Skip to content

Astra SDK Reference Configuration

Jump to bottom
Cedrick Lunven edited this page Feb 7, 2022 · 11 revisions

This page will cover the different options and configurations available to work with Astra SDK. To get more information regarding one API in particular please refer to the proper chapter in the menu on the left.

In the following tutorial we expect you to already have a database running and a token. If you need to create them have a look to the quickstart

📋 Table of content

  1. Architecture
  2. SDK design principles
  3. Configuration AstraClient
  4. Sample Code AstraClient
  5. Load-Balancing and Failover
  6. Working with AstraRC

1. Architecture

Datastax ASTRA is a the cloud platform of Datastax. It provides AstraDB and Astra Streaming services.

  • AstraDB is a DBAAS (database as a service) for Apache Cassandra™. You can start and run instances with a few clicks. The platform provides the databases nodes but also Stargate nodes with the tools to use the apis (swagger and graphQL playground).

  • Astra Streaming is a SAAS (software as a service) for Apache Pulsar™. You can create tenants with a few clicks and use all components of the message broker: namespaces, topics, subscriptions, functions....

📘 An Astra Database contains:

  • One to multiple region(s).
  • Each region will have 3 Cassandra nodes, 3 Stargate nodes, the CQL Console, Swagger-UI, GraphQL Playground and the load balancer to distribute the requests across the nodes.

As such, with Astra, there is no need to perform any load balancing at client side. You can eventually perform a failover across astra regions.

📘 Astra Communication and Network

Interface host protocol ports Paths
CqlSession ${dbid}-${dbregion}.db.astra.datastax.com tcp 29042,29080 --
Rest Api ${dbid}-${dbregion}.apps.astra.datastax.com http 443 api/rest*
Doc Api ${dbid}-${dbregion}.apps.astra.datastax.com http 443 api/rest*
Graph Api ${dbid}-${dbregion}.apps.astra.datastax.com http 443 api/graphql*
Grpc Api ${dbid}-${dbregion}.apps.astra.datastax.com tcp 443 --
Devops Api https://api.astra.datastax.com http 443 /v2*
Pulsar Client <clustername>.streaming.datastax.com tcp 6651,8001 --
Pulsar Admin <clustername>.api.streaming.datastax.com http 443 /*

2. SDK Design Principles

The Astra SDK library leverage on the Stargate SDK. As such it will reuse the same technologies for Cql, Http, gRPC, and graphQL, those are detailed here.

The Astra SDK adds some Http clients for the Devops Apis (databases, organization, streaming) reusing the same Http Component but also the pulsar-client and pulsar-admin needed to interact with Astra Streaming.

📘 Design principles

  • Users need a single class to setup the SDK, namely AstraClientConfig. There will be multiple ways to load the populate this object (builders, yaml) and an extension point is provided for you to create your own.

  • Users need a single class to use the SDK, namely AstraClient.

  • AstraClient will setup and reuse StargateClient class. As such the configuration will have a lot of similarities. Yet the Astra configuration is expected to be simpler as load-balancing is handle for us. Also the underlying StargateClient should not be hidden to developers.

StargateClient stargateClient = astraClient.getStargateClient();
  • Keep it Simple Stupid (KISS). A fluent Api is a great way to guide developers in the usage of the different functions available.
DatabasesClient apiDevopsDatabases = astraClient.apiDevopsDatabases();
OrganizationsClient apiDevopsOrg   = astraClient.apiDevopsOrganizations();
StreamingClient apiDevopsStreaming = astraClient.apiDevopsStreaming();
ApiDataClient apiDataRest          = astraClient.apiStargateData();
ApiDocumentClient apiDocument      = astraClient.apiStargateDocument();
ApiGraphQLClient apiGraphClient    = astraClient.apiStargateGraphQL();
ApiGrpcClient apiGrpc              = astraClient.apiStargateGrpc();
CqlSession                         = astraClient.cqlSession();
  • Initialize only Api(s) that can be used and disable others with no errors.
com.datastax.astra.sdk.config.AstraClientConfig : Initializing [AstraClient]
com.datastax.astra.sdk.AstraClient              : + API(s) Devops     [ENABLED]
com.datastax.astra.sdk.AstraClient              : + Using db with id [290eb696-c9ed-48b2-ac22-350f71baaee7] and region [us-east-1]
com.datastax.astra.sdk.AstraClient              : + Downloading bundles in: [/Users/cedricklunven/.astra]
com.datastax.astra.sdk.databases.DatabaseClient : + SecureBundle found : scb_290eb696-c9ed-48b2-ac22-350f71baaee7_us-east-1.zip
com.datastax.astra.sdk.databases.DatabaseClient : + SecureBundle found : scb_290eb696-c9ed-48b2-ac22-350f71baaee7_eu-central-1.zip
com.datastax.astra.sdk.AstraClient              : + Credentials used for Cql 'token' with ${token}
com.datastax.stargate.sdk.StargateClient        : Initializing [StargateClient]
com.datastax.stargate.sdk.StargateClient        : + Stargate nodes #[1] in [eu-central-1]
com.datastax.stargate.sdk.StargateClient        : + Stargate nodes #[1] in [us-east-1]
com.datastax.stargate.sdk.StargateClient        : + CqlSession   :[ ENABLED ]
com.datastax.stargate.sdk.rest.ApiDataClient    : + API Data     :[ ENABLED ]
com.datastax.stargate.sdk.doc.ApiDocumentClient : + API Document :[ ENABLED ]
com.datastax.stargate.sdk.gql.ApiGraphQLClient  : + API GraphQL  :[ ENABLED ]
com.datastax.stargate.sdk.grpc.ApiGrpcClient    : + API Grpc     :[ ENABLED ]
com.datastax.astra.sdk.AstraClient              : [AstraClient has been initialized.
  • Simplify user journey. To use the CqlSession with Astra you need to provide a zip file with the security certificates. This is file is called the Secure Cloud Bundle or SCB. The SDK will use the devops Api to download them for the user and save them by default in ~/.astra/scb_dbid_dbregion.zip.

3. Configuration AstraClient

In the following chapter we will describe the different parameters available for you to build the client class AstraClient through its builder.

AstraClientConfig astraClientConfig = AstraClient.builder();

🔽 Loading Parameters

In the builder there are 4 ways to retrieve configuration values and they will be loaded in the following order:

1. System Environment variables
2. Override by (if provided) read Java Arguments (-Dxx=yy)
3. Override by (if provided) file `~/.astrarc` with default section
4. Override by (if provided) explicit configuration in the builder (java code)

Not all parameters can be with environment variables (too many) here is the list:

Environment Variable Definition
ASTRA_DB_ID Unique identifier for your database
ASTRA_DB_REGION Default region used to connect to your database
ASTRA_DB_CLIENT_ID Part of the generated tokens, it acts as your username
ASTRA_DB_CLIENT_SECRET Part of the generated tokens, it acts as your password
ASTRA_DB_APPLICATION_TOKEN Part of the generated tokens, technical token used in Api Headers
ASTRA_DB_KEYSPACE Default selected keyspace in Astra
ASTRA_DB_SCB_FOLDER Folder where to download the secure connect bundles (one per region)

📑 Parameters List

📘 ApplicationName

Populate application name field in CqlSession.

astraClientConfig.withApplicationName("app_name")

📘 ClientId

This is your username when interacting with CQL API, it is part of the token generated.

astraClientConfig.withClientId("client_id")

📘 ClientSecret

This is your password when interacting with CQL API, it is part of the token generated.

astraClientConfig.withClientSecret("client_secret")

📘 CqlDriverOption: Driver Cql Option

*Each property in application.conf is identified by a key. Sometimes you want those keys to be set programmatically. Here the OptionsMap will be updated with this property.

//astraClientConfig.withCqlDriverOption(TypedDriverOption<T>, T);
astraClientConfig.withCqlDriverOption(
  TypedDriverOption.CONNECTION_CONNECT_TIMEOUT, 
  Duration.ofSeconds(10))

📘 CqlKeyspace: working Keyspace

Set current keyspace in the cqlSession.

astraClientConfig.withCqlKeyspace("keyspace")

📘 CqlMetricsRegistry: Cql Metrics Registry

Setup metrics (Dropwizard) at Cql driver level.

astraClientConfig.withCqlMetricsRegistry(myMeterMetricsRegistry)

📘 CqlRequestTracker: Cql drivers request tracker

Setup request tracker at Cql driver level.

astraClientConfig.withCqlRequestTracker(RequestTracker)

📘 CqlSessionBuilderCustomizer Cql Session builder customizer

Allow programmatic external configuration before creating CqlSession

astraClientConfig.withCqlSessionBuilderCustomizer(CqlSessionBuilderCustomizer)

📘 DatabaseId unique identifier for your database

Required for anything except devops apis, it will identify the database you want to work with

astraClientConfig.withDatabaseId("db_id")

📘 DatabaseRegion current region (datacenter) used to work with your db

Required for anything except devops apis, it will identify the database you want to work with

astraClientConfig.withDatabaseRegion("db_region")

📘 HttpRequestConfig: Http request configurations

*Fine tuning HTTP requests with ClientConfiguration (timeouts, keepalive, pooling...). Default are 20 seconds for timeouts.

astraClientConfig.withHttpRequestConfig(RequestConfig.custom()
   .setCookieSpec(StandardCookieSpec.STRICT)
   .setExpectContinueEnabled(true)
   .setConnectionRequestTimeout(Timeout.ofSeconds(5))
   .setConnectTimeout(Timeout.ofSeconds(5))
   .setTargetPreferredAuthSchemes(Arrays.asList(StandardAuthScheme.NTLM, StandardAuthScheme.DIGEST))
   .build())

📘 HttpRetryConfig: Http retry configurations

*Fine tuning for Http retries. But default a call is retried 3 times before the resources is considered unavailable by the internal load balancer. The retries are done in exponential backoff, after 100 millis, 200 millis and 400 millis.

astraClientConfig.withHttpRetryConfig(new RetryConfigBuilder()
   .retryOnAnyException()
   .withDelayBetweenTries(Duration.ofMillis(100))
   .withExponentialBackoff()
   .withMaxNumberOfTries(3)
   .build();)

📘 HttpObservers: Monitoring Observers

Define your observer and get notified for each request and retry. The class should implement ApiInvocationObserver.

public class SampleHttpObserver implements ApiInvocationObserver {

    /** {@inheritDoc} */
    @Override
    public void onCall(ApiInvocationEvent event) {
        System.out.println(event.getHost());
    }
    /** {@inheritDoc} */
    @Override
    public void onHttpSuccess(Status<String> s) {}

    /** {@inheritDoc} */
    @Override
    public void onHttpCompletion(Status<String> s) {}

    /** {@inheritDoc} */
    @Override
    public void onHttpFailure(Status<String> s) {}

    /** {@inheritDoc} */
    @Override
    public void onHttpFailedTry(Status<String> s) {}
}

You can register an observer by its name

astraClientConfig.addHttpObserver("sample", new SampleHttpObserver());

You can register all observer by providing a Map

Map<String, ApiInvocationObserver> observers = new HashMap<>();
observers.put("sample", new SampleHttpObserver());
observers.put("log", new AnsiLoggerObserver());
astraClientConfig.withHttpObservers(observers)

2 observers are provided out of the box AnsiLoggerObserver and AnsiLoggerObserverLight.

📘 withoutCqlSession: Disable Cql Session

To enforce a stateless client with no usage of the cqlSession you can disable CQL with this flag

astraClientConfig.withoutCqlSession()

📘 withSecureConnectBundleFolder: Set the folder to download secure bundle files

Location of the ZIP files required to initialize a connection to Astra.

astraClientConfig.withSecureConnectBundleFolder("/tmp")

📘 withToken: Set the authentication token

Mandatory for any api call except the cql session.

astraClientConfig.withToken("AstraCS:....")

4. Sample Code AstraClient

Here is a full fledge initialization of the AstraClient.

public AstraClient setupAstra() {
 return AstraClient.builder()
 
 // Astra Credentials Settings
 .withClientId("my_client_id")
 .withClientSecret("my_client_secret")
 .withToken("my_token")
                
 // Astra DB instance Settings
 .withDatabaseId("my_db_id")
 .withDatabaseRegion("my_db_region")
                
 // Cql Settings
 //.withoutCqlSession()
 .withApplicationName("SampleAPP")
 .withCqlDriverOption(TypedDriverOption.CONNECTION_CONNECT_TIMEOUT, Duration.ofSeconds(10))
 .withCqlDriverOption(TypedDriverOption.CONNECTION_INIT_QUERY_TIMEOUT, Duration.ofSeconds(10))
 .withCqlDriverOption(TypedDriverOption.CONNECTION_SET_KEYSPACE_TIMEOUT, Duration.ofSeconds(10))
 .withCqlDriverOption(TypedDriverOption.CONTROL_CONNECTION_TIMEOUT, Duration.ofSeconds(10))
 .withCqlKeyspace("quickstart")
 .withCqlSessionBuilderCustomizer(new CqlSessionBuilderCustomizer() {
   /** {@inheritDoc} */
   @Override
   public void customize(CqlSessionBuilder cqlSessionBuilder) {
     cqlSessionBuilder.withClassLoader(null);
   }})
 .withCqlMetricsRegistry(myMeterMetricsRegistry)
 //.withCqlRequestTracker(null)
 .withSecureConnectBundleFolder("~/.astra")
                
 // Http Client Settings
 .withHttpRequestConfig(RequestConfig.custom()
   .setCookieSpec(StandardCookieSpec.STRICT)
   .setExpectContinueEnabled(true)
   .setConnectionRequestTimeout(Timeout.ofSeconds(5))
   .setConnectTimeout(Timeout.ofSeconds(5))
   .setTargetPreferredAuthSchemes(Arrays.asList(StandardAuthScheme.NTLM, StandardAuthScheme.DIGEST))
   .build())
  .withHttpRetryConfig(new RetryConfigBuilder()
   .retryOnAnyException()
   .withDelayBetweenTries( Duration.ofMillis(100))
   .withExponentialBackoff()
   .withMaxNumberOfTries(10)
   .build())
  .addHttpObserver("logger_light", new AnsiLoggerObserver())
  .addHttpObserver("logger_full", new AnsiLoggerObserverLight())
  .build();
}

5. Load-Balancing and Failover

📘 Recommended Architectures

With AstraDB you can create multiple datacenters or regions for your instance. Each region will host 3 nodes of Cassandra and 3 nodes of Stargate. In this chapter will we cover design choices regarding Load Balancing and FailOver and how to use them with the SDK.

  • AstraDB load-balancing is done for your through Kubernetes Services (ingress), you have nothing to setup. The load balancing is done INSIDE A SINGLE REGION to keep the latencies low. Considering another

It is recommended to perform failover across regions/datacenters at application level with DNS based routing. Indeed if the failover is implemented at data layer we can see some spikes in the latencies (from 25ms to 150 ms in our tests on Astra).

📘 Failover

ℹ️ Code of this demonstration is available for download 📥 here.

6. Working with AstraRc

To setup the SDK you can provide values using the builder as listed above but not only you can save them locally in some configuration file (.astrarc).

📘 (a) Understanding configuration file ~/.astrarc

This file uses the ini configuration formatting. A key ASTRA_DB_APPLICATION_TOKEN is associated to its value. Values are grouped in sections identified by a [name].

Sample File with 2 sections:

[default]
ASTRA_DB_APPLICATION_TOKEN=AstraCS:GmSdU.....
ASTRA_DB_CLIENT_ID=GmSdUgw....
ASTRA_DB_CLIENT_SECRET=pL7QcZgN....
ASTRA_DB_ID=33a7527a-aaf9-4f9f-9758-96216efff3d5
ASTRA_DB_KEYSPACE=ks1
ASTRA_DB_REGION=eu-central-1
ASTRA_DB_SECURE_BUNDLE=

[sandbox]
ASTRA_DB_APPLICATION_TOKEN=AstraCS:GmSdU.....
ASTRA_DB_CLIENT_ID=GmSdUgw....
ASTRA_DB_CLIENT_SECRET=pL7QcZgN....
ASTRA_DB_ID=33a7527a-aaf9-4f9f-9758-96216efff3d5
ASTRA_DB_KEYSPACE=ks1
ASTRA_DB_REGION=eu-central-1
ASTRA_DB_SECURE_BUNDLE=

The file starts with a [default] section. If nothing name is provided it will be the one to load. In the SDK this file will be associated with the class com.datastax.astra.sdk.utils.AstraRc

📘 (b) Generate an ~/.astrarc

// Generate the file in default location user home (~/.astrarc) 
AstraRc.create("<your_token>");
        
// Generate the file in a defined location
AstraRc.create("<your_token>", new File("/tmp/astracrc"));

📘 (c) Load an ~/.astrarc file

// Load default file (~/.astrarc) 
AstraRc confAstraRc1 = AstraRc.load()
        
// Load a file in defined location
AstraRc confAstraRc2 = AstraRc.load("/tmp/astracrc");

📘 (d) Update an ~/.astrarc file

confAstraRc1.save("sectionName", "keyName", "keyValue");

Working with environment variables

The SDK can be and will be used in Docker and Kubernetes contextes. For those you provide parameters through environment variables. The SDK is ready for that here is the list of environment variables than you can provide:

Environment Variable Definition
STARGATE_USERNAME User identifier to login to Stargate
STARGATE_PASSWORD User password to login to Stargate
STARGATE_ENDPOINT_AUTH Authentication endpoint to get a Token
STARGATE_ENDPOINT_REST Endpoint for Data and Document APIs
STARGATE_ENDPOINT_GRAPHQL Endpoint for GraphQL APIs
STARGATE_ENDPOINT_CQL Contact Point adresse ip:port
STARGATE_LOCAL_DC Local DataCenter name
STARGATE_KEYSPACE Keyspace selected
STARGATE_ENABLE_CQL Even with user/[assword you can disable CQL to stay stateless
ASTRA_DB_ID Astra database identifier
ASTRA_DB_REGION Main region for the database
ASTRA_DB_APPLICATION_TOKEN Your Token AstraCS:...
ASTRA_DB_CLIENT_ID ClientId part of your token
ASTRA_DB_CLIENT_SECRET ClientSecret part of your token
ASTRA_DB_KEYSPACE Keyspace to use in ASTRA
ASTRA_DB_SECURE_BUNDLE Location where to find the secure connect bundle

The same variables can be provided as Java parameter using the syntax -DKEY=VALUE

java -jar myapp.jar -DSTARGATE_USERNAME=cassandra -DSTARGATE_PASSWORD=cassandra

🏠 Home | Document | Rest | Native Drivers | GraphQL | gRPC | Astra Devops |

I - Getting Started

Home
  1. Overview
  2. What's next
QuickStarts
  1. QuickStart Stargate
  2. QuickStart Astra
Reference Configurations
  1. Stargate SDK
  2. Astra SDK
Reference JavaDoc
  1. latest
  2. 0.3.4
  3. 0.2.6

II - Stargate Apis

Api Rest
  1. Overview
  2. Keyspaces
  3. Tables
  4. Columns
  5. Indexes
  6. User Defined Type
  7. Data
Api Document
  1. Overview
  2. Namespaces
  3. Collections
  4. Documents
  5. DocumentRepository
Api GraphQL
  1. Overview
  2. Cql First
  3. Schema First
  4. Federation
Api Grpc
  1. Overview
  2. BlockingStub
  3. FutureStub
  4. ReactiveStub
Native Drivers
  1. Overview
  2. Cql Session
  3. Object Mapper

III - Astra Apis

Api Devops Databases
  1. Overview
  2. Databases
  3. Regions
  4. AccessList
  5. Private Link
Api Devops Organizations
  1. Overview
  2. Roles
  3. Users
  4. Tokens
Api Devops Streaming
  1. Overview
  2. Tenants
  3. Clusters
  4. Pulsar client
  5. Pulsar admin
  6. Producers
  7. Consumers
  8. Subscriptions

IV - Integrations

Spring Framework
  1. Spring Boot Starter
  2. Spring Data Cassandra
  3. Spring Boot Actuator
  4. Spring Security
Clone this wiki locally