quickstart-confluent-cloud

Quick Start with Confluent Cloud

This folder contains a variant of the Quick Start app configured to use Confluent Cloud as the target Kafka cluster.

The docker-compose.yml file has been revised to realize the integration with Confluent Cloud as follows:

• Removal of the broker service, because replaced by the remote Kafka cluster.

• kafka-connector:

  • Definition of new environment variables to configure the remote endpoint, credentials, and topic name in the adapters.xml through the variable-expansion_ feature of Lightstreamer.

    ...
    environment:
      - bootstrap_server=${bootstrap_server}
      - api_key=${api_key}
      - secret=${secret}
    ...

  • Adaption of adapters.xml to include:

    • New Kafka cluster address retrieved from the environment variable bootstrap_server:

      <param name="bootstrap.servers">$env.bootstrap_server</param>

    • Encryption settings:

      <param name="encryption.enable">true</param>
      <param name="encryption.protocol">TLSv1.2</param>
      <param name="encryption.hostname.verification.enable">true</param>

    • Authentication settings, with credentials retrieved from environment variables api_key and secret:

      <param name="authentication.enable">true</param>
      <param name="authentication.mechanism">PLAIN</param>
      <param name="authentication.username">$env.api_key</param>
      <param name="authentication.password">$env.secret</param>

    NOTE This example assumes that the topic name is topic_0 (the default topic provided by Confluent Cloud). If required, update the topic mapping setting by replacing <confluent_topic_name> with the appropriate value, as follows (because variable-expansion is not available in the name attribute):

    <param name="map.<confluent_topic_name>.to">item-template.stock</param>

• producer:

  • Parameter --boostrap-servers retrieved from the environment variable bootstrap_server.
  • Provisioning of the producer.properties configuration file to enable SASL/PLAN over TLS, with username and password retrieved from the environment variables api_key and secret:

  # Configure SASL/PLAIN mechanism
  sasl.mechanism=PLAIN
  # Enable SSL encryption
  security.protocol=SASL_SSL
  # JAAS configuration
  sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="${api_key}" password="${secret}";

  NOTE If required, update the parameter --topic in the command attribute with the appropriate value.

Run

From this directory, run follow the command:

api_key=<API.key> secret=<secret> bootstrap_server=<bootstrap_server> topic=<topic> ./start.sh 

where

• API.key and secret are the credentials generated on the Confluent CLI or from the Confluent Cloud Console.
• bootstrap_server is the Kafla cluster address.

Then, point your browser to http://localhost:8080/QuickStart.