Please leave your feedback with the Docs team.
Toggle Menu

Filter Results:

  • Astra DB
  • Astra Streaming
  • Luna Support
↓↑to navigate
↵to select
escto close
    powered by Apache Cassandra™
    Go to DataStax Home
    Dark Mode

    Deploy the DataStax Apache Kafka™ Connector to stream records from an Apache Kafka topic to your DataStax Astra DB database.

    The DataStax Apache Kafka Connector download package includes a sample JSON properties file ( dse-sink-distributed.json.sample ).Use the sample file as a reference when configuring your deployment.The dse-sink-distributed.json.sample file is located in the conf directory of the DataStax Apache Kafka Connector distribution package.

    Prerequisites

    • Download and install the DataStax Apache Kafka Connector.
    • Configure the distributed worker configuration file connect-distributed.properties to fit your needs.Use this example from DataStax as a starting point.Specify the converter for the key.converter and value.converter properties that matches the form of your Kafka data.See Configuring converters in the Confluent documentation for more information on these properties.
    • From the directory where you installed Apache Kafka, start the distributed worker:
    • 1
    The worker startup process outputs a large number of informational messages.The following message displays after the process completes: `[2019-10-13 19:49:25,385] INFO Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder:852)`. Configure the JSON configuration file (such as dse-sink.json) to use the Astra DB secure connect bundle.
    • 1
    • name: Unique name for the connector. Default:  dse-sink 
    • connector.class: DataStax connector Java class provided in the  kafka-connect-dse-N.N.N.jar . Default:  com.datastax.kafkaconnector.DseSinkConnector 
    • cloud.secureConnectBundle: The full path to the secure connect bundle for your Astra DB database (`secure-connect-*database_name*.zip`).
    Download the secure connect bundle from the DataStax Cloud console.If this option is specified, you must also include the auth.username and auth.password for the database user.
    • auth.username: Astra DB database username
    When authorization is enabled, the DataStax connector login role must have a minimum of  modify  privileges on tables receiving data from the DataStax Apache Kafka® Connector.
    • *auth.password*: Astra DB database password for the specified username
    • Register the connector configuration with the distributed worker:
    • 1
    ip and port are the IP address and port number of the Kafka worker.Use the same port as the  rest.port  parameter set in  connect-distributed.properties .The default port is 8083.
    You configured the  dse-sink.json  or  dse-sink.properties  file when installing the DataStax Apache Kafka Connector.