Skip to content

Kafka Connect suite of connectors for Cloud storage (Amazon S3)

License

Notifications You must be signed in to change notification settings

confluentinc/kafka-connect-storage-cloud

Repository files navigation

Kafka Connect Connector for S3

FOSSA Status

kafka-connect-storage-cloud is the repository for Confluent's Kafka Connectors designed to be used to copy data from Kafka into Amazon S3.

Kafka Connect Sink Connector for Amazon Simple Storage Service (S3)

Documentation for this connector can be found here.

Blogpost for this connector can be found here.

Development

To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the kafka-connect-storage-common FAQ for guidance on this process.

You can build kafka-connect-storage-cloud with Maven using the standard lifecycle phases.

Running Integration Tests

Integration tests are run as part of mvn install; however one needs to first configure the environment variableAWS_CREDENTIALS_PATH to point to a json file path with following structure:

{
    "aws_access_key_id": "<key>",
    "aws_secret_access_key": "<secret>"
}

Contribute

License

This project is licensed under the Confluent Community License.

FOSSA Status