Add MongoDB Sink Connector on docker?

Published

I have a Ubuntu machine, where I followed this steps in order to run Confluent Platform with docker.

https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html

I can produce and subscribe to messages just fine.

I’m trying to add a MongoDB Sink Connector, in order to sync data with a mongo database.

  1. I’ve downloaded this zip file https://www.confluent.io/hub/hpgrahsl/kafka-connect-mongodb

  2. I’ve edited the etc/MongoDbSinkConnector.properties file with the correct mongo endpoint

  3. I’ve uploaded the zip to my Ubuntu machine

  4. I’ve created a file Dockerfile with the following content

    FROM confluentinc/cp-kafka-connect-base
    
    COPY hpgrahsl-kafka-connect-mongodb-1.4.0.zip /tmp/hpgrahsl-kafka-connect-mongodb-1.4.0.zip
    
    RUN confluent-hub install --no-prompt /tmp/hpgrahsl-kafka-connect-mongodb-1.4.0.zip
    
  5. I’ve executed the following command docker build . -t my-custom-image:1.0.0

    Sending build context to Docker daemon  15.03MB
    Step 1/3 : FROM confluentinc/cp-kafka-connect-base
     ---> 8fe065fffe44
    Step 2/3 : COPY hpgrahsl-kafka-connect-mongodb-1.4.0.zip /tmp/hpgrahsl-kafka-connect-mongodb- 
    1.4.0.zip
     ---> Using cache
     ---> ed2e4ec7ff97
    Step 3/3 : RUN confluent-hub install --no-prompt /tmp/hpgrahsl-kafka-connect-mongodb-1.4.0.zip
     ---> Using cache
     ---> 034f82e2e136
    Successfully built 034f82e2e136
    Successfully tagged my-custom-image:1.0.0
    

Am I missing something? My mongo does not get updated

Do I have to edit docker-compose.yml also?

How do I debug this connector, does it have logs?

Source: Docker Questions

Answers

Leave a Reply

Still Have Questions?


Our dedicated development team is here for you!

We can help you find answers to your question for as low as 5$.

Contact Us
faq