Questions tagged confluent-platform

Explore the latest questions and answers asked by our top developers.

Kafka Docker-Compose Configuration : Unexpected kafka request of type metadata during SASL handshake

I’ve been trying to set up authentication in kafka but I keep getting stuck, not sure what I am missing but this is my docker-compose (running version 3.7 of docker-compose) snippet services: zookeeper: image: confluentinc/cp-zookeeper:5.5.3 environment: ZOOKEEPER_CLIENT_PORT: 2181 KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/zookeeper_server_jaas.conf -Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Dzookeeper.allowSaslFailedClients=false -Dzookeeper.requireClientAuthScheme=sasl kafka: image: confluentinc/cp-enterprise-kafka:5.5.3 depends_on: [zookeeper] environment: KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181" KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_PLAINTEXT_HOST:SASL_PLAINTEXT KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://kafka:29092,SASL_PLAINTEXT_HOST://localhost:9092 […]

Apple M1: can’t start the confluent control center using docker

I would like to start the Confluent control center using docker as prescribed in the document: https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html This is the docker-compose.yaml file they provided; — version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:5.5.0 hostname: zookeeper container_name: zookeeper ports: – "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-server:5.5.0 hostname: broker container_name: broker depends_on: – zookeeper ports: – […]

what key.data in terms kafka rest v3 means?

I found format of create messages with predefined schema id here: https://docs.confluent.io/platform/current/kafka-rest/api.html#post–clusters-cluster_id-topics-topic_name-records It is slightly different from v2. I didn’t catch what key.data means? Tutorial shows only version v2 https://docs.confluent.io/platform/current/tutorials/examples/clients/docs/rest-proxy.html#basic-producer-and-consumer Source: Docker Questions

Multi-broker Confluent Kafka docker-compose.yml file

A while back I needed to instantiate a multi-broker Confluent Kafka stack using Confluent‘s docker images and docker-compose(1). Although Confluents documentation is quite excellent, it only offers a single-broker docker-compose.yml example; but not, say, a three-broker example from which one could study the pattern and generalize to any number of brokers needed. I pieced the […]

Kafka with multiple brokers raising "Connection to node 1 could not be established. Broker may not be available."

I’m trying to create a multiple broker setup locally (3 brokers, specifically), using confluentic images for both Zookeeper (6.2.0) and Kafka (5.5.1). When I docker-compose up, the project seems to boot fine. But when I try to consume my topic (–replication-factor 2 –partitions 10, tried with different partitions), the following errors are raised: [2021-10-05 01:55:29,696] […]

How to overwrite log4j.properties in confluent zookeeper

We are facing an issue that we are not able to overwrite log4j.properties. Here is our code Dockerfile FROM confluentinc/cp-zookeeper:6.2.0 #RUN chmod 777 /etc/kafka/connect-log4j.properties USER root COPY ./log4j.properties /etc/kafka/log4j.properties log4j.properties log4j.rootLogger=INFO, ROLLINGFILE log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n log4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender log4j.appender.ROLLINGFILE.Threshold=DEBUG log4j.appender.ROLLINGFILE.File=/var/log/zookeeper.log Note :- We are using this confluent zookeeper ‘confluentinc/cp-zookeeper:6.2.0’ Source: Docker Questions

How can I config broker to talk to other nodes? (Docker)

I create 3 broker nodes and 3 zookeeper nodes with docker-compose.yml, when I create the topics or produce and comesume message it’s show "WARN [AdminClient clientId=adminclient-1] Connection to node -2 (localhost/127.0.0.1:29092) could not be established. Broker may not be available. (or g.apache.kafka.clients.NetworkClient)" It’s seem like leader broker and follower can’t connect to each other. How […]

Upgrading Kafka Connect Avro Converter from 5.2.1 to 6.2.0 but no data comes in

I recently found out that Kafka Connect upgraded their version for the Kafka JDBC Sink Connector so I decided to upgrade the version in my docker file. However, once I updated the versions, I noticed that the Kafka Connect Avro Converter version error all my connectors from reading messages into the db. However, I copied […]

Confluent: How to add MongoDB Sink Connector on docker?

I have a Ubuntu machine, where I followed this steps in order to run Confluent with docker. https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html I can produce and subscribe to messages just fine. I’m trying to add a MongoDB Sink Connector, in order to sync data with a mongo database. I’ve downloaded this zip file https://www.confluent.io/hub/hpgrahsl/kafka-connect-mongodb I’ve edited the etc/MongoDbSinkConnector.properties file […]

Add MongoDB Sink Connector on docker?

I have a Ubuntu machine, where I followed this steps in order to run Confluent Platform with docker. https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html I can produce and subscribe to messages just fine. I’m trying to add a MongoDB Sink Connector, in order to sync data with a mongo database. I’ve downloaded this zip file https://www.confluent.io/hub/hpgrahsl/kafka-connect-mongodb I’ve edited the etc/MongoDbSinkConnector.properties […]

1 2 3 6

Still Have Questions?


Our dedicated development team is here for you!

We can help you find answers to your question for as low as 5$.

Contact Us
faq