Category : apache-kafka-connect

I am trying Aerospike Connect to Kafka. OS: Windows I referred https://enterprise.aerospike.com/docs/connect/streaming-from-asdb/installing/from-asdb-to-kafka-installing.html. I referred: ‘# With Aerospike 5.0 and later’ for deploying on docker I have these running in docker: My Windows path for the file: C:/aerospike-kafka-outbound/aerospike-kafka-outbound.yml C:UsersNupur>docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1114b8a97406 aerospike/aerospike-kafka-outbound:4.0.0 "/bin/sh -c ‘/opt/${…" 37 seconds ago ..

Read more

I’m using docker with kafka and clickhouse. I want to connect ‘KsqlDB table’ and ‘clickhouse’ using ‘kafka connect’. So I referred to this document and modified ‘docker composite’. here is my docker-compose — version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:6.2.1 hostname: zookeeper container_name: zookeeper ports: – "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-server:6.2.1 hostname: ..

Read more

I have a Ubuntu machine, where I followed this steps in order to run Confluent Platform with docker. https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html I can produce and subscribe to messages just fine. I’m trying to add a MongoDB Sink Connector, in order to sync data with a mongo database. I’ve downloaded this zip file https://www.confluent.io/hub/hpgrahsl/kafka-connect-mongodb I’ve edited the etc/MongoDbSinkConnector.properties ..

Read more

I am running Docker containers for kafka, zookeeper, kafka-connect, mongoDB & mosquitto within a network. When i register connector for mqtt source and send a message to bootstrap server: kafka:29092, I get the following warning when I listen to the topic. docker exec kafka kafka-console-consumer –bootstrap-server kafka:29092 –topic connect-custom –from-beginning –max-messages 42 [2021-08-25 11:53:04,555] WARN ..

Read more

Here’s the docker-compose file I am using for kafka and ksqldb setup, — version: ‘2’ services: zookeeper: image: confluentinc/cp-zookeeper:6.2.0 hostname: zookeeper container_name: zookeeper ports: – "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-server:6.2.0 hostname: broker container_name: broker depends_on: – zookeeper ports: – "9092:9092" – "9101:9101" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: ‘zookeeper:2181’ KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092 ..

Read more

Here is my docker-compose file: version: ‘3.7’ services: postgres: image: debezium/postgres:12 container_name: postgres networks: – broker-kafka environment: POSTGRES_PASSWORD: admin POSTGRES_USER: antriksh ports: – 5499:5432 zookeeper: image: confluentinc/cp-zookeeper:latest container_name: zookeeper networks: – broker-kafka ports: – 2181:2181 environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:latest container_name: kafka networks: – broker-kafka depends_on: – zookeeper ports: – 9092:9092 environment: ..

Read more

The "volumes" mapping does not seem to work when trying to embed Kafka connector in Ksqldb Server Below is my docker file ksqldb-server: image: confluentinc/ksqldb-server:0.18.0 hostname: ksqldb-server container_name: ksqldb-server depends_on: – broker – schema-registry ports: – "8088:8088" volumes: – "/usr/local/share/kafka/connectors/debezium-debezium-connector-mysql/lib:/usr/share/kafka/plugins/" environment: KSQL_LISTENERS: "http://0.0.0.0:8088" KSQL_BOOTSTRAP_SERVERS: "broker:9092" KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081" KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true" KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true" # Configuration to embed ..

Read more

I’ve recently tried installing the Debezium PostgreSQL Connector v1.6.0 via confluent-hub. When the plugin is loaded, the connector fails to load, raised by loadProperties in IoUtil, per this error: [2021-07-13 00:45:06,715] INFO Loading plugin from: /usr/share/confluent-hub-components/debezium-debezium-connector-postgresql (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [2021-07-13 00:45:07,375] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed) java.lang.ExceptionInInitializerError at io.debezium.connector.postgresql.PostgresConnector.version(PostgresConnector.java:47) at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:387) at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:392) at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:362) at ..

Read more