Questions tagged debezium

Explore the latest questions and answers asked by our top developers.

Connect to AWS DocumentDB from Docker Kafka Connect Container with SSL

I am trying to connect to my AWS Document DB from a local Kafka Connect Docker container using SSL. I’ve followed AWS documentation and created the rds-trustore.jks file from .pem file which is provided by AWS itself. Then I’ve set the environment variables stated here as below. CONNECT_SSL_TRUSTSTORE_LOCATION=/path/to/my/rds-trustore.jks CONNECT_SSL_TRUSTSTORE_PASSWORD=<password> The problem is, when I try […]

Failing to generate a filter on debezium connector

I have created a debezium connector to a docker MySQL container. I tried to set a filter for messages: { "name": "my_connector", "config": { "name": "my_connector", "connector.class": "io.debezium.connector.mysql.MySqlConnector", … "include.schema.changes": "true", "transforms": "filter, unwrap", "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", "transforms.unwrap.drop.tombstones": "true", "transforms.filter.type": "io.debezium.transforms.Filter", "transforms.filter.language": "jsr223.groovy", "transforms.filter.condition": "value.source.table == ‘table-name’ && (value.op == ‘d’ || value.op == ‘c’ || […]

By Tom Carmi
Published
Categorised as apache-kafka, debezium, docker, mysql Tagged , , ,

Debezium config for postgresql and elastic/Elasticsearch is not exposed from docker container(Probably)

I try to set up debezium, postgresql and elastic search to work together, but it seems like the elastic is not visible for kafka connect and do not receive any events. I noticed when I make a curl request to elastic I receive (52) Empty reply from server but when make this same request inside […]

By Vitoo
Published
Categorised as debezium, docker, elasticsearch, postgresql Tagged , , ,

CDC debezium (postgres) is not publishing events for certain table

I’ve set up a CDC pipeline in docker network using following scripts zookeper –restart always -p 2181:2181 -p 2888:2888 -p 3888:3888 –network=test-net -v /var/ibank/msdata/test/zookeper/config/zoo.cfg:/zookeper/conf/zoo.cfg -v /var/ibank/msdata/test/zookeper/data/main-data:/zookeper/data -v /var/ibank/msdata/test/zookeper/data/txns:/zookeper/txns -v /var/ibank/msdata/test/zookeper/data/logs:/zookeper/logs debezium/zookeeper:1.6 kafka –name kafka –restart=always –network=test-net -p 9092:9092 -p 29092:29092 -e ADVERTISED_HOST_NAME=kafka -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT -e ZOOKEEPER_CONNECT=zookeeper:2181 -e KAFKA_LISTENERS=PLAINTEXT_HOST://0.0.0.0:29092,PLAINTEXT://kafka:9092 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092 -v /var/ibank/msdata/test/kafka/data/kafka-data:/var/lib/kafka/data debezium/kafka:1.6 connect –restart=always […]

By Aika Sat
Published
Categorised as apache-kafka, debezium, docker, postgresql Tagged , , ,

Using Debezium, but cannot see changes in sink Postgres Docker container

I am learning Debezium, and I am new to Docker as well. I want to replicate changes from a source Postgres database to a destination Postgres (empty). Both are in separate Docker containers. I am using Debezium to replicate the changes. When I run the Docker stack, and make changes in source Postgres, I can […]

Can’t restart oracle database inside docker

So I’m trying to configure debezium on a Oracle database inside this docker image, according to the documentation described here. At the "Preparing the Database" the first config I have to do is: ORACLE_SID=ORACLCDB dbz_oracle sqlplus /nolog CONNECT sys/top_secret AS SYSDBA alter system set db_recovery_file_dest_size = 10G; alter system set db_recovery_file_dest = ‘/opt/oracle/oradata/recovery_area’ scope=spfile; shutdown […]

By Alexander Rumanovsk
Published
Categorised as debezium, docker, oracle Tagged , ,

Error configuring oracle database for debezium data mining

I’m trying to configure my oracle database for debezium to mine the changes on several tables. To do so, I’ve followed the tutorial described here. My oracle database is a Singleinstance database running inside a docker container, created using this code. So what I did is basically create several scripts that create the tablespaces, tables […]

By Alexander Rumanovsk
Published
Categorised as debezium, docker, oracle Tagged , ,

Connection to Kafka broker in Docker container fails

I’m trying to get a simple Debezium stack (with Docker Compose) running but the connection to the Kafka broker fails. Here is my simplified docker-compose.yml: version: "3" services: zookeeper: image: debezium/zookeeper:1.6 ports: – "2181:2181" – "2888:2888" – "3888:3888" kafka: image: debezium/kafka:1.6 links: – zookeeper ports: – "9092:9092" environment: BROKER_ID: 1 KAFKA_LISTENERS: LISTENER_BOB://kafka:29092,LISTENER_FRED://localhost:9092 KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka:29092,LISTENER_FRED://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: […]

By swissclash79
Published
Categorised as apache-kafka, debezium, docker Tagged , ,

Debezium with Postgres | Kafka Consumer not able to consume any message

Here is my docker-compose file: version: ‘3.7’ services: postgres: image: debezium/postgres:12 container_name: postgres networks: – broker-kafka environment: POSTGRES_PASSWORD: admin POSTGRES_USER: antriksh ports: – 5499:5432 zookeeper: image: confluentinc/cp-zookeeper:latest container_name: zookeeper networks: – broker-kafka ports: – 2181:2181 environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:latest container_name: kafka networks: – broker-kafka depends_on: – zookeeper ports: – 9092:9092 environment: […]

DockerFile – CMD

Hy guys, I’m newer in Docker. I have a problem with write a dockerfile for my application. My code is beloy: # define a imagem base FROM ubuntu:latest # define a owner image LABEL maintainer="MyCompany" # Update a image with packages RUN apt-get update && apt-get upgrade -y # Expose port 80 EXPOSE 8089 # […]

1 2 3

Still Have Questions?


Our dedicated development team is here for you!

We can help you find answers to your question for as low as 5$.

Contact Us
faq