Debezium Kafka Connect – Error Loading Plugin From Confluent Hub via Docker

Published

I’ve recently tried installing the Debezium PostgreSQL Connector v1.6.0 via confluent-hub.

When the plugin is loaded, the connector fails to load, raised by loadProperties in IoUtil, per this error:

[2021-07-13 00:45:06,715] INFO Loading plugin from: /usr/share/confluent-hub-components/debezium-debezium-connector-postgresql (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2021-07-13 00:45:07,375] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed)
java.lang.ExceptionInInitializerError
at io.debezium.connector.postgresql.PostgresConnector.version(PostgresConnector.java:47)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:387)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:392)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:362)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:334)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:268)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:260)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:229)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:206)
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:61)
at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:93)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)
Caused by: java.lang.NullPointerException: inStream parameter is null
at java.base/java.util.Objects.requireNonNull(Objects.java:246)
at java.base/java.util.Properties.load(Properties.java:406)
at io.debezium.util.IoUtil.loadProperties(IoUtil.java:491)
at io.debezium.util.IoUtil.loadProperties(IoUtil.java:521)
at io.debezium.connector.postgresql.Module.<clinit>(Module.java:19)
... 12 more

We essentially have one Dockerfile, where we run the following:

FROM confluentinc/cp-kafka-connect:6.2.0

RUN confluent-hub install --no-prompt debezium/debezium-connector-postgresql:1.6.0

Everything seems to build fine, and both the zookeeper and local broker is running fine. I have noticed similar behaviour in versions 6.1.1 and 6.2.0.

Can anyone provide any input?


Steps to recreate this issue:

  1. Create a network, and run the confluentinc/cp-zookeeper:6.2.0 image with the supplied "zookeeper.env" file for environment variables:
$ docker network create --driver bridge my_network
$ docker run --platform="linux/amd64" -p "2181:2181" --name="zookeeper" --env-file=zookeeper.env --network=my_network confluentinc/cp-zookeeper:6.2.0
  1. Run the confluentinc/cp-kafka:6.2.0 image with the supplied "broker.env" file for environment variables:
$ docker run -p "29092:29092" -p "9092:9092" -p "9101:9101" --platform="linux/amd64" --name="broker" --env-file=broker.env --network=my_network confluentinc/cp-kafka:6.2.0
  1. Build a Docker image from the "Dockerfile" attached which uses the base confluentinc/cp-kafka-connect:6.2.0 . This image will install debezium-connector-postgresql:1.6.0 from Confluent Hub. Run it with the supplied "connect.env" file for environment variables:
$ docker build . --platform='linux/amd64' -t 'connect'
$ docker run -p "8083:8083" --platform="linux/amd64" --name="connect" --env-file=connect.env --network=my_network connect

Relevant .env files and the Dockerfile are hosted on this Gist.


Platform details:

  • macOS Big Sur 11.2.3 / M1
  • Docker 3.5.2 (66501)
  • Docker Engine 20.10.7

Source: Docker Questions

Answers

Leave a Reply

Still Have Questions?


Our dedicated development team is here for you!

We can help you find answers to your question for as low as 5$.

Contact Us
faq