Kafka Connect to ElasticSearch - LOG4J2 or Connection refused error - elasticsearch

Question

I am trying to configure Kafka Connect with Elastic search.
But when I try to start it I am getting following ERRORs:
ERROR StatusLogger Log4j2 could not find a logging implementation.
ERROR Failed to create client to verify connection. (io.confluent.connect.elasticsearch.Validator:120)
ElasticsearchException[java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused]; nested: ExecutionException[java.net.ConnectException: Connection refused]; nested: ConnectException[Connection refused]
I am trying to start it with following command in terminal:
connect-standalone.sh config/connect-standalone.properties config/elasticsearch.properties
This is connect-standalone.properties:
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.flush.interval.ms=10000
# EDIT BELOW IF NEEDED
bootstrap.servers=localhost:9092
offset.storage.file.filename=/tmp/connect.offsets
plugin.path=/home/stjepan/kafka_2.13-3.2.3/connectors
This is elasticsearch.properties:
name=elasticsearch-sink
connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector
tasks.max=1
topics=wikimedia.recentchange
key.ignore=true
connection.url=localhost:9200
# connection.url=https://kafka-course-5842482143.eu-west-1.bonsaisearch.net
# connection.username=he6de7ka5o
# connection.password=yozz8ryqmg
type.name=kafka-connect
# necessary for this connector
schema.ignore=true
behavior.on.malformed.documents=IGNORE
# OVERRIDE
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
What am I doing wrong? Wikimedia Connector works fine...
I didn't try anything else because I don't find any idea what can I change except ports, but it didn't helped...

Answer 1

You're probably referring to the course material of https://www.conduktor.io/apache-kafka-for-beginners (or the related course on Packt Pub)
I worked around this issue by using an OpenSearch Kafka Sink Connector:
https://github.com/aiven/opensearch-connector-for-apache-kafka
under https://github.com/aiven/opensearch-connector-for-apache-kafka/blob/main/config/quickstart-elasticsearch.properties there is a Connector Config that you can merge into your existing config.
Hope that helps