Logstash local configuration - spring-boot

Im trying to configure local logstash config with spring boot in yml format.
Here it is:
appenders:
logstash:
enabled: true
input-type: tcp
destination: 127.0.0.1:4560
and after running app I catch in console:
WARN in net.logstash.logback.appender.LogstashTcpSocketAppender[logstash] - Log destination 127.0.0.1:5010: connection failed. java.net.ConnectException: Connection refused (Connection refused)
at java.net.ConnectException: Connection refused (Connection refused)
I checked all listeners and the port is not in use. I seems to be easy to fix but I dk how.

Related

Docker Desktop cannot access web from container

Tried almost every attempt found here in the community but didn't figure out how to access web through continers created in my Docker Desktop Environment.
This is my last attempt with Ubuntu (also tried Centos and Debian suites):
docker run -i -t --rm=true ubuntu /bin/bash
root#310f7a660efd:/# apt-get update -y
Err:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Could not connect to archive.ubuntu.com:80 (91.189.88.24). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.173). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.174). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.162). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.31). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.149). - connect (111: Connection refused)
Err:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Unable to connect to archive.ubuntu.com:http:
Err:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Unable to connect to archive.ubuntu.com:http:
Err:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
Could not connect to security.ubuntu.com:80 (91.189.91.14). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.24). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.162). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.31). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.174). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.91.26). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.91.24). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.91.23). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.173). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.149). - connect (111: Connection refused)
Reading package lists... Done
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease Could not connect to archive.ubuntu.com:80 (91.189.88.24). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.173). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.174). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.162). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.31). - connect (111: Connection refused) Could not connect to archive.ubuntu.com:80 (91.189.88.149). - connect (111: Connection refused)
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease Unable to connect to archive.ubuntu.com:http:
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease Unable to connect to archive.ubuntu.com:http:
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease Could not connect to security.ubuntu.com:80 (91.189.91.14). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.24). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.162). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.31). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.174). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.91.26). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.91.24). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.91.23). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.173). - connect (111: Connection refused) Could not connect to security.ubuntu.com:80 (91.189.88.149). - connect (111: Connection refused)
W: Some index files failed to download. They have been ignored, or old ones used instead.
If i ping google.com from the server no problem but just can't access repos.
Seems a proxy problem but just because I set proxy in the Docker settings it allows me to download images from dockerhub.
Also tried DNS fixed to 8.8.8.8.
My OS is Windows 10 and this is Docker info:
Client:
Debug Mode: false
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 3
Server Version: 19.03.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894--------------16b8817763c29fb
runc version: 425e105d5a03----------9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.184-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.952GiB
Name: docker-desktop
ID: UL2M:2GDU:GGIH:IO6C:EJWA:YUXM:5DFL:F6Q4:XTFE:K3MN:7474:TY4J
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 38
Goroutines: 58
System Time: 2019-10-07T13:34:51.1886632Z
EventsListeners: 1
HTTP Proxy: xxxxxxxxxxxxxxxx:3128
HTTPS Proxy: xxxxxxxxxxxxxxxx:3128
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
This is quite sad situation since if a container can't access web it's very limited.
Also if I put unreachable path from those stack in my chrome they are perfectly available.
Hope for some advice to put me in the right direction.

Connection Refused for Consul

I am starting a Spring Boot Application with Consul.
I am getting the following error
2019-08-30 12:34:22.650 ERROR 23428 --- [ main] o.s.boot.SpringApplication : Application run failed
com.ecwid.consul.transport.TransportException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8090 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect`
I changed the default port in bootstrap.properties file.
I also used another non-Consul Spring Boot Application and it worked fine for that use-case with same port.
8090 is not the default port for Consul. You didn't say if your other successful app was on the same host or not, but make sure Consul is actually listening on that port with netstat or ss.
By default, Consul listens for API requests on port 8500.

ElasticsearchSinkConnector can't connect to Elastic

Using the kafka-lenses-dev image, I ran into problems connecting Kafka to the local Elasticsearch 7.1.1 instance. As you can see, the connection is refused, although the instance is up and running and it's possible to curl docs in. Is the version of Elastic problematic for the connector or is the config wrong?
Config
connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector
type.name=kafka-connect
key.converter.schemas.enable=false
topics=cc_data
name=elastic-sink
value.converter.schemas.enable=false
connection.url=http://localhost:9200
key.ignore=true
Error
org.apache.kafka.connect.errors.ConnectException: Couldn't start ElasticsearchSinkTask due to connection error:
at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.<init>(JestElasticsearchClient.java:147)
at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.<init>(JestElasticsearchClient.java:114)
at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start(ElasticsearchSinkTask.java:120)
...
Caused by: io.searchbox.client.config.exception.CouldNotConnectException: Could not connect to http://localhost:9200
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:60)
at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.getServerVersion(JestElasticsearchClient.java:168)
at io.confluent.connect.elasticsearch.jest.JestElasticsearchClient.<init>(JestElasticsearchClient.java:145)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
...
Caused by: java.net.ConnectException: Connection refused (Connection refused)
...
You've specified Elasticsearch as being at localhost:9200. Unless Elasticsearch is running on the same container as Kafka Connect then this hostname is wrong. You need to specify the hostname of Elasticsearch in a form that is reachable from where Kafka Connect is running.
As an example, you can see an example of how to configure this based on this Docker Compose here.

hive is not working with HDP with Ambari

I have installed the HDP with Ambari 2.6.1. It mostly did everything automatically but Hive is unable to start properly.
I saw a post somewhere and I deleted the pid and killed the process as well in hope that I would restart it and it would work but now it's showing heartbeat lost on the machine.
Please guide me on what should I do?
I am listing the errors I got from ambari.
1.Hive Metastore:
Metastore on machine2.ambari.local failed (Traceback (mos&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'' returned 1. Error: Could not open client transport with JDBC Uri: jdbc:hive2://machine2.ambari.local:10000/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0) Error: Could not open client transport with JDBC Uri: jdbc:hive2://machine2.ambari.local:10000/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0) )" data-bindattr-40693="40693"> Connection failed on host machine2.ambari.local:10000 (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py", line 212, in execute ldap_password=ldap_password) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hive_check.py", line 81, in check_thrift_port_sasl timeou... t recent call last): File "/var/lib/ambari-agent/cache/common-servics/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py", line 203, in execute timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE, File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/
Hive Server2:
awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'' returned 1. Error: Could not open client transport with JDBC Uri: jdbc:hive2://machine2.ambari.local:10000/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0) Error: Could not open client transport with JDBC Uri: jdbc:hive2://machine2.ambari.local:10000/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0) )" data-bindattr-40693="40693"> Connection failed on host machine2.ambari.local:10000 (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py", line 212, in execute ldap_password=ldap_password) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hive_check.py", line 81, in check_thrift_port_sasl timeou...
There are following three different issues in this question
Hiveserver2 alert :Following error indicate port 10000 on machine2.ambari.local is not reachable, either there is no process (HiveServer2) running on port 10000 or some proxy issue.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://machine2.ambari.local:10000/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused)
Heartbeat Lost : In question it is mentioned that "heartbeat lost on the machine", this could be because ambari-agent process is no more running on host for which heartbeat lost is notified.
After deleting the pid and killing the process , by default it will not restart automatically.

ActiveMQ JMS TCP Transport is incrementally changing port number on retries

After some JMS connectivity problems I've noticed in logs:
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61141 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61156 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61148 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61161 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61192 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61197 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61226 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61273 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61241 failed: java.io.EOFException
Why JMS is retrying in such way? Does ActiveMQ client/broker technology have any port discovery/negotation protocol?
ActiveMQ has discovery using IP Multicast and failover mechanisms. Not clear why it reconnects to the same host, failover config may have the same host. Client configuration would be helpful to understand.

Resources