My spring boot application uses Kafka as an appender for my logs using log4j2. The properties for the same are given below:
<Kafka name="kafkaAppender" topic="logs" ignoreExceptions="false" >
<PatternLayout pattern="%m%n" />
<Property name="bootstrap.servers">127.0.0.1:9092</Property>
</Kafka>
<Async name="Async" ignoreExceptions="false" >
<AppenderRef ref="kafkaAppender"/>
</Async>
The errors I get are :
kafka-producer-network-thread | producer-2] 2018-04-26 17:12:22,590 WARN org.apache.kafka.common.network.Selector [276] - Error in I/O with /127.0.0.1
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_144]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_144]
at org.apache.kafka.common.network.Selector.poll(Selector.java:238) [kafka-clients-0.8.2.0.jar:?]
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) [kafka-clients-0.8.2.0.jar:?]
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) [kafka-clients-0.8.2.0.jar:?]
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) [kafka-clients-0.8.2.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
To fix the above errors I used failover appender given by log4j2 the configuration for which is given below:
<Failover name="Failover" primary="Async" ignoreExceptions="true" >
<Failovers>
<AppenderRef ref="RollingRandomAccessFile"/>
</Failovers>
</Failover>
Still instead of going to the appender given in the failover, the application prints the above errors continously and the below :
2018-04-26 17:05:37,196 restartedMain ERROR appender Failover has no parameter that matches element Failovers
Is there any way to fix the issue as people on log4j2 forums say that its a bug for Failover tag.
There is OPEN bug as of August 2019 in log4j2, which is causing the issue!
kindly find link here
Related
I have installed NiFi using Homebrew following the instructions on this page.
Once I go start NiFi using
nifi start
I get the following:
Java home: /usr/local/opt/openjdk#11/libexec/openjdk.jdk/Contents/Home
NiFi home: /usr/local/Cellar/nifi/1.15.0/libexec
Bootstrap Config File: /usr/local/Cellar/nifi/1.15.0/libexec/conf/bootstrap.conf
Error: Could not find or load main class org.apache.nifi.bootstrap.RunNiFi
Caused by: java.lang.ClassNotFoundException: org.apache.nifi.bootstrap.RunNiFi
I also see this error in the nifi-app.log
2021-12-08 13:06:37,463 ERROR [Write-Ahead Local State Provider Maintenance] o.a.n.c.s.p.l.WriteAheadLocalStateProvider Failed to checkpoint Write-Ahead Log used to stor$
java.io.FileNotFoundException: ./state/local/partition-0/1.journal (No such file or directory)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:187)
at org.wali.MinimalLockingWriteAheadLog$Partition.rollover(MinimalLockingWriteAheadLog.java:788)
at org.wali.MinimalLockingWriteAheadLog.checkpoint(MinimalLockingWriteAheadLog.java:534)
at org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider$CheckpointTask.run(WriteAheadLocalStateProvider.java:286)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Any ideas?
I got the same error. Mine was due to the 8443 being occupied. I think, you can either find what's using the 8443 or you can try to change the port nifi.web.https.port=8443 in ./bin/conf/nifi.properties to something else. I hope it helps if you are still facing the issues.
Good afternoon everyone, the problem is this I have a server with SonarQube, that when I try to start the windows service, it gets up but then it stops.
The following error appears in the sonarqube log:
2017.11.14 11:04:52 WARN sea[o.e.transport.netty] [sonar-1510653879773] exception caught on transport layer [[id: 0x346b46fb, /127.0.0.1:59330 => /127.0.0.1:9001]], closing connection
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method) ~[na:1.8.0_152]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43) ~[na:1.8.0_152]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_152]
at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.8.0_152]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_152]
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64) [elasticsearch-1.1.2.jar:na]
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) [elasticsearch-1.1.2.jar:na]
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) [elasticsearch-1.1.2.jar:na]
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) [elasticsearch-1.1.2.jar:na]
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [elasticsearch-1.1.2.jar:na]
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [elasticsearch-1.1.2.jar:na]
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [elasticsearch-1.1.2.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_152]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_152]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_152]
2017.11.14 11:04:52 INFO app[o.s.p.m.TerminatorThread] Process[search] is stopping
2017.11.14 11:04:52 INFO sea[o.s.p.StopWatcher] Stopping process
Do you know why this error?
I have set the sonar.properties correctly, including set the value of the sonar.search.port property to 0 as this link suggests: Sonar launch error, but the problem persists.
I hope you can give me a hand...
Regards!!!
UnComment below line in sonar property file and change port 9001 to 0
#sonar.search.port=9001
sonar.search.port=0
I had the same problem and i could fix it like this:
Go to this folder: sonarqube-x.x\conf
Open this file: sonar.properties
Find the word: #sonar.web.port
Change the value from 9000 to another port, like 9002
Save your changes
Start again your sonarqube
Access to the server with port 9000: http://localhost:9000
The reason could be the port number of sonarQube OR the one of elasticSearch instance used by sonarQube (I had a similar problem before), so the step to change both/one of those ports are :
Go to this folder: sonarqube-x.x\conf
Open this file: sonar.properties
For sonarQube port:
Find: #sonar.web.port
Change the value from 9000 to another port, like 9123; and un-comment the line (remove # in the beginning) sonar.web.port=9123
For sonarQube's elasticSearch instace port:
Find: #sonar.search.port
change this line To sonar.search.port=0 (this means that he will search for any available port and bind it)
Save your changes
Start again your sonarqube
Access to the server with the new specified sonarQube-port: http://localhost:9123
I experienced this error when upgrading SonarQube from version 5.6.7 to 6.7.1.
Originally I thought this was due to the port number but upon checking the web.log I noticed that there was an error relating to the LDAP plugin (2.2.0.608).
ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube org.sonar.plugins.ldap.LdapException: The property 'ldap.url' is empty and no realm configured to try auto-discovery.
Updating the sonar.properties file with the correct configuration allowed SonarQube to start.
I just occurred an exactly same question as you did.
I started SonarQube with MariaDB 5.5, but I found some error messages in sonarqube-x.x/logs/web.log:
2021.01.21 14:36:17 INFO web[][o.s.p.ProcessEntryPoint] Starting web
......
2021.01.21 14:36:19 ERROR web[][o.s.s.p.Platform] Web server startup failed: Unsupported mysql version: 5.5. Minimal supported version is 5.6.
So I changed my database to MySQL 5.7 and it started successfully.
Not quite sure you had the same problem, but just check these log files and see what actually happened during starting.
I'm actually working on topology taking data from kafka and persist them into elasticsearch. Ok first, I used the basic KafkaSpout from storm dependency to listen for data coming from a precise kafka topic and, I re-implemented the Elasticsearch bolt from the elasticsearch-hadoop project: https://github.com/elastic/elasticsearch-hadoop/blob/master/storm/src/main/java/org/elasticsearch/storm/EsBolt.java. The goal was to write on several indices in elasticsearch.
So, when I process the messages coming from kafka, I have some exceptions when the number of data grow up in the kafka queue. This is one part of the stack trace in the worker logs:
2016-04-13T22:24:44.641+0000 b.s.m.n.Client [ERROR] failed to send 580 messages to Netty-Client-ip-[internal-ip].ec2.internal/[internal-ip]:6700:
java.nio.channels.ClosedChannelException
2016-04-13T22:24:44.641+0000 b.s.m.n.Client [ERROR] failed to send 575 messages to Netty-Client-ip-[internal-ip].ec2.internal/[internal-ip]:6700:
java.nio.channels.ClosedChannelException
2016-04-13T22:25:05.970+0000 b.s.m.n.Client [WARN] Re-connection to ip-[internal-ip].ec2.internal/[internal-ip]:6701 was successful but 52890 messages
has been lost so far
2016-04-13T22:36:33.571+0000 b.s.m.n.StormClientHandler [INFO] Connection failed Netty-Client-ip-ip-[internal-ip].ec2.internal/[internal-ip]:6701
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_77]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_77]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_77]
at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.8.0_77]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_77]
at org.apache.storm.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64) [storm-core-0.9.6.jar:0.9.6]
at org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) [storm-core-0.9.6.jar:0.9.6]
at org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) [storm-core-0.9.6.jar:0.9.6]
at org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) [storm-core-0.9.6.jar:0.9.6]
at org.apache.storm.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [storm-core-0.9.6.jar:0.9.6]
at org.apache.storm.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [storm-core-0.9.6.jar:0.9.6]
at org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [storm-core-0.9.6.jar:0.9.6]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
I'm using a storm cluster of 3 nodes (1 nimbus+UI+Zookeeper and 2 supervisors). Storm version 0.9.6. Each of these machines have 4GB RAM and this is the content my storm.yml config file:
storm.zookeeper.servers:
- "nimbus-ip"
storm.local.dir: "/mnt/storm"
nimbus.seeds: ["nimbus-ip"]
storm.zookeeper.port: 2181
ui.port: 8080
nimbus.host: "nimbus-ip"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
storm.messaging.netty.max_wait_ms: 10000
Can anyone help me to know why workers can't communicate due to Netty-Client hostname resolution? I already saw one report of this issue in the 0.9.4 version of storm https://issues.apache.org/jira/browse/STORM-908. Is it possible that the 0.9.6 version does not fix this issue?
Many thanks!!
I got here from google looking for answers to a similar problem. In my case, the error was:
o.a.s.m.n.Client [ERROR] connection attempt 104 to Netty-Client-ip-XXX-XXX-XXX-XXX.ec2.internal/XXX.XXX.XXX.XXX:6703 failed: java.net.ConnectException: Connection refused: ip-XXX-XXX-XXX-XXX.ec2.internal/XXX.XXX.XXX.XXX:6703
This was appearing on a 2-node storm cluster (v1.0.1).
At first, I thought this was a networking issue with AWS (which is where I was deploying the nodes). I started to look at security group rules, /etc/hosts files etc etc, none of which helped.
After some searching I discovered this: https://issues.apache.org/jira/browse/STORM-1382 and figured that maybe the issue wasn't the network at all, but something on the other end wasn't running.
So, I ssh-d into a worker node and took a look at the supervisor log, which showed me something like this lots and lots:
o.a.s.d.supervisor [INFO] 30236e62-d2e1-4d5c-b75c-f54ef07653a4 still hasn't started
When I looked at the worker.log itself, I discovered there was a problem with the default java version. That was my problem, but other people's problems may be related to other reasons that a worker may fail.
Anyway, once I set the correct default java version it all kicked into life.
I'm trying to monitor Tomcat8 with JDK8 using JMX.
I have setup my agents and modified the startup.sh.
On my zabbix_java_gateway.log I get the following exception:
WARN com.zabbix.gateway.SocketProcessor - error processing request
com.zabbix.gateway.ZabbixException: java.net.SocketTimeoutException:
connection timed out:
service:jmx:rmi:///jndi/rmi://server1.example.com:10052/jmxrmi
at com.zabbix.gateway.JMXItemChecker.getValues(JMXItemChecker.java:97)
~[zabbix-java-gateway-2.4.7.jar:na]
at com.zabbix.gateway.SocketProcessor.run(SocketProcessor.java:63)
~[zabbix-java-gateway-2.4.7.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_71] Caused by: java.net.SocketTimeoutException: connection timed out:
service:jmx:rmi:///jndi/rmi://server1.example.com:10052/jmxrmi
at com.zabbix.gateway.ZabbixJMXConnectorFactory.connect(ZabbixJMXConnectorFactory.java:123)
~[zabbix-java-gateway-2.4.7.jar:na]
at com.zabbix.gateway.JMXItemChecker.getValues(JMXItemChecker.java:89)
~[zabbix-java-gateway-2.4.7.jar:na]
... 4 common frames omitted
On my startup.sh I added the following to the CATALINA_OPTS
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10052 -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.password.file=/opt/tomcat-latest/conf/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=/opt/tomcat-latest/conf/jmxremote.access
-Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=server1.example.com
My zabbix_agentd.conf contains the following:
PidFile=/tmp/zabbix_agentd.pid LogFile=/var/log/zabbix_agentd.log
LogFileSize=1 DebugLevel=3 Server=monitor.example.com
Hostname=server1.example.com ListenPort=10050 StartAgents=5 Timeout=30
I have already done the following:
successfully connected to the server using jconsole
remove authentication
telnet the server over port 10050 / 10052
The weird part is that the same setup works well for Tomcat6 with JDK7.
EDIT 1
I've updated the JDK version on the zabbix server to be newer than the JDK installed on my JAVA nodes - still same result - it ends with
ZBX_TCP_READ() failed: [4] Interrupted system call
UPDATE
So I figured it out eventually.
I had on my tomcat configuration file -Djava.rmi.server.hostname=server1.example.com
I miss understood that the hostname should be set to the monitoring server and to the monitored server hostname.
Apparently, there's a bug on Tomcat 6 and this directive does not work.
Remove it solved the problem completely.
Thanks,
Liron
I am facing one strange problem , I am working on one batch application using below tech STACK
Spring (IOC container) , spring batch , Hibernate , camel and Log4J for logging
I have configured log4J for file appender only and my all application logs goes to my log file , but strange thing is that all Camel logs goes to console appender (I have no console apender configured on my log4J properties file)
Spring – Camel Integration is XMl element :
<camelContext xmlns="http://camel.apache.org/schema/spring">
<routeBuilder ref="smtBatchRouter" />
</camelContext>
My Log4J configuration:
log4j.rootLogger=TRACE, file
log4j.logger.org.apache=TRACE
log4j.logger.org.directwebremoting=OFF
log4j.logger.org.hibernate=OFF
log4j.logger.org.springframework=WARN
log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.file=myapp.log
log4j.appender.file.ImmediateFlush=true
log4j.appender.file.bufferedIO=false
log4j.appender.file.DatePattern='.' dd-MM-yyyy
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d [%-15.15t] %-5p %-30.30c{1} - %m%n
log4j.appender.file.threshold=TRACE
Sample Camel Log that I am getting in console :
[main] INFO org.apache.camel.spring.SpringCamelContext - Apache Camel 2.11.0 (CamelContext: camel-1) started in 4.009 seconds
[Camel (camel-1) thread #0 - sftp://xyz.com/smtDataFolder] INFO org.apache.camel.component.file.remote.SftpOperations - JSCH -> Connecting to localhost port 22
[Camel (camel-1) thread #0 - sftp://xyz.com/smtDataFolder] INFO org.apache.camel.component.file.remote.SftpOperations - JSCH -> Connection established
[Camel (camel-1) thread #0 - sftp://xyz.com/smtDataFolder] INFO org.apache.camel.component.file.remote.SftpOperations - JSCH -> Remote version string: SSH-1.99-OpenSSH_4.6
I had the same issue and solved it.
Run your application with the flag:
-Dlog4j.debug
It will add diagnostic information to your console. In my case it was
SLF4J: Class path contains multiple SLF4J bindings.
So I just deleted redundant slf4j-jdk14-1.7.9.jar and left only slf4j-log4j12-1.7.19.jar.