SpringXD -> twitterstream --follow: ending with Http error - spring-xd

I'm trying to crate a stream that should follow #BBCBreaking (what should have 5402612 twitter ID), but I keep getting following Http error:
2016-03-28T02:13:12+0200 1.3.1.RELEASE INFO DeploymentSupervisor-0 zk.ZKStreamDeploymentHandler - Deployment status for stream 'mystream': DeploymentStatus{state=deployed}
2016-03-28T02:13:13+0200 1.3.1.RELEASE WARN twitterSource-1-1 twitter.TwitterStreamChannelAdapter - Http error, waiting for 5 seconds before restarting
2016-03-28T02:13:19+0200 1.3.1.RELEASE WARN twitterSource-1-1 twitter.TwitterStreamChannelAdapter - Http error, waiting for 10 seconds before restarting
2016-03-28T02:13:30+0200 1.3.1.RELEASE WARN twitterSource-1-1 twitter.TwitterStreamChannelAdapter - Http error, waiting for 20 seconds before restarting
my stream command is:
stream create --name mystream --definition "twitterstream --follow='5402612' | log" --deploy
running on SpringXD: 1.3.1.RELEASE
please, any idea that why the error?

You can debug such situations by enabling DEBUG logging - log config is in the xd/config folder in .groovy files; e.g. xd-singlenode-logback.groovy.
Set the loggers for org.springframework.integration and org.springframework.xd, org.springframework.xd.dirt.server to DEBUG and add a logger for org.springframework.social.twitter also at DEBUG.
Or you can set all of org.springframework and comment out the more specific ones.

Related

Spring Boot AWS logging level

I have several Spring Boot micro services that connect to Config Server and Eureka. All are running inside Docker container in ECS. They are very chatty and I would like to eliminate the non-critical messages.
2023-01-06 16:42:03.426 WARN 1 --- [tbeatExecutor-0]
c.n.d.s.t.d.RetryableEurekaHttpClient : Request execution failed with
message... 2023-01-06 16:43:28.082 INFO 1 --- [trap-executor-0]
c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via
configuration... 2023-01-06 16:50:03.679 INFO 1 --- [freshExecutor-0]
o.apache.http.impl.execchain.RetryExec... 2023-01-06 16:42:03.426 INFO
1 --- [tbeatExecutor-0] c.n.d.s.t.d.RedirectingEurekaHttpClient...
And each service has its application.properties file with the following:
logging.level.org.springframework.cloud.netflix.eureka.http=FATAL
logging.level.com.netflix.discovery=FATAL
logging.level.com.netflix.discovery.shared.resolver.aws = ERROR
logging.level.com.netflix.eureka.registry = ERROR
logging.level.org.springframework.web.servlet = FATAL
logging.level.org.hibernate = ERROR logging.level.com.zaxxer.hikari =
ERROR logging.level.org.apache.tomcat=ERROR
logging.level.org.apache.catalina=ERROR
But the messages continue to show up in the log files. I don't get why this is, since those messages are INFO and WARN.
In order to disable below error messages use the config below
logging.level.com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient=error
also in your config I see logging.level.org.springframework.web.servlet = FATAL logging.level.org.hibernate = this looks wrong, every line should have one gorup name = something, then new line.

Quarkus test LauncherUtil fails searching for "Listening on:" in logs

I am building a Lambda handler and tests are failing and Quarkus Test is looking for a message to appear in the logs in order to execute Pattern.compile("Listening on:\\s+(https?)://\\S*:(\\d+)") in CaptureListeningDataReader.
The failure happens because all the logs I see are:
2021-06-29 16:53:06,663 WARN [io.qua.config] (main) Unrecognized configuration key "quarkus.http.ssl-port" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-06-29 16:53:06,663 WARN [io.qua.config] (main) Unrecognized configuration key "quarkus.http.port" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
...
2021-06-29 16:53:07,515 INFO [io.quarkus] (main) foo-authorizer-lambda 1.0.0-SNAPSHOT on JVM (powered by Quarkus 1.13.7.Final) started in 1.249s.
Perhaps I'm missing a plugin assumed to be present, thus the two WARN messages and a missing "Listening on:" in the logs?

Spring Application not getting terminate for timeout exception

I have created a spring boot application to publish the message to the Kafka queue. For that, I am using spring cloud stream and Kafka binder as dependencies. Problem is my application is continuously trying to connect to Kafka broker if it is down for 2 minutes because of the default configuration.
I have reduced that time using the below property and set it to 1000 ms and getting the timeout exception
spring.kafka.properties.request.timeout.ms:1000.
But still, my spring application is running after the exception. I want it to fail if Kafka broker is not available to connect to. I have tried one more property for that spring.kafka.admin.fail-fast=true but still, the application is running.
I have also tried to search for some properties of spring cloud stream and Kafka binder that I can set to fail my application if Kafka broker is not available but couldn't find anything related to that.
Please, help me with this.
Please see below for the log of exception.
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:274)
at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner.createTopicAndPartitions(KafkaTopicProvisioner.java:351)
at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner.createTopicIfNecessary(KafkaTopicProvisioner.java:325)
at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner.createTopic(KafkaTopicProvisioner.java:302)
... 33 common frames omitted
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
2019-05-22 06:06:25 [main] DEBUG o.s.c.s.DefaultLifecycleProcessor - Successfully started bean 'outputBindingLifecycle'
2019-05-22 06:06:25 [main] DEBUG o.s.c.s.DefaultLifecycleProcessor - Starting beans in phase 2147482647
2019-05-22 06:06:25 [main] DEBUG o.s.c.s.binding.BindableProxyFactory - Binding inputs for :interface kafka.stream.RXXXStreams
2019-05-22 06:06:25 [main] DEBUG o.s.c.s.DefaultLifecycleProcessor - Successfully started bean 'inputBindingLifecycle'
2019-05-22 06:06:25 [main] DEBUG o.s.c.s.DefaultLifecycleProcessor - Starting beans in phase 2147483547
2019-05-22 06:06:25 [main] DEBUG o.s.c.s.DefaultLifecycleProcessor - Successfully started bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'
2019-05-22 06:06:25 [main] DEBUG o.s.b.a.l.ConditionEvaluationReportLoggingListener -
Do you have spring-boot-web libraries as dependency? If that's the case, your application will not exit. A full log will be also very helpful.

Apache NiFi closes immediately with error, org.apache.nifi.StdErr Failed to start web server

I have recently installed NiFi, and it was working fine for few days. But suddenly today when i try to open it using run-nifi.bat, the NiFi window is getting closed in few seconds stating the below error:
2019-04-11 23:07:40,146 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read status file C:\Users\DOWNLO~1\NIFI-1~1.1-B\NIFI-1~1.1\bin\..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2019-04-11 23:07:40,149 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 54149
2019-04-11 23:08:00,352 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Must configure HTTP or HTTPS connector
2019-04-11 23:08:00,352 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2019-04-11 23:08:00,419 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
I do looked out for the org.apache.nifi.StdErr Failed to start web server: Must configure HTTP or HTTPS connector error, but unfortunately I cant find a similar one. I'm sure that no settings or properties has been changed since installation. Any suggestion guys?
I was getting the same error.
You need to check nifi-app.log file to get more details on this type of error.
Here’s what I did: Remove the port information from nifi.properties for HTTPS and only keep the setting for HTTP. Then restart nifi again.
Keep one property enable like https or http. in my case https:127.0.0.1:8443/nifi/
it works fine me.

Not able to access RDS in my spring boot application inside docker

I wanted to dockerize my spring boot application. I used
docker build -t <Name> .
to build my project and then try to run it. But my spring boot application gives Communication link failure and is not able to connect to the rds database. I have setup database url correctly in properties file and my application is inside same VPC as the RDS. My docker file is:
FROM maven:3.5.2-jdk-8-alpine AS MAVEN_TOOL_CHAIN
COPY pom.xml /tmp/
WORKDIR /tmp/
RUN mvn verify clean --fail-never
COPY src /tmp/src/
RUN mvn package -DskipTests
FROM tomcat:8.0.51-jre8-alpine
COPY --from=MAVEN_TOOL_CHAIN /tmp/target/<ProjectName>.war $CATALINA_HOME/webapps/<ProjectName>.war
COPY ./conf/* $CATALINA_HOME/conf/
CMD ["catalina.sh","run"]
Stack trace of the error im getting is below:
2019-03-14 14:17:03.400 WARN 1 --- [pool-3-thread-1] o.h.e.j.s.SqlExceptionHelper : SQL Error: 0, SQLState: 08S01
2019-03-14 14:17:03.400 ERROR 1 --- [pool-3-thread-1] o.h.e.j.s.SqlExceptionHelper : Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
2019-03-14 14:17:03.400 INFO 1 --- [pool-3-thread-1] c.d.c.s.c.CampaignAnalyticsService : error in updating analytics Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
2019-03-14 14:17:03.400 INFO 1 --- [pool-3-thread-1] c.d.c.s.c.CampaignAnalyticsService : [543, Send, Announcement, -, Sukhna:, Run, for, fun]
2019-03-14 14:17:03.401 INFO 1 --- [pool-3-thread-1] c.z.h.HikariDataSource : HikariPool-1 - Starting...
2019-03-14 14:17:04.402 ERROR 1 --- [pool-3-thread-1] c.z.h.p.HikariPool : HikariPool-1 - Exception during pool initialization.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
I use the following command to run my image :
docker run -p 8081:8000 <Name>

Resources