I added the sleuth to a microservice and successfully get the Trace Id and span Id, but as you can see after sleuth, missing a part of the info before the log level in the log line.
I think maybe I can disable sleuth in a debug level, to have log line as before and then enable it in the info level.
do you know how or maybe a better solution?
before Sleuth:
2022-03-24 09:53:22.338 [(ID:9dcebfa9-b37f-4a94-a91b-45867ea90811)(CUSTOMER ID:*** TYPE:UNKNOWN)] DEBUG 3180716 --- [https-jsse-nio-48082-exec-4] c.h.d.s.staticdata.StaticDataService : start with version: 1.2 OS: ANDROID
after Sleuth:
2022-03-24 10:19:34.541 DEBUG [*****,efcbdb6dc513e468,efcbdb6dc513e468,true] 3368114 --- [https-jsse-nio-48082-exec-7] c.h.d.s.staticdata.StaticDataService : start with version: 1.2 OS: ios
Related
How to fix this issue?
Spring boot version:2.4.5
I am using gradle and dependencies used related to elastic search are:
implementation 'org.springframework.boot:spring-boot-starter-data-elasticsearch'
&
implementation 'co.elastic.clients:elasticsearch-java:7.17.3'
2023-02-10 20:51:45.788 INFO 1 --- [main] o.s.d.elasticsearch.support.VersionInfo : Version Spring Data Elasticsearch: 4.1.8
2023-02-10 20:51:45.793 INFO 1 --- [main] o.s.d.elasticsearch.support.VersionInfo : Version Elasticsearch Client in build: 7.9.3
2023-02-10 20:51:45.794 INFO 1 --- [main] o.s.d.elasticsearch.support.VersionInfo : Version Elasticsearch Client used: 7.9.3
2023-02-10 20:51:45.795 INFO 1 --- [main] o.s.d.elasticsearch.support.VersionInfo : Version Elasticsearch cluster: 8.6.1
2023-02-10 20:51:45.795 WARN 1 --- [main] o.s.d.elasticsearch.support.VersionInfo : Version mismatch in between Elasticsearch Client and Cluster: 7.9.3 - 8.6.1
The issue is a warning because the major version between the Elasticsearch client that is used and the Elasticsearch server do not match. Spring Data Elasticsearch was not tested and released against ES 8. So anything might happen - no guarantees.
How to solve this? Upgrade your application to Spring Boot 3, that uses Spring Data Elasticsearch 5.0, which in turn uses Elasticsearch 8.6.
Spring Data Elasticsearch 4.1 is out of maintenance since May 2021.
I have created a hilla app using this:
npx #vaadin/cli init --hilla --auth hilla-with-auth
Works fine!
Now I am trying to add that to an existing spring boot application, but I am having issues with Vite not starting as it should. No exception. No help in the debug output.
I have added:
relevant files in the root (package.json, vite.config.ts, etc.)
vaadin-featureflags.properties in resources folder
hilla dependencies in pom.xml as well as the build plugin
hill annotations to my application class (#Theme, #PWA) and made it extends SpringBootServletInitializer implements AppShellConfigurator
But even if everything seems to be initialized correctly, Vite does not start. Can anyone guide me in the right direction?
This is the most relevant output log:
2022-06-07 08:04:54.046 DEBUG 4947 --- [restartedMain] c.v.f.s.f.s.FullDependenciesScanner :
List of npm dependencies found in the project:
- #hilla/form 1.0.1 dev.hilla.EndpointController
....
2022-06-07 08:04:54.057 DEBUG 4947 --- [restartedMain] c.v.f.s.f.TaskGeneratePackageJson : writing file /Users/michael/Development/Previsto/previsto-server/target/flow-frontend/package.json.
2022-06-07 08:05:02.102 INFO 4947 --- [restartedMain] o.a.container.JSR356AsyncSupport : JSR 356 Mapping path /vaadinServlet
2022-06-07 08:05:02.176 INFO 4947 --- [restartedMain] c.v.f.s.DefaultDeploymentConfiguration :
Vaadin is running in DEBUG MODE.
When deploying application for production, remember to disable debug features. See more from https://vaadin.com/docs/
The following EXPERIMENTAL features are enabled:
- Use Vite for faster front-end builds
2022-06-07 08:05:02.207 DEBUG 4947 --- [restartedMain] c.v.f.s.c.PushRequestHandler : Using pre-initialized Atmosphere for servlet springServlet
2022-06-07 08:05:02.210 DEBUG 4947 --- [restartedMain] c.v.flow.server.VaadinServletService : Using 'com.vaadin.flow.server.communication.IndexHtmlRequestHandler' in client mode bootstrapping
2022-06-07 08:05:02.212 DEBUG 4947 --- [restartedMain] com.vaadin.flow.server.VaadinService : The application has the following routes:
2022-06-07 08:05:02.956 INFO 4947 --- [restartedMain] c.v.flow.server.frontend.FrontendTools : Project node version 16.10.0 is older than 16.14.0. Using node from /Users/michael/.vaadin.
2022-06-07 08:05:03.404 DEBUG 4947 --- [http-nio-8080-exec-1] c.v.f.s.s.VaadinDefaultRequestCache : Saving request to /
2022-06-07 08:05:03.471 DEBUG 4947 --- [http-nio-8080-exec-2] c.v.b.devserver.AbstractDevServerRunner : Requesting resource from Vite http://localhost:0/login
2022-06-07 08:05:03.486 ERROR 4947 --- [http-nio-8080-exec-2] o.a.c.c.C.[.[.[/].[springServlet] : Servlet.service() for servlet [springServlet] threw exception
java.net.ConnectException: Can't assign requested address (connect failed)
at ...
I just migrated to a new spring boot version lately, namely v2.6.2 and now I find the following exception of some of the services logs.
23:27:45.148 [main] INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat started on port(s): 8080 (https) with context path ''
23:27:47.046 [main] INFO o.a.c.i.engine.AbstractCamelContext - Routes startup (total:1 started:1)
23:27:47.048 [main] INFO o.a.c.i.engine.AbstractCamelContext - Started route1 (jms://queue:inp.contentextractor.entry)
23:27:47.049 [main] INFO o.a.c.i.engine.AbstractCamelContext - Apache Camel 3.14.0 (camel-1) started in 2s228ms (build:622ms init:1s32ms start:574ms)
23:27:47.061 [main] INFO eu.hermes.esb.cloud.Application - Started Application in 25.305 seconds (JVM running for 27.737)
23:28:20.356 [https-jsse-nio-8080-exec-4] INFO o.a.c.c.C.[Tomcat].[localhost].[/] - Initializing Spring DispatcherServlet 'dispatcherServlet'
23:28:20.357 [https-jsse-nio-8080-exec-4] INFO o.s.web.servlet.DispatcherServlet - Initializing Servlet 'dispatcherServlet'
23:28:20.359 [https-jsse-nio-8080-exec-4] INFO o.s.web.servlet.DispatcherServlet - Completed initialization in 2 ms
00:05:19.986 [https-jsse-nio-8080-Acceptor] ERROR org.apache.tomcat.util.net.Acceptor - Socket accept failed
java.io.IOException: Duplicate accept detected. This is a known OS bug. Please consider reporting that you are affected: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
at java.base/java.lang.Thread.run(Thread.java:834)
05:20:49.981 [https-jsse-nio-8080-Acceptor] ERROR org.apache.tomcat.util.net.Acceptor - Socket accept failed
java.io.IOException: Duplicate accept detected. This is a known OS bug. Please consider reporting that you are affected: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
at java.base/java.lang.Thread.run(Thread.java:834)
What does it mean and how harmful is it ?
Following the link provided in the bug (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298), and looking into comments made by community members, it seems that there is something in tomcat usage that uncovered a hidden issue in linux kernal 5.10-rc4 and was most likely fixed in 5.10-rc6.
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298/comments/15
This issue was reported across centOS, amazon linux, Arch linux and even Window 2019 - when using a lower linux kernel than 5.10.
The conclusion from this discussion seems to be the suggestion to update the OS itself to a stable version that uses linux kernel >= 5.10
I have a Spring Boot application which I'm building and running with Java 10. If I run the app using
java -jar
Everything works fine. The app starts just OK.
But if I put my app inside a Docker container with the exactly same Java version, my app throws this exception:
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.jgroups.logging.Slf4jLogImpl
at org.jgroups.logging.LogFactory.getLog(LogFactory.java:101)
at org.jgroups.conf.XmlConfigurator.<clinit>(XmlConfigurator.java:33)
at org.jgroups.conf.ConfiguratorFactory.getStackConfigurator(ConfiguratorFactory.java:62)
at org.jgroups.JChannel.<init>(JChannel.java:122)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.buildChannel(JGroupsTransport.java:591)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.initChannel(JGroupsTransport.java:405)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:389)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
... 104 common frames omitted
I'm using this version of Java:
java version "10.0.2" 2018-07-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)
Docker version is:
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:21:31 2018
OS/Arch: darwin/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:29:02 2018
OS/Arch: linux/amd64
Experimental: true
My Docker is using an Alpine base image alpine:latest. I'm installing java in my container from this link:
curl -jksSLH "Cookie: oraclelicense=accept-securebackup-cookie" -o /tmp/java.tar.gz \
http://download.oracle.com/otn-pub/java/jdk/10.0.2+13/19aef61b38124481863b1413dce1855f/jdk-10.0.2_linux-x64_bin.tar.gz
I'm really confused because from outside a docker container my app works fine, but inside a docker container it doesn't. Either case I'm using the same Java version.
UPDATE
We tried Oracle JDK and OpenJDK, same behavior
UPDATE 2
We even tried java -jar from inside the container, no luck
TL;DR
There are three options to resolve.
Upgrade to version 4.0.16 of JGroups, currently in SNAPSHOT. Edit: Now released here.
Make sure java properties for "user.language" and "user.country" are set.
Force JDKLogImpl with -Djgroups.use.jdk_logger=true. (mentioned by Perimosh)
Explanation
Ran into this issue in the following scenario.
Apache Camel + JGroups worked fine in a local environment. Deployed it elsewhere in a Docker instance, where we got the following stacktrace:
2018-11-19 13:38:03.063 INFO 582 --- [ main] o.a.camel.spring.boot.RoutesCollector : Loading additional Camel XML routes from: classpath:camel/*.xml
2018-11-19 13:38:03.064 INFO 582 --- [ main] o.a.camel.spring.boot.RoutesCollector : Loading additional Camel XML rests from: classpath:camel-rest/*.xml
2018-11-19 13:38:03.107 INFO 582 --- [ main] o.a.camel.spring.SpringCamelContext : Apache Camel 2.22.2 (CamelContext: camel-1) is starting
2018-11-19 13:38:03.111 INFO 582 --- [ main] o.a.c.m.ManagedManagementStrategy : JMX is enabled
2018-11-19 13:38:03.480 INFO 582 --- [ main] o.a.camel.spring.SpringCamelContext : StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2018-11-19 13:38:03.597 INFO 582 --- [ main] o.a.camel.spring.SpringCamelContext : Apache Camel 2.22.2 (CamelContext: camel-1) is shutting down
2018-11-19 13:38:03.616 WARN 582 --- [ main] o.a.camel.spring.SpringCamelContext : Error occurred while shutting down service: org.apache.camel.component.jgroups.cluster. JGroupsLockClusterService#10fa5af5. This exception will be ignored.
java.lang.NullPointerException: null
at org.apache.camel.component.jgroups.cluster.JGroupsLockClusterView.doStop(JGroupsLockClusterView.java:109)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102)
at org.apache.camel.impl.cluster.AbstractCamelClusterService.lambda$doStop$2(AbstractCamelClusterService.java:134)
at org.apache.camel.util.concurrent.LockHelper.doWithReadLockT(LockHelper.java:54)
at org.apache.camel.impl.cluster.AbstractCamelClusterService.doStop(AbstractCamelClusterService.java:130)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102)
at org.apache.camel.util.ServiceHelper.stopService(ServiceHelper.java:142)
at org.apache.camel.util.ServiceHelper.stopAndShutdownService(ServiceHelper.java:205)
at org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:3663)
at org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:3688)
at org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:3676)
at org.apache.camel.impl.DefaultCamelContext.doStop(DefaultCamelContext.java:3567)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102)
at org.apache.camel.impl.DefaultCamelContext.stop(DefaultCamelContext.java:3220)
at org.apache.camel.spring.SpringCamelContext.stop(SpringCamelContext.java:148)
...
2018-11-19 13:38:03.679 INFO 582 --- [ main] o.a.camel.spring.SpringCamelContext : Apache Camel 2.22.2 (CamelContext: camel-1) uptime 0.570 seconds
2018-11-19 13:38:03.680 INFO 582 --- [ main] o.a.camel.spring.SpringCamelContext : Apache Camel 2.22.2 (CamelContext: camel-1) is shutdown in 0.082 seconds
2018-11-19 13:38:03.716 INFO 582 --- [ main] org.mongodb.driver.connection : Closed connection [connectionId{localValue:2, serverValue:2}] to localhost:43115 because the pool has been closed.
As you can see, Apache Camel attempts to start, but never does and ends up shutting down. Thus, JGroups gets a NPE because it expects Camel to be up.
After debugging the code, it appeared that there was an exception being thrown during the Camel start up process which was getting eaten.
From there, discovered that the creation of an instance of Slf4jLogImpl in org.jgroups.logging.LogFactory#getLog(java.lang.Class<?>) (new Slf4jLogImpl(clazz)) was a problem Method threw 'java.lang.ExceptionInInitializerError' exception.:
java.lang.NullPointerException: null
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.jgroups.logging.LogFactory.getLog(LogFactory.java:101)
at org.jgroups.conf.XmlConfigurator.<clinit>(XmlConfigurator.java:33)
at org.jgroups.conf.ConfiguratorFactory.getXmlConfigurator(ConfiguratorFactory.java:210)
at org.jgroups.conf.ConfiguratorFactory.getStackConfigurator(ConfiguratorFactory.java:91)
at org.jgroups.JChannel.<init>(JChannel.java:130)
...
Running (new Slf4jLogImpl(clazz)) the second time onward in the debugger results in the following stacktrace that mirrors the original posted issue:
java.lang.NoClassDefFoundError: Could not initialize class org.jgroups.logging.Slf4jLogImpl
at org.jgroups.logging.LogFactory.getLog(LogFactory.java:101)
at org.jgroups.conf.XmlConfigurator.<clinit>(XmlConfigurator.java:33)
at org.jgroups.conf.ConfiguratorFactory.getXmlConfigurator(ConfiguratorFactory.java:210)
at org.jgroups.conf.ConfiguratorFactory.getStackConfigurator(ConfiguratorFactory.java:91)
at org.jgroups.JChannel.<init>(JChannel.java:130)
This difference in results is due to the class loader caching the result of the Class.forName() call previously to determine that the class definition was not found.
Finally, we tracked the previous NPE to be thrown from java.util.Locale#Locale(java.lang.String, java.lang.String, java.lang.String), since country was null. This is because JGroup's org.jgroups.logging.Slf4jLogImpl is definine a LOCALE field using java properties for "user.language" and "user.country". The former was not set in our docker instance, so Locale.java threw the NPE. Adding both of these java properties should fix this issue. Alternatively you can force using the JDKLogImpl so that the Slf4jLogImpl is never attempted to be instantiated. This was mentioned in the previous answer, by passing in -Djgroups.use.jdk_logger=true.
Edit: Fixed in the latest version released here.
Now, it looks like this will be fixed in the upcoming JGroup release of 4.0.16.Final (https://github.com/belaban/JGroups/commit/61578c657138f02178c32a564ac9eae7c3976093#diff-93eb0f6a8a4953312098be459bd7ce76). Until then, you can get the snapshot version with the fix at https://repository.jboss.org/nexus/content/repositories/snapshots/org/jgroups/jgroups/4.0.16-SNAPSHOT/.
This is not a real solution, but since it unblocked us, I will share it. Also, maybe somebody can think about the real problem by looking at this workaround. We added this JVM argument to by pass SLF4J for jgroups and use JDKLogImpl
-Djgroups.use.jdk_logger=true
We are using spring ehcache in our application, recently we upgraded the java version of the tomcat in which the app runs to 1.8.66.
Strangely there are no errors in the logs - but caching doesn't work on the linux machine.
On my local (windows) same version of java 1.8.66, it works like a charm... what could be the issue?
We are using aspecJ load time weaving with TomcatInstrumentableClassLoader - version 3.2.3.
Details of dependencies:
ehcache 2.7.4
aspectJ weaver 1.6.2
spring 3.1.1
INFO 2015-12-28 14:09:19,667 (org.springframework.cache.ehcache.EhCacheManagerFactoryBean.afterPropertiesSet(EhCacheManagerFactoryBean.java:100)) - Initializing EHCache CacheManager
INFO 2015-12-28 14:09:19,828 (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter.<clinit>(AnnotationSizeOfFilter.java:53)) - Using regular expression provided through VM argument net.sf.ehcache.pool.sizeof.ignore.pattern for IgnoreSizeOf annotation : ^.*cache\..*IgnoreSizeOf$
INFO 2015-12-28 14:09:19,873 (net.sf.ehcache.pool.sizeof.JvmInformation.<clinit>(JvmInformation.java:446)) - Detected JVM data model settings of: 64-Bit HotSpot JVM with Compressed OOPs
INFO 2015-12-28 14:09:20,190 (net.sf.ehcache.pool.sizeof.AgentLoader.getAgentFile(AgentLoader.java:198)) - Extracted agent jar to temporary file /opt/projects/tomcat/instance_7/temp/ehcache-sizeof-agent7845091970407661363.jar
INFO 2015-12-28 14:09:20,190 (net.sf.ehcache.pool.sizeof.AgentLoader.loadAgent(AgentLoader.java:138)) - Trying to load agent # /opt/projects/tomcat/instance_7/temp/ehcache-sizeof-agent7845091970407661363.jar
INFO 2015-12-28 14:09:20,199 (net.sf.ehcache.pool.impl.DefaultSizeOfEngine.<init>(DefaultSizeOfEngine.java:111)) - using Agent sizeof engine
INFO 2015-12-28 14:09:20,456 (net.sf.ehcache.pool.impl.DefaultSizeOfEngine.<init>(DefaultSizeOfEngine.java:111)) - using Agent sizeof engine
INFO 2015-12-28 14:09:20,746 (net.sf.ehcache.pool.impl.DefaultSizeOfEngine.<init>(DefaultSizeOfEngine.java:111)) - using Agent sizeof engine
INFO 2015-12-28 14:09:20,768 (net.sf.ehcache.pool.impl.DefaultSizeOfEngine.<init>(DefaultSizeOfEngine.java:111)) - using Agent sizeof engine
INFO 2015-12-28 14:09:20,789 (net.sf.ehcache.pool.impl.DefaultSizeOfEngine.<init>(DefaultSizeOfEngine.java:111)) - using Agent sizeof engine
I can see same logs in both Windows and Linux after application starts up
Could this be a issue with Linux JVM?
I think that it's a problem with spring 3.x.x and Java 8.
Take a look here Spring 3.2.x with Java 8