What depends on netty-tcnative in Quarkus? - quarkus

This pertains to a GitHub issue where TensorFlow is being loaded at runtime for native image:
https://github.com/awslabs/djl/issues/103#issuecomment-658549779
It causes a io.netty.internal.tcnative.SSLPrivateKeyMethod class not found error during native image compilation. If I add this dependency, the error goes away:
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative</artifactId>
<version>2.0.31.Final</version>
</dependency>
Here is the stack trace:
Fatal error:java.lang.NoClassDefFoundError
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:463)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:359)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:518)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:117)
at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:546)
Caused by: java.lang.NoClassDefFoundError: io/netty/internal/tcnative/SSLPrivateKeyMethod
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.getDeclaredMethods(Native Method)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.getDeclaredMethods(HotSpotResolvedObjectTypeImpl.java:958)
at com.oracle.svm.jni.access.JNIAccessibleMethod.anyMatchIgnoreReturnType(JNIAccessibleMethod.java:139)
at com.oracle.svm.jni.access.JNIAccessibleMember.findHidingSubclasses(JNIAccessibleMember.java:103)
at com.oracle.svm.jni.access.JNIAccessibleMember.findHidingSubclasses(JNIAccessibleMember.java:110)
at com.oracle.svm.jni.access.JNIAccessibleMember.setHidingSubclasses(JNIAccessibleMember.java:83)
at com.oracle.svm.jni.access.JNIAccessibleMethod.finishBeforeCompilation(JNIAccessibleMethod.java:135)
at com.oracle.svm.jni.access.JNIAccessFeature.beforeCompilation(JNIAccessFeature.java:348)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$doRun$2(NativeImageGenerator.java:595)
at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:70)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:595)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:451)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLPrivateKeyMethod
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 18 more
It is definitely related to one of the following configs, and does also seem to be impacted by JNI, but I am not sure how.
quarkus.native.enable-jni=true
quarkus.ssl.native=true
quarkus.native.additional-build-args=--initialize-at-run-time=org.tensorflow\\,ai.djl\\,org.bytedeco
In searching through the Quarkus GitHub repos, I cannot find reference to netty-tcnative. I really do not understand why this is needed as it seems to be related to Tomcat?
In order to fix this problem, or file a GitHub issue, need to understand where this dependency is expected.

Related

AWS glue NoClassDefFoundError on job.init()

Trying to debug AWS Glue scripts locally using Glue ETL library.
I have installed aws-glue-libs and spark-3.1.1-amzn-0-bin-3.2.1-amzn-3.tgz.
When I run job.init(), I get the following error trace:
py4j.protocol.Py4JJavaError: An error occurred while calling z:com.amazonaws.services.glue.util.Job.init.
: java.lang.NoClassDefFoundError: com/typesafe/config/ConfigMergeable
at com.amazonaws.services.glue.util.Job$.init(Job.scala:93)
at com.amazonaws.services.glue.util.Job.init(Job.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.typesafe.config.ConfigMergeable
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 13 more
This error wasn't resolved per say. But i found a workaround. Instead of running my scripts from pycharm, i run them using gluesparksubmit bash command. Now it doesn't throw error at job.init(). Still trying to figure out how to get access to data catalog when running glue scripts from local machine.
If it was for ConfigMergeable, check if proper jar file - config-1.3.3.jar - exists in your /opt/spark/jars dir.
Whole idea is that jars in /opt/spark/jars and ./aws-glue-libs/jarsv1 should match.

ClassNotFound exception when I run my packaged jar, it works when launching it using Idea

The strange thing is that I don't use the missing class DispatchApplication and because I have no error when launching it using idea, so i do not know how to debug it.
I tried to clean idea cache, clean local maven repository, restart my PC
but my colleague can run with a jar, and each of them don't know why this happen and how solve this
D:\projectFolder\midel-data\data-integration\data-integration-service\target>java -jar data-integration-service-1.0.jar
Exception in thread "main" java.lang.ClassNotFoundException: com.lty.dispatch.DispatchApplication
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:151)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:46)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
This looks to be an issue with dependencies.
You might want to check your IDE settings if you have explicitly enabled any additional jars in the class path.
Also, check out your pom.xml/.gradle file for any difference in dependency versions.
If this class is part of some custom jar, make sure you include it while packaging your jar.

Reactive version of quarkus-keycloak-admin-client extension?

Is there a reactive version of the quarkus-keycloak-admin-client extension?
When I add the dependency:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-keycloak-admin-client</artifactId>
</dependency>
I am getting the following exception:
2021-11-21 18:27:51,546 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[error]: Build step io.quarkus.deployment.steps.CapabilityAggregationStep#aggregateCapabilities threw an exception: java.lang.IllegalStateException: Please make sure there is only one provider of the following capabilities:
capability io.quarkus.rest.client is provided by:
- io.quarkus:quarkus-rest-client-reactive::jar:2.5.0.Final
- io.quarkus:quarkus-rest-client::jar:2.5.0.Final
at io.quarkus.deployment.steps.CapabilityAggregationStep.aggregateCapabilities(CapabilityAggregationStep.java:147)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:887)
at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
at java.base/java.lang.Thread.run(Thread.java:832)
at org.jboss.threads.JBossThread.run(JBossThread.java:501)
at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:330)
at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:252)
at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:60)
at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:93)
at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:450)
at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:67)
at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:149)
at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:105)
at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:145)
at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:63)
We don't have that yet. You can track this issue for updates on when that will change.
Actually, the error is saying that we have to choose one of the providers:
Build step io.quarkus.deployment.steps.CapabilityAggregationStep#aggregateCapabilities threw an exception: java.lang.IllegalStateException: Please make sure there is only one provider of the following capabilities:
capability io.quarkus.rest.client is provided by:
io.quarkus:quarkus-rest-client-reactive::jar:2.5.0.Final
io.quarkus:quarkus-rest-client::jar:2.5.0.Final
So, go to your pom.xml file and make sure to have one of them (quarkus-rest-client or quarkus-rest-client-reactive).
I believe you have to remove the quarkus-rest-client dependency and use only the quarkus-rest-client-reactive.

No Client ALPNProcessors JMeter HTTP2 Error even after installing jetty-alpn-java-client or jetty-alpn-conscrypt-client jar

I'm hitting the same Error mentioned in this post with Java 8, JMeter 5.1.1 Version: Java 12 JMeter 5 HTTP/2 request java.lang.NoClassDefFoundError: org/eclipse/jetty/alpn/client/ALPNClientConnectionFactory
I downloaded the jetty-alpn-java-client from here and installed in lib directory and restarted jmeter, I still see the same issue.
There is no TLS Configuration involved but still I downloaded jetty-alpn-conscrypt-client and installed from this link, Still seeing the same issue.
I don't have any code written, I Installed HTTP2 Sampler from JMeter Plugin Manager. Please let me know what am I missing.
Error Code:
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: No Client ALPNProcessors!
at org.eclipse.jetty.util.FuturePromise.get(FuturePromise.java:138)
at com.blazemeter.jmeter.http2.sampler.HTTP2Connection.connect(HTTP2Connection.java:68)
at com.blazemeter.jmeter.http2.sampler.HTTP2Request.setConnection(HTTP2Request.java:278)
at com.blazemeter.jmeter.http2.sampler.HTTP2Request.sample(HTTP2Request.java:138)
at com.blazemeter.jmeter.http2.sampler.HTTP2Request.sample(HTTP2Request.java:115)
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:622)
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:546)
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:486)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:253)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: No Client ALPNProcessors!
at org.eclipse.jetty.alpn.client.ALPNClientConnectionFactory.(ALPNClientConnectionFactory.java:57)
at org.eclipse.jetty.http2.client.HTTP2Client.lambda$doStart$1(HTTP2Client.java:155)
at org.eclipse.jetty.http2.client.HTTP2Client$ClientSelectorManager.newConnection(HTTP2Client.java:438)
at org.eclipse.jetty.io.ManagedSelector.createEndPoint(ManagedSelector.java:222)
at org.eclipse.jetty.io.ManagedSelector.access$1500(ManagedSelector.java:60)
at org.eclipse.jetty.io.ManagedSelector$CreateEndPoint.run(ManagedSelector.java:825)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:754)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:672)
... 1 more
Suppressed: java.lang.UnsupportedClassVersionError: org/eclipse/jetty/alpn/java/client/JDK9ClientALPNProcessor has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at org.eclipse.jetty.alpn.client.ALPNClientConnectionFactory.(ALPNClientConnectionFactory.java:65)
... 8 more
Suppressed: java.lang.NoClassDefFoundError: org/eclipse/jetty/alpn/ALPN
at org.eclipse.jetty.alpn.java.client.OpenJDK8ClientALPNProcessor.init(OpenJDK8ClientALPNProcessor.java:42)
at org.eclipse.jetty.alpn.client.ALPNClientConnectionFactory.(ALPNClientConnectionFactory.java:77)
... 8 more
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.alpn.ALPN
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more
You're using wrong alpn-boot library, you're trying to use 9.4.9.v20180320 which requires Java 9 and you're using Java 8.
there is a table at the bottom of this page: https://www.eclipse.org/jetty/documentation/9.4.x/alpn-chapter.html
where you can match the relevant version of the alpn-boot library with your Java 8 version.
Moreover, you should not be placing the .jar to JMeter's "lib" folder, you should set the relevant -Xbootclasspath/p:/path/to/alpn-boot.jar JVM property.
More information:
HTTP2 Plugin for JMeter
The New HTTP/2 Plugin for JMeter

Maven2 to Maven3 Migration java.lang.reflect.InvocationTargetException

I am trying to update to M3 but I have a few issues/LOTS to work out. I am stuck on one particular failure.
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
That causes a stack overflow and then
ERROR: Failed to parse POMs
hudson.util.IOException2: java.lang.reflect.InvocationTargetException
at hudson.maven.Maven3Builder.call(Maven3Builder.java:166)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:70)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:287)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.Exception: java.lang.reflect.InvocationTargetException
at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:161)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:112)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
... 11 more
Caused by: java.lang.StackOverflowError
at sun.nio.cs.SingleByteEncoder.encodeArrayLoop(SingleByteEncoder.java:91)
at sun.nio.cs.SingleByteEncoder.encodeLoop(SingleByteEncoder.java:130)
at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:544)
at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:252)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:106)
at java.io.OutputStreamWriter.write(OutputStreamWriter.java:190)
at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:111)
at java.io.PrintStream.write(PrintStream.java:476)
at java.io.PrintStream.print(PrintStream.java:619)
at org.apache.maven.cli.PrintStreamLogger.info(PrintStreamLogger.java:110)
at org.codehaus.plexus.logging.AbstractLogger.info(AbstractLogger.java:55)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:464)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:467)...about1000 more
What is going on...?
Updating maven assembly plugin to latest version fixed my problem.
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
Workaround, not a fix:
set MAVEN_OPTS=-Xss2m
What fixed it for me : Make sure you specify a version for maven assembly plugin and make sure it's updated.
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2.1</version>
The stack trace shows it is a StackOverflow error, try to increase stack size to 5MB or more by adding -Xss5M in VM arguments.
This failure log is from running in Hudson. Can you make sure it works on the command line first. That will make it much easier. Also if you want to run on Hudson I would suggest to make sure you use the new Maven 3 integration that allows you to do Maven 3 build steps in a free-style project and NOT use the Maven2/3 project type, since it has been declared legacy.

Resources