For some reason I can not create a native image using the quarkus-rest-client-jackson extension in Quarkus version 2.9.2. (I haven't tested out in a different one)
If I package it in a non-native image, everything works as expected and the image is creates successfully.
Thank you in advance for any detail.
GraalVM Native Image: Generating 'was-fits-scanner-1.0.0-SNAPSHOT-runner'...
========================================================================================================================
[1/7] Initializing... (17.5s # 0.13GB)
Version info: 'GraalVM 22.0.0.2 Java 11 CE'
3 user-provided feature(s)
- io.quarkus.runner.AutoFeature
- io.quarkus.runtime.graal.DisableLoggingAutoFeature
- io.quarkus.runtime.graal.ResourcesFeature
[2/7] Performing analysis... [] (14.1s # 0.74GB)
To see how the classes got initialized, use --trace-class-initialization=org.apache.http.conn.ssl.SSLConnectionSocketFactory
3,941 (85.67%) of 4,600 classes reachable
4,948 (64.71%) of 7,646 fields reachable
16,245 (80.04%) of 20,297 methods reachable
123 classes, 10 fields, and 380 methods registered for reflection
Error: Classes that should be initialized at run time got initialized during image building:
org.apache.http.conn.ssl.SSLConnectionSocketFactory the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'SSLConnectionSocketFactory.class'). To see why org.apache.http.conn.ssl.SSLConnectionSocketFactory got initialized use --trace-class-initialization=org.apache.http.conn.ssl.SSLConnectionSocketFactory
com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
org.apache.http.conn.ssl.SSLConnectionSocketFactory the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'SSLConnectionSocketFactory.class'). To see why org.apache.http.conn.ssl.SSLConnectionSocketFactory got initialized use --trace-class-initialization=org.apache.http.conn.ssl.SSLConnectionSocketFactory
at com.oracle.svm.core.util.UserError.abort(UserError.java:73)
at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:555)
at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:167)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$10(NativeImageGenerator.java:704)
at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:74)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$11(NativeImageGenerator.java:704)
at com.oracle.graal.pointsto.PointsToAnalysis.runAnalysis(PointsToAnalysis.java:755)
at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:702)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:537)
at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:494)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:426)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:587)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:126)
at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:617)
------------------------------------------------------------------------------------------------------------------------
0.6s (1.9% of total time) in 16 GCs | Peak RSS: 1.17GB | CPU load: 2.69
------------------------------------------------------------------------------------------------------------------------
Related
If you are working on a latest MacBook pro with M1 chip. Installing Cassandra and starting it might throw an error.
Steps
1: Installed Jdk8
2: Installed Cassandra 4.0.1
How to start: /opt/homebrew/opt/cassandra/bin/cassandra -f
Output:
ERROR [main] 2021-10-08 00:03:12,568 NativeLibraryDarwin.java:64 - Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /opt/homebrew/Cellar/cassandra/4.0.1/tmp/jna9964159388012624603.tmp: dlopen(/opt/homebrew/Cellar/cassandra/4.0.1/tmp/jna9964159388012624603.tmp, 1): no suitable image found. Did find:
/opt/homebrew/Cellar/cassandra/4.0.1/tmp/jna9964159388012624603.tmp: no matching architecture in universal wrapper
/opt/homebrew/Cellar/cassandra/4.0.1/tmp/jna9964159388012624603.tmp: no matching architecture in universal wrapper
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498)
at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694)
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2627)
at java.base/java.lang.Runtime.load0(Runtime.java:768)
at java.base/java.lang.System.load(System.java:1837)
at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
at com.sun.jna.Native.<clinit>(Native.java:195)
at com.sun.jna.NativeLibrary.<clinit>(NativeLibrary.java:87)
at org.apache.cassandra.utils.NativeLibraryDarwin.<clinit>(NativeLibraryDarwin.java:55)
at org.apache.cassandra.utils.NativeLibrary.<clinit>(NativeLibrary.java:98)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:258)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:763)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:887)
INFO [main] 2021-10-08 00:03:12,596 MonotonicClock.java:199 - Scheduling approximate time conversion task with an interval of 10000 milliseconds
INFO [main] 2021-10-08 00:03:12,597 MonotonicClock.java:335 - Scheduling approximate time-check task with a precision of 2 milliseconds
WARN [main] 2021-10-08 00:03:12,597 StartupChecks.java:143 - jemalloc shared library could not be preloaded to speed up memory allocations
WARN [main] 2021-10-08 00:03:12,597 StartupChecks.java:187 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
ERROR [main] 2021-10-08 00:03:12,597 CassandraDaemon.java:909 - The native library could not be initialized properly.
Solution:
Find where is jna-..*.jar located. you can do this using
find /opt/homebrew/ -name '*.jar' | grep cassandra
Browse to https://search.maven.org/artifact/net.java.dev.jna/jna/5.8.0/jar and download the jar from top-right downloads button on the page.
Replace your jna-..*.jar file with the one you downloaded. In my case i replaced jna-5.6.0.jar. Remember to keep the file name as your previous file name. Example I moved jna-5.8.0.jar and renamed as jna-5.6.0.jar
mv /Users/mansooralikhan/Downloads/jna-5.8.0.jar /opt/homebrew/Cellar/cassandra/4.0.1/libexec/jna-5.6.0.jar
Restart cassandra
To echo Mansoor's answer, I'll add that the version of JNA shipped with Apache Cassandra 4.0 unfortunately is not compatible with the M1 architecture.
To remedy this, head to the "downloads" section of the JNA GitHub repo, where you should find the latest version of the JNA jar file.
https://github.com/java-native-access/jna#jna
Download JNA version 5.8 or higher, and replace the existing jna-5.6.0.jar file in Cassandra's lib/ directory (tarball-based installs).
Just to add to the answers here, I wanted to note for posterity that the issue with arm64 on Apple M1 devices not supported by JNA 5.6.0 has been raised with the Apache Cassandra project previously.
More recently, it has been reported in CASSANDRA-17019. I brought it up with the project devs/contributors and hopefully it gets reviewed and resolved soon. Cheers!
I'm new to Julia and, after a suggestion, I started using nightly build 1.6.0-DEV.1371 due to the high loading time of packages in Julia 1.5.2.
So I tried to change the default directory of DEPOT_PATH and copied all files from ~/.julia to /opt/julia (owned by my user). Now, if I start Julia using the default directory and run the code below, it takes about 4 seconds. However, when I start julia in the new directory (JULIA_DEPOT_PATH=/opt/julia julia), the same code takes incredible 83 seconds. The same happens with Julia 1.5.2 (17s in the default directory, 200s in the new directory).
This is the code I'm using to measure the time.
t = time(); using Plots; time() - t
Is there some explanation about this strange (and annoying) behavior?
My Platform Info:
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i3-6100U CPU # 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
And I'm using a SSD.
There is a good chance that now you are having files from your previous Julia installation with the new Julia's JULIA_DEPOT_PATH. The normal approach is to set JULIA_DEPOT_PATH to an empty folder and then install them there.
Regarding startup times there are the following things you can do:
Precompile package (this is also happening when you use a new version of the package for the first time)
using Pkg
pkg"precompile"
Once Plots is precompiled it still takes around 10s to load each time.
If the above time is unacceptable build-in your Plots into Julia system image.
using PackageCompiler
create_sysimage(:Plots, sysimage_path="sys_plots.so", precompile_execution_file="precompile_plots.jl")
For the precompile_plots.jl you could use commands that you use regularly or perhaps test sets from Plots.jl.
Once done you will be starting Julia with the following comand:
julia --sysimage sys_plots.so
Building the system image takes long, however once done, loading Plots.jl will be a matter of milliseconds.
More information can be found here:
package compiler tutorial https://julialang.github.io/PackageCompiler.jl/dev/examples/plots/
excellent tutorial video https://live.juliacon.org/talk/Z8TE39
I'm trying to convert my python script, which contains the module pdftotext, into a standalone .exe.
When I test the .exe app in my anaconda env It works correctly but when I test it on another device It gaves me this error:
File "main.py", line 3, in <module> #line 3 is import pdftotext
"ImportError: DLL load failed: The specified module could not be found"
[7300] Failed to execute script main
I'm sure the problem concern the module pdftotext because I tried with the simple script below and works correctly:
a=input("Start")
print("Hello world")
b=input("End")
The error appears if I convert this script:
import pdftotext
a=input("Inserisci")
print("Hello world")
b=input("Fine")
Sorry for my poor english, I come from Italy. I hope I'm making myself clear, thanks to everyone
EDIT 1. I figured out the problem may be related to poppler (library used by pdftotext) but at the moment I can't understand which file hooks up to import poppler
EDIT 2. After some work I found out two thing that might help to understand better my situation:
The .exe application works on my device (even outside the anaconda env where I've installed poppler and pdftotext) but It doesn't work on others devices (I've tried two different windows laptop and the error is the same); the script without 'pdftotext' work on every devices
In the dist folder (build by pyinstaller) appears a single file with
the name pdftotext: the file is 'pdftotext.cp37-win_amd64.pyd' (i'm
not sure what is it). In my anaconda env there are only two file
which contains the string 'pdftotext': the files are
'pdftotext.cp37-win_amd64.pyd' and 'pdftotext.exe'
EDIT 3
Full error when I run main.exe on different device :
Traceback (most recent call last):
File "main.py",line 1, in <module>
ImportError: DLL load failed: The specified module could not be found
[7140] Failed to execute script main
Full pyinstaller log:
(envPDF) C:\Users\miche\Desktop\project>pyinstaller --additional-hooks-dir=hooks main.py
65 INFO: PyInstaller: 3.6
65 INFO: Python: 3.7.6 (conda)
65 INFO: Platform: Windows-10-10.0.18362-SP0
65 INFO: wrote C:\Users\miche\Desktop\project\main.spec
65 INFO: UPX is not available.
81 INFO: Extending PYTHONPATH with paths
['C:\\Users\\miche\\Desktop\\project', 'C:\\Users\\miche\\Desktop\\project']
81 INFO: checking Analysis
81 INFO: Building Analysis because Analysis-00.toc is non existent
81 INFO: Initializing module dependency graph...
81 INFO: Caching module graph hooks...
81 INFO: Analyzing base_library.zip ...
3232 INFO: Caching module dependency graph...
3326 INFO: running Analysis Analysis-00.toc
3343 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
required by c:\users\miche\anaconda3\envs\envpdf\python.exe
3608 INFO: Analyzing C:\Users\miche\Desktop\project\main.py
3624 INFO: Processing module hooks...
3624 INFO: Loading module hook "hook-encodings.py"...
3718 INFO: Loading module hook "hook-pydoc.py"...
3718 INFO: Loading module hook "hook-xml.py"...
3954 INFO: Loading module hook "hook-pdftotext.py"...
6537 INFO: Determining a mapping of distributions to packages...
29442 INFO: Packages required by pdftotext:
[]
33735 INFO: Looking for ctypes DLLs
33735 INFO: Analyzing run-time hooks ...
33746 INFO: Looking for dynamic libraries
34387 INFO: Looking for eggs
34387 INFO: Using Python library c:\users\miche\anaconda3\envs\envpdf\python37.dll
34390 INFO: Found binding redirects:
[]
34395 INFO: Warnings written to C:\Users\miche\Desktop\project\build\main\warn-main.txt
34430 INFO: Graph cross-reference written to C:\Users\miche\Desktop\project\build\main\xref-main.html
35274 INFO: checking PYZ
35274 INFO: Building PYZ because PYZ-00.toc is non existent
35274 INFO: Building PYZ (ZlibArchive) C:\Users\miche\Desktop\project\build\main\PYZ-00.pyz
35794 INFO: Building PYZ (ZlibArchive) C:\Users\miche\Desktop\project\build\main\PYZ-00.pyz completed successfully.
35802 INFO: checking PKG
35802 INFO: Building PKG because PKG-00.toc is non existent
35804 INFO: Building PKG (CArchive) PKG-00.pkg
35824 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
35824 INFO: Bootloader c:\users\miche\anaconda3\envs\envpdf\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run.exe
35824 INFO: checking EXE
35824 INFO: Building EXE because EXE-00.toc is non existent
35824 INFO: Building EXE from EXE-00.toc
35824 INFO: Appending archive to EXE C:\Users\miche\Desktop\project\build\main\main.exe
35824 INFO: Building EXE from EXE-00.toc completed successfully.
35875 INFO: checking COLLECT
35875 INFO: Building COLLECT because COLLECT-00.toc is non existent
35875 INFO: Building COLLECT COLLECT-00.toc
96644 INFO: Building COLLECT COLLECT-00.toc completed successfully.
What you need is a hook file for PyInstaller. To quote the documentation:
In summary, a “hook” file extends PyInstaller to adapt it to the special needs and methods used by a Python package. ... ... They help the Analysis phase find needed files.
The official hook docs can be found at https://pyinstaller.readthedocs.io/en/stable/hooks.html.
Edit: the following should work:
Create this directory structure:
- yourcode.py
- hooks
- hook-pdftotext.py
And in the hook file put the following:
from PyInstaller.utils.hooks import collect_all
datas, binaries, hiddenimports = collect_all('pdftotext')
And then build with:
$ pyinstaller --additional-hook-dir=hooks yourcode.py
I'm trying to run those Flink Benchmarks:
https://github.com/dataArtisans/flink-benchmarks
I've generated the jar file using maven with that command:
mvn clean package -Pbuild-jar
Then I'm trying to run the benchmark on a Flink Cluster with that command:
./bin/flink run -c org.apache.flink.benchmark.WindowBenchmarks ~/flinkBenchmarks/target/flink-hackathon-benchmarks-0.1.jar
I've used the -c option to add to the classpath the Main of the benchmark (WindowBenchmarks) I want to run.
Finally, I get that error:
# JMH version: 1.19
# VM version: JDK 1.8.0_151, VM 25.151-b12
# VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java
# VM options: -Dlog.file=/home/user/flink-1.3.2/flink-dist/target/flink-1.3.2-bin/flink-1.3.2/log/flink-user-client-mypc.log -Dlog4j.configuration=file:/home/user/flink-1.3.2/flink-dist/target/flink-1.3.2-bin/flink-1.3.2/conf/log4j-cli.properties -Dlogback.configurationFile=file:/home/user/flink-1.3.2/flink-dist/target/flink-1.3.2-bin/flink-1.3.2/conf/logback.xml -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
# Warmup: 10 iterations, 1 s each
# Measurement: 10 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.flink.benchmark.WindowBenchmarks.sessionWindow
# Run progress: 0.00% complete, ETA 00:04:00
# Fork: 1 of 3
Error: Could not find or load main class org.openjdk.jmh.runner.ForkedMain
<forked VM failed with exit code 1>
<stdout last='20 lines'>
</stdout>
<stderr last='20 lines'>
Error: Could not find or load main class org.openjdk.jmh.runner.ForkedMain
</stderr>
# Run complete. Total time: 00:00:00
Benchmark Mode Cnt Score Error Units
The program didn't contain a Flink job. Perhaps you forgot to call execute() on the execution environment.
I don't have any previous experience with Flink and Maven so I find out what is missing. My first thought was that it's a missing dependencies error, but they look fine. Any suggestions?
Thank you in advance!
flink-benchmarks is a repository that contains sets of micro benchmarks designed to run on single machine, not on the cluster. The main functions defined in the various classes (test cases) are 'JMH' runners, not Flink programs. As such you can either execute whole benchmark suite (which takes ~1hour):
mvn -Dflink.version=1.5.0 clean install exec:exec
or if you want to execute just one benchmark, the best approach is to execute selected main function manually. For example from your IDE (don't forget about selecting flink.version, default value for the property is defined in pom.xml).
There is also a possibility to execute single benchmark from console, but I haven't tried it for very long time.
I installed Julia v0.6.2 on Windows 10.
Every other package was installed without a problem, but Mongo.jl give me this error while building it.
Pkg.build("Mongo")
INFO: Building LibBSON
===============================[ ERROR: LibBSON ]===============================
LoadError: None of the selected providers can install dependency libbson.
Use BinDeps.debug(package_name) to see available providers
while loading C:\Users\"user"\.julia\v0.6\LibBSON\deps\build.jl, in expression starting on line 27
================================================================================
INFO: Building Mongo
================================[ ERROR: Mongo ]================================
LoadError: None of the selected providers can install dependency libmongoc.
Use BinDeps.debug(package_name) to see available providers
while loading C:\Users\"user"\.julia\v0.6\Mongo\deps\build.jl, in expression starting on line 26
================================================================================
================================[ BUILD ERRORS ]================================
WARNING: LibBSON and Mongo had build errors.
- packages with build errors remain installed in C:\Users\ciko9\.julia\v0.6
- build the package(s) and all dependencies with Pkg.build("LibBSON", "Mongo")
- build a single package by running its deps/build.jl script
================================================================================
I already opened an issue on github, but I'd like to fix it asap. Any idea on how to solve this problem?
The solution is first to install manually the mongo c drivers.
Second, to copy the mongo-c-drivers folder to the mongo e libbson folder inside julia, then make a new file deps.jl for each package and write this:
# Macro to load a library
macro checked_lib(libname, path)
((VERSION >= v"0.4.0-dev+3844" ? Base.Libdl.dlopen_e : Base.dlopen_e)(path) == C_NULL) && error("Unable to load \n\n$libname ($path)\n\nPlease re-run Pkg.build(package), and restart Julia.")
quote const $(esc(libname)) = $path end
end
# Load dependencies
#checked_lib libbson "C:\\Users\\"userName"\\.julia\\v0.6\\LibBSON\\mongo-c-driver\\bin\\libbson-1.0.dll"
# Load-hooks