I have Windows 10 and I followed this guide to install Spark and make it work on my OS, as long as using Jupyter Notebook tool. I used this command to instantiate the master and import the packages I needed for my job:
pyspark --packages graphframes:graphframes:0.8.1-spark3.0-s_2.12 --master local[2]
However, later, I figured out that any worker wasn't instantiated according to the aforementioned guide and my tasks were really slow. Therefore, taking inspiration from this, since I could not find any other way to connect workers to the Cluster manager due to the fact it was run by Docker, I tried to set up everything manually with the following commands:
bin\spark-class org.apache.spark.deploy.master.Master
The master was correctly instantiated, so I continued by the next command:
bin\spark-class org.apache.spark.deploy.worker.Worker spark://<master_ip>:<port> --host <IP_ADDR>
Which returned me the following error:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
21/04/01 14:14:21 INFO Master: Started daemon with process name: 8168#DESKTOP-A7EPMQG
21/04/01 14:14:21 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[main,5,main]
java.lang.ExceptionInInitializerError
at org.apache.spark.unsafe.array.ByteArrayMethods.<clinit>(ByteArrayMethods.java:54)
at org.apache.spark.internal.config.package$.<init>(package.scala:1006)
at org.apache.spark.internal.config.package$.<clinit>(package.scala)
at org.apache.spark.deploy.master.MasterArguments.<init>(MasterArguments.scala:57)
at org.apache.spark.deploy.master.Master$.main(Master.scala:1123)
at org.apache.spark.deploy.master.Master.main(Master.scala)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make private java.nio.DirectByteBuffer(long,int) accessible: module java.base does not "opens java.nio" to unnamed module #60015ef5
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:357)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:188)
at java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:181)
at org.apache.spark.unsafe.Platform.<clinit>(Platform.java:56)
... 6 more
From that moment on, none of the commands I used to run before were working anymore, and they returned the message you can see. I guess I messed up some Java stuff, but I do not understand what and where, honestly.
My java version is:
java version "16" 2021-03-16
Java(TM) SE Runtime Environment (build 16+36-2231)
Java HotSpot(TM) 64-Bit Server VM (build 16+36-2231, mixed mode, sharing)
I got the same error just now, the issue seems with Java version.
I installed java, python, spark etc. All latest versions... !
Followed steps mentioned in the below link..
https://phoenixnap.com/kb/install-spark-on-windows-10
Got the same error as you.. !
Downloaded Java SE 8 version from Oracle site..
https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html
Downloaded jdk-8u281-windows-x64.exe
Reset the JAVA_HOME.
Started spark-shell - it opened perfectly without any issues.
FYI: I don't have neither java or spark experience, if anyone feels something is wrong please correct me. Just that it worked for me, so providing the same solution here.. :)
Thanks,
Karun
I got a similar error on MacOS. The problem was with Java (I was using JDK 17), had to downgrade or use a different version.
Ended up using this:
https://adoptium.net/releases.html?variant=openjdk11
Download and install. Might have to remove your JDK17 version.
Easiest solution :
Latest version of Java (JDK) is not supported by Spark.
Please try installing JDK version 8. This will solve the error.
Related
I hava download JDK&deploy the Java_HOME and so on,i can use "javac" in Command,but when i use it like
nlp=StanfordCoreNLP(r'stanfordnlp',lang='zh')
there has a problem:
builtins.RuntimeError: Java not found.
Maybe you run out of memory. According to my experience, shutting down Java and restarting it can solve this issue.
I need help resolving this error.
java.lang.UnsatisfiedLinkError: no jp_co_epson_upos_firm_FirmNativeAccess_1_14_0001 in java.library.path
I want to use EpsonJavaPOS for printing.
Following are the steps I've succeeded so far.
Tried to install JavaPOS in windows 10(64 bit), But was partially successful
and hence copied the installation folder in windows 7 to windows 10.
I've configured the printer in setupPOS.exe application.
And as a next step I'm trying to run checkup health from checkuphealth.bat file
But receiving the above mentioned exception.
If I try to use the System.setProperty(JposPropertiesConst.JPOS_POPULATOR_FILE_PROP_NAME, "/path of jpos.xml");
as solution to the above mentioned exception, It's throwing null pointer exception.
Can anyone help on these issues?
Your process can't find the file "jp_co_epson_upos_firm_firmnativeaccess.dll"
One solution is to add the folder containing the file to the Windows PATH environment variable.
I found this file in my prior version of Java:
C:\Program Files\Java\jdk1.7.0_80\jre\bin
It was not installed in my Java 8 jre when I reinstalled the ADK driver after adding Java 8 to my system.
I have added it as a library in my application and it resolved this issue.
I am facing strange issue with sonarqube 5.0.1 , one one of the machine it is not starting. Here is the error log - sonar.log -
--> Wrapper Started as Daemon
Launching a JVM...
Unable to start JVM: No such file or directory (2)
JVM exited while loading the application.
JVM Restarts disabled. Shutting down.
<-- Wrapper Stopped
Machine is x86_64 GNU/Linux - Centos 5.1.
this box has java installed -
$java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
The same sonarqube package works on another machine.
Any idea what could be the issue here?
Thanks.
Issue was in the wrapper.conf where the java wrapper command was not getting resolved. It worked if I give the absolute path - ‘wrapper.java.command=/path/to/my/jdk/bin/java’
This could be an issue with an environment on a host.. not sure.
Few things that helped me in troubleshooting this -
log level changed to DEBUG in wrapper.conf
comments given in the wrapper.conf!
Thanks all for chiming in! Appreciate your inputs.
1.Just close all running jvm from task manager
2. Change the port of the sonar runner from the properties
I had the same symptoms (wrapper starts then immediately stops).
I tried these steps and finally succeeded (on a windows 10 pc):
1) in wrapper.conf, specified the java command:
wrapper.java.command=C:\Program Files\Java\jdk1.7...\bin\java.exe
That did not help.
2) Finally this fixed the problem. In the windows Services, open the Sonar service and then open Log On tab.
Changed the Log On to myself as follows:
I was facing the same issue on sonar startup. After reading this post , i modified the JDK path in below file and it works.
Modify the JDK path in wrapper.conf
wrapper.java.command=%JAVA_HOME%/bin/java
Install jdk 11
sudo yum install java-11-openjdk -y
sudo alternatives --config java
Select the JDK 11 version
Set the JDK 11 version in wrapper.conf
vi /opt/sonar/conf/wrapper.conf
wrapper.java.command=/usr/lib/jvm/java-11-openjdk-11.0.13.0.8-3.el8_5.x86_64/bin/java
Could you verify the Java version on the machine starting?
Java 6 is no more supported http://docs.sonarqube.org/display/SONAR/Requirements#Requirements-Prerequisite but from your error message, I don't know if this is the problem you meet.
Solution 1
Set java path globally
Solution 2
Go to sonarqube-{version}/conf directory
Edit wrapper.conf file
Replace wrapper.java.command=java with wrapper.java.command= {path-to-your-java-bin-directory}/java
eg: wrapper.java.command=/usr/java/bin/java
Try using a relative path, if your Sonar Folder is located in the same root folder as your jdk. For me my sonar and jdk are both under "Program Files", which has restrictive permission, hence the error.
E.g:
wrapper.java.command=../../../Java/jdk-11.0.4/bin/java
I am creating a Basic Weblogic server domain(11g, version 10.3.5.0 ). And in the JDK selection Window, in Available JDKs , ' JRockit SDK 1.7.0_55 # C:\Program Files\Java\jdk1.7.0_55' is selected..
The domain gets created, now i check the 'Start Admin Server' checkbox and click 'Done' . The process of starting server runs in the cmd, but it halts with the following error::
starting weblogic with Java version:
Unrecognized option: -jrockit
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
My Jdk(64-bit) is installed correctly according to me, JAVA_HOME and those things are correct. jrockit comes bundled with JDK , so why this error, any leads will be very helpful..
Thanks :)
I believe that WebLogic 10.3.5 only supports Suns Java JDK 1.6.0_24+ OR JRockit R28.1.3-1.6.0_24+.
It appears as though Weblogic 10.3.5 is not compatible with Java 7, try re-installing/configurring with a Java 6 version installed.
Source: Oracle Installation Docs
I have created a neo4j database on my windows machine.
I have transferred the content of the database directory to my linux machine. This is because I have the community edition which does not support the backup functions.
mtt#mttPC:/var/lib/neo4j/data/log$ sudo service neo4j-service start
WARNING: Max 1024 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
WARNING! You are using an unsupported Java runtime.
* Please use Oracle(R) Java(TM) 7 to run Neo4j Server. Download "Java Platform (JDK) 7" from:
http://www.oracle.com/technetwork/java/javase/downloads/index.html
* Please see http://docs.neo4j.org/ for Neo4j Server installation instructions.
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
Starting Neo4j Server...WARNING: not changing user
process [21498]... waiting for server to be ready..... Failed to start within 120 seconds.
Neo4j Server may have failed to start, please check the logs.
The file messages.log in the database directory says nothing.
Any idea? Are the windows and linux neo4js compatible? Thank you.
Edit
I have made a fresh install of neo4j on my ubuntu machine.
Now I finally get some logs:
2014-05-16 20:01:10.958+0000 ERROR [o.n.k.EmbeddedGraphDatabase]: Startup failed: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager#25984c63' was successfully initialized, but failed to start. Please see attached cause exception.: Component 'org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource#3d34dcb' was successfully initialized, but failed to start. Please see attached cause exception.: 'neostore' has a store version number that we cannot upgrade from. Expected 'NeoStore v0.A.0' but file is version 'NeoStore v0.A.2'.
2014-05-16 20:01:10.958+0000 INFO [o.n.k.EmbeddedGraphDatabase]: Shutdown started
I should be related to this but I am not sure how to proceed. Is the issue related to the fact that when I copied the database, I just stopped neo4j on my windows machine from the neo4j window?
There is no reason why a Neo4j database should not be transferable between operating systems. Can you please provide the output of data/log/console.log? First thought is that you may have permission issues. The files should be read/write for the user the Neo4j process will run as.