I have started a Flink Cluster via start-cluster.bat in the windows terminal.
Starting a local cluster with one JobManager process and one TaskManager process.
You can terminate the processes via CTRL-C in the spawned shell windows.
Web interface by default on http://localhost:8081/.
I was able to access the localhost URL but the Available Task Slots is given as 0. By default, it is expected to have one task (as configured in yamlfile). Anyone had a similar issue?
Have your processes started yet? It is possible that your JobManager startup failed. It is also possible that your system is running out of checks.
One possible solution:
Attach the remote JVM Debugger to TaskManager.
or: suspend=n
My Case:
FLINK_PROPERTIES=
jobmanager.rpc.address: jobmanager
taskmanager.memory.process.size: 2728m
taskmanager.memory.flink.size: 2280m
taskmanager.numberOfTaskSlots: 4
env.java.opts: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:5005"
parallelism.default: 4
Before:
The JobManager and TaskManager can be started normally with memory correctly configured. TaskManager log:
Starting taskexecutor as a console application on host bb166efde8f4.
Listening for transport dt_socket at address: 5005
After submitting the Job, it tries to create for about 3 minutes. And then fails with NoResourceAvailableException. Also, the dashboard show available task slot is 0.
Solution:
Set Remote JVM Debug in IDEA as follows:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 attach to remote jvm at localhost 5005
launch debugger.
submit the job.
Or you can set suspend=n instead of suspend=y in env var.
Now the available task slot is 4.
I have a Nagios monitoring system to monitor servers. I have a server in which WebSphere portal has been installed. I want to configure a system in which when there is high CPU usage/thread count increases, it automatically takes a thread dump. For this, I am using nagios plugin check_jvm. This plugin uses a jar called JvmInspector.jar. JvmInspector.jar should list all the jvm names which are running in the system.
Usage of JvmInspector.jar is as follows:
java -jar JvmInspector.jar all
When I am running this, it is showing an empty result although I have a running WebSphere portal server running. Inspite of this, I tried to run 'check_jvm' plugin but it shows the following result:
[root#dev03 libexec]# sudo -u root /usr/local/nagios/libexec/check_jvm -n WebSphere_Portal -p threads -w 105 -c 135
UNKNOWN Can't connect to the JVM:
Can anybody help me in this?
I have follow the instruction to install flow
https://www.youtube.com/watch?v=_HVx9Jqr34Q
it works perfectly. But if I restart my laptop, then i open "http://localhost:54321" it shows not connected.
Should I rerun the command "java -jar h2o.jar"? is that alway required if I want to open flow after computer reboot? is that an easy short cut to start flow?
Yes, you need to re-run java -jar h2o.jar after rebooting. Alternatively, you could have your OS start it, by running that command, during the boot process; the instructions for that vary by OS (and are outside the scope of StackOverflow, but are easy to google).
I have created a neo4j database on my windows machine.
I have transferred the content of the database directory to my linux machine. This is because I have the community edition which does not support the backup functions.
mtt#mttPC:/var/lib/neo4j/data/log$ sudo service neo4j-service start
WARNING: Max 1024 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
WARNING! You are using an unsupported Java runtime.
* Please use Oracle(R) Java(TM) 7 to run Neo4j Server. Download "Java Platform (JDK) 7" from:
http://www.oracle.com/technetwork/java/javase/downloads/index.html
* Please see http://docs.neo4j.org/ for Neo4j Server installation instructions.
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
Starting Neo4j Server...WARNING: not changing user
process [21498]... waiting for server to be ready..... Failed to start within 120 seconds.
Neo4j Server may have failed to start, please check the logs.
The file messages.log in the database directory says nothing.
Any idea? Are the windows and linux neo4js compatible? Thank you.
Edit
I have made a fresh install of neo4j on my ubuntu machine.
Now I finally get some logs:
2014-05-16 20:01:10.958+0000 ERROR [o.n.k.EmbeddedGraphDatabase]: Startup failed: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager#25984c63' was successfully initialized, but failed to start. Please see attached cause exception.: Component 'org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource#3d34dcb' was successfully initialized, but failed to start. Please see attached cause exception.: 'neostore' has a store version number that we cannot upgrade from. Expected 'NeoStore v0.A.0' but file is version 'NeoStore v0.A.2'.
2014-05-16 20:01:10.958+0000 INFO [o.n.k.EmbeddedGraphDatabase]: Shutdown started
I should be related to this but I am not sure how to proceed. Is the issue related to the fact that when I copied the database, I just stopped neo4j on my windows machine from the neo4j window?
There is no reason why a Neo4j database should not be transferable between operating systems. Can you please provide the output of data/log/console.log? First thought is that you may have permission issues. The files should be read/write for the user the Neo4j process will run as.
When I add the following Java options to enable debugging:
JAVA_OPTS="$JAVA_OPTS -noverify -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
I get the following error whenever I try to shutdown the tomcat:
ERROR: transport error 202: bind failed: Address already in use ["transport.c",L41]
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) ["debugInit.c",L500]
JDWP exit error JVMTI_ERROR_INTERNAL(113): No transports initializedFATAL ERROR in native method: JDWP No transports initialized, jvmtiError=JVMTI_ERROR_INTERNAL(113)
Thank you for a nice short explanation, PHeath! Following your advice, I found the best way to solve the problem is simply to use CATALINA_OPTS instead of JAVA_OPTS.
Looking into catalina.sh, one can see CATALINA_OPTS is only used by the "start" and "start-security" commands, whereas JAVA_OPTS is also used by the "stop" command (at least with Tomcat 6.0.33 on openSUSE 12.1).
At least if you have Tomcat installed on Linux using a package manager then modifying the CATALINA_OPTS variable in /etc/tomcat6/tomcat6.conf (or whatever path in your distribution) is cleaner than changing the catalina.sh script directly, for the package manager assumes that the user changes only configuration files and breaking this assumption may cause problems when upgrading the Tomcat packages (e. g. lost settings because the catalina.sh file is overwritten).
I think one should prefer CATALINA_OPTS over JAVA_OPTS not only for JDWP but for many other options as well: e. g. if one uses the heap size option -Xmx... then it would be reasonable to put it into CATALINA_OPTS, as the "stop" command does not need much heap.
You are trying to debug tomcat on startup, so it binds to port 5005 when the jvm starts.
When you run catalina.sh stop, it starts up another jvm which also tries to bind to port 5005.
You need to move the debug args to the run and start arguments (in catalina.sh) of tomcat, putting them straight into the JAVA_OPTS is the cause of the issue you're having.
The problem is your tomcat is still running on the debug port(5005) or some other service running on the same port(5005).
If tomcat still running, you can kill it
if it in linux environment ps -ef|grep java, and identify the process id of it. and kill the process using sudo kill -9 .
If it in windows environment got to task manager and kill the tomcat and java process.
Now you should be able to start the server in debug mood without any prob.
This can happen on debugging unit test through the tool(eclipse) which has been executed through the maven. To sole this you can flow the same process.
First close the Eclipse and kill the java process as well and start it again.
This is due to both applications are listening the same port number i.e 8000 while running in debug mode.
One quick solution is change the debug port to 8001 in startup.bat
SET DEBUGPORT=8001
It seems that the port 5005 is already in use. Check open ports with netstat command.
This may be because you already opened tomcat. Check your processes.
It appears you are starting Tomcat with the Debugger enabled, This causes the JVM to attach to the Process for Debugging, However in the catalina.sh there is a case statement for start, stop, restart, so on and so forth. Issuing the stop command still adds this in as it is part of your Global JAVA_OPTS and tries to start the debugger listening on the same port for the shutdown command. If you remove the address=50005 from your JAVA_OPTS or use the start jdpa commands to start the VM with the debugger this will fix your problem.
Look at the default catalina.sh in the latest Tomcat distribution if you need a clean copy. It sounds like someone has made changes inside yours that are invalid and causing JDPA to run on start, stop, any command issued.
set JPDA_ADDRESS=8001 in catalina.bat i.e debug port
and change all 3 ports in server.xml
In my case (Tomcat installed form tarball) I had those debug options unintentionally set in my env. This fixed the error:
$ unset JAVA_OPTS