I'm working with neo4j on AWS server ec2 ubuntu instance.
Everything worked great, but when I created an image of my ec2 instance, the instance had to reboot and every since i can't get my neo4j db server to run.
the only thing I see in my console.log file is the following:
2015-05-01 16:50:18.135+0000 INFO [API] Successfully shutdown Neo4j Server.
2015-05-01 16:50:18.304+0000 INFO [API] Successfully stopped database
2015-05-01 16:50:18.304+0000 INFO [API] Successfully shutdown database.
2015-05-01 16:53:34.972+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:08:22.223+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:08:49.951+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:10:06.715+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:10:44.192+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:12:36.974+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:13:28.628+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:22:54.172+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:25:42.891+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:30:59.901+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:32:29.831+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
2015-05-01 17:43:13.906+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
the server always says it failed to start, and I don't know what to do or where the problem is at..
Any ideas?
Because the server default start in (1200000ms)2minutes,it will fail over the time limit.
1.Check the /path/data/graph.db(or your db)/message.log
2.set /path/conf/neo4j-server.properties and add the 3rd step in it
3.org.neo4j.server.startup_timeout=0
the set is to disable the startup limit behavior.
You should assign an IAM role to your instance.
When neo4j is loading it executes this script:
/etc/neo4j/pre-neo4j.sh
which is calling:
aws ec2 describe-tags ...
Related
I am trying to run Terracotta server 10.11 from https://www.terracotta.org/downloads/ to connect it with Ehcache in my Spring boot application. But the problem is that when i run Terracotta server instance using the server\bin\start-tc-server.bat it does not give any error but put the server in DIAGNOSTIC state as shown in logs below.
2022-11-24 14:22:01,660 INFO - Terracotta 5.8.5, as of 2021-12-16 at 22:21:34 UTC (Revision 3695ab2f870d94491c564e87c266555a7d1c096b from UNKNOWN)
2022-11-24 14:22:01,660 INFO - Extensions:
2022-11-24 14:22:01,679 INFO - PID is 23344
2022-11-24 14:22:01,965 INFO - Did not find configuration directory at: C:\Users\user\terracotta\config
2022-11-24 14:22:01,965 INFO - Starting node from config file: C:\Users\user\Downloads\ehcache-clustered-3.10.0-kit\ehcache-clustered-3.10.0-kit\server\conf\cluster.cfg
2022-11-24 14:22:02,266 INFO - Found only one node information in config file: C:\Users\user\Downloads\ehcache-clustered-3.10.0-kit\ehcache-clustered-3.10.0-kit\server\conf\cluster.cfg
2022-11-24 14:22:02,267 INFO - Starting unconfigured node: default-node
2022-11-24 14:22:02,271 INFO - Bootstrapped nomad system with root: C:\Users\user\terracotta\config
2022-11-24 14:22:02,277 INFO - Startup configuration of the node:
client-lease-duration=150s
client-reconnect-window=120s
cluster-name=clustered
failover-priority=availability
offheap-resources=main\:512MB
stripe.1.node.1.bind-address=0.0.0.0
stripe.1.node.1.group-bind-address=0.0.0.0
stripe.1.node.1.group-port=9430
stripe.1.node.1.hostname=localhost
stripe.1.node.1.log-dir=%H/terracotta/logs
stripe.1.node.1.name=default-node
stripe.1.node.1.port=9410
stripe.1.stripe-name=default-stripe
2022-11-24 14:22:02,280 INFO - Logging directory is not set. Logging only to the console
2022-11-24 14:22:02,293 INFO - Available Max Runtime Memory: 1820MB
2022-11-24 14:22:02,314 INFO - Creating server nodeID: NodeID[localhost:9410]
2022-11-24 14:22:02,546 INFO - Initializing LeaseServiceProvider with default lease length of 150000 ms
2022-11-24 14:22:02,548 INFO - Initializing org.terracotta.lease.service.LeaseServiceProvider#4cf92ef3
2022-11-24 14:22:02,549 INFO - Initializing org.terracotta.client.message.tracker.OOOMessageHandlerProvider#40f5b3f9
2022-11-24 14:22:02,556 INFO - Registered MBean with name: DiagnosticRequestHandler
2022-11-24 14:22:02,557 INFO - Registered Diagnostic Service: org.terracotta.nomad.server.NomadServer
2022-11-24 14:22:02,557 INFO - Registered Diagnostic Service: org.terracotta.dynamic_config.api.service.DynamicConfigService
2022-11-24 14:22:02,558 INFO - Registered Diagnostic Service: org.terracotta.dynamic_config.api.service.TopologyService
2022-11-24 14:22:02,558 INFO - Initializing org.terracotta.diagnostic.server.DiagnosticServiceProvider#1bf35727
2022-11-24 14:22:02,561 INFO - Initializing org.terracotta.diagnostic.server.extensions.DiagnosticExtensionsServiceProvider#410ee45a
2022-11-24 14:22:02,804 INFO - Initializing org.terracotta.management.service.monitoring.MonitoringServiceProvider#65c7455b
2022-11-24 14:22:02,804 INFO - Initializing org.terracotta.platform.ServerInfoProvider#240d561b
2022-11-24 14:22:02,806 INFO - Registered dynamic configuration change handler for setting client-reconnect-window: org.terracotta.dynamic_config.server.service.handler.ClientReconnectWindowConfigChangeHandler#74d20602
2022-11-24 14:22:02,809 INFO - Registered dynamic configuration change handler for setting log-dir: org.terracotta.dynamic_config.server.service.handler.NodeLogDirChangeHandler#67c6fc00
2022-11-24 14:22:02,810 INFO - Registered dynamic configuration change handler for setting failover-priority: ConfigChangeHandler#accept()
2022-11-24 14:22:02,810 INFO - Registered dynamic configuration change handler for setting public-hostname: ConfigChangeHandler#accept()
2022-11-24 14:22:02,811 INFO - Registered dynamic configuration change handler for setting public-port: ConfigChangeHandler#accept()
2022-11-24 14:22:02,811 INFO - Registered dynamic configuration change handler for setting cluster-name: ConfigChangeHandler#accept()
2022-11-24 14:22:02,812 INFO - Registered dynamic configuration change handler for setting lock-context: ConfigChangeHandler#accept()
2022-11-24 14:22:02,812 INFO - Registered dynamic configuration change handler for setting logger-overrides: org.terracotta.dynamic_config.server.service.handler.LoggerOverrideConfigChangeHandler#3ba87843
2022-11-24 14:22:02,813 INFO - Registered dynamic configuration change handler for setting tc-properties: org.terracotta.dynamic_config.server.api.SelectingConfigChangeHandler#16df9889
2022-11-24 14:22:02,815 INFO - Initializing org.terracotta.dynamic_config.server.service.DynamicConfigServiceProvider#29ca0612
2022-11-24 14:22:02,815 INFO - Registering implementation-provided service com.tc.services.PlatformServiceProvider#16b645b2
2022-11-24 14:22:02,816 INFO - Registering implementation-provided service com.tc.services.EntityMessengerProvider#3c352805
2022-11-24 14:22:02,816 INFO - Initializing com.tc.objectserver.persistence.NullPlatformStorageServiceProvider#149f57c4
2022-11-24 14:22:02,818 INFO - Registering implementation-provided service com.tc.services.LocalMonitoringProducer#5baa3715
2022-11-24 14:22:02,830 INFO - Creating 4 worker comm threads for default-node - L2_L1
2022-11-24 14:22:02,910 INFO - Registering implementation-provided service com.tc.services.CommunicatorService#7d51aa32
2022-11-24 14:22:02,920 INFO - HealthChecker Started
2022-11-24 14:22:02,952 INFO - Started the server in diagnostic mode
2022-11-24 14:22:02,967 INFO - Server started as default-node
2022-11-24 14:22:02,959 INFO - Terracotta Server instance has started diagnostic listening on all interfaces (address:/0.0.0.0 port:9410)
2022-11-24 14:22:03,177 INFO - Moved to State[ DIAGNOSTIC ]
According to the documentation it should be in ACTIVE state to be running properly. Still i tried to make connection with the server from my Spring boot application but it was also unable to reach it and gave TimeoutException.
I am using the following command to run the server instance:
./start-tc-server.bat -f C:\Users\user\Downloads\ehcache-clustered-3.10.0-kit\ehcache-clustered-3.10.0-kit\server\conf\cluster.cfg
Does anyone have any clue why its not getting to ACTIVE state ? maybe try to run it on your end and see if the server gets to ACTIVE state. Or is there anything i am missing ?
Thanks in Advance.
P.S I tried running older version of Terracotta server from the same downloads page and it easily goes to active state but i cannot use old version since it is not compaitable with Ehcache 3.x
You need to activate the server.
In the kit that you downloaded navigate to /tools/bin and you need to run config-tool.bat activate -f ../../server/conf/cluster.cfg
This will create a folder C:\Users\{user}\terracotta that will contain the configs and logs for the terracotta server, so the next time you start it will use the configs in the folder and automatically go to activate state.
If you need to change configs delete the folder, restart terracotta and activate it again using the config tool.
I am trying to start apache server on my windows machine. But while doing so i get this error kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING and the process stops. Note that i have zookeeper process started before running this and it started fine.
console log for zookeeper below:-
[2021-02-20 16:24:30,818] INFO Server environment:java.library.path=C:\Program Files (x86)\Java\jdk-15.0.2\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;"C:\Users\Ripesh\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Java\jdk-15.0.2\bin";;. (org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,826] INFO Server environment:java.io.tmpdir=C:\Users\Ripesh\AppData\Local\Temp\
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,826] INFO Server environment:java.compiler=<NA>
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,834] INFO Server environment:os.name=Windows 10
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,834] INFO Server environment:os.arch=amd64
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,842] INFO Server environment:os.version=10.0
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,850] INFO Server environment:user.name=Ripesh
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,850] INFO Server environment:user.home=C:\Users\Ripesh
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,858] INFO Server environment:user.dir=C:\kafka2.7
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,858] INFO Server environment:os.memory.free=497MB
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,858] INFO Server environment:os.memory.max=512MB
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,866] INFO Server environment:os.memory.total=512MB
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,882] INFO minSessionTimeout set to 6000
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,882] INFO maxSessionTimeout set to 60000
(org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,898] INFO Created server with tickTime 3000 minSessionTimeout 6000
maxSessionTimeout 60000 datadir C:\kafka2.7\zookeeper\version-2 snapdir C:\kafka2.7\zookeeper\version-2 (org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:24:30,938] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2021-02-20 16:24:30,946] INFO Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers. (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-02-20 16:24:30,946] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2021-02-20 16:24:30,972] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase)
[2021-02-20 16:24:30,979] INFO Snapshotting: 0x0 to C:\kafka2.7\zookeeper\version-2\snapshot.0 (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2021-02-20 16:24:30,984] INFO Snapshotting: 0x0 to C:\kafka2.7\zookeeper\version-2\snapshot.0 (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
[2021-02-20 16:24:31,008] INFO Using checkIntervalMs=60000 maxPerMinute=10000 (org.apache.zookeeper.server.ContainerManager)
[2021-02-20 16:29:39,625] INFO Creating new log file: log.1 (org.apache.zookeeper.server.persistence.FileTxnLog)
[2021-02-20 16:29:58,708] INFO Expiring session 0x100011044520000, timeout of 18000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:48:34,467] INFO Expiring session 0x100011044520001, timeout of 18000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
[2021-02-20 16:48:34,471] INFO Expiring session 0x100011044520002, timeout of 18000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
Error log in console when kafka server started:-
[2021-02-21 06:30:58,108] INFO Socket connection established, initiating session, client: /[0:0:0:0:0:0:0:1]:52365, server: localhost/[0:0:0:0:0:0:0:1]:2181 (org.apache.zookeeper.ClientCnxn)
[2021-02-21 06:31:16,104] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2021-02-21 06:31:16,108] WARN Client session timed out, have not heard from server in 18001ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2021-02-21 06:31:16,114] INFO Client session timed out, have not heard from server in 18001ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2021-02-21 06:31:16,340] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2021-02-21 06:31:16,342] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2021-02-21 06:31:16,357] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2021-02-21 06:31:16,372] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:262)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:119)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1881)
at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:441)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:466)
at kafka.server.KafkaServer.startup(KafkaServer.scala:233)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:82)
at kafka.Kafka.main(Kafka.scala)
[2021-02-21 06:31:16,379] INFO shutting down (kafka.server.KafkaServer)
[2021-02-21 06:31:16,391] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2021-02-21 06:31:16,393] INFO shut down completed (kafka.server.KafkaServer)
[2021-02-21 06:31:16,394] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2021-02-21 06:31:16,397] INFO shutting down (kafka.server.KafkaServer)
PS C:\kafka2.7>
It looks from log there is a 18000msec timeout defined.Strangely after a few mins when i started kafka server process again i was able to start it without any issues. Note i started zookeeper from zookeper-server-start.bat provided in kafka package rather than apache zookeeper package.
Thanks
I am running a SpringBoot App. I have bootstrap-test.yml (located under src/test/resources/config), which looks like:
spring.cloud.config.enabled: false
spring.cloud.service-registry.auto-registration.enabled: false
spring.cloud.zookeeper.discovery.enabled: false
spring.cloud.zookeeper.discovery.register: false
pom:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zookeeper-discovery</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
However, when I try to run an integration test:
#ActiveProfiles("test")
#DirtiesContext
#SpringBootTest
public class TestClass......
ZooKeeper still attempts to connect "live":
16:02:14.752 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=493MB
16:02:14.752 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=8116MB
16:02:14.752 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=610MB
16:02:14.778 [main] INFO o.a.c.f.imps.CuratorFrameworkImpl - Starting
16:02:14.783 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState#76d828ff
16:02:14.788 [main] INFO o.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
16:02:14.798 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
16:02:14.805 [main] INFO o.a.c.f.imps.CuratorFrameworkImpl - Default schema
16:02:16.805 [main-SendThread(localhost:2181)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
I want to disable all zookeeper bootstrap actions for integration test. What gives, I'm frustrated and have no idea what is wrong: why is Zookeeper still trying to run when I disabled it? Thanks in advance.
this should work !
spring.cloud.zookeeper.enabled=false
for yaml,
spring:
cloud:
zookeeper:
enabled: false
I have installed Neo4j 2.3.2 Community Edition on Mac OS 10.10. I can launch the application and connect to it from localhost:7474/browser/. So far, so good.
I would like to launch Neo4j 2.3.2 from a Terminal window, so that I don't have the overhead of a windowed application running at the same time. When I run the following command...
$ ~/neo4j/bin/neo4j console
... I get this output in the Terminal window:
WARNING: Max 256 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
Starting Neo4j Server console-mode...
Unable to find any JVMs matching version "1.7".
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:-OmitStackTraceInFastThrow -XX:hashCode=5 -Dneo4j.ext.udc.source=tarball
2016-02-25 14:03:18.755+0000 INFO [API] Setting startup timeout to: 120000ms based on 120000
2016-02-25 14:03:58.356+0000 INFO [API] Successfully started database
2016-02-25 14:04:04.220+0000 INFO [API] Starting HTTP on port :7474 with 2 threads available
2016-02-25 14:04:13.512+0000 INFO [API] Enabling HTTPS on port :7473
09:04:20.201 [main] INFO org.eclipse.jetty.util.log - Logging initialized #98517ms
2016-02-25 14:04:23.034+0000 INFO [API] Mounting static content at [/webadmin] from [webadmin-html]
2016-02-25 14:04:25.785+0000 INFO [API] Mounting static content at [/browser] from [browser]
09:04:25.993 [main] INFO org.eclipse.jetty.server.Server - jetty-9.2.4.v20141103
09:04:26.722 [main] INFO o.e.j.server.handler.ContextHandler - Started o.e.j.s.h.MovedContextHandler#1611ba2{/,null,AVAILABLE}
09:04:27.794 [main] INFO o.e.j.w.StandardDescriptorProcessor - NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet
09:04:27.981 [main] INFO o.e.j.server.handler.ContextHandler - Started o.e.j.w.WebAppContext#132ea25{/webadmin,jar:file:/Users/james/neo4j/system/lib/neo4j-server-2.2.5-static-web.jar!/webadmin-html,AVAILABLE}
09:04:38.841 [main] INFO o.e.j.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler#60bfaa02{/db/manage,null,AVAILABLE}
09:04:39.326 [main] INFO o.e.j.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler#28e2e149{/db/data,null,AVAILABLE}
09:04:39.353 [main] INFO o.e.j.w.StandardDescriptorProcessor - NO JSP Support for /browser, did not find org.apache.jasper.servlet.JspServlet
09:04:39.355 [main] INFO o.e.j.server.handler.ContextHandler - Started o.e.j.w.WebAppContext#78e6aa71{/browser,jar:file:/Users/james/neo4j/system/lib/neo4j-browser-2.2.5.jar!/browser,AVAILABLE}
09:04:39.536 [main] INFO o.e.j.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler#4994d9ab{/,null,AVAILABLE}
09:04:39.745 [main] INFO o.e.jetty.server.ServerConnector - Started ServerConnector#2d19cf20{HTTP/1.1}{localhost:7474}
09:04:40.576 [main] INFO o.e.jetty.server.ServerConnector - Started ServerConnector#43c742c{SSL-HTTP/1.1}{localhost:7473}
09:04:40.577 [main] INFO org.eclipse.jetty.server.Server - Started #119058ms
2016-02-25 14:04:40.577+0000 INFO [API] Server started on: http://localhost:7474/
2016-02-25 14:04:40.590+0000 INFO [API] Remote interface ready and available at [http://localhost:7474/]
I have Java version 8, update 74 installed (build 1.8.0_74-b02), so I assume that I can ignore the warning Unable to find any JVMs matching version "1.7".
However, when I visit http://localhost:7474/ in Chrome Version 45.0.2454.85 (64-bit), I see three errors in the Developer Console: two files that fail to load and a subsequent script error.
localhost/:28 GET http://localhost:7474/browser/styles/68eddd94.main.css
localhost/:466 GET http://localhost:7474/browser/scripts/ded362b3.scripts.js
Uncaught Error: [$injector:modulerr] Failed to instantiate module neo4jApp due to:
Error: [$injector:nomod] Module 'neo4jApp' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.
As a result, the Neo4j interface does not appear in the browser window.
Is it possible to run Neo4j 2.3.2 from the Terminal, and if so, what do I need to do to get http://localhost:7474/ to load correctly?
Shift-reload, or test in an incognito window.
Looks like a JS file mismatch due to aggressive browser caching.
I installed Mesos in an OpenStack environment using these instructions from Mesosphere: https://open.mesosphere.com/getting-started/datacenter/install/. I ran the verification test as described and it was successful. UI for both Mesos and Marathon are working as expected.
When I run the Spark shell from my laptop I cannot connect. The shell hangs with the output below. I don't see anything in the Mesos master or slave logs that would indicate an error, so am not sure what to investigate next.
Any help would be appreciated.
TOMWATER-M-60SN:bin tomwater$ ./spark-shell --master mesos://zk://10.93.193.78:2181,10.93.193.79:2181,10.93.193.80:2181/mesos
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/08/06 15:39:02 INFO SecurityManager: Changing view acls to: tomwater
15/08/06 15:39:02 INFO SecurityManager: Changing modify acls to: tomwater
15/08/06 15:39:02 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(tomwater); users with modify permissions: Set(tomwater)
15/08/06 15:39:02 INFO HttpServer: Starting HTTP Server
15/08/06 15:39:02 INFO Utils: Successfully started service 'HTTP class server' on port 63056.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.4.1
/_/
Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_51)
Type in expressions to have them evaluated.
Type :help for more information.
15/08/06 15:39:05 INFO SparkContext: Running Spark version 1.4.1
15/08/06 15:39:05 INFO SecurityManager: Changing view acls to: tomwater
15/08/06 15:39:05 INFO SecurityManager: Changing modify acls to: tomwater
15/08/06 15:39:05 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(tomwater); users with modify permissions: Set(tomwater)
15/08/06 15:39:05 INFO Slf4jLogger: Slf4jLogger started
15/08/06 15:39:05 INFO Remoting: Starting remoting
15/08/06 15:39:05 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver#10.93.235.120:63057]
15/08/06 15:39:05 INFO Utils: Successfully started service 'sparkDriver' on port 63057.
15/08/06 15:39:05 INFO SparkEnv: Registering MapOutputTracker
15/08/06 15:39:05 INFO SparkEnv: Registering BlockManagerMaster
15/08/06 15:39:05 INFO DiskBlockManager: Created local directory at /private/var/folders/7g/p1nw5zg94yx5cck_6c4jgwh80000gp/T/spark-74145a91-396f-4989-b2c0-5902e32e9e16/blockmgr-511d3fdf-f84a-40dc-b6e5-daace4d3f786
15/08/06 15:39:05 INFO MemoryStore: MemoryStore started with capacity 265.1 MB
15/08/06 15:39:05 INFO HttpFileServer: HTTP File server directory is /private/var/folders/7g/p1nw5zg94yx5cck_6c4jgwh80000gp/T/spark-74145a91-396f-4989-b2c0-5902e32e9e16/httpd-4ce76073-5636-4656-9fba-633fbc1c16f4
15/08/06 15:39:05 INFO HttpServer: Starting HTTP Server
15/08/06 15:39:05 INFO Utils: Successfully started service 'HTTP file server' on port 63058.
15/08/06 15:39:05 INFO SparkEnv: Registering OutputCommitCoordinator
15/08/06 15:39:05 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/08/06 15:39:05 INFO SparkUI: Started SparkUI at http://10.93.235.120:4040
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#716: Client environment:host.name=TOMWATER-M-60SN
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#723: Client environment:os.name=Darwin
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#724: Client environment:os.arch=14.4.0
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#725: Client environment:os.version=Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#733: Client environment:user.name=tomwater
I0806 15:39:06.235976 547205120 sched.cpp:157] Version: 0.23.0
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#741: Client environment:user.home=/Users/tomwater
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#log_env#753: Client environment:user.dir=/Users/tomwater/development/tools/spark-1.4.1-bin-hadoop2.6/bin
2015-08-06 15:39:06,236:30782(0x1210e7000):ZOO_INFO#zookeeper_init#786: Initiating client connection, host=10.93.193.78:2181,10.93.193.79:2181,10.93.193.80:2181 sessionTimeout=10000 watcher=0x11eca0d00 sessionId=0 sessionPasswd=<null> context=0x7f8f7cffbaf0 flags=0
2015-08-06 15:39:06,333:30782(0x12147c000):ZOO_INFO#check_events#1703: initiated connection to server [10.93.193.78:2181]
2015-08-06 15:39:06,705:30782(0x12147c000):ZOO_INFO#check_events#1750: session establishment complete on server [10.93.193.78:2181], sessionId=0x14f0502209a0006, negotiated timeout=10000
I0806 15:39:06.707475 544960512 group.cpp:313] Group process (group(1)#10.93.235.120:63059) connected to ZooKeeper
I0806 15:39:06.707785 544960512 group.cpp:787] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I0806 15:39:06.707952 544960512 group.cpp:385] Trying to create path '/mesos' in ZooKeeper
I0806 15:39:06.712241 547741696 detector.cpp:138] Detected a new leader: (id='126')
I0806 15:39:06.712530 555130880 group.cpp:656] Trying to get '/mesos/info_0000000126' in ZooKeeper
W0806 15:39:06.714071 544960512 detector.cpp:444] Leading master master#192.168.1.69:5050 is using a Protobuf binary format when registering with ZooKeeper (info): this will be deprecated as of Mesos 0.24 (see MESOS-2340)
I0806 15:39:06.714269 544960512 detector.cpp:481] A new leading master (UPID=master#192.168.1.69:5050) is detected
I0806 15:39:06.714498 544960512 sched.cpp:254] New master detected at master#192.168.1.69:5050
I0806 15:39:06.714643 544960512 sched.cpp:264] No credentials provided. Attempting to register without authentication
I've just had this - obviously check that you can talk to the master mesos node (on port 5050 normally). However you also need to allow the mesos master to talk back to your client (it's an emphemeral port annoyingly).
If you strace it you can see what's going on.
strace -e trace=network -f -s 16384 -o /tmp/strace.log pyspark
Looking at the strace.log - first we ask for a random socket and listen on it:
28462 socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 254
28462 setsockopt(254, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
28462 bind(254, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
28462 getsockname(254, {sa_family=AF_INET, sin_port=htons(46975), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
28462 listen(254, 500000) = 0
Now's the interesting part - we talk to the mesos master (10.1.201.191:5050) and we tell it our IP and that port we opened (10.1.200.212:46975)
It then talks back to us (the accept()):
28507 connect(258, {sa_family=AF_INET, sin_port=htons(5050), sin_addr=inet_addr("10.1.201.191")}, 16) = -1 EINPROGRESS (Operation now in progress)
28510 getsockopt(258, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
28510 sendto(258, "POST /master/mesos.scheduler.Call HTTP/1.1\r\nUser-Agent: libprocess/scheduler-52db362e-d5dd-4109-97d3-e28e80f2391b#10.1.200.212:46975\r\nLibproce
ss-From: scheduler-52db362e-d5dd-4109-97d3-e28e80f2391b#10.1.200.212:46975\r\nConnection: Keep-Alive\r\nHost: \r\nTransfer-Encoding: chunked\r\n\r\n54\r\n\20\1\32P\n
N\n\6ubuntu\22\fPySparkShell:\34ip-10-1-200-212.ec2.internalJ\30http://10.1.200.212:4040\r\n0\r\n\r\n", 375, MSG_NOSIGNAL, NULL, 0) = 375
28510 accept(254, {sa_family=AF_INET, sin_port=htons(33743), sin_addr=inet_addr("10.1.201.191")}, [16]) = 259