Neo4j: OutOfMemoryError when running cypher query from neo4j-shell - shell

Question

I have a Neo4j database with about 130K nodes and probably somewhere between 17M relationships. My computer running Windows 10 has 16GB of RAM, 10GB (maximum) of which are allocated to the Neo4j-shell heap.
I want to run a query using the neo4j-shell from the command prompt and redirect the results to a csv file. The command I’m using for that is:
Neo4jShell -v -file query.cql > results.csv
Where the query is of the form:
MATCH (subject)-[:type1]->(intNode1:label)<-[:type2]-(intNode2:label)<-[:type3]-(object) RETURN subject.property1, object.property1;
The problem is that whenever I run this query, I get an OutOfMemory error (see error message at the bottom).
Does anyone have advice for how to run a query like this successfully? I feel like 10GB of RAM should be enough for a query like this given the size of the graph DB.
The error message I get is:
ERROR (-v for expanded information):
Error occurred in server thread; nested exception is:
java.lang.OutOfMemoryError: GC overhead limit exceeded
java.rmi.ServerError: Error occurred in server thread; nested exception is:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$2.run(Unknown Source)
at sun.rmi.transport.Transport$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
at sun.rmi.server.UnicastRef.invoke(Unknown Source)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown Source)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source)
at com.sun.proxy.$Proxy1.interpretLine(Unknown Source)
at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:149)
at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:133)
at org.neo4j.shell.StartClient.executeCommandStream(StartClient.java:393)
at org.neo4j.shell.StartClient.grabPromptOrJustExecuteCommand(StartClient.java:372)
at org.neo4j.shell.StartClient.startRemote(StartClient.java:330)
at org.neo4j.shell.StartClient.start(StartClient.java:196)
at org.neo4j.shell.StartClient.main(StartClient.java:135)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

Answer 1

Solution when I hade this problem was to add this line on the end of neo4j.properties
# Default values for the low-level graph engine
cache_type=none
neostore.nodestore.db.mapped_memory=50M
neostore.relationshipstore.db.mapped_memory=500M
neostore.propertystore.db.mapped_memory=100M
neostore.propertystore.db.strings.mapped_memory=100M
neostore.propertystore.db.arrays.mapped_memory=0M
also try to increase java in neo4j-wrapper.conf this line
#wrapper.java.initmemory=2048
#wrapper.java.maxmemory=2048
One more thing do you have index on node that you query?

Answer 2

You can provide more heap for Neo4jShell (use JAVA_OPTS=-Xmx4G -Xms4G -Xmn1G environment variable).
Did you try to run your query with profile? I presume you span up billions of paths as you don't have any restrictions.
You missed a label on subject and object, which cause the query planner to run a full-graph-scan.
MATCH (subject:label)-[:type1]->(intNode1:label)
<-[:type2]-(intNode2:label)
<-[:type3]-(object:label)
WITH distinct subject, object
RETURN subject.property1, object.property1;
You should reduce the in-between cardinality and also the output cardinality.
MATCH (subject:label)-[:type1]->(intNode1:label)
<-[:type2]-(intNode2:label)
WITH distinct subject, intNode2
MATCH (intNode2)<-[:type3]-(object:label)
WITH distinct subject, object
RETURN subject.property1, object.property1;
even better would be:
MATCH (subject:label)-[:type1]->(intNode1:label)
<-[:type2]-(intNode2:label)
WITH intNode2, collect(distinct subject) as subjects
MATCH (intNode2)<-[:type3]-(object:label)
WITH distinct object, subjects
UNWIND subjects as subject
RETURN subject.property1, object.property1;