AWS Neptune Performance - performance

Question

I'm working on transferring data from our database which is a rdf store DB to AWS Neptune, and I'm facing some performance issues.
I have a db.r4.large Neptune instance & ec2 instance on the same vpc as Neptune.
Basically, I'm trying to ingest data to Neptune using the following http request: <myinstance>:8182/sparql.
Actually, I send the http request from my ec2 instance, and it seems that Neptune processing time is slow. In addition, it seems that Neptune's processing is not parallel.
Below are my tests & results:
I sent the following request to Neptune:
time curl -X POST -d #/tmp/my_file_32m.txt http://myneptune-poc.c0zm6uyrnnwp.us-east-1.neptune.amazonaws.com:8182/sparql
/tmp/my_file_32m.txt contains sparql insert commands and the time for this request is 34.037s while Neptune claims that it took 21.846 s:
{
"type" : "Commit",
"totalElapsedMillis" : 21846
}
real 0m34.037s
user 0m0.044s
sys 0m0.062s
A tcpdump can clearly proves that the response from Neptune was received in a delay of 34 seconds.
When I sent a data of 100m it took more than 1 min.
When I sent the same file of 32m in parallel, time was multiple in 2 :
time xargs -I % -P 8 curl -vX POST -d #/tmp/my_file_32m.txt "http://myneptune-poc.c0zm6uyrnnwp.us-east-1.neptune.amazonaws.com:8182/sparql" < <(printf '%s\n' {1..2})<
{
"type" : "Commit",
"totalElapsedMillis" : 29797
}
{
"type" : "Commit",
"totalElapsedMillis" : 30362
}
real 0m57.752s
user 0m0.137s
sys 0m0.101s
I took a tcpdump and clearly see from the wireshark that the request was sent in parallel, but there is a delay of ~1 min till Neptune returned 200 OK for both requests.
Actually, it seems that Neptune's processing is not concurrent.
request was sent in time 12 and 200 ok for both requests was sent in time 69 which is exactly 57 seconds of delay.
I tried to increase my Neptune instance size to db.r4.xlarge and also to db.r4.2xlarge, db, but I got the same performance.
I tried to send a compressed data in a gzip format in order to improve times, but it seems that Neptune doesn't support it (checking in wireshark the request was sent correctly).
I would like to hear your opinion about my tests and the results:
why performance is slow for a single http request?
why Neptune's processing is not parallel?

Answer 1

You are comparing the output of time (client side round trip time) with server reported totalEllapsedMillis. The former includes your network transmission time where as the latter is just the time that the db took to compute the query from the time it accepted the request. Do you have any metrics on the time it took to transmit your 100MB file?
Neptune does process queries in parallel (in fact the amount of parallelism scales with your instance type). If your queries are really small compared to the time it spends on the wire, then it may appear like the results completed one after the other. I would like to see more granular details of your experiments to see if there is an issue with your setup.
For starters, what is the network lag between your client and the DB endpoint? (ie how long does it take for you to make a request to the /status API for example)