I'm uploading 2 binaries through a mutation:
The payload is something like this:
{"operationName":null,"variables":{"news":{"config":{"relatedID":null,"relatedTo":"Download","relatedLink":{"uid":"rc-upload-1666363697161-3","lastModified":1666363693912,"lastModifiedDate":"2022-10-21T14:48:13.912Z","name":"Metflife nuevo seguro complementario de salud.pdf","size":1413049,"type":"application/pdf","percent":0,"originFileObj":null,"status":"done"},"relatedLvl":""},"order":200,"title":"Prueba notifica PDF 01 - Paulo","description":"Prueba notifica PDF 01 - Paulo","mobileSetup":{"image":{"uid":"rc-upload-1666363697161-5","lastModified":1666363688030,"lastModifiedDate":"2022-10-21T14:48:08.030Z","name":"EXAMPLE.png","size":350501,"type":"image/png","percent":0,"originFileObj":null,"status":"done"}},"filters":{},"active":false},"_id":null},"query":"mutation ($news: NewsInput!) {\n createNewsItem(news: $news) {\n _id\n title\n }\n}\n"}
map:
{"1":["variables.news.config.relatedLink.originFileObj"],"2":["variables.news.mobileSetup.image.originFileObj"]}
1: (binary)
2: (binary)
My question is:
Through Apollo Server, is the any way to change the speed connection? I tested the same upload, same 2 files, in a simple express.js endpoint, in the same server, samne IP, same od, it takes like 3 or 4 seconds, but with this graphql resolver takes like 70 seconds (I know because the file is actually uploaded, but takes that time), and my server fails at 60seconds due timeout
Related
I'm working on transferring data from our database which is a rdf store DB to AWS Neptune, and I'm facing some performance issues.
I have a db.r4.large Neptune instance & ec2 instance on the same vpc as Neptune.
Basically, I'm trying to ingest data to Neptune using the following http request: <myinstance>:8182/sparql.
Actually, I send the http request from my ec2 instance, and it seems that Neptune processing time is slow. In addition, it seems that Neptune's processing is not parallel.
Below are my tests & results:
I sent the following request to Neptune:
time curl -X POST -d #/tmp/my_file_32m.txt http://myneptune-poc.c0zm6uyrnnwp.us-east-1.neptune.amazonaws.com:8182/sparql
/tmp/my_file_32m.txt contains sparql insert commands and the time for this request is 34.037s while Neptune claims that it took 21.846 s:
{
"type" : "Commit",
"totalElapsedMillis" : 21846
}
real 0m34.037s
user 0m0.044s
sys 0m0.062s
A tcpdump can clearly proves that the response from Neptune was received in a delay of 34 seconds.
When I sent a data of 100m it took more than 1 min.
When I sent the same file of 32m in parallel, time was multiple in 2 :
time xargs -I % -P 8 curl -vX POST -d #/tmp/my_file_32m.txt "http://myneptune-poc.c0zm6uyrnnwp.us-east-1.neptune.amazonaws.com:8182/sparql" < <(printf '%s\n' {1..2})<
{
"type" : "Commit",
"totalElapsedMillis" : 29797
}
{
"type" : "Commit",
"totalElapsedMillis" : 30362
}
real 0m57.752s
user 0m0.137s
sys 0m0.101s
I took a tcpdump and clearly see from the wireshark that the request was sent in parallel, but there is a delay of ~1 min till Neptune returned 200 OK for both requests.
Actually, it seems that Neptune's processing is not concurrent.
request was sent in time 12 and 200 ok for both requests was sent in time 69 which is exactly 57 seconds of delay.
I tried to increase my Neptune instance size to db.r4.xlarge and also to db.r4.2xlarge, db, but I got the same performance.
I tried to send a compressed data in a gzip format in order to improve times, but it seems that Neptune doesn't support it (checking in wireshark the request was sent correctly).
I would like to hear your opinion about my tests and the results:
why performance is slow for a single http request?
why Neptune's processing is not parallel?
You are comparing the output of time (client side round trip time) with server reported totalEllapsedMillis. The former includes your network transmission time where as the latter is just the time that the db took to compute the query from the time it accepted the request. Do you have any metrics on the time it took to transmit your 100MB file?
Neptune does process queries in parallel (in fact the amount of parallelism scales with your instance type). If your queries are really small compared to the time it spends on the wire, then it may appear like the results completed one after the other. I would like to see more granular details of your experiments to see if there is an issue with your setup.
For starters, what is the network lag between your client and the DB endpoint? (ie how long does it take for you to make a request to the /status API for example)
I have a working FB Bot built with Ruby which allows players to play a scavenger hunt.
Sometimes though, when I have multiple players in a team, FB is sending me a players 'Answer' webhook twice. I have looked into it and at first thought it was to do with the 20 second timeout if FB gets no 200 OK response (Docs here). After checking the logs though, I am receiving the second webhook from FB only 14 seconds later. See below:
# Webhook #1
{"object"=>"page", "entry"=>[{"id"=>"252445748474312", "time"=>1532153642358, "messaging"=>[{"sender"=>{"id"=>"1709242109154907"}, "recipient"=>{"id"=>"252445748474312"}, "timestamp"=>1532153641935, "message"=>{"mid"=>"0FeOChulGjuPgg3YJqEgajNsY8kMfNRt_bpIdeegEeE54h-KB8szcd-EQ-UHUT3850RwHgH4TxVYFkoFwxqhtg", "seq"=>402953, "text"=>"Larrikins"}}]}]}
# Webhook #2 (14 seconds later)
{"object"=>"page", "entry"=>[{"id"=>"252445748474312", "time"=>1532153656901, "messaging"=>[{"sender"=>{"id"=>"1709242109154907"}, "recipient"=>{"id"=>"252445748474312"}, "timestamp"=>1532153641935, "message"=>{"mid"=>"0FeOChulGjuPgg3YJqEgajNsY8kMfNRt_bpIdeegEeE54h-KB8szcd-EQ-UHUT3850RwHgH4TxVYFkoFwxqhtg", "seq"=>402953, "text"=>"Larrikins"}}]}]}
Notice both are exactly the same apart from the first "time" attribute (14 secs later).
Due to a number of methods and calls that I process after receiving the first webhook, the 200 OK response is only being sent back to FB once I have finished sending my messages in response (hence the 14 second delay).
So I have two questions:
Is the 14 second delay too long and that is why FB is resending? If so, how can I send a 200OK response straight away (head :ok)?
Is it another issue entirely?
You also ensure that "Echo" is disabled.
Go to Settings>Webhooks, edit events.
Asyncronous language like NodeJS is recomended, in my case y work with AWS SQS, I have workers that process the requests witout blocking (dont wait), I return 200,"ok" to FB to avoid that FB send again the message to my webhook.
Anothe apporach maybe store the mid in database, and check in each request if the mid exists, if exists the dont process the message. I was use Dynamo DB (AWS) with TTL enabled, thus with TTL my database autoclean every hour erasing old request.
I think it is the 15 second wait before replying, was also happening to me as Facebook auto retries when you don't reply fast enough. Te EEe Te's idea is solid, write some mechanism to cache mids and check if it is a duplicate before processing
I have an AJAX query on my client that passes two parameters to a server:
var url = window.location.origin + "/instanceStats"
$.getJSON(url, { 'unit' : unit, "stat" : stat }, function(data) {
instanceData[key] = data;
var count = showInstanceStats(targetElement, unit, stat, limiter);
});
The server itself is a very simple Python Flask application. On that particular URL, it grabs the "unit" and "stat" parameters from the query to determine the name of a CSV file and line within that file, grabs the line, and sends the data back to the client formatted as JSON (roughly 1KB).
Here is the funny thing: When I measure the time it takes for the data to come back, I observe that some queries are fast (between 20 and 40 ms), and some queries are slow (between 320 and 350 ms). Varying the "stat" parameter (i.e. selecting a different line in the CSV) doesn't seem to have any impact. The fast and slow queries usually switch back and forth (i.e. all even queries are fast, all odd ones are slow). The Python server itself reports roughly the same time for each query.
AJAX itself doesn't seem to have any impact either, as I can take the url that is constructed in the JS and paste it into the browser myself and get the same behavior. Here are some measurements from two subsequent queries:
Fast: http://i.imgur.com/VQ7qopd.png
Slow: http://i.imgur.com/YuG0ROM.png
This seems to be Chrome-specific, as I've tried it on Firefox and the same experiment yields roughly the same query time everytime (between 30 and 50 ms). This is unfortunate, as I want to deploy on both Chrome and Firefox.
What's causing this behavior, and how can I fix it?
I've run into this also. It only seems to happen when using localhost. If you use 127.0.0.1 (or even the computer name), it will not have the extra delay.
I'm having it too, and it's exactly the same: my Node.js application serves Ajax requests and no matter which /url I request it's either 30ms or 300ms and it switches back and forth: odd requests are long, even requests are short.
The thing I see in Chrome Web Inspector (aka Chrome DevTools) is that there is a long gap between "DNS lookup" and "Initial Connection".
They say it's OCSP related here:
http://www.webpagetest.org/forums/showthread.php?tid=12357
OCSP is some kind of certificate validation protocol:
https://en.wikipedia.org/wiki/Online_Certificate_Status_Protocol
Moving from localhost to 127.0.0.1 seems to fix it: response times are 30ms now.
I have Neo4j server running inside a virtual machine using Ubuntu 13.10 and I am accessing via REST using Cypher queries. The virtual machine has 4 GB of memory allocated to it.
I've changed the open file count to 40000, set the initial JVM heap to 1G and my neo4j.properties file is as follows:
neostore.nodestore.db.mapped_memory=250M
neostore.relationshipstore.db.mapped_memory=100M
neostore.propertystore.db.mapped_memory=100M
neostore.propertystore.db.strings.mapped_memory=100M
neostore.propertystore.db.arrays.mapped_memory=100M
keep_logical_logs=3 days
node_auto_indexing=true
node_keys_indexable=id
I've also updated sysctl based on the Neo4j Linux tuning guide:
vm.dirty_background_ratio = 50
vm.dirty_ratio = 80
Since I am testing queries, the basic routine is to run my suite of tests and then delete all of the nodes and run them all again. At the start of each test run, the database has 0 nodes in it. My suite of tests of about 100 queries is taking 22 seconds to run. Basic parameterized creates such as:
CREATE (x:user { email: {param0},
name: {param1},
displayname: {param2},
id: {param3},
href: {param4},
object: {param5} })
CREATE x-[:LOGIN]->(:login { password: {param6},
salt: {param7} } )
are currently taking over 170ms to execute (and that's the average, first query time is 700ms). During a test run, the CPU in the VM never exceeds 50% and memory usage is at a steady 1.4Gb.
Why would creating a single node in an empty database take 170ms? At this point unit testing is becoming almost impossible since it is so slow. This is my first time trying to tune Neo4j so I'm not really sure how to figure out where the problem is or what changes should be made.
Additional Details
I'm using Go 1.2 to make REST calls to the cypher endpoint (http://localhost:7474/db/data/cypher) of a locally installed Neo4j instance. I'm setting the request headers for content-type to "application/json", accept to "application/json" and "X-Stream" to true. I always return either an array of maps or nothing depending on the query.
It seems like the creates are the problem and are taking forever. For example:
2014/01/15 11:35:51 NewUser took 123.314938ms
2014/01/15 11:35:51 NewUser took 156.101784ms
2014/01/15 11:35:52 NewUser took 167.439442ms
2014/01/15 11:35:52 ValidatePassword took 4.287416ms
NewUser creates two new nodes and one relationship and is taking 167ms, while ValidatePassword is a read-only operation and it completes in 4ms. Also note that the three calls to NewUser are identical parameterized queries. While the creates are the big problem, I'm also a little concerned that Neo4j is taking 4ms to just find a labeled node when there are only 100 nodes in the database.
I do not restart the server in between test runs or delete the database. I issue a single delete all nodes query MATCH (n) OPTIONAL MATCH (n)-[r]-() DELETE n,r at the end of the test run. Running the same test suite multiple times back to back does not improve the query times.
Are your 100 queries all the same only with different parameters, or actually 100 different queries?
What you see is actually setup work. The parser has to load the parsing rules initially that takes a few ms. Also new queries that have not been seen are compiled, planned and put in the query cache.
So the first query always takes a bit longer. But as you parametrize all subsequent ones should be fast.
Can you confirm that?
I think you see the transactional overhead of flushing the transaction to disk.
Did you try to batch more requests into one? I.e. with the transactional endpoint? Or the /db/data/batch (but I'd rather use the new tx-endpoint /db/data/transaction).
Did you create an index for your lookup property for your validate query?
Can you do me a favor and test your create query without a label? I found some perf issues when testing that myself earlier this week.
Just ran a test with curl
for i in `seq 1 10`; do time curl -i -H content-type:application/json -H accept:application/json -H X-Stream:true -d #perf_test.json http://localhost:7474/db/data/cypher; done
I'm getting between 16 and 30ms per request externally including starting curl
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8; stream=true
Access-Control-Allow-Origin: *
Transfer-Encoding: chunked
Server: Jetty(9.0.5.v20130815)
{"columns":[],"data":[]}
real 0m0.016s
user 0m0.005s
sys 0m0.005s
Perhaps it is rather the VM (disk or network) or the cross-vm communication?
Did another test with ab and 1000 requests for both endpoints, got a mean of about 5 ms both times.
https://gist.github.com/jexp/8452037
Threw Node.JS on an AWS instance and was testing the request times, got some interesting results.
I used the following for the server:
var http = require('http');
http.createServer(function(req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.write('Hello World');
res.end();
}).listen(8080);
I have an average 90ms delay to this server, but the total request takes ~350+ms. Obviously a lot of time is wasted on the box. I made sure the DNS was cached prior to the test.
I did an Apache bench on the server with a cocurrency of 1000 - it finished 10,000 requests in 4.3 seconds... which means an average of 4.3 milliseconds.
UPDATE: Just for grins, I installed Apache + PHP on the same machine and did a simple "Hello World" echo and got a 92ms response time on average (two over ping).
Is there a setting somewhere that I am missing?
While Chrome Developer Tools is a good way to investigate front end performance, it gives you very rough estimate of actual server timings / cpu load. If you have ~350 ms total request time in dev tools, subtract from this number DNS lookup + Connecting + Sending + Receiving, then subtract roundtrip time (90 ms?) and after that you have first estimate. In your case I expect actual request time to be sub-millisecond. Try to run this code on server:
var http = require('http');
function hrdiff(t1, t2) {
var s = t2[0] - t1[0];
var mms = t2[1] - t1[1];
return s*1e9 + mms;
}
http.createServer(function(req, res) {
var t1 = process.hrtime();
res.writeHead(200, {'Content-Type': 'text/html'});
res.write('Hello World');
res.end();
var t2 = process.hrtime();
console.log(hrdiff(t1, t2));
}).listen(8080);
Based on ab result you should estimate average send+request+receive time to be at most 4.2 ms ( 4200 ms / 10000 req) (did you run it on server? what concurrency?)
I absolutely hate answering my own questions, but I want to pass along what I have discovered with future readers.
tl;dr: There is something wrong with res.write(). Use express.js or res.end()
I just got through conducting a bunch of tests. I setup multiple types of Node server and mixed in things like PHP and Nginx. Here are my findings.
As stated previously, with the snippet I included above, I was loosing around 250ms/request, but the Apache benchmarks did not replicate that issues. I then proceeded to do a PHP test and got results ranging from 2ms - 20ms over ping... a big difference.
This prompted some more research, I started a Nginx server and proxied the node through it, and somehow, that magically changed the response from 250ms to 15ms over ping. I was on par with that PHP script, but that is a really confusing result. Usually additional hops would slow things down.
Intrigued, I made an express.js server as well - and something even more interesting happened, the ping was 2ms over on its own. I dug around in the source for quite a while and noticed that it lacked a res.write() command, rather, it went straight to the res.end(). I started another server removing the "Hello World" from the res.write and added it to the res.end and amazingly, the ping was 0ms over ping.
I did some searching on this, wanted to see if it was a well-known issue and came across this SO question, who had the exact same problem. nodejs response speed and nginx
Overall, intresting stuff. Make sure you optimize your responses and send it all at once.
Best of luck to everyone!