I have a Perl script which uses LWP::UserAgent to retrieve a YouTube video's information via an HTTPS request. The script normally would take 2-3 seconds to retrieve three separate HTTPS requests. Now, it takes 15-20 minutes.
I wrote a similar script in Python where I use the following:
import google_auth_oauthlib.flow
import googleapiclient.discovery
import googleapiclient.errors
And this one request is taking 5-7 minutes.
Then - on Thursday night - the Perl script went back to only taking 2-3 seconds. This lasted for a few hours, and now the time is back to taking 15-20 minutes. The script seems to take forever on the HTTPS call - and not during any other time of the script.
If I take the HTTPS URL and put it in a web browser, it takes less than a second.
The script is over 2,000 lines long, so I can't post it here. But, all I am doing is trying to retrieve:
https://www.googleapis.com/youtube/v3/videos?part=statistics&key=$key&id=$video_comments_to_get,
where $key is my authorization key, and $video_comments_to_get is the 11 character YouTube video ID value.
This all started after YouTube's outage on November 11th. Any ideas why this would be taking so long? I only run this script 10-15 times a day - so it isn't like I am over my quota.
Related
I am trying to get execution date using
execution_dt = f"{{{{execution_date.in_timezone('{timezone}').strftime('{date_partition_format}')}}}}"
but the issue that I am facing is that the polling is happening for the previous hour right when DST is taking over, but is able to correct itself for subsequent runs.
So for example if the DST takes place and the clock has gone from 7th->8th hour, it would still try to poll for 7th hour, but in the subsequent run which is done after 2 hours, it will poll for 10th hour (in accordance with previous 8th hour).
I have a bash script that runs every five minutes to get an updated file from a ftp server. I know what time they generate the file which is every 5 minutes starting at 0 minutes of each hour. It usually takes them 30 seconds to generate the file and so I have mine offset by 1 minute (running every 5 minutes starting at 1 minute of each hour). However, there are times when their server bogs down and takes longer (sometimes minutes) to generate their file and it only takes about 13 seconds for me to download and process the file. When this happens I end up getting basically the first portion of the file and not the rest. Is there a way to verify that what I just downloaded matches what is on the ftp server? I was thinking maybe there was a way to check that the filesize of what I downloaded matches what is on the ftp server. Is that possible? My other thought was that if that is possible, depending on how quickly it can compare those two files, a delay may need to be built in to ensure it has time to for the file on the ftp server to have more data written to it if it is still in progress. Thoughts/suggestions? Thanks in advance.
I'm using load test in Visual Studio to test our web api services. But to my surprise I can't seem to test what I want to. Actually I have a single url in my .webtest file and try to send the same url time and again to see what is the avg. response time.
Here are the details
1.I use constant load of 1 user
2.Test duration of 1 hour
3.Think time of 10 seconds (not the think time between iterations)
4.The avg. response time that I get is 1.5 seconds
5.So the avg. test time comes out to be 11.5 seconds
6.Requests/sec are 0.088
7.And I'm using Sequential Test Order among 4 types of different tests
So these figures are making me think that every time a virtual user sends a request besides the specified think time it waits for the request to complete before he sends a new one (request). Thus technically the total think time becomes
Total think time = think time specified + avg. response time
But I don't want the user to wait for an already sent request to come back and then send a new one after a specified think time. I need to configure the load test in such a way that if the think time is 10 seconds then the user should send next request after every 10 seconds without waiting the first one to come back then think for another 10 seconds and then send a new request (hence making the total think time to 11.5 seconds in my case as mentioned above). And no matter what type of test I choose among 4 different types Visual Studio is always forcing the virtual user to wait for the completion of the request then add specified think time and then send a new one.
I know what Visual Studio load test is doing is more of a practical approach where the user sends the request wait till it comes back then think or interact with the website and then sends a new one.
Any help or suggestion would be appreciated towards what I'm trying to achieve.
In the properties of the scenario, set the "Test mix type" to be "Test mix based on user pace" and set the "Tests per user per hour" as appropriate. See here.
The suggestion in the question that:
Total think time = think time specified + avg. response time
is erroneous. To my mind adding the values does not provide a useful result. The two values on the right are as stated. Think time simulates the time a user spends reading the page, deciding what to do next and typing/clicking/etc their response. Response time is the "turn around" time between sending a request and getting the response. Adding them does not increase the think time in any sense, it just makes the total duration for handing the request in this specific test. Another test might make the same request with a different think time. Note that many web pages cause more than one request and response to be issued; JavaScript and other allow web pages to do many clever things.
I am running performance testing with concurrent users (50 threads) on a import function of the application in JMeter.
After clicked on import button, I made a call to get import status and keep looping it (using a While controller) until I receive a response with IsFinished=true.
But since I am running with 50 VU, it make 3-4 get status call every seconds and it overwhelmed the server.
So, I put a constant timer of 30 secs. It will then only made one get status call every 30 sec.
However, the wait time is not being counted toward the total time spent inside the while controller in the Summary Report.
I tested it with a single thread:
For a ~60 seconds import, it's showing 5 sec time spent in the while controller on the summary report.
How can I include the wait time in the report, so that I can get the total time spent for the import to complete?
Any help would be appreciated!
Thanks
Steven
enter image description here!
I think that you can use Transaction Controller which has the option to generate parent sample for all nested elements and include duration of pre/post processors and timers in the generated sample.
Transaction controller is commonly used where it is required to measure time for test actions sequence, but it can help to implement your scenario.
My understanding of the parse.com API rate limit is that it’s not a concurrent-job limit, it’s just the number of requests started in a given second. So if a user is, say, uploading a file from a slow network and it takes 30 seconds, that’s not 1 of my 30 req/s taken up that whole time. It’s just one request, the first second.
On my team, though, is a wonderful security guy whose job it is to worry. He thinks that if 30 users upload a file each, for 30 seconds, at a 30 r/s limit, no one else will be able to use our app until they are done.
Which one is correct?
Your understanding was correct. It's the number of requests started per second. The duration of the request does not come in to play.
Source: I work at Parse.
I think you are right. I've made some experiments with Parse, for example i reloaded a UITableview 10 or 20 times in one second (can't remember) for 3-4 minutes and checked the requests in the admin panel. The maximum value was always less than 30, but it doesn't matter, the point is that you can test it this way and get more informations.
Just create some test project and reload the SampleViewController.m (which contains a Parse query) 30 times in one second, after this you can check the data browser which will display the traffic by req/sec.
As a second option you can upload a bunch of images by current user in every second, since the upload time is longer than 1 sec, you can check what happens when you start uploading a bunch of images (or other data) in every second.