trying to run k6 performance test cases by using the following scenario's
how to hit x amount of API per min for
example: Produce 500 messages per minute → Check how API behaves
It should be same, next time we run the test case.
This is easily achievable with the constant-arrival-rate executor:
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
scenarios: {
500_mps: {
executor: 'constant-arrival-rate',
duration: '10m',
rate: 500,
timeUnit: '1m',
preAllocatedVUs: 10,
maxVUs: 100,
},
},
};
export default function() {
http.get('your api');
}
The above code will try to run the default function 500 times per minute for 10 minutes. It will use at most 100 VUs – if your API is too slow, k6 will not start more VUs and you will not reach the target load.
Related
I'm programming a contacts export from our database to Google Contacts using the Google People API. I'm programming the requests over URL via Google Apps Script.
The code below - using https://people.googleapis.com/v1/people:batchCreateContacts - works for 13 to about 15 single requests, but then Google returns this error message:
Quota exceeded for quota metric 'Critical read requests (Contact and Profile Reads)' and limit 'Critical read requests (Contact and Profile Reads) per minute per user' of service 'people.googleapis.com' for consumer 'project_number:***'.
For speed I send the request with batches of 10 parallel requests.
I have the following two questions regarding this problem:
Why, for creating contacts, I would hit a quotum regarding read requests?
Given the picture link below, why would sending 2 batches of 10 simultaneous requests (more precise: 13 to 15 single requests) hit that quotum limit anyway?
quotum limit of 90 read requests per user per minute as displayed on console.cloud.google.com
Thank you for any clarification!
Further reading: https://developers.google.com/people/api/rest/v1/people/batchCreateContacts
let payloads = [];
let lengthPayloads;
let limitPayload = 200;
/*Break up contacts in payload limits*/
contacts.forEach(function (contact, index) /*contacts is an array of objects for the API*/
{
if(!(index%limitPayload))
{
lengthPayloads = payloads.push(
{
'readMask': "userDefined",
'sources': ["READ_SOURCE_TYPE_CONTACT"],
'contacts': []
}
);
}
payloads[lengthPayloads-1]['contacts'].push(contact);
}
);
Logger.log("which makes "+payloads.length+" payloads");
let parallelRequests = [];
let lengthParallelRequests;
let limitParallelRequest = 10;
/*Break up payloads in parallel request limits*/
payloads.forEach(function (payload, index)
{
if(!(index%limitParallelRequest))
lengthParallelRequests = parallelRequests.push([]);
parallelRequests[lengthParallelRequests-1].push(
{
'url': "https://people.googleapis.com/v1/people:batchCreateContacts",
'method': "post",
'contentType': "application/json",
'payload': JSON.stringify(payload),
'headers': { 'Authorization': "Bearer " + token }, /*token is a token of a single user*/
'muteHttpExceptions': true
}
);
}
);
Logger.log("which makes "+parallelRequests.length+" parallelrequests");
let responses;
parallelRequests.forEach(function (parallelRequest)
{
responses = UrlFetchApp.fetchAll(parallelRequest); /* error occurs here*/
responses = responses.map(function (response) { return JSON.parse(response.getContentText()); });
responses.forEach(function (response)
{
if(response.error)
{
Logger.log(JSON.stringify(response));
throw response;
}
else Logger.log("ok");
}
);
Output of logs:
which makes 22 payloads
which makes 3 parallelrequests
ok (15 times)
(the error message)
I had raised the same issue in Google's issue tracker.
Seems that the single BatchCreateContacts or BatchUpdateContacts call consumes six (6) "Critical Read Request" quota per request. Still did not get an answer why for creating/updating contacts, we are hitting the limit of critical read requests.
Quota exceeded for quota metric 'Critical read requests (Contact and Profile Reads)' and limit 'Critical read requests (Contact and Profile Reads) per minute per user' of service 'people.googleapis.com' for consumer 'project_number:***'.
There are two types of quotas: project based quotas and user based quotas. Project based quotas are limits placed upon your project itself. User based quotes are more like flood protection they limit the number of requests a single user can make over a period of time.
When you send a batch request with 10 requests in it it counts as ten requests not as a single batch request. If you are trying to run this parallel then you are defiantly going to be overflowing the request per minute per user quota.
Slow down this is not a race.
Why, for creating contacts, I would hit a quota regarding read requests?
I would chock it up to a bad error message.
Given the picture link below, why would sending 13 to 15 requests hit that quota limit anyway? ((there are 3 read requests before this code)) quota limit of 90 read requests per user per minute as displayed on console.cloud.google.com
Well you are sending 13 * 10 = 130 per minute that would exceed the request per minute. There is also no way of knowing how fast your system is running it could be going faster as it will depend upon what else the server is doing at the time it gets your requests what minute they are actually being recorded in.
My advice is to just respect the quota limits and not try to understand why there are to many variables on Googles servers to be able to tack down what exactly a minute is. You could send 100 requests in 10 seconds and then try to send another 100 in 55 seconds and you will get the error you could also get the error after 65 seconds depend upon when they hit the server and when the server finished processing your initial 100 requests.
Again slow down.
I would like to test my "waiting for server reply" UI behavior. How do I do that reliably without pausing my test for a hardcoded delay?
Say, I have a button that triggers an http request and has to show certain animation / actions until the response arrives.
A dumb working method is:
cy.route({
delay: 1000,
response: "blah blah",
})
// triggger submission
cy.get('#my-submit-button').click()
// will disappear upon receiving response
cy.contains('Waiting for response...')
I am pretty much sure that the "waiting" text will appear within a second while the response paused, but then I commit the sin of pausing the test for a whole second.
If I start to shorten or remove the delay then I am running a risk of creating flaky tests since there's a chance the response would be processed before I check for the existence of the "Waiting..." text, which would've been removed by that moment.
Is there a way to ensure the response is produced only after the check for the "Waiting..." text without a hard delay?
I naïvely tried to do a cypress assertion from the route's onResponse, but cypress wasn't happy about that:
cy.route({
onResponse: xfr => {
cy.contains('Waiting for response...')
return xfr
},
response: "blah blah",
})
cy.get('#my-submit-button').click()
producing the https://on.cypress.io/returning-promise-and-commands-in-another-command error:
Error: Uncaught CypressError: Cypress detected that you returned a promise from a command while also invoking one or more cy commands in that promise.
The command that returned the promise was:
> cy.click()
The cy command you invoked inside the promise was:
> cy.contains()
Because Cypress commands are already promise-like, you don't need to wrap them or return your own promise.
Cypress will resolve your command with whatever the final Cypress command yields.
The reason this is an error instead of a warning is because Cypress internally queues commands serially whereas Promises execute as soon as they are invoked. Attempting to reconcile this would prevent Cypress from ever resolving.
Could this work?
cy.route({
delay: 1000,
response: "blah blah",
}).as('getResponse')
cy.wait('getResponse')
https://docs.cypress.io/guides/guides/network-requests.html#Waiting
You could record the time these things happen using cy.moment(), then compare it afterwards using .isBefore() or one of the other moment functions.
I'm using onResponse here to record the time the response comes in.
let timeDisappeared;
let timeResponded;
cy.route({
delay: 1000,
response: "blah blah",
onResponse: () => {
timeResponded = Cypress.moment()
}
})
// triggger submission
cy.get('#my-submit-button').click()
// waits for the text to appear
cy.contains('Waiting for response...').should('exist')
// waits for the text to disappear
cy.contains('Waiting for response...').should('not.exist').then(() => {
timeDisappeared = Cypress.moment()
})
expect(timeResponded.isBefore(timeDisappeared))
After saw your comment and check the other answers
I think Bredan's answer is the best solution to do this. You can
stub a response with delay: x seconds
check the text to appear. Record the time: timeAppear
check the text to disappear. Record the time: timeDisappear
Check if timeDisappear - timeAppear > x seconds
or Check timeDisappear - timeAppear - x seconds > certain seconds(You can define the tolerance)
This proves the text shows in the given response time.
You can extend the response Delay to longer value.
I modified Bredan's answer a bit to reflect the steps above,
let timeAppear;
let timeDisappear;
//delay the response to come back after 20 seconds
cy.route({
delay: 20000,
response: "blah blah"
})
// triggger submission
cy.get('#my-submit-button').click()
// waits for the text to appear and record the time starts.
cy.contains('Waiting for response...').should('exist').then(() => {
timeAppear = Cypress.moment()
})
// waits for the text to disappear and record the time ends
// add timeouts option here to make sure it is longer than delay
cy.contains('Waiting for response...',{timeout:30000}).should('not.exist').then(() => {
timeDisappear = Cypress.moment();
//check the time is above the delay time 2000 ms
expect(timeDisappear - timeAppear).to.be.above(20000);
//or check the tolerance value e.g.5000 ms
expect(timeDisappear - timeAppear - 20000).to.be.below(5000);
})
I am using JMeter Webdriver sampler for the application UI response time measurement. I am facing issue with the wait function. For example the login page loads between 10 to 120 secs. So I have the following code for the login page in Webdriver sampler for the page load check.
var ui=JavaImporter(org.openqa.selenium.support.ui)
var wait=new support_ui.WebDriverWait(WDS.browser,120)
wait.until(ui.ExpectedConditions.visibilityOfElementLocated(pkg.By.className('logout-btn-hover')))
The issue is even after the page load completely, JMeter is still waiting to perform the next action. This waiting time will reduce if I reduce the 120 secs.But sometimes the application will take 120 secs to load also so I need to keep 120 secs.
I am writing the time to a log file once the sampler completes its action. Because of the wait time issue I am not able to calculate time properly.
There are at least 2 errors in your script, it should look something like:
var ui=JavaImporter(org.openqa.selenium.support.ui)
var wait=new ui.WebDriverWait(WDS.browser,120)
wait.until(ui.ExpectedConditions.visibilityOfElementLocated(org.openqa.selenium.By.className('logout-btn-hover')))
Check out jmeter.log file for any suspicious entries, in particular for something like:
ERROR c.g.j.p.w.s.WebDriverSampler: Expected condition failed: waiting for visibility of element located by By.className: logout-btn-hover (tried for 120 second(s) with 500 milliseconds interval)
Double check your CSS selector
Consider refactoring your code to look for the logout button(?) in a loop with verbose logging for each step. Sample code:
var pkg = JavaImporter(org.openqa.selenium)
WDS.sampleResult.sampleStart()
WDS.browser.get('http://example.com')
var start = new Date().getTime()
var attempt = 1
while (new Date().getTime() - start < 5000) {
try {
var logout = WDS.browser.findElement(pkg.By.className('logout-btn-hover'))
WDS.log.info('Element found')
break
}
catch (err) {
WDS.log.info('Attempt # ' + attempt + ', Element not found')
java.lang.Thread.sleep(1000)
attempt++
}
}
WDS.sampleResult.sampleEnd()
Example output when the element is not found:
Example output when the element is found:
Check out The WebDriver Sampler: Your Top 10 Questions Answered article for more information on using WebDriver sampler in JMeter scripts.
I have a .Net Core application with Hangfire implementation.
There is a recurring job per minute as below:-
RecurringJob.AddOrUpdate<IS2SScheduledJobs>(x => x.ProcessInput(), Cron.MinuteInterval(1));
var hangfireOptions = new BackgroundJobServerOptions
{
WorkerCount = 20,
};
_server = new BackgroundJobServer(hangfireOptions);
The ProcessInput() internally checks the BlockingCollection() of some Ids to process, it keeps on continuously processing.
There is a time when the first ten ProcessInput() jobs keeps on processing with 10 workers where as the other new ProcessInput() jobs gets enqueued.
For this purpose, I wanted to increase the workers count, say around 50, so that there would be 50 ProcessInput() jobs being processed parallely.
Please suggest.
Thanks.
In the .NET Core version you can set the worker count when adding the Hangfire Server:
services.AddHangfireServer(options => options.WorkerCount = 50);
you can try increasing the worker process using this method. You have to know the number of processors running on your servers. To get 100 workers, you can try
var options = new BackgroundJobServerOptions { WorkerCount = Environment.ProcessorCount * 25 };
app.UseHangfireServer(options);
Source
I have 1000 http API requests to be made. I have kept them all as promises in an array. I want to execute them in "BATCHES" of 100 at a time - not more than that to avoid hitting any API rate-limit / throttling etc.
While bluebirdJS provides the .map() function with the concurrency option what it does is it limits the number of calls made AT A TIME. Meaning it will ensure that no more than 100 concurrent requests are being worked on at a time - as soon as the 1st request resolves, it will begin processing the 101st request - it doesn't wait for all the 100 to resolve first before starting with the next 100.
The "BATCHING" behavior i am looking for is to first process the 100 requests, and ONLY AFTER all of the 100 requests have completed it should begin with the next 100 requests.
Does BlueBirdJS provide any API out of the box to handle batches this way?
You can split big urls array to an array of batches. For each batch run Promise#map which will be resolved when all async operations are finished. And run these batches in sequence using Array#reduce.
let readBatch(urls) {
return Promise.map(url => request(url));
}
let read(urlBatches) {
return urlBatches.reduce((p, urls) => {
return p.then(() => readBatch(urls));
}, Promise.resolve());
}
const BATCH_SIZE = 100;
let urlBatches = [];
for (let i = 0; i < urls.length; i+= BATCH_SIZE) {
let batch = array.slice(i, i + BATCH_SIZE);
urlBatches.push(batch);
}
read(urlBatches)
.then(() => { ... }) // will be called when all 1000 urls are processed