I have been trying to figure out how to measure the amount of time it takes a thread (virtual user) in JMeter to fully complete. I'm not necessarily concerned with response times at the moment. The API that I'm attempting to load test works in an async fashion. I make a request to start a job, I'm given a job id then I use that job id to check the status of said job until it's complete. I'm interested in knowing how long it takes for each job to complete i.e. when the job starts (thread is created) and when the job is completed (thread is done working).
I've seen several people suggest using the Transaction Controller in similar situations but that, unless I'm misunderstanding, gives me the total response time for all the requests in the "transaction" which doesn't help me.
This is what I have setup so far in JMeter:
Which actually works great, I make the initial request to submit the job and extract the job id. In a while loop I check the status of the job using the extracted id every 10 seconds (Constant Timer) until the job is complete.
This is what the aggregate report looks like for 5 concurrent users, I can also make the labels be the same so that it's compacted further but none of this information tells me how long a thread took. From the number of samples I can surmise that half the threads took roughly 10 seconds to complete and the others I can multiply by 10 seconds (sleep timer) and get a rough estimate how long it took to complete but that would be difficult to do (or at least time intensive) for a couple hundred threads. I was really hoping JMeter had something out of the box that would give me this information in a nice report format.
I've also seen suggestions that the only way to get this type of information is to parse logs, was just wondering if anyone has solved a similar problem.
Your question contains the answer, just measure it
Add JSR223 Sampler to the beginning of your Thread Group and put the following code into "Script" area:
SampleResult.setIgnore()
vars.putObject('startTime', System.currentTimeMillis())
the first line tells JMeter to not to store the JSR223 sampler result (as I believe you don't need this) and the second line saves current timestamp into ${startTime} JMeter Variable
Add another JSR223 Sampler to the end of your Thread Group and use the following code there:
SampleResult.setIgnore()
def end = System.currentTimeMillis()
def start = vars.getObject('startTime')
log.info('Thread ' + (ctx.getThreadNum() + 1) + ' elapsed time: ' + (end - start))
here we get the current timestamp after the job ended and subtract it from the previous timestamp store in the first JSR223 Sampler. The delta is printed to jmeter.log file however you might rather want to store it into another JMeter Variable and expose it to the results via Sample Variables property so it would be added to .jtl results file
Demo:
See Top 8 JMeter Java Classes You Should Be Using with Groovy to learn more about what these SampleResult, vars, and ctx shorthands mean
Related
I have designed a scenario using multiple ultimate threadgroup. Executed the test for one hour. After the completion of test I have observed weird data in JTL file.
Egs: one of the transaction has 8 api calls. At the end of test 4 only executed. That transaction elapse time is showing 30sec but if you sum of the executed api it is around 10sec. I am surprised where is the remaining 20sec.
At the end of test 4 only executed - when you run a JMeter test for the specified duration of the time there is no guarantee that all Samplers will be executed the equal number of times. Test termination signal might be somewhere in the middle and all the samplers which are below won't be run
It's hard to provide a comprehensive answer without seeing your test plan and the .jtl file, blind shot: double check your Transaction Controller configuration, if you have it like this:
JMeter will add the time taken by all PreProcessors, PostProcessors and Timers into the overall Transaction elapsed time
what is the meaning of the duration in jmeter thread group.
I found this is official document saying that is end time while the startup delay is the start time.
But is it the end time since we start executing the test? or since the thread is created?
When I set the 50 threads and duration 1s, the first timestamp in the output file is 1607391972250 (2020-12-08 09:46:12.25), the thread-50 only send HTTP Request 1 in timestamp 1607391973232 (2020-12-08 09:46:13.232). So it cannot be the end time since each thread is created
But when I set 1000 threads, the timestamp of the last several thread will exceed 1s than the timestamp of first thread's first request. it cannot be the end time since the we start executing the test
So what is the real meaning of the duration?
Duration is neither the start time nor the end time but the timespan between the two.
If your total duration is 1 second and your ramp up time is 1 second it might lead to a bit confusion since some of the threads have only just started when the whole test is already over. The duration starts counting when you either hit the "play button" in the GUI or start the test via command line.
Startup delay will mostly make sense if you have several thread groups inside your test plan and want some part of the virtual users to start at a later point for either logical reasons (e.g. thread group A generates forum posts and group B starts searching those forum posts after 1 minute) or for some kind of load shaping (might be achievable within one thread group though). This is relative to the test start.
Anyway JMeter is a tool mostly used for load and performance testing. In this type of testing you will usually have the tests run for minutes or hours. So from my perspective your question is mostly theoretical unless you want to achieve something very specific which you did not further elaborate.
As per Thread Group documentation:
Specify Thread lifetime If selected, confines Thread operation time to the given bounds
Duration (seconds) If the scheduler checkbox is selected, one can choose a relative end time. JMeter will use this to calculate the End Time.
So "Duration" limits the maximum duration of a given single thread, JMeter starts threads within the bounds of the Ramp-Up period.
In your case you have Ramp-Up time set to 1 second and Duration set to 1 second, it means that JMeter will start 50 threads in 1 second (so the last one will be started in 1 second after test start) and then the last thread will be terminated in another 1 second so its total lifetime plus the time required to execute the first sampler if its response time is more than 1 second
You may find Custom Thread Groups easier to use, all of them provide easy configurable visual way of defining the workload, you can install them using JMeter Plugins Manager
I am new to JMeter. I have two scripts one script is web and another is a rest api call which posts metrics to server. Both the scripts are working fine. Now i wanted to implement a scenario.
Web Script should run first once the script is completed i need to capture test metrics like start time, end time, load rate (No.of threads), Pass or fail save to a variable and pass these values to the rest api call which will then run and post the metrics to the server.
Any help is appreciated
Start time - can be obtained as ${TESTSTART.MS} JMeter pre-defined variable
End time - can be obtained via __time() function, if you call it somewhere in tearDown Thread Group it will report the time when all the main Thread Group(s) are done
Number of threads - it's a kind of weird requirement because its you who define the number of virtual users. Anyway, you can obtain it at any moment of time using i.e. __groovy() function like:
${__groovy(ctx.getThreadGroup().getNumberOfThreads(),)} - returns the number of threads which are active currently
${__groovy(ctx.getThreadGroup().getNumThreads(),)} - returns the number of threads which are defined in the Thread Group
As you are planning for the given scenarios you need to do following things.
1) You need to user jp#gc listeners to measure the results in (response time, threads per minute/seconds, hits per second and many more)
You can find the list of listeners here >> https://jmeter-plugins.org/wiki/GraphsGeneratorListener/
2) You need to implement the test plan using regular expression extractor for taking values from the response requests which you can store in the variables and later pass on to the dependent requests. for documentation visit https://jmeter.apache.org/usermanual/regular_expressions.html
For general understanding you can go through the jmeter official documentation
https://jmeter.apache.org/usermanual/get-started.html
I hope it will help you
I have jmeter test plan in which I create a workflow which transitions statuses like Not Started to In Progress to Completed.
The problem I am having is how I can measure how long it took for workflow status from the time of creation to it get completed? Since we are logging the response time of api calls only.
Is there a way we can extract this info and also add it to the summary (jtl reports?) I was thinking if I could do it from beanshell post processor where I have the wait time calculated and also checking the status of workflow and I could write to the summary result (jtl). How can I do this?
I have the test plan set up something like this:
Thread Group
- Create Workflow API call
- WhileLoop(checking on a flag set by beanshell post processor)
- Wait for 5seconds
- Check for timeout using JSR223 sampler(java)(don't want test to run inevitably if something goes wrong so want to break the loop after a fixed amount of time)
- Get Workflow status API call
- Beanshell post processor to check the response from above call to see if status is Completed or wait time has exceeded. In either of these cases I set the while loop flag false if not repeat the loop wait for 5 seconds and so on.
For the test itself it works as expected.
Use transaction controller to measure the time taken. Please check the reference;-
The Transaction Controller generates an additional sample which
measures the overall time taken to perform the nested test elements.
Hope this helps.
I am checking a load with a minimum of 2000 threads in JMeter in the command line mode. I am using Graphic Generator also to get nice graphs. But at the end of the execution, I am getting an aggregated result inside the graphics generated results. What I actually wanted is the time taken for each thread in a nice format either in CSV or in Graph.
The command I am using is
sh jmeter -n -t /Project/Tests/test.jmx -l /Project/Tests/results.csv
Even though the results.CSV generates the whole but its not in a nice format. Can someone suggest me any other better options if available? Because my program is expecting each thread to return within 7 seconds if not my program will discard that thread. Hence i need to know how many threads are returned within 7 seconds.
Actually you should already have what you need.
You can figure out threads response times from .jtl results file, look into elapsed column. You can sort and see the most time-consuming sample results and how many of them exceed 7000 ms
There is Response Times Over Time graph which can show the trend of response times while the test is running
There is Response Times Distribution graph which can show the statistics of response times per number of requests executed
Both plugins can be installed using JMeter Plugins Manager
And finally you can use Duration Assertion so JMeter would fail requests which last longer than 7 seconds automatically