I have been using ApacheJMeter_core and ApacheJMeter_http libraries to create and run Jmeter testplan from java Junit file.
My test plan is of below structure
POST thread group
POST api call for duration 20 secs
GET thread group
GET api call for duration 20 secs
Summariser is added for each thread group
When i set loopController.setLoops(<some value>); I could see below correct logs in System out
GET_Result = 16 in 00:00:01 = 15.8/s Avg: 61 Min: 4 Max: 287 Err: 0 (0.00%)
POST_Result = 4 in 00:00:01 = 3.9/s Avg: 260 Min: 36 Max: 485 Err: 0 (0.00%)
But when I use loopController.setContinueForever(true); and getThreadGroup.setScheduler(true);getThreadGroup.setDuration(20); postThreadGroup.setScheduler(true);postThreadGroup.setDuration(20); I don't see results printing in System logs.
The log is as below
0 in 00:00:00 = ******/s Avg: 0 Min: 9223372036854775807 Max: -9223372036854775808 Err: 0 (0.00%)
Did I miss something? Appreciate any help
I would also like to know how the values can be asserted in the Junit file. Is it possible to get Avg, Standard Deviation of results and assert the values
Thanks
Replace your
loopController.setContinueForever(true);
with
loopController.setLoops(-1);
See Five Ways To Launch a JMeter Test without Using the JMeter GUI article for more information on different ways of headless JMeter tests execution including programmatic.
Be aware that the only officially recommended way of creating a JMeter test is via JMeter GUI, other approaches exist but you use them on your own risk. The easiest way of creating a JMeter test programmatically is using Taurus tool.
Related
When I am passing the command in command prompt then I am getting the below error-
C:\Users\ShivangiT\Downloads\apache-jmeter-5.3\apache-jmeter-5.3\bin>Jmeter.bat -Jjmeter.save.saveservice.output_format=xml -n -t \Users\ShivangiT\Downloads\apache-jmeter-5.3\apache-jmeter-5.3\bin\vieweventpage.jmx -l \Users\ShivangiT\Downloads\apache-jmeter-5.3\apache-jmeter-5.3\bin\rr.jtl
Creating summariser <summary>
Created the tree successfully using \Users\ShivangiT\Downloads\apache-jmeter-5.3\apache-jmeter-5.3\bin\vieweventpage.jmx
Starting standalone test # Fri Aug 21 07:29:38 BST 2020 (1597991378434)
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
summary = 41 in 00:00:14 = 2.9/s Avg: 5256 Min: 7 Max: 13688 Err: 13 (31.71%)
Tidying up ... # Fri Aug 21 07:29:52 BST 2020 (1597991392905)
... end of run
The JVM should have exited but did not.
The following non-daemon threads are still running (DestroyJavaVM is OK):
Thread[DestroyJavaVM,5,main], stackTrace:
Thread[AWT-EventQueue-0,6,main], stackTrace:sun.misc.Unsafe#park
java.util.concurrent.locks.LockSupport#park
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#await
java.awt.EventQueue#getNextEvent
java.awt.EventDispatchThread#pumpOneEventForFilters
java.awt.EventDispatchThread#pumpEventsForFilter
java.awt.EventDispatchThread#pumpEventsForHierarchy
java.awt.EventDispatchThread#pumpEvents
java.awt.EventDispatchThread#pumpEvents
java.awt.EventDispatchThread#run
Thread[AWT-Shutdown,5,system], stackTrace:java.lang.Object#wait
sun.awt.AWTAutoShutdown#run
java.lang.Thread#run
Can anybody please help me with this.
This is a known issue of JMeter 5.3 when test plan contains Http(s) Test script recorder.
The workaround is to remove it.
See:
https://bz.apache.org/bugzilla/show_bug.cgi?id=64479
Alternatively you can try nightly build:
https://ci.apache.org/projects/jmeter/nightlies/
Shiva, I would suggest that you should use the older version of JMeter always. The reason is very simple. There is not much of a change in JMeter since JMeter 4. JMeter is more inclined towards its compatibility with JDK 11 as currently, it supports JDK 8 flawlessly in the older versions. Use JMeter 4 from the JMeter official archive and you'll be able to execute everything smoothly. No need to look for workarounds. Make sure you use JMeter 4
Issue
I have a go package, with a test suite.
When I run the test suite for this package, the total runtime is ~ 7 seconds :
$ go test ./mydbpackage/ -count 1
ok mymodule/mydbpackage 7.253s
However, when I add a -cpuprofile=cpu.out option, the sampling does not cover the whole run :
$ go test ./mydbpackage/ -count 1 -cpuprofile=cpu.out
ok mymodule/mydbpackage 7.029s
$ go tool pprof -text -cum cpu.out
File: mydbpackage.test
Type: cpu
Time: Aug 6, 2020 at 9:42am (CEST)
Duration: 5.22s, Total samples = 780ms (14.95%) # <--- depending on the runs, I get 400ms to 1s
Showing nodes accounting for 780ms, 100% of 780ms total
flat flat% sum% cum cum%
0 0% 0% 440ms 56.41% testing.tRunner
10ms 1.28% 1.28% 220ms 28.21% database/sql.withLock
10ms 1.28% 2.56% 180ms 23.08% runtime.findrunnable
0 0% 2.56% 180ms 23.08% runtime.mcall
...
Looking at the collected samples :
# sample from another run :
$ go tool pprof -traces cpu.out | grep "ms " # get the first line of each sample
10ms runtime.nanotime
10ms fmt.(*readRune).ReadRune
30ms syscall.Syscall
10ms runtime.scanobject
10ms runtime.gentraceback
...
# 98 samples collected, for a total sum of 1.12s
The issue I see is : for some reason, the sampling profiler stops collecting samples, or is blocked/slowed down at some point.
Context
go version is 1.14.6, platform is linux/amd64
$ go version
go version go1.14.6 linux/amd64
This package contains code that interact with a database, and the tests are run against a live postgresql server.
One thing I tried : t.Skip() internally calls runtime.Goexit(), so I replaced calls to t.Skip and variants with a simple return ; but it didn't change the outcome.
Question
Why aren't more samples collected ? I there some known pattern that blocks/slows down the sampler, or terminates the sampler earlier than it should ?
#Volker guided me to the answer in his comments :
-cpuprofile creates a profile in which only goroutines actively using the CPU are sampled.
In my use case : my go code spends a lot of time waiting for the answers of the postgresql server.
Generating a trace using go test -trace=trace.out, and then extracting a network blocking profile using go tool trace -pprof=net trace.out > network.out yielded much more relevant information.
For reference, on top of opening the complete trace using go tool trace trace.out, here are the values you can pass to -pprof= :
from go tool trace docs :
net: network blocking profile
sync: synchronization blocking profile
syscall: syscall blocking profile
sched: scheduler latency profile
I have a total of 405 tests. They are all executed fine when running on a single thread. However, when trying to run it in parallel, it seems the number of tests is not being properly allocated per thread.
So, for example, executing it using 3 threads:
mvn integration-test -Dwebdriver.remote.url=http://selenium-hub.project.svc.cluster.local:4444/wd/hub \
-Dwebdriver.remote.driver=chrome \
-Dwebdriver.driver=chrome \
-Dconfig.threads=3 \
-Dserenity.batch.size=3 \
-Dserenity.batch.number=<"from 1 to 3"> \
-Dserenity.batch.strategy=DIVIDE_BY_TEST_COUNT \
-Dserenity.take.screenshots=FOR_EACH_ACTION
After triggered maven, as according to the sample above, the tests have been allocated as follows:
Thread 1: 106
Thread 2: 96
Thread 3: 103
Total: 305
The funny thing is that those numbers vary, changing the tests count per thread on every execution.
As well, it is like it is counting 4 threads instead of 3.
I'm running those tests using Jenkins, hosted in an Openshift environment.
Found a workaround by increasing the number of threads (e.g. from 3 to 4). It looks like somehow there is a limitation regarding the number of tests executed per thread.
I will search for this config. and keep this post updated in case I find something.
I have a master/slave for jmeter set up using jmeter 5.1
For time to time I am noticing the tests just hangs up while waiting for threads to shutdown.
In the jmeter.logs I am seeing:
2020-02-06 00:06:35,100 INFO o.a.j.r.Summariser: summary + 9 in 00:30:34 = 0.0/s Avg: 5647 Min: 5520 Max: 5833 Err: 0 (0.00%) Active: 1 Started: 4 Finished: 3
I tried waiting but it never finishes this 1 active thread and it causes issue for rest of the steps I have in the pipeline to read the jmeter test result file and generate HTML report.
Any suggestions how to debug this?
I saw this post:
Threads keep running even after test finishes in Jmeter
But would be nice to understand the issue, rather than just forcing the threads to stop.
Regards,
Vikas
If you want to "understand" the issue you need to understand what this thread is doing and the only way to get the information is taking a JVM thread dump, the options are in:
Starting from JMeter version 3.2 there is an option to take a thread dump directly from JMeter GUI
You can use jstack tool and provide to it the PID of the Java process where JMeter is running
On Linux you can use kill -3 command which will print the status of threads into the console window
You can also check jmeter-server.log for for any suspicious entries.
I have a mvn project in which is also integrated jmeter, to test performance. So far I have 6 thread groups in my test plan, all of them contains HTTP Requests. I run the tests using the command "mvn clean verify" from jmeter-maven plugin. Among the results I found multiple rows like this:
summary + 1 in 00:00:02 = 0.6/s Avg: 208 Min: 208 Max: 208 Err: 0 (0.00%) Active: 6 Started: 12 Finished: 6
I would need some extra information in console, especially the name and the avg time of each thread group or of the HTTP Request that ran. For example, something similar with aggregate report from GUI mode :
Label Samples Average Median 90% Line 95% Line 99% Line Min Max ...
AppleCodeRequest 6 196 119 279 284 284 108 284
PearCodeRequest 3 382 485 490 490 490 173 490
I want this because I am using a sh script to run the tests and I would like to trigger some performance issues before opening the html reports.
Is there any way to obtain this? Maby some user properties (even if I searched for one and no result) or some workaround ?
The easiest solution is going for a plugin like BlazeMeter Uploader, this way you will be able to observe real time test metrics in a fancy web UI. You can install BlazeMeter Uploader plugin using JMeter Plugins Manager
Alternative solution would be using JMeterPluginsCMD Command Line Tool.
Add the next lines to your pom.xml file
<configuration>
<jmeterExtensions>
<artifact>kg.apc:jmeter-plugins-cmd:2.2</artifact>
<artifact>kg.apc:jmeter-plugins-synthesis:2.2</artifact>
<artifact>kg.apc:jmeter-plugins-dummy:0.2</artifact>
<artifact>kg.apc:cmdrunner:2.0</artifact>
<artifact>kg.apc:jmeter-plugins-filterresults:2.2</artifact>
<artifact>kg.apc:jmeter-plugins-cmn-jmeter:0.6</artifact>
</jmeterExtensions>
<!-- The plugin uses some broken dependencies
An alternative is to set this to true and use excludedArtifacts, see below
-->
<downloadExtensionDependencies>false</downloadExtensionDependencies>
<propertiesJMeter>
<jmeter.save.saveservice.autoflush>true</jmeter.save.saveservice.autoflush>
</propertiesJMeter>
</configuration>
Add another Thread Group to your Test Plan with 1 user and infinite number of loops
Add JSR223 Sampler to your Thread Group
Put the following code into "Script" area:
SampleResult.setIgnore()
def resultFile = new File('../results').list().first()
"java -jar ../lib/ext/cmdrunner-2.0.jar --tool Reporter --generate-csv temp.csv --input-jtl ../results/$resultFile --plugin-type AggregateReport".execute().waitFor()
println("cat temp.csv".execute().text)
new File("temp.csv").delete()
Control how frequently you want to see this info using i.e. Constant Timer
You should be able to see the results in the console window: