So I'm trying to run JMeter from ALM. I have a bunch of scripts that will do load testing. At this point, I'm grasping at straws to get anything working. Originally I wanted to use LoadRunner, but I can't figure out how to actually run the scripts I have developed in that application either.
That said: I have written some code just to attempt to run the JMeter application with the relevant script. As I understand it, ALM runs the VBS locally, so local paths should work, and running software installed on a local machine should work...
Dim objShell
Set objShell = WScript.CreateObject( "WScript.Shell" )
objShell.Exec "C:\Users\[path]\apache-jmeter-2.12\bin\jmeter.bat -t C:\Users\[path]\script.jmx"
This executes fine, but when looking at it, nothing appears. Ideally I'd have this run with no GUI (-n flag for jmeter), but I want to at least verify that the thing even starts up.
Currently nothing is happening, and I can't figure out what to do. I've attempted loops that wait until something happens, but they end up being infinite loops.
Any help would be greatly appreciated. Or...if you know of a resource that could actually help run either LoadRunner or JMeter scripts from ALM, I would be so grateful.
Thanks!
Maybe your ALM script doesn't respect underlying OS properties like JAVA_HOME or you don't have java executable in your PATH variable.
Try changing your line to look as follows:
objShell.Exec "c:\java\bin\java.exe -jar C:\Users\[path]\apache-jmeter-2.12\bin\ApacheJMeter.jar -t C:\Users\[path]\script.jmx"
to see how it goes.
By the way, there are several more ways of launching JMeter test, to learn about options check out 5 Ways To Launch a JMeter Test without Using the JMeter GUI guide, hopefully it will help to identify and apply the best integration approach.
Why......What is optimal for performance unit tests is not optimal for performance business tests. You also have people building tests for JMETER who it is not their primary focus to engage in performance testing. The requirements will be soft, the data sparse, the liberties on think time and pacing great. You are looking at a low value path. This perspective comes from a reformed developer.
Just rebuild them natively. Your value will be higher and a better match to the requirements you are looking to validate. Need a starting point? Execute your JMETER scripts with the target of your LoadRunner recording proxy. Input = JMETER requests, output = native LoadRunner code (in C). No funky library compromises for trying to have one tool execute another one.
If you are hellbent on pursuing this path and cannot/will not be dissuaded then consider the JAVA template virtual user. This is likely your best path, assuming you are licensed for it. You will also find that the odds are close to certainty that you will spend more time achieving a stable model than you will in just rebuilding the existing test code.
Try this :
Dim objShell
Set objShell = WScript.CreateObject( "WScript.Shell" )
objShell.Exec """C:\Program Files\apache-jmeter-2.13\bin\JMeter.Bat"" -n -t ""C:\Program Files\apache-jmeter-2.13\bin\templates\DMS_APP.jmx"""
Related
I have a shell script deploying the utility jar file and starting the JMeter with the last test plan. Now I would like to close any existing JMeter IDEs before the deployment.
Ideally, the script should invoke the IDE's save menu before closing the GUI.
This will avoid losing any unsaved work.
Note
Killing the JMeter's process may not work in my use case.
I don't understand how this question is connected to JMeter.
It sounds like you're looking for a desktop automation tool like Appium, Taurus, Sikuli, LDTP, etc.
It might be the case that you "shell" script can also send "Save" signal to running JMeter instances and close them, however it depends on the operating system and "shell" type.
I am currently using JDK Flight Recorder with JDK 11 and came across some trouble in the CI/CD Plattform. Unfortunately, there is not too much documentation on the new Flight Recorder, but rather on the older one, which was still developed under Java.
When I try to start tests directly from the IDE, everything works fine and I get my recording files.
When I try to do the same thing, automatically, in the CI/CD Plattform, it causes time out and a lot of different indefinite failures, among them: trouble creating the file, the file is not even written, etc.
The JVM commands I used are the following (I put extra spaces for better readability):
-XX:+FlightRecorder
-XX:StartFlightRecording= name="UiTestServer", settings="profile", dumponexit=true, filename=""+System.getenv("CI_PROJECT_DIR") + "flightRecording/javaFlightRecorder.jfr"
The commands are the same that the IDE uses automatically, when starting the flight recording with right click on the specified test.
Does anybody know, whether the Flight Recorder has problems with such systems or specific services which might run parallely to it? I heard of some profiling tools, that are unable to perform on CI Plattforms.
If you need more detail, just ask me. Though, it might happen that I cannot tell anything related to the project.
Bit late as an answer, but JFR can definitely run in CI/CD environments. I have successfully attached JFR to our JMH microbenchmarks and published the results as artifacts in Atlassian Bamboo. Our Bamboo agents are running on AWS, so JFR itself should be good for most cloud environments.
JFR has been built to work in production systems, but if you want guarantees of low overhead (<1%), you should use the default settings, not profile.
'profile' is for a shorter period of time, i.e. 10 minutes, where it may be OK with additional overhead to gain more insight.
This is what I would recommend, for JDK 11 and later:
$ java -XX:StartFlightRecording:filename=/path
There is no need set dumponexit=true if a filename has been specified.
-XX:+FlightRecorder is only needed before JDK 8u40.
You can set a name if you like, but it's typically not needed. If you want to use jcmd and dump a recording, the name can be omitted.
The following command on Jenkins freestyle doesn't exit(runs forever) in Windows environment. However when I run the same command in CMD it works as expected.
matlab -nodesktop -nosplash -minimize -c "path_to_license" -r "run('path_to_run_all_tests_script')" -wait -logfile unittestlog.txt set output=%errorlevel% MORE unittestlog.txt EXIT %output%
What am I doing wrong?
EDIT:
This is the Matlab script that the command calls:
import matlab.unittest.TestSuite;
disp('Hello from TestSuite');
try
suite = TestSuite.fromPackage('test','IncludingSubpackages',true);
results = run(suite);
display(results);
catch e
disp(getReport(e,'extended'));
exit(1);
end
exit(any([results.Failed])+1);
My guess would be that this is a licensing issue - you need Jenkins to be running as the same user that is named in the MATLAB license (assuming of course that your MATLAB license is a named-user license - if it's a concurrent or named-machine license, please ignore me).
If it's the wrong user, then MATLAB will start up, but show a dialog box telling you of the licensing issue. However, since you've called it with -nodesktop, this dialog will not be visible, and MATLAB will then just hang around for ever as a "zombie" process without sending a return value to Jenkins. If you go into Task Manager, you may find that you have one or more of them hanging around, and you can force-quit them there.
Note that you should be careful about the licensing requirements if you're running MATLAB for unit testing or automated builds via Jenkins. If you have a named-user license, only that named user is allowed to use it, and that includes via Jenkins. So long as you're the only person who can kick off a Jenkins/MATLAB job, that's fine - but if you're exposing Jenkins to several users, all of whom can kick off a MATLAB job, you're violating the license agreement. If you need to do that, you should really acquire a concurrent license for Jenkins to run under. But it's a bit of a grey area, you might want to speak to your account manager about it.
Hope that helps!
Edit: I think you might also want to take a look at the exit codes you're using in your test script. You want MATLAB to exit with a success code of zero when the tests pass, and a non-zero failure code when the tests fail, or when an error occurs during testing. So you probably want your last line to be exit(any([results.Failed]));, not exit(any([results.Failed])+1);.
Problem was related to licensing. Jenkins should be run by the same user who has Matlab license.
In addition to Sam Roberts answer: To change Jenkins's user, go to services in computer management, find the Jenkins service and change the user from properties.
what are the two types of execution that can be done in jmeter?
This was asked in a interview to me. i was asked what are the ways u execute the script i.e GUI or non -GUI. I am working on jmeter but I have never come across these two words
GUI means "graphical user interface", like you run in microsoft windows.
We can run Jmeter in both GUI or Non-GUI (From command line).
I strongly suggest you do some reading about Jmeter on below website
http://jmeter.apache.org/usermanual/index.html
http://developer.amd.com/community/blog/2009/03/31/using-apache-jmeter-in-non-gui-mode/
hope this will help.
It is absolutely not recommended to use JMeter GUI (graphical user interface) for anything apart from developing or debugging test as AWT Event Thread will kill both your test and JMeter in case of more or less high load.
There is a variety of approaches on how to run a JMeter test in non-GUI mode including:
command-line mode
Apache Ant task
Apache Maven plugin
Jenkins plugin
Execution pre-defined .jmx from Java code or creating a test on-the-fly
Plugins for IDEs like Eclipse or Idea
See 5 Ways To Launch a JMeter Test without Using the JMeter GUI for more details on above.
I'm running a test script from batch file.
Because it is test, the programs are expected to fail once in a while. It is file as long as error code is returned so I can continue and mark specific test as failed.
However there is very annoying behavior of executable files under Microsoft Windows - if something fails it pop-ups window like:
This application has failed to start because foo.dll was not found, Re-installing the application may fix the problem
<OK>
Or even better:
The instruction at "..." referenced to memory at "..." ..
Click on OK to terminate the program
Click on CANCEL to debug the program
The result is known - the script execution blocks till somebody presses "Ok" button. And when we talk about automatic scripts that may run automatically at night in some headless virtual machine, it may be very problematic.
Is there a simple way to prevent such behavior and just make an application to exit with failure code - without changing the code of the program itself?
Is this possible at all?
The answer is following: You need to disable WER.
Simplest description for this I found at http://www.noktec.be/archives/259
Simply (ON XP): Right Click on My Computer > Advanced > Error Reporting > Disable
Voila - programs crash silently!
This does not solves problem when DLL is missing, but this is much rare case and this is good enough for me.
You can suppress AV's and such from showing a dialog box by running your application, or the script (the script engine, like cscript.exe), under a debugger.
Use Gflags.exe, or modify the registry directly, and set Image File Execution Options for the image in question. See this article for details on how to use the appropriate registry keys. You can set it up using a debugger commandline like "C:\Debuggers\ntsd.exe -g -G -c'command'", where you can pass commands to ignore certain types of exceptions in the -c"commmand" argument. This will effectively give you a tool to suppress interactive dialogs as a result of exceptions like AV, and will let the process continue (presumably to immediate end after the exception has occured).
This article explains the commands you can use to control exceptions and events from withing the debugger.
The -g and -G flags make sure that the process won't break into the debugger automatically during process start and end respectively. You'll have to play with the various exception suppression options to make sure that you 'eat' all possible first and second chance exceptiosn that might cause the process to break into the debugger.
Also, if you can tolerate a process being broken into the debugger (as against being stuck showing a dialog box), perhaps that would be a better option overall. You can evaluate each debug break in batch mode at a later time and decide which bugs you care to fix.
It is possible. We used to use IBM's Rational Robot product which could monitor the screen for specific items and, if found, send keystrokes to windows and other sorts of things.
We actually used it for fully automated unit and system testing, much like you're trying to do.
Now I thought that Robot has been through quite a few name changes so it may be hard to find but there it is, right on IBM's web page and with a free downloadable trial for you. It's not cheap, clocking in at a smidgeon under USD5,000 but it was worth it for us.
There's also TestComplete where you could get a licence for just unedr USD1,000 - it touts "Black-box testing - Functional testing of any Windows application" as one of its features and also has a downloadable demo to see if it's suitable before purchase.
However, you may be able to find another product to do the same sort of thing.
I initially thought of Expect but the ActiveState one seems to concentrate on console applications which leads me to believe it may not do graphics well.
The only other option I can suggest is to write your own program in VBScript. I've done this before to automate the starting of many processes (log on to work VPN, start mail, log in and so on) so I could be fully set up with one mouseclick instead of having to start everything manually.
You can use AppActivate to bring a window to the foreground and SendKeys to send arbitrary keypresses to it after that. It's possible you may be able to cobble together something from that if you want a cheaper solution.