Jmeter: Test plan which was working in windows failing in Centos - jmeter

I have a test plan which has two threads.
Each thread takes two CSV files as test data.
I have provided the CSV path as \testdata\csvtest1.csv this directory is located at src\test\jmeter\testdata, when I run this plan it works in Windows both in GUI mode and in non-gui mode via maven mvn clean verify.
But when I run this in Centos 7, it is giving below error I found in logs.
2018-10-04 13:56:24,739 INFO o.a.j.s.FileServer: Stored: \testdata\csvtest1.csv
2018-10-04 13:56:24,743 INFO o.a.j.s.FileServer: Stored: \testdata\csvtest2.csv
2018-10-04 13:56:24,740 ERROR o.a.j.t.JMeterThread: Test failed!
java.lang.IllegalArgumentException: File \testdata\csvtest2.csv must exist and be readable
So I manually copy-pasted test data directory with both CSV files inside Jmeter's bin directory. Still, it is giving the same error.
I have also tried solution here jMeter java.lang.IllegalArgumentException: File example.csv must exist and be readable and commented on answer but it didnt work.
Am I doing something wrong?
POM.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.demo.performancetesting</groupId>
<artifactId>demo-performance-testing</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<configuration>
<resultsFileFormat>xml</resultsFileFormat>
<generateReports>false</generateReports>
</configuration>
<executions>
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>de.codecentric</groupId>
<artifactId>jmeter-graph-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<inputFile>${project.build.directory}/jmeter/results/*.jtl</inputFile>
<graphs>
<graph>
<pluginType>ResponseTimesOverTime</pluginType>
<width>800</width>
<height>600</height>
<outputFile>${project.build.directory}/jmeter/results/BlazeDemoRequest.png</outputFile>
</graph>
</graphs>
</configuration>
</plugin>
</plugins>
</build>
</project>

Have you tried to read documentation?
Reference JMX files and CSV data
Once you have created your JMeter tests, you'll need to copy them to <Project Dir>/src/test/jmeter. By default this plugin will pick up all the .jmx files in that directory, to specify which tests should be run please see the project documentation.
You can also put data files in this folder and reference them in your plan.
So:
Copy your csvtest1.csv, csvtest2.csv, etc to the same location where your .jmx test(s) live, to wit src/test/jmeter folder
Reference them in the CSV Data Set Config elements just by name, i.e. csvtest1.csv
See Five Ways To Launch a JMeter Test without Using the JMeter GUI article for more information on various approaches to running JMeter tests, including using JMeter Maven Plugin.

Try to use / (slash) as file separator and make sure the files are readable from the user executing the test plan.

Related

JMeter Maven Plugin - How to provide external Properties file while running JUnit Performance Tests

I want to run JUnit tests for the performance testing using JMeter Maven Plugin.
During the JUnit tests run, I use one external properties file located inside <project>/src/resources/env.properties
This file has some configuration which I need to run the tests successfully.
I have given this file in src/test/jmeter location as well but the plugin is not picking up this file.
During performance tests run using JMeter Maven Plugin, This file is not found because of the relative path given inside JUnit tests as
String fileName = "src/resources/env.properties";
I get the below error during the JMeter test run
[INFO] fileName = env.properties
[INFO] java.io.FileNotFoundException: src/resources/env.properties (No such file or directory)
Can you please help, where to give the location of this file? do I need to give in pom.xml?
These tests working perfectly fine using the JMeter application where I put this file under <JMeterApp>/bin/src/resources and run the test.
Please find below pom.xml file
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com</groupId>
<artifactId>myProject</artifactId>
<version>1.0-SNAPSHOT</version>
<name>MyProject</name>
<url>http://maven.apache.org</url>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>3.4.0</version>
<executions>
<!-- Generate JMeter configuration -->
<execution>
<id>configuration</id>
<goals>
<goal>configure</goal>
</goals>
</execution>
<!-- Run JMeter tests -->
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
<!-- Fail build on errors in test -->
<execution>
<id>jmeter-check-results</id>
<goals>
<goal>results</goal>
</goals>
</execution>
</executions>
<configuration>
<generateReports>true</generateReports>
<jmeterExtensions>
<artifact>kg.apc:jmeter-plugins:pom:1.4.0</artifact>
<artifact>kg.apc:jmeter-plugins-json:2.7</artifact>
</jmeterExtensions>
<junitLibraries>
<artifact>com:myProject:1.0-SNAPSHOT</artifact>
</junitLibraries>
<propertiesFilesDirectory>src/test/jmeter/src/resources</propertiesFilesDirectory>
<downloadExtensionDependencies>false</downloadExtensionDependencies>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
When you run your tests via JMeter Maven Plugin the current working directory looks like:
your_project\target\some-guid-like-structure\jmeter\bin
so in order to access the file which lives under your_project\src\resources you need to refer it like:
String fileName = "../../../../src/resources/env.properties";
More information: How to Use the JMeter Maven Plugin

Change order of Execution for Jmeter and run Pre-Processor before Config or use Environmental variable for FilePath of CSV DataSet Config

I am reading CSV File contents to pass to my HTTP Requests, Which I can do correctly, But the issue is the CSV file path, to read fro, is in one of the Environment variables.
I tried using a JSR223 PreProcessor to read the environment variable and set the path into a property, Then use the property value in the FilePath field,
But due to the order of execution of JMeter elements, The Congif elements are executed before Preprocessors, Even If I put them in SetUp threads.
I am a newbie to Jmeter tests. Looking for somebody to help on either how to change the order of execution(If Possible) to run the Config elements after Preprocessors, Or on how to directly get the Environment variable into the FilePath field of CSV DataSet Config?
I've tried using Setup threads to Preprocess and then the next thread to run the config but it doesn't work.
You can parameterize the path to the CSV file using __P() function like:
Once done you should be able to set the property value via -J command-line argument like:
on Windows
jmeter -JfilePath=%your_environment_variable% -n -t test.jmx
on Unix and derivatives:
jmeter -JfilePath=$your_environment_variable -n -t test.jmx
See Apache JMeter Properties Customization Guide for more information on JMeter Properties and ways of setting and overriding them.
If you're running your JMeter test via Maven - you will need to declare the relevant property via propertiesUser section:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>jmeter</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.9.0</version>
<configuration>
<propertiesUser>
<filePath>${env.filePath}</filePath>
</propertiesUser>
</configuration>
<executions>
<!-- Run JMeter tests -->
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
<!-- Fail build on errors in test -->
<execution>
<id>jmeter-check-results</id>
<goals>
<goal>results</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

When I use Jmeter 5.0 and Jmeter Maven Plugin 2.7 together, it doesn't generate JTL file by default

My Jmeter test plan has two threads. Each thread uses different CSV file, runs separate test.
I am expecting it to generate two JTL files at the end of execution. In order to achieve this I have added separate Aggregate Table listener to each thread as suggested here Jmeter: Test plan has two thread groups but it generated only 1 jtl report. Still it wasn't generating JTL files. So I upgraded Jmeter from 3.2 to 5.0 and Jmeter Maven Plugin from 2.1.0 to 2.7.0. Now it is generating two separate CSV files not JTL files.
What wrong am I doing? Or is it a limitation of plugin that it won't generate JTL files?
I am using Maven 3, Windows 7, Java
POM.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.demo.performancetesting</groupId>
<artifactId>demo-performance-testing</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<executions>
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>de.codecentric</groupId>
<artifactId>jmeter-graph-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<configuration>
<resultsFileFormat>xml</resultsFileFormat>
<generateReports>false</generateReports>
</configuration>
<inputFile>${project.build.directory}/jmeter/results/*.jtl</inputFile>
<graphs>
<graph>
<pluginType>ResponseTimesOverTime</pluginType>
<width>800</width>
<height>600</height>
<outputFile>${project.build.directory}/jmeter/results/BlazeDemoRequest.png</outputFile>
</graph>
</graphs>
</configuration>
</plugin>
</plugins>
</build>
</project>
According to the JMeter Maven Plugin release notes
The current release of this plugin is 2.7.0, it requires JDK 8 and uses Apache JMeter 4.0
So I don't think you can really use JMeter 5.0 with the current version of the JMeter Maven Plugin.
Here are troubleshooting steps:
Download JMeter 4.0 from JMeter Archives page and make sure your test runs fine on it. If there are any issues, i.e. your test uses a deprecated feature which doesn't exist any more, or you're suffering from an incompatible change - fix your test accordingly. Once you can run your test in GUI mode - you should be able to run it using Maven
Check out the contents of target/logs file, it should contain a log file per your .jmx script which is(are) in src/test/jmeter folder, in the vast majority of cases JMeter log file contains enough information in order to be able to find out the root cause of the problem.
If you cannot figure out the cause yourself - next time make sure to update your question with:
Test Plan screenshot
Output of mvn verify command
JMeter log(s) file contents
If you want it to generate JTL (=XML) file add to maven plugin this:
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<executions>
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
<configuration>
<resultsFileFormat>xml</resultsFileFormat>
<generateReports>false</generateReports>
</configuration>
</plugin>
See:
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki/Test-Results-File-Format#4
In the configuration you show, you are making a mistake as you are adding this to jmeter-graph-maven-plugin plugin

Step by Step Instructions to Setup SoapUI project to run as part of CI with Maven

There currently is a Continuous Integration framework running Selenium scripts with maven 3.0.4 which are invoked by Jenkins. I've created a project in SoapUI containing a number of tests within a test suite. I'd like to be have the SoapUI scripts to run as part of the existing Continuous Integration process but can't seem to get it going. I'v done extensive research and thus this is my last resort. Can somebody please provide step by step instructions on setting this up? I've done the following so far:
Created a folder in the Workspace directory for the REST tests.
Created an src folder within the above mentioned directory (Workspace). The src folder consists of the following structure: src/test/soapui/xml file with the name of the soapui project
Directly within the src folder I have a pom.xml file with the following:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<name>Web Service Test Module</name>
<groupId>com.dstvo</groupId>
<artifactId>SOA-Tesing</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<description>Web Service soapUI integration tests</description>
<pluginRepositories>
<pluginRepository>
<id>eviwarePluginRepository</id>
<url>http://www.soapui.org/repository/maven2/</url>
</pluginRepository>
</pluginRepositories>
<build>
<plugins>
<plugin>
<groupId>eviware</groupId>
<artifactId>maven-soapui-plugin</artifactId>
<version>4.0.0</version>
<configuration>
<projectFile>src/test/soapui/REST-Testing-soapui-project.xml</projectFile>
<host>local host and port</host>
<outputFolder>${project.build.directory}/surefire-reports</outputFolder>
<junitReport>true</junitReport>
<printReport>false</printReport>
</configuration>
<executions>
<execution>
<id>Soa_Tests</id>
<goals>
<goal>test</goal>
</goals>
<phase>test</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I don't know if the pom.xml file is all that's needed in order to have the soapui tests run with maven or if there are some steps that I'm missing? Please help?
Here https://stackoverflow.com/a/17802822/2324993 I've already answer how to run soap ui tests via shell script.
So you just need to run this script with maven. You can do it for example with this maven plugin: http://mojo.codehaus.org/exec-maven-plugin/usage.html. You can use it to execute a shell script, or any other executable

How to make code coverage from FlexUnit work with Sonar?

Situation
I'm trying to get Sonar display the code coverage reports generated by FlexUnit from a Maven build job using Flex-Mojos but I'm not having any luck - all I ever get is a frustrating "-".
Build output
The result is that the dashboard always shows this (the left column):
(no, the unit tests don't run for over 90 minutes but rather 16 seconds; don't know what's off here)
The Sonar related console output is this:
So everything seems to work fine (no file-not-found errors other than the Cobertura one which I can't seem to get rid of in any way, no parse exceptions, etc).
Build setup
The pom.xml used for building the project looks like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>foo</groupId>
<artifactId>parent</artifactId>
<version>1.0.3</version>
</parent>
<groupId>foo</groupId>
<artifactId>bar</artifactId>
<version>0.1.0-SNAPSHOT</version>
<packaging>swc</packaging>
<name>Bar</name>
<dependencies>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito</artifactId>
<version>1.4M5</version>
<type>swc</type>
<scope>external</scope>
</dependency>
<dependency>
<groupId>com.adobe.flexunit</groupId>
<artifactId>flexunit</artifactId>
<version>4.1.0-8</version>
<classifier>flex</classifier>
<type>swc</type>
<scope>external</scope>
</dependency>
</dependencies>
<build>
<testSourceDirectory>src/test/flash</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.sonatype.flexmojos</groupId>
<artifactId>flexmojos-maven-plugin</artifactId>
<version>4.2-beta</version>
<configuration>
<coverage>true</coverage>
<debug>false</debug>
<optimize>true</optimize>
<omitTraceStatements>true</omitTraceStatements>
<verboseStacktraces>false</verboseStacktraces>
<defines>
<property>
<name>CONFIG::BUILD</name>
<value>"${project.version}"</value>
</property>
</defines>
</configuration>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>asdoc</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>${project.build.directory}/asdoc/tempdita</directory>
<targetPath>docs</targetPath>
<includes>
<include>**/*.xml</include>
</includes>
<excludes>
<exclude>ASDoc_Config.xml</exclude>
<exclude>overviews.xml</exclude>
</excludes>
</resource>
</resources>
</build>
<properties>
<sonar.language>flex</sonar.language>
<sonar.sources>src/main/flash</sonar.sources>
<sonar.tests>src/test/flash</sonar.tests>
<sonar.surefire.reportsPath>target/surefire-reports</sonar.surefire.reportsPath>
</properties>
</project>
I've tried several ways to run Sonar:
dynamicAnalysis=reuseReports + mvn clean install + mvn sonar:sonar
dynamicAnalysis=true + mvn clean install sonar:sonar -Dmaven.test.failure.ignore=true
dynamicAnalysis=true + mvn clean install -DskipTests=true + mvn sonar:sonar (<-- doesn't work: for some reason in this scenario the unit tests fail to run with a NullPointerException during execution of Flex-Mojos's test-run goal).
Is there a way to make displaying the coverage results in the Sonar dashboard work? Do I need additional plugins for that (Emma, Clover, whatever) to get the coverage from the standard Surefire reports to show up? Is there a known issue that prevents this from working? Am I doing something wrong?
Update
I've tried running Sonar with the Sonar-Runner. Interestingly, the dashboard then completely drops the code coverage widget. Checking the console output of the runner shows that the runner doesn't execute the FlexSurefireSensor (which the sonar:sonar Maven goal does):
The sonar-project.properties file contains:
sonar.projectKey=foo:bar
sonar.projectName=Bar
sonar.projectVersion=0.1.0-SNAPSHOT
sonar.language=flex
sources=src/main/flash
tests=src/test/flash
sonar.dynamicAnalysis=reuseReports
sonar.surefire.reportsPath=target/surefire-reports
I'm running it with mvn clean install followed by sonar-runner.
To get code coverage, you need to add the following property:
<sonar.cobertura.reportPath>target/site/coverage-cobertura/coverage.xml</sonar.cobertura.reportPath>
(or whatever the path to your coverage file is).
This is documented on the plugin home page.
I think the property you are missing in your POM is sonar.dynamicAnalysis:
<properties>
..
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.surefire.reportsPath>target/surefire-reports</sonar.surefire.reportsPath>
</properties>
The Flex plugin documentation describes how to enable Unit test and code coverage reporting. It also recommends using the Java Runner, but it should be functionally similar to the Maven plugin launcher.

Resources