Automating jmx files in CircleCI - jmeter

I have couple of jmx files which I recorded and downloaded using blazemeter plugin.
I would like to know
How Can I integrate to CircleCI?
How can I set it to run daily?
Any help will be highly appreciated.

When you add a CircleCI project you have a variety of languages to choose from:
I would recommend going for Maven(Java) as JMeter Maven Plugin is the easiest option for setting up and using out of other JMeter non-GUI execution options.
Organise your project structure as follows:
yourproject
.circleci
config.yml
src
test
jmeter
test.jmx
pom.xml
Make sure that config.yml file looks as follows:
# Java Maven CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/openjdk:8-jdk
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/repo
environment:
# Customize the JVM maximum heap limit
MAVEN_OPTS: -Xmx3200m
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "pom.xml" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run: mvn dependency:go-offline
- save_cache:
paths:
- ~/.m2
key: v1-dependencies-{{ checksum "pom.xml" }}
# run tests!
- run: mvn verify
- store_artifacts:
path: target/jmeter/reports
Make sure your pom.xml looks like:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.jmeter</groupId>
<artifactId>maven</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.8.0</version>
<executions>
<!-- Run JMeter tests -->
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
<!-- Fail build on errors in test -->
<execution>
<id>jmeter-check-results</id>
<goals>
<goal>results</goal>
</goals>
<configuration>
<generateReports>true</generateReports>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
That's it, now your build will be triggered on each commit:
You will be able to see HTML Reporting Dashboard in build artifacts
Scheduling your builds is also possible via cron-like expressions

Related

Cannot rename a war or jar file on a maven deploy github actions

When I try to deploy a war or jar file with github actions I can't rename the war/jar file to what I want; I tried changing the name using finalName tag but that didn't work it seems when I install it before hand it gives it the proper name, however when it deploys the name defaults back to artifactId-version, is their a maven commmand that I can put into github actions in order to fix this?
here is the build tag section of my pom file I do have another build tag at the beginning of this code sample, stack overflow isn't showing it for some reason
<sourceDirectory>WEB-INF/src</sourceDirectory>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webXml>WEB-INF\web.xml</webXml>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat6-maven-plugin</artifactId>
<version>2.2</version>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<id>tomcat-run</id>
<goals>
<goal>exec-war-only</goal>
</goals>
<phase>package</phase>
<configuration>
<!-- optional only if you want to use a preconfigured server.xml file -->
<!-- <serverXml>src/main/tomcatconf/server.xml</serverXml> -->
<warRunDependencies>
<warRunDependency>
<dependency>
<groupId>a groupId</groupId>
<artifactId>and artifactId</artifactId>
<version>version</version>
<type>war</type>
</dependency>
<contextPath>/</contextPath>
</warRunDependency>
</warRunDependencies>
<!-- naming is disabled by default so use true to enable it -->
<enableNaming>true</enableNaming>
<!-- extra dependencies to add jdbc driver, mail jars, etc. -->
<extraDependencies>
<extraDependency>
<groupId>org.apache.derby</groupId>
<artifactId>derby</artifactId>
<version>10.1.3.1</version>
</extraDependency>
<extraDependency>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
<version>1.4</version>
</extraDependency>
</extraDependencies>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
Note
All depends of your pom.xml. If you are using custom plugins related to the artifact final name, it would be complicated to override the final name.
Simple Jar (maven solution)
Jar is usually used for libraries and standalone apps. After spring-boot, is used for server side apps.
According to this: https://stackoverflow.com/a/39407739/3957754 you can override the final jar name with:
mvn clean package -Djar.finalName=acmejar
I tested with this simple code and it works!
And then in your yml:
name: Java CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up JDK 11
uses: actions/setup-java#v2
with:
java-version: '11'
distribution: 'adopt'
- name: Build with Maven
run: mvn clean package -Djar.finalName=acmejar verify
Some frameworks with custom maven configurations may cause this method to not work. In that case, use some of these approaches to override the jar name using maven:
How to override maven property in command line?
War, spring-boot, etc (maven solution)
As I said, maven projects with complex plugins, needs special treatment in order to override the final artifact name.
The probed solution is to execute some operations after everything else finishes. I used Apache Maven AntRun Plugin
If I run mvn clean package in this repository, the generated war is: jersey-simple-1.0.0.war
But if I add this plugin:
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<configuration>
<tasks>
<copy file="${project.build.directory}/${project.build.finalName}.war"
tofile="${project.build.directory}/${newName}.war"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
And run this: mvn clean package -DnewName=foo
A new war name is generated:
You could use the move task if you don't need to have the initial file:
<move file="file.orig" tofile="file.moved"/>
Finally add this line to your github action yml.
This is the repository with the ant run plugin.
Pure unix commands
You can run basic unix commands in Github Actions.
So if you know the initial name after the success mvn clean package, you could use the mv command to rename it:
jobs:
docit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: rename war
run: |
mv "${dirname}/original.war" "${dirname}/newName.war"

Spring Boot Maven Plugin > 2.4.x build image publish on GitLab registry

I'm currently developing a GitLab CI/CD pipeline which compiles, tests and builds a standard Spring Boot application.
I want to package it in a docker image and publish that to the GitLab registry to use it later on.
Spring Boot recently added the build-image goal to its maven plugin which also has the ability to publish the image to a registry.
My problem is, that I can't get the auth to work.
I'm using a maven:3.6.3-jdk-11-slim image for the job with the docker:dind service to have access to a docker daemon.
Building the image runs fine, but publishing fails.
I configured the maven plugin in the project pom to use properties for auth, which will be overwritten by the CLI in my CI/CD Job as follows:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<docker>
<publishRegistry>
<username>${CI_REGISTRY_USER}</username>
<password>${CI_REGISTRY_PASSWORD}</password>
<url>${CI_REGISTRY}</url>
</publishRegistry>
</docker>
</configuration>
</plugin>
Properties defined in the POM with no value (Will be filled in by CLI call) :
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>11</java.version>
<CI_REGISTRY/>
<CI_REGISTRY_USER/>
<CI_REGISTRY_PASSWORD/>
</properties>
My maven CLI call in the Pipeline/Job uses the GitLab registry variables :
docker image job:
stage: Build
image: maven:3.6.3-jdk-11-slim
services:
- docker:dind
script:
- echo "java.runtime.version=11" > system.properties
- mvn spring-boot:build-image -DCI_REGISTRY=$CI_REGISTRY -DCI_REGISTRY_USER=$CI_REGISTRY_USER -DCI_REGISTRY_PASSWORD=$CI_REGISTRY_PASSWORD -Dspring-boot.build-image.imageName=SpringBootImage_${CI_JOB_ID} -Dspring-boot.build-image.publish=true
I was following the instructions via GitLab and Spring Boot documentation, but cant seem to identify my problem.
GitLab Registry Auth documentation
Spring Boot Maven Plugin image publishing documentation
I know it's been a while but in case others are trying to accomplish this
This works well
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<mainClass>com.foo.Main</mainClass>
</configuration>
<executions>
<execution>
<goals>
<goal>build-image</goal>
</goals>
<phase>deploy</phase>
<configuration>
<image>
<name>${env.CI_REGISTRY_IMAGE}/${project.artifactId}:${project.version} </name>
<publish>true</publish>
</image>
<docker>
<publishRegistry>
<username>${env.CI_REGISTRY_USER}</username>
<password>${env.CI_REGISTRY_PASSWORD}</password>
</publishRegistry>
</docker>
</configuration>
</execution>
</executions>
</plugin>
It uses gitlab-ci built-in env vars to configure everything

Jmeter: Test plan which was working in windows failing in Centos

I have a test plan which has two threads.
Each thread takes two CSV files as test data.
I have provided the CSV path as \testdata\csvtest1.csv this directory is located at src\test\jmeter\testdata, when I run this plan it works in Windows both in GUI mode and in non-gui mode via maven mvn clean verify.
But when I run this in Centos 7, it is giving below error I found in logs.
2018-10-04 13:56:24,739 INFO o.a.j.s.FileServer: Stored: \testdata\csvtest1.csv
2018-10-04 13:56:24,743 INFO o.a.j.s.FileServer: Stored: \testdata\csvtest2.csv
2018-10-04 13:56:24,740 ERROR o.a.j.t.JMeterThread: Test failed!
java.lang.IllegalArgumentException: File \testdata\csvtest2.csv must exist and be readable
So I manually copy-pasted test data directory with both CSV files inside Jmeter's bin directory. Still, it is giving the same error.
I have also tried solution here jMeter java.lang.IllegalArgumentException: File example.csv must exist and be readable and commented on answer but it didnt work.
Am I doing something wrong?
POM.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.demo.performancetesting</groupId>
<artifactId>demo-performance-testing</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<configuration>
<resultsFileFormat>xml</resultsFileFormat>
<generateReports>false</generateReports>
</configuration>
<executions>
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>de.codecentric</groupId>
<artifactId>jmeter-graph-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<inputFile>${project.build.directory}/jmeter/results/*.jtl</inputFile>
<graphs>
<graph>
<pluginType>ResponseTimesOverTime</pluginType>
<width>800</width>
<height>600</height>
<outputFile>${project.build.directory}/jmeter/results/BlazeDemoRequest.png</outputFile>
</graph>
</graphs>
</configuration>
</plugin>
</plugins>
</build>
</project>
Have you tried to read documentation?
Reference JMX files and CSV data
Once you have created your JMeter tests, you'll need to copy them to <Project Dir>/src/test/jmeter. By default this plugin will pick up all the .jmx files in that directory, to specify which tests should be run please see the project documentation.
You can also put data files in this folder and reference them in your plan.
So:
Copy your csvtest1.csv, csvtest2.csv, etc to the same location where your .jmx test(s) live, to wit src/test/jmeter folder
Reference them in the CSV Data Set Config elements just by name, i.e. csvtest1.csv
See Five Ways To Launch a JMeter Test without Using the JMeter GUI article for more information on various approaches to running JMeter tests, including using JMeter Maven Plugin.
Try to use / (slash) as file separator and make sure the files are readable from the user executing the test plan.

When I use Jmeter 5.0 and Jmeter Maven Plugin 2.7 together, it doesn't generate JTL file by default

My Jmeter test plan has two threads. Each thread uses different CSV file, runs separate test.
I am expecting it to generate two JTL files at the end of execution. In order to achieve this I have added separate Aggregate Table listener to each thread as suggested here Jmeter: Test plan has two thread groups but it generated only 1 jtl report. Still it wasn't generating JTL files. So I upgraded Jmeter from 3.2 to 5.0 and Jmeter Maven Plugin from 2.1.0 to 2.7.0. Now it is generating two separate CSV files not JTL files.
What wrong am I doing? Or is it a limitation of plugin that it won't generate JTL files?
I am using Maven 3, Windows 7, Java
POM.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.demo.performancetesting</groupId>
<artifactId>demo-performance-testing</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<executions>
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>de.codecentric</groupId>
<artifactId>jmeter-graph-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<configuration>
<resultsFileFormat>xml</resultsFileFormat>
<generateReports>false</generateReports>
</configuration>
<inputFile>${project.build.directory}/jmeter/results/*.jtl</inputFile>
<graphs>
<graph>
<pluginType>ResponseTimesOverTime</pluginType>
<width>800</width>
<height>600</height>
<outputFile>${project.build.directory}/jmeter/results/BlazeDemoRequest.png</outputFile>
</graph>
</graphs>
</configuration>
</plugin>
</plugins>
</build>
</project>
According to the JMeter Maven Plugin release notes
The current release of this plugin is 2.7.0, it requires JDK 8 and uses Apache JMeter 4.0
So I don't think you can really use JMeter 5.0 with the current version of the JMeter Maven Plugin.
Here are troubleshooting steps:
Download JMeter 4.0 from JMeter Archives page and make sure your test runs fine on it. If there are any issues, i.e. your test uses a deprecated feature which doesn't exist any more, or you're suffering from an incompatible change - fix your test accordingly. Once you can run your test in GUI mode - you should be able to run it using Maven
Check out the contents of target/logs file, it should contain a log file per your .jmx script which is(are) in src/test/jmeter folder, in the vast majority of cases JMeter log file contains enough information in order to be able to find out the root cause of the problem.
If you cannot figure out the cause yourself - next time make sure to update your question with:
Test Plan screenshot
Output of mvn verify command
JMeter log(s) file contents
If you want it to generate JTL (=XML) file add to maven plugin this:
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<executions>
<execution>
<id>jmeter-tests</id>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
<configuration>
<resultsFileFormat>xml</resultsFileFormat>
<generateReports>false</generateReports>
</configuration>
</plugin>
See:
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki/Test-Results-File-Format#4
In the configuration you show, you are making a mistake as you are adding this to jmeter-graph-maven-plugin plugin

How to make code coverage from FlexUnit work with Sonar?

Situation
I'm trying to get Sonar display the code coverage reports generated by FlexUnit from a Maven build job using Flex-Mojos but I'm not having any luck - all I ever get is a frustrating "-".
Build output
The result is that the dashboard always shows this (the left column):
(no, the unit tests don't run for over 90 minutes but rather 16 seconds; don't know what's off here)
The Sonar related console output is this:
So everything seems to work fine (no file-not-found errors other than the Cobertura one which I can't seem to get rid of in any way, no parse exceptions, etc).
Build setup
The pom.xml used for building the project looks like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>foo</groupId>
<artifactId>parent</artifactId>
<version>1.0.3</version>
</parent>
<groupId>foo</groupId>
<artifactId>bar</artifactId>
<version>0.1.0-SNAPSHOT</version>
<packaging>swc</packaging>
<name>Bar</name>
<dependencies>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito</artifactId>
<version>1.4M5</version>
<type>swc</type>
<scope>external</scope>
</dependency>
<dependency>
<groupId>com.adobe.flexunit</groupId>
<artifactId>flexunit</artifactId>
<version>4.1.0-8</version>
<classifier>flex</classifier>
<type>swc</type>
<scope>external</scope>
</dependency>
</dependencies>
<build>
<testSourceDirectory>src/test/flash</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.sonatype.flexmojos</groupId>
<artifactId>flexmojos-maven-plugin</artifactId>
<version>4.2-beta</version>
<configuration>
<coverage>true</coverage>
<debug>false</debug>
<optimize>true</optimize>
<omitTraceStatements>true</omitTraceStatements>
<verboseStacktraces>false</verboseStacktraces>
<defines>
<property>
<name>CONFIG::BUILD</name>
<value>"${project.version}"</value>
</property>
</defines>
</configuration>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>asdoc</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>${project.build.directory}/asdoc/tempdita</directory>
<targetPath>docs</targetPath>
<includes>
<include>**/*.xml</include>
</includes>
<excludes>
<exclude>ASDoc_Config.xml</exclude>
<exclude>overviews.xml</exclude>
</excludes>
</resource>
</resources>
</build>
<properties>
<sonar.language>flex</sonar.language>
<sonar.sources>src/main/flash</sonar.sources>
<sonar.tests>src/test/flash</sonar.tests>
<sonar.surefire.reportsPath>target/surefire-reports</sonar.surefire.reportsPath>
</properties>
</project>
I've tried several ways to run Sonar:
dynamicAnalysis=reuseReports + mvn clean install + mvn sonar:sonar
dynamicAnalysis=true + mvn clean install sonar:sonar -Dmaven.test.failure.ignore=true
dynamicAnalysis=true + mvn clean install -DskipTests=true + mvn sonar:sonar (<-- doesn't work: for some reason in this scenario the unit tests fail to run with a NullPointerException during execution of Flex-Mojos's test-run goal).
Is there a way to make displaying the coverage results in the Sonar dashboard work? Do I need additional plugins for that (Emma, Clover, whatever) to get the coverage from the standard Surefire reports to show up? Is there a known issue that prevents this from working? Am I doing something wrong?
Update
I've tried running Sonar with the Sonar-Runner. Interestingly, the dashboard then completely drops the code coverage widget. Checking the console output of the runner shows that the runner doesn't execute the FlexSurefireSensor (which the sonar:sonar Maven goal does):
The sonar-project.properties file contains:
sonar.projectKey=foo:bar
sonar.projectName=Bar
sonar.projectVersion=0.1.0-SNAPSHOT
sonar.language=flex
sources=src/main/flash
tests=src/test/flash
sonar.dynamicAnalysis=reuseReports
sonar.surefire.reportsPath=target/surefire-reports
I'm running it with mvn clean install followed by sonar-runner.
To get code coverage, you need to add the following property:
<sonar.cobertura.reportPath>target/site/coverage-cobertura/coverage.xml</sonar.cobertura.reportPath>
(or whatever the path to your coverage file is).
This is documented on the plugin home page.
I think the property you are missing in your POM is sonar.dynamicAnalysis:
<properties>
..
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.surefire.reportsPath>target/surefire-reports</sonar.surefire.reportsPath>
</properties>
The Flex plugin documentation describes how to enable Unit test and code coverage reporting. It also recommends using the Java Runner, but it should be functionally similar to the Maven plugin launcher.

Resources