We use webpagetest to monitor our websites and see how it is performing. I know we can integrate Yslow with Phantom js and then Jenkins.
Is there any way we can integrate webpagetest.org with CI jenkins.
Thanks
You might try webpagetest-api which is a CLI tool and NodeJs module. You could fetch your test results and check for whatever metric you're after then output pass/no-pass.
Unfortunately CI integration does not come for free, you can either build your own integration with webpagetest-api node module or do some bash script magic with the CLI tool. Keep in mind webpagetest-api test (runTest) command is asynchronous, i.e.: it only schedule a test returning the test id so you can later check the results. There are some ways to integrate both test scheduling and fetching results, check it out my post on last performance calendar.
Related
I am trying to create a custom test report using Maven as my build tool, JUnit as my framework, along with selenium test cases. I was using maven's surefire report plugin but I need to include more information in my report. Can anyone direct me to a good tutorial on how to create a custom maven reporting tool?
We have a much better test automation dashboard based on just few api calls, ARES dashboard (built under Testastra and owned by ZenQ) is a much better option to try and it's absolutely free.
ARES, is an acronym for Test Automation Results dashboard. It's a TestAutomation framework/tool agonistic solution, that simplifies the collection of Test automation results and their analysis via live dashboard, daily/weekly trends, frequent failures etc. Website: http://www.testastra.com/#ares
Below repo has some code samples, documentation and usage of ARES test automation dashboard:https://github.com/testastra/ARES
Give it a try.
I have a general question about Jmeter. I am using Jmeter and I want to create Continuous integration solution using Jmeter. The problem is that there are several QA, that each one of them have test plans according his development. the problem is that if using Jmeter all test plan need to run as one test plan, and for each of test plan to create report, and than manually over each report and check for failed scenarios. So all the handling is much harder than I thought, Jmeter is wonderful solution but how to use it with several test plans, and if I want it to run nightly? and in the mornng to get single report with all test plans that created? Can someone please advise if Jmeter can be solution for CI nightly? is it possible to merge all reports to single report?, each test plan is for unique development, and we will have multiple testplans over time
Did you have a look at those plugins:
https://github.com/jmeter-maven-plugin/jmeter-maven-plugin which allows you to fail a build based on presence of errors in load test results which you could mix with https://jmeter-plugins.org/wiki/AutoStop/
https://plugins.jenkins.io/performance
JMeter per se is just a load testing tool, you need a continuous integration server. If you don't have one in your company there are several free and open source solutions you can consider like:
Jenkins
Buildbot
Cruise Control
Check out Jenkins vs. Other Open Source Continuous Integration Servers article to learn more about aforementioned tools, see sample build dashboards and commands to kick off a JMeter test.
I have a (slightly complex) spring webservice which communicates with multiple frontends via a RESTful API (JSON) and additionally with other devices via SOAP or REST. I'd like setup an automated test environment which is capable of the following things:
create preconditions via fixtures (POSTGRES DB)
send REST or SOAP messages against the API
is able to run certain task (requests against the API) at a specific
time/date
assert and validate the produced results (return of the API call or
check the DB)
run all tests independet from any frontend/UI
integrate the testing environment in my infrastructure (i.e. create a
docker container which runs all tests deployed by Jenkins)
preferably I'd like to build reusable components (i.e. for creating a user that is needed in multiple different tests and so on...). I know there are a lot of tools and frameworks (SoapUI, JMETER,...). But before trying them all and getting lost, I'd like to get an experience report from someone who has a simular setup.
we are using JMeter for API testing. We tried SOAPui but it had some memory issues. So we are pushing forward with JMeter and so far so good.
For your questions:
We are using MySQL, but this post seems to show how to set up a postgres connection in JMeter: https://hiromia.blogspot.com/2015/03/how-to-perform-load-testing-on.html
JMeter can send REST API requests
I'm not sure if this is possible but you could probably have your Jenkins job scheduled to run when you need the API to run the specific task at the specific time.
There are quite a few assertions in JMeter. I use the Response and the BeanShell Assertions a lot.
JMeter is independent from any front end UI which helps pinpoint bugs.
I have not run docker but I am running via Jenkins. This jenkins plugin has been helpful: https://wiki.jenkins.io/display/JENKINS/Log+Parser+Plugin
A few more Tips:
Use the HTTP Request Defaults element. It will save you from having to update all your HTTP requests.
Use the User Defined Variables to define variables you need.
You can combine user defined variables like: ${namePrefix}${myTime} but it will have to be in a 2nd User Defined Variable element(you cant combine them in the same element)
If you have multiple test environments, set up a user defined variable with a value like this: ${__P(testenv,staging)} This way, you can change it from a CLI like this: -Jtestenv=HOTFIX
We are using ExtentReports for pretty html results reports with a custom JSR223 Listener(find my old post on this site).
If your site uses cookies, use the HTTP Cookie Manager.
If you need things to happen in order, on the Test Plan element, check this option: Run Threat Groups consecutively. If you dont, JMeter runs them in a random order.
Hope this is helpful. Happy Testing!
I am working on a project where the core application needs a harness to send messages which will process the message and feed to a portal web application. I am writing the automated testing for the portal application using cucumber/ruby/capybara.
I have never used Jmeter but through a colleague i just understand the basics of the Jmeter GUI tool. I have been asked me to use Jmeter as harness tool so that I can later extend the functionality to do performance testing as well. How can I do this ? How can i use Jmeter in my automated scripts using cucumber/ruby/selenium or capybara to do my testing ?
I dont want to use GUI as the whole test case should be automated end to end, I have to figure out a way to integrate Jmeter functionality to do test data set up and then run my browser test to check whether the data exists.
i hope my question is clear, ask me if you are not. Hoping to get a solution as i dont think it is a uncommon question.
Note: i have seen the ruby-jmeter gem, i think it does partly what i need but it is actually a DSL to use jmeter in ruby.the problem with that is I have to stick the syntax of gem and have to figure all the options in gem. I have also read a blog that this can be done using Jmeter recording proxy and running the browser tests at the proxy which I am not sure how to do that. here is the link where this is mentioned:
https://groups.google.com/forum/#!topic/cukes/4ZGkf3a234Y
As per my understanding of the problem, please go through the below links
for recording the script using Jmeter:
http://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.pdf
for integrating selenium in Jmeter:
There are actually 2 ways of achieving this:
exporting all your selenium scripts as *.jar and copying it to /junit folder under /lib folder of Jmeter.
Use JUnit Sampler to execute your test cases:
http://intensetesting.wordpress.com/2013/09/25/integrating-jmeter-with-selenium-code/
use WebDeriver plugin for Jmeter and write your selenium test cases in Jmeter:
http://jmeter-plugins.org/wiki/WebDriverTutorial/
Also you can run Jmeter via command line.
I'm still new to PHPUnit Testing and seleniumRC, but i have managed to get them both working, so now i was wondering if it is possible to sent out an email when the test fails and passes after every test is run. the mail should go to the developer and the testing manager. is it possible to do that? and it would be very nice to generate a whole report and sent it out on all the test results. can someone please give me a proper direction which i can follow on how to get around this.
Thank you in advance
D~~
You have a few options available. For a small project, maybe a plain php script that redirects phpunit output to a file, parses it and acts accordingly. ob_start() could also be your friend for this task.
Getting into more complicated options, you could look also into using a couple of phing tasks for this. Then, last but not least, the holy-grail: very flexible for most any build tasks and best of all automated -> look into continuous integration tools such as the jenkins.
For small one-man team projects I opt for the simplest.