Our webapp has several very long running requests where we show the user a progress bar. It doesn't makes sense to use the same APDEX values for these requests.
The APDEX table includes columns showing the thresholds for each page, which leads me to believe there should be a way to do it. Why include them per page instead of just saying once at the top.
I would also be totally happy just excluding these pages from the APDEX completely.
I can't find anything about exlcuding or customizing APDEX in the docs.
You cannot customize thresholds per transaction, it is a enhancement request that has not yet been implemented:
https://bz.apache.org/bugzilla/show_bug.cgi?id=60112
Edit on 14/04/2017:
since JMeter 3.2, it is now possible to customize thresholds per transaction
jmeter.reportgenerator.apdex_satisfied_threshold=500
jmeter.reportgenerator.apdex_tolerated_threshold=1500
jmeter.reportgenerator.apdex_per_transaction=sample(\\d+):1000|2000;\
sampleA:3000|4000;\
scenarioB:5000|6000
You can cutomize the template that contain the index, it's a freemarker template located in:
bin/report-template/index.html.fmkr
I suggest you copy existing folder, modify the file and reference it with property in user.properties file:
jmeter.reportgenerator.exporter.html.property.template_dir=path to the copied and modified template folder
An even nicer idea would be to implement the enhancement and contribute it back to JMeter.
It's possible with JMeter 3.3 like this
jmeter.reportgenerator.apdex_per_transaction=sample(\\d+):1000|2000;\
sampleA:3000|4000;\
scenarioB:5000|6000
Documentation is here: https://jmeter.apache.org/changes_history.html
JMeter 3.2 had a bug, which makes this feature unusable: https://bz.apache.org/bugzilla/show_bug.cgi?id=61292
Related
I am running performance test using JMeter 5.0 and then at the end of the test a HTML report is generated. I would like to know if there is anyway to view actual request with the endpoint it served what was the response and ability to view all requests in HTML report. I know JMeter has something called view result tree and it provides most of the information i need but i am not sure how to display result tree or something similar in HTML report.
View results is tree is rather resource intensive listener so you want to use it with caution. Typically you want to filter and display only errors so you can analyse them.
If you would like HTML reports then you can use a modular extension for JMeter. Its default behavior is to read and process samples from CSV files to generate HTML files containing graph views. It can generate the report at end of a load test or on demand.
You can generate the HTML report using menu item Tools → Generate HTML report:
This page explains different configs and options to tweak and tune the report generation.
Hope this helps.
As of current JMeter version 5.2
JMeter can only generate the HTML Reporting Dashboard from the CSV files
The dashboard generator is a modular extension of JMeter. Its default behavior is to read and process samples from CSV files to generate HTML files containing graph views. It can generate the report at end of a load test or on demand.
There is no possibility to store response data in CSV mode:
response_data is currently not supported for CSV output
So there is no easy way to implement your issue unless you use i.e. Flexible File Writer to store request/response data into a separate file and modify report-template (you will need to learn FreeMarker) to display the information you need.
use this command, -g .jtl file -o reportlocationpath, you will get nicely generated html report along lots of graphs as shown below:
I would like to know if there is anyway to view actual request with
the endpoint it served
My guess is that you already have all the metrics required for each request but would like to see individual request payload and the response. Just in case this guess is wrong, since you already have the report and are still not able to see the individual requests, I am assuming that you have checked on "Generate Parent Sample" across each Transaction Controller. Uncheck that and you should be able to see individual request level response time, 90th percentile etc. Of course you will have to re-run the test to get these individual request level values
However, if you want to literally see the request payload and response in HTML report, there is no way we can do that. Only View Results Tree can do that for you within the scope of JMeter's GUI and not outside of this.
what was the response
As mentioned above, as of today, there is no possibility to see the response of individual requests along with their payload in the HTML report and this is for a good reason. We do not require the requests/responses while we run a load test in our HTML report. It is necessary for debugging and until the script is ready for execution. But post that, we wouldn't require the responses. If you still want to ensure that the correct responses are received, you can always add assertions (although best practice is not to add too many assertions in a load test)
Just an example of why HTML report across any load testing tool will not show responses of a request -
Assume there is a load test of 10K users for 10 iterations and the Thread Group has around 10 transactions each having an average of 7 requests, there is going to be 7 million responses to go through. Having this in a HTML report is not just tedious but literally doesn't make sense.
I know JMeter has something called view result tree and it provides
most of the information i need
The View Results Tree is the holy grail for debugging and for ensuring if requests and response are appropriate during script creation stage.
I think your primary concern is to ensure the correctness of the responses rather than get the whole response of each request. In that case, as I had mentioned above, you can always use assertions. Several other custom alternatives include capturing specific parts of response and writing them to a file or something like that by leveraging JSR223 sampler or using the if controller to validate. Of course this is going to add some overhead on JMeter depending on the type of test and script that you are going to execute
but i am not sure how to display result tree or something similar in
HTML report
But if you still insist on having something in HTML and if I have failed to convince as to why this is something that one must not pursue, you can capture the entire response of each request in a variable through Regex Extractor and write them to a file with HTML tags in it. However, you will have to implement logic to collate responses and organize them for your need.
One more way would be to have the JTL save all data and modify this up in some text editor and beautify it with HTML
FYI on JTL and its configuration
Hope this helps!
I have a Load Test project that works perfectly and is able to saves all the perf counter in the LoadTest db as expected.
I wanted to add some specific scenario attributes information to my test run.
Therefore when I create a report in Excel at the end, ill be able to filter based on those attributes
Example:
Environment Attribute: (QA, PreProd, Production)
Target Attribute: (UI, API,..)
I searched everywhere but couldn't find the information. Not sure if I have to create a new table in that DB and populate it myself, or if there is another easier way.
The closest I have achieved to doing what you ask is by modifying the "reporting names" of requests to include the wanted information. On one test I added a test case that executed a small number of times (so few that the overall results would not be skewed). It had two requests. The first collected some data and saved it to a context parameter. The second had a PreRequest plugin that wrote the data to the ReportingName field of that request. (Experiments showed that setting the field in a PostRequest plugin had no effect.)
Another approach I have used is have a PostWebTest plugin that writes a line of useful data to a logging file, so I get one line per test case executed.
It may be possible to add another table to the load test database, but I would wonder whether it would be properly handled by the Open, Import, Export and Delete commands of the Open and Manage Test Results window.
I am using JMeter for load testing and using listeners for getting the response results but I am not sure which are the most commonly used listeners which will give data for analysis.
I know Table view and tree view but those basic ones, kindly advice which listeners should I use.
JMeter documentation provides a very good overview of the listeners and when/how to use them.
While you are debugging and developing your plan, there's nothing better than View Results Tree, which also serves as a tester for RegEx, CSS/JQuery and XPath tester. However this particular listener must be disabled or deleted during the real load test, as it will eventually crash JMeter with OOM exception.
During the real load test you need to record statistics (how long requests took, etc.) and errors. In non-interactive mode, the best is to use Simple Data Writer with CSV format, which is considered to be very efficient. If you use interactive mode, or both (interactive and non-interactive modes), it's very convenient to use Aggregate Report or
Summary Report, since they display stats right away, and you can see immediately if something goes wrong. They also have ability to write to file, just like Simple Data Writer.
Finally, if you want to include some custom result collecting (not provided by any listeners), you can use BeanShell Listener or BSF Listener
In terms of organization, I find it convenient to separate successes and failures, so I always have 2 listeners:
For successes (in "Log/display only" option Successes is checked) I either record only statistics using Aggregate/Summary report (if test will run interactively and for a long time) or record file in CSV format (if I need a raw data about each request).
I always record failures (in "Log/display only" option Errors is checked) into file in XML format (for example using Simple Data Writer). XML format is not that efficient, but test is not supposed to have many failures (if it does, basically it should be stopped and analyzed). But XML format allows to record failing request and response headers and body, which is convenient for further debugging.
Hope this helps.
While executing the test better to avoid adding the listeners, only thing you can add simple data writer alone from that listeners you can generate any type listeners as you need.
while making the script ready you can use any type of listeners that will no be any issues.
We are working on a site in AEM 6.1 which has news and events content with most pages having info on recent and related news/events based on tagging that are dynamic. We are using dispatcher. Please suggest on some caching techniques that could be implemented at application level apart from the dispatcher. Thanks.
Aim of implementing the caching on dispatcher is to allow less hits on your app server and serve as much as possible from web server. In short improving response time from your app server. But in some cases we can't cache too much on web server if results change on app server frequently.
On app server we can have following solutions implemented to get results quickly on top of having dispatcher in place.
Make sure your content hierarchy where you are ingesting news items have as less number of article as possible. Divide your hierarchy based on following structure. Year >> Month >> Day >> Hour (this can be ignored if content flow is less) >> news items.
Having this structure in place, write path based query so that you don't have to traverse in whole content hierarchy.
There is a concept of transient node in CQ, for each news item which is getting created in CQ, update the transient node with newly created item. Means for recent news you don't have to traverse content structure just refer to transient node which has reference to newly created news item.
You could also write a cron job which gets executed in background and takes care of collating views namely top recent news.
To complement the answer of Rupesh I would say that definitely use dispatcher cache as much as you can and for using local caching strategies in AEM try using guava cache it is a very good and easy to use tool there is also a lot of information on how you can set it up and use it for your specific needs. Hope it helps.
I would suggest the following:
For recent news/events, write a scheduler
(https://sling.apache.org/documentation/bundles/scheduler-service-commons-scheduler.html) that will compute the list of recent news/events and write it to a specific node as properties, example:
/tmp/recent
news [/path/to/news1,/path/to/news2]
events [/path/to/event1,/path/to/event2]
Most recent always at the end of the array. Your code need to limit it to the amount of max recent you want to have.
Let's say you want to have the last 5 changed and a 6th page is changed, then you just pop and push(new_page_path)
This could run once a day or at the frequency which you feel is the best depending on your requirements.
If you need instant update, then you can additionally write a Listener when a page is changed/deleted and update the recent list. In this case I would suggest putting the code that deal with updating the recent list into a service and use that service in both the scheduler and the listener.
Listener and scheduler need to run on both author and publisher and on publisher trigger dispatcher cache invalidation for /tmp/recent afterwards.
In order to render the recent list without having to invalidate the whole pages, I would suggest you use SSI for that, that means have a component in your page that will render an SSI include to /tmp/recent.news.html or /tmp/recent.events.html depending on whether you want to render recent news or events.
Give the node /tmp/recent the resourceType for handling the "news" and "events" selector and implement that resourceType to render the content.
For the related
Use the Tag Manager (https://docs.adobe.com/docs/en/cq/5-6-1/javadoc/com/day/cq/tagging/TagManager.html) "find" method to lookup for all news/events having the same tag as the current page. I assume your news and events pages have a dedicated template or resource type.
Also I would suggest having a dedicated component that would include that content using SSI include. Let's say your page has 2 tags, ns/tag1 and ns/tag2, then you could perform the SSI include like this:
SSI include /etc/tags/ns/tag1.related_news.html
SSI include /etc/tags/ns/tag1.related_events.html
SSI include /etc/tags/ns/tag2.related_news.html
SSI include /etc/tags/ns/tag2.related_events.html
depending on what you want to include
Write a component under /apps/cq/tagging/components/tag (sling:resourceSuperType= /libs/cq/tagging/components/tag) that will provide the rendering for the "related_news" and "related_events" selector and list all related pages.
The advantage with this approach is that you can share the related page for each tag and whenever the tag is changed/deleted then the cache gets invalidated automatically.
In both cases (recent and related) configure the dispatcher to cache the output.
I am using Jmeter to test a web page. Obviously it has css, images, js etc. How can I group the response times by css,js and images so that I can clearly see the response time broken by resource types.
You pose an interesting question. The listeners in JMeter show what's at the sibling/child level. Therefore, instinctively, if you want just one type of resource in a given report, you'd need to put all the requests in a single controller and give that controller a listener.
You can get around restructuring your test by renaming your samples in a consistent manner. Something like: "CSS - actual request name".
Then, using Aggregate Report, you can copy the results into a spreadsheet, sort by name, and get your metrics that way.
I think you can use parallel controller to group the type of static files, ajax, etc. And use a transaction controller to group the page you would like to request, under the transaction controller you can add the parallel controller as hierarchy.