In JMeter i need to perform a large search and count the number of rows which are returned. Max rows are 50000.
The number of rows which are returned are shown on the website after a search. "Number of returned rows: xx".
Or I can count the rows inside the HTTP response.
I have tried to use a regex post-processer to count the amount of rows which are returned, the problem is that JMeter freezes since the http-response is so large.
I have also tried to extract the text directly from the website unsuccesfully. I guess one cant do that since the information is not in the HTTP-response?
--So:
Is there some faster and less demanding way to counter all the returned rows inside a HTTP-response body?
Or is there some way to get the text directly from the website?
Thank you.
It looks like your application is buggy, I don't think that returning 50000 entries in a single shot is something people should be doing as there is creates extra network traffic and consumes a lot of resources on server and client(browser) side. I would rather expect some form of Pagination when it comes to operating large amounts of data.
If you're totally sure that your application works as expected you can try using Boundary Extractor which is available since JMeter 4.0
Due to the specifics of internal implementation it consumes less resources and acts faster than the Regular Expression Extractor therefore the load you will be able to conduct from a single machine will be higher.
Check out The Boundary Extractor vs. the Regular Expression Extractor in JMeter article for more information
yes you can get that count in matchNr which is coming after search string. use Regular expression to match any name or id,
do match No. -1
ex. regex variable name is totalcount so then you can fetch that count by using ${totalcount_matchNr}
Related
Anyone know how I can retrieve the total number of matches, by match type (queue) for a specific summoner? Some sites like wol.gg claim they can calculate your lifetime match history, but I don't see how the Riot API supports that with API request limits, 7 day range limits on match history, etc.
Any insight would be helpful, thanks!
To get all of the available matches for a user, you need to call the Matches API in a while loop, incrementing the starting index each time until there are no new matches. I have some sample code you can look at in one of my past projects where I do exactly this, as well as caching the results in a database. It's too much to type out here, but you can see the logic for some of Riot's historical API:
https://github.com/ErikOverflow/Graphs-GG-Server/blob/master/services/matches.js
you can get all match history in this link https://matchhistory.euw.leagueoflegends.com/en/#match-history/EUW1/211103650
I checked the request send when you want more match history and it seem that riot ask
https://acs.leagueoflegends.com/v1/stats/player_history/EUW1/211103650?begIndex=135&endIndex=150&
with begindex and endindex param
I perform a bing API search for webpages and the query cameras.
The first "page" of results (offset=0, count=50) returns 49 actual results. It also returns a totalEstimatedMatches of 114000000 -- 114 million. Neat, that's a lot of results.
The second "page" of results (offset=49, count=50) performs similarly...
...until I reach page 7 (offset=314, count=50). Suddenly totalEstimatedMatches is 544.
And the actual count of results returned per-page trails off precipitously from there. In fact, over 43 "pages" of results, I get 413 actual results, of which only 311 have unique URLs.
This appears to happen for any query after a small number of pages.
Is this expected behavior? There's no hint from the API documentation that exhaustive pagination should lead to this behavior... but there you have it.
Here's a screenshot:
Each time the API is called, the search API obtains a group of possible matches starting at in the result set, and then filters out the results based on different parameters (e.g spam, duplicates, safesearch setting, etc), finally leaving a final result set. If the final result after filtering and optimization is more than the count parameter then the number of results equal to count would be returned. If the parameter is more than the final result set count then the final result set is returned which will be less than the count parameter. If the search API is called again, passing in the offset parameter to get the next set of results, then the filtering process happens again on the next set of results which means it may also be less than count.
You should not expect the full count parameter number of results to always be returned for each API call. If further search results beyond the number returned are required then the query should be called again, passing in the offset parameter with a value equal to the number of results returned in the previous API call. This also means that when making subsequent API calls, the offset parameter should never be a hard coded value and should always be calculated based on the results of previous queries.
totalEstimatedMatches can also add to confusion around the Bing Search API results. The word ‘estimated’ is important because the number is an estimation based on an initial quick result set, prior to the filtering described above. Additionally, the totalEstimatedMatches value can change as you iterate through the result set by making subsequent API calls with increasing offset values. The totalEstimatedMatches should only be used as a rough guide indicating the magnitude of the possible result set, and it should not be used to determine the number of results that will ultimately be returned. To query all of the possible results you should continue making API calls, passing in offset with a value of the sum of the results returned in previous calls, until that sum is greater than totalEstimatedMatches of the most recent API call.
Note that you can see this same behavior by going to bing.com directly and using a query such as https://www.bing.com/search?q=bill+gates&count=50. Notice that you will get around 34 results with a totalEstimatedMatches of ~567,000 (valid as of June 2017, future searches may change), and if you click the 'next page' arrow you will see that the next query executed will start at the offset of the 34 returned in the first query (ie. https://www.bing.com/search?q=bill+gates&count=50&first=34). If you click ‘next’ several more times you may see the totalEstimatedMatches also change from page to page.
This seems to be expected behavior. The Web Search API is not a crawler API, thus it only delivers results, that the algorithms deem relevant for a human. Simply put, most humans won't skim through more than a few pages of results, furthermore they expect to find relevant results on the first page.
If you could retrieve the results in the millions, you could simply copy their search index and Bing would be out of business.
Search indices seem to be things of political and economic power, as far as I know there are only four relevant search indices world wide: from Google, from Microsoft (Bing), from Russia, and from China.
Those who control the search, control the Spice... ;-)
I need to simulate a test scenario where my application sends a request with 100s of queries. On the back-end, this request is broken down into requests containing a single query each. So a request from Jmeter with 100 queries will become 100 requests on the back-end. Now - the response from the back-end could either contain the requested data for each of those queries OR contain a unique queryID. Sending back a queryID is server's way of telling that this query is still running. For example, if Jmeter sends a request with 100 queries, it might get back data for 80 and 20 unique queryIDs. So my application under test makes a callback request with those 20 queryIDs every 15 seconds until it gets back the requested data or timeout.
Here is what I have implemented so far.
-main_request_with_100_queries
--XPath_extractor_to_extract_any_queryIDs_found
-if_controller_to_check_if_queryID_MatchNr_is_greater_than_0
--15_second_pause
--beanshell_preprocessor_to_create_the_request_body_with_all_queryIDs
--callback_request_with_queryIDs
What I want to implement is to have another XPath extractor for my callback_request and if any queryIDs are found, then go back to the if_controller
I'm trying to make this work by using a module_controller but so far no luck. Has anyone ever implemented something like this? Can anyone suggest some ideas?
You can use While Controller to keep making the request until there is a queryID in the response.
While Controller [ "${querid.present}"=="true" ]
HTTP Request
Pre Processor [to_create_the_request_body_with_all_queryIDs]
Post Processor [to check for query ID. if no query id - change querid.present to false ]
If possible, try to use Regular Expression Extractor. xpath is very slow and might affect your performance of the script. Check here for more details.
Creating modular test script in JMeter.
I want to correlate this 181-418-5889 in the following statement: regSend&transferNumber=181-418-5889".
I used the regular web_reg_save_param: But it failed... any suggestion?
You are using the statement in the wrong location, such as using it just before the request is sent containing the correlated value versus just before the location where the response containing the value is sent to the client
You are not receiving the correct page response and as a result you may not be able to collect the value. The page may be an HTTP 200 page but the content could be completely off. Always check for an appropriate expected result
Your left boundary, right boundary and other parameters are incorrect to collect the value you need
You have not been through training and you are being forced by your management to learn this tool via trial and error
1- I am not using the statement in the wrong location since I did find the needed value I want to correlate via the Tree function and put it just before the statement that hold this value
2- The Page is not an HTTP 200
3- The Left and right boundary are correct since I checked the text if it does exist twice in the response body.
4- I know the tool (Loadrunner) but in fact, the application is developed under ZK platform and I am not sure if ZK and Loadrunner are compatible knowing that I did implement the dtid function in my script to have a static desktop id each time I replay the process.
I am wondering how the each command on a Parse Query counts towards the request execution limits. I am building an app that will need to perform a function on many objects (could be more than 1000) in a parse class.
For example (in JavaScript),
var query = new Parse.Query(Parse.User);
query.equalTo('anObjectIWant',true); //there could be more than 1000 objects I want
query.each(function(object){
doSomething(object); //doSomething does NOT involve another Parse request
});
So, will the above code count as 1 request towards my Parse application execution limit (you get 30/second free), or will each object (each recurrence of calling "each") use one request (so 1000 objects would be 1000 requests)?
I have evaluated the resource usage by observing the number of API requests made by query.each() for different result set sizes. The bottom line is that (at the moment of writing) this function is using the default query result count limit of 100. Thus if your query matches up to 100 results it will make 1 API request, 2 API requests for 101-200 and so forth.
This behavior can not be changed by manually increasing the limit to the maximum using query.limit(1000). If you do this you will get an error when you call query.each() afterwards (this is also mentioned in the documentation).
Therefore it has to be considered to manually implement this functionality (e.g., by recursive query.find()) which allows you to set the query limit to 1000 and thus, in the best case, only consumes one-tenth of the API requests query.each() would consume.
This would count as 1 or 2 depending on :
If it is run from cloudcode function =2,when 1 is for cloudcode call + 1 for query. Since queries get their results all at once it is single call.
If this should be place within "beforeSave" functions or similar then only query would be counted, 1 API call.
So you should be pretty fine as long as you don't trigger another parse API for each result.
I would not be surprised if the .each method would query the server each iteration.
You can actually check this using their "control panel", just look at the amount of requests beeing made.
We left Parse after doing some prototyping, one of the reasons was that while using proper and sugested code from the parse website, I managed to create 6500 requests a day beeing the only one using the app.
Using our own API, we are down to not more than 100.