Can I use variables across all the threads in the thread groups in jmeter? - jmeter

I'm trying to create a test plan for rate-limiting behavior.
I set a rule that blocks after X requests per minute, and I want to check that I get response code 200 until I reached the X requests, and from then, to get 429. I created a counter that shared between all the threads, but it seems to be a mess because it's not a thread-safe.
This is my beanshell "once only controller":
String props_pre_fix = ${section_id} + "-" + ${START.HMS};
props.remove("props_pre_fix" + ${section_id}, props_pre_fix);
props.put("props_pre_fix" + ${section_id}, props_pre_fix);
props.put(props_pre_fix + "_last_response_code", "200");
props.put(props_pre_fix + "_my_counter", "0");
and this is the beanshell assertion:
String props_pre_fix = props.get("props_pre_fix" + ${section_id});
//log.info("props_pre_fix " + props_pre_fix);
//extract my counter from props
int my_counter = Integer.parseInt(props.get(props_pre_fix + "_my_counter"));
//extract last response code
String last_response_code = props.get(props_pre_fix + "_last_response_code");
log.info("last_response_code " + last_response_code);
//if last seconds is greater than current seconds it means we are in a new minute - set counter to zero
if(last_response_code.equals("429") && ResponseCode.equals("200")){
log.info("we moved to a new minute - my_counter should be zero");
my_counter = 0;
}
//increase counter
my_counter++;
log.info("set counter with value: " + my_counter);
//save counter
props.put(props_pre_fix + "_my_counter", my_counter + "");
log.info("counter has set with value: " + my_counter);
if (ResponseCode.equals("200")) {
props.put(props_pre_fix + "_last_response_code", "200");
if(my_counter <= ${current_limit}){
Failure = false;
}
else {
Failure = true;
FailureMessage = "leakage of " + (my_counter - ${current_limit}) + " requests";
}
}
else if (ResponseCode.equals("429")) {
props.put(props_pre_fix + "_last_response_code", "429");
if(my_counter > ${current_limit}){
Failure = false;
}
}
I'm using props to share the counter, but I obviously feel that this is not the right way to do it.
Can you suggest me how to do that?

I don't think that it is possible to automatically test this requirement using JMeter Assertions because you don't have access to the current throughput so I would rather recommend considering cross-checking Response Codes per Second and Transactions per Second charts (can be installed using JMeter Plugins Manager)
All the 200 and 429 responses can be marked as successful using Response Assertion configured like:
If for some reason you still want to do this programmatically you might want to take a look at Summariser class source which is used for displaying current throughput in the STDOUT.
Also be informed that starting from JMeter 3.1 you should be using JSR223 Test Elements and Groovy language for scripting.

Related

Keep original order of requests in JMeter reports

I use following code to label requests by response times.
if (prev.getTime() > 170 && prev.getTime() < 340) {
prev.setSampleLabel(prev.getSampleLabel() + " > 170")
} else if (prev.getTime() > 340 && prev.getTime() < 4000) {
prev.setSampleLabel(prev.getSampleLabel() + " > 340")
} else if (prev.getTime() > 4000 && prev.getTime() < 8000) {
prev.setSampleLabel(prev.getSampleLabel() + " > 4000")
} else if (prev.getTime() > 8000) {
prev.setSampleLabel(prev.getSampleLabel() + " > 8000")
}
Aggregate report and Summary report contain name of requests in a different order than original one in Thread group. Total number of samples per request is not visible in this way.
JMeter's Aggregate Report and Summary Report listeners always store the Sample Results in their execution order and JMeter executes Samplers upside down (or according to the Logic Controllers)
As you can see, Sampler 4 is the first one because it has been executed first, however it is possible to sort requests by label by clicking the column header since JMeter 3.2):
Another option to sort requests by label is generating HTML Reporting Dashboard

Why the TiDB performance drop for 10 times when the updated field value is random?

I set up the TiDB, TiKV and PD cluster in order to benchmark them with YCSB tool, connected by the MySQL driver.
The cluster consists of 5 instances for each of TiDB, TiKV and PD.
Each node run a single TiDB, TiKV and PD instance.
However, when I play around the YCSB code in the update statement, I notice that if the value of the updated field is fixed and hardcoded, the total throughput is ~30K tps and the latency at ~30ms. If the updated field value is random, the total throughput is ~2k tps and the latency is around ~300ms.
The update statement creation code is as follow:
#Override
public String createUpdateStatement(StatementType updateType) {
String[] fieldKeys = updateType.getFieldString().split(",");
StringBuilder update = new StringBuilder("UPDATE ");
update.append(updateType.getTableName());
update.append(" SET ");
for (int i = 0; i < fieldKeys.length; i++) {
update.append(fieldKeys[i]);
String randStr = RandomCharStr(); // 1) 3K tps with 300ms latency
//String randStr = "Hardcode-Field-Value"; // 2) 20K tps with 20ms latency
update.append(" = '" + randStr + "'");
if (i < fieldKeys.length - 1) {
update.append(", ");
}
}
// update.append(fieldKey);
update.append(" WHERE ");
update.append(JdbcDBClient.PRIMARY_KEY);
update.append(" = ?");
return update.toString();
}
How do we account for this performance gap?
Is it due to the DistSQL query cache, as discussed in this post?
I manage to figure this out from this post (Same transaction returns different results when i ran multiply times) and pr (https://github.com/pingcap/tidb/issues/7644).
It is because TiDB will not perform the txn if the updated field is identical to the previous value.

Servicenow script infinite recursion

I have the following ServiceNow script which inserts the record into live_message table.
(function executeRule(current, previous/*null when async*/) {
var requestBody;
var responseBody;
var status;
var request;
var response;
try {
request = new sn_ws.RESTMessageV2("LiveMessageWebhook", "post");
request.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
var parameters = "&chat_message=" + current.chat_message.toString();
parameters = parameters + "&context=" + current.context.toString();
parameters = parameters + "&formatted_message=" + current.formatted_message.toString();
parameters = parameters + "&has_attachments=" + current.has_attachments.toString();
parameters = parameters + "&has_links=" + current.has_links.toString();
parameters = parameters + "&has_tags=" + current.has_tags.toString();
parameters = parameters + "&ID=" + current.id.toString();
parameters = parameters + "&in_reply_to=" + current.in_reply_to.toString();
parameters = parameters + "&isLiked=" + current.is_liked.toString();
parameters = parameters + "&lastActivity=" + current.last_activity.toString();
parameters = parameters + "&lastMessage=" + current.last_message.toString();
parameters = parameters + "&likeCount=" + current.like_count.toString();
parameters = parameters + "&message=" + current.message.toString();
parameters = parameters + "&next_reply_order_chunk=" + current.next_reply_order_chunk.toString();
parameters = parameters + "&order=" + current.order.toString();
parameters = parameters + "&poll=" + current.poll.toString();
parameters = parameters + "&private=" + current.private_message.toString();
parameters = parameters + "&profile=" + current.profile.toString();
parameters = parameters + "&reflected_field=" + current.reflected_field.toString();
parameters = parameters + "&reply_to=" + current.reply_to.toString();
parameters = parameters + "&state=" + current.state.toString();
parameters = parameters + "&group_type=" + current.sys_class_name.toString();
parameters = parameters + "&created_by=" + current.sys_created_by.toString();
parameters = parameters + "&created_on=" + current.sys_created_on.toString();
parameters = parameters + "&domain=" + current.sys_domain.toString();
parameters = parameters + "&domain_path=" + current.sys_domain_path.toString();
parameters = parameters + "&sys_ID=" + current.sys_id.toString();
parameters = parameters + "&to_profile=" + current.sys_domain.toString();
parameters = parameters + "&updates=" + current.sys_mod_count.toString();
parameters = parameters + "&updated_by=" + current.sys_updated_by.toString();
parameters = parameters + "&updated_on=" + current.sys_updated_on.toString();
request.setRequestBody(encodeURI(parameters));
var l = request.getRequestBody();
response = request.execute();
responseBody = response.haveError()
? response.getErrorMessage()
: response.getBody();
status = response.getStatusCode();
{
var gr = new GlideRecord('live_message');
gr.initialize();
gr.chat_message = current.chat_message;
gr.context = current.context.toString();
gr.formatted_message = "abc";
gr.group = current.group;
gr.has_attachments = current.has_attachments;
gr.has_links = current.has_links;
gr.has_tags = current.has_tags;
gr.id = current.id;
gr.in_reply_to = "admin";
gr.is_liked = current.is_liked;
gr.last_activity = current.last_activity;
gr.last_message = current.last_message;
gr.like_count = current.like_count;
gr.message = "abc";
gr.next_reply_order_chunk = current.next_reply_order_chunk;
gr.order = current.order;
gr.poll = current.poll;
gr.private_message = current.private_message;
gr.profile = current.profile;
gr.reflected_field = current.reflected_field;
gr.reply_to = current.reply_to;
gr.state = current.state;
gr.sys_class_name = current.sys_class_name;
gr.sys_created_by = current.sys_created_by;
gr.sys_created_on = current.sys_created_on;
gr.sys_domain = current.sys_domain;
gr.sys_domain_path = current.sys_domain_path;
gr.sys_mod_count = current.sys_mod_count;
gr.sys_updated_by = current.sys_updated_by;
gr.sys_updated_on = current.sys_updated_on;
gr.to_profile = current.to_profile;
gr.insertWithReferences();
}
} catch (ex) {
responseBody = 'Exception: ' + ex;
status = '900';
requestBody = request
? request.getRequestBody()
: null;
} finally {
gs.info("Final: Request Body: " + requestBody);
gs.info("Final: Response: " + responseBody);
gs.info("Final: HTTP Status: " + status);
gs.addInfoMessage('Final: Finished');
}
})(current, previous);
But after inserting the record it again goes back to the try block again and does the whole thing again and now with the inserted record
How is this business rule configured?
Is it possible that the changes made by this script, could also trigger the script to run?
Here's an example of what I mean:
Imagine I have a business rule that executes whenever a record in the table "u_arbitrary_counter" is updated.
Imagine that the function of this business rule is that whenever such a record is updated, we increase the value of the "u_counter" field by one, like so:
current.setValue('u_counter', parseInt(current.getValue('u_counter')) + 1);
current.update();
By using .update(), I'm forcing an update to the database, EVEN if this is a "before" business rule.
Pro Tip: Note that "before" business rules run on the data that's about to be
saved to the database, BEFORE the actual database operation has taken
place. Any changes to the "current" object in a "before" business rule
will be saved even without using current.update, because you're
modifying the data that's about to be sent to the database anyway, in
the natural course of this operation.
So using current.update() in a BEFORE business rule isn't a great idea. For the same reason though, performing any other operation which necessarily leads to a database update which could trigger this same business rule is a bad idea. It doesn't have to be current.update() -- Instead, imagine if rather than updating the current record, I did something like...
var gr = new GlideRecord('u_arbitrary_counter'); //the same table this BR is running on
gr.initialize();
gr.setValue('u_counter', parseInt(current.getValue('u_counter')) + 1);
gr.insert(); //This triggers a database action!
This is no good for the same reason.
Ditto for any REST calls which would trigger a DB action on this table which may trigger this business rule.
The solution
If this is indeed the cause of your issue (and without knowing the configuration of your business rule, I can't be sure if it is or not), there is actually a way to tell the system not to run any further business rules as a result of the operation your script performs. Right before your gr.insertWithReferences(); line (line 91), IF that block of code is the issue (which I'm fairly confident it is), add:
gr.setWorkflow(false);
This prevents business rules from running.
Unfortunately, since you're placing the record into the live_message table, that may not be a good idea either, since the messenger may require some business rule to propagate or present that information once inserted; I'm not sure.
If that is the case, I would recommend adding a condition to your business rule so that it only runs under certain circumstances, and then craft your inserted record to not meet those criteria.
Pro Tip: PS - I notice you access a table field using current.field_name.toString() in some places, and directly access
current.field_name in others. While the former is acceptable (not
that I recommend it in most cases), the latter should almost never be
used -- except in the case of journal fields. Instead, I strongly
recommend using the appropriate getter/setter methods, to avoid
pass-by-reference issues and confusion, and to ensure you're not
relying on JavaScript's coersion, which doesn't always play nice with
Mozilla Rhino, which ServiceNow runs on the back-end. Example issue
here. For more info, please see my article on using getValue()
and setValue() in ServiceNow.

how to record the arrival time of an event in esper

I want to record a data processing time in esper and I choose Bollinger Band as example. In Bollinger Band, there is called Moving Average (MA). that MA obtained from the result of calculate the stock price average. In this case, I set win:length(20). So, the MA can be obtained from the result of calculate the stock price average from 20 events that exists in the data window view. The following is code that i created.
public class BollingerBand {
static double startTime, finishTime;
public static void main (String [] args){
Configuration configuration = new Configuration();
configuration.addEventType("Stock", Stock.class);
EPServiceProvider epService = EPServiceProviderManager.getDefaultProvider(configuration);
AdapterInputSource source = new AdapterInputSource("BollingerBand.csv");
EPStatement statement = epService.getEPAdministrator().createEPL("insert into Aggregation " +
"select prevcount(symbol), symbol, avg(price) as SimpleMovingAverage, stddev(price) as StandardDeviation, " +
"last(price) as price, last(timestamp) as date from Stock.std:groupwin(symbol).win:length(20)" +
" group by symbol having count(*) >=20");
statement.addListener(new UpdateListener() {
public void update(EventBean[] newEvents, EventBean[] oldEvents) {
// TODO Auto-generated method stub
//System.out.println("Event Receive : "+newEvents[0].getUnderlying());
startTime = System.currentTimeMillis();
System.out.println("\nStart time : " + startTime + " miliseconds\n");
}
});
EPStatement statement2 = epService.getEPAdministrator().createEPL("select symbol, " +
"SimpleMovingAverage + 2*StandardDeviation as UpperBand," +
"SimpleMovingAverage as MiddleBand," +
"SimpleMovingAverage - 2*StandardDeviation as LowerBand," +
"price," +
"4*StandardDeviation/SimpleMovingAverage as Bandwidth," +
"(price - (SimpleMovingAverage - (2 * StandardDeviation))) / ((SimpleMovingAverage + " +
"(2 * StandardDeviation)) - (SimpleMovingAverage - (2 * StandardDeviation))) as PercentB," +
"date from Aggregation");
statement2.addListener(new UpdateListener() {
public void update(EventBean[] newEvents, EventBean[] oldEvents) {
// TODO Auto-generated method stub
//System.out.println("Event Receive : "+newEvents[0].getUnderlying());
finishTime = System.currentTimeMillis();
System.out.println("Start time : " + startTime + " miliseconds");
System.out.println("Finish time : " + finishTime + " miliseconds");
System.out.println("Processing time : " + (finishTime-startTime) + " miliseconds");
}
});
(new CSVInputAdapter(epService, source, "Stock")).start();
}
}
From the above code, time will be recorded if the average has calculated. But what i need is I want the time is recorded when the 20th event and next event enter to data window view. It's as the start time and finish time obtained from bollinger band calculation result. My question is how to record time of the 20th event and in the same time next event enter to the window view data. please help
The CSV adapter doesn't provide a callback when events are sent. You could easily change its code however. Or you could use a different CSV reader and send the events via runtime API.
Maybe have some sort of TickCounter in which there's a map that takes a key value pair of (item_count and timestamp). You update this in your second UpdateListener and of course you can always lookup the item of key 20.
By the way, I've used your Bollinger Band Calculation but using Storm and the EsperBolt. Blogged about it here:
http://chanchal.wordpress.com/2014/07/08/using-esperbolt-and-storm-to-calculate-bollinger-bands/

Entity Framework SaveChanges() first call is very slow

I appreciate that this issue has been raised a couple of times before, but I can't find a definitive answer (maybe there isn't one!).
Anyway the title tells it all really. Create a new context, add a new entity, SaveChanges() takes 20 seconds. Add second entity in same context, SaveChanges() instant.
Any thoughts on this? :-)
============ UPDATE =============
I've created a very simple app running against my existing model to show the issue...
public void Go()
{
ModelContainer context = new ModelContainer(DbHelper.GenerateConnectionString());
for (int i = 1; i <= 5; i++)
{
DateTime start = DateTime.Now;
Order order = context.Orders.Single(c => c.Reference == "AA05056");
DateTime end = DateTime.Now;
double millisecs = (end - start).TotalMilliseconds;
Console.WriteLine("Query " + i + " = " + millisecs + "ms (" + millisecs / 1000 + "s)");
start = DateTime.Now;
order.Note = start.ToLongTimeString();
context.SaveChanges();
end = DateTime.Now;
millisecs = (end - start).TotalMilliseconds;
Console.WriteLine("SaveChanges " + i + " = " + millisecs + "ms (" + millisecs / 1000 + "s)");
Thread.Sleep(1000);
}
Console.ReadKey();
}
Please do not comment on my code - unless it is an invalid test ;)
The results are:
Query 1 = 3999.2288ms (3.9992288s)
SaveChanges 1 = 3391.194ms (3.391194s)
Query 2 = 18.001ms (0.018001s)
SaveChanges 2 = 4.0002ms (0.0040002s)
Query 3 = 14.0008ms (0.0140008s)
SaveChanges 3 = 3.0002ms (0.0030002s)
Query 4 = 13.0008ms (0.0130008s)
SaveChanges 4 = 3.0002ms (0.0030002s)
Query 5 = 10.0005ms (0.0100005s)
SaveChanges 5 = 3.0002ms (0.0030002s)
The first query takes time which I assume is the view generation? Or db connection?
The first save takes nearly 4 seconds which for the more complex save in my app takes over 20 seconds which is not acceptable.
Not sure where to go with this now :-(
UPDATE...
SQL Profiler shows first query and update are fast and are not different for first. So I know delay is Entity Framework as suspected.
It might not be the SaveChanges call - the first time you make any call to the database in EF, it has to do some initial code generation from the metadata. You can pre-generate this though at compile-time: http://msdn.microsoft.com/en-us/library/bb896240.aspx
I would be surprised if that's the only problem, but it might help.
Also have a look here: http://msdn.microsoft.com/en-us/library/cc853327.aspx
I would run the following code on app start up and see how long it takes and if after that the first SaveChanges is fast.
public static void UpdateDatabase()
{
//Note: Using SetInitializer is reconnended by Ladislav Mrnka with reputation 275k
//http://stackoverflow.com/questions/9281423/entity-framework-4-3-run-migrations-at-application-start
Database.SetInitializer<DAL.MyDbContext>(
new MigrateDatabaseToLatestVersion<DAL.MyDbContext,
Migrations.MyDbContext.Configuration>());
using (var db = new DAL.MyDbContext()) {
db.Database.Initialize(false);//Execute the migrations now, not at the first access
}
}

Resources