In my thread I get active_status parameter from Cassandra DataBase. I need to run on each active_status var been created from Cassandra sample (active_status_1, active_status_2, etc..) and check if it's active.
When not active it should skip and do nothing and if it's active I need to do an action only once and then stop (so even if I have 10 active results, it will perform the action ("report" in my case) only on the first one).
active_status vars from debug response
media_id vars from debug response
http request where I use the counter
My action (report) request is: story_id=evE0-001-085121182-5&media_id=1021372295263777595_175887011&who=admin&reason=offensive&blogger_id=Amit even though "1021372295263777595_175887011" is not ACTIVE
Use :
User Parameters PreProcessor once as a child of a Sampler before the ForEach Controller to init RUN_ONCE to false:
and once as a child of the HTTP Request that runs on active status to set RUN_ONCE to true
ForEach Controller with following settings:
If Controller and put in condition:
${__jexl2( "${status}" == "ACTIVE" && "${RUN_ONCE}" == "false")}
- Test Plan would look like this:
You counter should be in ForEachController not in If Controller as it will not increment correctly and give the ID for the correct status
You can use 'while controller' with a condition:
${__javaScript( "${some_var}" = "something" && "${some_other_var}" < "${something_else}"; )}
One of those expressions should refer to counter, another to status.
Related
I am new to JMeter,
In my test I am creating a JDBC Connection to oracle DB and running a query which fetching me the count of records, which I want to validate must be equal to the SAMPLE-NUMBER (which is a defined variable in the user defined variable).
SELECT COUNT(*) FROM event_log WHERE audit_context_key LIKE '288017ec-0dcf-4fd5-9565-e8ad15e65cd2' AND event_desc = 'Success'
Response Body:
COUNT(*)
2
UserDefined Variable
You can do this, defined the variable name in JDBC request,
for example, TOTALCOUNT and add a JSR223 Assertion with the following code,
1.upto(vars.get('TOTALCOUNT_#') as int, {
if (vars.get('TOTALCOUNT_' + it) == '${__groovy(vars.get('SAMPLE-NUMBER'),)}') {
AssertionResult.setFailure(false);
}
})
Response Assertion can do the trick for you:
In the JDBC Request define "Variable Names", i.e. ACTUAL_COUNT
Once done you can compare the ACTUAL_COUNT variable value with the SAMPLE-NUMBER variable like:
I am implemented airflow , can we add data sanity logic. suppose I have Task1 which do the following task
1.Read the data from the data source--RAW DATA.
2. do join with dimensional table to get the some relation detail product name etc.
3. Store output file some location after step 2.
There is a task2 that stored the output file into database. but before task2 execution i need to some data validation like count of RAW DATA should equal to the store output file count i.e after joining
like count(raw_data) = count(raw_data_join_with_dimensional) , if it is true then trigger the Task2 else send the alert and failed the job.
For that use case a possible workflow could be:
check_op = SQLCheckOperator(
task_id='check_task',
sql='YOUR VALIDATION SQL',
conn_id='YOUR CONN',
)
t2_op = YourNextOperator()
failure_op = EmailOperator(subject='check has failed', to='YOUR EMAIL', trigger_rule='one_failed')
check_op >> [t2_op, failure_op]
It works as follows:
SQLCheckOperator runs the query against the DB. If query returns False the check has failed thus the operator will be in Failure state. If the query returns value the query consider as success thus the operator will be in Success state.
EmailOperator will be triggered if SQLCheckOperator status is failure otherwise YourNextOperator will be triggered.
Edit: Go see #Elad's answer, there's a much more specific operator for this task.
The airflow.sensors.sql_sensor.SqlSensor can be used to build a task that can check data quality:
from airflow.sensors import sql_sensor
...
check_data_task = sql_sensor.SqlSensor(
task_id="check_data",
conn_id="YourConnectionIdentifier",
sql="SELECT CASE WHEN data_is_valid THEN 1 ELSE 0 END ...",
timeout=0
)
The key being that your sql argument return "at least one cell that contains a non-zero / empty string value" -- according to the documentation. The timeout=0 means that it will check once and fail if your data check query doesn't "pass"
getproperty values passed from Thread Group 1 to Thread group2
Result from BeanShell assertion
Step 1- USing jdbc request to get data from database with 2 columns and multiple rows.
Step 2 - From ThreadGroup 1, Set property to the database results using ${__setProperty(StateCodeProperty,${stateDetails})};
Step 3 - Access in Thread Group 2 by get property using beanshell assertion- String result = (vars.get("${__property(StateCodeProperty)}")); I need help on how to separate the columns and use it in api call. –
In any case if you want to access the DB results in different Thread group then you can try to do something like this inside beanshell assertion (not sure though) -
ArrayList results = ${__property(StateCodeProperty)}; //it should return the object as an arraylist
for (int i; i < results.size(); i++) {
if (results.get(i).get("statecode").equals("NY")) { //iterating the results, 'statecode' is the name of your 1st column, similarly you can do for 'State'
//Do your comparisons or whatever you like here
}
}
I have a Dataset, obtained from a DataBase query, of about 5,000 elements. I would like to divide this data into chunks and then have the 'users' (threads) make a HTTP request.
The purpose of this is we have a site that gives realtime information on transient data, I want to simulate multiple concurrent requests against the service.
1 - Tried to create a test plan where the DB query was done and then processed via a HTTP request via a ForEach controller. This works fine when I have only 1 'user', however; if I increase the user count to 2+ then the DB query is run 2+ times and each 'user' runs through the entire 5,000+ data points
2 - I tried moving the DB query into it's own Thread Group and then using BeanShell to put the data into the environment (props.add(...)). This worked in that the data was there but again each 'user' in the http request Thread Group iterated all the data.
Ideally what I would like is to take the data, and have the HTTP Request Thread Group divide the data so that Thread 1 takes the first 2,500 and that Thread 2 takes the second 2,500 (or if there are 4 'users' then thread 1 takes the first 1,250, thread 2 the next 1,250 and so on).
I just started looking at JMeter and I don't think it can do this "automatically" but I wanted to ask in case I'm missing something obvious.
Put a Counter element to testplan with:
Starting value: 1
Increment: 1
Reference name: (for example) cid
and disabled "Track counter independently ...".
Then add JSR223 or BeanShell sampler and write a simple code:
Integer cid = Integer.valueOf(vars.get("cid"));
Integer dataShift = 2500;
Integer startReadDataFrom = (cid - 1) * 2500;
vars.put('startReadDataFrom', String.valueOf(startReadDataFrom));
Then you can use variable ${startReadDataFrom} as a starting point to read data for every thread (0, 2500, 5000, 7500, ...).
The fastest and the easiest way is to store the data from the database into a CSV file, once done you should be able to use CSV Data Set Config and its Sharing Mode feature according to your requirements.
The storing of the data could be done as follows:
Define Result variable name in your JDBC Request Sampler:
Add JSR223 PostProcessor as a child of the JDBC Request sampler
Put the following code into "Script" area:
resultSet = vars.getObject("resultSet")
result = new StringBuilder()
for (Object row : resultSet ) {
iter = row.entrySet().iterator()
while (iter.hasNext()) {
pair = iter.next()
result.append(pair.getValue())
result.append(",")
}
result.append(System.getProperty("line.separator"))
}
org.apache.commons.io.FileUtils.writeStringToFile(new File("data.csv"), result.toString(), "UTF-8")
Once execution will be finished you should see data.csv file in "bin" folder of your JMeter installation containing the data from the database
I have some functional tests created via JMeter. It is pretty huge but i can't handle one simple check.
I generate properties using BSF pre processor with help of JS. Parameter (lets call it "payment_fee") should be generated only if other parameter (lets call it "role") has a value = 1 .In this case we post pre generated integer into payment_fee and everything works well. But if role =2 then we should post nothing into payment_fee.
The problem is, i don't know how to say to JMeter: In case if role = 1 use variable with pre generated payment_fee but if role = 2, you shouldn't use this variable so just post an empty value for payment_fee . Server waits for an integer so empty string or NULL had been rejected.
For more clarification:
I will try to explain more clear.
Here is a part of my code
var role = Math.floor(Math.random()*3+1)
var paymentType = ["creditcard","cash"]
var randomPay = installerType[Math.floor(Math.random()*installerType.length)];
var payment = "";
var paymentFee;
if (role == 1){
payment+=randomPay,
paymentFee = Math.floor((Math.random() * 999) + 1) / 10.00
}
vars.put("role", role);
vars.put("payment", payment);
vars.put("paymentFee", paymentFee);
And if role == 1 i should post paymentFee value. Like this - http://prntscr.com/b50kk1 BUT! if role == 2 || role == 3 I should remove this value, so it should be like this http://prnt.sc/b50l82
I don't fully understand what you're trying to do as your 2 statements clash:
But if role =2 then we should post nothing into payment_fee
Server waits for an integer so empty string or NULL had been rejected
You should know few bits about JMeter properties:
Properties are global for the whole JVM. Once you set property it will "live" until you exit JMeter.
Properties can be accesses by all threads of all Thread Groups.
So it might be the case when you define property earlier or by another thread and expect it to be not set later on.
Also BSF PreProcessor and JavaScript isn't the best combination from performance perspective, consider switching to JSR223 PreProcessor and Groovy language.