Using JMeter-Bean shell sampler, I have extracted and split(with ',' delimiter) dynamic string. Currently I got stuck how this split values can be inserted to different columns of a database table.
Here is the code snippet which prints all the values after split. After splitting the string, each value will store into an array. You can retrieve by specifying the array position.
You can use the variable name e.g. aftersplit[0], aftersplit[1], and so on in the insert query.
String mystring = "here is my, dynamic, random, and unique string";
String[] aftersplit = mystring.split(",");
System.out.println(aftersplit[0]);
System.out.println(aftersplit[1]);
System.out.println(aftersplit[2]);
//To print all the values after splitting
for (int i=0; i < aftersplit.length; i++){
System.out.println(aftersplit[i]);
}
I would recommend the following approach: store your dynamic values into JMeter Variables having number postfix, example code:
String source = "foo,bar,baz";
int counter = 1;
for (String token : source.split(",")) {
vars.put("token_" + counter, token);
counter++;
}
It produces the following JMeter Variables:
token_1=foo
token_2=bar
token_3=baz
Then add ForEach Controller to iterate the generated variables and JDBC Request sampler as a child of the ForEach Controller to insert them into the database. See The Real Secret to Building a Database Test Plan With JMeter to learn how to establish database connection and execute arbitrary SQL queries using JMeter
Related
Is there a way to add each session Id that is retrieved from a Loop Controller to a list and assign it to a property for use in the following thread group? Below I used a couple of Dummy Sampler to explain my requirement.
I had 3 users stored in a list to retrieve 3 session ids in the setUp Thread Group.
JSR223 PreProcessor
List usernames = Arrays.asList('Peter', 'Alex', 'Mary');
props.put('accounts', usernames);
I was able to read a username from this property to get a session id in the response accordingly per iteration in the Loop Controller.
"sessionId": "this_is_my_session_id-${__groovy(props.get('accounts').get(${__jm__LoopController__idx} % 3),)}-${__jm__LoopController__idx} "
I parsed the 3 session ids out by a JSR223 PostProcessor
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
def jsonSlurper = new JsonSlurper();
def response = jsonSlurper.parseText(prev.getResponseDataAsString());
def json = JsonOutput.toJson(response.sessionId)
def sessionId = new JsonSlurper().parseText(json)
log.info('The session id is:' + sessionId)
ArrayList<String> sessionIds = new ArrayList<String>();
props.put("sessionIds", sessionIds.add(sessionId))
I needed to add these 3 session ids to a list and assign it to a property so that I can use one session id inside the property per VU/thread in the following Thread Group. But it didn't work as expected. It threw error saying No such property: sessionIds
${__groovy(props.get(sessionIds).get(${__jm__UseSession__idx} % 3),)}
We don't know what do you "expect"
Most probably the problem is here:
props.put("sessionIds", sessionIds.add(sessionId))
Collection.add() function returns a boolean value so it puts true to the sessionIds property instead of the real value of the ArrayList.
So I believe you need to change it to something like:
sessionIds.add(sessionId)
props.put("sessionIds", sessionIds)
if you're going to run the JSR223 Test Element in the loop you can also reconsider the way you're initializing the sessionIds and implement the following logic:
If sessionIds property exists - read its value
If it doesn't exist - create a new ArrayList
Something like:
ArrayList<String> sessionIds = props.get("sessionIds") ?: new ArrayList<String>()
More information on Groovy scripting in JMeter: Apache Groovy: What Is Groovy Used For?
getproperty values passed from Thread Group 1 to Thread group2
Result from BeanShell assertion
Step 1- USing jdbc request to get data from database with 2 columns and multiple rows.
Step 2 - From ThreadGroup 1, Set property to the database results using ${__setProperty(StateCodeProperty,${stateDetails})};
Step 3 - Access in Thread Group 2 by get property using beanshell assertion- String result = (vars.get("${__property(StateCodeProperty)}")); I need help on how to separate the columns and use it in api call. –
In any case if you want to access the DB results in different Thread group then you can try to do something like this inside beanshell assertion (not sure though) -
ArrayList results = ${__property(StateCodeProperty)}; //it should return the object as an arraylist
for (int i; i < results.size(); i++) {
if (results.get(i).get("statecode").equals("NY")) { //iterating the results, 'statecode' is the name of your 1st column, similarly you can do for 'State'
//Do your comparisons or whatever you like here
}
}
Using Apache Cayenne I am trying to figure out how to avoid doing iterative calls to the DB when I have a Collection of attribute values.
Say we have a persistent object Person with an attribute name of type String. If I have a List containing names of the Person I would like to generate an expression that allows a single DB call rather than iterating over the list and getting each Person in turn.
This doesn't actually exist but I would like something like this:
List<String> names = ...;
ExpressionFactory.orLikeExp(Person.NAME_PROPERTY, names);
You can use ExpressionFactory.join(..):
List<Expression> pairs = new ArrayList<>(names.size());
for(String name : names) {
// use an expression appropriate for comparison...
// "like" in this example, but can be "equals", etc.
pairs.add(ExpressionFactory.likeExp(Person.NAME_PROPERTY, name));
}
Expression e = ExpressionFactory.joinExp(Expression.OR, pairs);
I have been stuck for a while with this problem and I have no clue. I am trying to upload multiple CSV files which has dates but I wanted the dates stored as date variables so I use the date variables to form part of the column in a table using script componet and I have no idea how to create the dates as date variables in SSIS.
CSV files look as shown below when opened in Excel.
CSV data 1:
Relative Date: 02/01/2013
Run Date: 15/01/2013
Organisation,AreaCode,ACount
Chadwell,RM6,50
Primrose,RM6,60
CSV data 2:
Relative Date: 14/02/2013
Run Date: 17/02/2013
Organisation,AreaCode,ACount
Second Ave,E12,110
Fourth Avenue, E12,130
I want the Relative Date and Run Date stored as date variables. I hope I made sense.
Your best solution would be to use a Script Task in your control flow. With this you would pre-process your CSV files - you can easily parse the first two rows, retrieving your wanted dates and storing them into two variables created beforehand. (http://msdn.microsoft.com/en-us/library/ms135941.aspx)
Important to make sure when passing the variables into the script task you set them as ReadWriteVariables. Use these variables in any way you desire afterwards.
Updated Quick Walkthrough:
I presume that the CSV files you will want to import will be located in the same directory:
Add a Foreach Loop Container which will loop through the files in your specified directory and inside, a Script Task which will be responsible for parsing the two dates in each of your files and a Data Flow Task which you will use for your file import.
Create the variables you will be using - one for the FileName/Path, two for the two dates you want to retrieve. These you won't fill in as it will be done automatically in your process.
Set-up your Foreach Loop Container:
Select a Foreach File Enumerator
Select a directory folder that will contain your files. (Even better, add a variable that will take in a path you specify. This can then be read into the enumerator using its expression builder)
Wildcard for the files that will be searched in that directory.
You also need to map each filename the enumerator generates to the variable you created earlier.
Open up your Script Task, add the three variables to the ReadWriteVariables section. This is important, otherwise you won't be able to write to your variables.
This is the script I used for the purpose. Not necessarily the best, works for this example.
public void Main()
{
string filePath = this.Dts.Variables["User::FileName"].Value.ToString();
using (StreamReader reader = new System.IO.StreamReader(filePath))
{
string line = "";
bool getNext = true;
while (getNext && (line = reader.ReadLine()) != null)
{
if(line.Contains("Relative Date"))
{
string date = getDate(line);
this.Dts.Variables["User::RelativeDate"].Value = date;
// Test Event Information
bool fireAgain = false;
this.Dts.Events.FireInformation(1, "Rel Date", date,
"", 0, ref fireAgain);
}
else if (line.Contains("Run Date"))
{
string date = getDate(line);
this.Dts.Variables["User::RunDate"].Value = date;
// Test Event Information
bool fireAgain = false;
this.Dts.Events.FireInformation(1, "Run Date", date,
"", 0, ref fireAgain);
break;
}
}
}
Dts.TaskResult = (int)ScriptResults.Success;
}
private string getDate(string line)
{
Regex r = new Regex(#"\d{2}/\d{2}/\d{4}");
MatchCollection matches = r.Matches(line);
return matches[matches.Count - 1].Value;
}
The results from the execution of the Script Task for the two CSV files. The dates can now be used in any way you fancy in your Data Flow Task. Make sure you skip the first rows you don't need to import in your Source configuration.
How to use variable mapping while using Oracle OLE DB provider? I have done the following:
Execute SQL Task: Full result set to hold results of the query.
Foreach ADO Enumerator: ADO object source above variable (Object data type).
Variable Mapping: 1 field.
The variable is setup as Evaluate as an Express (True)
Data Flow: SQL Command from variable, as SELECT columnName FROM table where columnName = ?
Basically what I am trying to do is use the results of a query from a SQL Server table, (ie ..account numbers) and pull records from Oracle reference the results from the SQL query
It feels like you're mixing items. The Parameterization ? is a placeholder for a variable which, in an OLE DB Source component, you'd click on the Parameters button and map.
However, since you're using the SQL Command from a Variables, that doesn't allow you to use the Parameterization option, probably because the risk of a user changing the shape of the result set, via Expressions, is too high.
So, pick one - either "SQL Command" with proper parametetization or "SQL Command from Variable" where you add in your parameters in terrible string building fashion like Dynamically assign value to variable in SSIS SQL Server 2005/2008/2008R2 people, be aware that you are limited to 4k characters in a string variable that uses Expressions.
Based on the comment of "Basically what I am trying to do is use the results of a query from a SQL Server table, (ie ..account numbers) and pull records from Oracle reference the results from the SQL query"
There's two ways of going about this. With what you've currently developed, my above answer still stands. You are shredding the account numbers and using those as the filter in your query to Oracle. This will issue a query to Oracle for each account number you have. That may or may not be desirable.
The upside to this approach is that it will allow you to retrieve multiple rows. Assuming you are pulling Sales Order type of information, one account number likely has many sales order rows.
However, if you are working with something that has a zero to one mapping with the account numbers, like account level data, then you can simplify the approach you are taking. Move your SQL Server query to an OLE DB Source component within your data flow.
Then, what you are looking for is the Lookup Component. That allows you to enrich an existing row of data with additional data. Here you will specify a query like "SELECT AllTheColumnsICareAbout, AccountNumber FROM schema.Table ". Then you will map the AccountNumber from the OLE DB Source to the one in the Lookup Component and the click the checkmark next to all the columns you want to augment the existing row with.
I believe what you are asking is how to use SSIS to push data to Oracle OleDb provider.
I will assume that Oracle is the destination. The idea of using data destinations with variable columns is not supported out of the box. You should be able to use the SSIS API or other means, I take a simpler approach.
I recently set up a package to get all tables from a database and create dynamic CSV output. One file for each table. You could do something similar.
Switch out the streamwriter part with a section to 1. Create the table in destination. 2. Insert records into Oracle. I am not sure if you will need to do single inserts to Oracle. In another project that works in reverse, dynamic csv into SQL. SInce I work with SQL server, I load a datatable and use SQLBulkCopy class to use bulk loading which provides excellent performance.
public void Main()
{
string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
try
{
string TableName = Dts.Variables["User::CurrentTable"].Value.ToString();
string FileDelimiter = ",";
string TextQualifier = "\"";
string FileExtension = ".csv";
//USE ADO.NET Connection from SSIS Package to get data from table
SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)(Dts.Connections["connection manager name"].AcquireConnection(Dts.Transaction) as SqlConnection);
//Read data from table or view to data table
string query = "Select * From [" + TableName + "]";
SqlCommand cmd = new SqlCommand(query, myADONETConnection);
//myADONETConnection.Open();
DataTable d_table = new DataTable();
d_table.Load(cmd.ExecuteReader());
//myADONETConnection.Close();
string FileFullPath = Dts.Variables["$Project::ExcelToCsvFolder"].Value.ToString() + "\\Output\\" + TableName + FileExtension;
StreamWriter sw = null;
sw = new StreamWriter(FileFullPath, false);
// Write the Header Row to File
int ColumnCount = d_table.Columns.Count;
for (int ic = 0; ic < ColumnCount; ic++)
{
sw.Write(TextQualifier + d_table.Columns[ic] + TextQualifier);
if (ic < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
// Write All Rows to the File
foreach (DataRow dr in d_table.Rows)
{
for (int ir = 0; ir < ColumnCount; ir++)
{
if (!Convert.IsDBNull(dr[ir]))
{
sw.Write(TextQualifier + dr[ir].ToString() + TextQualifier);
}
if (ir < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
}
sw.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception exception)
{
// Create Log File for Errors
//using (StreamWriter sw = File.CreateText(Dts.Variables["User::LogFolder"].Value.ToString() + "\\" +
// "ErrorLog_" + datetime + ".log"))
//{
// sw.WriteLine(exception.ToString());
//}
Dts.TaskResult = (int)ScriptResults.Failure;
throw;
}
Dts.TaskResult = (int)ScriptResults.Success;