Jmeter suite to upload 1000 multiple format files through put request - performance

I am facing one problem during uploading multiple files on server. I implemented some logic and fetched files one by one from a folder . But in HTTP request my URL is something like {Container name} followed by {Filename}.
My problem is in filename parameter value what should i passed so that every files will be uploaded on server .
My code is as below:-
File folder = new File("c:\\Test1\\Test");
File[] files = folder.listFiles(new FileFilter() {
public boolean accept(File file) {
return file.isFile();
}
});
for (int i=0; i < files.length; i++) {
vars.put("file_" + i, files[i].getAbsolutePath());
}
In file path i am passing this ${filesToUpload} ,parameter name and mimetype

I don't think your way is correct (unless you want to build the full raw body), you should rather invoke HTTPSamplerBase.setHTTPFiles() function using the following example steps:
Add JSR223 PreProcessor as a child of the HTTP Request sampler to which you want to add files
Put the following code into "Script" area:
import org.apache.commons.io.FileUtils
import org.apache.commons.io.filefilter.TrueFileFilter
import org.apache.jmeter.protocol.http.util.HTTPFileArg
import java.nio.file.Files
def files = FileUtils.listFiles(new File("c:/Test1/Test"), TrueFileFilter.INSTANCE, TrueFileFilter.INSTANCE)
def arguments = []
files.eachWithIndex { file, index ->
arguments.add(new HTTPFileArg(file.getAbsolutePath(), "your_parameter_name", Files.probeContentType(file.toPath())))
}
sampler.setHTTPFiles(arguments as HTTPFileArg[])
That's it, the code will dynamically read the files from folder and add the files from it to the request, just make sure to amend your_parameter_name to the real respective value
More information on Groovy scripting in JMeter: Apache Groovy - Why and How You Should Use It

Related

Files are overwriting instated of appending in BeanShell PostProcessor Jmeter while running 1000 threads

I have written below code under beanshall post-processor. But when I am running 1000 threads the files are overwriting existing content instated of appending. It is working for 1-5 threads. Can anyone help me on this?
import org.apache.commons.io.FileUtils;
import java.util.ArrayList;
import java.util.Collections;
File fExceptionLog = new File("${logPath}/ExceptionLog.txt");
String extExceptionData= FileUtils.readFileToString(fExceptionLog);
id=vars.get("id");
try{
String cDatestamp="${__time(yyyyMMddHHmmssSSS)}";
String cResponce = prev.getResponseDataAsString();
String cRequest = prev.getQueryString();
String cResponceCode=prev.getResponseCode();
cTransactionName = prev.getSampleLabel();
cResponseTime = prev.getTime();
cSize = prev.getBytesAsLong();
cIsSuccessful =prev.isSuccessful();
File fRequestLog = new File("${logPath}/RequestLog.txt");
File fHitLog = new File("${logPath}/HitLog.txt");
File fResponceLog = new File("${logPath}/ResponceLog.txt");
File fErrorLog = new File("${logPath}/ErrorLog.txt");
String extHitData = FileUtils.readFileToString(fHitLog);
String extRequestData = FileUtils.readFileToString(fRequestLog);
String extResponceData = FileUtils.readFileToString(fResponceLog);
String extErrorData = FileUtils.readFileToString(fErrorLog);
log.info("cResponceCode"+cResponceCode);
FileUtils.writeStringToFile(fHitLog,extHitData+id+"~"+cDatestamp+"~"+cTransactionName+"~"+cResponceCode+"~"+cResponseTime+"~"+cSize+"~"+cIsSuccessful+"\n");
if(cResponceCode.equals("200")){
FileUtils.writeStringToFile(fRequestLog,extRequestData+id+"~"+cDatestamp+"~"+cTransactionName+"~"+cResponce+"\n");
FileUtils.writeStringToFile(fResponceLog,extResponceData+id+"~"+cDatestamp+"~"+cResponceCode+"~"+cResponce+"\n");
}else{
FileUtils.writeStringToFile(fErrorLog,extErrorData+id+"~"+cDatestamp+"~"+cTransactionName+"~"+cResponce+"\n"+id+"~"+cDatestamp+"~"+cResponceCode+"~"+cResponce+"\n");
}
}catch(Exception e){
FileUtils.writeStringToFile(fExceptionLog,extExceptionData+id+"~"+cDatestamp+"~"+cTransactionName+"~"+e+"\n");
}
You're violating at least 3 JMeter Best Practices
You're referring JMeter Variables like ${logPath} while you should be using vars shorthand instead like vars.get("logPath")
You're using Beanshell while starting from JMeter 3.1 you should be using JSR223 and Groovy
And last but not the least, you yourself introduced a race condition so when several threads will be concurrently writing the same file it will result in data loss. You can put this Beanshell test element (along with the parent Sampler(s)) under the Critical Section Controller, but it will reduce concurrency of the parent sampler(s) to only one at a time
If you need to write some some metrics into a custom file in your own format I would rather recommend consider migrating to the Flexible File Writer which is extremely "flexible" with regards to what values is to store and it accumulates multiple entries in memory and flushes them periodically in batch manner so all the data will be stored without collisions.
You can install Flexible File Writer using JMeter Plugins Manager

Jmeter: how to initialise header manager element globally

I wanted to use the same set of headers in multiple jmx files. So I wanted to initialise it once and have to use it across my jmx files.
Can anyone help me in meeting my requirement? Thanks in advance.
That’s not possible.
To be able to apply a Header Manager to all plan, it should have the largest scope but using Include or Module controller means reduced scope.
Thanks to scope stil, you can set your Header Manager as child of test plan and it will apply to whole requests.
You could use properties and __P function to make those configurable in user.properties
You can do this as follows:
Create a CSV file called headers.csv to hold your headers like:
header-1-name,header-1-value
header-2-name,header-2-value
and store it in "bin" folder of your JMeter installation
Add empty HTTP Header Manager to the top level of your Test Plan
Add setUp Thread Group to your Test Plan
Add JSR223 Sampler to the setUp Thread Group
Put the following code into "Script" area:
import org.apache.jmeter.protocol.http.control.Header
import org.apache.jmeter.protocol.http.control.HeaderManager
import org.apache.jmeter.threads.JMeterContext
import org.apache.jmeter.threads.JMeterContextService
import org.apache.jorphan.collections.SearchByClass
SampleResult.setIgnore()
def engine = ctx.getEngine()
def testPlanTree = org.apache.commons.lang3.reflect.FieldUtils.readDeclaredField(engine, "test", true)
def headerManagerSearch = new SearchByClass<>(HeaderManager.class)
testPlanTree.traverse(headerManagerSearch)
def headerManagers = headerManagerSearch.getSearchResults()
headerManagers.any { headerManager ->
new File('headers.csv').readLines().each { line ->
def values = line.split(',')
headerManager.add(new Header(values[0], values[1]))
}
}
If you want you can "externalize" points 3 and 4 via Test Fragment

How to get the downloaded xlsx file from the api endpoint in karate?

I have an endpoint that downloads an xlsx file. In my test, I need to check the content of the file (not comparing the file with another file, but reading the content and checking). I am using karate framework for testing and I am trying to use apache POI for working with the excel sheet. However, the response I get from karate when calling the download endpoint is a String. For creating an excel file with POI I need an InputStream or the path to the actual file. I have tried the conversion, but it does not work.
I guess I am missing some connection here, or maybe the conversion is bad, I am new to karate and to the whole thing.
I appreciate any help, thanks!
Given url baseUrl
Given path downloadURI
When method GET
Then status 200
And match header Content-disposition contains 'attachment'
And match header Content-disposition contains 'example.xlsx'
And match header Content-Type == 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
* def value= FileChecker.createExcelFile(response)
* print value
And the Java code:
public static String createExcelFile(String excel) throws IOException, InvalidFormatException {
InputStream stream = IOUtils.toInputStream(excel, Charset.forName("UTF-8"));
Workbook workbook = WorkbookFactory.create(stream);
return ("Workbook has " + workbook.getNumberOfSheets() + " Sheets : ");
}
When running the scenario, I get the following error:
javascript evaluation failed: FileChecker.createExcelFile(response), java.io.IOException: Failed to read zip entry source
When testing the same endpoint in Postman, I am getting a valid excelsheet.
In Karate 0.9.X onwards you have a responseBytes variable which will be raw bytes, which may be what you need.
* def value = FileChecker.createExcelFile(responseBytes)
And you can change your method signature to be:
public static String createExcelFile(byte[] excel) {}
You should be easily able to convert a byte array to an InputStream and take it from there.
P.S. just saying that it "works in Postman" is not helpful :P
TO download zip file from Karate tests as binary bite array
Scenario: To verify and get the ADCI Uri from deployment
Given url basicURL + DeployUri +ArtifactUri
And headers {authorization:'#(authToken)',accept:'application/json',tenant:'#(tenantUUId)',Content-Type:'application/zip'}
When method get
Then status 200
And def responsebytes = responseBytes

How to use java program in bean pre-processor

The below program reads the XML file and compress into gzip.
I have a couple of questions here.
Can I use the following program directly in JMeter BeanShell pre-processor?
I want to use the output variable as input to JSON request. Is it possible in Jmeter?
Screen shot and details will be appreciated.
public static void main(String[] args) throws Exception {
String line = null;
String sb = "";
File f=new File("D:\\RetailTransactionLog_9419_001_590.xml");
FileReader fr=new FileReader(f);
BufferedReader br=new BufferedReader(fr);
while((line=br.readLine())!=null)
{
sb= sb + line;
}
br.close();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gzos = new GZIPOutputStream(baos);
gzos.write(sb.getBytes("UTF-8"));
gzos.close();
String base64CompressedString = Base64.getEncoder().encodeToString(baos.toByteArray());
System.out.println(base64CompressedString);
Of course, you can put your piece of java code directly in a JMeter BeanShell pre-processor and much more !
Insert you pre-processor component as child of your JSON request (as in my script example in attachment).
You don’t need to import java.io package like BufferedReader, ByteArrayOutputStream, File, FileReader, IOException, Base64…
Remove also the main signature public static void main(String[] args) …
You only have to import "java.util.zip.GZIPOutputStream" (A)
I’ve also replace your System.out.println(base64CompressedString) by log.info(base64CompressedString) just to visualize in the jmeter console your output (B)…
And finally add at the end, the code (C) to reference your result in the variable of your choice ("a" in my example).
You just have to call your variable after with ${a} in your json request like in my JMX script :
Unzip attachments http://uplea.com/dl/9F734367B43FB93 :
"ReadAndCompressMyFile.jmx" under /bin and put "test.xml" under C: or change the path in your code.
I’ve used a dummy sampler instead of your json request.
After running my script, you can see in View Result Tree (Request tab) and in the console, the value of "a" (corresponding to base64CompressedString).
PS : To run my script with the "dummy sampler", you need to add jmeter-plugins-dummy-0.1.jar under /lib/ext of your jmeter directory.
Hope to help you...
GZIP compression is rather "heavy" operation, if you will have lots of virtual users - it may become a bottleneck so consider using other approaches:
There is __base64Encode() function available via JMeter plugins
If for any reason it is not enough - go for JSR223 Sampler and Groovy language, Groovy is mostly compatible with Java so your code should work fine (just remove main method)

How do you use jxl in jmeter

I am trying to read an xls file in jmeter using beanshell postprocessor. The error i get in the logs is: "Typed variable declaration : Class: Workbook not found in namespace"
Following is my code.
import java.io.*;
import jxl.Cell;
import jxl.Sheet;
import jxl.Workbook;
FileInputStream(vars.get("jmeterScriptPath")+"\\"+vars.get("VIN")+".xls", true);
Workbook wb = Workbook.getWorkbook(new java.io.File("C:\\datasheet\\RIGUD000000000051.xls"));
Sheet sh = wb.getSheet("RIGUD000000000051");
// To get the number of rows present in sheet
int totalNoOfRows = sh.getRows();
// To get the number of columns present in sheet
int totalNoOfCols = sh.getColumns();
for (int row = 0; row < totalNoOfRows; row++) {
for (int col = 0; col < totalNoOfCols; col++) {
System.out.print(sh.getCell(col, row).getContents() + "\t");
vars.put("responseContent",sh.getCell(col, row).getContents());
}
System.out.println();
}
In order to get it working follow next two steps:
Copy jxl.jar to JMeter's "lib" folder
Restart JMeter - it's required to pick the jxl.jar up.
You should be good to do with your code.
P.S. I would recommend to switch to Apache POI as JExcelApi supports only Excel 2003 documents and now is almost 2016. If you download tika-app.jar it will enable preview of all supported formats in View Results Tree listener and you also will get read/write access to them all from scripting test elements.
See How to Extract Data From Files With JMeter guide for more information on working with binary files in JMeter.

Resources