I am trying to modify the name of some files with NiFi getting a value from a JSON an adding to the original filename (for example filename.csv (original name) + january (name that provides the incoming JSON)). To do so, I am redirecting the CSV files to updateAttribute to change de Filename.
From the other hand, I am receiving a JSON that has an attribute that will be part of the name of the file.
On the evaluateJsonPath the configuration is the next (I am receiving it correctly the value):
And finally I am trying to merge the values in the UpdateAttribute processor (here is where it doesn't work properly):
The response I am getting is _filename.csv
You have quotes around name inside your Expression Language expression, try
${name}_${filename} or
${name:append('_'):append(${filename})}
Related
http request default. http request config. csv data config used. csv data file used .I wanted to add data from csv file, but if i pass single parameter also its adding "&" character before parameter.
ex: https://www.linkedin.com/start/join?&name=A0A1A0
Just remove this ? sign from the /api/dealer/searchByPostalCode
JMeter automatically detects query string beginning in the URL path and tries to merge parameters with the ones you specify in the path to avoid syntax errors.
Given you don't provide any parameters in the "Path" section you should remove ? from there.
Another possible reason is that your CSV data contains this & character, double check it using Debug Sampler and View Results Tree listener combination.
As per the official documentation:
As a special case, the string "\t" (without quotes) in the delimiter
field is treated as a Tab.
When the end of file (EOF) is reached, and the recycle option is true,
reading starts again with the first line of the file.
Source: http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config
So I'd request you to remove \n in delimiter field from the CSV Data set config and then try.
I am using JMeter 5.1.1.
I have a CSV file looks like this
When I call HTTP Request like this
I get this
Jmeter doesn't take the value but just pass the name of the variable to the URI.
But if I take the second column CustomerParam by putting ${CustomerParam} , then JMeter will grab the value (instead of the name of the variable).
I am wondering how can I get the values of the first column ?
Thanks.
I found the root cause.
When creating the CSV file using the newest Microsoft Excel 365 , save it as CSV , not CSV UTF-8.
When I saved the file using CSV, JMeter works perfectly fine. But when I saved it using CSV UTF-8, then the issue I described earlier does appear.
If Variables are referenced as follows:
${VARIABLE}
If an undefined function or variable is referenced, JMeter does not report/log an error - the reference is returned unchanged. For example, if UNDEF is not defined as a variable, then the value of ${UNDEF} is ${UNDEF}.
Input the Variable name in your CSV Data Set Config as follows (If your CSV data set looks exactly above where variable names are present in the first line):
Or, you can just remove the variable names from the first line of your CSV file and make the config as below:
I cannot reproduce your issue using the same CSV Data Set config:
The evidence that the variables have been read (assumes Debug Sampler and View Results Tree listener combination)
So double check the following:
Your CSV file integrity as it might be an issue with the data itself, i.e. if there is a BOM at the beginning - you will have to remove it as the BOM will be added to the first variable name (use aforementioned Debug Sampler and View Results Tree listener to verify which variables are defined)
An problem with the CSV file operating-system wise (i.e. typo in location or file permission issue). Normally JMeter should report any errors connected with failure of reading the CSV file in jmeter.log
I want to merge two FlowFile by filename attribute. The UpdateAttribute contains filename -> ${UUID()}. Then EvaluateJsonPath has filname -> $.filename. However finally I get two FlowFiles with different filename attributes that cannot be merged:
Output ofEvaluateJsonPath: the value of filename is an empty string
Output of QueryElasticsearchHttp: the value of filename is 1.
How to make these two outputs to have the pair of same values of filename?
If you want to merge by filename just put an UpdateAttribute right before MergeContent and set filename to a constant value like 'myfielname'.
There seems to be no reason for the UpdateAttribute at the beginning of the flow, since you are going to overwrite filename in the EvaluateJsonPath processor, plus I think all flow files should already have filename set to the flow file's uuid, unless it came from a GetFile which set the filename from the file on disk.
I have created a RouteOnAttribute processor in nifi dataflow, i want it to select only certain files to parse on.
in the properties i have created a property "filetofetch"
with following expression
${filename:contains('INCOMING.D151221')
I need to fetch the file name INCOMING.D< YYYYMMDD>
so today 21 MARTS 2017 the filename would be
INCOMING.D20170321
I have tried with something like this to extraxt file name
${filename:contains('INCOMING.D'+ ${now():format('yyyymmdd')} )}
But i cannot concat with the date prefix
any suggestions ?
Havmaage,
You can concat by use append in Expression language like below.,
You have to use the updateAttribute to store the 'Incoming.D' in one attribute name like below.
fileStartsWith:Incoming.D
Date:${now():format('yyyyMMdd')
Then finally use routeonattribute to check like below.
${filename:contains(${fileStartsWith:append(${date})})}
You cann't be concat with '+' in Nifi.
EDIT-1:
Date:${now():format('yyyyMMdd')
Using JMeter to support functional API testing and have run across a problem with reading data from a CSV file. The data from the file is used in building a POST data body which contains something like this:
"wibbles" : ${wibble-var},
${wibble-var} is read from a CSV file and has the format :
["wibble1","wibble2","wibble3"]
... there are over 1000 wibble values in the list.
If "wibbles" : ["wibble1","wibble2","wibble3"]... is hard-coded into the POST body, then JMeter is happy, builds the POST request and does the business, but it's proved impossible to create a CSV file with even the 3 value example above, that JMeter will parse. JMeter skips the thread containing the 'CSV read' without building the POST request or sending it, so there's no response to examine, and a Debug Sampler is similarly skipped. I've heard rumours that doubling up the quotes can work but haven't been able to find the right syntax. Can anyone throw any light on this issue? Thanks
Double quotes will work if you can get "wibble1,wibble2,wibble3" & if you set Allow quoted data to true in CSV data set config
You can get this value and then use beanshell preprocessor to convert to the format "wibble1","wibble2","wibble3".
If you want to get in this format "wibble1","wibble2","wibble3" directly, you can use \t as the delimiter & modify the data in the CSV file accordingly.
Trial and error led to the following solution.
The format of the single data variable I needed to parse is ["value1","value2","value3"] (i.e a JSON array.) and this is exactly what the CSV file contained (with a header name of course on the first row), including the [ and ] brackets.
I modified the parameterised POST body to:
"wibbles": [${wibble-var}],
-- that is, I moved the square brackets out of the CSV file so that the CSV file now just contained the quoted elements of the array:
"value1","value2","value3" etc
I then set the delimiter in the CSV Data Set Config to |
And Allow Quoted Data to FALSE. <--- This was a bit counter intuitive but without it JMeter would not read the whole comma separated list of 2000 quoted strings as a single variable.
With these changes in place the script executed correctly.
Thanks again for the responses, I will definitely look at the __String functions mentioned.
I would go for the following options:
If your "wibbles" are a single string which you need to pass a a JSON Array it might be a lot easier to access them via __StringFromFile() or __FileToString() functions like:
"wibbles" : ${_StringFromFile(/path/to/file/containing/wibbles,,,)},
If you need to access individual "wibbles" and your CSV file is basically a JSON file:
Add HTTP Request Sampler to your test plan (before one which sends these "wibbles") and configure it as follows:
Protocol: file
Path: c:/testdata/yourfile.csv
Add JSON Path PostProcessor and use a JSON Path query to store the "wibbles" into a JMeter Variable(s)