How can I get ParameterContext value with dynamic name in Apache NiFi?
For example, for each flowfile I need to get parameter according to ${type} attribute of this flowfile.
Like: #{${type}_parameter}.
Is it possible to do this in NiFi 1.19.1?
I tried ${literal('{${type}_parameter}'):prepend('#'):evaluateELString()}, but it always returns empty string.
Related
I have .csv file with several fields. One of them (for example 3th) contains email. How can I add additional filed that will contain only serverName from email field?
Example:
input:
01;city;name#servername.com;age;
result:
01;city;name#servername.com;age;servername;
I guess it possible through ReplaceText processor, but I can't choose correct value in "Search Value" and "Replacement Value" fields.
You can convert your flowfile, to a record with the help of ConvertRecord.
It allows you to pass to a JSON (or something else) format to whatever you prefer.
Then you can add a new field, like the following, with an UpdateRecordProcessor:
I recommend you the following reading:
Update Record tutorial
I have this flow in NiFi:
GetFile -> ConvertExcelToCSVProcessor -> ReplaceText ->
PutDatabaseRecord.
It's working ok but I want to set "table name" property of
PutDatabaseRecord
based on csv file name (and if it's posible to customize it). I can't find anything in docs or web.
Thanks!
According to documentation, the parameter table name of the PutDatabaseRecord processor supports nifi expression language.
So, if the attribute filename of your flow file contains value MyTableName.csv you could use expression with regular expression to convert file name to table name in PutDatabaseRecord processor like this:
Table Name = ${filename:replaceAll('\\..*','')}
I'm sending requests via ListenHttp processor, for example
curl -X POST --data "myschema.mytable" http://localhost:44221/contentListener
how can I then use that in my DB query?
I want to execute in the ExecuteSQL processor select * from myschema.mytable
As written by #daggett in the comments, the body of the post request is written to the FlowFile content.
You may extract the FlowFile content into an attribute using the ExtractText processor.
In case the whole request body consists of the table name you may use (.+) as RegEx to store it in an FlowFile attribute, as seen below.
This will result in the whole content being written to the defined attribute.
To prevent you from being vulnerable against SQL injection you should escape any user based input in your queries. Thus instead of executing select * from myschema.mytable you should rather use select * from ? as query.
To let the ExecuteSQL processor known what to escape and insert instead of the question mark, you must add two attributes - sql.args.1.type and sql.args.1.value - to your FlowFile before sending it to the ExecuteSQL processor.
You may use an UpdateAttribute processor to do so. The type varchar is resembled by the number 12. To find out the correct mappings consult the documentation.
You then may use it as part of your SQL-Query inside the ExecuteSQL processor.
Remarks
You should be aware of potential out-of-memory errors in case the content of the post requests gets to large. In general it is NOT recommended to store the whole content as attribute as attributes are generally stored in memory. In case you are able to alter the http request signature I recommend you to use a common format - e.g. JSON - and only extract the wanted value using a corresponding processor instead of ExtractText.
Providing the variable as HTTP header
As suggested by #mattyb in the comments, instead of using the body of a POST request you could also provide the value as HTTP header and read it from there.
That way you may remove the ReplaceText processor entirely.
To do so you may alter the request to something along the lines of:
curl --request POST \
--url http://localhost:31337/contentListener \
--header 'databaseName: nyan.cat'
Inside the ListenHttp processor you have to set the value ^databaseName$ for the HTTP Headers to receive as Attributes (Regex) property.
This will result in the attribute databaseName being set as with the previous approach.
$.id:append('-') & ${id:append('-')} in Evaluate JsonPathProcessor gives me no results, what would we be the correct way to append a text at the end of an incoming attribute in Apache Nifi.
If your Apache NiFi had a sample input JSON document like:
{
"id": "foo"
}
Then you can extract and format attributes in two steps:
EvaluateJsonPath to extract a value from the input JSON document to a NiFi flowfile attribute. For the sample, you might add a custom property json.id with the JsonPath of $.id. A flowfile attribute called json.id will be added to the flowfile with the value foo.
UpdateAttribute to use NiFi Expression Language to format the id. You can assign any attribute, like formatted.id using an expression that references the attribute extracted earlier, json.id:
I am trying to get the SESSSTARTTIME in a mapping variable. Is it possible? I don't want to use an expression for single output port.