NiFi - How can i change ReplacementValue on ReplaceText? - apache-nifi

Question

I'm using ExecuteSQL,SplitAvro,ConvertAvroToJSON,EvaluateJsonPath,ReplaceText,ExecuteSQL.
I am trying to replace the content in the flowfile using replaceText processor.
Now, i can replace like this. -> INSERT INTO values (${id},'${name}') . ReplaceText process send excute sql like this :
INSERT into x values(1,'xx')
INSERT into x values(2,'yy')
INSERT into x values(3,'zz')
I'm sending INSERT query for each line.
But i want to send executesql process like this
INSERT INTO x values (1,'xx'),(2,'yy'),(3,'zz')

Answer 1

i'm not sure it's the best approach but you could do this:
SplitAvro # i guess you are splitting the records here (here you should get fragment.* attributes)
ConvertAvroToJSON # converting each record to json
EvaluateJsonPath # getting id,name values from json
ReplaceText # (${id}, '${name}')
MergeContent # merge rows back to single file with header and delimiter
Binary Concatenation
Header = Insert into X values
Demarcator = ,
ExecuteSQL

Answer 2

Although it won't generate exactly the SQL you're looking for, take a look at ConvertRecord and/or JoltTransformRecord -> PutDatabaseRecord. The former is used to get each of your Avro records into the form you want (id, name), and PutDatabaseRecord will use a PreparedStatement in batches to send the records to the database. It might not be quite as efficient as a single INSERT, but should be much more efficient than a Split -> Convert -> ExecuteSQL with separate INSERTs per FlowFile.
To truly get the SQL you want, you'll likely need a scripted processor such as InvokeScriptedProcessor with a RecordReader, I have a blog post on the subject.