Apache NiFi. add new CSV field based on existing with modification - apache-nifi

I have .csv file with several fields. One of them (for example 3th) contains email. How can I add additional filed that will contain only serverName from email field?
Example:
input:
01;city;name#servername.com;age;
result:
01;city;name#servername.com;age;servername;
I guess it possible through ReplaceText processor, but I can't choose correct value in "Search Value" and "Replacement Value" fields.

You can convert your flowfile, to a record with the help of ConvertRecord.
It allows you to pass to a JSON (or something else) format to whatever you prefer.
Then you can add a new field, like the following, with an UpdateRecordProcessor:
I recommend you the following reading:
Update Record tutorial

Related

MongoDB ruby driver update single field

https://docs.mongodb.com/ruby-driver/v2.14/tutorials/ruby-driver-crud-operations/#updating
I' been doing this
videos.find({"id": "c024f2bd"}).update_one({"title": "this is testing"})
When i look over the database it replace the entire document with just this field, my other field were all gone and empty. How can i update just single field? i've read the document it doesn't seem to have option parameter where i can define update field only don't replace.
You should use $set. Try this:
videos.update_one({"id": "c024f2bd"}, {"$set": {"title": "this is testing"}})

Specify alternate JSONpaths

Suppose I have one JSON log that outputs the following information:
{"timestamp":"someText","alert":"someMoreText","Level":someInt}
And I have another JSON log that outputs the same kind of information but with different label's:
{"ts":"someText","alert":"someMoreText","Level":someInt}
The difference being "timestamp" and "ts" have different names yet same quality information. How would I reference, with one JSON path call, either one of the alternate names, if such a technique is possible?
So for example, if I wanted it to reference the timestamp of both logs, I would want to use something like $.[timestamp|ts]
Using the new record processors, you might be able to do something like...
Define a schema that has both 'timestamp' and 'ts'
Send all the records with 'ts' to an UpdateRecord processor
Set the UpdateRecord processor to make /timestamp = /ts
Define another version of the schema that doesn't have 'ts'
Use a ConvertRecord processor with a writer that uses the second schema
That last step would rewrite the records without the 'ts' field.
Alternatively, you could try defining a schema with a 'timestamp' field and an alias of 'ts' which should let any of the record processors access both fields by using 'timestamp'. It would depend what you are doing in your flow to see if it can be achieved with the record processors.

Parsing CSV using logstash

Is there a way the top row of the csv can be used to make column headers for the elasticsearch index? As in my csv, columns are not fixed in numbers or names.
Also the requirement is to convert all the datatypes based on a regex to integer/float. Can mutate do that?
You can do it using Embulk tools 'guess' command option. This command will interpret the data and automatically assign format to it.
Look at 'Loading a csv file' section in this link,
http://www.embulk.org/docs/recipe/scheduled-csv-load-to-elasticsearch-kibana4.html

How do i rename a scripted field in kibana?

Is there a way i can change only the name of a scripted field? I'm using Kibana 4.3.0. For eg. I want to rename Bytes Read(GB) to Bytes Read. When i click on edit field, it doesn't allow name changes.
You can not change the name of a scripted field, you only can copy the script and paste in a new scripted field with the new name.

Using custom fields in freemarker

I am trying to print custom field in PDF template, got from a saved search with the naming syntax Bank Name(Custom). How to use this name in the freemarker pdf template ? I am not able to fetch the value using any of the below : subs.bankname where subs is the result of the saved search.
A NetSuite custom field is prefixed with "cust" (ie. custrecord, custentity, custbody, etc), so you need to find out the correct field's id in order to display it using the Freemarker syntax. Also, as "subs" is the result of a Saved Search, you might need to interact with all rows.
In your case it would be something like (to display the first row):
${subs[0].cust_bankname}
Or the following to interact with all rows:
<#list subs as sub>
${sub.cust_bankname}
</#list>
I hope it helps.

Resources