I'm trying to replace an empty field with nulls in an UpdateRecord processor.
/title ${field.value:replaceEmpty(null)}
This fails because "null" is not a valid keyword. How does one specify null in the nifi expression language?
You can use the literal() function to return a String value that is the exact input to the function, and you can nest that inside your replaceEmpty method. Try using the expression ${field.value:replaceEmpty(${literal('null')})}.
If you are doing this in the UpdateRecord processor, you want to use Apache NiFi RecordPath syntax, not Expression Language. I believe the CSVReader and others parse even a field value containing only spaces to empty, so a regular expression like replaceRegex( /title, '^(?![\s\S])$', 'null' ) doesn't work.
My suggestion would be to file a Jira requesting this capability. In the mean time, do not use UpdateRecord for this, but rather ReplaceText with a regular expression like ,\s?, for an empty CSV value and replace it with null.
There is a trick using RecordPath, if the field value is blank you can do this to get a null value.
/fieldName[not(isBlank(/fieldName))]
It is giving answer as
{
"fieldname" : "null"
}
here null is a string not a null value.
Related
The output of ExecuteSqlRecord is fed to EvaluateJsonPath and it returns empty string.
Output of ExecuteSqlRecord:
[
{
"X_LAST_DAY": "1618459200000",
"X_FIRST_DAY_3MON_PREV": "1610427600000",
"X_FIRST_DAY_1MON_PREV": "1615525200000",
"X_LAST_DAY_1MON_PREV": "1617163200000"
}
]
The attribute values are coming as 'Empty String Set'. Why is it coming as empty and what am I doing wrong?
In the EvaluateJsonPath, I have also tried different options like setting the following.
Return Type - auto-detect
Null Value Representation - empty string
your data is array, which should be split. use SplitJson before EvaluateJsonPath
The Apache Nifi ReplaceText processor isn't behaving the way I expect. I can't figure out why the expression evaluation is inserting zero length strings where the data should be going.
The ReplaceText processor's configuration is:
Replacement strategy is: Always Replace.
Evaluation Mode is: Entire text.
The processor chain is: QueryDatabaseTable->SplitAvro->UpdateAttribute->ReplaceText->PutSQL
The replacement value in the ReplaceText processor is:
INSERT INTO public.journal (payee, amount, billed_date, paid_date)
VALUES ('${payee}', ${amount}, '${billed_date}', '${paid_date}');
It should become….
INSERT INTO public.journal (payee, amount, billed_date, paid_date)
VALUES ('Dead End LLC', 2000.000, ‘2018-02-01’, ‘2018-02-01’);
Instead I get:
INSERT INTO public.journal (payee, amount, billed_date, paid_date)
VALUES (‘’, , ‘’, ‘’);
Which is especially frustrating when I look at the output of the preceding UpdateAttribute processor step and see…
[ {
"payee" : "Dead End LLC",
"amount" : "2000.00",
"billed_date" : "2018-02-01",
"paid_date" : "2018-02-02"
} ]
This breaks my brain since the expression processing appears to be working just fine but not pulling in the right data (which my naive implementation assumes will be there.)
Previous reading that got me to where I am now:
Database Extract
Database Insert
The reason you are getting empty string is because the expressions for '${payee}', ${amount}, '${billed_date}', '${paid_date} are evaluating to no value, and that is because you probably do not have flow file attributes with those names.
You cannot go directly from a value in Avro in the content of a flow file into NiFi's expression language, you would need to first extract the values from content into flow file attributes.
Something like this would probably work...
QueryDatabaseTable-> SplitAvro-> ConvertAvroToJson -> EvaluteJsonPath -> UpdateAttribute -> ReplaceText -> PutSQL
In EvluateJsonPath is where you would extract the values from the json into flow file attributes.
What does this vbscript code mean?
<%=read_strt(objRS, "ccnumber", "")%>
I want to know whats the meaning or used of read_strt on the code?
Any ideas?
The statement writes the return value of a call to the function "read_strt" to the HTML output stream. The function seems to read the value of a field/column in the current row of a database query (recordset), convert its type to String and/or test it against 'bad' (e.g. Null) values; in that case the default value ("", empty string) is returned.
In Freemarker, I want to treat something that may be missing (not defined on given object or in given Map), have a null value, be an empty string "" or a string with whitespace only (" \t\n"), AKA blank string. Or a real value like "foo".
In case it's anything besides a non-blank string, I want the expression to return a default value.
How can I handle this in Freemarker?
So far, this is what I use:
${ obj.propery???then(obj.property?trim!"default") }
But I can imagine something more ellegant, like
${ obj.property!!?trim!"default" }
and even with trimming (which is quite common operation for templates):
${ obj.property!!!"default" }
Is there something such in Freemarker? (Besides coding my own method or macro.)
There's nothing in FreeMarker for that, at least not in 2.3.24. I think the best way to address that is wiriting a trimToNull function, and then you can write trimToNull(obj.property)!"default" and trimToNull(obj.property)??, etc.
I had the same challenge and came to this solution:
${ (obj.property!"")?trim }
It also trims the empty string but is easier to read than writing an function or a if.
I'm banging my head against the wall on this one. I was looking at some old database reporting code written in VB6 and came across this line (the code is moving data from a "source" database into a reporting database):
rsTarget!VehYear = Trim(Str(rsSource!VehYear))
When rsSource!VehYear is Null, the above line generates an "Invalid use of Null" run-time error. If I break on the above line and type the following in the Immediate pane:
?rsSource!VehYear
It outputs Null. Fine, that makes sense. Next, I try to reproduce the error:
?Str(rsSource!VehYear)
I get an "Invalid use of Null" error.
However, if I type the following into the Immediate window:
?Str(Null)
I don't get an error. It simply outputs Null.
If I repeat the same experiment with Trim() instead of Str(), everything works fine. ?Trim(rsSource!VehYear) returns Null, as does ?Trim(Null). No run-time errors.
So, my question is, how can Str(rsSource!VehYear) possibly throw an "Invalid use of Null" error when Str(Null) does not, when I know that rsSource!VehYear is equal to Null?
Update: If I type the following in the Immediate window, it works as expected (no error occurs):
?Str(rsSource!VehYear.Value)
This outputs Null. Now, I know that rsSource!VehYear is actually an ADODB.Field instance, but Value is its default property, so Str should be operating on the Value property (which is Null). Even the error message ("Invalid use of Null") suggests that Str is receiving a Null parameter, but how can it treat Null differently in one case and not the other?
My only guess is the internal implementation of Str() is somehow failing to get the default property, and the "Invalid use of Null" error is happening for a different reason (something other than the parameter is causing the "Invalid use of Null", perhaps when it is trying to retrieve the default property from the Field object).
Does anyone have a more detailed, technical explanation for what is actually happening here?
In short:
?Str(rsSource!VehYear)
throws an "Invalid use of Null" error when rsSource!VehYear is Null, but
?Str(rsSource!VehYear.Value)
returns Null.
However, both Trim(rsSource!VehYear) and Trim(rsSource!VehYear.Value) return Null.
If you need a value other than a string, try using IsNull instead:
rsTarget!VehYear = IIf(IsNull(rsSource!VehYear), 0, rsSource!VehYear)
' Note 0 is the default value
The Str function will specifically check if a Null value is passed in and deal with it accordingly. When you pass in an object it attempts to convert the result of a default method to a String. The result of the default method is not passed into the Str method, but Field object is, so a check for the initial Null will fail. The Str function will continue to check the parameter type for datatypes that it supports when it realizes that it has an object, it will attempt to retrieve the default value. It doesn't re-attempt to deal with the default value as it did with the passed in argument, so the attempt to return a Null as a String will fail. It seems MS didn't expect a default value to be Null or any other invalid value for Str. For instance Str doesn't support an empty string either.
This was my workaround in the vb6-days:
rsTarget!VehYear = Trim(Str(rsSource!VehYear & ""))
the & "" will make sure there is allways at least an empty string to work with.
From memory, null database fields are Nothing (or possibly vbNull), which do not have the same rules applied to them as Null. You should just be able to do a quick check:
If (rsSource!VehYear Is Nothing) Then
' Null
Else
' Not null
End If