I am currently trying to convert a nested sub field that contains hexadecimal string to an int type field using Logstash filter
ruby{
code => 'event.set("[transactions][gasprice_int]", event.get("[transactions][gasPrice]").to_i(16))'
}
but it's returning the error
[ERROR][logstash.filters.ruby ][main][2342d24206691f4db46a60285e910d102a6310e78cf8af43c9a2f1a1d66f58a8] Ruby exception occurred: wrong number of arguments calling `to_i` (given 1, expected 0)
I also tried looping through json objects in transactions field using
transactions_num = event.get("[transactions]").size
transactions_num.times do |index|
event.set("[transactions][#{index}][gasprice_int]", event.get("[transactions][#{index}][gasPrice].to_i(16)"))
end
but this also returned an error of
[ERROR][logstash.filters.ruby ][main][99b05fdb6022cc15b0f97ba10cabb3e7c1a8fabb8e0c47d6660861badffdb28e] Ruby exception occurred: Invalid FieldReference: `[transactions][0][gasPrice].to_i(16)`
This method of conversion of hex-string to int type using a ruby filter worked when I wasn't dealing with nested fields, so can anyone please help me how to correctly address nested fields in this case?
btw,
this is the code that still works
ruby {
code => 'event.set("difficulty_int", event.get("difficulty").to_i(16))'
}
I think you have answered this one yourself in your final example - the to_i should not be inside the double quotes. So
...event.get("[transactions][#{index}][gasPrice].to_i(16)"))
should be
...event.get("[transactions][#{index}][gasPrice]").to_i(16))
Related
Hi I am trying to extract a value from a Netsuite hash inside custom fields, and some others, which typically look like this - `
"custbody_delivery_ticket_number"=>
{
"script_id"=>"custbody_delivery_ticket_number",
"internal_id"=>"2701",
"type"=>"platformCore:DateCustomFieldRef",
"attributes"=> {
"value"=>"123abc"
}
}` and want the value of it inside of attributes.
Have tried many different ways, but one in particular -
delivery_ticket_number: "#{netsuite_sales_orders.custom_field_list.custom_fields.select['custbody_nef_meter_ticket_number']['attributes']['value']}",
throws error for class Enumerator, NoMethodError: undefined method `[]' for #Enumerator:0x00005589ec778730 which indicates may be getting close, but doing something wrong.
If anyone has any idea how to get values from these kind of hashes?
(Am told by the system admin that it is the correct custbody identifier)
Many Thanks
Eventually fixed this, grabbing Netsuite custom fields with a select of script_id by name,and map as below:
delivery_date:netsuite_sales_order.custom_fields_list.custom_fields.select { |field| field.script_id == 'custbody_delivery_date' }.map { |field| field.value }.first
First selecting the script_id by unique name, then mapping to the value. Was able to get any custom field like this, preferable as they can move and might not have the same index if use an index to grab them, fetching an incorrect value. This way ensures getting the correct data even if the item is moved up or down in the custom fields list.
Thanks for everyones help!
I'm trying to save a json file with already inputed data with the code snippet below:
def read_rentals
return [] unless File.exist?('rentals.json')
rentals_json = JSON.parse(File.read('rentals.json'))
rentals_json.map do |rental|
Rental.new(rental['date'], #person[rental['person_index']], #books[rental['book_index']])
end
end
but I get the error message:
block in read_rentals': no implicit conversion from nil to integer (TypeError)
it stopped my script from running. robocop Rental.new line above as the error case.
The JWT.encode method expects a hash of claims. You should simply get the first value of the array and put that in as a parameter. There is no other way around. This is simply the current state of the ruby-jwt implementation.
I found the solution:
in my rentals.json, my person_index was set as null. i changed it to an integer, and the script was working
[
{
"date": "04/23",
"book_index": 0,
"person_index": 0
}
]
I've seen the JSON array questions here and I'm still a little lost, so could use some extra help.
Here's the setup:
My Flow calls a sproc on my DB and that sproc returns this JSON:
{
"ResultSets": {
"Table1": [
{
"OrderID": 9518338,
"BasketID": 9518338,
"RefID": 65178176,
"SiteConfigID": 237
}
]
},
"OutputParameters": {}
}
Then I use a PARSE JSON action to get what looks like the same result, but now I'm told it's parsed and I can call variables.
Issue is when I try to call just, say, SiteConfigID, I get "The output you selected is inside a collection and needs to be looped over to be accessed. This action cannot be inside a foreach."
After some research, I know what's going on here. Table1 is an Array, and I need to tell PowerAutomate to just grab the first record of that array so it knows it's working with just a record instead of a full array. Fair enough. So I spin up a "Return Values to Virtual Power Agents" action just to see my output. I know I'm supposed to use a 'first' expression or a 'get [0] from array expression here, but I can't seem to make them work. Below are what I've tried and the errors I get:
Tried:
first(body('Parse-Sproc')?['Table1/SiteConfigID'])
Got: InvalidTemplate. Unable to process template language expressions in action 'Return_value(s)_to_Power_Virtual_Agents' inputs at line '0' and column '0': 'The template language function 'first' expects its parameter be an array or a string. The provided value is of type 'Null'. Please see https://aka.ms/logicexpressions#first for usage details.'.
Also Tried:
body('Parse-Sproc')?['Table1/SiteconfigID']
which just returns a null valued variable
Finally I tried
outputs('Parse-Sproc')?['Table1']?['value'][0]?['SiteConfigID']
Which STILL gives me a null-valued variable. It's the worst.
In that last expression, I also switched the variable type in the return to pva action to a string instead of a number, no dice.
Also, changed 'outputs' in that expression for 'body' .. also no dice
Here is a screenie of the setup:
To be clear: the end result i'm looking for is for the system to just return "SiteConfigID" as a string or an int so that I can pipe that into a virtual agent.
I believe this is what you need as an expression ...
body('Parse-Sproc')?['ResultSets']['Table1'][0]?['SiteConfigID']
You can see I'm just traversing down to the object and through the array to get the value.
Naturally, I don't have your exact flow but if I use your JSON and load it up into Parse JSON step to get the schema, I am able to get the result. I do get a different schema to you though so will be interesting to see if it directly translates.
I have a custom gem which creates a AR query with input that comes from an elasticsearch instance.
# record_ids: are the returned ids of the ES results
# order: is the order of the of the ids that ES returns
search_class.where(search_class.primary_key => record_ids).order(order)
Right now the implementation is that I build the order string directly into the order variable so it looks like this: ["\"positions\".\"id\" = 'fcdc924a-21da-440e-8d20-eec9a71321a7' DESC"]
This works fine but throws a deprecation warning which ultimately will not work in rails6.
DEPRECATION WARNING: Dangerous query method (method whose arguments are used as raw SQL) called with non-attribute argument(s): "\"positions\".\"id\" = 'fcdc924a-21da-440e-8d20-eec9a71321a7' DESC". Non-attribute arguments will be disallowed in Rails 6.0. This method should not be called with user-provided values, such as request parameters or model attributes. Known-safe values can be passed by wrapping them in Arel.sql()
So I tried couple of different approaches but all of them with no success.
order = ["\"positions\".\"id\" = 'fcdc924a-21da-440e-8d20-eec9a71321a7' DESC"]
# Does not work since order is an array
.order(Arel.sql(order))
# No errors but only returns an ActiveRecord_Relation
# on .inspect it returns `PG::SyntaxError: ERROR: syntax error at or near "["`
.order(Arel.sql("#{order}"))
# .to_sql: ORDER BY [\"\\\"positions\\\".\\\"id\\\" = 'fcdc924a-21da-440e-8d20-eec9a71321a7' DESC\"]"
order = ['fcdc924a-21da-440e-8d20-eec9a71321a7', ...]
# Won't work since its only for integer values
.order("idx(ARRAY#{order}, #{search_class.primary_key})")
# .to_sql ORDER BY idx(ARRAY[\"fcdc924a-21da-440e-8d20-eec9a71321a7\", ...], id)
# Only returns an ActiveRecord_Relation
# on .inspect it returns `PG::InFailedSqlTransaction: ERROR:`
.order("array_position(ARRAY#{order}, #{search_class.primary_key})")
# .to_sql : ORDER BY array_position(ARRAY[\"fcdc924a-21da-440e-8d20-eec9a71321a7\", ...], id)
I am sort of stuck since rails forces attribute arguments in the future and an has no option to opt out of this. Since the order is a code generated array and I have full control of the values I am curious how I can implement this. Maybe someone had this issue before an give some useful insight or idea?
You could try to apply Arel.sql to the elements of the array, that should work, ie
search_class.where(search_class.primary_key => record_ids)
.order(order.map {|i| i.is_a?(String) ? Arel.sql(i) : i})
Why does this test pass, while the month value is obviously invalid (13)?
#Test
public void test() {
String format = "uuuuMM";
String value = "201713";
DateTimeFormatter.ofPattern(format).withResolverStyle(ResolverStyle.STRICT)
.parse(value);
}
When using a temporal query, the expected DateTimeParseException is thrown:
DateTimeFormatter.ofPattern(format).withResolverStyle(ResolverStyle.STRICT)
.parse(value, YearMonth::from);
What happens when no TemporalQuery is specified?
EDIT: the 13 value seems to be a special one, as I learned thanks to the answer of ΦXocę 웃 Пepeúpa ツ (see Undecimber).
But the exception is not thrown even with another value, like 50:
#Test
public void test() {
String format = "uuuuMM";
String value = "201750";
DateTimeFormatter.ofPattern(format).withResolverStyle(ResolverStyle.STRICT)
.parse(value);
}
I've made some debugging here and found that part of the parsing process is to check the fields against the formatter's chronology.
When you create a DateTimeFormatter, by default it uses an IsoChronology, which is used to resolve the date fields. During this resolving phase, the method java.time.chrono.AbstractChronology::resolveDate is called.
If you look at the source, you'll see the following logic:
if (fieldValues.containsKey(YEAR)) {
if (fieldValues.containsKey(MONTH_OF_YEAR)) {
if (fieldValues.containsKey(DAY_OF_MONTH)) {
return resolveYMD(fieldValues, resolverStyle);
}
....
return null;
As the input has only the year and month fields, fieldValues.containsKey(DAY_OF_MONTH) returns false, the method returns null and no other check is made as you can see in the Parsed class.
So, when parsing 201750 or 201713 without a TemporalQuery, no additional check is made because of the logic above, and the parse method returns a java.time.format.Parsed object, as you can see by the following code:
DateTimeFormatter fmt = DateTimeFormatter.ofPattern("uuuuMM").withResolverStyle(ResolverStyle.STRICT);
TemporalAccessor parsed = fmt.parse("201750");
System.out.println(parsed.getClass());
System.out.println(parsed);
The output is:
class java.time.format.Parsed
{Year=2017, MonthOfYear=50},ISO
Note that the type of the returned object is java.time.format.Parsed and printing it shows the fields that were parsed (year and month).
When you call parse with a TemporalQuery, though, the Parsed object is passed to the query and its fields are validated (of course it depends on the query, but the API built-in ones always validate).
In the case of YearMonth::from, it checks if the year and month are valid using the respective ChronoField's (MONTH_OF_YEAR and YEAR) and the month field accepts only values from 1 to 12.
That's why just calling parse(value) doesn't throw an exception, but calling with a TemporalQuery does.
Just to check the logic above when all the date fields (year, month and day) are present:
DateTimeFormatter fmt = DateTimeFormatter.ofPattern("uuuuMMdd").withResolverStyle(ResolverStyle.STRICT);
fmt.parse("20175010");
This throws:
Exception in thread "main" java.time.format.DateTimeParseException: Text '20175010' could not be parsed: Invalid value for MonthOfYear (valid values 1 - 12): 50
As all the date fields are present, fieldValues.containsKey(DAY_OF_MONTH) returns true and now it checks if it's a valid date (using the resolveYMD method).
The month 13 is called : Undecimber
The gregorian calendar that many of us use allows 12 months only but java includes support for calendars which permit thirteen months so it depends on what calendar system you are talking about
For example, the actual maximum value of the MONTH field is 12 in some years, and 13 in other years in the Hebrew calendar system. So the month 13 is valid
It is a little odd that an exception is not thrown when parse is called without a given TemporalQuery. Some of the documentation for the single argument parse method:
This parses the entire text producing a temporal object. It is typically more useful to use parse(CharSequence, TemporalQuery). The result of this method is TemporalAccessor which has been resolved, applying basic validation checks to help ensure a valid date-time.
Note that it says it is "typically more useful to use parse(CharSequence, TemporalQuery)". In your examples, parse is returning a java.time.format.Parsed object, which is not really used for anything other than creating a different TemporalAccessor.
Note that if you try to create a YearMonth from the returned value, an exception is thrown:
YearMonth.from(DateTimeFormatter.ofPattern(format)
.withResolverStyle(ResolverStyle.STRICT).parse(value));
throws
Exception in thread "main" java.time.DateTimeException: Unable to obtain YearMonth from TemporalAccessor: {Year=2017, MonthOfYear=50},ISO of type java.time.format.Parsed
at java.time.YearMonth.from(YearMonth.java:263)
at anl.nfolds.Test.main(Test.java:21)
Caused by: java.time.DateTimeException: Invalid value for MonthOfYear (valid values 1 - 12): 50
at java.time.temporal.TemporalAccessor.get(TemporalAccessor.java:224)
at java.time.YearMonth.from(YearMonth.java:260)
... 1 more
Documentation for Parsed:
A store of parsed data.
This class is used during parsing to collect the data. Part of the parsing process involves handling optional blocks and multiple copies of the data get created to support the necessary backtracking.
Once parsing is completed, this class can be used as the resultant TemporalAccessor. In most cases, it is only exposed once the fields have been resolved.
Since:1.8
#implSpecThis class is a mutable context intended for use from a single thread. Usage of the class is thread-safe within standard parsing as a new instance of this class is automatically created for each parse and parsing is single-threaded