I am receiving a JSON object with an array property. I would like to search the array and save only the element that matches my criteria. My input looks like this:
{
"identifier": [
{ "system" : "Source1", "value" : "TheValueIDontWant"},
{ "system" : "Source2", "value" : "TheValueIWant"}
]
}
and I would like my output to look like this:
{
"SourceID": "TheValueIWant"
}
So in this case, I want to search the identifier array for the element which has Source2 as the system and save its corresponding value to my new property.
Is there a way to do this in Logstash?
Thanks
Got this answer from someone on the elastic forum. Using ruby was indeed the answer and this is how:
ruby {
code => '
ids = event.get("identifier")
if ids.is_a? Array
ids.each { |x|
if x["system"] == "Source2"
event.set("SourceID", x["value"])
end
}
end
'
}
Related
Currently, I have this kind of JSON array with the same field, what I wanted is to split this data into an independent field and the field name is based on a "name" field
events.parameters (this is the field name of the JSON array)
{
"name": "USER_EMAIL",
"value": "dummy#yahoo.com"
},
{
"name": "DEVICE_ID",
"value": "Wdk39Iw-akOsiwkaALw"
},
{
"name": "SERIAL_NUMBER",
"value": "9KJUIHG"
}
expected output:
events.parameters.USER_EMAIL : dummy#yahoo.com
events.parameters.DEVICE_ID: Wdk39Iw-akOsiwkaALw
events.parameters.SERIAL_NUMBER : 9KJUIHG
Thanks.
Tldr;
There is no filter that does exactly what you are looking for.
You will have to use the ruby filter
I just fixed the problem, for everyone wondering here's my ruby script
if [events][parameters] {
ruby {
code => '
event.get("[events][parameters]").each { |a|
name = a["name"]
value = a["value"]
event.set("[events][parameters_split][#{name}]", value)
}
'
}
}
the output was just like what I wanted.
Cheers!
When there is a single value, whereJsonContains does the job. But in this case, I have an array which is as a result of saving to JSON column being converted into string. Here is the example:
[
{
"key":"cUyV6kW3noxxW85G",
"value":"value-1",
},
{
"key":"R8dHf4vWBS8M4W5G",
"value":"value-2",
"multidimensional_array":{
"table":[
{
"attributes":{
"array":"[\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\",\"10\"]"
}
}
],
}
},
{
"key":"cUyV6kW3noxxW85G",
"value":"value-3",
},
]
I want to get those elements which contain specific value inside array. Here is my try, but it's not working, no results are being returned:
ExampleTable::whereJsonContains('example_column', ['multidimensional_array' => ['table' => ['attributes' => ['array' => $specific_value]]]])->get();
Is it even possible to do it this way, or do I need to get every single result which has multidimensional_array and json_decode() array so I can iterate over it to look for what I need?
take a look at the documentation: https://laravel.com/docs/master/queries#json-where-clauses
You should use the whereJsonContains a bit different;
ExampleTable::whereJsonContains('multidimensional_array->table->attributes->array', $specific_value)->get();
This question is a followup to this question.
If my documents look like so:
{"documentid":1,
"documentStats":[ {"foo_1_1":1}, {"foo_2_1":5}, {"boo_1_1":3} ]
}
What would be the correct groovy script to be used in a script_field for returning the sum of all documentStats per document that match a particular pattern, (e.g., contain _1_)
Similar to the referred question, there's a one-liner that does the same thing with your new structure:
{
"query" : {
...
},
"script_fields" : {
"sum" : {
"script" : "_source.documentStats.findAll{ it.keySet()[0] =~'_1_' }.collect{it.values()}.flatten().sum()"
}
}
}
I don't know ES, but in pure Groovy you would do:
document.documentStats.collectMany { Map entry ->
// assumes each entry has a single key and a single int value
def item = entry.entrySet()[0]
item.key.contains('_1_') ? [item.value] : []
}.sum()
Hope this helps.
In Elasticsearch index , I have field called category , and I want to rename it to cat in the returned array of objects in stead of array of actual value , something like MySQL SELECT category as cat
I tried to use partial_fields
, it returns an array
"partial_fields" : {
"cat" : {
"include" : ["category"]
}
}
but it returns
"fields": {
"cat": [
{
"category": 1
}
]
}
in fact I want it to be something like
"fields": {
"cat": [1]
}
is there any way to do this ?
That's not possible, unfortunately. You'll have to handle this in your application.
I'm searching for a value in an array of a sub arrays. What will the code in C# look like?
db.File.find({
Properties: {
$elemMatch: {
$elemMatch: {
$in:
['AWS-Uploaded']
}
}
}
});
A simplified version of the documents looks like this:
{
"_id" : ObjectId("4f3b83acec76021c6827769e"),
"Extension" : ".mov",
"Length" : NumberLong(7910975),
"Properties" : [
["MediaId", "20898180"],
["AWS-Uploaded", "11/08/2013 16:15:50"]
]
}
For the query object for returning documents where an array element exists, this should do the job
Query.ElemMatch("Properties",
Query.Exists('AWS-Uploaded',true)
);
To use ElemMatch with query operators, like where MediaId=20898180 would be:
Query.ElemMatch("Properties",
Query.EQ('MediaId',20898180)
);
Hope that helps