I've searched for affirmative docs to no avail.
Is it possible to work with sincedb's inode number in logstash.conf?
something like:
input { ... file ... }
filter { ... blah blah ... }
mutate {
add_field => { "sincedb_inode_number" => $SINCEDB_INODE-NUMBER_FOR_CURRENT_MATCH }
}
output {
stdout { codec => rubydebug }
gelf {
id => "graylog"
host => "graylog"
protocol => "TCP"
port => 12201
}
}
Related
I have an input log stream in logstash 8.5.3 like this:
{
"ibm_product": "IBM App Connect Enterprise",
"messageFlowUniqueName": "BROKER12.T2.TT.MQTEST",
"#timestamp": "2023-02-15T13:01:17.030747900Z",
"localTransactionId": "b5b9418a-d9aa-4d6a-b7c6-6106751d55ce-24",
"ibm_recordtype": "monitor",
"type": "ace_monitoring_event",
"eventName": "MQ Input.CatchTerminal"
}
I've set up my logstash conf as follows:
input {
http {
port => 3333
}
}
filter {
grok {
match => {
"message" => ["%{JSON:payload_raw}"]
}
pattern_definitions => {
"JSON" => "{.*$"
}
}
if [eventName] == "MQ Input.CatchTerminal" {
mutate {
add_field => { "status" => "Failure"}
}
}
mutate {
copy => {
"message" => "message_copy"
}
}
json {
source => "payload_raw"
target => "payload"
}
}
output {
elasticsearch{
hosts => ["172.20.191.90:9200"]
}
stdout { codec => json_lines }
}
I can't get the 'if' condition to process. I get a "_grokparsefailure" in my stream. Can someone help me with coding the 'if' statement?
I have many records where the msg is 'a'. Some of these records have the same type.
I'm trying to write a query that counts the number of with msg 'a', but doesn't count duplicates.
Example:
1: msg = 'a', type = 'b'
2: msg = 'a', type = 'b'
3: msg = 'a', type
= 'c'
This should return a count of two because the first and second records have the same type and are only counted once.
Here is my query so far.
body: {
query: {
bool: {
must: [
{
range: {
"#timestamp" => { from: 'now-1d', to: 'now' }
}
},
{ match: { msg: 'a' }}
]
}
}
}
Any help is appreciated!
Try using aggregations they'll count it for you :)
Read here:
https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-aggregations-bucket-terms-aggregation.html
And try something like this:
body:{
query: {
bool: {
must: [
{
range: {
"#timestamp" => { from: 'now-1d', to: 'now' }
}
},
{ match: { msg: 'a' }}
]
}
}
},
aggs:{
"type":{
"terms":{
"field":"type"
}
}
}
}
I have an event in logstash that looks like:
{
"terms" : { "A" : 1, "B" : 0.5, "c" : 1.6 }
}
I would like to change it to:
{
"terms" : [ "A", "B", "C" ]
}
I didn't find any documentation about a for loop or get the keys of dictionary.
I would like to do something like:
filter {
for key in [terms]{
mutate {
merge => ["tmp_terms", key]
}
mutate {
remove_field => ["terms"]
rename => ["tmp_terms", "terms"]
}
}
Any suggestions ?
Logstash doesn't have a loop construct but you can use the ruby plugin:
filter {
ruby {
code => "event['terms'] = event['terms'].keys"
}
}
I have this array:
params[:types] = [type1, type2, type3...]
I would like to populate my hash the following way using the above array:
params[:hash] = {
"type1" => {
something: something
},
"type2" => {
something: something
},
}
Using a for loop like for index in i ...params[:types] just populates the hash with the last value in the array.
You can use the each_with_object method to do this:
params = {}
params[:types] = ["type1", "type2", "type3"]
params[:types].each_with_object({}) { |k, h| h[k] = { "something" => "something" } }
That last line will return:
=> {"type1"=>{"something"=>"something"}, "type2"=>{"something"=>"something"}, "type3"=>{"something"=>"something"}}
Here is a code snippet example that does what you need.
hash = {}
array.each do |a|
hash[a.to_s] = { "something" => "something" }
end
output:
hash
=> {
"type1" => {
"something" => "something"
},
"type2" => {
"something" => "something"
},
"type3" => {
"something" => "something"
}
}
You could do this:
params = { types: ["type1", "type2", "type3"] }
Hash[params[:types].product([{"something" => "something"}])]
#=> {"type1"=>{"something"=>"something"},
# "type2"=>{"something"=>"something"},
# "type3"=>{"something"=>"something"}}
or with Ruby 2.1,
params[:types].product([{"something" => "something"}]).to_h
If you want a different hash for each element of params[:types]:
hashes = [{ "something1"=>"something1" }, { "something2"=>"something2" },
{ "something3"=>"something3" }]
then
Hash[params[:types].zip(hashes)]
#=> {"type1"=>{"something1"=>"something1"},
# "type2"=>{"something2"=>"something2"},
# "type3"=>{"something3"=>"something3"}}
I have the following:
value = 42
array = ["this","is","a","test"]
how can I convert that to get this
{ "this" => { "is" => { "a" => { "test" => 42 } } } }
the array is always flat.
Thank you!
Try this:
array.reverse.inject(value) { |assigned_value, key| { key => assigned_value } }
#=> {"this"=>{"is"=>{"a"=>{"test"=>42}}}}