Logstash configuration file error(Answer not working) - elasticsearch

Question

The only thing that is certain about [url][queryString] is that it begins with 404; or that the key is long.I need to remove such keys.
If I use the ruby code below it gives cannot convert linked hashmap to string exception.
input {
file {
# Wildcards work, here :)
path => ["C:\Users\ppurush\Desktop\test\*.log"]
start_position => "beginning"
}
}
filter {
ruby {
code=>
"
require json
my_hash = JSON.parse([url][queryString])
my_hash.delete_if { |key,value| key.to_s.match(/^404;/) }
"
}
}
output {
stdout{}
elasticsearch {
host => localhost
}
}

Answer 1

You get a ruby exception because your ruby code is invalid. Try this instead:
filter {
ruby {
init => "require 'json'"
code => "
my_hash = JSON.parse( event['url']['queryString'] )
my_hash.delete_if { |key,value| key.to_s.match(/^404;/) }
"
}
}
This works if your event has a 'url' => 'queryString' field which contains valid json. You might already have some kind of filter to achieve this (e.g. grok). You might also consider using logstash's built-in json filter and maybe drop to delete certain events.
EDIT:
Suppose your input is plain json (I had to tidy this up):
{"id":"val1","host":"val2","app":"val3","#timestamp":"2015-08-04T19:00:03.642932‌​2Z","#timestampEnd":"2015-08-04T19:00:03.6429322Z","vid":"val4","vidNew":"val5","se‌​ssionId":"val6","url":{"rawUrl":"val7","path":"val8","queryString":{"404;dfdgfdgf‌​ghfhjghhhhhhhhhhhhh":""}},"net":{"method":"GET","status":"200","size":"0","timeTa‌​kenMillis":"0"},"context":{"SearchType":""}}
You can use codec => "json" in your file input.
input {
file {
path => ["C:\Users\ppurush\Desktop\test\*.log"]
start_position => "beginning"
codec => "json"
}
}
You will get a field:
"url" => {
"rawUrl" => "val7",
"path" => "val8",
"queryString" => {
"404;dfdgfdgf‌​ghfhjghhhhhhhhhhhhh" => ""
}
}
So 404;dfdgfdgf‌​ghfhjghhhhhhhhhhhhh is a variable, too. To check for it and delete the event you could do something like this:
if [url][queryString][404;dfdgfdgf‌​ghfhjghhhhhhhhhhhhh] {
drop {}
}