i have a dynamical field, the field format looks like
A-B-C::D_[randomNum]
the field is dynamic because the randonNUM ,
i want to change the '-' to '_' and remove the [randomNUM]
and it's will be looks like as follow,
A_B_C::D
Is there any plugin / strategy to solve this problem?
You should be able to achieve this with a mutate/gsub filter
filter {
mutate {
gsub => [
# replace random num suffix
"fieldname", "_\d+", "",
# replace all dashes with underscores
"fieldname", "-", "_"
]
}
}
Make sure to replace fieldname with your actual field name.
UPDATE
Given your comments, it turned out it's the field names that are dynamic and not the value. For this reason, you cannot use the above solution but the next one should work, i.e. using the ruby filter:
filter {
ruby {
code => "
newhash = {}
event.to_hash.each {|key, value|
if key =~ /^CISCO/ then
newkey = key.gsub(/_\d+/, '').gsub('-', '_')
newhash[newkey] = event[key]
event.remove(key)
end
}
newhash.each {|key,value|
event[key] = value
}
"
}
}
After this filter runs, your event will have the field A_B_C::D instead of the original A-B-C::D_num
Related
I have this as input, I need to parse all the keys and values and put in a separate event, can you please help me how to achieve, Actually I'm only one key and value when I using the below ruby code.
"entities": {
"PROCESS_GROUP_INSTANCE-EA653E6874E21338": "WebSphere AS nzapwa145Cell (nzapwa146 / ePIMS_pwa146)",
"PROCESS_GROUP_INSTANCE-64A2967FB7FC0C5B": "WebSphere AS ndmProd (nzapwa127 / ePIMS_Report_pwa127)",
"PROCESS_GROUP_INSTANCE-4738C9392BDBC296": "EPS Store - 900008014",
"PROCESS_GROUP_INSTANCE-BA196B9B53FF6323": "EPS Store - 900008040",
"PROCESS_GROUP_INSTANCE-D5D2DAB06C8FDAAF": "ws-server.jar prod-sj-userprofile-adapter-service- - -*",
"PROCESS_GROUP_INSTANCE-3EABAD79933CE911": "/apps/conf/httpd-p2_cdis.conf - KPATHS",
}
Code:
ruby {
code => '
event.get("[result][entities]").each { |key, value|
event.set("hostId", key)
event.set("serverName", value)
}
'
}
Output for above code:
{"serverName":"/apps/conf/httpd-p2_cdis.conf - KPATHS)","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-3EABAD79933CE911"}
But desired output will be like this :
{"serverName":"WebSphere AS nzapwa145Cell (nzapwa146 / ePIMS_pwa146)","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-EA653E6874E21338"}
{"serverName":"WebSphere AS ndmProd (nzapwa127 / ePIMS_Report_pwa127)","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-64A2967FB7FC0C5B"}
{"serverName":"EPS Store - 900008014","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-4738C9392BDBC296"}
{"serverName":"ws-server.jar prod-sj-userprofile-adapter-service- - -*","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-BA196B9B53FF6323"}
{"serverName":"/apps/conf/httpd-p2_cdis.conf - KPATHS","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-D5D2DAB06C8FDAAF"}
{"serverName":"/apps/conf/httpd-p2_cdis.conf - KPATHS)","#timestamp":"2018-08-25T15:17:11.762Z","hostId":"PROCESS_GROUP_INSTANCE-3EABAD79933CE911"}
Can you please help me to get the solution, I'm facing problem with ruby code.
I'm not a Logstash expert, but I think your Ruby filter overwrites (sets) the value hostId and serverName multiple times while iterating over all given key-value pairs. That leaves you with the values that were set in the last iteration. After all, there is only one event, so your keys need to be unique. To fix that, you need to extend your event's key field by either nesting it in tree structure or creating unique key strings.
Here, hostId is the unique key :
ruby {
code => '
event.get("[result][entities]").each do |key, value|
event.set(key, value)
end
'
}
Here an additional index is added to ensure unique keys:
ruby {
code => '
idx = 0
event.get("[result][entities]").each do |key, value|
event.set("[#{idx}][hostId]", key)
event.set("[#{idx}][serverName]", value)
idx += 1
end
'
}
I hope you find that helpful.
I need a simple way to search through a string to match another string.
example:
domain.com
data that looks like this:
test[0][:element] = "foo.com domain.com bar.com"
The domains are separated by a space.
test[] is an array of hashes.
I was thinking something along these lines of:
test[0][:element].select #?
'foo.com domain.com bar.com'.split.include?('domain.com') # => true
Now if I understand you correctly, you want to find all hashes which have a value, which contains that domain:
test.select { |hash| hash.values.any? { |value| value.split.include?('domain.com') } }
I have a JSON document returned from a query to the Google Books API, e.g:
{
"items": [
{
"volumeInfo": {
"industryIdentifiers": [
{
"type": "OTHER",
"identifier": "OCLC:841804665"
}
]
}
},
{
"volumeInfo": {
"industryIdentifiers": [
{
"type": "ISBN_10",
"identifier": "156898118X"...
I need the ISBN number (type: ISBN_10 or ISBN_13) and I've written a simple loop that traverses the parsed JSON (parsed = json.parse(my_uri_response)). In this loop, I have a next if k['type'] = "OTHER" which sets "type" to "OTHER".
How do I best extract just one ISBN number from my JSON example? Not all of them, just one.
Something like XPath search would be helpful.
JSONPath may be just what you're looking for:
require 'jsonpath'
json = #your raw JSON above
path = JsonPath.new('$..industryIdentifiers[?(#.type == "ISBN_10")].identifier')
puts path.on(json)
Result:
156898118X
See this page for how XPath translates to JSONPath. It helped me determine the JSONPath above.
how about:
parsed['items'].map { |book|
book['volume_info']['industryIdentifiers'].find{ |prop|
['ISBN_10', 'ISBN_13'].include? prop['type']
}['identifier']
}
If you receive undefined method [] for nil:NilClass this means that you have an element within items array, which has no volume_info key, or that you have a volume with a set of industryIdentifiers without ISBN. Code below should cover all those cases (+ the case when you have volumeInfo without industry_identifiers:
parsed['items'].map { |book|
identifiers = book['volume_info'] && book['volume_info']['industryIdentifiers']
isbn_identifier = idetifiers && identifiers.find{ |prop|
['ISBN_10', 'ISBN_13'].include? prop['type']}['identifier']
}
isbn_identifier && isbn_identifier['identifier']
}.compact
If you happen to have the andand gem, this might be written as:
parsed['items'].map { |book|
book['volume_info'].andand['industryIdentifiers'].andand.find{ |prop|
['ISBN_10', 'ISBN_13'].include? prop['type']
}.andand['identifier']
}.compact
Note that this will return only one ISBN for each volume. If you have volumes with both ISBN_10 and ISBN_13 and you want to get both, instead of find you'll need to use select method and .map{|i| i[:identifier]} in place of .andand['identifier'].
I'm working with Nokogiri and I'm a newbye. I'm parsing an HTML document to match some placeholder, and after match I must replace the widget placeholder with some generated HTML.
I create this method:
doc = Nokogiri::HTML.fragment(raw)
matches = doc.xpath(".//widget")
if matches.present?
matches.each do |match|
media_replace(..)
else
self.body = raw
end
I have some matches, and every match has this attributes.
matches.first.attributes
{"data_id"=>#(Attr:0x3fdd42e2cebc { name = "data_id", value = "5" }),
"data_type"=>#(Attr:0x3fdd42e2ce94 { name = "data_type", value = "gallery" })}
How can I extract these attributes(gallery and 5) to pass them to my media_replace method?
Media_replace method return to me an 'html': how can I replace every 'match' with the returned HTML?
To get attribute values from a node you can use the [] method. For example:
media_replace(match['data_id'], match['data_gallery'])
To replace the node, use the replace or swap methods (assuming media_replace returns a string or other compatible object):
new_html = media_replace(...)
match.replace(new_html)
In a ruby on rails app, I build an array of Project Names and project id values, but want to truncate the length of the names. Current code is:
names = active_projects.collect {|proj| [proj.name, proj.id]}
I have tried to add a truncate function to the block, but am getting undefined method for class error.
Thanks in advance - I just cannot wrap my head around this yet.
Assuming I understood the question properly:
max_length = 10 # this is the length after which we will truncate
names = active_projects.map { |project|
name = project.name.to_s[0..max_length] # I am calling #to_s because the question didn't specify if project.name is a String or not
name << "…" if project.name.to_s.length > max_length # add an ellipsis if we truncated the name
id = project.id
[name, id]
}
Try Following
name=[]
active_projects.collect {|proj| name << [proj.name, proj.id]}
EDITED this should be
names= active_projects.collect {|proj| [proj.name.to_s[0..10], proj.id]}
In a Rails application you can use the truncate method for this.
If your code isn't in a view then you will need to include the TextHelper module in order to make the method available:
include ActionView::Helpers::TextHelper
you can then do:
names = active_projects.collect { |proj| [truncate(proj.name), proj.id] }
The default behaviour is to truncate to 30 characters and replace the removed characters with '...' but this can be overridden as follows:
names = active_projects.collect {
# truncate to 10 characters and don't use '...' suffix
|proj| [truncate(proj.name, :length => 10, :omission => ''), proj.id]
}