How do I add attributes to a nested XML array using Savon? - ruby

I am using savon version 2.2 to call some SOAP services. For simple services everything is working OK. However one service has a complex structure like the example below with both repeating groups and attributes on each group:
<sch:RequestControl>
<sch:requestID>6989668868</sch:requestID>
</sch:RequestControl>
<sch:InquiryParam>
<sch:crmParam name="AccountNumber">1234567</sch:crmParam>
<sch:crmParam name="History">1</sch:crmParam>
</sch:InquiryParam>
My current message looks like this:
<RequestControl>
<requestID>6989668868</requestID>
</RequestControl>
<InquiryParam>
<crmParam>1234567</crmParam>
<attributes>
<crmParam>
<name>AccountNumber</name>
</crmParam>
</attributes>
</InquiryParam>
<InquiryParam>
<crmParam>1</crmParam>
<attributes>
<crmParam>
<name>History</name>
</crmParam>
</attributes>
</InquiryParam>
The above is produced using this logic:
message = { :RequestControl =>
{ :requestID => 6989668868 },
:InquiryParam => [
{ :crmParam => { :content! => #account_number } ,
:attributes => { "crmParam" => {"name" => "AccountNumber"} } },
{ :crmParam => { :content! => #history } ,
:attributes => { "crmParam" => {"name" => "History"} } } ]
}
I've tried various combinations using :crmParam => { :content! => #account_number, :attributes => {'name'=>'AccountNumber'} } and similar based on the savon and gyoku documentation but have run onto a brick wall in getting the XML to format like the example. I know I can brute force the message by assigning it to xml but that makes it difficult to see what's going on.
Can anyone suggest a fix to have the attributes inside the crmParam tags?

I'm not sure Savon handles a Ruby array as you'd like, but the following script should give you a better idea what you can do.
require 'savon'
c = Savon.client(endpoint: "http://www.example.com",
namespace: "urn:ns.example.com",
log: true,
log_level: :debug,
pretty_print_xml: true)
r = c.call(:call,
:message => {
:InquiryParam => [
{"crmParam" => 123,
:attributes! => { "crmParam" => { "name" => "AccountNumber" }}},
{"crmParam" => 456,
:attributes! => { "crmParam" => { "name" => "history" }}}
]
}

Related

jira-ruby: issue.save returning false

I am trying to create an issue with jira-ruby in the terminal. So far I have done the following (where username, password, site and project have been replaced with the proper values). I have been able to fetch issues, but not to create them. Jira-ruby return false when i try and save an issue
options = {
:username => "username",
:password => "password",
:site => 'site',
:context_path => '',
:auth_type => :basic
}
client = JIRA::Client.new(options)
issue = client.Issue.build
issue.save({
"fields" => {
"summary" => "blarg from in example.rb",
"project" => {"key" => "mykey"},
"issuetype" => {"id" => "1"}
}
})
=> false
issue.attrs
=> {"errorMessages"=>[], "errors"=>{"issuetype"=>"issue type is required"}, "summary"=>"blarg from in example.rb", "key"=>"somekey", "id"=>"someid", "self"=>"site", "exception"=>{"class"=>"Net::HTTPBadRequest", "code"=>"400", "message"=>"Bad Request"}}
What is the problem?

Config file, logstash ruby filter event.get("message").match() Error

In the logstash config file I am trying to just get the XML data to parse.
This is my config file:
input {
file {
path => "/home/elastic-stack/logstash-7.3.2/event-data/telmetry.log"
start_position => "beginning"
type => "sandbox-out"
codec => multiline {
pattern => "^</datastore-contents-xml>"
negate => "true"
what => "next"
}
}
http {
host => "127.0.0.1"
port => 8080
type => "sandbox-out"
}
}
filter {
grok {
match => { "message" => "\[%{USER:host_name} %{IP:ip_address} %{USER:session-id} %{NUMBER:session-id-num}\]"}
}
grok {
match => { "message" => "\Subscription Id \: %{BASE16NUM:subcription-id:int}"}
}
grok {
match => { "message" => "\Event time \: %{TIMESTAMP_ISO8601:event-time}"}
}
grok {
match => {"message" => "\<%{USERNAME:Statistic}\>"}
}
mutate {
remove_field => ["headers", "host_name", "session-id","message"]
}
date {
match => ["timestamp","dd/MMM/yyyy:HH:mm:ss Z"]
}
ruby { code => 'event.set("justXml", event.get("message").match(/.+(<datastore-contents-xml.*)/m)[1])' }
xml {
#remove_namespaces => "true"
#not even the namspace option is working to access the http link
source => "justXml"
target => "xml-content"
#force_array => "false"
xpath => [
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='name']/text()" , "name" ,
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='total-memory']/text()" , "total-memory",
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='used-memory']/text()" , "used-memory",
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='free-memory']/text()" , "free-memory" ,
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='lowest-memory']/text()" , "lowest-memory" ,
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='highest-memory']/text()" , "highest-memory"
]
#logstash is not dectecting any of these xpaths in the config
}
mutate {
convert => {
"total-memory" => "integer"
"used-memory" => "integer"
"free-memory" => "integer"
"lowest-memory" => "integer"
"highest-memory" => "integer"
}
}
}
output {
stdout {
codec => rubydebug
}
file {
path => "%{type}_%{+dd_MM_yyyy}.log"
}
}
Desired output:
{
"ip_address" => "10.10.20.30",
"subcription-id" => 2147483650,
"event-time" => "2019-09-12 13:13:30.290000+00:00",
"host" => "127.0.0.1",
"Statistic" => "memory-statistic",
"type" => "sandbox-out",
"#version" => "1",
"#timestamp" => 2019-09-26T10:03:00.620Z,
"session-id-num" => "35"
"yang-model" => "http://cisco.com/ns/yang/Cisco-IOS-XE-memory-oper"
"name" => "Processor"
"total-memory" => 2238677360
"used-memory" => 340449924
"free-memory" => 1898227436
"lowest-usage" => 1897220640
"highest-usage" => 1264110388
}
ERROR:
[2019-09-27T09:18:55,622][ERROR][logstash.filters.ruby ] Ruby exception occurred: undefined method `match' for nil:NilClass
/home/elastic-stack/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
"ip_address" => "10.10.20.30",
"subcription-id" => 2147483650,
"session-id-num" => "35",
"tags" => [
[0] "_rubyexception"
],
"Statistic" => "memory-statistic",
"event-time" => "2019-09-12 13:13:30.290000+00:00",
"type" => "sandbox-out",
"#version" => "1",
"host" => "127.0.0.1",
"#timestamp" => 2019-09-27T07:18:54.868Z
By the error I can already know that the problem is with the ruby filter but I do not know how to resolve it.
This data generate by Cisco Telemetry and I am trying to ingest it using Elastic Stack.
The error seems to be that the event has no message field, so you cannot call match on a non existing thing.
I see you are calling match on the message field in this ruby code:
ruby { code => 'event.set("justXml", event.get("message").match(/.+(<datastore-contents-xml.*)/m)[1])' }
However you are removing the message field from the event a few lines earlier:
mutate {
remove_field => ["headers", "host_name", "session-id","message"]
}
Solution is to remove the message field only when you don't need it anymore, I would move the remove_field mutate to the end of the filter block.
One more suggestion if I may add. You have multiple grok filters running on the same, message field:
grok {
match => { "message" => "\[%{USER:host_name} %{IP:ip_address} %{USER:session-id} %{NUMBER:session-id-num}\]"}
}
grok {
match => { "message" => "\Subscription Id \: %{BASE16NUM:subcription-id:int}"}
}
grok {
match => { "message" => "\Event time \: %{TIMESTAMP_ISO8601:event-time}"}
}
grok {
match => {"message" => "\<%{USERNAME:Statistic}\>"}
}
This can be simplified into this (you can check to the Grok filter docs:
grok {
break_on_match => false,
match => {
"message" => [
"\[%{USER:host_name} %{IP:ip_address} %{USER:session-id} %{NUMBER:session-id-num}\]",
"\Subscription Id \: %{BASE16NUM:subcription-id:int}",
"\Event time \: %{TIMESTAMP_ISO8601:event-time}",
"\<%{USERNAME:Statistic}\>"
]
}
}
This way you need only one instance of the grok filter, as it will go through the patterns in the list and because of break_on_match=>false it will not finish after the first successful match, but will make sure to extract all fields it can based on all the patterns in the list.

Savon: Wrong Element Name Prefix. How can I change it?

So here is the code that I have:
resp = client.call(
:producer_query,
message: {
'cmn:Carrier' => '',
'cmn:ProducerCriteria' => { 'cmn:EntityType' => 'Individual',
'cmn:CustomerId' => 5555,
:attributes! => {'CustomerId' => {'type' => 'AGENTCD'}}},
'cmn:SectionConfiguration' => { 'cmn:SectionType' => 'Associations', :attributes! => {'cmn:SectionType' => {'activeOnly' => 'false'}}},
:attributes! => { 'cmn:Carrier' => { "id" => 55555 }}
}
) do
wsse_auth ENV['ID'], ENV['PASSWORD'], :digest
end
And it produces something that looks like:
<soapenv:Body>
<cmn:ProducerQuery>
<cmn:Carrier id="5555"/>
<cmn:ProducerCriteria>
<cmn:EntityType>Individual</cmn:EntityType>
<cmn:CustomerId>55555</cmn:CustomerId>
</cmn:ProducerCriteria>
<cmn:SectionConfiguration>
<cmn:SectionType activeOnly="false">Associations</cmn:SectionType>
</cmn:SectionConfiguration>
</cmn:ProducerQuery>
</soapenv:Body>
I've added in the 'cmn' to everything, but the ProducerQuery to make it match what I think it should be. However, I think it really should read 'tran' instead. I can control the 'cmn' for everything but the part that reads <cmn:ProducerQuery>. How can I make it read <cmn:ProducerQuery>?

Upload Captions YouTube Data API ruby

I am trying to upload captions to YouTube using the Data API. However I can't find in the reference or in the forum any example in Ruby. In specific how to send the actual caption file (xml).
body = {
:snippet => {
:videoId => videoId,
:language => "English",
:name => "English"
}
}
captions_insert_response = client.execute(
:api_method => youtube.captions.insert,
:parameters => {
:part => body.keys.join(',')
},
:body_object => body
)
where and how do I add the caption file? I tried doing it like uploading a video, but it didn't seem to work. This line was added after ":body_object"
:media => Google::APIClient::UploadIO.new(captions_file, 'text/xml')
Thanks
I solved the issue changing the language in the snippet to "en".
This is the complete request if someone needs it.
body = {
:snippet => {
:videoId => videoId,
:language => "en",
:name => "English"
}
}
captions_insert_response = client.execute(
:api_method => youtube.captions.insert,
:body_object => body,
:media => Google::APIClient::UploadIO.new(captions_file, 'text/xml'),
:parameters => {
'uploadType' => 'multipart',
:part => body.keys.join(',')
}
)

How do I use savon nested attributes! hash?

I'm looking at using Ruby savon for SOAP. For purely masochistic reasons I have to deal with SOAP elements having attributes.
So, no problem, there is an example on the savon docs site which highlights this ability:
{ :person => "Eve", :attributes! => { :person => { :id => 666 } } }.to_soap_xml
"<person id=\"666\">Eve</person>"
My problem is how do I set attributes on child elements, for example, say I add an address child element to person:
{ :person => {:address => ""}, :attributes! => { :person => { :id => 666 } } }.to_soap_xml
Now I want to add an id attribute to the address element:
It's no go if I nest address in the attributes hash:
{ :person => {:address => ""}, :attributes! => { :person => { :id => 666, :address => {:id => 44 }} }}.to_soap_xml
So my question is, how can I get this?
<person id=666><address id=44></address></person>
I ran across the issue of the previous answer no longer working. Eventually I found https://github.com/savonrb/savon/issues/518 which lead me to the correct syntax to add attributes now.
So the previous example would now be done as
{
:person => {
:#id => 666,
:address => {
:#id => 44
}
}
}
Which would generate the following xml
<person id="666">
<address id="44"/>
</person>
You were close - just needed to put the :attributes! key in the same hash that contains the value.
{
:person => {
:address => "",
:attributes! => { :address => { :id => 44 } }
},
:attributes! => { :person => { :id => 666 } }
}.to_soap_xml
# => "<person id=\"666\"><address id=\"44\"></address></person>"

Resources