I am using savon version 2.2 to call some SOAP services. For simple services everything is working OK. However one service has a complex structure like the example below with both repeating groups and attributes on each group:
<sch:RequestControl>
<sch:requestID>6989668868</sch:requestID>
</sch:RequestControl>
<sch:InquiryParam>
<sch:crmParam name="AccountNumber">1234567</sch:crmParam>
<sch:crmParam name="History">1</sch:crmParam>
</sch:InquiryParam>
My current message looks like this:
<RequestControl>
<requestID>6989668868</requestID>
</RequestControl>
<InquiryParam>
<crmParam>1234567</crmParam>
<attributes>
<crmParam>
<name>AccountNumber</name>
</crmParam>
</attributes>
</InquiryParam>
<InquiryParam>
<crmParam>1</crmParam>
<attributes>
<crmParam>
<name>History</name>
</crmParam>
</attributes>
</InquiryParam>
The above is produced using this logic:
message = { :RequestControl =>
{ :requestID => 6989668868 },
:InquiryParam => [
{ :crmParam => { :content! => #account_number } ,
:attributes => { "crmParam" => {"name" => "AccountNumber"} } },
{ :crmParam => { :content! => #history } ,
:attributes => { "crmParam" => {"name" => "History"} } } ]
}
I've tried various combinations using :crmParam => { :content! => #account_number, :attributes => {'name'=>'AccountNumber'} } and similar based on the savon and gyoku documentation but have run onto a brick wall in getting the XML to format like the example. I know I can brute force the message by assigning it to xml but that makes it difficult to see what's going on.
Can anyone suggest a fix to have the attributes inside the crmParam tags?
I'm not sure Savon handles a Ruby array as you'd like, but the following script should give you a better idea what you can do.
require 'savon'
c = Savon.client(endpoint: "http://www.example.com",
namespace: "urn:ns.example.com",
log: true,
log_level: :debug,
pretty_print_xml: true)
r = c.call(:call,
:message => {
:InquiryParam => [
{"crmParam" => 123,
:attributes! => { "crmParam" => { "name" => "AccountNumber" }}},
{"crmParam" => 456,
:attributes! => { "crmParam" => { "name" => "history" }}}
]
}
Related
I am trying to create an issue with jira-ruby in the terminal. So far I have done the following (where username, password, site and project have been replaced with the proper values). I have been able to fetch issues, but not to create them. Jira-ruby return false when i try and save an issue
options = {
:username => "username",
:password => "password",
:site => 'site',
:context_path => '',
:auth_type => :basic
}
client = JIRA::Client.new(options)
issue = client.Issue.build
issue.save({
"fields" => {
"summary" => "blarg from in example.rb",
"project" => {"key" => "mykey"},
"issuetype" => {"id" => "1"}
}
})
=> false
issue.attrs
=> {"errorMessages"=>[], "errors"=>{"issuetype"=>"issue type is required"}, "summary"=>"blarg from in example.rb", "key"=>"somekey", "id"=>"someid", "self"=>"site", "exception"=>{"class"=>"Net::HTTPBadRequest", "code"=>"400", "message"=>"Bad Request"}}
What is the problem?
In the logstash config file I am trying to just get the XML data to parse.
This is my config file:
input {
file {
path => "/home/elastic-stack/logstash-7.3.2/event-data/telmetry.log"
start_position => "beginning"
type => "sandbox-out"
codec => multiline {
pattern => "^</datastore-contents-xml>"
negate => "true"
what => "next"
}
}
http {
host => "127.0.0.1"
port => 8080
type => "sandbox-out"
}
}
filter {
grok {
match => { "message" => "\[%{USER:host_name} %{IP:ip_address} %{USER:session-id} %{NUMBER:session-id-num}\]"}
}
grok {
match => { "message" => "\Subscription Id \: %{BASE16NUM:subcription-id:int}"}
}
grok {
match => { "message" => "\Event time \: %{TIMESTAMP_ISO8601:event-time}"}
}
grok {
match => {"message" => "\<%{USERNAME:Statistic}\>"}
}
mutate {
remove_field => ["headers", "host_name", "session-id","message"]
}
date {
match => ["timestamp","dd/MMM/yyyy:HH:mm:ss Z"]
}
ruby { code => 'event.set("justXml", event.get("message").match(/.+(<datastore-contents-xml.*)/m)[1])' }
xml {
#remove_namespaces => "true"
#not even the namspace option is working to access the http link
source => "justXml"
target => "xml-content"
#force_array => "false"
xpath => [
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='name']/text()" , "name" ,
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='total-memory']/text()" , "total-memory",
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='used-memory']/text()" , "used-memory",
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='free-memory']/text()" , "free-memory" ,
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='lowest-memory']/text()" , "lowest-memory" ,
"//*[name()='datastore-contents-xml']/*[name()='memory-statistics']/*[name()='memory-statistic'][1]/*[name()='highest-memory']/text()" , "highest-memory"
]
#logstash is not dectecting any of these xpaths in the config
}
mutate {
convert => {
"total-memory" => "integer"
"used-memory" => "integer"
"free-memory" => "integer"
"lowest-memory" => "integer"
"highest-memory" => "integer"
}
}
}
output {
stdout {
codec => rubydebug
}
file {
path => "%{type}_%{+dd_MM_yyyy}.log"
}
}
Desired output:
{
"ip_address" => "10.10.20.30",
"subcription-id" => 2147483650,
"event-time" => "2019-09-12 13:13:30.290000+00:00",
"host" => "127.0.0.1",
"Statistic" => "memory-statistic",
"type" => "sandbox-out",
"#version" => "1",
"#timestamp" => 2019-09-26T10:03:00.620Z,
"session-id-num" => "35"
"yang-model" => "http://cisco.com/ns/yang/Cisco-IOS-XE-memory-oper"
"name" => "Processor"
"total-memory" => 2238677360
"used-memory" => 340449924
"free-memory" => 1898227436
"lowest-usage" => 1897220640
"highest-usage" => 1264110388
}
ERROR:
[2019-09-27T09:18:55,622][ERROR][logstash.filters.ruby ] Ruby exception occurred: undefined method `match' for nil:NilClass
/home/elastic-stack/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
"ip_address" => "10.10.20.30",
"subcription-id" => 2147483650,
"session-id-num" => "35",
"tags" => [
[0] "_rubyexception"
],
"Statistic" => "memory-statistic",
"event-time" => "2019-09-12 13:13:30.290000+00:00",
"type" => "sandbox-out",
"#version" => "1",
"host" => "127.0.0.1",
"#timestamp" => 2019-09-27T07:18:54.868Z
By the error I can already know that the problem is with the ruby filter but I do not know how to resolve it.
This data generate by Cisco Telemetry and I am trying to ingest it using Elastic Stack.
The error seems to be that the event has no message field, so you cannot call match on a non existing thing.
I see you are calling match on the message field in this ruby code:
ruby { code => 'event.set("justXml", event.get("message").match(/.+(<datastore-contents-xml.*)/m)[1])' }
However you are removing the message field from the event a few lines earlier:
mutate {
remove_field => ["headers", "host_name", "session-id","message"]
}
Solution is to remove the message field only when you don't need it anymore, I would move the remove_field mutate to the end of the filter block.
One more suggestion if I may add. You have multiple grok filters running on the same, message field:
grok {
match => { "message" => "\[%{USER:host_name} %{IP:ip_address} %{USER:session-id} %{NUMBER:session-id-num}\]"}
}
grok {
match => { "message" => "\Subscription Id \: %{BASE16NUM:subcription-id:int}"}
}
grok {
match => { "message" => "\Event time \: %{TIMESTAMP_ISO8601:event-time}"}
}
grok {
match => {"message" => "\<%{USERNAME:Statistic}\>"}
}
This can be simplified into this (you can check to the Grok filter docs:
grok {
break_on_match => false,
match => {
"message" => [
"\[%{USER:host_name} %{IP:ip_address} %{USER:session-id} %{NUMBER:session-id-num}\]",
"\Subscription Id \: %{BASE16NUM:subcription-id:int}",
"\Event time \: %{TIMESTAMP_ISO8601:event-time}",
"\<%{USERNAME:Statistic}\>"
]
}
}
This way you need only one instance of the grok filter, as it will go through the patterns in the list and because of break_on_match=>false it will not finish after the first successful match, but will make sure to extract all fields it can based on all the patterns in the list.
So here is the code that I have:
resp = client.call(
:producer_query,
message: {
'cmn:Carrier' => '',
'cmn:ProducerCriteria' => { 'cmn:EntityType' => 'Individual',
'cmn:CustomerId' => 5555,
:attributes! => {'CustomerId' => {'type' => 'AGENTCD'}}},
'cmn:SectionConfiguration' => { 'cmn:SectionType' => 'Associations', :attributes! => {'cmn:SectionType' => {'activeOnly' => 'false'}}},
:attributes! => { 'cmn:Carrier' => { "id" => 55555 }}
}
) do
wsse_auth ENV['ID'], ENV['PASSWORD'], :digest
end
And it produces something that looks like:
<soapenv:Body>
<cmn:ProducerQuery>
<cmn:Carrier id="5555"/>
<cmn:ProducerCriteria>
<cmn:EntityType>Individual</cmn:EntityType>
<cmn:CustomerId>55555</cmn:CustomerId>
</cmn:ProducerCriteria>
<cmn:SectionConfiguration>
<cmn:SectionType activeOnly="false">Associations</cmn:SectionType>
</cmn:SectionConfiguration>
</cmn:ProducerQuery>
</soapenv:Body>
I've added in the 'cmn' to everything, but the ProducerQuery to make it match what I think it should be. However, I think it really should read 'tran' instead. I can control the 'cmn' for everything but the part that reads <cmn:ProducerQuery>. How can I make it read <cmn:ProducerQuery>?
I am trying to upload captions to YouTube using the Data API. However I can't find in the reference or in the forum any example in Ruby. In specific how to send the actual caption file (xml).
body = {
:snippet => {
:videoId => videoId,
:language => "English",
:name => "English"
}
}
captions_insert_response = client.execute(
:api_method => youtube.captions.insert,
:parameters => {
:part => body.keys.join(',')
},
:body_object => body
)
where and how do I add the caption file? I tried doing it like uploading a video, but it didn't seem to work. This line was added after ":body_object"
:media => Google::APIClient::UploadIO.new(captions_file, 'text/xml')
Thanks
I solved the issue changing the language in the snippet to "en".
This is the complete request if someone needs it.
body = {
:snippet => {
:videoId => videoId,
:language => "en",
:name => "English"
}
}
captions_insert_response = client.execute(
:api_method => youtube.captions.insert,
:body_object => body,
:media => Google::APIClient::UploadIO.new(captions_file, 'text/xml'),
:parameters => {
'uploadType' => 'multipart',
:part => body.keys.join(',')
}
)
I'm looking at using Ruby savon for SOAP. For purely masochistic reasons I have to deal with SOAP elements having attributes.
So, no problem, there is an example on the savon docs site which highlights this ability:
{ :person => "Eve", :attributes! => { :person => { :id => 666 } } }.to_soap_xml
"<person id=\"666\">Eve</person>"
My problem is how do I set attributes on child elements, for example, say I add an address child element to person:
{ :person => {:address => ""}, :attributes! => { :person => { :id => 666 } } }.to_soap_xml
Now I want to add an id attribute to the address element:
It's no go if I nest address in the attributes hash:
{ :person => {:address => ""}, :attributes! => { :person => { :id => 666, :address => {:id => 44 }} }}.to_soap_xml
So my question is, how can I get this?
<person id=666><address id=44></address></person>
I ran across the issue of the previous answer no longer working. Eventually I found https://github.com/savonrb/savon/issues/518 which lead me to the correct syntax to add attributes now.
So the previous example would now be done as
{
:person => {
:#id => 666,
:address => {
:#id => 44
}
}
}
Which would generate the following xml
<person id="666">
<address id="44"/>
</person>
You were close - just needed to put the :attributes! key in the same hash that contains the value.
{
:person => {
:address => "",
:attributes! => { :address => { :id => 44 } }
},
:attributes! => { :person => { :id => 666 } }
}.to_soap_xml
# => "<person id=\"666\"><address id=\"44\"></address></person>"