Ruby to_yaml stringifies my json - ruby

I am trying to convert a ruby hash to yaml. I'd like part of the hash be valid json; however, when I try to serialize the json string, it is converted to yaml in quotes.
For example, when I just have a simple string, the ouput is as follows (note foo is not in quotations):
request = {}
request['body'] = 'foo'
request.to_yaml # outputs: body: foo
However, when I add something to the beginning of the string, like { foo the output for body gets quoted:
request['body'] = '{ foo'
request.to_yaml # outputs: body: '{ foo'
How can I get around this? I've tried JSON.parse and, though that make work, I can't be guaranteed that this input will actually be json (could be xml, etc...) -- I just want to give back whatever was given to me but not "stringified".
Basically, I want to give an object that looks like:
{ 'request' => {
'url' => '/posts',
'method' => 'GET',
'headers' => [
'Content-Type' => 'application/json'
]
},
'response' => {
'code' => 200,
'body' => '[{"id":"ef4b3a","title":"this is the title"},{"id":"a98c4f","title":"title of the second post"}]'
}
}
Which returns:
request:
url: /posts
method: GET
headers:
- Content-Type: application/json
response:
code: 200
body:
[{"id":"ef4b3a","title":"this is the title"},{"id":"a98c4f","title":"title of the second post"}]
The reason being: right now, I can go from yaml to the correct ruby hash but I can't go the other way.

The method my_hash.to_yaml() just takes a hash and converts it to YAML without doing anything special to the values. The method does not care whether your string is JSON or XML, it just treats it as a string.
So why is my JSON being put into quotes when other strings aren't?
Good question! The reason is simple: curly braces are a valid part of YAML syntax.
This:
my_key: { sub: 1, keys: 2}
Is called flow mapping syntax in YAML, and it allows you make nested mappings in one line. To escape strings which have curly braces in them, YAML uses quotes:
my_key: "{ sub: 1, keys: 2}" # this is just a string
Of course, the quotes are optional for all strings:
my_key: "foo" #same as my_key: foo
Okay, but I want to_yaml() to find my JSON string and convert it to YAML mappings like the rest of the hash.
Well then, you need to convert your JSON string to a hash like the rest of your hash. to_yaml() converts a hash to YAML. It doesn't convert strings to YAML. The proper method for doing this is to use JSON.parse, as you mentioned:
request['body'] = JSON.parse( '{"id":"ef4b3a"}' )
But the string might not be JSON! It might be XML or some other smelly string.
This is exactly why to_yaml() doesn't convert strings. A wise programmer once told me: "Strings are strings. Strings are not data structures. Strings are strings."
If you want to convert a string into a data structure, you need to validate it and parse it. Because there's no guarantee that a string will be valid, it's your responsibility as a programmer to determine whether your data is JSON or XML or just bad, and to decide how you want to respond to each bit of data.
Since it looks like you're parsing web pages, you might want to consider using the same bit of data other web clients use to parse these things:
{ 'request' => {
'url' => '/posts',
'method' => 'GET',
'headers' => [
'Content-Type' => 'application/json' #<== this guy, right here!
]
},
'response' => {
'code' => 200,
'body' => '[{"id":"ef4b3a","title":"this is the title"},{"id":"a98c4f","title":"title of the second post"}]'
}
}
If the content-type doesn't agree with the body then you should throw an error because your input data is bad.

The reason '{ foo' requires quote is because this is part of the YAML specification 7.3.3 Plain Style.
Excerpt
Plain scalars must never contain the “: ” and “#” character combinations. Such combinations would cause ambiguity with mapping key: value pairs and comments. In addition, inside flow collections, or when used as implicit keys, plain scalars must
not contain the “[”, “]”, “{”, “}” and “,” characters. These characters would cause ambiguity with flow collection structures.
Based on the above even your stated "return" value is incorrect and the body is probably enclosed in single quotes e.g.
response:
code: 200
body: '[{"id":"ef4b3a","title":"this is the title"},{"id":"a98c4f","title":"title of the second post"}]'
Otherwise it would create ambiguity with "Flow Sequences" ([,]) and "Flow Mappings" ({,}).
If you would like result of the JSON, XML or other notation language to be represented appropriately (read objectively) then you will need to determine the correct parser (may be from the "Content-Type") and parse it before converting it YAML

Related

WebMock stub_request with hash_including -- matching only part of json

I wold like to test whether a request contained some inner value in json. Let assume the request body is:
level1: {
level2a: "value1",
level2b: "value2",
...
},
...
}
Then I have the following code:
WebMock.stub_request(:post, "http://127.0.0.1:/path")
.with(body: hash_including({level1: {level2a: 'value1'}}))
.to_return(status: 200, body: '{}', headers: {})
The code above does not work unfortunately. I've been hoping it'll work with matching only sub-set of json.
According to the doc: Matches a hash that includes the specified key(s) or key/value pairs. Ignores any additional keys. I am not sure if it works the way I want.
The request is too big here though for quoting it all.
Any ideas for why the test is not working and how to fix or rewrite it?

Ruby string interpolation with substitution

I have a given method that adds keys to urls with:
url % {:key => key}
But for one url I need the key to be escaped with CGI.escape. I cannot change the method, I can only change the url, but substitution does not work:
"https://www.example.com?search=#{CGI.escape(%{key})}"
Is there a way to achieve this only by changing the url string? I cannot use additional variables or change the method, thus I cannot do the escaping in the method and send the escaped key to the url string.
It isn't clear how your given method is supposed to work. Can you give an example where the method works, and one where it doesn't? Ignoring the method part of your question, and focusing on the URL bit,
>> key = "Baby Yoda"
=> "Baby Yoda"
>> %{key}
=> "key"
is the expected result, regardless of whether you have a variable named key, set to any value. See: https://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Literals#The_.25_Notation
Unless you have a method defined which overloads '%' to do something else special for URLs, but that isn't clear in your question.
If you just want to CGI escape the value of 'key' within your URL string, don't use the percent notation:
>> key = 'Baby Yoda'
=> "Baby Yoda"
>> "https://www.example.com?search=#{CGI.escape(key)}"
=> "https://www.example.com?search=Baby+Yoda"
It just seems not possible. I worked around by defining a syntax ${...}
"https://www.example.com?search=${CGI.escape(%{key})}"
Then I first do subtitution of %{key} and then use eval to do CGI.Escape (or any method for that matter) with
gsub(/\${(.+?)}/) { |e| eval($1) }

Adding a variable to URI

I'm pulling an integer from a JSON response, and then using that integer as the upper-limit in a loop:
data = response.parsed_response["meta"]
pages = data['total']
(1..pages.to_i).step(1) do |page_num|
response2 = client.get('/item?page=#{page_num}&per_page=1', :headers => {'Authorization' => auth, 'Content-Type' => 'application/json'})
However, I seem to be unable to pass the variable into the URI in a way that is usable with HTTParty:
/opt/chef/embedded/lib/ruby/1.9.1/uri/common.rb:176:in `split': bad URI(is not URI?): /items?page=#{page_num}&per_page=1 (URI::InvalidURIError)
I tried other methods ('/item?page=' + page_num + '&...', for example) that were, likewise unsuccessful. I'm not where what I'm overlooking, but is there another, more correct way of doing this.
The output is being returned as an integer, and is usable, but it seems not to be getting passed into the URI string above.
I came across this thread:
URI with variable
but I was unsure that this applied, and an attempt to adapt the solution was unsuccessful (undefined method `strip' for 10:Fixnum (NoMethodError))
Instead of:
'/item?page=#{page_num}&per_page=1'
Use:
"/item?page=#{page_num}&per_page=1"
Strings with single-quotation marks don't allow interpolation; thus to interpolate, you must use double-quotation marks strings.
And just FYI: When it comes to an integer range, you need not specify "step(1)" as it is the default value.

how do I parse out this URL-encoded string with a JSON array using Ruby?

When I use a webhook with Mandrill and post to my Iron Worker, I get the following Raw (this is from RequestBin, as well) -- I didn't include the whole payload, just an example:
puts payload =>
mandrill_events=%5B%7B%22event%22%3A%22inbound%22%2C%22msg%22%3A%7B%22dkim%22%3A%7B%22signed%22%3Atrue%2C%22valid%22%3Atrue%7D%2C%22email%22%3A%22kaya%40hellokaya.com%22%2C%22from_email%22%3A%22example.sender%40mandrillapp.com%22%2C%22headers%22%3A%7B%22Content-Type%22%3A%22multipart%5C%2Falternative%3B+boundary%3D%5C%22_av-7r7zDhHxVEAo2yMWasfuFw%5C%22%22%2C%22Date%22%3A%22Fri%2C+10+May+2013+19%3A28%3A20+%2B0000%22%2C%22Dkim-Signature%22%3A%5B%22v%3D1%3B+a%3Drsa-
I tried to extract the value of the parameter mandrill_events using:
puts params = CGI::parse(#payload) =>
{"mandrill_events"=>["[{\"event\":\"inbound\",\"msg\":{\"dkim\":{\"signed\":true,\"valid\":true},\"email\":\"kaya#hellokaya.com\",\"from_email\":\"example.sender#mandrillapp.com\",\"headers\":{\"Content-Type\":\"multipart\\/alternative; boundary=\\\"_av-7r7zDhHxVEAo2yMWasfuFw\\\"\",\"Date\":\"Fri, 10 May 2013 19:28:20 +0000\",\"Dkim-Signature\":[\"v=1; a=rsa-sha1; c=relaxed\\/relaxed; s=mandrill; d=mail115.us4.mandrillapp.com; h=From:Sender:Subject:List-Unsubscribe:To:Message-Id:Date:MIME-Version:Content-Type; i=example.sender#mail115.us4.mandrillapp.com;
Then I am stuck. I want to extract the email value in the JSON.array.
I thought to try the following;
puts json_params = JSON.parse(params)
But I am now feeling there must be a better way....
How can I extract the elements from the JSON array in this URL-encoded string?

Setting a string containing a variable to be checked when string is used - Ruby 1.9.3

Apologies in advance for what I think might be a stupid question. I promise I have searched, but I'm not sure if I've searched for the correct things. I'm a very new programmer, Ruby is my first serious language, and I taught myself it over the past few months. I've also never asked a question on StackOverflow, so hopefully this is acceptably clear and meets the other requirements for question-asking. Here is the question:
I have a branching method that calls various different APIs and various different URLs within them, based upon the values passed to the method. As is, it looks like:
if api == x
url = "http://url.x.com/api/#{#variable}"
elsif api == y
url = "http://url.y.com/api/public/#{#var1}_#{#var2}/#{#variable}"
etc.
The url's being called are different for each operation, as are the necessary variables. The variables used in the requests are being created as instance variables at the beginning of the method. The possible values of the instance variables are stored in a large hash or are being passed into the method by the call itself. The hash is structured like:
$globalhash =
{
"api_1" =>
{
"call_type_1" => {"info_1" => "", "info_2" => ""},
"call_type_2" => {"info_1" => "", "info_2" => ""}
},
"api_2" =>
{
"call_type_1" => {"info_1" => "", "info_2" => ""},
"call_type_2" => {"info_1" => "", "info_2" => ""}
}
}
The problem I have is that this branching section goes on for a long time in the code--partially because I've done it suboptimally, I'm sure. I'm also sure that my code would be much more efficient if the branching section didn't exist. Ideally, instead of the branching section, I'd like to make this happen:
url = $globalhash[#api][#call_type]["url"]
The value that pulls would be a URL specific to the call type and the api--the address, formatting, and various other differences included. So some values would be:
http://api.x.com/#var1_#var2/#variable
http://api.y.com/public/#variable
and so on. So the structures vary, and I need to access the instance variables stored within the method call, but I need to do so dynamically. The issue I've had is that every way I've tried to implement this results in the values of the instance variables in the strings for "url" being set when $globalhash is read as the program begins to run, with them all being nil. I need it to check the variable when the request is being made, and not before, basically. I have a similar (same?) issue with setting the post_data for the request--if I could find a way to set it in the hash, I'd have cleaner code that runs faster. The post_data is as:
post_data = {'value' => #var1, 'value2' => #var2, 'value3' => #var3}
and so on. The post_data is different for each API and for each call_type; the values are different as are the requested variables for them. I'd like to implement a set of key-value pairs in the hash that look vaguely like:
$globalhash = {"api_1" => {"call_type_1" => {"url" => "http://api.x.com/#{#variable}", "post_data" => "{'value' => #var1, 'value2' => #var2, etc.}"}}}
Except, of course, it would need to work--the variables it needs to reference are nil when $globalhash is being read. I'm not sure how best to solve this.
The program works as-is, but I have a very derpy-looking ~80 lines of branching code that figures out the structure of the URL and the post_data, and then requests it.
The only solution I've considered is creating an instance hash within the method to replace the branching code, that only gets created after declaring the instance variables. However, I'm afraid that would create a similar problem to the current one, of it being inefficient to create a huge new hash every time.
Thanks for reading, and I appreciate your help. I hope the pseudo-code is acceptable, I found it the best way to explain my question.
EDIT: Found a solution. Code:
url = eval "\"#{$globalhash["api_1"]["call_type_1"]["url"]}\""
["url"] references 'http://api.x.com/#{#variable}', single quotes prevent interpolation within the hash, eval function puts it in double quotes and string interpolation collects the instance variable at the time it is called. It's an ugly methodology BUT it does let me greatly shorten my code. Source: https://www.ruby-forum.com/topic/3237101
Instead of creating the hash using the instance variables...
post_data = {'value' => #var1, 'value2' => #var2, 'value3' => #var3}
...you could initialize the hash using strings representing the instance variables.
post_data = {'value' => '#var1', 'value2' => '#var2', 'value3' => '#var3'}
Then when you reference the hash variable, you would wrap the call in an eval.
post_data = {'value' => '#var1', 'value2' => '#var2', 'value3' => '#var3'}
#var2 = 'this is test data'
eval(post_data['value2'])
=> "this is test data"
Similarly I would use placeholders in the url...
url = 'http://api.x.com/[#variable]'
#variable = "cotton_tree"
my_url = url.dup
my_url.sub!(/\[.*?\]/, eval(my_url.match(/\[(.*?)\]/)[1])) while my_url.match(/\[(.*?)\]/)
p my_url
=> "http://api.x.com/cotton_tree"

Resources