Sending array variables using HTTParty - ruby

I'm sending a POST request using HTTParty. One of the variables required is an array. This is the code I'm using to send:
response = HTTParty.post url, :body =>
{"key"=>'XYZ123',
"content"=>
[{"placename"=>"placeholder",
"placecontent"=>"sample content"}],
etc. }
The API needs to see:
"content": [
{
"placename": "placeholder",
"placecontent": "sample content"
}
],
However, when I check the request received logs on the API, I see that my code is producing:
"content": [
{
"placename": "placeholder"
},
{
"placecontent": "sample content"
}
],
How can I stop the array record from being split in two?
Thanks.
EDIT:
The desired output of the code is the equivalent of:
...&content[0][placename]=placeholder&content[0][placecontent]=sample%20content...

By default, HTTParty uses HashConversions to convert a Hash body to parameters:
Examples:
{ :name => "Bob",
:address => {
:street => '111 Ruby Ave.',
:city => 'Ruby Central',
:phones => ['111-111-1111', '222-222-2222']
}
}.to_params
#=> "name=Bob&address[city]=Ruby Central&address[phones][]=111-111-1111&address[phones][]=222-222-2222&address[street]=111
Ruby Ave."
You can override this with your own convertor by using HTTParty.query_string_normalizer:
Override the way query strings are normalized. Helpful for overriding
the default rails normalization of Array queries.
For a query:
get '/', :query => {:selected_ids => [1,2,3]}
The default query string normalizer returns:
/?selected_ids[]=1&selected_ids[]=2&selected_ids[]=3
Let’s change it to this:
/?selected_ids=1&selected_ids=2&selected_ids=3
Pass a Proc to the query normalizer which accepts the yielded query.
#example Modifying Array query strings
class ServiceWrapper
include HTTParty
query_string_normalizer proc { |query|
query.map do |key, value|
value.map {|v| "#{key}=#{v}"}
end.join('&')
}
end
#param [Proc] normalizer custom query string normalizer. #yield [Hash,
String] query string #yieldreturn [Array] an array that will later be
joined with ‘&’
or simply pass it in your options:
response = HTTParty.post url, :body =>
{"key"=>'XYZ123',
"content"=>
[{"placename"=>"placeholder",
"placecontent"=>"sample content"}]},
:query_string_normalizer => -> (h) { ... your own implementation here ...}
To get a serialization of a[1]=val1&a[2]=val2 instead of a[]=val1&a[]=val2 you can create your own HashConversions based on the current one
class MyHashConversions
def to_params(hash)
params = hash.map { |k,v| normalize_param(k,v) }.join
params.chop! # trailing &
params
end
def normalize_param(key, value)
param = ''
stack = []
if value.is_a?(Array)
#### THE CHANGE IS HERE
param << value.each_with_index.map { |element, i| normalize_param("#{key}[#{i}]", element) }.join
####
elsif value.is_a?(Hash)
stack << [key,value]
else
param << "#{key}=#{URI.encode(value.to_s, Regexp.new("[^#{URI::PATTERN::UNRESERVED}]"))}&"
end
stack.each do |parent, hash|
hash.each do |k, v|
if v.is_a?(Hash)
stack << ["#{parent}[#{k}]", v]
else
param << normalize_param("#{parent}[#{k}]", v)
end
end
end
param
end
end
The code above is not tested, but if it works, and is generic enough, you might consider forking the project and github, making the fix there, so it will work out of the box :)

Related

Converting Ruby Hash into string with escapes

I have a Hash which needs to be converted in a String with escaped characters.
{name: "fakename"}
and should end up like this:
'name:\'fakename\'
I don't know how this type of string is called. Maybe there is an already existing method, which I simply don't know...
At the end I would do something like this:
name = {name: "fakename"}
metadata = {}
metadata['foo'] = 'bar'
"#{name} AND #{metadata}"
which ends up in that:
'name:\'fakename\' AND metadata[\'foo\']:\'bar\''
Context: This query a requirement to search Stripe API: https://stripe.com/docs/api/customers/search
If possible I would use Stripe's gem.
In case you can't use it, this piece of code extracted from the gem should help you encode the query parameters.
require 'cgi'
# Copied from here: https://github.com/stripe/stripe-ruby/blob/a06b1477e7c28f299222de454fa387e53bfd2c66/lib/stripe/util.rb
class Util
def self.flatten_params(params, parent_key = nil)
result = []
# do not sort the final output because arrays (and arrays of hashes
# especially) can be order sensitive, but do sort incoming parameters
params.each do |key, value|
calculated_key = parent_key ? "#{parent_key}[#{key}]" : key.to_s
if value.is_a?(Hash)
result += flatten_params(value, calculated_key)
elsif value.is_a?(Array)
result += flatten_params_array(value, calculated_key)
else
result << [calculated_key, value]
end
end
result
end
def self.flatten_params_array(value, calculated_key)
result = []
value.each_with_index do |elem, i|
if elem.is_a?(Hash)
result += flatten_params(elem, "#{calculated_key}[#{i}]")
elsif elem.is_a?(Array)
result += flatten_params_array(elem, calculated_key)
else
result << ["#{calculated_key}[#{i}]", elem]
end
end
result
end
def self.url_encode(key)
CGI.escape(key.to_s).
# Don't use strict form encoding by changing the square bracket control
# characters back to their literals. This is fine by the server, and
# makes these parameter strings easier to read.
gsub("%5B", "[").gsub("%5D", "]")
end
end
params = { name: 'fakename', metadata: { foo: 'bar' } }
Util.flatten_params(params).map { |k, v| "#{Util.url_encode(k)}=#{Util.url_encode(v)}" }.join("&")
I use it now with that string, which works... Quite straigt forward:
"email:\'#{email}\'"
email = "test#test.com"
key = "foo"
value = "bar"
["email:\'#{email}\'", "metadata[\'#{key}\']:\'#{value}\'"].join(" AND ")
=> "email:'test#test.com' AND metadata['foo']:'bar'"
which is accepted by Stripe API

Ruby - Safely navigated hash when a key could be a string or further hash

I'm stuck trying to safely navigate a hash from json.
The json could have a string, eg:
or it could be further nested:
h1 = { location: { formatted: 'Australia', code: 'AU' } }
h2 = { location: 'Australia' }
h2.dig('location', 'formatted')
Then String does not have #dig method
Basically I'm trying to load the JSON then populate the rails model with the data available which may be optional. It seems backwards to check every nested step with an if.
Hash#dig has no magic. It reduces the arguments recursively calling Hash#[] on what was returned from the previous call.
h1 = { location: { formatted: 'Australia', code: 'AU' } }
h1.dig :location, :code
#⇒ "AU"
It works, because h1[:location] had returned a hash.
h2 = { location: 'Australia' }
h2.dig :location, :code
It raises, because h2[:location] had returned a String.
That said, the solution would be to reimplement Hash#dig, as usually :)
Explicitly taking into account that it’s extremely trivial. Just take a list of keys to dig and (surprise) reduce, returning either the value, or nil.
%i|location code|.reduce(h2) do |acc, e|
acc.is_a?(Hash) ? acc[e] : nil
end
#⇒ nil
%i|location code|.reduce(h1) do |acc, e|
acc.is_a?(Hash) ? acc[e] : nil
end
#⇒ "AU"
Shameless plug. You might find the gem iteraptor I had created for this exact purpose useful.
You can use a simple piece of code like that:
def nested_value(hash, key)
return hash if key == ''
keys = key.split('.')
value = hash[keys.first] || hash[keys.first.to_sym]
return value unless value.is_a?(Hash)
nested_value(value, keys[1..-1].join('.'))
end
h1 = { location: { formatted: 'Australia', code: 'AU' } }
h2 = { 'location' => 'Australia' }
p nested_value(h1, 'location.formatted') # => Australia
p nested_value(h2, 'location.formatted') # => Australia
You can also use that method for getting any nested value of a hash by providing key in format foo.bar.baz.qux. Also the method doesn't worry whether a hash has string keys or symbol keys.
I don't know if this lead to the expected behaviour (see examples below) but you can define a patch for the Hash class as follow:
module MyHashPatch
def safe_dig(params) # ok, call as you like..
tmp = self
res = nil
params.each do |param|
if (tmp.is_a? Hash) && (tmp.has_key? param)
tmp = tmp[param]
res = tmp
else
break
end
end
res
end
end
Hash.include MyHashPatch
Then test on your hashes:
h1 = { location: { formatted: 'Australia', code: 'AU' } }
h2 = { location: 'Australia' }
h1.safe_dig([:location, :formatted]) #=> "Australia"
h2.safe_dig([:location, :formatted]) #=> "Australia"
h1.safe_dig([:location, :code]) #=> "AU"
h2.safe_dig([:location, :code]) #=> "Australia"

Deep Convert OpenStruct to JSON

I have an OpenStruct that is nested with many other OpenStructs. What's the best way to deeply convert them all to JSON?
Ideally:
x = OpenStruct.new
x.y = OpenStruct.new
x.y.z = OpenStruct.new
z = 'hello'
x.to_json
// {y: z: 'hello'}
Reality
{ <OpenStruct= ....> }
There is no default methods to accomplish such task because the built-in #to_hash returns the Hash representation but it doesn't deep converts the values.
If a value is an OpenStruct, it's returned as such and it's not converted into an Hash.
However, this is not that complicated to solve. You can create a method that traverses each key/value in an OpenStruct instance (e.g. using each_pair), recursively descends into the nested OpenStructs if the value is an OpenStruct and returns an Hash of just Ruby basic types.
Such Hash can then easily be serialized using either .to_json or JSON.dump(hash).
This is a very quick example, with an update from #Yuval Rimar for arrays of OpenStructs:
def openstruct_to_hash(object, hash = {})
case object
when OpenStruct then
object.each_pair do |key, value|
hash[key] = openstruct_to_hash(value)
end
hash
when Array then
object.map { |v| openstruct_to_hash(v) }
else object
end
end
openstruct_to_hash(OpenStruct.new(foo: 1, bar: OpenStruct.new(baz: 2)))
# => {:foo=>1, :bar=>{:baz=>2}}
Fixes to above solution to handle arrays
def open_struct_to_hash(object, hash = {})
object.each_pair do |key, value|
hash[key] = case value
when OpenStruct then open_struct_to_hash(value)
when Array then value.map { |v| open_struct_to_hash(v) }
else value
end
end
hash
end
Here's yet another approach, modified from lancegatlin's answer. Also adding the method to the OpenStruct class itself.
class OpenStruct
def deep_to_h
each_pair.map do |key, value|
[
key,
case value
when OpenStruct then value.deep_to_h
when Array then value.map {|el| el === OpenStruct ? el.deep_to_h : el}
else value
end
]
end.to_h
end
in initializers/open_struct.rb:
require 'ostruct'
# Because #table is a instance variable of OpenStruct and Object#as_json returns Hash of instance variables.
class OpenStruct
def as_json(options = nil)
#table.as_json(options)
end
end
Usage:
OpenStruct.new({ a: { b: 123 } }).as_json
# Result
{
"a" => {
"b" => 123
}
}
Edit:
This seems to do almost the same thing (notice the keys are symbols instead of strings)
OpenStruct.new({ a: { b: 123 } }).marshal_dump
# Result
{
:a => {
:b => 123
}
}
Same function that can accept arrays as an input too
def openstruct_to_hash(object, hash = {})
case object
when OpenStruct then
object.each_pair do |key, value|
hash[key] = openstruct_to_hash(value)
end
hash
when Array then
object.map { |v| openstruct_to_hash(v) }
else object
end
end
None of the above worked for me with a copy and paste. Adding this solution:
require 'json'
class OpenStruct
def deep_to_h
each_pair.map do |key, value|
[
key,
case value
when OpenStruct then value.deep_to_h
when Array then value.map {|el| el.class == OpenStruct ? el.deep_to_h : el}
else value
end
]
end.to_h
end
end
json=<<HERE
{
"string": "fooval",
"string_array": [
"arrayval"
],
"int": 2,
"hash_array": [
{
"string": "barval",
"string2": "bazval"
},
{
"string": "barval2",
"string2": "bazval2"
}
]
}
HERE
os = JSON.parse(json, object_class: OpenStruct)
puts JSON.pretty_generate os.to_h
puts JSON.pretty_generate os.deep_to_h

Ruby map a hash and extract the keys with values if nested or not

I’m using the PaperTrail gem to track changes in my Job model. Each time I change a record, the object is serialized and stored inside a changeset field on my Version model.
To display the values that have changed, I’m deserializing the object and iterating through it using map. I need help creating a method that extracts all the keys and values from this hash, regardless of whether they’re nested or not. The nesting will never be more than one level deep.
Here is my current code:
# deserializing the object into a hash
json_string = v.changeset.to_json
serialized_json = JSON.parse(json_string)
# current method to extract keys with values, but doesn’t pick up nested ones
serialized_json.map { |k,v|
puts k.titleize
puts v[1].present? ? v[1] : '<empty>'
}
# serialized_json (1st output)
{
"name" => [
[0] nil,
[1] "Epping Forest"
],
"job_address" => [
[0] nil,
[1] "51 Epping Forest, Essex, ES1 SAD"
]
}
# serialized_json (2nd output)
{
"quote" => [
[0] {
"quote" => {
"url" => nil
}
},
[1] {
"quote" => {
"url" => "/tmp/uploads/1433181671-22986-9554/file.pdf"
}
}
]
}
The values I’d like to extract from my method are:
"Name"
"Epping Forest"
"Quote Url"
"/tmp/uploads/1433181671-22986-9554/file.pdf"
In your below code, I assume you're putting [0] and [1] as index numbers to help the readers of your code, as it is not valid Ruby syntax.
"name" => [
[0] nil,
[1] "Epping Forest"
],
As for extracting 1 level deep keys and values, since it's a bit unclear to me how you would like your output formatted, I'ma ssuming a 1-D array is fine...in which case, something like this would give you the keys and values:
serialized_json hash w/valid Ruby syntax
{
"name" => [
nil,
"Epping Forest"
],
"job_address" => [
nil,
"51 Epping Forest, Essex, ES1 SAD"
]
}
Then:
serialized_json.map { |k, v| [k, v] }.flatten.reject { |x| x.nil? }
outputs:
["name", "Epping Forest", "job_address", "51 Epping Forest, Essex, ES1 SAD"]
I hope this helps :)
#str = ""
def hash_checks(hash)
hash.each do |k,v|
if v.is_a?(Array)
v.each_with_index do |element, index|
if element.is_a?(Hash)
#str = element[index] + #str if hash_check(element)
else
#str << k.titleize + "\n" + element + "\n" unless element.nil?
end
end
elsif v.is_a?(Hash)
#str << k.titleize if hash_check(v)
end
end
puts #str
end
def hash_check(hash)
add_tag = false
hash.each do |k,v|
if v.is_a?(Hash)
#str = k.titleize + " " + #str if hash_check(v)
else
#str << k.titleize + "\n" + v + "\n" unless v.nil?
add_tag = true
end
end
add_tag
end
I've excluded the '<empty>' value for nil because the data you would want to extract is the one that matters.
There's an inconsistency on the data you want though, since on your first serialized json, there are 2 non-nil values, but you just want one.
This method includes both, and runs even for multi-level nests :)

Creating an md5 hash of a number, string, array, or hash in Ruby

I need to create a signature string for a variable in Ruby, where the variable can be a number, a string, a hash, or an array. The hash values and array elements can also be any of these types.
This string will be used to compare the values in a database (Mongo, in this case).
My first thought was to create an MD5 hash of a JSON encoded value, like so: (body is the variable referred to above)
def createsig(body)
Digest::MD5.hexdigest(JSON.generate(body))
end
This nearly works, but JSON.generate does not encode the keys of a hash in the same order each time, so createsig({:a=>'a',:b=>'b'}) does not always equal createsig({:b=>'b',:a=>'a'}).
What is the best way to create a signature string to fit this need?
Note: For the detail oriented among us, I know that you can't JSON.generate() a number or a string. In these cases, I would just call MD5.hexdigest() directly.
I coding up the following pretty quickly and don't have time to really test it here at work, but it ought to do the job. Let me know if you find any issues with it and I'll take a look.
This should properly flatten out and sort the arrays and hashes, and you'd need to have to some pretty strange looking strings for there to be any collisions.
def createsig(body)
Digest::MD5.hexdigest( sigflat body )
end
def sigflat(body)
if body.class == Hash
arr = []
body.each do |key, value|
arr << "#{sigflat key}=>#{sigflat value}"
end
body = arr
end
if body.class == Array
str = ''
body.map! do |value|
sigflat value
end.sort!.each do |value|
str << value
end
end
if body.class != String
body = body.to_s << body.class.to_s
end
body
end
> sigflat({:a => {:b => 'b', :c => 'c'}, :d => 'd'}) == sigflat({:d => 'd', :a => {:c => 'c', :b => 'b'}})
=> true
If you could only get a string representation of body and not have the Ruby 1.8 hash come back with different orders from one time to the other, you could reliably hash that string representation. Let's get our hands dirty with some monkey patches:
require 'digest/md5'
class Object
def md5key
to_s
end
end
class Array
def md5key
map(&:md5key).join
end
end
class Hash
def md5key
sort.map(&:md5key).join
end
end
Now any object (of the types mentioned in the question) respond to md5key by returning a reliable key to use for creating a checksum, so:
def createsig(o)
Digest::MD5.hexdigest(o.md5key)
end
Example:
body = [
{
'bar' => [
345,
"baz",
],
'qux' => 7,
},
"foo",
123,
]
p body.md5key # => "bar345bazqux7foo123"
p createsig(body) # => "3a92036374de88118faf19483fe2572e"
Note: This hash representation does not encode the structure, only the concatenation of the values. Therefore ["a", "b", "c"] will hash the same as ["abc"].
Here's my solution. I walk the data structure and build up a list of pieces that get joined into a single string. In order to ensure that the class types seen affect the hash, I inject a single unicode character that encodes basic type information along the way. (For example, we want ["1", "2", "3"].objsum != [1,2,3].objsum)
I did this as a refinement on Object, it's easily ported to a monkey patch. To use it just require the file and run "using ObjSum".
module ObjSum
refine Object do
def objsum
parts = []
queue = [self]
while queue.size > 0
item = queue.shift
if item.kind_of?(Hash)
parts << "\\000"
item.keys.sort.each do |k|
queue << k
queue << item[k]
end
elsif item.kind_of?(Set)
parts << "\\001"
item.to_a.sort.each { |i| queue << i }
elsif item.kind_of?(Enumerable)
parts << "\\002"
item.each { |i| queue << i }
elsif item.kind_of?(Fixnum)
parts << "\\003"
parts << item.to_s
elsif item.kind_of?(Float)
parts << "\\004"
parts << item.to_s
else
parts << item.to_s
end
end
Digest::MD5.hexdigest(parts.join)
end
end
end
Just my 2 cents:
module Ext
module Hash
module InstanceMethods
# Return a string suitable for generating content signature.
# Signature image does not depend on order of keys.
#
# {:a => 1, :b => 2}.signature_image == {:b => 2, :a => 1}.signature_image # => true
# {{:a => 1, :b => 2} => 3}.signature_image == {{:b => 2, :a => 1} => 3}.signature_image # => true
# etc.
#
# NOTE: Signature images of identical content generated under different versions of Ruby are NOT GUARANTEED to be identical.
def signature_image
# Store normalized key-value pairs here.
ar = []
each do |k, v|
ar << [
k.is_a?(::Hash) ? k.signature_image : [k.class.to_s, k.inspect].join(":"),
v.is_a?(::Hash) ? v.signature_image : [v.class.to_s, v.inspect].join(":"),
]
end
ar.sort.inspect
end
end
end
end
class Hash #:nodoc:
include Ext::Hash::InstanceMethods
end
These days there is a formally defined method for canonicalizing JSON, for exactly this reason: https://datatracker.ietf.org/doc/html/draft-rundgren-json-canonicalization-scheme-16
There is a ruby implementation here: https://github.com/dryruby/json-canonicalization
Depending on your needs, you could call ary.inspect or ary.to_yaml, even.

Resources