Store a byte array - ruby

The following is a sinatra route:
post '/accounts/:id/video' do |acc_id|
acc_id = params[:id].to_s
user = db_find('thrill.users',{_id: acc_id})
if user
db_update('thrill.users', { _id: acc_id},
{ '$set' =>
{ video: request.body.to_s }})
res(200, :sys_message => 'Video stored.')
else
res(201, :sys_message => "User not found")
end
end
It gets a response from the client with a video file in the form of a byte array. If I store it directly in the database, it is stored in the following format:
PhusionPassenger::Utils::TeeInput:0x007f48f8485b50
This has to be stored in the database in a format that can be retrieved later and sent back to the client. How can I read the byte array? The preferred format for storing would be in a form of a url which the client can access and stream the video. How can this be done?


From the Rack specification, about the request body (which is what request.body is:
The input stream is an IO-like object which contains the raw HTTP POST data. When applicable, its external encoding must be “ASCII-8BIT” and it must be opened in binary mode, for Ruby 1.9 compatibility. The input stream must respond to gets, each, read and rewind.
The PhusionPassenger::Utils::TeeInput class meets these requirements. Calling to_s on an instance of this calls will just give the Ruby string representation of the object. To get the contents you need to call read, e.g.
{ video: request.body.read }
This will read the entire body into a string which you can then pass to Mongo.

Related

Using Sinatra to Parse JSON data from url

I'm using Sinatrarb to complete a task
I need to:
Parse the data of a JSON object from a url,
Single out one of attributes of the json data and store it as a variable
Run some arithmetic on the variable
Return the result as a new variable
then post this to a new url as a new json object.
I have seen bits and pieces of information all over including information on parsing JSON data in ruby and information on open-uri but I believe it would be very valuable having someone break this down step by step as most similar solutions given to this are either outdated or steeply complex.
Thanks in advance.

Here's a simple guide. I've done the same task recently.
Let's use this JSON (put it in a file called 'simple.json'):
{
"name": "obscurite",
"favorites": {
"icecream": [
"chocolate",
"pistachio"
],
"cars": [
"ferrari",
"porsche",
"lamborghini"
]
},
"location": "NYC",
"age": 100}
Parse the data of a JSON object from a url.
Step 1 is to add support for JSON parsing:
require 'json'
Step 2 is to load in the JSON data from our new .json file:
json_file = File.read('simple.json')
json_data = JSON.parse(json_file)
Single out one of attributes of the json data and store it as a variable
Our data is in the form of a Hash on the outside (curly braces with key:values). Some of the values are also hashes ('favorites' and 'cars'). The values of those inner hashes are lists (Arrays in Ruby). So what we have is a hash of hashes, where some hashes are arrays.
Let's pull out my location:
puts json_data['location'] # NYC
That was easy. It was just a top level key/value. Let's go deeper and pull out my favorite icecream(s):
puts json_data['favorites']['icecream'] # chocolate pistachio
Now only my second favorite car:
puts json_data['favorites']['cars'][1] # porsche
Run some arithmetic on the variable
Step 3. Let's get my age and cut it down by 50 years. Being 100 is tough!
new_age = json_data['age'] / 2
puts new_age
Return the result as a new variable
Step 4. Let's put the new age back into the json
json_data['age'] = new_age
puts json_data['age'] # 50
then post this to a new url as a new json object.
Step 5. Add the ability for your program to do an HTTP POST. Add this up at top:
require 'net/http'
and then you can post anywhere you want. I found a fake web service you could use, if you just want to make sure the request got there.
# use this guy's fake web service page as a test. handy!
uri = URI.parse("http://jsonplaceholder.typicode.com/posts")
header = {'Content-Type'=> 'text/json'}
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.request_uri, header)
request.body = json_data.to_json
response = http.request(request)
# Did we get something back?
puts response.body
On linux or mac you can open a localhost port and listen as a test:
nc -4 -k -l -v localhost 1234
To POST to this port change the uri to:
uri = URI.parse("http://localhost:1234")
Hope this helps. Let me know if you get stuck and I'll try to lend a hand. I'm not a ruby expert, but wanted to help a fellow explorer. Good luck.

Ruby split and parse batched HTTP Response (multipart/mixed)

I'm using the GMail API that allows me to get a batched response of multiple Gmail objects.
This comes back in the form of a multipart/mixed HTTP response with a set of separate HTTP responses separated by a boundary as defined in the header.
Each of the HTTP sub-Responses is a JSON format.
i.e.
result.response.response_headers = {...
"content-type"=>"multipart/mixed; boundary=batch_abcdefg"...
}
result.response.body = "----batch_abcdefg
<the response header>
{some JSON}
--batch_abcdefg
<another response header>
{some JSON}
--batch_abcdefg--"
Is there a library or an easy way to convert those responses from the string into a set of separate HTTP responses or JSON objects?

Thanks to Tholle above...
def parse_batch_response(response, json=true)
# Not the same delimiter in the response as we specify ourselves in the request,
# so we have to extract it.
# This should give us exactly what we need.
delimiter = response.split("\r\n")[0].strip
parts = response.split(delimiter)
# The first part will always be an empty string. Just remove it.
parts.shift
# The last part will be the "--". Just remove it.
parts.pop
if json
# collects the response body as json
results = parts.map{ |part| JSON.parse(part.match(/{.+}/m).to_s)}
else
# collates the separate responses as strings so you can do something with them
# e.g. you need the response codes
results = parts.map{ |part| part}
end
result
end

Zlib::DataError: incorrect header check

I have a string, but I don't know the type of encoding.
Here's what the raw data looks like:
{
"securityProxyResponseEnvelope":{
"resultCode":"OK",
"apiResponse":"{zlibe}9mtdE350h9rd4h7wlFX3AkeCtNsb40FFaEZAl/CfcNNKrhGXawhgAs2rI4rnEgIwLpgJkKl+qkB0kzJ6+ZFmmz12Pl9/9MPdA1unUKL5OdHcWmAZim3ZjDXFGCW4zruCS/IOSiU1qVKAF5qIbocB4+2rAF7zH18SRtmXM8YW3eYs5w1NPjmYkM31W8x7QvrKkzFscH3kqDwmYn0I2gNNOtfwuKjWd5snunyqxPopZHNX3CBdW/pj4+N0tJXjAoHorCe8Ypmjxnvh3zthkLTbiBLgeULH1hGvVtkI0C9PGMyt/92upVW6qHxqCYoO/LTJK1tq6OpBnMRBNZDDntSRkrzp+1RpvzbBxFtwQ9jh45eSthbG5hq+D2oJkW5zrGi6TM8eG4ztCqRoO9dEvz2JbQsDCTPz70+C6iPYdkvOyqji18ysLjBbGcHw1j45YItcurVxp0FChxXrnHZwu6m430xKEp7ONxvgEZurt3T8qAjrkrbHfd8jRjDydUXYsMoa",
"session":"n3qp6jzHwZkXWSMW3VBF:jitqBjBmlZbrgcEgY7Od",
"parameters":{
}
}
}
I want to decompress the string in data['securityProxyResponseEnvelope']['apiResponse'].
Here's what I'm doing:
#clear_string_from_data = '9mtdE350h9rd4h7wlFX3AkeCtNsb40FFaEZAl/CfcNNKrhGXawhgAs2rI4rnEgIwLpgJkKl+qkB0kzJ6+ZFmmz12Pl9/9MPdA1unUKL5OdHcWmAZim3ZjDXFGCW4zruCS/IOSiU1qVKAF5qIbocB4+2rAF7zH18SRtmXM8YW3eYs5w1NPjmYkM31W8x7QvrKkzFscH3kqDwmYn0I2gNNOtfwuKjWd5snunyqxPopZHNX3CBdW/pj4+N0tJXjAoHorCe8Ypmjxnvh3zthkLTbiBLgeULH1hGvVtkI0C9PGMyt/92upVW6qHxqCYoO/LTJK1tq6OpBnMRBNZDDntSRkrzp+1RpvzbBxFtwQ9jh45eSthbG5hq+D2oJkW5zrGi6TM8eG4ztCqRoO9dEvz2JbQsDCTPz70+C6iPYdkvOyqji18ysLjBbGcHw1j45YItcurVxp0FChxXrnHZwu6m430xKEp7ONxvgEZurt3T8qAjrkrbHfd8jRjDydUXYsMoa'
#decoded = Base64.decode64(#clear_string_from_data)
#inflated = Zlib::Inflate.inflate(#decoded)
But this returns
#=> Zlib::DataError: incorrect header check
What's causing this and what could I try next to decompress the data?

What's causing it is that it is not zlib data. You should ask whoever is producing that raw data.

I was getting this when trying to call inflate on a data that hadn't been deflated by Zlib. In my case it was for a unit test and I sent in a plain string and simply forgot to call .deflate on it first.
In your case, if you do this instead you don't get the error:
#decoded = Zlib::Deflate.deflate(#clear_string_from_data)
#inflated = Zlib::Inflate.inflate(#decoded)

how do I parse out this URL-encoded string with a JSON array using Ruby?

When I use a webhook with Mandrill and post to my Iron Worker, I get the following Raw (this is from RequestBin, as well) -- I didn't include the whole payload, just an example:
puts payload =>
mandrill_events=%5B%7B%22event%22%3A%22inbound%22%2C%22msg%22%3A%7B%22dkim%22%3A%7B%22signed%22%3Atrue%2C%22valid%22%3Atrue%7D%2C%22email%22%3A%22kaya%40hellokaya.com%22%2C%22from_email%22%3A%22example.sender%40mandrillapp.com%22%2C%22headers%22%3A%7B%22Content-Type%22%3A%22multipart%5C%2Falternative%3B+boundary%3D%5C%22_av-7r7zDhHxVEAo2yMWasfuFw%5C%22%22%2C%22Date%22%3A%22Fri%2C+10+May+2013+19%3A28%3A20+%2B0000%22%2C%22Dkim-Signature%22%3A%5B%22v%3D1%3B+a%3Drsa-
I tried to extract the value of the parameter mandrill_events using:
puts params = CGI::parse(#payload) =>
{"mandrill_events"=>["[{\"event\":\"inbound\",\"msg\":{\"dkim\":{\"signed\":true,\"valid\":true},\"email\":\"kaya#hellokaya.com\",\"from_email\":\"example.sender#mandrillapp.com\",\"headers\":{\"Content-Type\":\"multipart\\/alternative; boundary=\\\"_av-7r7zDhHxVEAo2yMWasfuFw\\\"\",\"Date\":\"Fri, 10 May 2013 19:28:20 +0000\",\"Dkim-Signature\":[\"v=1; a=rsa-sha1; c=relaxed\\/relaxed; s=mandrill; d=mail115.us4.mandrillapp.com; h=From:Sender:Subject:List-Unsubscribe:To:Message-Id:Date:MIME-Version:Content-Type; i=example.sender#mail115.us4.mandrillapp.com;
Then I am stuck. I want to extract the email value in the JSON.array.
I thought to try the following;
puts json_params = JSON.parse(params)
But I am now feeling there must be a better way....
How can I extract the elements from the JSON array in this URL-encoded string?

Ruby Sinatra and JSON objects from toodledo API 2.0

I have a small problem with receiving JSON objects. I'm using Ruby 1.9.3 and my goal is to receive my tasks from an API via RestClient and print them more or less pretty onto the page.
I created a route /test:
get '/test' do
json_ip_url = "http://api.toodledo.com/2/tasks/get.php?key=198196ae24792467eec09ac2191*****;modafter=1234567890;fields=folder,star,priority"
ip_details = RestClient.get(json_ip_url)
test = JSON.pretty_generate(ip_details) # => throws exception
end
The JSON#pretty_generate line throws an error, "only generation of JSON objects or arrays allowed". What am I doing wrong here?
Update:
I'am now able to output via pretty_generate, but what do I have to do, to get the elements of it. Here is the JSON Data, it seems to me its an Array with Objects inside of it?
[{"num":"18","total":"18"},{"id":"11980343","title":"Add some items to your todo list","modified":1391670256,"completed":0,"folder":"0","star":"0"},{"id":"11980345","title":"Visit the Settings section and configure your account","modified":1391670256,"completed":0,"folder":"0","star":"0"},{"id":"11980347","title":"Watch our tutorial videos in the Help section","modified":1391670256,"completed":0,"folder":"0","star":"1"},{"id":"12607789","title":"test","modified":1392285802,"completed":0,"folder":"0","star":"0"},{"id":"12636039","title":"My Task","modified":1392308705,"completed":0,"folder":"0","star":"0"},{"id":"12636041","title":"Another","modified":1392308705,"completed":0,"folder":"0","star":"1"},{"id":"12636143","title":"My Task","modified":1392308789,"completed":0,"folder":"0","star":"0"},{"id":"12636145","title":"Another","modified":1392308789,"completed":0,"folder":"0","star":"1"},{"id":"12636449","title":"My Task","modified":1392308950,"completed":0,"folder":"0","star":"0"},{"id":"12636451","title":"Another","modified":1392308950,"completed":0,"folder":"0","star":"1"},{"id":"12636621","title":"My Task","modified":1392309061,"completed":0,"folder":"0","star":"0"},{"id":"12636623","title":"Another","modified":1392309061,"completed":0,"folder":"0","star":"1"},{"id":"12636665","title":"My Task","modified":1392309085,"completed":0,"folder":"0","star":"0"},{"id":"12636667","title":"Another","modified":1392309085,"completed":0,"folder":"0","star":"1"},{"id":"12636733","title":"My Task","modified":1392309137,"completed":0,"folder":"0","star":"0"},{"id":"12636735","title":"Another","modified":1392309137,"completed":0,"folder":"0","star":"1"},{"id":"12637135","title":"My Task","modified":1392309501,"completed":0,"folder":"0","star":"0"},{"id":"12637137","title":"Another","modified":1392309501,"completed":0,"folder":"0","star":"1"}]
The Code I used for pretty_generate:
get '/save' do
jdata = params[:data]
response = RestClient.get 'http://api.toodledo.com/2/tasks/get.php?key=da21e24e2a00ba9d45008974aed00***;modafter=1234567890;fields=folder,star,priority', {:accept => :json}
test = JSON.parse(response)
test.to_json
output = JSON.pretty_generate(test)
puts output

RestClient#get returns the raw response as a string (and not a hash or array) when called without a block, so ip_details isn't a structure that JSON#pretty_generate knows how to handle. You need to use JSON#parse to turn the response into a hash or array first.

Resources