I am building a small Sinatra API that will run some calculations and store results on the server. The POST route requires a JSON payload. The JSON payload is a combination of floats, arrays of floats and base64 encoded data.
This is the route
post '/calc' do
rad_data = JSON.parse(request.body.read)
# Perform calculations
end
Everything works fine when the json is around 2.8MB, but when the size gets to 3.9MB I receive the error
2021-11-20 17:33:43 +0000 Rack app ("POST /calc" - (78.110.164.58)): #<RangeError: RangeError>
(It is also weird that 78.110.164.58 is not the ip address of the machine and i have no idea where it comes from.)
The structure of the payload is exactly the same. The larger one has simply longer arrays. Also, if I load the file directly on the server everything works, so it is not a problem with the json content.
Based on this answer I have added the key_space_limit to the beginning of the app, but it makes no difference.
require 'sinatra'
require 'sinatra/reloader'
require 'rack'
if Rack::Utils.respond_to?("key_space_limit=")
puts "Yes"
Rack::Utils.key_space_limit = 2**64
end
Just in case it helps, I send my request with Ruby rest-client
RestClient.post(uri_post, JSON.dump(rad_data))
Related
i have been trying to get the api data using httparty and been doing it like this
require 'httparty'
doc = HTTParty.get('http://ip-api.com/php/105.49.23.183')
puts doc['status']
The api response is
a:14:{s:6:"status";s:7:"success";s:7:"country";s:5:"Kenya";s:11:"countryCode";s:2:"KE";s:6:"region";s:2:"30";s:10:"regionName";s:16:"Nairobi Province";s:4:"city";s:7:"Nairobi";s:3:"zip";s:5:"09831";s:3:"lat";d:-1.25909;s:3:"lon";d:36.7858;s:8:"timezone";s:14:"Africa/Nairobi";s:3:"isp";s:17:"Safaricom Limited";s:3:"org";s:3:"SFC";s:2:"as";s:25:"AS33771 Safaricom Limited";s:5:"query";s:13:"105.49.23.183";}
but the output when i run my script is just
status
but i need it to return the data responding to the status key which is success. I can't figure it out how to do this. Initially in python this method works but ruby is different
Just use the JSON formatted API endpoint:
require 'httparty'
doc = HTTParty.get('http://ip-api.com/json/105.49.23.183')
puts doc['status']
#=> "success"
Notice the json instead of the php in the URL path. The serialized PHP endpoint is deprecated anyway.
I'm trying to upload a file with XHR request using PUT method with Sinatra.
My first idea was to upload the file and writing the stream directly into a MongoDB GridFS
#fs.open("test.iso", "w") do |f|
f.write request.body.read
end
It works, but, it loads the entire file into the RAM and it write it into the MongoDB GridFS.
I'd like to avoid this behavior by writing it continuously to the GridFS (stream the file, not loading it and put it in the GridFS) without loading the entire file into the RAM : because for huge files (like 1GB or more) it's clearly a bad practice (due to RAM consumption).
How can I do that ?
EDIT :
Can I have Sinatra / Rack not read the entire request body into memory? method is creating a TempFile, the thing is I want to only work with streams to optimize memory consumption on server-side.
Like a php://input would do in PHP.
EDIT 2 :
Here is how I'm currently handling it :
HTML/JS part
Ruby/Sinatra part
It looks like the streaming support in Sinatra is only in the opposite direction for long running GET requests out to the client.
If you do a POST multipart upload, Sinatra will write the data to a tempfile and provide you with the details in the params Hash.
require 'sinatra'
require 'fileutils'
post '/upload' do
tempfile = params['file'][:tempfile]
filename = params['file'][:filename]
FileUtils.mv(tempfile.path, "test.iso")
"posted"
end
While in the same sinatra directory:
$ echo "testtest" > /tmp/file_to_upload
$ curl --form "file=#/tmp/file_to_upload" http://localhost:4567/upload
posted
$ cat test.iso
testtest
When I run the curl command
curl -v -H "Content-type: application/json" -X POST -d '{"name":"abc", "id":"12", "subject":"my subject"}' http://localhost:9292
to send a POST request with data to my Rack application, my code prints out {}. That is coming from puts req.POST() in the code below.
Why does it print out {} instead of the POST data? And how do I correctly access the POST data in my Rack application?
require 'json'
class Greeter
def call(env)
req = Rack::Request.new(env)
if req.post?
puts req.POST()
end
[200, {"Content-Type" => "application/json"}, [{x:"Hello World!"}.to_json]]
end
end
run Greeter.new
From reading the docs for POST, looks like it is giving you parsed data based on other content types. If you want to process "application/json", you probably need to
JSON.parse( req.body.read )
instead. To check this, try
puts req.body.read
where you currently have puts req.POST.
req.body is an I/O object, not a string. See the body documentation and view the source. You can see that this is in fact the same as mudasobwa's answer.
Note that other code in a Rack application may expect to read the same I/O, such as the param parsers in Sinatra or Rails. To ensure that they see the same data and not get an error, you may want to call req.body.rewind, possibly both before and after reading the request body in your code. However, if you are in such a situation, you might instead consider whether your framework has options to process JSON directly via some option on the controller or request content-type handler declaration etc - most likely there will be an option to handle this kind of request within the framework.
Try:
env['rack.input'].read
I found it in "How to receive a JSON object with Rack" and, though it still sounds weird to me, it likely works.
You can try:
req.params
Hope this can help you.
I'm implementing a simple tumblr api tool for my girlfriend, and I'm having some trouble working with the data handed to me by Tumblr.
The structure handed back can be found here (scroll down a little for the example.)
The following string of hash keys works while I'm working manually in IRB:
followers_result['response']['users']
But when I launch the server and try to walk the json hash, I get a "Can't convert string to integer" typeerror. Using the following code, response['users'] is identified as the problem field. (I expanded the hashtree query a bit here.)
followers_result = JSON.parse(#followers_response.body)
response = followers_result['response']
users = response['users']
users.each do |follower|
followers << follower['name']
end
I'm having the same problem with my other request, which requires a few more nodes down the tree... Anyone know why this would be different between server and irb?
(One difference is that the server is requesting via OAuth whereas my irb tests are requesting via net/http, but I'm taking the exact same OAuth query and adding my API key on to do the tests with. I'm not sure how to manually request it via OAuth to make sure that the server is getting the same document because of the three-legged authentication.)
Thanks for any help/suggestions,
Cameron
In a little app I'm building, I'm using the the twitter_oauth gem (source of the methods I'm using), which incidentally means I'm dealing with the JSON ruby gem.
I'm using the messages method, whose source is as follows:
def messages(page=1)
oauth_response = access_token.get("/direct_messages.json?page=#{page}")
JSON.parse(oauth_response.body)
end
It parses the JSON produced by Twitter, using the JSON.parse method. Now, what I want to do is filter the response, so as to show only the messages sent by a certain user. In other words, I want to be able to get an accounts message's on a per user basis.
I went through the JSON gem docs, but couldn't find an easy way to do this. I normally don't work with JSON (I prefer XML), but since the Twitter_oauth gem relies on it, I'm forced to learn it (unless I change the gem's source code or overwrite it - not of my preference).
Does any one know a pragmatic way of sorting JSON in Ruby?
Why do you have to work with JSON at all? JSON.parse gives you back deserialized structures (Ruby arrays and hashes), so all you have to do (I presume, you haven't pasted sample JSON output here) is to do select on whatever output JSON.parse (or, in your case, messages) method returns.