I have GET-type URLs with the parameters at the end of the URL string. I need to restructure them so that the stuff before the ? becomes the endpoint and the stuff after gets sent out as parameters in a correctly-formatted POST request payload.
For example:
http://ecample.com?test=true&message=hello
needs to be sent out as a POST request to the URL http://example.com with a request payload of:
{"test":true,"message":"hello"}
Any ideas or quick tricks to get this so I can get the POST response?
Meditate on this:
require 'uri'
scheme, userinfo, host, port, registry, path, opaque, query, fragment = URI.split('http://example.com?test=true&message=hello')
scheme # => "http"
userinfo # => nil
host # => "example.com"
port # => nil
registry # => nil
path # => ""
opaque # => nil
query # => "test=true&message=hello"
fragment # => nil
uri = URI.parse('http://example.com?test=true&message=hello')
server = '%s://%s' % [uri.scheme, uri.host] # => "http://example.com"
parameters = Hash[URI.decode_www_form(uri.query)] # => {"test"=>"true", "message"=>"hello"}
At this point you can use whatever you want to connect to the server and send the parameters using GET, POST or any of the other request types.
URI is built into Ruby and has all the methods necessary to take apart URLs correctly and rebuild them.
When learning a computer language, it's mandatory to read through all the libraries and become familiar with their offerings. I do it many times, not to know exactly where a particular method is and its parameters, but to remember that it exists somewhere and then I can search and find it. Any modern language that can talk to web services will have such functionality available to it; It's your job to read the documentation and become familiar with it.
Related
I'm using Sinatrarb to complete a task
I need to:
Parse the data of a JSON object from a url,
Single out one of attributes of the json data and store it as a variable
Run some arithmetic on the variable
Return the result as a new variable
then post this to a new url as a new json object.
I have seen bits and pieces of information all over including information on parsing JSON data in ruby and information on open-uri but I believe it would be very valuable having someone break this down step by step as most similar solutions given to this are either outdated or steeply complex.
Thanks in advance.
Here's a simple guide. I've done the same task recently.
Let's use this JSON (put it in a file called 'simple.json'):
{
"name": "obscurite",
"favorites": {
"icecream": [
"chocolate",
"pistachio"
],
"cars": [
"ferrari",
"porsche",
"lamborghini"
]
},
"location": "NYC",
"age": 100}
Parse the data of a JSON object from a url.
Step 1 is to add support for JSON parsing:
require 'json'
Step 2 is to load in the JSON data from our new .json file:
json_file = File.read('simple.json')
json_data = JSON.parse(json_file)
Single out one of attributes of the json data and store it as a variable
Our data is in the form of a Hash on the outside (curly braces with key:values). Some of the values are also hashes ('favorites' and 'cars'). The values of those inner hashes are lists (Arrays in Ruby). So what we have is a hash of hashes, where some hashes are arrays.
Let's pull out my location:
puts json_data['location'] # NYC
That was easy. It was just a top level key/value. Let's go deeper and pull out my favorite icecream(s):
puts json_data['favorites']['icecream'] # chocolate pistachio
Now only my second favorite car:
puts json_data['favorites']['cars'][1] # porsche
Run some arithmetic on the variable
Step 3. Let's get my age and cut it down by 50 years. Being 100 is tough!
new_age = json_data['age'] / 2
puts new_age
Return the result as a new variable
Step 4. Let's put the new age back into the json
json_data['age'] = new_age
puts json_data['age'] # 50
then post this to a new url as a new json object.
Step 5. Add the ability for your program to do an HTTP POST. Add this up at top:
require 'net/http'
and then you can post anywhere you want. I found a fake web service you could use, if you just want to make sure the request got there.
# use this guy's fake web service page as a test. handy!
uri = URI.parse("http://jsonplaceholder.typicode.com/posts")
header = {'Content-Type'=> 'text/json'}
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.request_uri, header)
request.body = json_data.to_json
response = http.request(request)
# Did we get something back?
puts response.body
On linux or mac you can open a localhost port and listen as a test:
nc -4 -k -l -v localhost 1234
To POST to this port change the uri to:
uri = URI.parse("http://localhost:1234")
Hope this helps. Let me know if you get stuck and I'll try to lend a hand. I'm not a ruby expert, but wanted to help a fellow explorer. Good luck.
I am attempting to update the 'ip' parameter in a json object in an API.
I have the following case:
when "put"
uri = URI.parse("http://#{ip}:#{port}/api/v1/address_data/1.json")
jobj = Hash.new
jobj['ip'] = "1.1.1.1"
http = Net::HTTP.new(uri.hostname, uri.port)
response = http.send_request('PUT', '/api/v1/address_data/1.json', data = jobj.to_s)
end
This does not work, but this does:
curl -X PUT http://ip:port/api/v1/address_data/1.json -d "ip=1.1.1.1"
How do I more accurately translate the curl into a Put request in Ruby? I have tried several methods I've found through google searching, but none of them have had successful results.
A few things:
You're not sending JSON in the Ruby example, it's a string representation of a Ruby hash which isn't the same. You need the JSON module or similar.
In the Ruby code you're attempting to send a JSON object (which would look like {"ip":"1.1.1.1"} and in the curl example you're sending it in application/x-www-form-urlencoded format, so they're currently not equivalent.
Also I'd look at the type of data the server expects from your requests: both Ruby and curl send a request header of Content-Type: application/x-www-form-urlencoded by default, and you're expecting to send JSON. This is why the curl example works: the data format you're using and the header matches. Note the .json in the URL shouldn't really make any difference; the header takes precedence.
Your call to send_request has you picking out the data parameter as a Python-style keyword argument. Ruby doesn't do that: what you're actually doing there is assigning a local variable in-line with the call.
So try something like this:
require 'json' # put this at the top of the file
uri = URI.parse("http://#{ip}:#{port}/api/v1/address_data/1.json")
jobj = {"ip" => "1.1.1.1"}
http = Net::HTTP.new(uri.hostname, uri.port)
response = http.send_request('PUT', uri.path, JSON.dump(jobj),
{'Content-Type' => 'application/json'})
And just a friendly reminder, saying something "doesn't work" doesn't usually give enough information to people that might answer your question: try and remember to paste in error messages, stack traces, and things like that :)
I am trying to write short script to do HTTP authentication using GET request
This makes GET request.
def try_login(u, p)
path1 = '/index.php'
path2 = '?uuser=#{myuser}&ppass=#{mypass}'
r = send_request_raw({
'URI' => "#{path1}#{path2}",
'method' => 'GET'
})
...continued...
But this code does not work because error says:
undefined local variable or method `myuser'
--> Basically I am trying to send one (1) GET request with login parameters, and the app responds with a specific data. And I do not know how to put placeholders for user and pass in this GET request.
...
Next, I am checking the HTTP response. Response comes in as JSON mime like this:
Success response
{"param1":1,"param2"="Auth Success","menu":0,"userdesc":"My User","user":"uuser","pass":"ppass","check":"success"}
Fail response
{"param1":-1,"param2"="Auth Fail","check":"fail"}
--> How can I check the response body for this kind of data.
I have been trying all day now, but stuck totally. Please advice.
Edit:
I do not understand why some one down voted this question saying little to no research on my part. Until before yesterday morning, I had absolutely zero idea about ruby code & working with it. And then I spent numerous hours looking at many different examples, making my script and testing it out. When it still didn't work, I asked my question here. Please, if you still want to down vote, do it but please, at least share some pointers to solve this as well.
def try_login(u, p)
path1 = '/index.php'
path2 = '?uuser=#{myuser}&ppass=#{mypass}'
r = send_request_raw({
'URI' => "#{path1}#{path2}",
'method' => 'GET'
})
...continued...
Should be:
def try_login(u, p)
path1 = '/index.php'
path2 = "?uuser=#{u}&ppass=#{p}"
r = send_request_raw({
'URI' => "#{path1}#{path2}",
'method' => 'GET'
})
...continued...
For parsing JSON in Ruby, I would recommend you take a look at this answer to another question.
Edit: The reason try_login(u, p) isn't working as you would expect is because Ruby does not do string interpolation for single quoted (') strings. Additionally, myuser and mypass do not appear to be the correct variables.
I have a url such as:
http://www.relevantmagazine.com/life/relationship/blog/23317-pursuing-singleness
And would like to extract just relevantmagazine from it.
Currently I have:
#urlroot = URI.parse(#link.url).host
But it returns www.relevantmagazine.com can anyone help me?
Using a gem for this might be overkill, but anyway: There's a handy gem called domainatrix that can extract the sitename for your while dealing with things like two element top-level domains and more.
url = Domainatrix.parse("http://www.pauldix.net")
url.url # => "http://www.pauldix.net" (the original url)
url.public_suffix # => "net"
url.domain # => "pauldix"
url.canonical # => "net.pauldix"
url = Domainatrix.parse("http://foo.bar.pauldix.co.uk/asdf.html?q=arg")
url.public_suffix # => "co.uk"
url.domain # => "pauldix"
url.subdomain # => "foo.bar"
url.path # => "/asdf.html?q=arg"
url.canonical # => "uk.co.pauldix.bar.foo/asdf.html?q=arg"
how about
#urlroot = URI.parse(#link.url).host.gsub("www.", "").split(".")[0]
Try this regular expression:
regex = %r{http://[w]*[\.]*[^/|$]*}
If you had the following url strings, it gives the following:
url = 'http://www.google.com/?q=blah'
url.scan(regex) => ["http://www.google.com"]
url = 'http://google.com/?q=blah'
url.scan(regex) => ["http://google.com"]
url = 'http://google.com'
url.scan(regex) => ["http://google.com"]
url = 'http://foo.bar.pauldix.co.uk/asdf.html?q=arg'
url.scan(regex) => ["http://foo.bar.pauldix.co.uk"]
It's not perfect, but it will strip out everything but the prefix and the host name. You can then easily clean up the prefix with some other code knowing now you only need to look for an http:// or http://www. at the beginning of the string. Another thought is you may need to tweak the regex I gave you a little if you are also going to parse https://. I hope this helps you get started!
Edit:
I reread the question, and realized my answer doesn't really do what you asked. I suppose it might be helpful to know if you know if the urls you're parsing will have a set format like always have the www. If it does, you could use a regular expression that extracts everything between the first and second period in the url. If not, perhaps you could tweak my regex so that it's everything between the / or www. and the first period. That might be the easiest way to get just the site name with none of the www. or the .com or .au.uk and such.
Revised regex:
regex = %r{http://[w]*[\.]*[^\.]*}
url = 'http://foo.bar.pauldix.co.uk/asdf.html?q=arg'
url.scan(regex) => ["http://foo"]
It'll be weird. If you use the regex stuff, you'll probably have to do it incrementally to clean up the url to extract the part you want.
Maybe you can just split it?
URI.parse(#link.url).host.split('.')[1]
Keep in mind that some registered domains may have more than one component to the registered country domain, like .co.uk or .co.jp or .com.au for example.
I found the answer inspired by tadman's answer and the answer in another question
#urlroot = URI.parse(item.url).host
#urlroot = #urlroot.start_with?('www.') ? #urlroot[4..-1] : #urlroot
#urlroot = #urlroot.split('.')[0]
First line get the host, second line gets removes the www. if they is one and third line get everything before the next dot.
I'm using sendgrid's event api which sends a codeblock like this to a postback url of my choice:
Array
(
[email] => fgdfg#gmail.com
[timestamp] => 1323698899
[smtp-id] => <4ee60acf8e3d1_55dd862cf147044#mbjoppa.mail>
[response] => 250 2.0.0 OK 1323698899 o30s15072o427yhl.103
[event] => delivered
)
They don't have XML or JSON and I need to extract the email and event parts of this block.
Any idea how I do this with rails? Basically this block is sent to a postback URL of my choice but I'm not sure how to use it from there.
I never used sendgrid but after a quick look it seems like they are just sending you a standard post request with parameters so you best bet is to define a simple action like the following and see what you get:
def sendgrid_event
Rails.logger.info(params)
# chances are that this will contains what you are looking for:
# params['email']
# params['event']
end