Hello Im using HTTParty to call for a remote json file that I need to extract the URL's to use in one of my tests..
the json format goes something like:
"manifest" : {
"header" : {
"generated" : "xxxxxxxxxxxxxxx",
"name" : "xxxxxxxxxxx",
"version" : "1.0.0"
},
"files" : [ {
"file" : "blimp.zip",
"url" : "http://www.xxx.xx/restaurants_blimp.zip",
"checksum" : "ee98c9455b8d7ba6556f53256f95"
}, {
"file" : "yard.zip",
"url" : "www.xxx.xx/yard.zip",
"checksum" : "e66aa3d123f804f34afc622b5"
}
on irb I can get all the sub hashes inside example: ['manifest']['files'] and I can only get the url if I expecify which one.. like for example puts file['manifest']['files']['1']['url'] <-- this does work on irb but since I need to get ALL url's this is why I use .each but it gives me a cant convert to string error or similar
#!/usr/bin/env ruby
require 'httparty'
HOST=ARGV[0]
ID=ARGV[1]
VERSION=ARGV[2]
class MyApi
include HTTParty
end
file = MyApi.get("http://#{HOST}/v1/dc/manifest/#{ID}/#{VERSION}")
file.each do |item|
puts item['manifest']['files']['url']
end
not working but I can on IRB do a:
puts item['manifest']['files'][2]['url'] <-- and this will give me the url but with the .each will just complaint about cant convert to string or similar
Try the following:
#!/usr/bin/env ruby
require 'httparty'
(HOST, ID, VERSION) = ARGV
class MyApi
include HTTParty
format :json
end
response = MyApi.get("http://#{HOST}/v1/dc/manifest/#{ID}/#{VERSION}")
puts response.inspect
The addition of the format :json tells HTTParty to parse the response as JSON. Then you'll get a hash you can iterate over properly.
Try:
file['manifest']['files'].each do |item|
puts item['url']
end
Related
I created a body for a PUT request with Ruby. When I print out the body, I don't see any problems. But when I try to print out the body from the actual PUT request, I start getting an error.
To elaborate, here is my code:
#data={param1: "nameserver",
param2: {code: "XYZ", name: "NAME", start: "2017"}}
puts "data = #{#data} " #This works fine
#putResponse = HTTParty.put(base_url,
:body => #data.to_json,
:headers => header)
puts "putResponse.body is #{#putResponse.body}" #This is where I get the error
So as you can see, the line puts "data = #{#data} " works fine. It prints out
data = {:param1=>"nameserver", :param2=>{:code=>"XYZ", :name=>"NAME", :start=>"2017"}}
But the line puts puts "putResponse.body is #{#putResponse.body}" doesn't work. This is what it prints out:
putResponse.body is {"errors":[{"message":"The specified resource does not exist."}],"error_report_id":443}
So what's the problem here? (I'm using HTTParty to make my PUT request)
EDIT:
Here's how I got the host and header:
config = JSON.parse(File.read('config.json'))
puts "config: #{config}"
access_token = config['canvas']['access_token']
puts "access_token: #{access_token}"
host = config['canvas']['host']
puts "host: #{host}"
base_url = 'http://#{host}/api/v1/users/self/custom_data/program_of_study'
puts "base_url: #{base_url}"
header = {'Authorization': 'Bearer ' "#{$access_token}", "Content-Type" => 'application/json', 'Accept' => 'application/json'}
Dealing with PUT in Sinatra, is similar to dealing with POST - this is why the documentation may be scarce in this aspect.
This is a simple example of a Sinatra endpoint that receives PUT arguments from a curl request:
# server.rb
require 'sinatra'
put '/' do
params.inspect
end
And test it with this curl command:
$ curl -X PUT -d n=hello http://localhost:4567/
The params hash will be available to you inside any Sinatra endpoint, including parameters received by any HTTP method.
In response to your comment, it is hard to debug without seeing your entire code.
I suggest you run the tests provided in this answer, and gradually modify it to fit your actual code, to understand what breaks.
With the above server.rb running, the below test works without errors:
# test.rb
require 'httparty'
data = {
param1: "nameserver",
param2: { code: "XYZ", name: "NAME", start: "2017" }
}
response = HTTParty.put 'http://localhost:4567/', body: data
puts response.body
# => {"param1"=>"nameserver", "param2"=>{"code"=>"XYZ", "name"=>"NAME", "start"=>"2017"}}
{
"TOTAL" : "520"
,
"PROD_101379" : {
"IMG" : "1406301107587209.jpg",
"NAME" : "hello sunny",
"LINK" : "/product/productDetail.do?seq=101379",
"SEQ" : "101379",
"PRICE" : "18000",
"MILEAGE" : "2"
}
,
"PROD_101378" : {
"IMG" : "",
"NAME" : "special gift",
"LINK" : "/product/productDetail.do?seq=101378",
"SEQ" : "101378",
"PRICE" : "3000",
"MILEAGE" : "2"
}
,
"PROD_101376" : {
"IMG" : "1405020190326241.jpg",
"NAME" : "it radiant",
"LINK" : "/product/productDetail.do?seq=101376",
"SEQ" : "101376",
"PRICE" : "45000",
"MILEAGE" : "2"
}
,
"PAGER" : "<div class="paging"><a href="javascript:pageForm('0');
}
I am learning how to program in Ruby, using Nokogiri gem to parse internet data.
Above data was received after requesting with POST using Net/HTTP, then parsing it with Json gem.
JSON.parse(x.body)
I am trying to turn that data into CSV like so:
IMG | Name | Link | SEQ | Price | Mileage |
for each PROD_xxxxxx arrays.
I've read the gem documentation, and looked at other questions here to realize that much like Nokogiri parsing HTML(or more like Array & Hash), I should be able to parse JSON format.
I found that I can get the value of something by:
json_parsed["TOTAL"]
which would give me "520"
I get use the same approach to get at the "PROD_xxxxxx" nested data, but that would give me only specific nested data for that name.
I would like to be able to loop through them, so I've tried something like
json_parsed["PROD*"].each do |each|
but looks like this is not the correct way to use the syntax.
If can I loop through each and get "IMG", "NAME", "LINK", ...etc, I am trying to use CSV to:
CSV.open(fname,"w") do |csv|
Json_parsed[each-PROD].each do |each|
name = each["NAME"]
img = each["IMG"]
csv << [name, img]
end
end
However, if there is a better way than the above approach to turn the JSON data into CSV (maybe this is possible as JSON is a structured data, like CSV?) I would appreciate the suggestion.
Thanks,
The easiest way is to iterate through values of PROD_* keys:
h = '{ ... }':
require 'json'
hash = (JSON.parse h).select { |k,v| k =~ /PROD_/ } # select PROD_* only
require 'csv'
CSV.open('filename', 'w') do |csv|
csv << ['IMG', 'Name', 'Link', 'SEQ', 'Price', 'Mileage']
hash.each { |k,v|
csv << v.values
}
end
Hope it helps.
I'm a rookie in Ruby language. I'm trying to write a json file with ruby to import it after to a Mongodb collection. I need the document maintain proper indentation to then fill it comfortably
At this moment, I'm doing it in this way, but I'm sure that isn't the recommened way
out_file = File.new('file.json', "w+")
str = <<-eos
{
"key1": #{#value1},
"key2" : #{#value2},
"key3" : {
"subkey_3_1" : {
"key" : #{#value},
"questions" : #{#invalid_questions}
},
"subkey_3_2" : {
"key" : #{value},
"array_key" : [
for i in 1..50
# Here, must be create 50 hash pair-value like this.
{},
{},
{},
...
end
]
}
}
}
eos
out_file.puts(str)
out_file.close
This is the final structure that I want.Thanks, and sorry for not explaining right from the start
How can I define it in ruby?
str = <<-eos
"key" : [
#{for i in 1..50 {
...something content...
}.join("\n") }
]
eos
However - why do you want a string here - I don't know what you are trying to do, but there must be a better way of doing it.
UPDATE:
Yep, as mentioned by #ArupRakshit you need to create the hash first and call to_json on it. If you don't have this method, you need to install gem called active_support and require 'active_support/core_ext' (no need to do this for rails app). Do not build json response manually.
I am trying to build a quick hack that "likes" all the recent photos on instagram of a particular tag.
I have authenticated and used the JSON gem to turn the JSON from the API into a Ruby Hash like this:
def get_content (tag_name)
uri = URI.parse("https://api.instagram.com/v1/tags/#{tag_name}/media/recent? access_token=#{#api_token}")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Get.new(uri.request_uri)
json_output = http.request(request)
#tags = JSON.parse(json_output.body)
end
This outputs a hash with arrays as keys nested like the original JSON ( ex. http://instagr.am/developer/endpoints/tags/)
I am trying to iterate over and retrieve all the "id"s of the photos.
However when I use each method:
#tags.each do |item|
puts item["id"]
end
I get an error:
instagram.rb:23:in `[]': can't convert String into Integer (TypeError)
from instagram.rb:23:in `block in like_content'
from instagram.rb:22:in `each'
from instagram.rb:22:in `like_content'
from instagram.rb:42:in `<main>'
instagram.rb:23:in `[]': can't convert String into Integer (TypeError)
You're getting this error because in puts item["id"], item is an Array, not a Hash, so Ruby tries to convert what you put between [] into an integer index, but it can't because it's a string ("id").
This arises from the fact that json_output.body is a Hash. Take a second look at the example JSON response in the documentation:
{ "data" : [
{ "type" : "image",
// ...
"id" : "22699663",
"location" : null
},
// ...
]
}
This whole structure becomes a single Hash with one key, "data", so when you call #tags.each you're actually calling Hash#each, and since "data"'s value is an Array when you call item["id"] you're calling Array#[] with the wrong kind of parameter.
Long story short, what you actually want to do is probably this:
#tags = JSON.parse( json_output.body )[ "data" ]
..then #tags will be the Array you want instead of a Hash and you can iterate over its members like you wanted:
#tags.each do |item|
puts item["id"]
end
I have some code written in Ruby 1.9.2 patch level 136 and I'm have an issue where when I perform a find via the _id in the raw ruby mongo driver I get a nil when trying to use a value from a csv file. Here's the code:
require 'mongo'
require 'csv'
require 'bson'
# Games database
gamedb = Mongo::Connection.new("localhost", 27017).db("gamedb")
#games = gamedb.collection("games")
# Loop over CSV data.
CSV.foreach("/tmp/somedata.csv") do |row|
puts row[0] # Puts the ObjectId
#game = #games.find( { "_id" => row[0] } ).first
puts #game.inspect
end
The CSV file looks like this:
_id,game_title,platform,upc_db_match,upc
4ecdacc339c7d7a2a6000002,TMNT,PSP,TMNT,085391157663
4ecdacc339c7d7a2a6000004,Super Mario Galaxy,Wii,Super Mario Galaxy,045496900434
4ecdacc339c7d7a2a6000005,Beowulf,PSP,Beowulf,097363473046
The first column is the objectId in Mongo that I already have. If I perform a local find from the mongo command line the values in the first column, I get the data I want. However, the code above returns nil on the #game.inspect call.
I've tried the following variations, which all produce nil:
#game = #games.find( { "_id" => row[0].to_s } ).first
#game = #games.find( { "_id" => row[0].to_s.strip } ).first
I've even tried building the ObjectId with the BSON classes as such:
#game = #games.find( { "_id" => BSON::ObjectId(row[0]) } ).first
or
#game = #games.find( { "_id" => BSON::ObjectId("#{row[0]}") } ).first
Both of which output the following error:
/Users/donnfelker/.rvm/gems/ruby-1.9.2-p136#upc-etl/gems/bson-1.4.0/lib/bson/types/object_id.rb:126:in `from_string': illegal ObjectId format: _id (BSON::InvalidObjectId)
from /Users/donnfelker/.rvm/gems/ruby-1.9.2-p136#upc-etl/gems/bson-1.4.0/lib/bson/types/object_id.rb:26:in `ObjectId'
from migrate_upc_from_csv.rb:14:in `block in <main>'
from /Users/donnfelker/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/csv.rb:1768:in `each'
from /Users/donnfelker/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/csv.rb:1202:in `block in foreach'
from /Users/donnfelker/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/csv.rb:1340:in `open'
from /Users/donnfelker/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/csv.rb:1201:in `foreach'
from migrate_upc_from_csv.rb:10:in `<main>'
The crazy thing is, if I manually create the BSON ObjectId by hand it works (as shown below):
#game = #games.find( { "_id" => BSON::ObjectId("4ecdacc339c7d7a2a6000004") } ).first
When I run #game.inspect I get my data back, as I would expect. However, If I change this to use row[0], I get nil.
Why? What am I doing wrong?
System Details
$ gem list
*** LOCAL GEMS ***
bson (1.4.0)
bson_ext (1.4.0)
mongo (1.4.0)
RVM Version: rvm 1.6.9
Ruby Version: ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-darwin10.6.0]
Mongo Version:
[initandlisten] db version v1.8.2, pdfile version 4.5
[initandlisten] git version: 433bbaa14aaba6860da15bd4de8edf600f56501b
Again, why? What am I doing wrong here? Thanks!
The first row is not being read as a header, to do that pass in :headers => true like this:
require 'csv'
# Loop over CSV data.
CSV.foreach("/tmp/somedata.csv", :headers => true) do |row|
puts row[0] # Puts the ObjectId
end
If you do not pass the :headers parameter in you can see the first row[0] object is the string "_id":
_id
4ecdacc339c7d7a2a6000002
4ecdacc339c7d7a2a6000004
4ecdacc339c7d7a2a6000005
When you include it, you are golden:
4ecdacc339c7d7a2a6000002
4ecdacc339c7d7a2a6000004
4ecdacc339c7d7a2a6000005
Are you sure your CSV parsing code isn't treating the headers as a first line of data and actually tries to do BSON::ObjectId("_id")? The error message kinda looks like it. Try with FasterCSV.foreach('/tmp/somedata.csv', :headers => true) and using row['_id'] (IIRC you'll still have to use BSON::ObjectID).