Parsing JSON with multiple pages in Ruby - ruby

Question

I understand how to parse JSON, but I don’t understand how to parse it if it contains links to other pages.
I would be grateful for your help!
api.example.com/v0/accounts
On the first request for a JSON file, we get:
{
"response": "OK",
"nfts": [
{
"token_id": "35507806371588763669298464310896317145981867843055556101069010709538683224114"
}
],
"total": null,
"continuation": "1634866413000"
}
There is a line: continuation, which is a link to the next request, and so it repeats many more times.
On next request, the link changes to api.example.com/v0/accounts&continuation=1634866413000
My code now looks like this:
class Source
include Mongoid::Document
include Mongoid::Timestamps
require 'json'
after_save :add_items
def add_items
json= HTTParty.get("https://api.example.com/v0/accounts")
json.dig('nfts')
load_items_ethereum.each do |item|
Item.create!(
:token_id => item['token_id'],
)
end
end
end

Answer 1

Low-level HTTP clients like HTTParty typically don't handle iteration. You'll need to do it yourself, using a loop until there's no continuation field, e.g.:
begin
continuation_param = "?continuation=#{continuation_id}" if continuation_id
json = HTTParty.get("https://api.example.com/v0/accounts#{continuation_param}")
continuation_id = json.dig('continuation');
# process latest payload, append it to a running list, etc.
end while continuation_id
(And for production, best practice would be to keep a counter so you can bail after N iterations, to avoid an infinite loop.)