RubyMotion async programming with BubbleWrap - ruby

I am confused with how to write decent code when using a lot of asynchronous code.
In the following code snippet I log in to get the authentication cookie and use that cookie for the next request to get a list of projects name (as an example):
def self.populateProjectsTable(projects_controller)
payload = {email: "email", password: "pass"}
HTTP.post("http://example.com/login", {payload: payload}) do |response|
authCookie = response.headers['Set-Cookie']
HTTP.get("http://example.com/projects.json", {cookie: authCookie}) do |response|
projects = JSON.parse(response.body.to_str)
projects_controller.projects = projects
projects_controller.reloadData
end
end
end
While this will work the code feels dirty. Not really following the single responsibility principle. I would like to extract this in a few methods:
def self.populateProjectsTable(projects_controller)
#taskList = TaskList.new
#taskList.doLogin
projects = #taskList.getProjects
projects_controller.projects = projects
projects_controller.reloadData
end
def doLogin
payload = {email: "email", password: "pass"}
HTTP.post("http://example.com/login", {payload: payload}) do |response|
#authCookie = response.headers['Set-Cookie']
end
end
def getProjects
HTTP.get("http://example.com/projects.json", {cookie: #authCookie}) do |response|
projects = JSON.parse(response.body.to_str)
end
end
This obviously does not work. The getProjects method is called before doLogin is finished and the projects are only known in the scope of the block, not giving back the data to the populateProjectsTable method.
How does one program such applications without the nesting shown in the first example?

You're not going to totally get away from the nesting. Taking Alan's answer and massaging it a bit, this is what I've come up with. It involves passing a block through a couple of methods.
def self.populateProjectsTable(projects_controller)
#taskList = TaskList.new
#taskList.loginAndGetProjects do |projects|
projects_controller.projects = projects
projects_controller.reloadData
end
end
def loginAndGetProjects(&block)
payload = {email: "email", password: "pass"}
HTTP.post("http://example.com/login", {payload: payload}) do |response|
#authCookie = response.headers['Set-Cookie']
getProjects(&block)
end
end
def getProjects(&block)
HTTP.get("http://example.com/projects.json", {cookie: #authCookie}) do |response|
projects = JSON.parse(response.body.to_str)
block.call(projects)
end
end

I've had a similar problem trying to wrap methods that themselves took blocks. I wanted the new wrapper methods to still be able to take blocks. Here's what I did in ParseModel:
# with block:
# ParseModel::Cloud.callFunction("myFunction", {"myParam" => "myValue"}) do |result, error|
# # do something...
# end
# without block:
# ParseModel::Cloud.callFunction("myFunction", {"myParam" => "myValue"})
module ParseModel
class Cloud
def self.callFunction(function, params={}, &block)
return PFCloud.callFunction(function, withParameters:params) unless block_given?
PFCloud.callFunctionInBackground(function, withParameters:params, block:lambda do |result, error|
block.call(result, error)
end)
end
end
end
Applying this concept to your problem, you could rewrite your methods to take blocks themselves. Here's a bit of a refactor that I think might be helpful:
def self.populateProjectsTable(projects_controller)
#taskList = TaskList.new
#taskList.doLogin do |login_response|
authCookie = login_response.headers['Set-Cookie']
#taskList.getProjects(authCookie) do |projects_response|
projects = JSON.parse(projects_response.body.to_str)
projects_controller.projects = projects
projects_controller.reloadData
end
end
end
def doLogin(&block)
payload = {email: "email", password: "pass"}
HTTP.post("http://example.com/login", {payload: payload}) do |response|
block.call(response)
end
end
def getProjects(cookie, &block)
HTTP.get("http://example.com/projects.json", {cookie: cookie}) do |response|
block.call(response)
end
end
I don't think you're totally out of the woods regarding SRP, but this should be a good start.

+1 for Jamon's answer.
I might suggest using a class to manage your session and splitting out the API into a module if you like SRP. This is especially helpful as you add additional API calls. Here I queue up requests that will be satisfied once login is completed. Later you can add handling for timeouts, etc.
module ProjectApi
def get_projects(&block)
with_session do
HTTP.get("http://example.com/projects.json", {cookie: #auth_cookie}) do |response|
projects = JSON.parse(response.body.to_str)
block.call(projects)
end
end
end
end
class MySession
include ProjectApi
def initialize(login, password)
#login = login
#password = password
#state = nil
#requests = []
end
def active?
#state == :active
end
def with_session(&block)
#requests << &block
active? ? handle_requests : login(true)
end
private
def login(do_handle_requests = false)
payload = {login: #login, password: #password}
#state = nil
HTTP.post("http://example.com/login", {payload: payload}) do |response|
#state = :active
#auth_cookie = response.headers['Set-Cookie']}
handle_requests if do_handle_requests
end
end
def handle_requests
while request = #requests.shift do
request.call
end if active?
end
end
def self.populateProjectsTable(projects_controller)
#session ||= MySession.new('mylogin', 'mypassword')
#session.get_projects do |projects|
projects_controller.projects = projects
projects_controller.reloadData
end
end

Related

How to combine yield with retry loops while preserving original context?

require_relative 'config/environment'
HTTP_ERRORS = [
RestClient::Exception
]
module API
class Client
def initialize
#client = RawClient.new
end
def search(params = {})
call { #client.search(params) }
end
def call
raise 'No block specified' unless block_given?
loop do # Keep retrying on error
begin
return yield
rescue *HTTP_ERRORS => e
puts "#{e.response&.request.url}"
sleep 5
end
end
end
end
class RawClient
BASE_URL = 'https://www.google.com'
def search(params = {})
go "search/#{params.delete(:query)}", params
end
private
def go(path, params = {})
RestClient.get(BASE_URL + '/' + path, params: params)
end
end
end
API::Client.new.search(query: 'tulips', per_page: 10)
Will output
https://www.google.com/search/tulips?per_page=10 # First time
https://www.google.com/search/?per_page=10 # On retry
I thought I was being clever here: have a flexible and unified way to pass parameters (ie. search(query: 'tulips', per_page: 10)) and let the client implementation figure out what goes into the url itself (ie. query) and what should be passed as GET parameters (ie. per_page).
But the query param is lost from the params after the first retry, because the hash is passed by reference and delete makes a permanent change to it. The second time yield is called, it apparently preserves the context and params won't have the deleted query anymore in it.
What would be an elegant way to solve this? Doing call { #client.search(params.dup) } seems a bit excessive.

How do I implement hashids in ruby on rails

I will go ahead and apologize upfront as I am new to ruby and rails and I cannot for the life of me figure out how to implement using hashids in my project. The project is a simple image host. I have it already working using Base58 to encode the sql ID and then decode it in the controller. However I wanted to make the URLs more random hence switching to hashids.
I have placed the hashids.rb file in my lib directory from here: https://github.com/peterhellberg/hashids.rb
Now some of the confusion starts here. Do I need to initialize hashids on every page that uses hashids.encode and hashids.decode via
hashids = Hashids.new("mysalt")
I found this post (http://zogovic.com/post/75234760043/youtube-like-ids-for-your-activerecord-models) which leads me to believe I can put it into an initializer however after doing that I am still getting NameError (undefined local variable or method `hashids' for ImageManager:Class)
so in my ImageManager.rb class I have
require 'hashids'
class ImageManager
class << self
def save_image(imgpath, name)
mime = %x(/usr/bin/exiftool -MIMEType #{imgpath})[34..-1].rstrip
if mime.nil? || !VALID_MIME.include?(mime)
return { status: 'failure', message: "#{name} uses an invalid format." }
end
hash = Digest::MD5.file(imgpath).hexdigest
image = Image.find_by_imghash(hash)
if image.nil?
image = Image.new
image.mimetype = mime
image.imghash = hash
unless image.save!
return { status: 'failure', message: "Failed to save #{name}." }
end
unless File.directory?(Rails.root.join('uploads'))
Dir.mkdir(Rails.root.join('uploads'))
end
#File.open(Rails.root.join('uploads', "#{Base58.encode(image.id)}.png"), 'wb') { |f| f.write(File.open(imgpath, 'rb').read) }
File.open(Rails.root.join('uploads', "#{hashids.encode(image.id)}.png"), 'wb') { |f| f.write(File.open(imgpath, 'rb').read) }
end
link = ImageLink.new
link.image = image
link.save
#return { status: 'success', message: Base58.encode(link.id) }
return { status: 'success', message: hashids.encode(link.id) }
end
private
VALID_MIME = %w(image/png image/jpeg image/gif)
end
end
And in my controller I have:
require 'hashids'
class MainController < ApplicationController
MAX_FILE_SIZE = 10 * 1024 * 1024
MAX_CACHE_SIZE = 128 * 1024 * 1024
#links = Hash.new
#files = Hash.new
#tstamps = Hash.new
#sizes = Hash.new
#cache_size = 0
class << self
attr_accessor :links
attr_accessor :files
attr_accessor :tstamps
attr_accessor :sizes
attr_accessor :cache_size
attr_accessor :hashids
end
def index
end
def transparency
end
def image
##imglist = params[:id].split(',').map{ |id| ImageLink.find(Base58.decode(id)) }
#imglist = params[:id].split(',').map{ |id| ImageLink.find(hashids.decode(id)) }
end
def image_direct
#linkid = Base58.decode(params[:id])
linkid = hashids.decode(params[:id])
file =
if Rails.env.production?
puts "#{Base58.encode(ImageLink.find(linkid).image.id)}.png"
File.open(Rails.root.join('uploads', "#{Base58.encode(ImageLink.find(linkid).image.id)}.png"), 'rb') { |f| f.read }
else
puts "#{hashids.encode(ImageLink.find(linkid).image.id)}.png"
File.open(Rails.root.join('uploads', "#{hashids.encode(ImageLink.find(linkid).image.id)}.png"), 'rb') { |f| f.read }
end
send_data(file, type: ImageLink.find(linkid).image.mimetype, disposition: 'inline')
end
def upload
imgparam = params[:image]
if imgparam.is_a?(String)
name = File.basename(imgparam)
imgpath = save_to_tempfile(imgparam).path
else
name = imgparam.original_filename
imgpath = imgparam.tempfile.path
end
File.chmod(0666, imgpath)
%x(/usr/bin/exiftool -all= -overwrite_original #{imgpath})
logger.debug %x(which exiftool)
render json: ImageManager.save_image(imgpath, name)
end
private
def save_to_tempfile(url)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = uri.scheme == 'https'
http.start do
resp = http.get(uri.path)
file = Tempfile.new('urlupload', Dir.tmpdir, :encoding => 'ascii-8bit')
file.write(resp.body)
file.flush
return file
end
end
end
Then in my image.html.erb view I have this:
<%
#imglist.each_with_index { |link, i|
id = hashids.encode(link.id)
ext = link.image.mimetype.split('/')[1]
if ext == 'jpeg'
ext = 'jpg'
end
puts id + '.' + ext
%>
Now if I add
hashids = Hashids.new("mysalt")
in ImageManager.rb main_controller.rb and in my image.html.erb I am getting this error:
ActionView::Template::Error (undefined method `id' for #<Array:0x000000062f69c0>)
So all in all implementing hashids.encode/decode is not as easy as implementing Base58.encode/decode and I am confused on how to get it working... Any help would be greatly appreciated.
I would suggest loading it as a gem by including it into your Gemfile and running bundle install. It will save you the hassle of requiring it in every file and allow you to manage updates using Bundler.
Yes, you do need to initialize it wherever it is going to be used with the same salt. Would suggest that you define the salt as a constant, perhaps in application.rb.
The link you provided injects hashids into ActiveRecord, which means it will not work anywhere else. I would not recommend the same approach as it will require a high level of familiarity with Rails.
You might want to spend some time understanding ActiveRecord and ActiveModel. Will save you a lot of reinventing the wheel. :)
Before everythink you should just to test if Hashlib is included in your project, you can run command rails c in your project folder and make just a small test :
>> my_id = ImageLink.last.id
>> puts Hashids.new(my_id)
If not working, add the gem in gemfile (that anyway make a lot more sence).
Then, I think you should add a getter for your hash_id in your ImageLink model.
Even you don't want to save your hash in the database, this hash have it's pllace in your model. See virtual property for more info.
Remember "Skinny Controller, Fat Model".
class ImageLink < ActiveRecord::Base
def hash_id()
# cache the hash
#hash_id ||= Hashids.new(id)
end
def extension()
# you could add the logic of extension here also.
ext = image.mimetype.split('/')[1]
if ext == 'jpeg'
'jpg'
else
ext
end
end
end
Change the return in your ImageManager#save_image
link = ImageLink.new
link.image = image
# Be sure your image have been saved (validation errors, etc.)
if link.save
{ status: 'success', message: link.hash_id }
else
{status: 'failure', message: link.errors.join(", ")}
end
In your template
<%
#imglist.each_with_index do |link, i|
puts link.hash_id + '.' + link.extension
end # <- I prefer the do..end to not forgot the ending parenthesis
%>
All this code is not tested...
I was looking for something similar where I can disguise the ids of my records. I came across act_as_hashids.
https://github.com/dtaniwaki/acts_as_hashids
This little gem integrates seamlessly. You can still find your records through the ids. Or with the hash. On nested records you can use the method with_hashids.
To get the hash you use to_param on the object itself which result in a string similar to this ePQgabdg.
Since I just implemented this I can't tell how useful this gem will be. So far I just had to adjust my code a little bit.
I also gave the records a virtual attribute hashid so I can access it easily.
attr_accessor :hashid
after_find :set_hashid
private
def set_hashid
self.hashid = self.to_param
end

API integration error HTTParty

I'm learning how to work with HTTParty and API and I'm having an issue with my code.
Users/admin/.rbenv/versions/2.0.0-p481/lib/ruby/2.0.0/uri/generic.rb:214:in `initialize': the scheme http does not accept registry part: :80 (or bad hostname?)
I've tried using debug_output STDOUT both as an argument to my method and after including HTTParty to have a clue but with no success. Nothing gets displayed:
require 'httparty'
class LolObserver
include HTTParty
default_timeout(1) #timeout after 1 second
attr_reader :api_key, :playerid
attr_accessor :region
def initialize(region,playerid,apikey)
#region = region_server(region)
#playerid = playerid
#api_key = apikey
end
def region_server(region)
case region
when "euw"
self.class.base_uri "https://euw.api.pvp.net"
self.region = "EUW1"
when "na"
self.class.base_uri "https://na.api.pvp.net"
self.region = "NA1"
end
end
def handle_timeouts
begin
yield
#Timeout::Error, is raised if a chunk of the response cannot be read within the read_timeout.
#Timeout::Error, is raised if a connection cannot be created within the open_timeout.
rescue Net::OpenTimeout, Net::ReadTimeout
#todo
end
end
def base_path
"/observer-mode/rest/consumer/getSpectatorGameInfo"
end
def current_game_info
handle_timeouts do
url = "#{ base_path }/#{region}/#{playerid}?api_key=#{api_key}"
puts '------------------------------'
puts url
HTTParty.get(url,:debug_output => $stdout)
end
end
end
I verified my URL which is fine so I'm lost as to where the problem is coming from.
I tested with a static base_uri and it doesn't change anything.
The odd thing is when I do:
HTTParty.get("https://euw.api.pvp.net/observer-mode/rest/consumer/getSpectatorGameInfo/EUW1/randomid?api_key=myapikey")
Everything is working fine and I'm getting a response.
HTTParty doesn't seem to like the way you set your base_uri.
Unless you need it to be like that just add another attr_reader called domain and it will work.
require 'httparty'
class LolObserver
include HTTParty
default_timeout(1) #timeout after 1 second
attr_reader :api_key, :playerid, :domain
attr_accessor :region
def initialize(region,playerid,apikey)
#region = region_server(region)
#playerid = playerid
#api_key = apikey
end
def region_server(region)
case region
when "euw"
#domain = "https://euw.api.pvp.net"
self.region = "EUW1"
when "na"
#domain = "https://na.api.pvp.net"
self.region = "NA1"
end
end
def handle_timeouts
begin
yield
#Timeout::Error, is raised if a chunk of the response cannot be read within the read_timeout.
#Timeout::Error, is raised if a connection cannot be created within the open_timeout.
rescue Net::OpenTimeout, Net::ReadTimeout
#todo
end
end
def base_path
"/observer-mode/rest/consumer/getSpectatorGameInfo"
end
def current_game_info
handle_timeouts do
url = "#{domain}/#{ base_path }/#{region}/#{playerid}?api_key=#{api_key}"
puts '------------------------------'
puts url
HTTParty.get(url,:debug_output => $stdout)
end
end
end

How to test that a block is called within a thread?

I am working on wrapping the ruby-mqtt gem into a class which implements a subscribe and publish method. The subscribe method connects to the server and listens in a separate thread because this call is synchronous.
module PubSub
class MQTT
attr_accessor :host, :port, :username, :password
def initialize(params = {})
params.each do |attr, value|
self.public_send("#{attr}=", value)
end if params
super()
end
def connection_options
{
remote_host: self.host,
remote_port: self.port,
username: self.username,
password: self.password,
}
end
def subscribe(name, &block)
channel = name
connect_opts = connection_options
code_block = block
::Thread.new do
::MQTT::Client.connect(connect_opts) do |c|
c.get(channel) do |topic, message|
puts "channel: #{topic} data: #{message.inspect}"
code_block.call topic, message
end
end
end
end
def publish(channel = nil, data)
::MQTT::Client.connect(connection_options) do |c|
c.publish(channel, data)
end
end
end
end
I have a test that I have written using rspec to test the class but it does not pass.
mqtt = ::PubSub::MQTT.new({host: "localhost",port: 1883})
block = lambda { |channel, data| puts "channel: #{channel} data: #{data.inspect}"}
block.should_receive(:call).with("channel", {"some" => "data"})
thr = mqtt.subscribe("channel", &block)
mqtt.publish("channel", {"some" => "data"})
When I run the following ruby-mqtt-example I have now problems at all.
uri = URI.parse ENV['CLOUDMQTT_URL'] || 'mqtt://localhost:1883'
conn_opts = {
remote_host: uri.host,
remote_port: uri.port,
username: uri.user,
password: uri.password,
}
# Subscribe example
Thread.new do
puts conn_opts
MQTT::Client.connect(conn_opts) do |c|
# The block will be called when you messages arrive to the topic
c.get('test') do |topic, message|
puts "#{topic}: #{message}"
end
end
end
# Publish example
puts conn_opts
MQTT::Client.connect(conn_opts) do |c|
# publish a message to the topic 'test'
loop do
c.publish('test', 'Hello World')
sleep 1
end
end
So my question is, what am I doing wrong when I simply create a class and separate out the publish and subscribe logic? My guess is that it has something to do with Threading in the function call but I can't seem to figure it out. Any help is much appreciated.
UPDATE
I believe I know why the test is not passing and it is because when I pass a lambda in to subscribe expecting it to receive a call it actually will not receive the call when it exits the method or until publish is called. So I would like to rephrase the question to: How do I test that a block is called within a thread? If someone answers, "you don't", then the question is: How do you test that block is being called in an infinite loop like in the example of calling get within ruby-mqtt gem.
The RSpec expectations machinery will work fine with threads, as evidenced by the following example, which passes:
def foo(&block)
block.call(42)
end
describe "" do
it "" do
l = lambda {}
expect(l).to receive(:call).with(42)
Thread.new { foo(&l) }.join
end
end
The join waits for the thread(s) to finish before going further.

Same request sent twice has two different responses

Please consider this test:
def test_ok_on_second_request
bad_response = #request.get "/bad-response"
assert_equal 404, bad_response.status
good_response = #request.get "/test-title"
assert_equal 200, good_response.status
assert_equal "text/html", good_response.content_type
end
I have assured that /test-title is a valid path. The assertion that's supposed to return 200 is in fact returning 404. How is Rack behaving in order to return two different results for the same request?
This is the code for the Server class inside the project:
module Blogrite
class Server
attr_accessor :status, :mimetype, :body, :provider
def initialize *args, &block
#status, #mimetype = 200, "text/html"
provider = args[0][:with].nil? ? :filesystem : args[0][:with]
#provider = Blogrite.const_get(provider.capitalize).new
# p "Server is running with #{#provider.class}."
end
def call env
begin
article = go env['PATH_INFO'].delete("/")
rescue Blogrite::Article::NoBodyError
#status = 404
end
#status = 404 if !article
#status = 403 if env["REQUEST_METHOD"] == 'POST'
#mimetype = "text/css" if env["PATH_INFO"].include?("css")
#body = if article then article.render
elsif env.respond_to?(:to_yaml) then "<pre>#{env.to_yaml}</pre>"
else "oops"
end
[#status,{ "Content-Type" => #mimetype},[#body]]
end
def go path
f = #provider.fetch path
Article.parse f unless f.nil?
end
end
end
The whole workflow is too big for me to paste it in but you can check the project out on Github. I appreciate your help, thank you.
The solution for the problem is as simple as initializing #status inside the call function.
class Server
attr_accessor :status, :mimetype, :body, :provider
def initialize *args, &block
- #status, #mimetype = 200, "text/html"
provider = args[0][:with].nil? ? :filesystem : args[0][:with]
#provider = Blogrite.const_get(provider.capitalize).new
# p "Server is running with #{#provider.class}."
end
def call env
begin
- article = go env['PATH_INFO'].delete("/")
+ #status, #mimetype = 200, "text/html"
+ article = go env['PATH_INFO'].delete("/")
rescue Blogrite::Article::NoBodyError
#status = 404
end
That way the rack instance – that is called only once – stays out of the request's way. Every call function should have its own defaults, not the server class.
Thanks to #rubenfonseca for helping me out.

Resources