I have an external library that needs to be instantiated differently depending on whether the parameter is a local file or an online file.
Right now I have this (I assume the file is always online):
def initialize(path)
url_image = open(path)
#image = Magick::ImageList.new
#image.from_blob(url_image.read)
end
What would be the best way to differentiate if the file is locally stored? I have thought of doing this:
def initialize(path, is_online = true)
if is_online
url_image = open(path)
#image = Magick::ImageList.new
#image.from_blob(url_image.read)
else
#image = Magick::ImageList.new(path)
end
end
As a general rule if a URI has a host, it's not a local file.
require 'uri'
if URI(path).host.nil?
#local
else
#external
end
Then your interface doesn't need to change.
Related
I want to add onlyGenerateCoverageForSpecifiedTargets property to TestAction object programmatically. According to the documentation this property is not yet supported. So I need to add a custom property to an object. Also I need to add CodeCoverageTargets group.
Here is my code:
scheme = Xcodeproj::XCScheme.new
scheme.add_build_target(app_target)
scheme.set_launch_target(app_target)
scheme.add_test_target(target)
test_action = scheme.test_action
test_action.code_coverage_enabled = true
# add onlyGenerateCoverageForSpecifiedTargets = true
scheme.test_action = test_action
scheme.save_as(xcode_proj_dir, name)
Here is xml structure when I add property from Xcode GUI.
<TestAction
buildConfiguration = "Debug"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
shouldUseLaunchSchemeArgsEnv = "YES"
codeCoverageEnabled = "YES"
onlyGenerateCoverageForSpecifiedTargets = "YES">
<MacroExpansion>
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "D7CE66BC1C7DE6F700FC64CC"
BuildableName = "AppName.app"
BlueprintName = "AppName"
ReferencedContainer = "container:buddyui.xcodeproj">
</BuildableReference>
</MacroExpansion>
<CodeCoverageTargets>
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "D7CE66BC1C7DE6F700FC64CC"
BuildableName = "AppName.app"
BlueprintName = "AppName"
ReferencedContainer = "container:buddyui.xcodeproj">
</BuildableReference>
</CodeCoverageTargets>
I'll say it first: I know nothing about the Xcodeproj Gem nor the logic behind Xcode metadata. Take my code as a starter for further improvements.
You have a few ways of achieving what you asked:
MonkeyPatch Xcodeproj. That is what I did, sorry for that :-P
Extend Xcodeproj classes. That would be the recommended solution.
Manipulate the XML file or the XCScheme object directly, with REXML.
Here comes my proposal. I added a few methods to TestAction (based on the code of similar existing methods) and created the additional class CodeCoverageTargets (based on the class MacroExpansion). As I don't know how Xcode works, I chose to create the method add_code_coverage_targets in XCScheme instead of overwriting set_launch_target (where MacroExpansion is instantiated).
require 'xcodeproj'
class Xcodeproj::XCScheme
def add_code_coverage_targets(build_target)
code_cov_targets = CodeCoverageTargets.new(build_target)
test_action.add_code_coverage_targets(code_cov_targets)
end
class CodeCoverageTargets < XMLElementWrapper
def initialize(target_or_node = nil)
create_xml_element_with_fallback(target_or_node, 'CodeCoverageTargets') do
self.buildable_reference = BuildableReference.new(target_or_node) if target_or_node
end
end
def buildable_reference
#buildable_reference ||= BuildableReference.new #xml_element.elements['BuildableReference']
end
def buildable_reference=(ref)
#xml_element.delete_element('BuildableReference')
#xml_element.add_element(ref.xml_element)
#buildable_reference = ref
end
end
class TestAction
def only_generate_coverage_for_specified_targets?
string_to_bool(#xml_element.attributes['onlyGenerateCoverageForSpecifiedTargets'])
end
def only_generate_coverage_for_specified_targets=(flag)
#xml_element.attributes['onlyGenerateCoverageForSpecifiedTargets'] = bool_to_string(flag)
end
def code_coverage_targets
#xml_element.get_elements('CodeCoverageTargets').map do |node|
CodeCoverageTargets.new(node)
end
end
def add_code_coverage_targets(code_coverage_targets)
#xml_element.add_element(code_coverage_targets.xml_element)
end
end
end
You can use it like this:
xcode_proj_dir = 'Desktop/SO/66719313/DummyApp.xcodeproj'
xcode_proj = Xcodeproj::Project.open(xcode_proj_dir)
app_target = xcode_proj.targets.first
scheme = Xcodeproj::XCScheme.new
scheme.add_build_target(app_target)
scheme.set_launch_target(app_target)
#scheme.add_test_target(app_target)
scheme.add_code_coverage_targets(app_target) # new method
test_action = scheme.test_action
test_action.code_coverage_enabled = true
test_action.only_generate_coverage_for_specified_targets = true # new method
puts test_action
You can simply create a module which adds the behaviour you want (i.e. your method), but I'm not sure this will fix your real problem
module AddOnlyGenerateCoverageForSpecifiedTargets
attr_accessor :only_generate_coverage_for_specified_targets
end
Then include the module in the class you want it. To include it in one object only, include it in its singleton_class:
test_action.singleton_class.include AddOnlyGenerateCoverageForSpecifiedTargets
# Now you can use it
test_action.only_generate_coverage_for_specified_targets = true
Our processing server works mainly with TempFiles as it makes things easier on our side: no need to take care of deleting them as they get garbage collected or handle name collisions, etc.
Lately, we are having problems with TempFiles getting GCed too early in the process. Specially with one of our services that will convert a Foo file from a url to some Bar file and upload it to our servers.
For sake of clarity I added bellow a case scenario in order to make discussion easier and have an example at hand.
This workflow does the following:
Get a url as parameter
Download the Foo file as a TempFile
Duplicate it to a new TempFile
Download the related assets to TempFiles
Link the related assets into the local dup TempFile
Convert the Foo to Bar format
Upload it to our server
At times the conversion fail and everything points to the fact that our local Foo file is pointing to related assets that have been created and GCed before the conversion.
My two questions:
Is it possible that my TempFiles get GCed too early? I read about Ruby GCed system it was very conservative to avoid those scenarios.
How can I avoid this from happening? I could try to save all related assets from download_and_replace_uri(node) and passing them as a return to keep it alive while the instance of ConvertService is still existing. But I'm not sure if this would solve it.
myfile.foo
{
"buffers": [
{ "uri": "http://example.com/any_file.jpg" },
{ "uri": "http://example.com/any_file.png" },
{ "uri": "http://example.com/any_file.jpmp3" }
]
}
main.rb
ConvertService.new('http://example.com/myfile.foo')
ConvertService
class ConvertService
def initialize(url)
#url = url
#bar_file = Tempfile.new
end
def call
import_foo
convert_foo
upload_bar
end
private
def import_foo
#foo_file = ImportService.new(#url).call.edited_file
end
def convert_foo
`create-bar "#{#foo_file.path}" "#{#bar_file.path}"`
end
def upload_bar
UploadBarService.new(#bar_file).call
end
end
ImportService
class ImportService
def initialize(url)
#url = url
#edited_file ||= Tempfile.new
end
def call
download
duplicate
replace
end
private
def download
#original = DownloadFileService.new(#url).call.file
end
def duplicate
FileUtils.cp(#original.path, #edited_file.path)
end
def replace
file = File.read(#edited_file.path)
json = JSON.parse(file, symbolize_names: true)
json[:buffers]&.each do |node|
node[:uri] = DownloadFileService.new(node[:uri]).call.file.path
end
write_to_disk(#edited_file.path, json.to_json)
end
end
DownloadFileService
module Helper
class DownloadFileService < ApplicationHelperService
def initialize(url)
#url = url
#file = Tempfile.new
end
def call
uri = URI.parse(#url)
Net::HTTP.start(
uri.host,
uri.port,
use_ssl: uri.scheme == 'https'
) do |http|
response = http.request(Net::HTTP::Get.new(uri.path))
#file.binmode
#file.write(response.body)
#file.flush
end
end
end
end
UploadBarService
module Helper
class UploadBarService < ApplicationHelperService
def initialize(file)
#file = file
end
def call
HTTParty.post('http://example.com/upload', body: { file: #file })
# NOTE: End points returns the url for the uploaded file
end
end
end
Because of the complexity of your code and missing parts which may be obfuscated to us, the simple answer to your problem is to insure that your tempfile instance objects remain in memory throughout the lifecycle in which they are needed, otherwise they will get garbage collected immediately, removing the tempfile from the file system, and will lead to the the missing tempfile state you've encountered.
The Ruby Document for Tempfile states "When a Tempfile object is garbage collected, or when the Ruby interpreter exits, its associated temporary file is automatically deleted."
As per comments, others may find this conversation helpful when running into this problem.
I created a CLI program that uses Scraper class to scrape site. I am Using Nokogiri and Open-URI. The error on top is popping up. I looked online and did not find help.
I made sure the site doesn't have typos.
from the CLI class I create a new Scraper class using the site as arg
class KefotoScraper::CLI
attr_accessor :kefoto_scraper
def initialize
site = "https://www.kefotos.mx"
#kefoto_scraper = Scraper.new(site)
end
end
In Scraper I have the following code:
class Scraper
attr_accessor :doc, :product_names, :site, :name, :link
def initialize(site)
#site = site
#doc = doc
#product_names = product_names
#name = name
#link = link
#price_range = [].uniq
scrape_product
end
def get_html
#doc = Nokogiri::HTML(open(#site))
#product_names = doc.css(".navbar-nav li")
product_names
end
def scrape_product
get_html.each {|product|
#name = product.css("span").text
plink = product.css("a").attr("href").text
#link = "#{site}#{link}"
link_doc = Nokogiri::HTML(open(#link))
pr = link_doc.scan(/[\$£](\d{1,3}(,\d{3})*(\.\d*)?)/)
prices = pr_link.text
prices.each {|price|
if #price_range.include?(price[0]) == false
#price_range << price[0]
end
}
new_product = Products.new(#name, #price_range)
puts new_product
}
end
end
I get the following error:
scraper.rb:18:in `initialize': No such file or directory # rb_sysopen - https://www.kefotos.mx (Errno::ENOENT)
open by default operates on local files, not URLs. That error means "I can't find a file on your hard drive named https://www.kefotos.mx".
You can let it work on URIs by requiring the open-uri library:
require 'open-uri'
This will make your code work, but it is a much better practice to use a proper HTTP client to read HTTP resources, as an attacker could potentially use an overloaded open() to access files on your machine's hard drive.
For example, if you were to use just net/http:
# At the top of your scraper.rb:
require 'net/http'
# Then, in your class:
link_doc = Nokogiri::HTML(Net::HTTP.get(URI(#link)))
I will go ahead and apologize upfront as I am new to ruby and rails and I cannot for the life of me figure out how to implement using hashids in my project. The project is a simple image host. I have it already working using Base58 to encode the sql ID and then decode it in the controller. However I wanted to make the URLs more random hence switching to hashids.
I have placed the hashids.rb file in my lib directory from here: https://github.com/peterhellberg/hashids.rb
Now some of the confusion starts here. Do I need to initialize hashids on every page that uses hashids.encode and hashids.decode via
hashids = Hashids.new("mysalt")
I found this post (http://zogovic.com/post/75234760043/youtube-like-ids-for-your-activerecord-models) which leads me to believe I can put it into an initializer however after doing that I am still getting NameError (undefined local variable or method `hashids' for ImageManager:Class)
so in my ImageManager.rb class I have
require 'hashids'
class ImageManager
class << self
def save_image(imgpath, name)
mime = %x(/usr/bin/exiftool -MIMEType #{imgpath})[34..-1].rstrip
if mime.nil? || !VALID_MIME.include?(mime)
return { status: 'failure', message: "#{name} uses an invalid format." }
end
hash = Digest::MD5.file(imgpath).hexdigest
image = Image.find_by_imghash(hash)
if image.nil?
image = Image.new
image.mimetype = mime
image.imghash = hash
unless image.save!
return { status: 'failure', message: "Failed to save #{name}." }
end
unless File.directory?(Rails.root.join('uploads'))
Dir.mkdir(Rails.root.join('uploads'))
end
#File.open(Rails.root.join('uploads', "#{Base58.encode(image.id)}.png"), 'wb') { |f| f.write(File.open(imgpath, 'rb').read) }
File.open(Rails.root.join('uploads', "#{hashids.encode(image.id)}.png"), 'wb') { |f| f.write(File.open(imgpath, 'rb').read) }
end
link = ImageLink.new
link.image = image
link.save
#return { status: 'success', message: Base58.encode(link.id) }
return { status: 'success', message: hashids.encode(link.id) }
end
private
VALID_MIME = %w(image/png image/jpeg image/gif)
end
end
And in my controller I have:
require 'hashids'
class MainController < ApplicationController
MAX_FILE_SIZE = 10 * 1024 * 1024
MAX_CACHE_SIZE = 128 * 1024 * 1024
#links = Hash.new
#files = Hash.new
#tstamps = Hash.new
#sizes = Hash.new
#cache_size = 0
class << self
attr_accessor :links
attr_accessor :files
attr_accessor :tstamps
attr_accessor :sizes
attr_accessor :cache_size
attr_accessor :hashids
end
def index
end
def transparency
end
def image
##imglist = params[:id].split(',').map{ |id| ImageLink.find(Base58.decode(id)) }
#imglist = params[:id].split(',').map{ |id| ImageLink.find(hashids.decode(id)) }
end
def image_direct
#linkid = Base58.decode(params[:id])
linkid = hashids.decode(params[:id])
file =
if Rails.env.production?
puts "#{Base58.encode(ImageLink.find(linkid).image.id)}.png"
File.open(Rails.root.join('uploads', "#{Base58.encode(ImageLink.find(linkid).image.id)}.png"), 'rb') { |f| f.read }
else
puts "#{hashids.encode(ImageLink.find(linkid).image.id)}.png"
File.open(Rails.root.join('uploads', "#{hashids.encode(ImageLink.find(linkid).image.id)}.png"), 'rb') { |f| f.read }
end
send_data(file, type: ImageLink.find(linkid).image.mimetype, disposition: 'inline')
end
def upload
imgparam = params[:image]
if imgparam.is_a?(String)
name = File.basename(imgparam)
imgpath = save_to_tempfile(imgparam).path
else
name = imgparam.original_filename
imgpath = imgparam.tempfile.path
end
File.chmod(0666, imgpath)
%x(/usr/bin/exiftool -all= -overwrite_original #{imgpath})
logger.debug %x(which exiftool)
render json: ImageManager.save_image(imgpath, name)
end
private
def save_to_tempfile(url)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = uri.scheme == 'https'
http.start do
resp = http.get(uri.path)
file = Tempfile.new('urlupload', Dir.tmpdir, :encoding => 'ascii-8bit')
file.write(resp.body)
file.flush
return file
end
end
end
Then in my image.html.erb view I have this:
<%
#imglist.each_with_index { |link, i|
id = hashids.encode(link.id)
ext = link.image.mimetype.split('/')[1]
if ext == 'jpeg'
ext = 'jpg'
end
puts id + '.' + ext
%>
Now if I add
hashids = Hashids.new("mysalt")
in ImageManager.rb main_controller.rb and in my image.html.erb I am getting this error:
ActionView::Template::Error (undefined method `id' for #<Array:0x000000062f69c0>)
So all in all implementing hashids.encode/decode is not as easy as implementing Base58.encode/decode and I am confused on how to get it working... Any help would be greatly appreciated.
I would suggest loading it as a gem by including it into your Gemfile and running bundle install. It will save you the hassle of requiring it in every file and allow you to manage updates using Bundler.
Yes, you do need to initialize it wherever it is going to be used with the same salt. Would suggest that you define the salt as a constant, perhaps in application.rb.
The link you provided injects hashids into ActiveRecord, which means it will not work anywhere else. I would not recommend the same approach as it will require a high level of familiarity with Rails.
You might want to spend some time understanding ActiveRecord and ActiveModel. Will save you a lot of reinventing the wheel. :)
Before everythink you should just to test if Hashlib is included in your project, you can run command rails c in your project folder and make just a small test :
>> my_id = ImageLink.last.id
>> puts Hashids.new(my_id)
If not working, add the gem in gemfile (that anyway make a lot more sence).
Then, I think you should add a getter for your hash_id in your ImageLink model.
Even you don't want to save your hash in the database, this hash have it's pllace in your model. See virtual property for more info.
Remember "Skinny Controller, Fat Model".
class ImageLink < ActiveRecord::Base
def hash_id()
# cache the hash
#hash_id ||= Hashids.new(id)
end
def extension()
# you could add the logic of extension here also.
ext = image.mimetype.split('/')[1]
if ext == 'jpeg'
'jpg'
else
ext
end
end
end
Change the return in your ImageManager#save_image
link = ImageLink.new
link.image = image
# Be sure your image have been saved (validation errors, etc.)
if link.save
{ status: 'success', message: link.hash_id }
else
{status: 'failure', message: link.errors.join(", ")}
end
In your template
<%
#imglist.each_with_index do |link, i|
puts link.hash_id + '.' + link.extension
end # <- I prefer the do..end to not forgot the ending parenthesis
%>
All this code is not tested...
I was looking for something similar where I can disguise the ids of my records. I came across act_as_hashids.
https://github.com/dtaniwaki/acts_as_hashids
This little gem integrates seamlessly. You can still find your records through the ids. Or with the hash. On nested records you can use the method with_hashids.
To get the hash you use to_param on the object itself which result in a string similar to this ePQgabdg.
Since I just implemented this I can't tell how useful this gem will be. So far I just had to adjust my code a little bit.
I also gave the records a virtual attribute hashid so I can access it easily.
attr_accessor :hashid
after_find :set_hashid
private
def set_hashid
self.hashid = self.to_param
end
I am making a command line tool and I am using yaml to make a config file. But right now I can only access the tool when I am in the same directory as that of myprogram.yml.
private
CONFIG_FILE = 'myprogram.yml'
def write_config
config = {}
config['username']=#username
config['password']=#password
File.open(CONFIG_FILE, 'w') do |f|
f.write config.to_yaml
end
end
def read_config
config = YAML.load_file(CONFIG_FILE)
#username = config['username']
#password = config['password']
end
How can I make this file to be accessed from any directory on my computer?
You'll want to give the absolute directory of the myprogram.yml file. Right now it will look in the directory where you are executing the ruby script from. By making it absolute, the script can run anywhere and know where to find the config file.
Example:
private
CONFIG_FILE = '/Users/myuser/config/myprogram.yml'
def write_config
config = {}
config['username']=#username
config['password']=#password
File.open(CONFIG_FILE, 'w') do |f|
f.write config.to_yaml
end
end
def read_config
config = YAML.load_file(CONFIG_FILE)
#username = config['username']
#password = config['password']
end