My project uses Paperclip and Amazon S3, but I need a development/test environment that don't connect directly to S3. I've tried to use FakeS3, but with no luck, since I am using aws-sdk version 2 (and all other websites shows how to proceed using the v1).
There's a way to do that? How?
My Gemfile:
gem 'aws-sdk', '~> 2.5', '>= 2.5.3'
gem 'paperclip', '~> 5.1'
group :development, :test do
gem 'fakes3', '~> 0.2.4'
end
The aws-ruby-sdk v2 offers something that allow you to test your code that uses AWS sdk.
stub_data and stub_responses
If offers many options and one of them is:
# stub data in the constructor
client = Aws::S3::Client.new(stub_responses: {
list_buckets: { buckets: [{name: 'my-bucket' }] },
get_object: { body: 'data' },
})
client.list_buckets.buckets.map(&:name) #=> ['my-bucket']
client.get_object(bucket:'name', key:'key').body.read #=> 'data'
This way you control what gets returned by the SDK without needing to use the real service.
http://docs.aws.amazon.com/sdkforruby/api/Aws/ClientStubs.html
Related
For the sake of security internal Ruby gems in the Gemfile should always be referenced inside a source block so it never tries to fetch them from rubygems.org. I'd like to automate finding where people fail to do this, so would like to parse the Gemfile, find any gems that match our internal names, and check that rubygems.org isn't in their possible sources list.
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
I've seen you can parse the Gemfile
Bundler::Definition.build('Gemfile', '', {})
But can't find anything in the returned data structure that shows me the available / allowed sources per gem
If I include the Gemfile.lock I see more source info, but it doesn't seem right because every gem lists all my sources regardless of if they're in a source block
Bundler::Definition.build('Gemfile', 'Gemfile.lock', {}).
locked_gems.
specs.
map {|g| [g.full_name, g.source.remotes.map(&:hostname).join(', ')]}
=> ["rails-6.0.3.4", "my.private.gemserver, rubygems.org"],
["my-private-gem1-1.0.0", "my.private.gemserver, rubygems.org"],
["my-private-gem2-1.0.0", "my.private.gemserver, rubygems.org"]]
Any thoughts on how to parse the Gemfile to find that my-private-gem1 is outside a source block?
Figured it out finally, just took awhile digging through the Bundler methods - and a coworker's help.
Bundler::Definition.
build('Gemfile', '', nil).
dependencies.
map {|dep| [dep.name, dep.source&.remotes&.map(&:hostname)&.join(', ')]}
=>
[["rails", nil],
["my-private-gem1", nil],
["my-private-gem2", "my.private.gemserver"]]
Now I can easily search that resulting data structure for any private gems that aren't locked down to my private gem server.
Preface
While I was writing this answer, the OP found a Bundler-specific answer. However, I offer a more generalizable solution below. This solution also offers user feedback that may make it easier to fix the file.
Finding Candidate Gems by Column Alignment, with Whitelisting
If you can safely assume that your Gemfile is always properly indented, the KISS solution may be to simply identify the gems that aren't indented within a group definition. For example:
# example Gemfile to test against
GEMFILE = <<~'EOF'
source 'https://rubygems.org'
gem 'rails'
gem 'my-private-gem1' # this should be in the source block below
source PRIVATE_GEM_REPO do
gem 'my-private-gem2'
end
EOF
# gems that are acceptable in a non-group context
whitelist = Regexp.new %w[rails sass-rails webpacker].join(?|)
UngroupedGem = Struct.new :line_no, :line_txt, :gem_name
ungrouped_gems = []
GEMFILE.lines.each_with_index do |line_txt, line_no|
next if line_txt =~ whitelist or line_txt !~ /^\s*gem/
gem_name = line_txt.match(/(?<=(['"]))(.*?)(?=\1)/)[0]
ungrouped_gems.append(
UngroupedGem.new line_no.succ, line_txt, gem_name
).compact!
end
# tell the user what actions to take
if ungrouped_gems.any?
puts "Line No.\tGem Name"
ungrouped_gems.each { printf "%d\t\t%s\n", _1.line_no, _1.gem_name }
else
puts "No gems need to be moved."
end
With this example, it will print:
Line No. Gem Name
3 my-private-gem1
5 my-private-gem2
which will give you a solid idea of what lines in the Gemfile need to be moved, and which specific gems are involved.
While doing a create on storage_accounts
I get:
NoMethodError: undefined method allow_blob_public_access= for #Azure::Storage::Mgmt::V2019_06_01::Models::StorageAccountCreateParameter
This is the link to the StorageAccountCreateParameter on microsofts github.
My code looks like this:
sa_create_params = StorageModels::StorageAccountCreateParameters.new.tap do |sacp|
sacp.kind = 'StorageV2'
sacp.kind = payload['StorageAccountType'] if payload && payload['StorageAccountType']
sacp.sku = sku
sacp.location = params['region']
sacp.access_tier = 'hot'
sacp.access_tier = payload['AccessTier'] if payload && payload['AccessTier']
sacp.tags = system_tags(params)
sacp.allow_blob_public_access = false
end
Without the last line, regarding public access, it works just fine. I've tried upversioning the gems (hence the current version). And looking at their github it looks pretty self-explanatory. I'm at a loss, all help is much appreciated.
I'm using the following gems:
ms_rest_azure 0.11.0
azure_mgmt_storage 0.21.0
Update: it seems the name should be sacp.properties.allow_blob_public_acces as per link.
But this also throws a NoMethodError
I updated all azure gems to the latest version.
Relevant gems:
spec.add_dependency 'ms_rest_azure', '~> 0.12.0'
Spec.add_dependency 'azure_mgmt_storage', '~> 0.22.0'
This did the trick
I use free heroku instance to run my Dashing project. In result, it looses the value passed previously, when my instance sleeps. I was recommended to use Redis to keep history. I tryed to follow the instruction given here. In result I got the following config.ru (as part of my dashing project):
require 'dashing'
require 'redis-objects'
require 'yaml'
configure do
set :auth_token, 'my-token'
set :default_dashboard, 'def' # https://github.com/Shopify/dashing/wiki/How-To:-Change-the-default-dashboard
helpers do
def protected!
# Put any authentication code you want in here.
# This method is run before accessing any resource.
end
end
end
def redis?
ENV.has_key? 'REDISTOGO_URL'
end
if redis?
redis_uri = URI.parse(ENV['REDISTOGO_URL'])
Redis.current = Redis.new(:host => redis_uri.host,
:port => redis_uri.port,
:password => redis_uri.password)
set :history, Redis::HashKey.new('dashing-history')
elsif File.exists?(settings.history_file)
set history: YAML.load_file(settings.history_file)
else
set history: {}
end
map Sinatra::Application.assets_prefix do
run Sinatra::Application.sprockets
end
run Sinatra::Application
and the following Gemfile:
source 'https://rubygems.org'
gem 'dashing'
gem 'redis-objects'
## Remove this if you don't need a twitter widget.
gem 'twitter', '>= 5.9.0'
But it didn't help. What I did incorrectly?
I also tried to use this tutorial. But it was giving me an error at line redis_uri = URI.parse(ENV["REDISTOGO_URL"]) (something like wrong url is given).
The problem was that the app requires the add-on Redis To Go
If Redis To Go is configured, REDISTOGO_URL is added to environment variables, it will work
For more information on how to setup Redis To Go, read the heroku article
Adding Redis to an application provides benefits, you may be using RedisToGo to power simple Resque or Sidekiq jobs, or using the raw power of Redis 2.6 Lua Scripting to do some crazy fast operations. Redis can be used a database, but it’s often used as a complementary datastore. With over 140 commands, the possibilities are endless.
https://rubygems.org/api/v1/search.json provides only 30 gems, but in my local machine I can get all gems using gem list --remote.
I read http://guides.rubygems.org/rubygems-org-api/ , and an API which can get all gems in the list does not exist. How can I get the list via the web? Or, does anyone provide this?
I want the gem's name at a minimum but, if possible, I want to get the name and version.
I think gem list --remote is via web, so I can get all gem lists.
You could use https://rubygems.org/latest_specs.4.8.gz which returns gzipped marshal dumped array like this:
[["abscss", Gem::Version.new("0.0.1"), "ruby"],
["absee", Gem::Version.new("1.0"), "ruby"],
["absentee_camper", Gem::Version.new("0.0.7"), "ruby"],
["absgit", Gem::Version.new("0.3.0"), "ruby"],
["absinthe", Gem::Version.new("0.0.3"), "ruby"],
["absolute", Gem::Version.new("0.0.5"), "ruby"],
["AbsoluteRenamer", Gem::Version.new("1.1.2"), "ruby"],
["AbsoluteRenamer-date", Gem::Version.new("0.1.0"), "ruby"]]
But if you're in Ruby, I strongly recommend using the gem fetcher:
require 'rubygems/spec_fetcher'
fetcher = Gem::SpecFetcher.fetcher
tuples = fetcher.detect(:released) { true }
tuples is now an array of tuples of [Gem::NameTuple, Gem::Remote]. Some examples of what you can do with this:
tuples[1337][0].name # => GraphvizR
tuples[1337][0].version.to_s # => "0.1.0"
You can provide the page query parameter to paginate through the list, e.g.:
https://rubygems.org/api/v1/search.json?query=main&page=2
In the OpsCode Wiki there is there following documentation:
require 'ohai'
# ...
# Profit! ;-)
How can I print the JSON data provided by the 'ohai' command but using IRB? I tried to see the code in application.rb but I get empty data.
require 'ohai/application'
ohai = Ohai::System.new
ohai.json_pretty_print
=> "{\n\n}"
I am not trying to do this within Chef (or Shef), I just want to use the ohai gem itself, in my own app.
Poking about in the Ohai::Application class (what you get when you run ohai), #run_application instantiates Ohai::System and, unless it was configured by a file, it calls all_plugins to populate it with data.
Presumably, Ohai::System#all_plugins delegates the data collection to the lib/ohai/plugins directory.
$ irb -rohai
> system = Ohai::System.new
=> #<Ohai::System:0x00000100988950 #data={}, #seen_plugins={}, #providers={}, #plugin_path="", #hints={}>
> system.all_plugins
=> true
> puts system.to_json
{"languages":{"ruby":{"platform":"x86_64-darwin10.8.0","version":"1.9.2", ...
> system.to_json.size
=> 42395
I too searched for a simple and good library to do that...
I came across
Usagewatch
and after 5 minutes had my monitoring.rb done..
Install : gem install usagewatch
Install on mac : gem install usagewatch_ext
Refer to this article for more info