Environment variables locally and Heroku - heroku

I have a sinatra app in which i have a yml file to set environment variables, i call them using this method
module MyConfig
def config
environment = ENV["RACK_ENV"] || "development"
YAML.load_file("./config/config.yml")[environment]
end
end
so when i want to use a variable i do this for example
aws_access_key_id = config['aws_access_key']
I have a .gitignore file that ignores config.yml when pushing to github for example.So when I push to heroku these environment variables will not be accessible?
So this leaves me with using the heroku way of setting them like so
heroku config:add aws_access_key= myapikey
but heroku accesses these like
aws_access_key_id = ENV['aws_access_key']
How can i set my dev environment to use method config and heroku use ENV, am i looking at this the wrong way? or does my config method do this for me?
Any help appreciated
RAKEFILE
require 'active_support/core_ext'
require './config/config.rb'
require 'bundler/setup'
Bundler.require(:default)
include MyConfig
AssetSync.configure do |con|
con.fog_provider = 'AWS'
con.fog_region = 'eu-west-1'
con.fog_directory = config['fog_directory']
con.aws_access_key_id = config['aws_access_key']
con.aws_secret_access_key = config['aws_secret_key']
con.prefix = "assets"
con.public_path = Pathname("./public")
end
namespace :assets do
desc "Precompile assets"
task :precompile do
AssetSync.sync
end
end

Update:
I now use the dotenv gem instead of the example below. So instead of ignoring the env.rb file, I now ignore the .env file with Git.
Original post:
Try this,
# /env.rb
ENV['aws_bucket'] = 'my_bucket'
ENV['aws_access_key'] = 'my_access_key'
ENV['aws_access_secret'] = 'my_access_secret'
This file sets the same ENV values as heroku config would do.
# /config.rb
require './env' if File.exists?('env.rb')
The env.rb will only get required if it exists.
# /.gitignore
/env.rb
The env.rb has been added to the .gitignore file so it isn't kept in Git.
You would then access the values using ENV['key'] instead of config['key'].
You might need to change the path to the env.rb if it's not in the same directory as the config.rb file.
EDIT:
From looking at your Rakefile in the previous question, you need to change it to this:
# Rakefile
require 'bundler/setup'
Bundler.require(:default)
require './env' if File.exists?('env.rb')
AssetSync.configure do |con|
con.fog_provider = 'AWS'
con.fog_region = 'eu-west-1'
con.fog_directory = ENV['aws_bucket']
con.aws_access_key_id = ENV['aws_access_key']
con.aws_secret_access_key = ENV['aws_access_secret']
con.prefix = "assets"
con.public_path = Pathname("./public")
end
namespace :assets do
desc "Precompile assets"
task :precompile do
AssetSync.sync
end
end
I've assumed that the only method in /config/config.rb was the config method so I've removed the,
require './config/config.rb'
include MyConfig
And swapped the config[key] for the ENV[key] values defined in env.rb. You may need to change the key names to match up.

You could delete the yaml, and describe the environment variables in a .env file then start your app with foreman start. See https://devcenter.heroku.com/articles/config-vars#local-setup
Or keep your hybrid system, where you load a yaml in dev, and use environment variables on heroku.

I do something similar to Sam's suggestion, but a little bit different. I have a YAML config file too, but I wrap the reading of it in a Rake task, which then runs the app.
# in the Rakefile
require 'yaml'
def set_connstring
s = %Q!postgres://#{ENV["DB_APP"]}#localhost/#{ENV["DB_APP"]}!
ENV['DATABASE_URL'] ||= ENV["RACK_ENV"] == "test" ? "#{s}.test" : s
end
def basic_environment
warn " Setting up environment..."
file = File.expand_path( File.join File.dirname(__FILE__), "./config.yml" )
if File.exist? file
YAML.load_file(file).each do |k,v|
warn "-> #{k}"
ENV[k.upcase] = v
end
end
set_connstring()
end
namespace :app do
desc "Set up the environment locally"
task :environment do
basic_environment()
end
desc "Run the app locally"
task :run_local => "app:environment" do
exec "bin/rackup config.ru -p #{ENV['RUN_LOCAL_PORT']}"
end
end
It means I can run it locally without any code inside the app to deal with this.
Edit: a quick aside, I notice you have Bundler.require(:default) in your Rakefile. If you use bundle install --binstubs then Bundler installs all executables into a dir named "bin/" within the project. Then, if you run any of those executables they automatically use the libraries installed by Bundler, no need to require via Bundler. See http://gembundler.com/v1.2/man/bundle-exec.1.html.

Exporting directly from heroku admin:
Settings -> Reveal Config Vars
Then open browser js console, paste this and type enter...
k=[];
$(".config-var-list input").map(function(y, x){k.push($(x).val())});
v=[];
$(".config-var-list textarea").map(function(y, x){v.push($(x).val())});
ret="";
k.map(function(x, i){ret+=k[i]+"\t"+v[2*i]+"\n"});
console.info(ret);

Related

Ruby Gem: Install configuration files to the user-home directory

I have to make a REST-Client in Ruby.
The client must be runnable from the command-line like a binary and also it must be "requirable" in a ruby script and provide different functions.
My gemspec does exactly what it should.
But i have no idea how to install a configuration file (YAML) in the user-home folder?
The config file should be in the user directory to provide easy access for the user.
Is this even possible?
Should i check on the first run if there is a config file and create it?
Can i execute an own installation routine while installing a gem?
I did exactly the same thing in Python and it worked fine, so the Ruby client should behave similar.
For such decisions, I wrote gem persey. If you look at the description of the use of this gem, you can see that it provides what you expect:
# Rails.root are not initialized here
app_path = File.expand_path('../../', __FILE__)
# ...
# config with secret keys
# you don't want store this config in repository and copy to secret folder on host machine
my_secret_key_config = '/home/user/secret/keys.yml'
# ...
# Persey.init ENV["environment"] do # set current environment
Persey.init Rails.env do # set current environment
source :yaml, my_secret_key_config, :secret # no comments. It's secret!
env :production do
# ...
end
env :development, :parent => :production do
# ...
end
end

ENV variables not being read sinatra

So i am trying to serve my static assets from Amazon s3 locally and for Heroku, I was loading a yml file but that doesn't work as Heroku doesn't accept symlinks.
So i was given the suggestion to use ENV variables as Heroku also uses these. I have a rake task to precompile the assets to AWS. two problems at the moment
1) My ENV variables are not being read.(Fog directory can't be blank, Aws access key can't be blank, Aws secret access key can't be blank
2) When running a rake task i also get the error 'already initialized constant VALID_CHARACTER'
So the activesupport constant is being loaded twice?
My setup
env.rb
ENV['aws_bucket'] = 'bucketname'
ENV['aws_access_key'] = 'myaccesskey'
ENV['aws_secret_key'] = 'mysecretkey'
Rakefile
require 'bundler/setup'
Bundler.require(:default)
require './env' if File.exists?('env.rb')
AssetSync.configure do |con|
con.fog_provider = 'AWS'
con.fog_region = 'eu-west-1'
con.fog_directory = ENV['aws_bucket']
con.aws_access_key_id = ENV['aws_access_key']
con.aws_secret_access_key = ENV['aws_secret_key']
con.prefix = "assets"
con.public_path = Pathname("./public")
end
namespace :assets do
desc "Precompile assets"
task :precompile do
AssetSync.sync
end
Gemfile
source :rubygems
gem 'sinatra'
gem 'pony'
gem 'sinatra-flash'
gem 'heroku'
gem 'asset_sync', git: 'git://github.com/ejholmes/asset_sync.git', branch: 'sinatra'
UPDATE
AssetSync has activesupport in it's gemspec so that's getting included any way. It seems to be conflicting with the constant defined in the mail gem from the pony gemspec.
So with the Pony gem removed i can precompile assets locally, but when i try and compile for heroku nothing happens, it starts the rake task but then just goes back to the terminal ready for a new command.
The other thing is i need Pony for my mailer, how can i get around this?
To get rid of the clash between Pony and when running Rake locally, put the gems into different groups, e.g.
# Gemfile
group :assets do
gem 'asset_sync', git: 'git://github.com/ejholmes/asset_sync.git', branch: 'sinatra'
end
group :mail do
gem "pony"
end
# more…
in the Rakefile
Bundler.require(:assets,:database,:whatever_else_you_need)
in the rackup/app file
Bundler.require(default,:assets,:database,:mail,:whatever_else_you_need)
As to your other problem, you should set the env vars for production via heroku config (see https://devcenter.heroku.com/articles/config-vars) and load them locally using the Rakefile as I said in the other question you asked about this. The env vars will live for the extent of the Ruby process, so if you load them in via Rake and also start the local server within the same Rake process you'll get Sinatra picking up all the env vars.
Edit: env vars will last as long as the process that added them, so if you put them in a dependent task the following task will have access to them:
namespace :assets do
task :environment do
AssetSync.configure do |con|
con.fog_provider = 'AWS'
con.fog_region = 'eu-west-1'
con.fog_directory = ENV['aws_bucket']
con.aws_access_key_id = ENV['aws_access_key']
con.aws_secret_access_key = ENV['aws_secret_key']
con.prefix = "assets"
con.public_path = Pathname("./public")
end
end
desc "Precompile assets"
task :precompile => :"assets:environment" do
AssetSync.sync
end
You might want to split this into different questions. This makes it easier to help you. As for your first question: I assume you didn't put the env.rb under version control?
Why are your env vars not being picked up by Sinatra? Because you configure Fog in the Rakefile, and Simatra never sees that file. It's used by rake only.
I'd suggest you put the Fog configuration into a third file and require that in both Rakefile and Sinatra app.

How to get the current Rack environment in Rake?

Is there a way to get information about the current Rack environment in Rake? For example, how can I tell whether Rack is running in development or production mode?
I understand that Rake is not Rack-aware. I'm trying to avoid replicating code in nearly-identical Rake tasks between production and dev environments.
Question is old but never fetched the best practice answer or a satisfying answer at all.
The real question is: How to go sure which environment is used in a Rake task in order to load the correct configuration / hitting into correct if-conditions.
Note: As Rake doesn't give much about Rack (Rake is not using HTTP) to
rely on the RACK_ENV is basically wrong but common and handy if a Rake
task loads your main Sinatra application (the RACK_ENV is required to let
Sinatras development? / test? / production? being set correctly).
The answer: Set the environment with each Rake task call.
Command line call:
/usr/bin/rake namespace:task_name RACK_ENV=production
Cronjob call (in crontab):
cd /into/your/app/root && /usr/bin/rake namespace:task_name RACK_ENV=production --silent
Note: To specify the path of the Rake bin is not necessary if you have it in your global system variables. Your path might differs from mine used in the examples, check on Unix systems with: whereis rake
You can check the RACK_ENV in your tasks via:
puts ENV["RACK_ENV"]
As other environment variable, you can retrieve it using:
ENV['RACK_ENV']
Considering it's a Sinatra application, and that you've set the environment into config/environment.rb, you can add the following to your Rakefile:
task :environment do
require File.expand_path('config/environment', File.dirname(__FILE__))
end
task :your_task => :environment do
# task
end
Then, you can retrieve the environment (depending how you set it up in your environment.rb) with ENV['RACK_ENV'] or Sinatra::Application.environment.
Considering there isn't a config/environment.rb config file, only the application file, for instance hello_world.rb, the following works:
hello_world.rb:
require 'sinatra'
set :environment, :production
get '/' do
'Hello World'
end
Rakefile:
task :environment do
require File.expand_path('hello_world', File.dirname(__FILE__)) # your Sinatra app
end
task :your_task => :environment do
puts Sinatra::Application.environment
end
When doing rake your_task you should obtain:
> rake your_task
production
After 2.5 years, I want to share what I've found to be the best solution.
Create a .env file in the root folder of the application, and add a flag specifying the application environment:
ENVIRONMENT=development
Then use Brandon Keepers' dotenv gem to load all environment variables from this file. Now you can use any environment variables specified in .env within Rake tasks.
Rake will rely on the explicit value set in .env, so you must create separate .env files for each environment you plan on using (e.g. dev, test, staging, production, etc).
Sample Rakefile:
require 'dotenv/tasks'
task :default => :help
desc 'Show this help menu'
task :help do
puts "Available rake tasks:"
system('rake --tasks')
end
# Will run in any environment
desc 'Demo task'
task :demo_task => :dotenv do
puts "Running demo task in '#{ENV['ENVIRONMENT']}' mode"
end
# Will only run if ENVIRONMENT value in .env file is set to 'production'
desc 'Production-only task'
task :production_task => :dotenv do
if ENV['ENVIRONMENT'] == 'production'
puts "Running 'Production-only' task"
else
puts "Won't run, because the environment is not set to PRODUCTION!"
end
end
# Will only run if ENVIRONMENT value in .env file is set to 'development'
desc 'Development-only task'
task :dev_task => :dotenv do
if ENV['ENVIRONMENT'] == 'development'
puts "Running 'Development-only' task"
else
puts "Won't run, because the environment is not set to DEVELOPMENT!"
end
end
If you want to use the environment variables within your Rack or Sinatra app (which you probably do), add the following to the application's config or bootstrap block:
require 'dotenv'
Dotenv.load

Using a rackup file to custom configure an application instance

tl;dr How can I get a single Sinatra app to start up very differently on different servers via customizations to config.ru?
Background
I have a single web application written using Sinatra that's run on different servers. Currently the codebase for these servers is forked because there are some non-trivial differences in the way (discrete) parts of them work. For example:
one server authenticates users via an intranet LDAP server, while another server uses a simpler local database table lookup.
one server uses an external cron job to periodically update some statistics, while another (Windows-based) server uses an internal sleepy Thread.
one server stores certain metadata in a local table, while another server pulls the metadata from an external Wiki via screen scraping (!).
…and so on.
I'd like to get these code bases completely shared (single Git repo). I envision that each server would have one slightly-differing configuration file that causes the app to be started up differently.
Abandoned Solutions
I could change the behavior of the app based on environment variables. As there are a not-tiny number of variations in behavior, I'd rather not hide the settings in environment variables.
I could create my own "server-settings.rb" file that is unique to each machine, require it in my app.rb, and then change the configuration there. However, this seems to possibly be re-inventing the wheel. I already have a file named config.ru for each server. Shouldn't I be using this?
The Current Code
My config.ru for the app currently is simply:
require ::File.join( ::File.dirname(__FILE__), 'app' )
run MyApp.new
And the app.rb that it requires is, in essence:
require 'sinatra'
require_relative 'helpers/login' # customized for LDAP lookup on this server
class MyApp < Sinatra::Application
use Rack::Session::Cookie, key:'foo.bar', path:'/', secret:'ohnoes'
set :protection, except: [:path_traversal, :session_hijacking]
configure :production do
# run various code that depends on server settings, e.g.
Snapshotter.start # there is no cron on this machine, so we do it ourselves
end
configure :development do
# run various code that depends on server settings
end
end
The Question
I'd like to make config.ru live up to its name, and have it look something like this:
require ::File.join( ::File.dirname(__FILE__), 'app' )
run MyApp.new( auth: :ldap, snapshot:false, metadata: :remote_wiki, … )
How can I modify my application to change its configuration behavior based on settings supplied via config.ru? Or is this an abuse of config.ru, trying to use it for totally the wrong thing?
As soon as I started reading the question the first answer to pop into my head was "environment variable" but you scotched that straight away :)
I'll go with a mixture of one of your coulds and the desired outcome code, as it's how I structure things…
Because I want to be able to test my applications more easily, I take most of the Ruby out of the config.ru and into a separate config.rb file and leave config.ru to be a bootstrap file. So my standard skel is:
config.ru
# encoding: UTF-8
require 'rubygems'
require 'bundler'
Bundler.setup
root = File.expand_path File.dirname(__FILE__)
require File.join( root , "./app/config.rb" )
# everything was moved into a separate module/file to make it easier to set up tests
map "/" do
run APP_NAME.app
end
app/config.rb
# encoding: utf-8
require_relative File.expand_path(File.join File.dirname(__FILE__), "../lib/ext/warn.rb")
require_relative "./init.rb" # config
require_relative "./main.rb" # routes and helpers
require 'encrypted_cookie'
# standard cookie settings
COOKIE_SETTINGS = {
:key => 'usr',
:path => "/",
:expire_after => 86400, # In seconds, 1 day
:secret => ENV["LLAVE"],
:httponly => true
}
module APP_NAME # overall name of the app
require 'rack/ssl' # force SSL
require 'rack/csrf'
if ENV["RACK_ENV"] == "development"
require 'pry'
require 'pry-nav'
end
# from http://devcenter.heroku.com/articles/ruby#logging
$stdout.sync = true
ONE_MONTH = 60 * 60 * 24 * 30
def self.app
Rack::Builder.app do
cookie_settings = COOKIE_SETTINGS
# more security if in production
cookie_settings.merge!( :secure => true ) if ENV["RACK_ENV"] == "production"
# AES encryption of cookies
use Rack::Session::EncryptedCookie, cookie_settings
if ENV["RACK_ENV"] == "production"
use Rack::SSL, :hsts => {:expires => ONE_MONTH}
end
# to stop XSS
use Rack::Csrf, :raise => true unless ENV["RACK_ENV"] == "test"
run App # the main Sinatra app
end
end # self.app
end # APP_NAME
The initial reason I did this was making it easy to run the app in specs:
shared_context "All routes" do
include Rack::Test::Methods
let(:app){ APP_NAME.app }
end
but it makes sense to me to keep this code with the rest of the application code, so to speak, as I can bundle things together, run other apps etc. I've used this to conditionally load different examples into the specs in a few projects (it helps cut down on duplicated effort and check the examples really work), so I don't see why you couldn't use it to conditionally load configurations.
This way you get to choose to use a conditional in the config.ru as to which config.rb file you would use, or use an env var in the config.rb as to which definiton of self.app to use , or pass in an options hash to self.app…
With your set up I'd rename the APP_NAME module to MyApp, and the Sinatra class to App (because quite often I'll have an website that runs a front end and an API, so the Sinatra classes get named by their function (App, API etc) and wrapped in a module named after the site) and end up with:
config.ru
map "/" do
run MyApp.app( auth: :ldap, snapshot:false, metadata: :remote_wiki )
end
config.rb
def self.app( opts={} )
opts = DEFAULT_OPTIONS.merge opts
# …
run App
end
It'll be interesting to see how other people tackle this.

Creating a class that wraps my config.yml file, and using it in my rake files

This is a non-rails app, just a simple ruby script that uses rake etc. to automate some things.
My folder layout is this:
/scripts/Rakefile
/scripts/config/config.yml
/scripts/tasks/*.rake (various rake files with namespaces to organize them)
/scripts/lib/settings.rb
Now I want to create a Settings class that will load the config yaml file, and then expose properties/methods for the contents of the yaml file.
The yaml file has separate sections for development and production.
development:
scripts_path: '/dev/mygit/app1/scripts/'
production:
scripts_path: '/var/lib/app1/scripts/'
My rakefile so far looks like:
$LOAD_PATH.unshift File.expand_path('..', __FILE__)
#imports
require 'fileutils'
require 'rubygems'
require 'active_record'
require 'yaml'
require 'logger'
require 'ar/models'
require 'lib/app1'
env = ENV['ENV'] || 'development'
config = YAML::load(File.open('config/config.yml'))[env]
Dir.glob('tasks/*.rake').each { |r| import r }
I need help with the Settings.rb file, is this right?
module App1
class Settings
def initialize(config_path, env)
config = YAML.load(File.open(config_path))
end
def scripts_path
end
end
end
How can I pass in the env, and then read the correct value from the config for each method like scripts_path etc?
Now suppose each *.rake file needs to reference my Settings.rb file somehow (to get the config related information). How should I do this? Since my settings needs the path of the config.yml file, do I have to do this in each rake file?
Update
Sorry, this isn't a Rails app, just some ruby scripts.
I would do it quite simple.
You don't need a complex solution.
require 'ostruct'
require 'yaml'
MY_ENV = ENV['ENV'] || 'development'
CONFIG = OpenStruct.new(YAML.load_file("config/config.yml")[MY_ENV])
Stick this at the top of your rakefile
and CONFIG will be available in all rake tasks.
Just call CONFIG.scripts_path
Inside my applications I do something of this sort.
# config/application.yml
development:
some_variable: a string
production:
some_variable: a different string
Then in application.rb I load it up.
# config/application.rb
module MyApp
def self.config
#config ||= OpenStruct.new(YAML.load_file("config/application.yml")[Rails.env.to_s])
end
class Application < Rails::Application
...
In this case, anywhere the environment is loaded I can say
MyApp.config.some_variable
To get access to this inside a rake task, I just need to include environment
task :something => :environment do
MyApp.config.some_variable
# do something with it
end

Resources