What object does ENV return in Ruby? - ruby
There's an ENV variable we all know. It returns an Object. But the return value of ENV is visually similar to a Hash. But it's really not.
For example:
> ENV
=> {"SHELL"=>"/bin/bash", "SESSION_MANAGER"=>"local/archlinux:#/tmp/.ICE-unix/613,unix/archlinux:/tmp/.ICE-unix/613", "COLORTERM"=>"truecolor", "XDG_CONFIG_DIRS"=>"/etc/xdg", "XDG_MENU_PREFIX"=>"xfce-", "SSH_AUTH_SOCK"=>"/tmp/ssh-Al0pdO1R5970/agent.622", "DESKTOP_SESSION"=>"Xfce Session", "SSH_AGENT_PID"=>"623", "GTK_MODULES"=>"canberra-gtk-module:canberra-gtk-module", "XDG_SEAT"=>"seat0", "PWD"=>"/home/sourav", "LOGNAME"=>"sourav", "XDG_SESSION_TYPE"=>"x11", "XAUTHORITY"=>"/home/sourav/.Xauthority", "HOME"=>"/home/sourav", "LANG"=>"en_GB.UTF-8", "XDG_CURRENT_DESKTOP"=>"XFCE", "VTE_VERSION"=>"5603", "INVOCATION_ID"=>"6d4dc7a91cc141e691436cb850e18f8d", "GLADE_CATALOG_PATH"=>":", "XDG_SESSION_CLASS"=>"user", "TERM"=>"xterm-256color", "USER"=>"sourav", "DISPLAY"=>":0.0", "SHLVL"=>"2", "XDG_VTNR"=>"1", "XDG_SESSION_ID"=>"1", "TILIX_ID"=>"f2480263-263e-408f-be36-8105e71256a6", "MOZ_PLUGIN_PATH"=>"/usr/lib/mozilla/plugins", "GLADE_MODULE_PATH"=>":", "XDG_RUNTIME_DIR"=>"/run/user/1000", "GLADE_PIXMAP_PATH"=>":", "JOURNAL_STREAM"=>"9:25041", "XDG_DATA_DIRS"=>"/usr/local/share:/usr/share", "PATH"=>"/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/sourav/.rvm/bin:/home/sourav/.rvm/bin:/home/sourav/.gem/ruby/2.6.0/bin", "DBUS_SESSION_BUS_ADDRESS"=>"unix:path=/run/user/1000/bus", "MAIL"=>"/var/spool/mail/sourav", "_"=>"/home/sourav/.irb", "LINES"=>"24", "COLUMNS"=>"80"}
Which looks similar to:
> ENV.to_h
=> {"SHELL"=>"/bin/bash", "SESSION_MANAGER"=>"local/archlinux:#/tmp/.ICE-unix/613,unix/archlinux:/tmp/.ICE-unix/613", "COLORTERM"=>"truecolor", "XDG_CONFIG_DIRS"=>"/etc/xdg", "XDG_MENU_PREFIX"=>"xfce-", "SSH_AUTH_SOCK"=>"/tmp/ssh-Al0pdO1R5970/agent.622", "DESKTOP_SESSION"=>"Xfce Session", "SSH_AGENT_PID"=>"623", "GTK_MODULES"=>"canberra-gtk-module:canberra-gtk-module", "XDG_SEAT"=>"seat0", "PWD"=>"/home/sourav", "LOGNAME"=>"sourav", "XDG_SESSION_TYPE"=>"x11", "XAUTHORITY"=>"/home/sourav/.Xauthority", "HOME"=>"/home/sourav", "LANG"=>"en_GB.UTF-8", "XDG_CURRENT_DESKTOP"=>"XFCE", "VTE_VERSION"=>"5603", "INVOCATION_ID"=>"6d4dc7a91cc141e691436cb850e18f8d", "GLADE_CATALOG_PATH"=>":", "XDG_SESSION_CLASS"=>"user", "TERM"=>"xterm-256color", "USER"=>"sourav", "DISPLAY"=>":0.0", "SHLVL"=>"2", "XDG_VTNR"=>"1", "XDG_SESSION_ID"=>"1", "TILIX_ID"=>"f2480263-263e-408f-be36-8105e71256a6", "MOZ_PLUGIN_PATH"=>"/usr/lib/mozilla/plugins", "GLADE_MODULE_PATH"=>":", "XDG_RUNTIME_DIR"=>"/run/user/1000", "GLADE_PIXMAP_PATH"=>":", "JOURNAL_STREAM"=>"9:25041", "XDG_DATA_DIRS"=>"/usr/local/share:/usr/share", "PATH"=>"/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/sourav/.rvm/bin:/home/sourav/.rvm/bin:/home/sourav/.gem/ruby/2.6.0/bin", "DBUS_SESSION_BUS_ADDRESS"=>"unix:path=/run/user/1000/bus", "MAIL"=>"/var/spool/mail/sourav", "_"=>"/home/sourav/.irb", "LINES"=>"24", "COLUMNS"=>"80"}
But:
> ENV.to_h.eql?(ENV)
=> false
So what kind of object does ENV return?
It's a custom Object with Hash-like functionality mostly implemented in C.
https://github.com/ruby/ruby/blob/trunk/hash.c#L4944
There is nothing in the Ruby Language Specification that requires ENV to be implemented in any particular way. As long as it responds to the right messages in the right way, it can be anything it wants.
For example, in Rubinius, ENV is implemented in a class called Rubinius::EnvironmentVariables that implements part of the Hash protocol and also mixes in Enumerable: https://github.com/rubinius/rubinius/blob/master/core/env.rb .
Related
Accessing environment variables in a YAML file for Ruby project (using ${ENVVAR} syntax)
I am building an open source project using Ruby for testing HTTP services: https://github.com/Comcast/http-blackbox-test-tool
I want to be able to reference environment variables in my test-plan.yaml file. I could use ERB, however I don't want to support embedding any random Ruby code and ERB syntax is odd for non-rubyists, I just want to access environment variables using the commonly used Unix style ${ENV_VAR} syntax.
e.g.
order-lunch-app-health:
request:
url: ${ORDER_APP_URL}
headers:
content-type: 'application/text'
method: get
expectedResponse:
statusCode: 200
maxRetryCount: 5
All examples I have found for Ruby use ERB. Does anyone have a suggestion on the best way to deal with this? I an open to using another tool to preprocess the YAML and then send that to the Ruby application.
I believe something like this should work under most circumstances:
require 'yaml'
def load_yaml(file)
content = File.read file
content.gsub! /\${([^}]+)}/ do
ENV[$1]
end
YAML.load content
end
p load_yaml 'sample.yml'
As opposed to my original answer, this is both simpler and handles undefined ENV variables well.
Try with this YAML:
# sample.yml
path: ${PATH}
home: ${HOME}
error: ${NO_SUCH_VAR}
Original Answer (left here for reference)
There are several ways to do it. If you want to allow your users to use the ${VAR} syntax, then perhaps one way would be to first convert these variables to Ruby string substitution format %{VAR} and then evaluate all environment variables together.
Here is a rough proof of concept:
require 'yaml'
# Transform environments to a hash of { symbol: value }
env_hash = ENV.to_h.transform_keys(&:to_sym)
# Load the file and convert ${ANYTHING} to %{ANYTHING}
content = File.read 'sample.yml'
content.gsub! /\${([^}]+)}/, "%{\\1}"
# Use Ruby string substitution to replace %{VARS}
content %= env_hash
# Done
yaml = YAML.load content
p yaml
Use it with this sample.yml for instance:
# sample.yml
path: ${PATH}
home: ${HOME}
There are many ways this can be improved upon of course.
Preprocessing is easy, and I recommend you use a YAML loaderd/dumper
based solution, as the replacement might require quotes around the
replacement scalar. (E.g. you substitute the string true, if that
were not quoted, the resulting YAML would be read as a boolean).
Assuming your "source" is in input.yaml and your env. variable
ORDER_APP_URL set to https://some.site/and/url. And the following
script in expand.py:
import sys
import os
from pathlib import Path
import ruamel.yaml
def substenv(d, env):
if isinstance(d, dict):
for k, v in d.items():
if isinstance(v, str) and '${' in v:
d[k] = v.replace('${', '{').format(**env)
else:
substenv(v, env)
elif isinstance(d, list):
for idx, item in enumerate(d):
if isinstance(v, str) and '${' in v:
d[idx] = item.replace('${', '{').format(**env)
else:
substenv(item, env)
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
data = yaml.load(Path(sys.argv[1]))
substenv(data, os.environ)
yaml.dump(data, Path(sys.argv[2]))
You can then do:
python expand.py input.yaml output.yaml
which writes output.yaml:
order-lunch-app-health:
request:
url: https://some.site/and/url
headers:
content-type: 'application/text'
method: get
expectedResponse:
statusCode: 200
maxRetryCount: 5
Please note that the spurious quotes around 'application/text' are preserved, as would be any comments
in the original file.
Quotes around the substituted URL are not necessary, but the would have been added if they were.
The substenv routine recursively traverses the loaded data, and substitutes even if the substitution is in mid-scalar, and if there are more than substitution in one scalar. You can "tighten" the test:
if isinstance(v, str) and '${' in v:
if that would match too many strings loaded from YAML.
ExecJS: keeping the context between two calls
I'm currently trying to use ExecJS to run Handlebars for one of the product I work on (note: I know the handlebars.rb gem which is really cool and I used it for some times but there is issues to get it installed on Windows, so I try another homemade solution).
One of the problem I'm having is that the Javascript context is not kept between each "call" to ExecJS.
Here the code where I instantiate the #js attribute:
class Context
attr_reader :js, :partials, :helpers
def initialize
src = File.open(::Handlebars::Source.bundled_path, 'r').read
#js = ExecJS.compile(src)
end
end
And here's a test showing the issue:
let(:ctx) { Hiptest::Handlebars::Context.new }
it "does not keep context properly (or I'm using the tool wrong" do
ctx.js.eval('my_variable = 42')
expect(ctx.js.eval('my_variable')).to eq(42)
end
And now when I run it:
rspec spec/handlebars_spec.rb:10 1 ↵
I, [2015-02-21T16:57:30.485774 #35939] INFO -- : Not reporting to Code Climate because ENV['CODECLIMATE_REPO_TOKEN'] is not set.
Run options: include {:locations=>{"./spec/handlebars_spec.rb"=>[10]}}
F
Failures:
1) Hiptest::Handlebars Context does not keep context properly (or I'm using the tool wrong
Failure/Error: expect(ctx.js.eval('my_variable')).to eq(42)
ExecJS::ProgramError:
ReferenceError: Can't find variable: my_variable
Note: I got the same issue with "exec" instead of "eval".
That is a silly example. What I really want to do it to run "Handlebars.registerPartial" and later on "Handlebars.compile". But when trying to use the partials in the template it fails because the one registered previously is lost.
Note that I've found a workaround but I find it pretty ugly :/
def register_partial(name, content)
#partials[name] = content
end
def call(*args)
#context.js.call([
"(function (partials, helpers, tmpl, args) {",
" Object.keys(partials).forEach(function (key) {",
" Handlebars.registerPartial(key, partials[key]);",
" })",
" return Handlebars.compile(tmpl).apply(null, args);",
"})"].join("\n"), #partials, #template, args)
end
Any idea on how to fix the issue ?
Only the context you create when you call ExecJS.compile is preserved between evals. Anything you want preserved needs to be part of the initial compile.
Ruby: add new fields to YAML file
I've been searching around for a bit and couldn't find anything that really helped me. Especially because sometimes things don't seem to be consistant.
I have the following YAML that I use to store data/ configuration stuff:
---
global:
name: Core Config
cfg_version: 0.0.1
databases:
main_database:
name: Main
path: ~/Documents/main.reevault
read_only: false
...
I know how to update fields with:
cfg = YAML.load_file("test.yml")
cfg['global']['name'] = 'New Name'
File.open("test.yml", "w"){ |f| YAML.dump(cfg, f) }
And that's essentially everybody on the internet talks about. However here is my problem: I want to dynamically be able to add new fields to that file. e.g. under the "databases" section have a "secondary_db" field with it's own name, path and read_only boolean. I would have expected to do that by just adding stuff to the hash:
cfg['global']['databases']['second_db'] = nil
cfg['global']['databases']['second_db']['name'] = "Secondary Database"
cfg['global']['databases']['second_db']['path'] = "http://someurl.remote/blob/db.reevault"
cfg['global']['databases']['second_db']['read_only'] = "true"
File.open("test.yml", "w"){ |f| YAML.dump(cfg, f) }
But I get this error:
`<main>': undefined method `[]=' for nil:NilClass (NoMethodError)
My question now is: how do I do this? Is there a way with the YAML interface? Or do I have to write stuff into the file manually? I would prefer something via the YAML module as it takes care of formatting/ indentation for me.
Hope someone can help me.
Yo have to initialize cfg['global']['database']['second_db'] to be a hash not nil. Try this cfg['global']['database']['second_db'] = {}
How to mock/stub the config initializer hash in rails 3
Environment : Rails 3.1.1 and Rspec 2.10.1
I am loading all my application configuration through an external YAML file. My initializer (config/initializers/load_config.rb) looks like this
AppConfig = YAML.load_file("#{RAILS_ROOT}/config/config.yml")[RAILS_ENV]
And my YAML file sits under config/config.yml
development:
client_system: SWN
b2c_agent_number: '10500'
advocacy_agent_number: 16202
motorcycle_agent_number: '10400'
tso_agent_number: '39160'
feesecure_eligible_months_for_monthly_payments: 1..12
test:
client_system: SWN
b2c_agent_number: '10500'
advocacy_agent_number: 16202
motorcycle_agent_number: '10400'
tso_agent_number: '39160'
feesecure_eligible_months_for_monthly_payments: 1..11
And I access these values as, For example AppConfig['feesecure_eligible_months_for_monthly_payments']
In one of my tests I need AppConfig['feesecure_eligible_months_for_monthly_payments'] to return a different value but am not sure how to accomplish this. I tried the following approach with no luck
describe 'monthly_option_available?' do
before :each do
#policy = FeeSecure::Policy.new
#settlement_breakdown = SettlementBreakdown.new
#policy.stub(:settlement_breakdown).and_return(#settlement_breakdown)
#date = Date.today
Date.should_receive(:today).and_return(#date)
#config = mock(AppConfig)
AppConfig.stub(:feesecure_eligible_months_for_monthly_payments).and_return('1..7')
end
.....
end
In my respective class I am doing something like this
class Policy
def eligible_month?
eval(AppConfig['feesecure_eligible_months_for_monthly_payments']).include?(Date.today.month)
end
....
end
Can someone please point me in the right direction!!
The method that is being called when you do AppConfig['foo'] is the [] method, which takes one argument (the key to retrieve)
Therefore what you could do in your test is
AppConfig.stub(:[]).and_return('1..11')
You can use with to setup different expectations based on the value of the argument, ie
AppConfig.stub(:[]).with('foo').and_return('1..11')
AppConfig.stub(:[]).with('bar').and_return(3)
You don't need to setup a mock AppConfig object - you can stick your stub straight on the 'real' one.
Forcing a package's function to use user-provided function
I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions Namespaces are sealed once they are loaded. Sealing means that imports and exports cannot be changed and that internal variable bindings cannot be changed. Sealing allows a simpler implementation strategy for the namespace mechanism. Sealing also allows code analysis and compilation tools to accurately identify the definition corresponding to a global variable reference in a function body. The namespace controls the search strategy for variables used by functions in the package. If not found locally, R searches the package namespace first, then the imports, then the base namespace and then the normal search path.