Source shell script into environment within a ruby script - ruby

If I'm writing a shell script and I want to "source" some external (c-)shell scripts to set up my environment, I can just make calls like this:
source /file/I/want/to/source.csh
I want to replace a shell script that does this with a ruby script. Can I do a similar thing in the ruby script?
Update:
Just tried it with test_script.csh:
#!/bin/csh
setenv HAPPYTIMES True
...and test_script.rb:
#!/usr/bin/env ruby
system "~/test_script.csh"
system "echo $HAPPYTIMES"
Sadly, no HAPPYTIMES as of yet.

Given the following Ruby
# Read in the bash environment, after an optional command.
# Returns Array of key/value pairs.
def bash_env(cmd=nil)
env = `#{cmd + ';' if cmd} printenv`
env.split(/\n/).map {|l| l.split(/=/)}
end
# Source a given file, and compare environment before and after.
# Returns Hash of any keys that have changed.
def bash_source(file)
Hash[ bash_env(". #{File.realpath file}") - bash_env() ]
end
# Find variables changed as a result of sourcing the given file,
# and update in ENV.
def source_env_from(file)
bash_source(file).each {|k,v| ENV[k] = v }
end
and the following test.sh:
#!/usr/bin/env bash
export FOO='bar'
you should get:
irb(main):019:0> source_env_from('test.sh')
=> {"FOO"=>"bar"}
irb(main):020:0> ENV['FOO']
=> "bar"
Enjoy!

The reason this isn't working for you is b/c ruby runs its system commands in separate shells. So when one system command finishes, the shell that had sourced your file closes, and any environment variables set in that shell are forgotten.
If you don't know the name of the sourced file until runtime, then Roboprog's answer is a good approach. However, if you know the name of the sourced file ahead of time, you can do a quick hack with the hashbang line.
% echo sourcer.rb
#!/usr/bin/env ruby
exec "csh -c 'source #{ARGV[0]} && /usr/bin/env ruby #{ARGV[1]}'"
% echo my-script.rb
#!/usr/bin/env ruby sourcer.rb /path/to/file/I/want/to/source.csh
puts "HAPPYTIMES = #{ENV['HAPPYTIMES']}"
% ./my-script.rb
HAPPYTIMES = True
All of these will only help you use the set enviroment variables in your ruby script, not set them in your shell (since they're forgotten as soon as the ruby process completes). For that, you're stuck with the source command.

Improving a little on #takeccho's answer... Checks, and a few whistles. First, the sourced environment is cleaned via env -i, which is a safety measure but might be not desired in some cases. Second, via set -a, all variables set in the file are "exported" from the shell and thus imported into ruby. This is useful for simulating/overriding behavior found in environment files used with init scripts and systemd env files.
def ShSource(filename)
# Inspired by user takeccho at http://stackoverflow.com/a/26381374/3849157
# Sources sh-script or env file and imports resulting environment
fail(ArgumentError,"File #{filename} invalid or doesn't exist.") \
unless File.exist?(filename)
_newhashstr=`env -i sh -c 'set -a;source #{filename} && ruby -e "p ENV"'`
fail(ArgumentError,"Failure to parse or process #{filename} environment")\
unless _newhashstr.match(/^\{("[^"]+"=>".*?",\s*)*("[^"]+"=>".*?")\}$/)
_newhash=eval(_newhashstr)
%w[ SHLVL PWD _ ].each{|k|_newhash.delete(k) }
_newhash.each{|k,v| ENV[k]=v } # ENV does not have #merge!
end
Theory of operation: When ruby outputs the ENV object using p, it does so in a way that ruby can read it back in as an object. So we use the shell to source the target file, and ruby (in a sub-shell) to output the environment in that serializable form. We then capture ruby's output and eval it within our ruby-process. Clearly this is not without some risk, so to mitigate the risk, we (1) validate the filename that is passed in, and (2) validate using a regexp that the thing we get back from the ruby-subshell is, in fact, a serializable hash-string. Once we're sure of that, we do the eval which creates a new hash. We then "manually" merge the hash with ENV, which is an Object and not a regular Hash. If it were a Hash, we could have used the #merge! method.
EDIT: sh -a exported things like PATH. We must also remove SHLVL and PWD before the hash-merge.

I had have same probrem. and I resolve like below.
#!/usr/local/bin/ruby
def source(filename)
ENV.replace(eval(`tcsh -c 'source #{filename} && ruby -e "p ENV"'`))
end
p "***old env*****************************"
p ENV
source "/file/I/want/to/source.csh"
p "+++new env+++++++++++++++++++++++++++++"
p ENV
'eval' is a very powerful method.
It jump over the process easily.

You are going to have to write a function to run something like the following, and capture the output ("backtick" operation):
/bin/csh -e '. my_script ; env'
Loop on each line, match against something like
/^(\w+)=(.*)$/
Then use the first match capture as the var name, and the second capture as the var value.
(yes, I'm hedging on the fact that I know Perl way better than Ruby, but the approach would be the same)

system 'source /file/I/want/to/source.sh'
Not sure that this will do what you want though. It will execute the source command in a subshell. Try it and see it it does what you're after.

Related

How to pass shell script variables back to ruby?

I have a ruby script executing a shell script. How can I pass shell script data back to the ruby script.
desc "Runs all the tests"
lane :test do
sh "../somescript.sh"
print variables_inside_my_script // i want to access my script data here.
end
I'm able to do the reverse scenario using environment variables from ruby to shell script.
desc "Runs all the tests"
lane :test do
puts ENV["test"]
sh "../somescript.sh" // access test using $test
end
Thanks
It's not so clear what variables_inside_my_script is supposed to mean here, but as a rule operating systems do not allow one to "export" variables from a subshell to the parent, so rubyists often invoke the subcommand with backticks (or equivalent) so that the parent can read the output (stdout) of the subshell, e.g.
output = %x[ ls ]
There are alternative techniques that may be useful depending on what you really need -- see e.g.
Exporting an Environment Variable in Ruby
http://tech.natemurray.com/2007/03/ruby-shell-commands.html
http://blog.bigbinary.com/2012/10/18/backtick-system-exec-in-ruby.html
If the shell script is under your control, have the script write the environment definitions in Ruby syntax to STDOUT. Within Ruby, you eval the output:
eval `scriptsettings.sh`
If your script produces other output, write the environment definitions to a temporary file and use the load command to read them.

source a file containing environment variables including "dash" character in bash?

I'm using bash and have a file called x.config that contains the following:
MY_VAR=Something1
ANOTHER=Something2
To load these as environment variables I just use source:
$ source x.config
But this doesn't work if MY_VAR is called MY-VAR:
MY-VAR=Something1
ANOTHER=Something2
If I do the same thing I get:
x.config:1: command not found: MY-VAR=Something1
I've tried escaping - and a lot of other things but I'm stuck. Does anyone know a workaround for this?
A pure bash workaround that might work for you is to re-run the script using env to set the environment. Add this to the beginning of your script.
if [[ ! -v myscript_env_set ]]; then
export myscript_env_set=1
readarray -t newenv < x.config
exec env "${newenv[#]}" "$0" "$#"
fi
# rest of the script here
This assumes that x.config doesn't contain anything except variable assignments. If myscript_env_set is not in the current environment, put it there so that the next invocation skips this block. Then read the assignments into an array to pass to env. Using exec replaces the current process with another invocation of the script, but with the desired variables in the environment.
A dash (-) in an environment variable is not portable, and as you noticed, will cause a lot of problems. You can't set these from bash. Fix the application you want to invoke.
That being said, if you can't change the target app, you can do this from python:
#!/usr/bin/python
import os
with open('x.config') as f:
for line in f:
name, value = line.strip().split('=')
os.environ[name] = value
os.system('/path/to/your/app')
This is a very simplistic config reader, and for a more complex syntax you might want to use ConfigParser.

How can I import environment variables from a shell script into the current Ruby environment?

I have a series of bash files that I would like to source into the current Ruby environment. Here's an example:
$ echo "export FOO=bar" > foo.sh
$ irb
> `source $(pwd)/foo.sh`
> puts ENV['FOO']
=> nil
Is there a way to source foo.sh into the parent environment without having to manually parse it?
All shell invocations in Ruby run in a subprocess, whether you use system() or backticks or Process or any other mechanism to execute them. It is not possible for a subprocess like this to effect the current Ruby process.
If you want to source a shell script prior to executing some Ruby code, you can create a wrapper:
#!/bin/sh
source foo.sh
ruby some_ruby_file.rb
If you really want to, you can try to parse out variable exports from the shell script and then set Ruby's ENV hash directly, but that's almost certainly a bad idea. It'd be hard to write, error-prone, and unmaintainable.
Either use a wrapper as above, or come up with a different way to save your environment config, such as a YAML file or some other conventional configuration solution.
TL;DR
You can't change the environment of a parent process from a child process or subshell, but you can certainly source or parse the file into the current process.
"Source" a Ruby Script
You can source a Ruby script with the Kernel#load or Kernel#require methods. That will import the contents of the file into your current process.
Parsing a Shell Script Into Ruby
If your source file is a shell script, you can't simply load it as Ruby; you will need to perform some kind of parsing of the file. This may be a security risk, unless you trust the contents, format, and source of the file you're reading in.
Assuming that you trust your input sources, and given a sample file like:
#!/usr/bin/bash
export FOO='bar'
echo FOO
echo bar
echo "FOO=$FOO"
you could do something like this:
# Find variables in the general form of "export x=y"
env_vars = File.read('/tmp/file.txt').scan /export\s+(\S+)=(\S+)/
#=> [["FOO", "'bar'"]]
# Parse each variable into the Ruby ENV key/value pair, removing
# outer quotes on the value if present.
env_vars.each { |v| ENV[v.first] = v.last.gsub /\A['"]|['"]\Z/, '' }
# Verify you have the value you expect.
ENV['FOO']
#=> "bar"
This will add each variable found via the String#scan method into ENV, where it can then be accessed by its key.
Caveats for Parsing
This works fine in casual testing, but you may need to modify your regular expression if you are not exporting the variable on the same line where you define it.
In addition, it is up to you to sanitize or validate input.
I strongly recommend only setting environment variables you're expecting to see (e.g. use Array#select to whitelist acceptable ENV keys), and ensuring the values for each variable you set are sane and safe for your particular use case.
A Better Option for Options
In general, if you're trying to set configuration options from inside a script, you'd be better off loading a YAML file or using OptionParser. YAML files in particular are easy to read, easy to parse, human editable, and (relatively) easy to sanitize. Your mileage may vary.
Got a few almost answers, none that worked entirely for my case here's what I ended up doing:
file = Pathname.new "tmp-variables.txt"
`env -i /bin/sh -c 'source <myfile-here.sh> && env > #{file}'`
file.each_line do |line|
line.match(/(?<key>[^=]+)=(?<value>.+)/) {|match| ENV[match[:key]] = match[:value] }
end
Here's an example script that shows it in action:
require 'pathname'
file = Pathname.new "tmp-variables.txt"
`mkdir profile.d`
`echo "export FOO=${FOO:-'bar'}" > profile.d/node`
`touch #{file}`
`env -i /bin/sh -c 'source profile.d/node && env > #{file}'`
file.each_line do |line|
line.match(/(?<key>[^=]+)=(?<value>.+)/) {|match| ENV[match[:key]] = match[:value] }
end
puts ENV['FOO']
Some tricks here env -i executes the command (-c) without any environment variables. I tried using this trick How do I source environment variables for a command shell in a Ruby script? but set gives you all the environment variables, i just want the ones from that file.
Jim's idea was good, but couldn't use due to my constraints. CodeGnome was on the right path, but we cannot read the file wholesale without evaluating otherwise we mis things like file = Pathname.new "tmp-variables.txt". Thanks all, this was quite a team effort. I've given you all up-votes.
You can import .env files using the dotenv gem.
https://github.com/bkeepers/dotenv

How to require files by default without mentioning explicitly

I defined a custom instance method in the String class that I want to use in my other ruby files. I can do it by require-ing the file (that I defined my custom method in), but I want to use it naturally (without having to require).
For example: I defined this in my 'custom_string.rb' file:
class String
def set_style(style)
puts "\n#{self}"
self.size.times do
print style
end
end
end
Then, to use my set_style method in my 'test.rb' file, I have to do this:
require 'custom_string'
puts "hello".set_style("*")
I'm not using a Rails project. Is there a way to include my file into ruby by default (from the ruby command line) so it is available to all files in Ruby?
If you do not require 'custom_string' but found away for this to be automatically included, what happens when you run your program at another location, different server, share the code on github etc. The code would no longer execute as expected. The results that you post when asking for help would no longer match up with other peoples. This sounds like a bad idea to change Ruby behaviour in an untraceable manor.
If you just want irb to have this behaviour then you can add the require to your ~/.irbrc.
#Hauleth solution of adding it the command line allows the change of behaviour to be traced. An alias can be added to .bashrc or other shell rc to give this behaviour by default.
Ruby 1.9 help:
$ ruby --help
Usage: ruby [switches] [--] [programfile] [arguments]
-0[octal] specify record separator (\0, if no argument)
-a autosplit mode with -n or -p (splits $_ into $F)
-c check syntax only
-Cdirectory cd to directory, before executing your script
-d set debugging flags (set $DEBUG to true)
-e 'command' one line of script. Several -e's allowed. Omit [programfile]
-Eex[:in] specify the default external and internal character encodings
-Fpattern split() pattern for autosplit (-a)
-i[extension] edit ARGV files in place (make backup if extension supplied)
-Idirectory specify $LOAD_PATH directory (may be used more than once)
-l enable line ending processing
-n assume 'while gets(); ... end' loop around your script
-p assume loop like -n but print line also like sed
-rlibrary require the library, before executing your script
-s enable some switch parsing for switches after script name
-S look for the script using PATH environment variable
-T[level=1] turn on tainting checks
-v print version number, then turn on verbose mode
-w turn warnings on for your script
-W[level=2] set warning level; 0=silence, 1=medium, 2=verbose
-x[directory] strip off text before #!ruby line and perhaps cd to directory
--copyright print the copyright
--version print the version
Take a look on:
-rlibrary require the library, before executing your script

How do I source environment variables for a command shell in a Ruby script?

I'm trying to run some third party bash scripts from within my ruby program.
Before I can run them they require me to source a file. On the command line it all works fine but within Ruby it doesn't work. I've found out that system commands will open a new child shell process and any sourcing will be done in that and can't be seen from the parent shell process running the Ruby script. When the system call ends, the child shell is also killed.
How do i get round this problem?
Do this:
$ source whatever.sh
$ set > variables.txt
And then in Ruby:
File.readlines("variables.txt").each do |line|
values = line.split("=")
ENV[values[0]] = values[1]
end
After you've ran this, your environment should be good to go.
I'm not sure if I understand your question correctly. Do you try to source a shell script before running another one? In this case the answer is simple:
#!/bin/env ruby
system "source <path_to_source_file> && <command>"
If the source file contains variables which your command should use you have to export them. It is also possible to set environment variables within your Ruby script by using ENV['<name_of_var>'] = <value>.
Update: Jan 26, 2010 - 15:10
You can use IO.popen to open a new shell:
IO.popen("/bin/bash", "w") do |shell|
shell.puts "source <path_to_source_file>"
shell.puts "<command>"
end
This is horrible, but..
env = %x{. /some/shell/script/which/setups/your/env && env}
env.split("\n").each do |line|
key, value = line.split("=", 2)
ENV[key] ||= value unless value.nil? or value.empty?
end
I did this because I didn't want to have to write a file or mix up my ruby code with this stuff:
#!/usr/bin/env bash
eval $(echo "$(cat .env) $1" | tr '\n' ' ')
I put this in ~/bin/run_with_env.sh and then for example I can run:
% run_with_env.sh "rails console"
Modifying ENV works only for a single thread and I do the below instead.
But a question I have how do I use a merged COPY of the environment with IO.popen ? somehow it doesn't seem to work.
with "system" I can do:
# ... create new "path" here
subEnv = Hash.new;
subEnv.merge!(ENV); # copy old environment
subEnv['PATH'] = path; # set new values in copy
system(subEnv, commandLine);

Resources