I have a ruby script executing a shell script. How can I pass shell script data back to the ruby script.
desc "Runs all the tests"
lane :test do
sh "../somescript.sh"
print variables_inside_my_script // i want to access my script data here.
end
I'm able to do the reverse scenario using environment variables from ruby to shell script.
desc "Runs all the tests"
lane :test do
puts ENV["test"]
sh "../somescript.sh" // access test using $test
end
Thanks
It's not so clear what variables_inside_my_script is supposed to mean here, but as a rule operating systems do not allow one to "export" variables from a subshell to the parent, so rubyists often invoke the subcommand with backticks (or equivalent) so that the parent can read the output (stdout) of the subshell, e.g.
output = %x[ ls ]
There are alternative techniques that may be useful depending on what you really need -- see e.g.
Exporting an Environment Variable in Ruby
http://tech.natemurray.com/2007/03/ruby-shell-commands.html
http://blog.bigbinary.com/2012/10/18/backtick-system-exec-in-ruby.html
If the shell script is under your control, have the script write the environment definitions in Ruby syntax to STDOUT. Within Ruby, you eval the output:
eval `scriptsettings.sh`
If your script produces other output, write the environment definitions to a temporary file and use the load command to read them.
Related
In Ruby, on osx, how can I change the prompt from within a ruby script?
For example,
I'd like a ruby script to run some code and depending on the result, do PS1='\w$ '. or PS1='\t$ '
i.e.
~ $PS1='\w $'
~ $
~ $PS1='\t $'
20:52:23 $
So as a start I've just tried a script to change the prompt
~/$ cat scripttochangeprompt.rb
`export PS1='\t$ '`
~/$
but it doesn't change the prompt.
~/$ ruby ./scripttochangeprompt.rb
~/$
I can guess that it's creating a new console session to run the command then closing that new console session. But how can I get it to run it in this console session?
Ruby will always run in a subprocess. A subprocess cannot change the environment of a parent process. However, it can output things in a format that is convenient for parent process to evaluate. This is how e.g. ssh-agent sets its environment variables.
#!/usr/bin/env ruby
puts "export PS1='\t$ '"
Then execute it like this:
eval `changeprompt.rb`
There are several ways to work with shell command execution and you use the one that executes the command in the current ruby process and returns its result to your script. With such an approach this proccess cannot mutate a shell it was called from, so it is unfitting. Other ways of interaction with the shell also use child processes or current ruby process one way or another. Unfortunately, in most OS you cannot set an environment variable of parent process(namely, shell) from child process(namely, your ruby script). Here are stackoverflow issue and an article on that matter. There are some hacks in the issue provided that can satisfy you in practice, but they seem obscure and awkward to me.
I have a bash script that when ran, produces output like this:
VAR1=test
VAR2=test
I want to pass these variables as environment variables to an npm script, which in this case is just running mocha in the current directory, like if I was running VAR1=test VAR2=test mocha.
Thanks in advance.
It's not a great design for a bash script to output shell variable assignments like that, but you can work around it with a wrapper script runwithvars:
#!/bin/bash
set -a # Auto-export all new variables
eval "$(mybashscript)" # Perform whichever actions the script outputs
exec "$#" # Execute the specified command
Now you can use runwithvars mocha to run mocha with those variables.
Note that if the script outputs key-value pairs instead of shell variable assignments, e.g. VAR1=some value with spaces instead of VAR1='some value with spaces', then this answer does not apply and could be fragile or dangerous.
Use export:
export VAR1=test
export VAR2=test
I'm working on a jenkins install with two script components. The bash bits run first and then groovy. I'd like to be able to pass a value (property? Other?) from the bash script->groovy script.
Is this possible? Do I need to write the value to a property file and read it back in groovy?
EDIT: my goal from this was to generate a build # in bash and pass this to groovy so I could set the description and build # in the jenkins display. It appears that groovy isn't available on the build server so I'm looking for another direction. Currently experimenting with the 'postbuild' plugin and the 'env-inject' plugin. Open to suggestions.
Here are a few things to consider to make this successful:
Make sure you're trying to accomplish this with one "Execute shell" in Jenkins or from a script.
Export the shell variable so that the variable will be present in the child process that will execute your groovy script.
# foo.sh
export foo=bar
groovy myscript.groovy
# myscript.groovy
def env = System.getenv()
String myvar=env['foo']
println myvar
Running foo.sh should produce the following:
./foo.sh
bar
If for some reason you prefer not to export the variable (there may be good and valid reasons for this), you can still explicitly pass the variable to the groovy script as a "system property":
# foo.sh
foo=bar
groovy -Dfoo="$foo" yourscript.groovy
# yourscript.groovy
String yourvar = System.properties['foo']
println yourvar
which produces the following results:
$ ./foo.sh
bar
$
I just worked on this problem for days and thought I might share what I discovered. I had to access a variable in a groovy file from a .sh file and had difficulty at first grabbing the variable. There is a simple way to do it, though. Here's what I did:
In your bash file, save the value in a variable. Then in the groovy script, do this:
variableToGet = sh(returnStdout: true, script: """
. ${DIRECTORY}/bash_file.sh
echo \$VARIABLE
""").trim()
Hope this helps. This problem was a good challenge! It's important to note, however, that standard out will return a String, regardless of what type of variable you are grabbing. If you need to use an integer value, you can then use the integer value with Integer.parseInt(variableToGet)
The best way is setting an environment variable to share the information from bash into groovy. You could pipe things as well using standard in/out as well.
So if you are setting the env in a bash script it wont be available outside of that script. Instead of doing a bash script put the script inline in your command in jenkins. Run your bash code then call the groovy script.
Something like below
#do somebash scripting
VARIABLE="something"
#call your groovy code
groovy util.groovy
your groovy code (util.groovy):
String variable = env['VARIABLE']
I have a series of bash files that I would like to source into the current Ruby environment. Here's an example:
$ echo "export FOO=bar" > foo.sh
$ irb
> `source $(pwd)/foo.sh`
> puts ENV['FOO']
=> nil
Is there a way to source foo.sh into the parent environment without having to manually parse it?
All shell invocations in Ruby run in a subprocess, whether you use system() or backticks or Process or any other mechanism to execute them. It is not possible for a subprocess like this to effect the current Ruby process.
If you want to source a shell script prior to executing some Ruby code, you can create a wrapper:
#!/bin/sh
source foo.sh
ruby some_ruby_file.rb
If you really want to, you can try to parse out variable exports from the shell script and then set Ruby's ENV hash directly, but that's almost certainly a bad idea. It'd be hard to write, error-prone, and unmaintainable.
Either use a wrapper as above, or come up with a different way to save your environment config, such as a YAML file or some other conventional configuration solution.
TL;DR
You can't change the environment of a parent process from a child process or subshell, but you can certainly source or parse the file into the current process.
"Source" a Ruby Script
You can source a Ruby script with the Kernel#load or Kernel#require methods. That will import the contents of the file into your current process.
Parsing a Shell Script Into Ruby
If your source file is a shell script, you can't simply load it as Ruby; you will need to perform some kind of parsing of the file. This may be a security risk, unless you trust the contents, format, and source of the file you're reading in.
Assuming that you trust your input sources, and given a sample file like:
#!/usr/bin/bash
export FOO='bar'
echo FOO
echo bar
echo "FOO=$FOO"
you could do something like this:
# Find variables in the general form of "export x=y"
env_vars = File.read('/tmp/file.txt').scan /export\s+(\S+)=(\S+)/
#=> [["FOO", "'bar'"]]
# Parse each variable into the Ruby ENV key/value pair, removing
# outer quotes on the value if present.
env_vars.each { |v| ENV[v.first] = v.last.gsub /\A['"]|['"]\Z/, '' }
# Verify you have the value you expect.
ENV['FOO']
#=> "bar"
This will add each variable found via the String#scan method into ENV, where it can then be accessed by its key.
Caveats for Parsing
This works fine in casual testing, but you may need to modify your regular expression if you are not exporting the variable on the same line where you define it.
In addition, it is up to you to sanitize or validate input.
I strongly recommend only setting environment variables you're expecting to see (e.g. use Array#select to whitelist acceptable ENV keys), and ensuring the values for each variable you set are sane and safe for your particular use case.
A Better Option for Options
In general, if you're trying to set configuration options from inside a script, you'd be better off loading a YAML file or using OptionParser. YAML files in particular are easy to read, easy to parse, human editable, and (relatively) easy to sanitize. Your mileage may vary.
Got a few almost answers, none that worked entirely for my case here's what I ended up doing:
file = Pathname.new "tmp-variables.txt"
`env -i /bin/sh -c 'source <myfile-here.sh> && env > #{file}'`
file.each_line do |line|
line.match(/(?<key>[^=]+)=(?<value>.+)/) {|match| ENV[match[:key]] = match[:value] }
end
Here's an example script that shows it in action:
require 'pathname'
file = Pathname.new "tmp-variables.txt"
`mkdir profile.d`
`echo "export FOO=${FOO:-'bar'}" > profile.d/node`
`touch #{file}`
`env -i /bin/sh -c 'source profile.d/node && env > #{file}'`
file.each_line do |line|
line.match(/(?<key>[^=]+)=(?<value>.+)/) {|match| ENV[match[:key]] = match[:value] }
end
puts ENV['FOO']
Some tricks here env -i executes the command (-c) without any environment variables. I tried using this trick How do I source environment variables for a command shell in a Ruby script? but set gives you all the environment variables, i just want the ones from that file.
Jim's idea was good, but couldn't use due to my constraints. CodeGnome was on the right path, but we cannot read the file wholesale without evaluating otherwise we mis things like file = Pathname.new "tmp-variables.txt". Thanks all, this was quite a team effort. I've given you all up-votes.
You can import .env files using the dotenv gem.
https://github.com/bkeepers/dotenv
If I'm writing a shell script and I want to "source" some external (c-)shell scripts to set up my environment, I can just make calls like this:
source /file/I/want/to/source.csh
I want to replace a shell script that does this with a ruby script. Can I do a similar thing in the ruby script?
Update:
Just tried it with test_script.csh:
#!/bin/csh
setenv HAPPYTIMES True
...and test_script.rb:
#!/usr/bin/env ruby
system "~/test_script.csh"
system "echo $HAPPYTIMES"
Sadly, no HAPPYTIMES as of yet.
Given the following Ruby
# Read in the bash environment, after an optional command.
# Returns Array of key/value pairs.
def bash_env(cmd=nil)
env = `#{cmd + ';' if cmd} printenv`
env.split(/\n/).map {|l| l.split(/=/)}
end
# Source a given file, and compare environment before and after.
# Returns Hash of any keys that have changed.
def bash_source(file)
Hash[ bash_env(". #{File.realpath file}") - bash_env() ]
end
# Find variables changed as a result of sourcing the given file,
# and update in ENV.
def source_env_from(file)
bash_source(file).each {|k,v| ENV[k] = v }
end
and the following test.sh:
#!/usr/bin/env bash
export FOO='bar'
you should get:
irb(main):019:0> source_env_from('test.sh')
=> {"FOO"=>"bar"}
irb(main):020:0> ENV['FOO']
=> "bar"
Enjoy!
The reason this isn't working for you is b/c ruby runs its system commands in separate shells. So when one system command finishes, the shell that had sourced your file closes, and any environment variables set in that shell are forgotten.
If you don't know the name of the sourced file until runtime, then Roboprog's answer is a good approach. However, if you know the name of the sourced file ahead of time, you can do a quick hack with the hashbang line.
% echo sourcer.rb
#!/usr/bin/env ruby
exec "csh -c 'source #{ARGV[0]} && /usr/bin/env ruby #{ARGV[1]}'"
% echo my-script.rb
#!/usr/bin/env ruby sourcer.rb /path/to/file/I/want/to/source.csh
puts "HAPPYTIMES = #{ENV['HAPPYTIMES']}"
% ./my-script.rb
HAPPYTIMES = True
All of these will only help you use the set enviroment variables in your ruby script, not set them in your shell (since they're forgotten as soon as the ruby process completes). For that, you're stuck with the source command.
Improving a little on #takeccho's answer... Checks, and a few whistles. First, the sourced environment is cleaned via env -i, which is a safety measure but might be not desired in some cases. Second, via set -a, all variables set in the file are "exported" from the shell and thus imported into ruby. This is useful for simulating/overriding behavior found in environment files used with init scripts and systemd env files.
def ShSource(filename)
# Inspired by user takeccho at http://stackoverflow.com/a/26381374/3849157
# Sources sh-script or env file and imports resulting environment
fail(ArgumentError,"File #{filename} invalid or doesn't exist.") \
unless File.exist?(filename)
_newhashstr=`env -i sh -c 'set -a;source #{filename} && ruby -e "p ENV"'`
fail(ArgumentError,"Failure to parse or process #{filename} environment")\
unless _newhashstr.match(/^\{("[^"]+"=>".*?",\s*)*("[^"]+"=>".*?")\}$/)
_newhash=eval(_newhashstr)
%w[ SHLVL PWD _ ].each{|k|_newhash.delete(k) }
_newhash.each{|k,v| ENV[k]=v } # ENV does not have #merge!
end
Theory of operation: When ruby outputs the ENV object using p, it does so in a way that ruby can read it back in as an object. So we use the shell to source the target file, and ruby (in a sub-shell) to output the environment in that serializable form. We then capture ruby's output and eval it within our ruby-process. Clearly this is not without some risk, so to mitigate the risk, we (1) validate the filename that is passed in, and (2) validate using a regexp that the thing we get back from the ruby-subshell is, in fact, a serializable hash-string. Once we're sure of that, we do the eval which creates a new hash. We then "manually" merge the hash with ENV, which is an Object and not a regular Hash. If it were a Hash, we could have used the #merge! method.
EDIT: sh -a exported things like PATH. We must also remove SHLVL and PWD before the hash-merge.
I had have same probrem. and I resolve like below.
#!/usr/local/bin/ruby
def source(filename)
ENV.replace(eval(`tcsh -c 'source #{filename} && ruby -e "p ENV"'`))
end
p "***old env*****************************"
p ENV
source "/file/I/want/to/source.csh"
p "+++new env+++++++++++++++++++++++++++++"
p ENV
'eval' is a very powerful method.
It jump over the process easily.
You are going to have to write a function to run something like the following, and capture the output ("backtick" operation):
/bin/csh -e '. my_script ; env'
Loop on each line, match against something like
/^(\w+)=(.*)$/
Then use the first match capture as the var name, and the second capture as the var value.
(yes, I'm hedging on the fact that I know Perl way better than Ruby, but the approach would be the same)
system 'source /file/I/want/to/source.sh'
Not sure that this will do what you want though. It will execute the source command in a subshell. Try it and see it it does what you're after.