I have this shell script in a file named a.sh:
export A='a'
export B=b
export C=1
export D=$USER # say $USER is root
There are some other similar files b.sh, c.sh, etc.
I need to read the shell file, say a.sh, from ruby script and convert it to a Ruby hash:
{ 'A' => 'a', 'B' => 'b', 'C' => 1, 'D' => 'root' }
How to achieve that?
If you have run the bash script prior to the ruby one, you can get doing something like this:
Hash[([ 'A', 'B', 'C', 'D' ] & ENV.keys).map {|x| [x, ENV[x]] }]
where array [ 'A', 'B', 'C', 'D' ] contains valid variable names to create the Hash.
If you need to parse the bash script in ruby, do as follows:
vars = {}
IO.read('shell.sh').each do| line |
if line =~ /^export\s([A-Za-z_][A-Za-z_0-9]*)=\s*(?:['"]([^'"]*)['"]|(.*))\s*$/
(name, value) = [ $1, $2 || $3 ]
value.gsub!( /\$(?:([A-Za-z_][A-Za-z_0-9]*)|{([^{}]+)})/ ) do| match |
ENV[ match[1..-1] ]
end
vars[ name ] = value.gsub(/#.*/, '').strip
end
end
vars
# => {"A"=>"a", "B"=>"b", "C"=>"1", "D"=>"malo"}
Related
I need to reuse a textfile that is filled with one-liners such:
export NODE_CODE="mio12"
How can I do that in my Ruby program the var is created and assign as it is in the text file?
If the file were a Ruby file, you could require it and be able to access the variables after that:
# variables.rb
VAR1 = "variable 1"
VAR2 = 2
# ruby.rb
require "variables"
puts VAR1
If you're not so lucky, you could read the file and then loop through the lines, looking for lines that match your criteria (Rubular is great here) and making use of Ruby's instance_variable_set method. The gsub is to deal with extra quotes when the matcher grabs a variable set as a string.
# variables.txt
export VAR1="variable 1"
export VAR2=2
# ruby.rb
variable_line = Regexp.new('export\s(\w*)=(.*)')
File.readlines("variables.txt").each do |line|
if match = variable_line.match(line)
instance_variable_set("##{match[1].downcase}", match[2].gsub("\"", ""))
end
end
puts #var1
puts #var2
Creating a hash from this file can be a fairly simple thing.
For var.txt:
export BLAH=42
export WOOBLE=67
File.readlines("var.txt").each_with_object({}) { |line, h|
h[$1] = $2 if line =~ /^ export \s+ (.+?) \s* \= \s* (.+) $/x
}
# => {"BLAH"=>"42", "WOOBLE"=>"67"}
I have a file (filename.txt) with the following structure:
>line1
ABC
DEF
GHI
>line2
JKL
MNO
PQR
>line3
STU
VWX
YZ
I would like to shuffle the characters in the strings that do not start wit >. The output would (for example) look like the following:
>line1
DGC
FEI
HBA
>line2
JRP
OKN
QML
>line3
SZV
YXT
UW
This is what I tried to shuffle the characters for each >line[number]: ruby -lpe '$_ = $_.chars.shuffle * "" if !/^>/' filename.txt. The command works (see my post BASH - Shuffle characters in strings from file) but it shuffles line by line. I was wondering how I could modify the command to shuffle characters between all strings of each >line[number]). Using ruby is not a requirement.
First, we need to solve the problem: how to shuffle all characters in multiple lines:
echo -e 'ABC\nDEF\nGHI' |grep -o . |shuf |tr -d '\n'
GDAFHEIBC
and, we also need an array to record the length of each line in origin strings.
s=GDAFHEIBC
lens=(3 3 3)
start=0
for len in "${lens[#]}"; do
echo ${s:${start}:${len}}
((start+=len))
done
GDA
FHE
IBC
So, the origin multiple lines:
ABC
DEF
GHI
have been shuffled to:
GDA
FHE
IBC
Now, we can do our jobs:
lens=()
string=""
function shuffle_lines {
local start=0
local shuffled_string=$(grep -o . <<< ${string} |shuf |tr -d '\n')
for len in "${lens[#]}"; do
echo ${shuffled_string:${start}:${len}}
((start+=len))
done
lens=()
string=""
}
while read -r line; do
if [[ "${line}" =~ ^\> ]]; then
shuffle_lines
echo "${line}"
else
string+="${line}"
lens+=(${#line})
fi
done <filename.txt
shuffle_lines
Examples:
$ cat filename.txt
>line1
ABC
DEF
GHI
>line2
JKL
MNO
PQR
>line3
STU
VWX
YZ
>line4
0123
456
78
9
$ ./solution.sh
>line1
HFG
BED
AIC
>line2
JOP
KMQ
RLN
>line3
UVW
TYZ
XS
>line4
1963
245
08
7
#!/bin/bash
# echo > output.txt # uncomment to write in a file output.txt
mix(){
{
echo "$title"
line="$( fold -w1 <<< "$line" | shuf )"
echo "${line//$'\n'}" | fold -w3
} # >> output.txt # uncomment to write in a file output.txt
unset line
}
while read -r; do
if [[ $REPLY =~ ^\> ]]; then
mix
title="$REPLY"
else
line+="$REPLY"
fi
done < filename.txt
mix # final mix after loop's exit, otherwise line3 will be not mixed
exit
edited with comment of gniourf-gniourf
First create a test file.
str =<<FINI
>line1
ABC
DEF
GHI
>line2
JKL
MNO
PQR
>line3
STU
VWX
YZ
FINI
File.write('test', str)
#=> 56
Now read the file and perform the desired operations.
result = File.read('test').split(/(>line\d+)/).map do |s|
if s.match?(/\A(?:|>line\d+)\z/)
s
else
a = s.scan(/\p{Lu}/).shuffle
s.gsub(/\p{Lu}/) { a.shift }
end
end.join
# ">line1\nECF\nHIA\nGBD\n>line2\nJNP\nKLR\nOQM\n>line3\nTXY\nUZV\nSW\n"
puts result
>line1
ECF
HIA
GBD
>line2
JNP
KLR
OQM
>line3
TXY
UZV
SW
To do this from the command convert the code to a string with statements separated by a semicolon.
ruby -e "puts (File.read('test').split(/(>line\d+)/).map do |s|; if s.match?(/\A(?:|>line\d+)\z/); s; else; a = s.scan(/\p{Lu}/).shuffle; s.gsub(/\p{Lu}/) { a.shift }; end; end).join"
The steps are as follows.
a = File.read('test')
#=> ">line1\nABC\nDEF\nGHI\n>line2\nJKL\nMNO\nPQR\n>line3\nSTU\nVWX\nYZ\n"
b = a.split(/(>line\d+)/)
#=> ["", ">line1", "\nABC\nDEF\nGHI\n", ">line2", "\nJKL\nMNO\nPQR\n",
# ">line3", "\nSTU\nVWX\nYZ\n"]
Notice that the regular expression that is split's argument places >line\d+ within a capture group. Without that, ">line1", ">line2" and ">line3" would not be included in b.
c = b.map do |s|
if s.match?(/\A(?:|>line\d+)\z/)
s
else
a = s.scan(/\p{Lu}/).shuffle
s.gsub(/\p{Lu}/) { a.shift }
end
end
#=> ["", ">line1", "\nEAC\nIHB\nDGF\n", ">line2", "\nKQJ\nROL\nMPN\n",
# ">line3", "\nSUY\nXTV\nZW\n"]
c.join
#=> ">line1\nEAC\nIHB\nDGF\n>line2\nKQJ\nROL\nMPN\n>line3\nSUY\nXTV\nZW\n"
Now consider more closely the calculation of c. The first element of b is passed to the block and the block variable s is set to its value:
s = ""
We then compute
s.match?(/\A(?:|>line\d+)\z/)
#=> true
so "" is returned from the block. The regular expression can be expressed as follows.
/
\A # match the beginning of the string
(?: # begin a non-capture group
# match an empty space
| # or
>line\d+ # match '>line' followed by one or more digits
) # end non-capture group
\z # match the end of the string
/x # free-spacing regex definition mode.
Within the non-capture group an empty space was matched.
The next element of b is then passed to the block.
s = ">line1"
Again
s.match?(/\A(?:|>line\d+)\z/)
#=> true
so s is return from the block.
Now the third element of b is passed to the block. (Finally, something interesting.)
s = "\nABC\nDEF\nGHI\n"
d = s.scan(/\p{Lu}/)
#=> ["A", "B", "C", "D", "E", "F", "G", "H", "I"]
a = d.shuffle
#=> ["D", "C", "G", "H", "B", "F", "I", "E", "A"]
s.gsub(/\p{Lu}/) { a.shift }
#=> "\nDCG\nHBF\nIEA\n"
The remaining calculations are similar.
I would like to take a string that contains positional argument markers (not named), supply it with an array (not hash) of values, and have it evaluated.
The use case as an example would be somewhat like ARGV.
For example,
# given:
string = "echo $1 ; echo $#"
values = ["hello", "world"]
# expected result:
"echo hello ; echo hello world"
The below function is the best I could come up with:
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
# Create a variable that can replace $# with all arguments, and quote
# arguments that had "more than one word" originally
all_arguments = arguments.map{|a| a =~ /\s/ ? "\"#{a}\"" : a}.join ' '
# Replace all $1 - $9 with their respective argument ($1 ==> arguments[0])
string.gsub!(/\$(\d)/) { arguments[$1.to_i - 1] }
# Replace $# or $* with all arguments
string.gsub!(/\$[*|#]/, all_arguments)
return string
end
And it seems to me like it can and should be simpler.
I was hoping to find something that is closer to the Kernel.sprintf method of doing things - like "string with %{marker}" % {marker: 'value'}
So, although this issue is almost solved for me (I think), I would love to know if there is something I missed that can make it more elegant.
It seems like you're trying to reproduce Bash-style variable expansion, which is an extremely complex problem. At the very least, though, you can simplify your code in two ways:
Use Kernel.sprintf's built in positional argument feature. The below code does this by substituting e.g. $1 with the sprintf equivalent %1$s.
Use Shellwords from the standard library to escape arguments with spaces etc.
require 'shellwords'
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
tmpl = string.gsub(/\$(\d+)/, '%\1$s')
(tmpl % arguments).gsub(/\$[*#]/, arguments.shelljoin)
end
string = "echo $1 ; echo $#"
values = ["hello", "world"]
puts evaluate_args(string, values)
# => echo hello ; echo hello world
If you didn't have the $* requirement I'd suggest just dropping the Bash-like format and just using sprintf, since it covers everything else you mentioned. Even so, you could further simplify things by using sprintf formatting for everything else:
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
string.gsub('%#', arguments.shelljoin) % arguments
end
string = "echo %1$s ; echo %#"
values = ["hello", "world"]
puts evaluate_args(string, values)
# => echo hello ; echo hello world
Edit
If you want to use %{1} with sprintf you could turn the input array into a hash where the integer indexes are turned into symbol keys, e.g. ["hello", "world"] becomes { :"1" => "hello", :"2" => "world" }:
require "shellwords"
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
string % {
:* => arguments.shelljoin,
**arguments.map.with_index {|val,idx| [ :"#{idx + 1}", val ] }.to_h
}
end
string = "echo %{1} ; echo %{*}"
values = ["hello", "world"]
puts evaluate_args(string, values)
# => echo hello ; echo hello world
string = "echo $1 ; echo $# ; echo $2 ; echo $cat"
values = ["hello", "World War II"]
vals = values.map { |s| s.include?(' ') ? "\"#{s}\"" : s }
#=> ["hello", "\"World War II\""]
all_vals = vals.join(' ')
#=> "hello \"World War II\""
string.gsub(/\$\d+|\$[#*]/) { |s| s[/\$\d/] ? vals[s[1..-1].to_i-1] : all_vals }
#=> "echo hello ; echo hello \"World War II\" ; echo \"World War II\" ; echo $cat" $cat"
I have this line in a bash script and I can't figure out what the "=>" means? I don't think it means equal to or greater than but maybe it does. Thoughts?
"echo '\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },' >> /tmp/user_data.config\n",
It doesn't mean anything, because it's inside a string. Consider:
$ echo 'foo'
foo
$ echo 'foo => bar'
foo => bar
The => doesn't have any significance; it's just part of the string that echo writes to its output.
In the case of your code, the echo command and its string argument are followed by >> /tmp/user_data.config, which just means that the output will be appended to the user_data.config file. Like so:
$ touch /tmp/out.txt
$ echo 'foo => bar' >> /tmp/out.txt
$ echo 'baz => qux' >> /tmp/out.txt
$ cat /tmp/out.txt
foo => bar
baz => qux
The => is part of the string that is being echo'd. Try running the command (without the " at the beginning and the \n", at the end) in BASH and you'll see it just echoes the string and appends it to /tmp/user_data.config
$ echo '\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },' >> /tmp/user_data.config
$ cat /tmp/user_data.config
\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },
I want to read a file which has name value pair on a remote server. As per the requirement I need to shell into the remote server, read the file, and then grep for the values. Example:
/domain/srvr/primary = ABC
/host/DEF/second = DEF
/host/XYZ/second = XYZ
/host/GHI/second = GHI
:
:
:
Now I want to read this file and make an array of all secondary servers (ex: DEF, XYZ, GHI) but I am getting nil value.
primary = #ssh.exec!("cd /home/dir; grep 'srvr/primary' #{filename} | awk '{print $3}'")
secondary = #ssh.exec!("cd /home/dir; grep '\<host.*second\>' #{filename} | awk '{print $3}'")`
It prints the primary server name properly but returns nil for secondary servers array. I tried to use split("\n") but errors out by saying undefined method 'split' for nil:NilClass.
Need help in getting all the secondary servers in an array.
You can use something like this
file_contents.
split("\n").
map {|line| line.split(" = ") }.
find_all {|pair| pair[0] =~ /second$/ }.
map(&:last)
and probably get file contents with cat or downloading the file through ssh. If in the same server just use File.read.
If you can only use bash or just prefer to, you can use
grep -p "^\/host.*second" path/to/file | cut -d" " -f 3
The -p option will enable perl regexp syntax on grep which will give you all the capabilities you want to search through the file. Then cut will split each string by the delimiter -d ang get the field of 1 based index -f. In this case string = server, so server is on the third field.
Instead of trying to do the work on the remote host, you can simplify the task and only grab the data and process it locally. You can use Net::SCP (docs), Net::FTP or Net::SFTP to easily retrieve the data.
I'd use something like this to grab the desired data once the text has been received:
data = <<EOT
/domain/srvr/primary = ABC
/host/DEF/second = DEF
/host/XYZ/second = XYZ
/host/GHI/second = GHI
EOT
data.split("\n").grep(/\bsecond\b/).map{ |l| l.split.last }
# => ["DEF", "XYZ", "GHI"]
or:
data.split("\n").grep(/\bsecond\b/).map{ |l| l[/= (\S+)/, 1] }
# => ["DEF", "XYZ", "GHI"]
or:
data.split("\n").grep(/\bsecond\b/).map{ |l| l.rstrip[/\S+$/] }
# => ["DEF", "XYZ", "GHI"]
Just to make it more interesting:
require 'fruity'
data = <<EOT
/domain/srvr/primary = ABC
/host/DEF/second = DEF
/host/XYZ/second = XYZ
/host/GHI/second = GHI
EOT
compare do
p1 { data.split("\n").grep(/\bsecond\b/).map{ |l| l[/= (\S+)/, 1] } }
p2 { data.split("\n").grep(/\bsecond\b/).map{ |l| l.rstrip[/\S+$/] } }
p3 { data.split("\n").grep(/\bsecond\b/).map{ |l| l.rstrip[/\S+$/] } }
end
# >> Running each test 256 times. Test will take about 1 second.
# >> p1 is faster than p2 by 3.9x ± 0.01
# >> p2 is similar to p3