I know something like this is possible
out = `echo 1`
$?.to_i == 0 or raise “Failed"
Yet I’m unable to merge these 2 statements, so that the output will be captured into a variable and the command will fail (also printing the captured output) if the shell command returns with error.
Preferably into a 1 lines, if possible. Something like
out = `echo 1` && $?.to_i == 0 or raise “Failed. Output:” + out
only prettier.
Look at the Open3 class. It has a number of methods that will let you do what you want.
In particular, capture2 is the closest to what you're doing. From the docs:
::capture2 captures the standard output of a command.
stdout_str, status = Open3.capture2([env,] cmd... [, opts])
Pay attention to that optional env parameter. Without that your called application will have no environment information so you might want to consider passing in the ENV hash, allowing the child to have the same environment settings as the running code. If you want to restrict what is passed you can selectively add key/value pairs to a hash, or use ENV.dup then delete selected key/value pairs.
Related
I'm defining a variable as a composition of other variables and some text, and I'm trying to get this variable to not expand its containing variables on the assigning. But I want it to expand when called later. That way I could reuse the same template to print different results as the inner variables keep changing. I'm truing to avoid eval as much as possible as I will be receiving some of the inner variables from third parties, and I do not know what to expect.
My use case, as below, is to have some "calling stack" so I can log all messages with the same format and keep a record of the script, function, and line of the logged message in some format like this: script.sh:this_function:42.
My attempted solution
called.sh:
#!/bin/bash
SCRIPT_NAME="`basename "${BASH_SOURCE[0]}"`"
CURR_STACK="${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"
echo "${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"
echo "${CURR_STACK}"
echo
function _func_1 {
echo "${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"
echo "${CURR_STACK}"
}
_func_1
So, I intend to get the same results while printing the "${CURR_STACK}" as when printing the previous line.
If there is some built-in or other clever way to log this 'call stack', by all means, let me know! I'll gladly wave my code good-bye, but I'd still like to know how to prevent the variables from expanding right away on the assigning of CURR_STACK, but still keep them able to expand further ahead.
Am I missing some shopt?
What I've tried:
Case 1 (expanding on line 4):
CURR_STACK="${SCRIPT_NAME}:${FUNNAME[0]}:${LINENO[0]}"
CURR_STACK="`echo "${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}"`"
CURR_STACK="`echo "\${SCRIPT_NAME}:\${FUNCNAME[0]}:\${LINENO[0]}"`"
called.sh::7 <------------------| These are control lines
called.sh::4 <---------------. .------------| With the results I expect to get.
X
called.sh:_func_1:12 <---´ `-------| Both indicate that the values expanded
called.sh::4 <-------------------------| on line 4 - when CURR_STACK was set.
Case 2 (not expanding at all):
CURR_STACK="\${SCRIPT_NAME}:\${FUNNAME[0]}:\${LINENO[0]}"
CURR_STACK=\${SCRIPT_NAME}:\${FUNCNAME[0]}:\${LINENO[0]}
CURR_STACK="`echo '${SCRIPT_NAME}:${FUNCNAME[0]}:${LINENO[0]}'`"
called.sh::7
${SCRIPT_NAME}:${FUNNAME[0]}:${LINENO[0]} <-------.----| No expansion at all!...
/
called.sh::12 /
${SCRIPT_NAME}:${FUNNAME[0]}:${LINENO[0]} <----´
Shell variables are store plain inert text(*), not executable code; there isn't really any concept of delayed evaluation here. To make something that does something when used, create a function instead of a variable:
print_curr_stack() {
echo "$(basename "${BASH_SOURCE[1]}"):${FUNCNAME[1]}:${BASH_LINENO[0]}"
}
# ...
echo "We are now at $(print_curr_stack)"
# Or just run it directly:
print_curr_stack
Note: using BASH_SOURCE[1] and FUNCNAME[1] gets info about context the function was run from, rather than where it is in the function itself. But for some reason I'm not clear on, BASH_LINENO[1] gets the wrong info, and BASH_LINENO[0] is what you want.
You could also write it to allow the caller to specify additional text to print:
print_curr_stack() {
echo "$#" "$(basename "${BASH_SOURCE[1]}"):${FUNCNAME[1]}:${BASH_LINENO[0]}"
}
# ...
print_curr_stack "We are now at"
(* There's an exception to what I said about variables just contain inert text: some variables -- like $LINENO, $RANDOM, etc -- are handled specially by the shell itself. But you can't create new ones like this except by modifying the shell itself.)
Are you familiar with eval?
$ a=this; b=is; c=a; d=test;
$ e='echo "$a $b $c $d"';
$ eval $e;
this is a test
$ b='is NOT'; # modify one of the variables
$ eval $e;
this is NOT a test
$ f=$(eval $e); # capture the value of the "eval" statement
$ echo $f;
this is NOT a test
I'm using bash 4.4.19(1)-release.
At the start of my program I read customer configuration values from the command line, configuration file(s), and the environment (in decreasing order of precedence). I validate these configuration values against internal definitions, failing out if required values are missing or if the customer values don't match against accepted regular expressions. This approach is a hard requirement and I'm stuck using BASH for this.
The whole configuration process involves the parsing of several YAML files and takes about a second to complete. I'd like to only have to do this once in order to preserve performance. Upon completion, all of the configured values are placed in a global associative array declared as follows:
declare -gA CONFIG_VALUES
A basic helper function has been written for accessing this array:
# A wrapper for accessing the CONFIG_VALUES array.
function get_config_value {
local key="${1^^}"
local output
output="${CONFIG_VALUES[${key}]}"
echo "$output"
}
This works perfectly fine when all of the commands are run within the same shell. This even works when the get_config_value function is called from a child process. Where this breaks down is when it's called from a child process and the value in the array contains slashes. This leads to errors such as the following (line 156 is "output="${CONFIG_VALUES[${key}]}"):
config.sh: line 156: path/to/some/file: syntax error: operand expected (error token is "/to/some/file")
This is particularly obnoxious because it seems to be reading the value "path/to/some/file" just fine. It simply decides to announce a syntax error after doing so and falls over dead instead of echoing the value.
I've been trying to circumvent this by running the array lookup in a subshell, capturing the syntax failure, and grepping it for the value I need:
# A wrapper for accessing the CONFIG_VALUES array.
function get_config_value {
local key="${1^^}"
local output
if output="$(echo "${CONFIG_VALUES[${key}]}" 2>&1)"; then
echo "$output"
else
grep -oP "(?<=: ).*(?=: syntax error: operand expected)" <<< "$output"
fi
}
Unfortunately, it seems that BASH won't let me ignore the "syntax error" like that. I'm not sure where to go from here (well... Python, but I don't get to make that decision).
Any ideas?
I'm dealing with a pipeline of predominantly shell and Perl files, all of which pass parameters (paths) to the next. I decided it would be better to use a single file to store all the paths and just call that for every file. The issue is I am using awk to grab the files at the beginning of each file, and it's turning out to be a lot of repetition.
My question is: I do not know if there is a way to store key-value pairs in a file so shell can natively do something with the key and return the value? It needs to access an external file, because the pipeline uses many scripts and a map in a specific file would result in parameters being passed everywhere. Is there some little quirk I do not know of that performs a map function on an external file?
You can make a file of env var assignments and source that file as need, ie.
$ cat myEnvFile
path1=/x/y/z
path2=/w/xy
path3=/r/s/t
otherOpt1="-x"
Inside your script you can source with either . myEnvFile or the more versbose version of the same feature sourc myEnvFile (assuming bash shell) , i.e.
$cat myScript
#!/bin/bash
. /path/to/myEnvFile
# main logic below
....
# references to defined var
if [[ -d $path2 ]] ; then
cd $path2
else
echo "no pa4h2=$path2 found, can't continue" 1>&1
exit 1
fi
Based on how you've described your problem this should work well, and provide a-one-stop-shop for all of your variable settings.
IHTH
In bash, there's mapfile, but that reads the lines of a file into a numerically-indexed array. To read a whitespace-separated file into an associative array, I would
declare -A map
while read key value; do
map[$key]=$value
done < filename
However this sounds like an XY problem. Can you give us an example (in code) of what you're actually doing? When I see long piplines of grep|awk|sed, there's usually a way to simplify. For example, is passing data by parameters better than passing via stdout|stdin?
In other words, I'm questioning your statement "I decided it would be better..."
I have one loop that creates a group of variables like DISK1, DISK2... where the number at the end of the variable name gets created by the loop and then loaded with a path to a device name. Now I want to use those variables in another loop to execute a shell command, but the variable doesn't give its contents to the shell command.
for (( counter=1 ; counter<=devcount ; counter++))
do
TEMP="\$DISK$counter"
# $TEMP should hold the variable name of the disk, which holds the device name
# TEMP was only for testing, but still has same problem as $DISK$counter
eval echo $TEMP #This echos correctly
STATD$counter=$(eval "smartctl -H -l error \$DISK$counter" | grep -v "5.41" | grep -v "Joe")
eval echo \$STATD$counter
done
Don't use eval ever, except maybe if there is no other way AND you really know what you are doing.
The STATD$counter=$(...) should give an error. That's not a valid assignment because the string "STATD$counter" is not a valid variable name. What will happen is (using a concrete example, if counter happened to be 3 and your pipeline in the $( ) output "output", bash will only expand that line as far as "STATD3=output" so it will try to find a command named "STATD3=output" and run it. Odds are this is not what you intended.
It sounds like everything you want to do can be accomplished with arrays instead. If you are not familiar with bash arrays take a look at Greg's Wiki, in particular this page or the bash man page to find out how to use them.
For example, in the loop you didn't post in your question: make disk (not DISK: don't use all upper case variable names) an array like so
disk+=( "new value" )
or even
disk[counter]="new value"
Then in the loop in your question, you can make statd an array as well and assign it with values from disk by
statd[counter]="... ${disk[counter]} ..."
It's worth saying again: avoid using eval.
I am trying to read part of a file and stop and a particular line, using bash. I am not very familiar with bash, but I've been reading the manual and various references, and I don't understand why something like the following does not work (but instead produces a syntax error):
while { read -u 4 line } && (test "$line" != "$header_line")
do
echo in loop, line=$line
done
I think I could write a loop that tests a "done" variable, and then do my real tests inside the loop and set "done" appropriately, but I am curious as to 1) why the above does not work, and 2) is there some small correction that would make it work? I still fairly confused about when to use [, (, {, or ((, so perhaps some other combination would work, though I have tried several.
(Note: The "read -u 4 line" works fine when I call it above the loop. I have opened a file on file descriptor 4.)
I think what you want is more like this:
while read -u 4 line && test "$line" != "$header_line"
do
...
done
Braces (the {} characters) are used to separate variables from other parts of a string when whitespace cannot be used. For example, echo "${var}x" will print the value of the variable var followed by an x, but echo "$varx" will print the value of the variable varx.
Brackets (the [] characters) are used as a shortcut for the test program. [ is another name for test, but when test detects that it was called with [ it required a ] as its last argument. The point is clarity.
Parenthesis (the () characters) are used in a number of different situations. They generally start subshells, although not always (I'm not really certain in case #3 here):
Retrieving a single exit code from a series of processes, or a single output stream from a sequence of commands. For example, (echo "Hi" ; echo "Bye") | sed -e "s/Hi/Hello/" will print two lines, "Hello" and "Bye". It is the easiest way to get multiple echo statements to produce a single stream.
Evaluating commands as if they were variables: $(expr 1 + 1) will act like a variable, but will produce the value 2.
Performing math: $((5 * 4 / 3 + 2 % 1)) will evaluate like a variable, but will compute the result of that mathematical expression.
The && operator is a list operator - he seperates two commands and only executes when the first is true, but in this case the first is the while and he is expecting his do stuff. And then he reaches do and the while stuff is already history.
Your intention is to put it into the expression. So you put it together with (). E.g. this a solution with just a small change
while ( read -u 4 line && test "$line" != "$header_line" )