This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 4 years ago.
I am trying to use a variable to store grep's options:
#!/bin/sh
set -xe
GREP_OPTS="-e \"test this\""
echo "I want to test this." | grep $GREP_OPTS
Output:
+ GREP_OPTS=-e "test this"
+ echo I want to test this.
+ grep -e "test this"
grep: this": No such file or directory
how can I make this work?
Word-Splitting is occurring on your GREP_OPTS="-e \"test this\"" resulting in the command being
grep -e '"test' 'this"'
Resulting in the exact error:
grep: this": No such file or directory
(of course there is no file named "this\"")
See BashFAQ-50 - I'm trying to put a command in a variable, but the complex cases always fail.
In order to prevent word splitting use an array for options instead of trying to use a single variable, e.g.
#!/bin/sh
set -xe
GREP_OPTS=(-e "test this")
echo "I want to test this." | grep "${GREP_OPTS[#]}"
Example Use/Output
$ bash grepopts.sh
+ GREP_OPTS=(-e "test this")
+ echo 'I want to test this.'
+ grep -e 'test this'
I want to test this.
Let me know if you have further questions.
Related
This question already has answers here:
How can I loop over the output of a shell command?
(4 answers)
Closed 2 months ago.
i am building a bash script that is supposed to put each line of the output of one command to an variable and then run some commands on that, i am doing it like
for i in `cmd`
do
echo i=$i
lang=$(echo $i | cut -d '"' -f 6)
echo lang=$lang
#some stuff
done
my problem is that for is using space and newlines for separation to different $i's and i want it to do create only new $i's with newline delimiters cause every line may have a cupple of spaces and i want them no matter that handled as it own...
google but found nothing really helping me, only suggestions to use xargs which dosnt help me cause i need to use not one command but a cupple after creating some variables and running some if statements that desiside which command is to run if any...
If you want to read cmd's output line by line you can do it using
while loop and bash's internal read command
cmd | while IFS= read -r i
do
echo "i=${i}"
lang="$(echo "${i}" | cut -d '"' -f 6)"
echo "lang=${lang}"
#some stuff
done
Use " around a variable's de-reference to avoid problems with spaces inside it's value.
This question already has answers here:
Forcing bash to expand variables in a string loaded from a file
(13 answers)
Closed 1 year ago.
I would like to create a templating system that executes bash within a text file.
For example, let's consider we created a simple template.yaml file:
my_path: $(echo ${PATH})
my_ip: $(curl -s http://whatismyip.akamai.com/)
some_const: "foo bar"
some_val: $(echo -n $MY_VAR | base64)
The desire is to execute each one, such that the result may look like:
my_path: /Users/roman/foo
my_ip: 1.2.3.4
some_const: "foo bar"
some_val: ABC
How would I go about doing such a substitution?
Reasons for wanting this:
There are many values, and doing something like a sed or envsubst isn't practical
It would be common to apply a series of piped transformations
The configuration file would be populated from numerous sources, all of them essentially bash commands
I do need to create a yaml file of a specific format (ultimately used by another tool)
I could create aliases etc to increase readability
By having it execute in it's own shell none of the semi-sensitive values are stored as in history or as files.
I'm not married to this approach, and would happily attempt a recommendation that fulfils the reasons.
This might work, you can try though:
you can create a script: script.sh which will take one argument as .yaml file and will expand the variables inside that file:
script.sh :
echo 'cat <<EOF' > temp.sh
cat "$1" >> temp.sh
echo 'EOF' >> temp.sh
bash temp.sh
rm temp.sh
and you can invoke the script as from the command line : ./script.sh template.yaml
Thank you to #joshmeranda for pointing me in the right direction, this solved my problem
echo -e "$(eval "echo -e \"`<template.yaml`\"")"
While eval can be dangerous, in my case its usage is controlled.
This question already has answers here:
Why does a space in a variable assignment give an error in Bash? [duplicate]
(3 answers)
How do I set a variable to the output of a command in Bash?
(15 answers)
Closed 3 years ago.
I'm new to bash script, it is interesting, but somehow I'm struggling with everything.
I have a file, separated by tab "\t" with 2 infos : a string and a number.
I'd like to use both info on each line into a bash script in order to look into a file for those infos.
I'm not even there, I'm struggling to give the arguments from the two columns as two arguments for bash.
#/!bin/bash
FILE="${1}"
while read -r line
do
READ_ID_WH= "echo ${line} | cut -f 1"
POS_HOTSPOT= echo '${line} | cut -f 2'
echo "read id is : ${READ_ID_WH} with position ${POS_HOTSPOT}"
done < ${FILE}
and my file is :
ABCD\t1120
ABC\t1121
I'm launching my command with
./script.sh file_of_data.tsv
What I finally get is :
script.sh: line 8: echo ABCD 1120 | cut -f 1: command not found
I tried a lot of possibilities by browsing SO, and I can't make it to divide my line into two arguments to be used separately in my script :(
Hope you can help me :)
Best,
The quotes cause the shell to look for a command whose name is the string between the quotes.
Apparently you are looking for
while IFS=$'\t' read -r id hotspot; do
echo "read id is: $id with position $hotspot"
done <"$1"
You generally want to avoid capturing things into variables you only use once, but the syntax for that is
id=$(echo "$line" | cut -f1)
See also Correct Bash and shell script variable capitalization and When to wrap quotes around a shell variable?. You can never have whitespace on either side of the = assignment operator (or rather, incorrect whitespace changes the semantics to something you probably don't want).
You have a space after the equals sign on lines 5 and 6, so it thinks you are looking for an executable file named echo ABCD 1120 | cut -f 1 and asking to execute it while assigning the variable READ_ID_WH to the empty string, rather than assigning the string you want to the variable.
This question already has answers here:
Send string to stdin
(6 answers)
Command not found error in Bash variable assignment
(5 answers)
How do I set a variable to the output of a command in Bash?
(15 answers)
Closed 3 years ago.
I am trying to assign a variable to the result of a regex substitution in bash. For instance, when I run
echo $initialvar | perl -pe 's/.+(Some_Dir\/)(.+)/\2/'
I get the desired output from the echo. How would I assign a newvar to the resulting output?
I've tried:
newvar= "$($initialvar | perl -pe 's/.+(Some_Dir\/)(.+)/\2/')"
newvar= echo $initialvar | perl -pe 's/.+(Some_Dir\/)(.+)/\2/'
but none of them work
As for your question, it seems to consist of two challenges: One being assigning the output of running a command to a BASH variable, and the other being piping the content of a variable to the standard input of a Perl program.
One
foo=$(bar)
runs bar and saves its output to the BASH variable $foo.
The other
Any of
echo "$foo" | bar
bar <(echo "$foo")
bar <<< "$foo"
runs bar with the content of $foo piped to its standard input
Combining the two
baz=$(bar <<< "$foo")
sets $baz to the value produced by bar having the content of $foo sent to its standard input.
Secondly, a few Perl-related suggestions peripherally related to your question:
I might instead use perl -nE:
-n will loop through each line like -p, but won't print by default.
-E will evaluate like -e, but with experimental features like say enabled.
You can avoid escaping the slash in Some_Dir/ by using another separator, e.g. s!...!...! or m!...!. Instead of replacing with s//, since you're just printing "Some_Dir/" if it matches, you might as well go and do that directly:
perl -nE 'say (m!Some_Dir/! ? "Some_Dir/" : $_)'
So:
newvar=$(perl -nE 'say (m!Some_Dir/! ? "Some_Dir/" : $_)' <<< "$initialvar")
You can use either of the following:
newvar=$(echo "$initialvar" | perl -pe 's/.+(Some_Dir\/)(.+)/\2/')
newvar=`echo "$initialvar" | perl -pe 's/.+(Some_Dir\/)(.+)/\2/'`
This question already has answers here:
grep for expression containing variable
(2 answers)
Closed 9 years ago.
In my bash script, I am attempting to parse through a status file and detect errors based on some keywords. I store these prefixes in a list, and then loop through them.
Bash script:
status_page="/path/to/file.txt"
list="aaa bbb ccc ddd"
for pre in $list
do
echo "grep '\w\w\w${pre}-.*\.lin failed' ${status_page}" # debug
if grep '\w\w\w${pre}-.*\.lin failed' ${status_page}; then
echo "Found error!"
exit 8;
fi
done
/path/to/file.txt:
xyzfff-tool.lin failed
xyzggg-exec.lin failed
rstccc-tool.lin failed
The bash script should catch the line rstccc-tool.lin failed, but it skips right over it.
For debugging, I print the grep commands verbatim, and when I copy that line and issue the command in my shell (tcsh), it returns that line...
Shell:
$ grep '\w\w\wccc-.*\.lin failed' /path/to/file.txt
rstccc-tool.lin failed
$ echo $?
0
If grep can find the line when I issue the command normally, how come it won't find it when the bash script is calling grep?
The variable won't be expanded in single quotes. Try with double quotes:
if grep "\w\w\w${pre}-.*\.lin failed" "${status_page}"; then
The ${pre} portion of that script is not parsing it correctly. I believe you need that line to say:
if grep '\w\w\w'${pre}'-.*\.lin failed' ${status_page}; then
... where the ${pre} is outside the quotation, such that bash will do the correct string replacement before sending it to grep.