This question already has answers here:
Checking the success of a command in a bash `if [ .. ]` statement
(1 answer)
When to wrap quotes around a shell variable?
(5 answers)
Closed last year.
I am trying to pass a regular expression as a parameter. What should I fix in my code?
My goal is to send find and the regular expression string, then use grep on the parameter so I can do whatever I want with what grep finds (which is print the count of occurrences).
This is what I send:
$ ./lab12.sh find [Gg]reen
Here's my bash code:
if [[ "$1" == "find" ]]
then
declare -i cnt=0
for file in /tmp/beatles/*.txt ; do
if [[ grep -e $2 ]] //problem is here...
then
((cnt=cnt+1))
fi
done
echo "$cnt songs contain the pattern "$2""
fi
The if statement takes a command. [[ being one, and grep is another, writing [[ grep ... ]] is essentially as wrong as writing vim grep, or cat grep etc, just use:
if grep -q -e "$pattern"
then
...
instead.
The -q switch to grep will disable output, but set the exit status to 0 (success) when the pattern is matches, and 1 (failure) otherwise, and the if statement will only execute the then block if the command succeded.
Using -q will allow grep to exit as soon as the first line is matches.
And as always, remember to wrap your paremeter expansions in double quotes, to avoid pathname expansion and wordsplitting.
Note that square brackets [...] will be interpreted by your calling shell, and you should escape them, or wrap the whole pattern in quotes.
It's always recommended use single quotes, as the only special character is another single quote.
$ ./lab12.sh find '[Gg]reen'
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 1 year ago.
I have to build a bash script that removes upper and lower lines from a file.
The script ask the person for a word, searches the word in the file and removes 4 upper lines, 9 lower lines and the line that contains the word.
Bellow are the commands that works:
vi -e -c 'g/word/.-4,-d' -c 'wq' fileName
sed -i '/word/,+9d' fileName
The problem is that I want to ask the user for a word that I will use as a variable to do all that.
The bellow code doesn't work!
#!/bin/bash
read -p "Insert the word:" word
vi -e -c 'g/$word/.-4,-d' -c 'wq' fileName
sed -i '/$word/,+9d' fileName
What should I do to solve it?
something like this will work, assumes the search word appears only once in the file
$ awk -v b=4 -v a=9 -v word="$word" 'NR==FNR{if($0~word) n=NR; next}
FNR<n-b || FNR>n+a' file{,}
double scans the file, first to find the line number of the match, second round prints out the lines based on before/after context set.
Also doesn't handle if the search word not found. If you want to print everything add !n || to the last condition
You can do it one shot with vi or ed, no need to use sed. Variables inside single quotes won't be interpolated, you need double quotes for that. See Difference between single and double quotes in Bash for details.
vi -e -c 'g/'"$word"'/.-4,+9d' -c 'wq' fileName
printf '/%s/\n-4,+9d\nwq\n' "$word" | ed -s fileName > /dev/null
I don't know ed enough to prevent the line matching the given word from being printed, hence the use of /dev/null
I have a file and its name looks like:
12U12345._L001_R1_001.fastq.gz
I want to assign to a variable just the 12U12345 part.
So far I have:
variable=`basename $fastq | sed {s'/_S[0-9]*_L001_R1_001.fastq.gz//'}`
Note: $fastq is a variable with the full path to the file in it.
This solution currently returns the full file name, any ideas how to get this right?
Just use the built-in parameter expansion provided by the shell, instead of spawning a separate process
fastq="12U12345._L001_R1_001.fastq.gz"
printf '%s\n' "${fastq%%.*}"
12U12345
or use printf() itself to store to a new variable in one-shot
printf -v numericPart '%s' "${fastq%%.*}"
printf '%s\n' "${numericPart}"
Also bash has a built-in regular expression comparison operator, represented by =~ using which you could do
fastq="12U12345._L001_R1_001.fastq.gz"
regex='^([[:alnum:]]+)\.(.*)'
if [[ $fastq =~ $regex ]]; then
numericPart="${BASH_REMATCH[1]}"
printf '%s\n' "${numericPart}"
fi
You could use cut:
$> fastq="/path/to/12U12345._L001_R1_001.fastq.gz"
$> variable=$(basename "$fastq" | cut -d '.' -f 1)
$> echo "$variable"
12U12345
Also, please note that:
It's better to wrap your variable inside quotes. Otherwise you command won't work with filenames that contain space(s).
You should use $() instead of the backticks.
Using Bash Parameter Expansion to extract the basename and then extract the portion of the filename you want:
fastq="/path/to/12U12345._L001_R1_001.fastq.gz"
file="${fastq##*/}" # gives 12U12345._L001_R1_001.fastq.gz
string="${file%%.*}" # gives 12U12345
Note that Bash doesn't allow us to nest the parameter expansion. Otherwise, we could have combined statements 2 and 3 above.
I don't usually work in bash but grep could be a really fast solution in this case. I have read a lot of questions on grep and variable assignment in bash yet I do not see the error. I have tried several flavours of double quotes around $pattern, used `...`` or $(...) but nothing worked.
So here's what I try to do:
I have two files. The first contains several names. Each of them I want to use as a pattern for grep in order to search them in another file. Therefore I loop through the lines of the first file and assign the name to the variable pattern.
This step works as the variable is printed out properly.
But somehow grep does not recognize/interpret the variable. When I substitute "$pattern" with an actual name everything is fine as well. Therefore I don't think the variable assignment has a problem but the interpretation of "$pattern" as the string it should represent.
Any help is greatly appreciated!
#!/bin/bash
while IFS='' read -r line || [[ -n $line ]]; do
a=( $line )
pattern="${a[2]}"
echo "Text read from file: $pattern"
var=$(grep "$pattern" 9606.protein.aliases.v10.txt)
echo "Matched Line in Alias is: $var"
done < "$1"
> bash match_Uniprot_StringDB.sh ~/Chromatin_Computation/.../KDM.protein.tb
output:
Text read from file: "UBE2B"
Matched Line in Alias is:
Text read from file: "UTY"
Matched Line in Alias is:
EDIT
The solution drvtiny suggested works. It is necessary to get rid of the double quotes to match the string. Adding the following lines makes the script work.
pattern="${pattern#\"}"
pattern="${pattern%\"}"
Please, look at "-f FILE" option in man grep.
I advise that this option do exactly what you need without any bash loops or such other "hacks" :)
And yes, according to the output of your code, you read pattern including double quotes literally. In other words, you read from file ~/Chromatin_Computation/.../KDM.protein.tb this string:
"UBE2B"
But not
UBE2B
as you probably expect.
Maybe you need to remove double quotes on the boundaries of your $pattern?
Try to do this after reading pattern:
pattern=${pattern#\"}
pattern=${pattern%\"}
This question already has answers here:
How can I preserve quotes in printing a bash script's arguments
(7 answers)
Closed 3 years ago.
I have a Bash script where I want to keep quotes in the arguments passed.
Example:
./test.sh this is "some test"
then I want to use those arguments, and re-use them, including quotes and quotes around the whole argument list.
I tried using \"$#\", but that removes the quotes inside the list.
How do I accomplish this?
using "$#" will substitute the arguments as a list, without re-splitting them on whitespace (they were split once when the shell script was invoked), which is generally exactly what you want if you just want to re-pass the arguments to another program.
Note that this is a special form and is only recognized as such if it appears exactly this way. If you add anything else in the quotes the result will get combined into a single argument.
What are you trying to do and in what way is it not working?
There are two safe ways to do this:
1. Shell parameter expansion: ${variable#Q}:
When expanding a variable via ${variable#Q}:
The expansion is a string that is the value of parameter quoted in a format that can be reused as input.
Example:
$ expand-q() { for i; do echo ${i#Q}; done; } # Same as for `i in "$#"`...
$ expand-q word "two words" 'new
> line' "single'quote" 'double"quote'
word
'two words'
$'new\nline'
'single'\''quote'
'double"quote'
2. printf %q "$quote-me"
printf supports quoting internally. The manual's entry for printf says:
%q Causes printf to output the corresponding argument in a format that can be reused as shell input.
Example:
$ cat test.sh
#!/bin/bash
printf "%q\n" "$#"
$
$ ./test.sh this is "some test" 'new
>line' "single'quote" 'double"quote'
this
is
some\ test
$'new\nline'
single\'quote
double\"quote
$
Note the 2nd way is a bit cleaner if displaying the quoted text to a human.
Related: For bash, POSIX sh and zsh: Quote string with single quotes rather than backslashes
Yuku's answer only works if you're the only user of your script, while Dennis Williamson's is great if you're mainly interested in printing the strings, and expect them to have no quotes-in-quotes.
Here's a version that can be used if you want to pass all arguments as one big quoted-string argument to the -c parameter of bash or su:
#!/bin/bash
C=''
for i in "$#"; do
i="${i//\\/\\\\}"
C="$C \"${i//\"/\\\"}\""
done
bash -c "$C"
So, all the arguments get a quote around them (harmless if it wasn't there before, for this purpose), but we also escape any escapes and then escape any quotes that were already in an argument (the syntax ${var//from/to} does global substring substitution).
You could of course only quote stuff which already had whitespace in it, but it won't matter here. One utility of a script like this is to be able to have a certain predefined set of environment variables (or, with su, to run stuff as a certain user, without that mess of double-quoting everything).
Update: I recently had reason to do this in a POSIX way with minimal forking, which lead to this script (the last printf there outputs the command line used to invoke the script, which you should be able to copy-paste in order to invoke it with equivalent arguments):
#!/bin/sh
C=''
for i in "$#"; do
case "$i" in
*\'*)
i=`printf "%s" "$i" | sed "s/'/'\"'\"'/g"`
;;
*) : ;;
esac
C="$C '$i'"
done
printf "$0%s\n" "$C"
I switched to '' since shells also interpret things like $ and !! in ""-quotes.
If it's safe to make the assumption that an argument that contains white space must have been (and should be) quoted, then you can add them like this:
#!/bin/bash
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
i=\"$i\"
fi
echo "$i"
done
Here is a sample run:
$ ./argtest abc def "ghi jkl" $'mno\tpqr' $'stu\nvwx'
abc
def
"ghi jkl"
"mno pqr"
"stu
vwx"
You can also insert literal tabs and newlines using Ctrl-V Tab and Ctrl-V Ctrl-J within double or single quotes instead of using escapes within $'...'.
A note on inserting characters in Bash: If you're using Vi key bindings (set -o vi) in Bash (Emacs is the default - set -o emacs), you'll need to be in insert mode in order to insert characters. In Emacs mode, you're always in insert mode.
I needed this for forwarding all arguments to another interpreter.
What ended up right for me is:
bash -c "$(printf ' %q' "$#")"
Example (when named as forward.sh):
$ ./forward.sh echo "3 4"
3 4
$ ./forward.sh bash -c "bash -c 'echo 3'"
3
(Of course the actual script I use is more complex, involving in my case nohup and redirections etc., but this is the key part.)
Like Tom Hale said, one way to do this is with printf using %q to quote-escape.
For example:
send_all_args.sh
#!/bin/bash
if [ "$#" -lt 1 ]; then
quoted_args=""
else
quoted_args="$(printf " %q" "${#}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_args}"
send_fewer_args.sh
#!/bin/bash
if [ "$#" -lt 2 ]; then
quoted_last_args=""
else
quoted_last_args="$(printf " %q" "${#:2}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_last_args}"
receiver.sh
#!/bin/bash
for arg in "$#"; do
echo "$arg"
done
Example usage:
$ ./send_all_args.sh
$ ./send_all_args.sh a b
a
b
$ ./send_all_args.sh "a' b" 'c "e '
a' b
c "e
$ ./send_fewer_args.sh
$ ./send_fewer_args.sh a
$ ./send_fewer_args.sh a b
b
$ ./send_fewer_args.sh "a' b" 'c "e '
c "e
$ ./send_fewer_args.sh "a' b" 'c "e ' 'f " g'
c "e
f " g
Just use:
"${#}"
For example:
# cat t2.sh
for I in "${#}"
do
echo "Param: $I"
done
# cat t1.sh
./t2.sh "${#}"
# ./t1.sh "This is a test" "This is another line" a b "and also c"
Param: This is a test
Param: This is another line
Param: a
Param: b
Param: and also c
Changed unhammer's example to use array.
printargs() { printf "'%s' " "$#"; echo; }; # http://superuser.com/a/361133/126847
C=()
for i in "$#"; do
C+=("$i") # Need quotes here to append as a single array element.
done
printargs "${C[#]}" # Pass array to a program as a list of arguments.
My problem was similar and I used mixed ideas posted here.
We have a server with a PHP script that sends e-mails. And then we have a second server that connects to the 1st server via SSH and executes it.
The script name is the same on both servers and both are actually executed via a bash script.
On server 1 (local) bash script we have just:
/usr/bin/php /usr/local/myscript/myscript.php "$#"
This resides on /usr/local/bin/myscript and is called by the remote server. It works fine even for arguments with spaces.
But then at the remote server we can't use the same logic because the 1st server will not receive the quotes from "$#". I used the ideas from JohnMudd and Dennis Williamson to recreate the options and parameters array with the quotations. I like the idea of adding escaped quotations only when the item has spaces in it.
So the remote script runs with:
CSMOPTS=()
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
CSMOPTS+=(\"$i\")
else
CSMOPTS+=($i)
fi
done
/usr/bin/ssh "$USER#$SERVER" "/usr/local/bin/myscript ${CSMOPTS[#]}"
Note that I use "${CSMOPTS[#]}" to pass the options array to the remote server.
Thanks for eveyone that posted here! It really helped me! :)
Quotes are interpreted by bash and are not stored in command line arguments or variable values.
If you want to use quoted arguments, you have to quote them each time you use them:
val="$3"
echo "Hello World" > "$val"
As Gary S. Weaver shown in his source code tips, the trick is to call bash with parameter '-c' and then quote the next.
e.g.
bash -c "<your program> <parameters>"
or
docker exec -it <my docker> bash -c "$SCRIPT $quoted_args"
If you need to pass all arguments to bash from another programming language (for example, if you'd want to execute bash -c or emit_bash_code | bash), use this:
escape all single quote characters you have with '\''.
then, surround the result with singular quotes
The argument of abc'def will thus be converted to 'abc'\''def'. The characters '\'' are interpreted as following: the already existing quoting is terminated with the first first quote, then the escaped singular single quote \' comes, then the new quoting starts.
Yes, seems that it is not possible to ever preserve the quotes, but for the issue I was dealing with it wasn't necessary.
I have a bash function that will search down folder recursively and grep for a string, the problem is passing a string that has spaces, such as "find this string". Passing this to the bash script will then take the base argument $n and pass it to grep, this has grep believing these are different arguments. The way I solved this by using the fact that when you quote bash to call the function it groups the items in the quotes into a single argument. I just needed to decorate that argument with quotes and pass it to the grep command.
If you know what argument you are receiving in bash that needs quotes for its next step you can just decorate with with quotes.
Just use single quotes around the string with the double quotes:
./test.sh this is '"some test"'
So the double quotes of inside the single quotes were also interpreted as string.
But I would recommend to put the whole string between single quotes:
./test.sh 'this is "some test" '
In order to understand what the shell is doing or rather interpreting arguments in scripts, you can write a little script like this:
#!/bin/bash
echo $#
echo "$#"
Then you'll see and test, what's going on when calling a script with different strings