I'm trying to run an application that needs a parameter passed in single or double quotes:
bwa mem -R '#RG\tID:foo\tLB=foo\tPL=illumina\tPU=1234.1\tSM=bar' ...
I'm trying to run this in a script:
#!/usr/bin/bash
var=foo_bar
first=$(echo $var | cut -d '_' -f 1)
second=$(echo $var | cut -d '_' -f 2)
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second"
echo \'$readgroup\'
'#RG\tID:foo\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=bar'
However, when I use this as a parameter in my application call:
var=foo_bar
first=$(echo $var | cut -d '_' -f 1)
second=$(echo $var | cut -d '_' -f 2)
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second"
bwa mem -R \'$readgroup\' ...
it's interpreted as
\'#RG\tID:HUM-7\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=HUM-7\'
What am I doing wrong?
The application doesn't care about the quotes; they are only used to protect the string from the shell.
var=foo_bar
IFS=_ read first second <<< "$var"
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second"
bwa mem -R "$readgroup"
If the \t are actually supposed to be literal tab characters, consider using printf instead of an assignment statement.
printf -v readgroup '#RG\tID:%s\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=%s' "$first" "$second"
Should be like:
readgroup="'#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second'" [with '' wrapping]
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second" [without]
Important thing is to use -e option for echo [ See man echo ]
echo -e $readgroup
Related
My current code:
while read -r rbv_line || [[ -n "$rbv_line" ]]; do
if [[ "${rbv_line}" =~ ${rbv_reg} ]]; then
rbv_downcase="${BASH_REMATCH[0],,}" &&
ruby_version="${rbv_downcase//[^0-9a-z\.\-]/}" &&
((reg_matches="${reg_matches}"+1))
printf "\n"
printf "Setting Ruby version: %s\n" "${ruby_version}"
break
fi
done < "${1}"
It does what I want. But I would love to know if I can simplify this code even more, hoping someone can help me understand the syntax.
If you see these two lines:
rbv_downcase="${BASH_REMATCH[0],,}" &&
ruby_version="${rbv_downcase//[^0-9a-z\.\-]/}" &&
Initially I tried to combine those into one using something like this:
ruby_version="${BASH_REMATCH[0],,//[^0-9a-z\.\-]/}"
That does not work.
Is there a way to combine those two parameter expansions (,, and the //[^0-9a-z\.\-]/) or is passing it through an intermediary variable the right approach?
You can view the latest version of the code here:
https://github.com/octopusnz/scripts
You cannot combine multiple parameter expansions, but...
... you can simplify this code!
The biggest gain is by using already available tools.
Instead of looping, let's use grep. It's supposed to do something when RegEx pattern is occurred, so:
grep -E "$rbv_reg" "$1" # -E is for extended RegEx
I guess your pattern isn't case sensitive, so let's disable it with -i flag.
The loop breaks after match, so let's pass -m 1 to stop processing file after first match.
You want to convert uppercase to lowercase, so let's pipe it through tr:
grep -m 1 -E -i "$rbv_reg" "$1" | tr '[:upper:]' '[:lower:]'
You then replace some characters with //[^0-9a-z\.\-]/, piping it to sed will do the trick:
grep -m 1 -E -i "$rbv_reg" "$1" | tr '[:upper:]' '[:lower:]' | sed 's/[^0-9a-z\.\-]//g'
And at the very end, let's grab the output to variable:
ruby_version="$( grep -m 1 -E -i '$rbv_reg' '$1' | tr '[:upper:]' '[:lower:]' | sed 's/[^0-9a-z\.\-]//g' )"
Since you are printing new line anyway, let's use simple echo instead of printf
All what's left is if [ -n "$ruby_version" ] to increment reg_matches
At the end, we got:
ruby_version="$(
grep -m 1 -E -i '$rbv_reg' '$1' |
tr '[:upper:]' '[:lower:]' |
sed 's/[^0-9a-z\.\-]//g'
)"
if [ -n "$ruby_version" ]; then
reg_matches="$((reg_matches+1))"
echo
echo "Setting Ruby version: $ruby_version"
fi
The advantage of above code is the fact it isn't really Bash dependent and should work in any POSIX Bourne compatible shell.
I had problem with cut variables from string in " quotes. I have some scripts to write for my sys classes, I had a problem with a script in which I had to read input from the user in the form of (a="var1", b="var2")
I tried the code below
#!/bin/bash
read input
a=$($input | cut -d '"' -f3)
echo $a
it returns me a error "not found a command" on line 3 I tried to double brackets like
a=$(($input | cut -d '"' -f3)
but it's still wrong.
In a comment the OP gave a working answer (should post it as an answer):
#!/bin/bash
read input
a=$(echo $input | cut -d '"' -f2)
b=$(echo $input | cut -d '"' -f4)
echo sum: $(( a + b))
echo difference: $(( a - b))
This will work for user input that is exactly like a="8", b="5".
Never trust input.
You might want to add the check
if [[ ${input} =~ ^[a-z]+=\"[0-9]+\",\ [a-z]+=\"[0-9]+\"$ ]]; then
echo "Use your code"
else
echo "Incorrect input"
fi
And when you add a check, you might want to execute the input (after replacing the comma with a semicolon).
input='testa="8", testb="5"'
if [[ ${input} =~ ^[a-z]+=\"[0-9]+\",\ [a-z]+=\"[0-9]+\"$ ]];
then
eval $(tr "," ";" <<< ${input})
set | grep -E "^test[ab]="
else
echo no
fi
EDIT:
#PesaThe commented correctly about BASH_REMATCH:
When you use bash and a test on the input you can use
if [[ ${input} =~ ^[a-z]+=\"([0-9]+)\",\ [a-z]+=\"([0-9])+\"$ ]];
then
a="${BASH_REMATCH[1]}"
b="${BASH_REMATCH[2]}"
fi
To extract the digit 1 from a string "var1" you would use a Bash substring replacement most likely:
$ s="var1"
$ echo "${s//[^0-9]/}"
1
Or,
$ a="${s//[^0-9]/}"
$ echo "$a"
1
This works by replacing any non digits in a string with nothing. Which works in your example with a single number field in the string but may not be what you need if you have multiple number fields:
$ s2="1 and a 2 and 3"
$ echo "${s2//[^0-9]/}"
123
In this case, you would use sed or grep awk or a Bash regex to capture the individual number fields and keep them distinct:
$ echo "$s2" | grep -o -E '[[:digit:]]+'
1
2
3
I have file with a few lines:
x 1
y 2
z 3 t
I need to pass each line as paramater to some program:
$ program "x 1" "y 2" "z 3 t"
I know how to do it with two commands:
$ readarray -t a < file
$ program "${a[#]}"
How can i do it with one command? Something like that:
$ program ??? file ???
The (default) options of your readarray command indicate that your file items are separated by newlines.
So in order to achieve what you want in one command, you can take advantage of the special IFS variable to use word splitting w.r.t. newlines (see e.g. this doc) and call your program with a non-quoted command substitution:
IFS=$'\n'; program $(cat file)
As suggested by #CharlesDuffy:
you may want to disable globbing by running beforehand set -f, and if you want to keep these modifications local, you can enclose the whole in a subshell:
( set -f; IFS=$'\n'; program $(cat file) )
to avoid the performance penalty of the parens and of the /bin/cat process, you can write instead:
( set -f; IFS=$'\n'; exec program $(<file) )
where $(<file) is a Bash equivalent to to $(cat file) (faster as it doesn't require forking /bin/cat), and exec consumes the subshell created by the parens.
However, note that the exec trick won't work and should be removed if program is not a real program in the PATH (that is, you'll get exec: program: not found if program is just a function defined in your script).
Passing a set of params should be more organized :
In this example case I'm looking for a file containing chk_disk_issue=something etc.. so I set the values by reading a config file which I pass in as a param.
# -- read specific variables from the config file (if found) --
if [ -f "${file}" ] ;then
while IFS= read -r line ;do
if ! [[ $line = *"#"* ]]; then
var="$(echo $line | cut -d'=' -f1)"
case "$var" in
chk_disk_issue)
chk_disk_issue="$(echo $line | tr -d '[:space:]' | cut -d'=' -f2 | sed 's/[^0-9]*//g')"
;;
chk_mem_issue)
chk_mem_issue="$(echo $line | tr -d '[:space:]' | cut -d'=' -f2 | sed 's/[^0-9]*//g')"
;;
chk_cpu_issue)
chk_cpu_issue="$(echo $line | tr -d '[:space:]' | cut -d'=' -f2 | sed 's/[^0-9]*//g')"
;;
esac
fi
done < "${file}"
fi
if these are not params then find a way for your script to read them as data inside of the script and pass in the file name.
In a shell script I would like to replace all underscore characters with a blank space in a function that use tr but a receive an error because I don't know of to pass a space in a variable to tr
function sanitizeDirName() {
local name=$1
local f=$2
local r=$3
echo ${name##*/} | grep -E -o $re | tr $f $r
}
sanitizeDirName "~/test_1" "_" " "
Thank you
You need to quote your variables, since they are populated from user input and could have whitespaces or metacharacters (as tripleee pointed out in the comments):
echo ${name##*/} | grep -E -o "$re" | tr "$f" "$r"
If you want to remove _ then you might want to use the -d flag
echo ${name##*/} | grep -E -o $re | tr -d $f
In the goal to create a file from a one line (bash) command, the goal is to output the contents of any text file - in this example a bash script - and wrap each line inside a command that is able to output that same line when pasted in a Terminal window.
Example source input file:
Line 1
Line 2
Line 3
Example desired output:
echo 'Line 1';echo 'Line 2';echo 'Line 3';
Note: whether printf, echo or another command is used to create the output, doesn't matter as long as the source is human readable.
One hurdle were the single quotes, that would not be recreated. Therefore use the form $'string', which are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
Another requirement is to re-create tab characters from the old file in the new file. Therefore the wish is to replace <\tab> characters with \t.
Our tries to do this with sed or tr fail. How to replace tabs with their escape \t counterpart and still being able to output lines with original quotes?
Input file /Library/Scripts/BootRepairMount.sh contains:
$ cat /Library/Scripts/BootRepairMount.sh
#!/bin/bash
sleep 18
for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')
do
if [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then
echo "$OUTPUT is not mounted, repair and mount"
diskutil repairVolume $OUTPUT
diskutil mount $OUTPUT
fi
done
The best shell one line command we could create is:
$ oldifs=$IFS;printf '\n';printf '{';while IFS= read -r p;do [[ "$p" == *"'"* ]] && echo -n "echo $'$p';" || echo -n "echo '$p';"; done < /Library/Scripts/BootRepairMount.sh | tr '\t' '\134\164';printf '}';printf '\n\n';IFS=$oldifs
Which returns this faulty output:
{echo '#!/bin/bash';echo 'sleep 18';echo $'for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')';echo 'do';echo '\if [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then';echo '\\echo "$OUTPUT is not mounted, repair and mount"';echo '\\diskutil repairVolume $OUTPUT';echo '\\diskutil mount $OUTPUT';echo '\fi';echo 'done';}
Desired output is:
{echo '#!/bin/bash';echo 'sleep 18';echo $'for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')';echo 'do';echo '\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then';echo '\t\techo "$OUTPUT is not mounted, repair and mount"';echo '\t\tdiskutil repairVolume $OUTPUT';echo '\t\tdiskutil mount $OUTPUT';echo '\tfi';echo 'done';}
Bash one line command version 2
$ oldifs=$IFS;printf '\n';printf '{';while IFS= read -r p;do [[ "$p" == *"'"* ]] && printf 'printf $'\''%q'\'';' "$p" || printf 'printf '\''%q'\'';' "$p"; done < /Library/Scripts/BootRepairMount.sh;printf '}';printf '\n\n';IFS=$oldifs
returns output that is heavy escaped:
{printf '\#\!/bin/bash';printf 'sleep\ 18';printf $'for\ OUTPUT\ in\ \$\(diskutil\ list\ \|\ grep\ \':\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Apple_HFS\'\ \|\ awk\ \'\{\ print\ \$NF\ \}\'\)';printf 'do';printf '$'\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then'';printf '$'\t\techo "$OUTPUT is not mounted, repair and mount"'';printf '$'\t\tdiskutil repairVolume $OUTPUT'';printf '$'\t\tdiskutil mount $OUTPUT'';printf '$'\tfi'';printf 'done';}
that never gets unescaped back to its original values in Mac OS X 10.7.5.
printf '\#\!/bin/bash';
outputs:
\#\!/bin/bash
As well as:
echo -e '\#\!/bin/bash'
does output the unescaped value
\#\!/bin/bash
-e is not a valid command switch for the Mac OS X 10.7.5 echo command, according to its man page.
bash's builtin command printf has %q format code that handles this:
printf '\n{ '; while IFS= read -r p; do printf "echo %q; " "$p"; done < /Library/Scripts/BootRepairMount.sh; printf '}\n\n'
Unfortunately, it doesn't always choose quoting/escaping modes that're easy to read. Specifically, it tends to prefer escaping individual metacharacters (e.g. spaces) rather than enclosing them in quotes:
{ echo \#\!/bin/bash; echo sleep\ 18; echo for\ OUTPUT\ in\ \$(diskutil\ list\ \|\ grep\ \':\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Apple_HFS\'\ \|\ awk\ \'{\ print\ \$NF\ }\'); echo do; echo $'\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then'; echo $'\t\techo "$OUTPUT is not mounted, repair and mount"'; echo $'\t\tdiskutil repairVolume $OUTPUT'; echo $'\t\tdiskutil mount $OUTPUT'; echo $'\tfi'; echo done; }
If I understand right you want paste one long line to the Terminal.app and want get the "source code" of original script. So, need a script what will generate the one-line script.
Maybe a bit unusual solution, but it is easy and simple.
here is the test script called test.sh (instead of your BootReapirMount.sh)
for i in {1..10}
do
date
done
Here is the generator script mkecho.sh
#!/bin/bash
[[ ! -f "$1" ]] && echo "Need filename" && exit 1
asc=$(gzip < "$1" | base64)
echo "base64 -D <<<'$asc'| gzip -d"
Now, run:
bash mkecho.sh test.sh
you will get the next:
base64 -D <<<'H4sIAASwqFEAA0vLL1LIVMjMU6g21NMzNKjlSsnn4kxJLEkFMvJSuQBZFmY0HwAAAA=='| gzip -d
If you copy and paste the above into the terminal, it will will display the original test.sh
Variant2
If you want directly execute the script, you should modify the mkecho.sh to the next mkeval.sh
#!/bin/bash
[[ ! -f "$1" ]] && echo "Need filename" && exit 1
asc=$(gzip < "$1" | base64)
echo -n 'eval "$(base64 -D <<<"'
echo -n $asc
echo -n '" | gzip -d)"'
echo
When run
bash mkeval.sh test.sh
will get
eval "$(base64 -D <<<"H4sIAASwqFEAA0vLL1LIVMjMU6g21NMzNKjlSsnn4kxJLEkFMvJSuQBZFmY0HwAAAA==" | gzip -d)"
and finally when you copy and paste it into the terminal, you run the test.sh and will get:
Fri May 31 16:25:08 CEST 2013
... 8 lined deleted...
Fri May 31 16:25:08 CEST 2013
Warning: because the script is NOT TESTED for every possible conditions, nor for redirects and so on - I really don't recommending using the eval verision.
sed 's/\\t/\\/g'
$ echo 'ffsd \tif [[ -z $' | sed 's/\\t/\\/g'
ffsd \if [[ -z $