I would like to do that:
i="1"; echo -e '#!/usr/bin/env bash\nmyprogram -i "input_${i}.txt"'
and pipe it to a job scheduler.
However, this doesn't replace the variable i by its value. Instead, I obtain this:
#!/usr/bin/env bash
myprogram -i "input_${i}.txt"
I played a bit with option -e of echo and with single-/double-quote but could not make it work. For instance, I get this:
i="1"; echo -e "#!/usr/bin/env bash\nmyprogram -i \"input_${i}.txt\""
-bash: !/usr/bin/env: event not found
My bash version is 4.1.2.
Try this:
i="1"; echo -e '#!/usr/bin/env bash\nmyprogram -i '"\"input_${i}.txt\""
You can echo single- and double-quoted strings at the same time.
Try also escaping the exclamation mark:
\! should be okay, and will not be read as an "event" by bash.
i="1"; echo -e '#!/usr/bin/env bash\nmyprogram -i "input_'"${i}"'.txt"'
Basically, use single quotes until you need to interpolate, then close the single quotes, open the double quotes, add the interpolation, close the double quotes, reopen single quotes, and finish the string. In the shell, quotation marks don't delimit a word; they just change the interpretation of the part of a word falling between them.
Related
I'm building a shell script with echo. I have something like:
echo "sed -i \"\\|charlie\.url\\s*=\\s*.*|c\\charlie.url = ${CHARLIE_URL}\" foo.conf" >> bar.sh
i.e. replace the line in foo.conf containing the current charlie.url (not necesarily at the begining, 'cause the line could be commented) for a new line with a new url.
I would expect the output to bar.sh to be
sed -i "\|charlie\.url\s*=\s*.*|c\charlie.url = ${CHARLIE_URL}" foo.conf
Nevertheless, the c\\charlie is interpreted as c \c harlie, instead of
c\ charlie, which generates the following output:
sed -i "\|charlie\.url\s*=\s*.*|c
I have found that I could prevent this by using single instead of doubles quotes, but in that case ${CHARLIE_URL} (which I do need to expand) does not get expanded.
How should my echo argument look like?
I'm using dash (#!/bin/sh under Ubuntu), but I could also use bash or zsh.
Instead of echo, you can try cat :
cat << EOF >> bar.sh
sed -i "\|charlie\.url\s*=\s*.*|c\charlie.url = ${CHARLIE_URL}" foo.conf
EOF
#!/bin/bash
# bash will expand content within "" and stuff like $URL gets expanded
# also double-backslash reduces to a single; bash wouldn't expand stuff within ''
URL="https..."
echo double: "c\\charlie.url, URL $URL"
echo single: 'c\\charlie.url, URL $URL'
# if you need to output a \\
echo "\\\\"
The command line is using your first slash to quote your second one. If CHARLIE_URL=foo, your echo is actually outputting
sed -i "\|charlie\.url\s*=\s*.*|c\charlie.url = foo" foo.conf
Try using single quotes, but close/open them around the variable.
echo 'sed -i "\\|charlie\.url\\s*=\\s*.*|c\\charlie.url = '"${CHARLIE_URL}"'" foo.conf'
This produces
sed -i "\\|charlie\.url\\s*=\\s*.*|c\\charlie.url = foo" foo.conf
You could also, as mentioned, quote the slash that quotes the slash, and then quote the slash being quoted by the other slash, so that the subsequent iterations boil them down to what you want... but that's generally a mess, and leads to leaning-toothpick syndrome.
Someone made a good here-doc suggestion already, so I won't repeat that, but you can also use a "here string" which I generally prefer.
cat <<< "sed -i '\|charlie\.url\s*=\s*.*|c\charlie.url = ${CHARLIE_URL}' foo.conf"
I switched the double-quotes in your script for singles, assuming what you want is to output the literal value of $CHARLIE_URL into your script. If you want the script to use the variable, with whatever value is assigned at runtime, then quote that too -
cat <<< "sed -i '\|charlie\.url\s*=\s*.*|c\charlie.url = \${CHARLIE_URL}' foo.conf"
I am trying to find and replace a specific text content using the sed command and to run it via a shell script.
Below is the sample script that I am using:
fp=/asd/filename.txt
fd="sed -i -E 's ($2).* $2:$3 g' ${fp}"
eval $fd
and executing the same by passing the arguments:
./test.sh update asd asdfgh
But if the argument string contains $ , it breaks the commands and it is replacing with wrong values, like
./test.sh update asd $apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1.
How can I make sure that the values inside the variables are not expanded because of the $?
Updated
sh file test.sh
set -xv
fp="/asd/filename.txt"
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
text file filename.txt
hello:world
Outputs
1)
./test.sh update hello WORLD
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
++ sed -iE 's/(hello).*/hello:WORLD/g' /asd/filename.txt
2)
./test.sh update hello '$apr1$hosgaxyv$D0KXp5dCyZ2BUYCS9BmHu1'
sed -iE "s/(${2//'$'/'\$'}).*/${2//'$'/'\$'}:${3//'$'/'\$'}/g" "$fp"
++ sed -iE 's/(hello).*/hello:'\''$'\''apr1'\''$'\''hosgaxyv'\''$'\''D0KXp5dCyZ2BUYCS9BmHu1/g' /asd/filename.txt
In both the case , its not replacing the content
You don't need eval here at all:
fp=/asd/filename.txt
sed -i -E "s/(${2//'$'/'\$'}).*/\1:${3//'$'/'\$'}/g" "$fp"
The whole sed command is in double quotes so variables can expand.
I've replaced the blank as the s separator with / (doesn't really matter in the example).
I've used \1 to reference the first capture group instead of repeating the variable in the substitution.
Most importantly, I've used ${2//'$'/'\$'} instead of $2 (and similar for $3). This escapes every $ sign as \$; this is required because of the double quoting, or the $ get eaten by the shell before sed gets to see them.
When you call your script, you must escape any $ in the input, or the shell tries to expand them as variable names:
./test.sh update asd '$apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1.'
Put the command-line arguments that are filenames in single quotes:
./test.sh update 'asd' '$apr1$HnIF6bOt$9m3NzAwr.aG1Yp.t.bpIS1'
must protect all the script arguments with quotes if having space and special shell char, and escape it if it's a dollar $, and -Ei instead of -iE even better drop it first for test, may add it later if being really sure
I admit i won't understant your regex so let's just get in the gist of solution, no need eval;
fp=/asd/filename.txt
sed -Ei "s/($2).*/$2:$3/g" $fp
./test.sh update asd '\$apr1\$HnIF6bOt\$9m3NzAwr.aG1Yp.t.bpIS1.'
Someone with better experience then me regarding bashscript that could take a look at this code and explain why no string replacements is taking place. If I run the same command (text input instead of variables) in the terminal, then it works.
#!/bin/bash
echo "important to escape every \"/\" character"
read -p "Specify the old string you want to replace? (from) " FROM
read -p "Specify the new string you want to use instead? (to) " TO
cp ../backup/mysql/dump.sql ../backup/mysql/dump.sql.backup.$(date +"%Y-%m-%d-%H-%M-%S") \
&& sed -i 's/$FROM/$TO/g' ../backup/mysql/dump.sql
Use double quotes for variables expansion instead of single quotes: "s/$FROM/$TO/g"
You can even have the cp implicitly within the sed command itself using -i(inplace) option.
sed -i.backup.$(date +"%Y-%m-%d-%H-%M-%S") "s/$FROM/$TO/g" ../backup/mysql/dump.sql
This will create the backup file as well.
I'd like to employ perl one-liner calculate resulting filenames using regexp substitutions. When doing dry run and simply printing the results it gives me the desired result (no quotes there yet):
for i in *_\ *; do echo "${i}" $(perl -ne 'print s/(?<![_ ])_ /-/gr' <<< "${i}"); done
but when changed to mv it breaks:
for i in *_\ *; do mv "${i}" $(perl -ne 'print s/(?<![_ ])_ /-/gr' <<< "${i}"); done
mv: target ‘9781430249146.pdf’ is not a directory
apparently perl's output is reinterpreted and white spaces cause problem.
When I put double quotes around it the perl code gets evaluated first by bash, which makes another problem:
for i in *_\ *; do mv "${i}" "$(perl -ne 'print s/(?<![_ ])_ /-/gr' <<< "${i}")"; done
-bash: ![_: event not found
Any way to quote just the output from command substitution (not the command itself)?
If you want your command substitution to be treated as a single word by bash, you should enclose it in double quotes. In order to prevent ! from being interpreted by the shell, you should disable history substitution using one of the following two methods:
set +o histexpand
or
set +H
In the below shell script I try to print A2D(Vlog-Ams-#Cross) with special characters escaped. For example replace ( with \( but sed won't have any effect.
#! /bin/sh
parameter="A2D(Vlog-Ams-#Cross)"
echo $parameter
parameterz=`echo "$parameter" | sed 's/(/\\(/g'`
echo $parameterz
The output is
A2D(Vlog-Ams-#Cross)
A2D(Vlog-Ams-#Cross)
If I do the same on my c-shell terminal, it works fine.
Any ideas?
You use backslashs within a backtick command and that's tricky. If the sed command didn't occur within backticks, it would work correctly. When the shell looks for the closing backtick, however, it removes one level of backslash quoting, so you get
sed 's/(/\(/g'
and that's a no-op. If your shell permits it, use $(...) instead of backticks; in this way you avoid these quoting problems.
In your replacement \\( the first \ escapes the second \. But you must escape the (, too:
$ echo 'A2D(Vlog-Ams-#Cross)' | sed -e 's/(/\\\(/g' -e 's/)/\\\)/g'
A2D\(Vlog-Ams-#Cross\)