This question already has answers here:
Using dollar sign in sed for both variable replacement and character
(3 answers)
Closed 3 years ago.
In sed, we use double quotes to read environment variable indicated by a dollar sign, what should i do if I want to use the dollar sign for end-of-line pattern at the same time?
Here is example of my command (not working),
cat inputfile | xargs -n4 sh -c 'sed -ne "$1,$2p" bigfile|sed -e "$s/$/\n+--$3-----+/" > $0.outputfile'
I pipe the content of a input file (containing the output filename, target line number (start and end), and some tags to add to each file) to xargs which break the big file into smaller files. Example of input file:
dataA
59
88
sometagA
dataB
91
236
sometagB
....
In the second sed command: sed -e "$s/$/\n+--$3-----+/", $3 is environment variable from xargs, and i want to use the other $s and $ to target end-of-file and end-of-line pattern respectively, as i intent to insert some tags to each output file. I cannot use single and double quote at the same time as the outter xargs already used single quote.
Use backslash to escape a dollar sign from the shell.
xargs -n4 sh -c 'sed -ne "$1,$2p" bigfile |
sed -e "\$s/\$/\n+--$3-----+/" > "$0".outputfile' <inputfile
You only pass four arguments and they are numbered from zero, so I guess you mean $3, not $4.
Notice also how to avoid the useless cat.
The two sed scripts could probably be merged into something like
sed -n "$1,$2!d;$2{;s/\$/+--$3-----+/p;q;}p"
escape it from bash shell to have an end anchor
sed -ne "$1,$2p" bigfile|sed -e "$s/\$/\n+--$3-----+/"
escape it once more from sed to have a literal dollar sign e.g;
sed -ne "$1,$2p" bigfile|sed -e "$s/\$/\n+--$3-----+/| sed 's/.*/& pays \\$99/'
Related
I have into a text file the following line :
\[Omega]BD=100;
I would like to replace with gsed the value 100 by a shell variable (zsh shell), here 600 :
I tried :
$ i=600
$ gsed 's/\[Omega]BD=.*/\[Omega]BD=\'\\"$i"\\';/' text_to_modify.txt | grep 600
but it returns me :
\[Omega]BD=\600; and not \[Omega]BD=600;
The is an additional backslash that I don't want, I wonder how could I remove this backslash. I would like to keep the two single quotes of gsed 's/.../.../'
Using sed;
i=600
$ sed "/\[Omega]/s/[[:digit:]]\+/$i/" input_file
\[Omega]BD=600;
You may use this sed command:
i=600
sed -E "s/(\\\\\[Omega]BD=).*/\1$i;/" file
\[Omega]BD=600;
We require additional escaping i.e. \\\\ to match a single \ because we are using double quotes around full sed command.
Or we can avoid you can use this combination of single and double quotes to avoid extra escaping:
sed -E 's/(\\\[Omega]BD=).*/\1'"$i;/" file
I wish to grep a line staring which start with Alphabet followed by 5 numbers.
MY approach is
eg
'A12345'
'A123456'
output should be only line starting with Alphabet followed by 5 numbers
ie 'A12345'
My approach : it doesn't work
grep -E '[A-Z]''[0-9]{5}'
# PCRE allows \xHH escapes
$ grep -P '^\x27[A-Z][0-9]{5}\x27' ip.txt
'A12345'
# double quotes can also be used here since there's no clash
$ grep -E "^'[A-Z][0-9]{5}'" ip.txt
'A12345'
# this is same as ^' followed by [A-Z][0-9]{5} and then another '
$ grep -E "^'"'[A-Z][0-9]{5}'"'" ip.txt
'A12345'
Here's an example where double quotes can be problematic. See https://mywiki.wooledge.org/Quotes and Difference between single and double quotes in Bash for more details.
$ echo '1a$(z)b' | grep -E "a$(z)b"
z: command not found
Cannot seem to find an answer to this one online...
I have a string variable (externally sourced) with new lines "\n" encoded as strings.
I want to replace those strings with actual new line carriage returns. The code below can achieve this...
echo $EXT_DESCR | sed 's/\\n/\n/g'
But when I try to store the result of this into it's own variable, it converts them back to strings
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
How can this be achieved, or what I'm I doing wrong?
Here's my code I've been testing to try get the right results
EXT_DESCR="This is a text\nWith a new line"
echo $EXT_DESCR | sed 's/\\n/\n/g'
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
echo ""
echo "$NEW_DESCR"
No need for sed, using parameter expansion:
$ foo='1\n2\n3'; echo "${foo//'\n'/$'\n'}"
1
2
3
With bash 4.4 or newer, you can use the E operator in ${parameter#operator}:
$ foo='1\n2\n3'; echo "${foo#E}"
1
2
3
Other answers contain alternative solutions. (I especially like the parameter expansion one.)
Here's what's wrong with your attempt:
In
echo $EXT_DESCR | sed 's/\\n/\n/g'
the sed command is in single quotes, so sed gets s/\\n/\n/g as is.
In
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
the whole command is in backticks, so a round of backslash processing is applied. That leads to sed getting the code s/\n/\n/g, which does nothing.
A possible fix for this code:
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\\\n/\\n/g'`
By doubling up the backslashes, we end up with the right command in sed.
Or (easier):
NEW_DESCR=$(echo $EXT_DESCR | sed 's/\\n/\n/g')
Instead of backticks use $( ), which has less esoteric escaping rules.
Note: Don't use ALL_UPPERCASE for your shell variables. UPPERCASE is (informally) reserved for system variables such as HOME and special built-in variables such as IFS or RANDOM.
Depending on what exactly you need it for:
echo -e $EXT_DESCR
might be all you need.
From echo man page:
-e
enable interpretation of backslash escapes
This printf would do the job by interpreting all escaped constructs:
printf -v NEW_DESCR "%b" "$EXT_DESCR"
-v option will store output in a variable so no need to use command substitution here.
Problem with your approach is use of old back-ticks. You could do:
NEW_DESCR=$(echo "$EXT_DESCR" | sed 's/\\n/\n/g')
Assuming you're using gnu sed as BSD sed won't work with this approach.
This question already has answers here:
sed substitution with Bash variables
(6 answers)
Closed 5 years ago.
I have a variable in a config, which I would like to replace with a value.
ROOT="rROOT"
I would like to replace that with
ROOT="$root"
So with the value of $root (Important are the quotation marks).
So far I have tried it that way
sed -i s/'ROOT=rROOT'/'ROOT="$root"'/g $directory/settings.conf
The result is that
ROOT="$root"
But this is stored in the variable $root (It is always a path)
root: /
How can I replace rROOT with the value of $root?
Sed Version: (GNU sed) 4.2.2
Ok, don't like to ruin my scripts for testing:
cat tageslog.sh | sed "s/alt=185094/alt=\${root}/g"
Use double quotes, but mask the dollar sign, so it doesn't get lost and root interpreted while calling sed.
Use ${root} instead of "$root".
sed "s/ROOT=.rRoot./ROOT=\${root}/g" $directory/settings.conf
if this works, use the -i switch:
sed -i "s/ROOT=.rRoot./ROOT=\${root}/g" $directory/settings.conf
You have different problems:
Testing with echo
I think you tested your command with
echo ROOT="rROOT" | sed s/'ROOT=rROOT'/'ROOT="$root"'/g
The double quotes won't appear in the output of the echo, so you will end up with a command working for ROOT=rROOT.
When the inputfile has qouble quotes, you do not have to insert them.
Testing with the double quotes is possible with
echo 'ROOT="rROOT"' | sed s/'ROOT=rROOT'/'ROOT="$root"'/g
Place the double quotes outside the single quotes;
You can test this with echo:
echo 'Showing "$PWD" in double quotes is "'$PWD'"'
echo 'With additional spaces the last part is " '$PWD' " '
Root vaiable has slashes that will confuse sed.
The root variable is replaced before sed tries to understand the command.
You can use another character, like #, in the sed command:
sed 's#before#after#'
When your input has double quotes:
echo 'ROOT="rROOT"' | sed 's#ROOT="rROOT"#ROOT="'$root'"#g'
# or by remembering strings
echo 'ROOT="rROOT"' | sed -r 's#(ROOT=")rROOT(")#\1'$root'\2#g'
Input without double quotes
echo 'ROOT=rROOT' | sed 's#ROOT=rROOT#ROOT="'$root'"#g'
# or by remembering strings
echo 'ROOT=rROOT' | sed -r 's#(ROOT=)rROOT#\1"'$root'"#g'
This question already has answers here:
Sed Insert Multiple Lines
(3 answers)
Closed 1 year ago.
I have a multiline variable that I captured from STDOUT.
I want to insert an echo command using this multiline variable to line 15 in another script (target).
#!/bin/bash
TEST=`cat foo`
echo "$TEST"
sed -i "15i echo \"$TEST\" > someotherfile" target
Contents of foo :
apples
oranges
bananas
carrots
I thought the sed command read in line feeds, which I confirmed my foo has:
user#test$ cat foo | tr -cd '\n' | wc -c
4
When I run my test.sh script, I see what's in $TEST, but am getting an error for the sed command:
user#test$ ./test.sh
apples
oranges
bananas
carrots
sed: -e expression #1, char 18: unknown command: `o'
What am I doing wrong?
Thanks in advance.
GNU sed is assumed, as implied by the syntax used in the question.
#!/bin/bash
# Read contents of file 'foo' into shell variable $test.
test=$(<foo)
# \-escape the newlines in $test for use in Sed.
testEscapedForSed=${test//$'\n'/\\$'\n'}
sed -i "15i echo \"$testEscapedForSed\" > someotherfile" target
Your problem was that passing multi-line strings to sed functions such as i (insert) requires the newlines embedded in those strings to be \-escaped, so that sed knows where the string ends and additional commands, if any, start.
A (nonstandard) parameter expansion is used to replace all newlines in $test with themselves prefixed by \, using ANSI C-quoted string $'\n' to generate actual newline chars.
Also note:
I've renamed TEST to test, because all-uppercase shell-variable names should be avoided.
I've used modern command-substitution syntax $(..) in lieu of legacy syntax `...`.
$(<foo) is a slightly more efficient - although nonstandard - way of reading the content of a file at once.
Try:
Solution1:
awk 'NR==15{print;system("cat foo");next} 1' Input_file
No need to get the complete file into a variable, we could simply print it whichever line of Input_file you want to print it.
Solution2:
line=15; sed -e "${line}r foo" target
Or (in script mode)
cat script.ksh
line=15;
sed -e "${line}r foo" target
Where you could change the number of line where you want to insert the lines from another file.
The i command in sed inserts the lines of text that end with a newline, up until a line that doesn't end with a backslash. The a and c commands are similar. Classic sed doesn't like the first line to appear on the same line as the i command; GNU sed isn't as fussy.
If you were writing the command manually, you'd need to write:
15i\
echo "apples\
oranges\
bananas\
carrots" > someotherfile
At issue now is "how do you want to create this given the file foo contains the list of names?". Sometimes, using sed to generate the sed script is useful. However, it can also be intricate if you need to get backslashes at the ends of lines which are subject to an i (or a or c) command, and it is simpler to circumvent the problem.
{
echo "15i\\"
sed -e '1s/^/echo "/' -e 's/$/\\/' -e '$s/\\$/" > someotherfile/' foo
} | sed -f /dev/stdin target
GNU sed can read its script from standard input using -f -; BSD (macOS) sed doesn't like that, but you can use -f /dev/stdin instead (which also works with GNU sed), at least on systems where there is a /dev/stdin.
Interesting issue.
As already mentioned the whole story for sed to be able to insert multiline text in another file is that this new multiline text must have actually literral \n for line breaks.
So we can use sed to convert real new line chars to literal \n:
$ a=$(tr '\n' '\\' <file3 |sed 's#[\]$##' |sed "s#[\]#\0n#g")
#Alternative: a=$(sed "s#[\]#\0n#g" <(sed 's#[\]$##' <(tr '\n' '\\' <file3)))
$ echo "$a"
apples\noranges\nbananas\ncarrots
How this translation works:
* First we replace all new lines with a single backslash using tr
* Then we remove the backslash from the end of the string
* Then we replace all other backslashes with backaslash and n char.
Since now variable $a contains literal \n between lines, sed will translate them back to actuall new lines:
$ cat file4
Line1
line2
line3
$ sed "2i $a" file4
Line1
apples
oranges
bananas
carrots
line2
line3
Result:
Mutliline replacement can be done with two commands:
$ a=$(tr '\n' '\\' <file3 |sed 's#[\]$##' |sed "s#[\]#\0n#g")
$ sed "2i $a" file4
sed 2i means insert a text before line2. 2a can be used in order to insert something after line2.
Remark:
According to this post which seems to be a duplicate, translation of new lines to literal \n seems that can be done with just :
a=$(echo ${a} | tr '\n' "\\n")
But this method never worked in my system.
Remark2:
The sed operation sed "2i $a" = insert variable $a before line 2 , can be also expressed as sed "1 s/.*/\0\n$a/" = replace all chars of first line with the same chars \0 plus a new line \n plus the contents of variable $a => insert $a after line1 = insert $a before line2.