This question already has answers here:
sed edit file in place
(15 answers)
Closed 5 years ago.
Let's say we want to replace the letter "a" with its Morse code equivalent from a text.txt file to a hidden .out.txt file using sed, with the contents of the original file being "this is a test".
#!/bin/bash
touch .out.txt
cat text.txt > .out.txt
sed 's/a/.-/g' .out.txt
The output, as expected, is "this is .- test".
However, let's say we want to do multiple instances of sed, without constantly printing the result of each instance of sed using the -n parameter. In this case:
#!/bin/bash
touch .out.txt
cat text.txt > .out.txt
sed -n 's/a/.-/g' .out.txt
sed -n 's/t/-/g' .out.txt
cat .out.txt
However, in this case, the output of cat is the same as the contents of text.txt, "this is a test".
Is there a possible substitute for -n, or in general a way to prevent sed from printing anything with our wanted result?
Check your seds sed --help for the syntax of -ioption.
This is necessary, because the syntax is different depending on which sed you are using, i.e. I cannot simply tell you.
Then use it accordingly to have the effects of sed take place in-place on the file.
Related
This question already has answers here:
Is there a cleaner way of getting the last N characters of every line?
(4 answers)
Closed 2 years ago.
I have a text file with thousands of lines. The last 7 characters on each line are a mix of letters and numbers (eg AAP8945 or GGR6645). I want to save these in a separate file.
Excuse the noob question, but I can't work it out.
With GNU grep
Assuming you have GNU grep:
grep -o -E '.{7}$' input > output
The -o option means 'output only what matches' (rather than the whole line). This is the key feature which makes it possible to use grep for the job. Without support for -o (or an equivalent option), grep is the wrong tool for the job.
The -E option is for extended regular expressions, and it means that the . (any character) is matched 7 times and then matches the end of line.
Without GNU grep
If you don't have GNU grep (or a compatible grep with the -o option or equivalent), then you can use sed instead (GNU or any other variant):
sed -e 's/.*\(.\{7\}\)$/\1/' input > output
This matches the start of the line (.*) and captures the last 7 characters (\(…\)) of the line; it replaces the whole with the captured part, and prints the result. If your variant of sed has extended regular expressions (usually -E or sometimes -r), then:
sed -E -e 's/.*(.{7})$/\1/' input > output
The difference is in the number of backslashes needed.
Both of those will print any short lines in their entirety. If those should be omitted, use:
sed -n -e 's/.*\(.\{7\}\)$/\1/p' input > output
sed -n -E -e 's/.*(.{7})$/\1/p' input > output
grep -Eo '.{7}$'
Or without grep:
rev input|cut -c -7|rev >output
The double rev is necessary here because I can not specify a position of the text from the right with cut.
This question already has answers here:
Why does reading and writing to the same file in a pipeline produce unreliable results?
(2 answers)
Closed 3 years ago.
echo "hello" | tee test.txt
cat test.txt
sudo sed -e "s|abc|def|g" test.txt | tee test.txt
cat test.txt
Output:
The output of 2nd command and last command are different, where as the command is same.
Question:
The following line in above script gives an output, but why it is not redirected to output file?
sudo sed -e "s|abc|def|g" test.txt
sudo sed -e "s|abc|def|g" test.txt | tee test.txt
Reading from and writing to test.txt in the same command line is error-prone. sed is trying to read from the file at the same time that tee wants to truncate it and write to it.
You can use sed -i to modify a file in place. There's no need for tee. (There's also no need for sudo. You made the file, no reason to ask for root access to read it.)
sed -e "s|abc|def|g" -i test.txt
You shouldn't use the same file for both input and output.
tee test.txt is emptying the output file when it starts up. If this happens before sed reads the file, sed will see an empty file. Since you're running sed through sudo, it's likely to take longer to start up, so this is very likely.
This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 6 years ago.
I'm creating a variable from an array which build up multiple -e clauses for a sed command.
The resulting variable is something like:
sedArgs="-e 's/search1/replace1/g' -e 's/search2/replace2/g' -e 's/search3/replace3/g'"
But when I try to call sed with this as the argument I get the error sed: -e expression #1, char 1: unknown command: ''
I've tried to call sed the following ways:
cat $myFile | sed $sedArgs
cat $myFile | sed ${sedArgs}
cat $myFile | sed `echo $sedArgs`
cat $myFile | sed "$sedArgs"
cat $myFile | sed `echo "$sedArgs"`
and all give the same error.
UPDATE - Duplicate question
As has been identified, this is a 'quotes expansion' issue - I thought it was something sed specific, but the duplicate question that has been identified put me on the right track.
I managed to resolve the issue by creating the sedArgs string as:
sedArgs="-e s/search1/replace1/g -e s/search2/replace2/g -e s/search3/replace3/g"
and calling it with:
cat $myFile | sed $sedArgs
which works perfectly.
Then I took the advice of tripleee and kicked the useless cat out!
sed $sedArgs $myFile
also works perfectly.
Use BASH arrays instead of simple string:
# sed arguments in an array
sedArgs=(-e 's/search1/replace1/g' -e 's/search2/replace2/g' -e 's/search3/replace3/g')
# then use it as
sed "${sedArgs[#]}" file
Here is no sane way to do that, but you can pass the script as a single string.
sedArgs='s/search1/replace1/g
s/search2/replace2/g
s/search3/replace3/g'
: then
sed "$sedArgs" "$myFile"
The single-quoted string spans multiple lines; this is scary when you first see it, but perfectly normal shell script. Notice also how the cat is useless as ever, and how the file name needs to be quoted, too.
I have a large number of words in a text file to replace.
This script is working up until the sed command where I get:
sed: 1: "*.js": invalid command code *
PS... Bash isn't one of my strong points - this doesn't need to be pretty or efficient
cd '/Users/xxxxxx/Sites/xxxxxx'
echo `pwd`;
for line in `cat myFile.txt`
do
export IFS=":"
i=0
list=()
for word in $line; do
list[$i]=$word
i=$[i+1]
done
echo ${list[0]}
echo ${list[1]}
sed -i "s/{$list[0]}/{$list[1]}/g" *.js
done
You're running BSD sed (under OS X), therefore the -i flag requires an argument specifying what you want the suffix to be.
Also, no files match the glob *.js.
This looks like a simple typo:
sed -i "s/{$list[0]}/{$list[1]}/g" *.js
Should be:
sed -i "s/${list[0]}/${list[1]}/g" *.js
(just like the echo lines above)
So myFile.txt contains a list of from:to substitutions, and you are looping over each of those. Why don't you create a sed script from this file instead?
cd '/Users/xxxxxx/Sites/xxxxxx'
sed -e 's/^/s:/' -e 's/$/:/' myFile.txt |
# Output from first sed script is a sed script!
# It contains substitutions like this:
# s:from:to:
# s:other:substitute:
sed -f - -i~ *.js
Your sed might not like the -f - which means sed should read its script from standard input. If that is the case, perhaps you can create a temporary script like this instead;
sed -e 's/^/s:/' -e 's/$/:/' myFile.txt >script.sed
sed -f script.sed -i~ *.js
Another approach, if you don't feel very confident with sed and think you are going to forget in a week what the meaning of that voodoo symbols is, could be using IFS in a more efficient way:
IFS=":"
cat myFile.txt | while read PATTERN REPLACEMENT # You feed the while loop with stdout lines and read fields separated by ":"
do
sed -i "s/${PATTERN}/${REPLACEMENT}/g"
done
The only pitfall I can see (it may be more) is that if whether PATTERN or REPLACEMENT contain a slash (/) they are going to destroy your sed expression.
You can change the sed separator with a non-printable character and you should be safe.
Anyway, if you know whats on your myFile.txt you can just use any.
This question already has answers here:
Replace a word with multiple lines using sed?
(11 answers)
Closed 4 years ago.
I am trying to replace a word with a text which spans multiple lines. I know that I can simply use the newline character \n to solve this problem, but I want to keep the string "clean" of any unwanted formatting.
The below example obviously does not work:
read -r -d '' TEST <<EOI
a
b
c
EOI
sed -e "s/TOREPLACE/${TEST}/" file.txt
Any ideas of how to achieve this WITHOUT modifying the part which starts with read and ends with EOI?
Given that you're using Bash, you can use it to substitute \n for the newlines:
sed -e "s/TOREPLACE/${TEST//$'\n'/\\n}/" file.txt
To be properly robust, you'll want to escape /, & and \, too:
TEST="${TEST//\\/\\\\}"
TEST="${TEST//\//\\/}"
TEST="${TEST//&/\\&}"
TEST="${TEST//$'\n'/\\n}"
sed -e "s/TOREPLACE/$TEST/" file.txt
If your match is for a whole line and you're using GNU sed, then it might be easier to use its r command instead:
sed -e $'/TOREPLACE/{;z;r/dev/stdin\n}' file.txt <<<"$TEST"
You can just write the script as follows:
sed -e 's/TOREPLACE/a\
b\
c\
/g' file.txt
A little cryptic, but it works. Note also that the file won't be modified in place unless you use the -i option.
tricky... but my solution would be :-
read -r -d '' TEST <<EOI
a
b
c
EOI
sed -e "s/TOREPLACE/`echo "$TEST"|awk '{printf("%s\\\\n", $0);}'|sed -e 's/\\\n$//'`/g" file.txt
Important:
Make sure you use the correct backticks, single quotes, double quotes and spaces
else it will not work.
An interesting question..
This may get you closer to a solution for your use case.
read -r -d '' TEST <<EOI
a\\
b\\
c
EOI
echo TOREPLACE | sed -e "s/TOREPLACE/${TEST}/"
a
b
c
I hope this helps.