So I have been using the following general syntax to be able to remove text on a line within a configuration file after the '=' sign:
sed -i 's|\(.*\txt_to_remove=\).*|\1|g' file.txt
Let's say I have a text file:
$ cat tmp.txt
datee='10'
alter='purple'
hr='green'
new_val='100'
and if I consecutively use the command I provided above to try and delete everything after '=':
sed -i 's|\(.*\datee=\).*|\1|g' tmp.txt
sed -i 's|\(.*\alter=\).*|\1|g' tmp.txt
sed -i 's|\(.*\hr=\).*|\1|g' tmp.txt
sed -i 's|\(.*\new_val=\).*|\1|g' tmp.txt
The results will be:
$ cat tmp.txt
datee=
alter='purple'
hr=
new_val='100'
The question is, why did the commands not work for the 'alter' or 'new_val', and what can be done to make sure that it always works for each individual instance of using the command?
Do them all at once with:
sed -i 's/^\(\(datee\|alter\|hr\|new_val\)=\).*/\1/g' tmp.txt
You have a backslash before the keyword that you want to replace. Backslash is the escape prefix, and \a and \n are escape sequences, so they don't match what's in the file.
Get rid of these backslashes.
sed -i 's|\(.*datee=\).*|\1|g' tmp.txt
sed -i 's|\(.*alter=\).*|\1|g' tmp.txt
sed -i 's|\(.*hr=\).*|\1|g' tmp.txt
sed -i 's|\(.*new_val=\).*|\1|g' tmp.txt
try this sed expression sed 's/=.*//'
I tried and got the desired results
cat tmp.txt | sed 's/=.*//'
datee
alter
hr
new_val
Related
I need to write a shell script to insert a parameter string after every big letter in a file.
$parameter="4"
Example input.txt
AppLe
House
Example output.txt
A4ppL4e
H4ouse
I've tried to use
sed '/[A-Z]/i\$1\'
Can anyone help me?
THX
With GNU/BSD/busybox sed which support the -i option:
param=4
sed -i'' -e 's/\([[:upper:]]\)/\1'"$param"'/g' input.txt
This replaces each uppercase letter inside the captured group \(...\) globally with the first captured group \1 and the value of variable param in-place.
With standard sed you need a temporary file or sponge from the moreutils package:
param=4
sed 's/\([[:upper:]]\)/\1'"$param"'/g' input.txt > temp && mv temp input.txt
param=4
sed 's/\([[:upper:]]\)/\1'"$param"'/g' input.txt | sponge input.txt
Use a file editor like ed to edit files:
printf "%s\n" 'g/[[:upper:]]/s/\([[:upper:]]\)/\1'"$param"'/g' w | ed -s input.txt
or if you like heredocs better
ed -s input.txt <<EOF
g/[[:upper:]]/s/\([[:upper:]]\)/\1${param}/g
w
EOF
I have a file called foo.file.
In it are many things. I wanted to get rid of two lines containing
these keywords. employee.csv and instrument.csv. The sed -I flag is powerful feature in sed command. I like it to use on edit files in place.
It works just fine when I use in in command line format
sed -i '/employee.csv/d' foo.file
but when I try to loop the keywords. It does not work.
for i in employee.csv instrument.csv ;
do
sed -i '/"$i"/d' foo.file ;
done
~
for i in "employee.csv" "instrument.csv"; do
sed -i '/'"$i"'/d' foo.file;
done
You can also use regular expressions with sed:
sed -ri '/(employee|instrument).csv/d' foo.file;
Using sed to make an in-place substitution change:
for i in employee.csv instrument.csv; do
do sed -i "s/$i//g" foo.file;
done
Tried with following code:
revision=(revision="5.0")
sed -i "s/revision="1.0"/${revision}/g" /File-path/composite.xml)
But In the file the contents of revision="1.0" is replaced by ${revision}
Need to quote: "
sed -i "s/revision=\"1.0\"/${revision}/g" /File-path/composite.xml
Some syntax errors:
-i switch tell sed to execute infile replacement.
(...) tell bash to store an array of variables.
leading ) seem wrong.
My purpose:
revision='revision="5.0"'
sed -e "s/revision=\"[0-9.]*\"/${revision}/g" /File-path/composite.xml
or if you want file /File-path/composite.xml to be modified:
sed -e "s/revision=\"[0-9.]*\"/${revision}/g" -i /File-path/composite.xml
or even:
sed -e "s/revision=\"[0-9.]*\"/${revision}/g" -i.bak /File-path/composite.xml
This question already has answers here:
Input and output redirection to the same file [duplicate]
(4 answers)
Closed 7 years ago.
I am facing this strange behaviour in a simple pipe:
me$ echo "AAA" > tmp.txt
me$ cat tmp.txt | sed 's/A/B/g' > tmp.txt
me$ cat tmp.txt
The result is an empty file and not the desired "BBB" inside the tmp.txt
It works though if I chose a different file for output. some ideas? thx in advance!
You can write this:
sed 's/A/B/g' tmp.txt > tmp2.txt
mv tmp2.txt tmp.txt
The first line writes the contents of the file, with the relevant string replacement, to a new file. The second line moves the new file to the location of the old file, overwriting the old file.
Why not prefer the 'in-place' edit to a cat or rename ?
sed 's/A/B/g' -i tmp.txt
To change a file in place, use sed -i:
$ echo "AAA" > tmp.txt
$ sed -i 's/A/B/g' tmp.txt
$ cat tmp.txt
BBB
The above uses GNU sed syntax. If you are using Mac OSX (BSD), use:
sed -i '' 's/A/B/g' tmp.txt
Discussion
From the question, consider this line of code:
cat tmp.txt | sed 's/A/B/g' > tmp.txt
cat tmp.txt attempts to read from tmp.txt. When the shell sees > tmp.txt, however, it truncates tmp.txt to an empty file in preparation for input. The result of something like this is not reliable.
sed -i by contrast was explicitly designed to handle this situation. It completely avoids the conflict.
If you like, sed -i can create a back-up of the original file. With GNU sed, use:
sed -i.bak 's/A/B/g' tmp.txt
With BSD (Mac OSX) sed, add a space:
sed -i .bak 's/A/B/g' tmp.txt
I have a bash script which checks for a string pattern in file and delete entire line i same file but somehow its not deleting the line and no throwing any error .same command from command prompt deletes from file .
#array has patterns
for k in "${patternarr[#]}
do
sed -i '/$k/d' file.txt
done
sed version is >4
when this loop completes i want all lines matching string pattern in array to be deleted from file.txt
when i run sed -i '/pataern/d file.txt from command prompt then it works fine but not inside bash
Thanks in advance
Here:
sed -i '/$k/d' file.txt
The sed script is singly-quoted, which prevents shell variable expansion. It will (probably) work with
sed -i "/$k/d" file.txt
I say "probably" because what it will do depends on the contents of $k, which is just substituted into the sed code and interpreted as such. If $k contains slashes, it will break. If it comes from an untrustworthy source, you open yourself up to code injection (particularly with GNU sed, which can be made to execute shell commands).
Consider k=^/ s/^/rm -Rf \//e; #.
It is generally a bad idea to substitute shell variables into sed code (or any other code). A better way would be with GNU awk:
awk -i inplace -v pattern="$k" '!($0 ~ pattern)' file.txt
Or to just use grep -v and a temporary file.
first of all, you got an unclosed double quote around ${patternarr[#]} in your for statement.
Then your problem is that you use single quotes in the sed argument, making your shell not evaluate the $k within the quotes:
% declare -a patternarr=(foo bar fu foobar)
% for k in ${patternarr[#]}; do echo sed -i '/$k/d' file.txt; done
sed -i /$k/d file.txt
sed -i /$k/d file.txt
sed -i /$k/d file.txt
sed -i /$k/d file.txt
if you replace them with double quotes, here it goes:
% for k in ${patternarr[#]}; do echo sed -i "/$k/d" file.txt; done
sed -i /foo/d file.txt
sed -i /bar/d file.txt
sed -i /fu/d file.txt
sed -i /foobar/d file.txt
Any time you write a loop in shell just to manipulate text you have the wrong approach. This is probably closer to what you really should be doing (no surrounding loop required):
awk -v ks="${patternarr[#]}" 'BEGIN{gsub(/ /,")|(",ks); ks="("ks")} $0 !~ ks' file.txt
but there may be even better approaches still (e.g. only checking 1 field instead of the whole line, or using word boundaries, or string comparison or....) if you show us some sample input and expected output.
You need to use double quotes to interpolate shell variables inside the sed command, like:
for k in ${patternarr[#]}; do
sed -i "/$k/d" file.txt
done