This question already has answers here:
Find and replace in file and overwrite file doesn't work, it empties the file
(12 answers)
Sed to delete a range of lines from a specific match line TILL a specific match line (not including last line)
(3 answers)
Closed 8 years ago.
I am writing a bash shell script, and in it I am trying to delete lines from a text file between 2 markers
START
...
...
END
Do eliminate this I have tried a few things, and every time it leave my text file blank.
sed '/START/,/END/d' myfile.txt > myfile.txt
sed '/START/d' myfile.txt > myfile.txt
As long as I have a sed command in my code, my entire file gets erased and not just the section I am looking to erase.
You are redirecting stdout to the same file as stdin. When you do this, your redirection is interpreted buy the shell and it opens a new file for write with that name. Your existing file is overwritten by the newly created blank file. You will need to redirect the output to a different file or you can edit the file by using the -i option to sed.
You can't redirect to a file that you are reading. That will delete your file's contents, as you've noticed.
Instead, either redirect to a different file, or edit in place:
sed -i ... myfile.txt
A way in awk.
awk '/START/{x=1}/END/{x=0}!x{print > ARGV[1]}' myfile.txt
Portable
awk '/START/{x=1}/END/{x=0}!x{print > "myfile.txt"}' myfile.txt
Related
This question already has answers here:
Bash - remove all lines beginning with 'P'
(5 answers)
Closed 3 years ago.
I'm trying to copy a file but skip a specific line in the file that starts with 'An', using the bash in mac terminal.
The file only has 4 lines:
Kalle Andersson 036-134571
Bengt Pettersson 031-111111
Anders Johansson 08-806712
Per Eriksson 0140-12321
I know how to copy the file using the command cp and to grab a specific line in the file using the grep command.
I do not know how i can delete a specific line i the file.
I have used the cp command:
cp file1.txt file2.txt
to copy the file.
And I used the
grep 'An' file2.txt
I expect the result where the new file have the three lines:
Kalle Andersson 036-134571
Bengt Pettersson 031-111111
Per Eriksson 0140-12321.
Is there an way I can do this in a single command?
As Aaron said:
grep -vE '^An' file1.txt > file2.txt
What you do here is use grep with the -v option. That means print every line, except the one that matches. Furthermore, you instruct the shell to redirect the output of the grep to file2.txt. That is the meaning of the>.
There are a lot of commands in Unix/Linux that can be used for this. sed is an obvious candidate, awk can do it, as in
awk '{if (!/^An/) print}' file1.txt > file2.txt
Another option is ed:
ed file1.txt <<EOF
1
/^An
d
w file2.txt
q
EOF
This question already has answers here:
Bash tool to get nth line from a file
(22 answers)
How do I set a variable to the output of a command in Bash?
(15 answers)
Closed 3 years ago.
So, I am creating a script in bash and I am trying to read a text file which is saved in the same directory. I need to read the nth line of that file and then save it to a variable to be used later but I'm not sure how I can do this
What I have currently tried is listed below but it essentially reads the line from the file, saves it a variable and then deletes that line from the file and repeats. This is a hack and although it works, isn't what I want, I can't get the nth value and it's deleting from the file which I definitely don't want.
read -r first<"test.txt" // reads first line and stores in first
sed -i -e "1d" "test.txt" . // removes first line
read -r second<"test.txt" // reads first line and stores in second
sed -i -e "1d" "test.txt" . // removes first line
If I wanted to get the 2nd line for example, I have seen sed '2q;d' file but not sure how/where the result is saved. It gets printed in terminal? Any help appreciated, thanks!w
sed '2q;d' file
prints the second line in file to the terminal.
To populate a variable with it, use bash's command expansion feature:
$ var=$(sed '2q;d' file)
$ echo "$var"
this is second line
Simple solution using head and tail:
a=$(head -2 test.txt | tail -1 )
Saves the second line of test.txt to the variable $a.
Have a directory full of file names that end with .mp3 and have a code in it that i would like to pipe into a text file.
I need to get the last 11 characters before the .mp3 part of a file in a certain directory and pipe that into a text file (with bash on mac osx)
How do I accomplish this? With sed?
If I'm understanding correctly, you have a list of files with names like "abcdefghijklmnopqrstuvwxyz.mp3" and want to extract "pqrstuvwxyz". You can do this directly in bash without invoking any fancy sed business:
for F in *.mp3; do STRIP=${F/.mp3}; echo ${STRIP: -11}; done > list.txt
The first STRIP variable is the name of each file F with the .mp3 extension removed. Then you echo the last 11 characters and save to a file.
There's a nice page on bash substitutions here. sed is great but I personally find it's overkill for these simple cases.
Along with good above answers, can be done via awk
for f in `ls *.mp3`;
echo $f|awk -F. '{printf substr($1,length($1)-11,length($1)),$2;}'
done
This question already has answers here:
Remove the last line from a file in Bash
(16 answers)
Closed 5 years ago.
I asked a question regarding popping the last line of a text file in PHP, and now, is it possible to re-write the logic in shell script?
I tried this to obtain the last line:
tail -n 1 my_log_file.log
but I am not sure how can I remove the last line and save the file.
P.S. given that I use Ubuntu server.
To get the file content without the last line, you can use
head -n-1 logfile.log
(I am not sure this is supported everywhere)
or
sed '$d' logfile.log
What you want is truncate the file just before the last line without having to read the file entirely.
truncate -s -"$(tail -n1 file | wc -c)" file
That's assuming the file is not currently being written to.
truncate is part of the GNU coreutils (so generally found on recent Linux distributions) and is not a standardized Unix or POSIX command. Many "dd" implementations can be used to truncate a file as well.
(Solution is based on sch's answer so credit should go to him/her)
This approach will allow you to efficiently retrieve the last line of the file and truncate the file to remove that line. This can better deal with large inputs as the file is not read sequentially.
# retrieve last line from file
LAST=$(tail -n 1 my_log_file.log)
# truncate file
let TRUNCATE_SIZE="${#LAST} + 1"
truncate -s -"$TRUNCATE_SIZE" my_log_file.log
# ... $LAST contains 'popped' last line
Note that this will not work as expected if the file is modified between the calls to tail and truncate.
One way is:
sed '$d' < f1 > f2 ; mv f2 f1
This question already has answers here:
How to delete from a text file, all lines that contain a specific string?
(21 answers)
Closed 5 years ago.
I want to delete a line containing a specific string from the file. How can I do this without using awk? I tried to use sed but I could not achieve it.
This should do it:
sed -e s/deletethis//g -i *
sed -e "s/deletethis//g" -i.backup *
sed -e "s/deletethis//g" -i .backup *
it will replace all occurrences of "deletethis" with "" (nothing) in all files (*), editing them in place.
In the second form the pattern can be edited a little safer, and it makes backups of any modified files, by suffixing them with ".backup".
The third form is the way some versions of sed like it. (e.g. Mac OS X)
man sed for more information.
sed -i '/pattern/d' file
Use 'd' to delete a line. This works at least with GNU-Sed.
If your Sed doesn't have the option, to change a file in place, maybe you can use an intermediate file, to store the modification:
sed '/pattern/d' file > tmpfile && mv tmpfile file
Writing directly to the source doesn't work: sed '/pattern/d' FILE > FILE so make a copy before trying out, if you doubt it. The redirection to a new file FILE will open a new file FILE before reading from it, so reading from it will result in an empty input.
Try the vim-way:
ex -s +"g/foo/d" -cwq file.txt