I want to remove the first line of all files in a folder if the file starts with uuid and so I have a working sed command looking like this:
$ sed -i '' '/^uuid/d' *
which works fine and removes all lines starting with uuid.
Now I want to improve the script by only removing the first line if it starts with uuid since some of the files has multiple uuid:s and only the one on the first line should be deleted. So now I improved the command to look like this:
$ sed -i '' '1{/^uuid/d;}' *
Now this command also works but only on the first file in the folder and even if I run a simple (just remove first line) version like:
$ sed -i '' '1d' *
it still only affects the first file.
Why is this?
I'm on Mac (so the BSD version of sed as I've come to understand) and I also tried installing the gnu-sed version via Brew, $ brew install gnu-sed --with-default-names, with no luck.
I've read sed - 25 examples to delete a line or pattern in a file,
sed - 20 examples to remove / delete characters from a file and googled sed delete first line in files
UPDATE 1: As proposed in the comments by john1024 I've tested with the -s option but not sure how to use it.
$ sed -s '1d' ./*
sed: illegal option -- s
When I check man sed I can find the -s & --seperateoption so I must do something wrong here.
UPDATE 2: Ok, progress ... find . -iname '*.yml' -exec sed -i '' -e '1{/uuid/d;}' {} \; does the trick but I get error message saying sed: can't read : No such file or directory
Thanks for any help or guidance!
:ola
sed -i '' '1{/^uuid/d;}' * will modify only the first file,
because the line numbers are counted cumulatively across files,
so "line 1" occurs only once, it's the first line of the first file.
To do something with the first line of multiple files,
you need to run sed once per file.
You can do that using a simple for loop:
for f in *; do sed -i '' '1{/^uuid/d;}' "$f"; done
Related
I've a list of csv-files and would like to use a for loop to edit the content for each file. I'd like to do that with sed. I have this sed commands which works fine when testing it on one file:
sed 's/[ "-]//g'
So now I want to execute this command for each file in a folder. I've tried this but so far no luck:
for i in *.csv; do sed 's/[ "-]//g' > $i.csv; done
I would like that he would overwrite each file with the edit performed by sed. The sed commands removes all spaces, the " and the '-' character.
Small changes,
for i in *.csv
do
sed -i 's/[ "-]//g' "$i"
done
Changes
when you iterate through the for you get the filenames in $i as example one.csv, two.csv etc. You can directly use these as input to the sed command.
-i Is for inline changes, the sed will do the substitution and updates the file for you. No output redirection is required.
In the code you wrote, I guess you missed any inputs to the sed command
In my case i want to replace every first occurrence of a particular string in each line for several text files, i've use the following:
//want to replace 16 with 1 in each files only for the first occurance
sed -i 's/16/1/' *.txt
In your case, In terminal you can try this
sed 's/[ "-]//g' *.csv
In certain scenarios it might be worth considering finding the files and executing a command on them like explained in this answer (as stated there, make sure echo $PATH doesn't contain .)
find /path/to/csv/ -type f '*.csv' -execdir sed -i 's/[ "-]//g' {} \;
here we:
find all files (type f) which end with .csv in the folder /path/to/csv/
sed the found files in place, ie we replace the original files with the changed version instead of creating numbered csv files ($i.csv)
I have a bunch of files in the directory that has this pattern: version="0". It can be any number inside. But I don't want to affect the <?xml version="1.0" ?> parts. This line can be not on the first line, so we can't just skip the first.
The main problem is that sed and awk's gsub don't work with lookbehind. I decided that it acceptable to do double work. replace all versions and then fix all xml versions. But sed with -r missunderstands the capturing groups.
What I have so far:
sed -r 's#(\<\?xml .*)version="[^"]*"(.*\?\>)#\1version="1.0"\2#g' fixing xmls
To change all version attributes within an XML document, the following XMLStarlet command will suffice:
xmlstarlet ed --inplace \
-u '//*[#version="0"]/#version' -v 1 \
/your/directory/*.xml
I think I kind of made it:
find test -exec sed -i 's/version="[^"]*"/version="800"/g' {} \; -print | xargs -I FILE sed -i 's#\(<?xml \)version="[^"]*"\(.*\)#\1version="1.0"\2#g' FILE
where the 800 is desirable value. but still double work.
Don't escape the< and > if you don't use them for word boundary. Try this:
sed -r 's#(<\?xml .*)version="[^"]*"(.*\?>)#\1version="1.0"\2#g' file
That said you should avoid the any character .* pattern that is greedy. A safer command would be:
sed -r 's#(<\?xml[^>]*)version="[^"]*"([^>]*)\?>#\1version="1.0"\2?>#g' file
So, disclaimer: I am pretty new to using bash and zsh, so there is a chance the answer is really simple. Nonetheless. I checked previous postings and couldn't find anything. (edit: I have tried this in both bash and zsh shells- same problem.)
I have a directory with many files and am trying to remove the first line from each file.
So say the directory contains: file1.txt file2.txt file3.txt ... etc.
I am using the sed command (non-GNU):
sed -i -e "1d" *.txt
For some reason, this is only removing the first line of the first file. I thought that the *.txt would affect all files matching the pattern in directory. Strangely, it is creating the file duplicates with -e appended, but both the duplicate and original are the same.
I tried this with other commands (e.g. ls *.txt) and it works fine. Is there something about sed I am missing?
Thank you in advance.
Different versions of sed in differing operating systems support various parameters.
OpenBSD (5.4) sed
The -i flag is unavailable. You can use the following /bin/sh syntax:
for i in *.txt
do
f=`mktemp -p .`
sed -e "1d" "${i}" > "${f}" && mv -- "${f}" "${i}"
done
FreeBSD (11-CURRENT) sed
The -i flag requires an extension, even if it's empty. Thus must be written as sed -i "" -e "1d" *.txt
GNU sed
This looks to see if the argument following -i is another option (or possibly a command). If so, it assumes an in-place modification. If it appears to be a file extension such as ".bak", it will rename the original with the ".bak" and then modify it into the original file's name.
There might be other variations on other platforms, but those are the three I have at hand.
use it without -e !
for one file use:
sed -i '1d' filename
for all files use :
sed -i '1d' *.txt
or
files=/path/to/files/*.extension ; for var in $files ; do sed -i '1d' $var ; done
.for me i use ubuntu and debian based systems , this method is working for me 100% , but for other platformes i'm not sure , so this is other method :
replace first line with emty pattern , and remove empty lines , (double commands):
for files in $(ls /path/to/files/*.txt); do sed -i "s/$(head -1 "$files")//g" "$files" ; sed -i '/^$/d' "$files" ; done
Note: if your files contain splash '/' , then it will give error , so in this case sed command should look like this ( sed -i "s[$(head -1 "$files")[[g" )
hope that's what you're looking for :)
The issue here is that the line number isn't reset when sed opens a new file, so 1 only matches the first line of the first file.
One solution is to use a shell loop, calling sed once for each file. Gumnos' answer shows how to do this in the most widely compatible way, although if you have a version of sed supporting the -i flag, you could do this instead:
for i in *.txt; do
sed -i.bak '1d' "$i"
done
It is possible to avoid creating the backup file by passing an empty suffix but personally, I don't think it's such a bad thing. One day you'll be grateful for it!
It appears that you're not working with GNU tools but if you were, I would recommend using GNU awk for this task. The variable FNR is useful here, as it keeps track of the record number for each file individually, allowing you to do this:
gawk -i inplace 'FNR>1' *.txt
Using the inplace extension, this allows you to remove the first line from each of your files, by only printing the lines where FNR is greater than 1.
Testing it out:
$ seq 5 > file1
$ seq 5 > file2
$ gawk -i inplace 'FNR>1' file1 file2
$ cat file1
2
3
4
5
$ cat file2
2
3
4
5
The last argument you are passing to the Sed is the problem
try something like this.
var=(`find *txt`)
for file in "${var[#]}"
do
sed -i -e 1d $file
done
This did the trick for me.
Following sed command is not working on my lion mac.
find . -type f -exec sed -i 's/user_dashboard/user/g' {} \;
I am getting this error
sed: 1: "./vendor/assets/javascr ...": invalid command code .
The OSX version of sed is not the same as those found in most Linux systems.
It extends the -i option to give you the opportunity save a file with a different extension, but requires input for that extension.
If you just want to overwrite the file in place, you need to use sed -i "" ...sedCmd.... fileName to rename your file in-place.
Per #JamesMcMahon 's comment, see here for the full doc for OSX/BSD sed.
I hope this helps.
-i probably has a different meaning (not "in-place") in your version of sed. Try using gsed if available or replacing -i with -e and using a temporary file (and a mv afterwards) to emulate it.
replacing text inside a file on the fly with sed on mac is possible,
the command is just a little different.
with: -i , you have to specify a postfix, which sed will use to save the original file after it modified it.
run the command as:
$ sed -i _backup -E 's/foo/bar/' /tmp/jestinkt.txt
ending with both the modified /tmp/jestinkt.txt and the original /tmp/jestinkt.txt_backup
I'd like edit a file with sed on OS X. I'm using the following command:
sed 's/oldword/newword/' file.txt
The output is sent to the terminal. file.txt is not modified. The changes are saved to file2.txt with this command:
sed 's/oldword/newword/' file1.txt > file2.txt
However I don't want another file. I just want to edit file1.txt. How can I do this?
I've tried the -i flag. This results in the following error:
sed: 1: "file1.txt": invalid command code f
You can use the -i flag correctly by providing it with a suffix to add to the backed-up file. Extending your example:
sed -i.bu 's/oldword/newword/' file1.txt
Will give you two files: one with the name file1.txt that contains the substitution, and one with the name file1.txt.bu that has the original content.
Mildly dangerous
If you want to destructively overwrite the original file, use something like:
sed -i '' 's/oldword/newword/' file1.txt
^ note the space
Because of the way the line gets parsed, a space is required between the option flag and its argument because the argument is zero-length.
Other than possibly trashing your original, I’m not aware of any further dangers of tricking sed this way. It should be noted, however, that if this invocation of sed is part of a script, The Unix Way™ would (IMHO) be to use sed non-destructively, test that it exited cleanly, and only then remove the extraneous file.
I've similar problem with MacOS
sed -i '' 's/oldword/newword/' file1.txt
doesn't works, but
sed -i"any_symbol" 's/oldword/newword/' file1.txt
works well.
The -i flag probably doesn't work for you, because you followed an example for GNU sed while macOS uses BSD sed and they have a slightly different syntax.
All the other answers tell you how to correct the syntax to work with BSD sed. The alternative is to install GNU sed on your macOS with:
brew install gsed
and then use it instead of the sed version shipped with macOS (note the g prefix), e.g:
gsed -i 's/oldword/newword/' file1.txt
If you want GNU sed commands to be always portable to your macOS, you could prepend "gnubin" directory to your path, by adding something like this to your .bashrc/.zshrc file (run brew info gsed to see what exactly you need to do):
export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
and from then on the GNU sed becomes your default sed and you can simply run:
sed -i 's/oldword/newword/' file1.txt
sed -i -- "s/https/http/g" file.txt
You can use -i'' (--in-place) for sed as already suggested. See: The -i in-place argument, however note that -i option is non-standard FreeBSD extensions and may not be available on other operating systems. Secondly sed is a Stream EDitor, not a file editor.
Alternative way is to use built-in substitution in Vim Ex mode, like:
$ ex +%s/foo/bar/g -scwq file.txt
and for multiple-files:
$ ex +'bufdo!%s/foo/bar/g' -scxa *.*
To edit all files recursively you can use **/*.* if shell supports that (enable by shopt -s globstar).
Another way is to use gawk and its new "inplace" extension such as:
$ gawk -i inplace '{ gsub(/foo/, "bar") }; { print }' file1
This creates backup files. E.g. sed -i -e 's/hello/hello world/' testfile for me, creates a backup file, testfile-e, in the same dir.
You can use:
sed -i -e 's/<string-to-find>/<string-to-replace>/' <your-file-path>
Example:
sed -i -e 's/Hello/Bye/' file.txt
This works flawless in Mac.
If you need to substitute more than one different words:
sed -i '' -e 's/_tools/tools/' -e 's/_static/static/' test.txt