I have some 100 files which have my name in it, RAHUL. I want it replaced with another term RAHUL2. I have a file which contains the list of files and i want to fetch it in a sed to do the changes.
files :
C:/desktop/file1.txt
C:/desktop/rahul/file1.txt
C:/desktop/rahul/file3.txt
C:/desktop/rahul/file4.txt
C:/desktop/rahul/file6.txt
C:/desktop/rahul/file8.txt
C:/desktop/rahul/file9.txt
and in each file data, i want to replace all occurance of term RAHUL with RAHUL2
I assume you are using some Cygwin environment on Windows, since you have sed. Then you can use find to list all the files and execute sed on them:
find C:/desktop -type f -name 'file*.txt' -exec sed -i 's/RAHUL/RAHUL2/g' {} \;
Please make a backup of the original files if you are using sed -i but aren't sure if the command is working already. This because sed -i will overwrite the original file. You have been warned.
hek2mgl's answer is good if you want to search for all files matching file*.txt pattern on your C:/desktop directory.
Now as you already have a file containing the list of files to edit, here is another way to proceed:
while read FILE ; do sed -i 's/RAHUL/RAHUL2/g' $FILE ; done < files_to_edit.txt
The read command will read your input one line after another. Input is files_to_edit.txt file, as indicated by the < input redirection operator.
Remark about -i option remains valid: it will edit your files in place, so make backup, or at least run it first on a couple of files (potentially without -i option, just to check what output is).
Related
I want to append a String to all lines of files inside a directory hierarchy. I know how to do this for a single file by the sed command :
sed -e 's/$/ sample string/' -i test.txt
but how to write a script that does this for all files found in a directory hierarchy (and its sub-directories and sub-sub-directories so on...)?
for all files found in a directory hierarchy (and its sub-directories and sub-sub-directories so on...)
This is a job for find, plain and simple.
For example, the command:
find . -type f -exec sed -i 's/$/ sample string/' {} \;
will execute that sed command on all regular files in and under the current directory.
The find command has lots more options (depth limiting, not crossing filesystem boundaries, only working on certain file masks, and so on) that you can use but the ones above are probably all you need for your immediate question.
Keep in mind that sed -i is in-place editing so, if you make a mistake, you better make sure you've backed up the original files before-hand.
In the case where you want to create new files (as requested in a comment), you can bypass the -i and create new files. The safest way to do this is probably by putting the commands you want to execute in an executable script (e.g., myprog.sh):
#!/usr/bin/env bash
origName="$1"
newName="$1.new"
sed s/$/ sample string/' "${origName}" > "${newName}"
Then call that from your find:
find . -type f -exec myprog.sh {} \;
Then you can make the target file name an arbitrarily complex value calculated from the original.
I'm trying to create a bash script based on a input file (list.txt). The input File contains a list of files with absolute path. The output should be a bash script (move.sh) which moves the files to another location, preserve the folder structure, but changing the target folder name slightly before.
the Input list.txt File example looks like this :
/In/Folder_1/SomeFoldername1/somefilename_x.mp3
/In/Folder_2/SomeFoldername2/somefilename_y.mp3
/In/Folder_3/SomeFoldername3/somefilename_z.mp3
The output file (move.sh) should looks like this after creation :
mv "/In/Folder_1/SomeFoldername1/somefilename_x.mp3" /gain/Folder_1/
mv "/In/Folder_2/SomeFoldername2/somefilename_y.mp3" /gain/Folder_2/
mv "/In/Folder_3/SomeFoldername3/somefilename_z.mp3" /gain/Folder_3/
The folder structure should be preserved, more or less.
after executing the created bash script (move.sh), the result should looks like this :
/gain/Folder_1/somefilename_x.mp3
/gain/Folder_2/somefilename_y.mp3
/gain/Folder_3/somefilename_z.mp3
What I've done so far.
1. create a list of files with absolute path
find /In/ -iname "*.mp3" -type f > /home/maars/mp3/list.txt
2. create the move.sh script
cp -a /home/maars/mp3/list.txt /home/maars/mp3/move.sh
# read the list and split the absolute path into fields
while IFS= read -r line;do
fields=($(printf "%s" "$line"|cut -d'/' --output-delimiter=' ' -f1-))
done < /home/maars/mp3/move.sh
# add the target path based on variables at the end of the line
sed -i -E "s|\.mp3|\.mp3"\"" /gain/"${fields[1]}"/|g" /home/maars/mp3/move.sh
sed -i "s|/In/|mv "\""/In/|g" /home/maars/mp3/move.sh
The script just use the value of ${fields[1]}, which is Folder_1 and put this in all lines at the end. Instead of Folder_2 and Folder_3.
The current result looks like
mv "/In/Folder_1/SomeFoldername1/somefilename_x.mp3" /gain/Folder_1/
mv "/In/Folder_2/SomeFoldername2/somefilename_y.mp3" /gain/Folder_1/
mv "/In/Folder_3/SomeFoldername3/somefilename_z.mp3" /gain/Folder_1/
rsync is not an option since I need the full control of files to be moved.
What could I do better to solve this issue ?
EDIT : #Socowi helped me a lot by pointing me in the right direction. After I did a deep dive into the World of Regex, I could solve my Issues. Thank you very much
The script just use the value of ${fields[1]}, which is Folder_1 and put this in all lines at the end. Instead of Folder_2 and Folder_3.
You iterate over all lines and update fields for every line. After you finished the loop, fields retains its value (from the last line). You would have to move the sed commands into your loop and make sure that only the current line is replaced by sed. However, there's a better way – see down below.
What could I do better
There are a lot of things you could improve, for instance
Creating the array fields with mapfile -d/ fields instead of printf+cut+($()). That way, you also wouldn't have problems with spaces in paths.
Use sed only once instead of creating the array fields and using multiple sed commands. You can replace step 2 with this small script:
cp -a /home/maars/mp3/list.txt /home/maars/mp3/move.sh
sed -i -E 's|^/[^/]*/([^/]*).*$|mv "&" "/gain/\1"|' /home/maars/mp3/move.sh
However, the best optimization would be to drop that three step approach and use only one script to find and move the files:
find /In/ -iname "*.mp3" -type f -exec rename -n 's|^/.*?/(.*?)/.*/(.*)$|/gain/$1/$2|' {} +
The -n option will print what will be renamed without actually renaming anything . Remove the -n when you are happy with the result. Here is the output:
rename(/In/Folder_1/SomeFoldername1/somefilename_x.mp3, /gain/Folder_1/somefilename_x.mp3)
rename(/In/Folder_2/SomeFoldername2/somefilename_y.mp3, /gain/Folder_2/somefilename_y.mp3)
rename(/In/Folder_3/SomeFoldername3/somefilename_z.mp3, /gain/Folder_3/somefilename_z.mp3)
It's not builtin to bash, but the mmv command is nice for this kind of mv where you need to use wildcards in paths. Something like the following should work:
mmv "in/*/*/*" "#1/#3"
Note that this won't create the directories for you - but in your example above it looks like these already exist?
I am getting into linux and optimizing my workflow and would love an example. I have a whole bunch of .scss inside of nested folders and I need to check if the first line is blank, and if so delete it, then re-save the file. I'm working on windows at work, but like writing bash. I've experimented with :
grep -r "/^\n/"
But seems to return every blank line. Then I'm not too sure how to delete it and then re-save.
This may get you started:
find . -name '*.scss' -exec sed -i.bak '1{/^$/d}' {} \;
To understand this command, we can break it into two parts:
find . -name '*.scss' -exec ... \;
Starting with the current directory, this looks recursively for files with names ending with .scss and, when it finds one, it runs the command that follows -exec on it.
sed -i.bak '1{/^$/d}' {}
sed is a stream editor. The option -i.bak tells it to change files in-place, leaving behind back backup file. Before find runs this command, it will replace {} with the actual name of the file that it found.
1{...}' tells sed to select the first line of the file and apply to it the commands in braces.
/^$/ is a regular expression. It matches a line if the line is empty.
d tells sed to delete any matching line.
So, let's put that all together: if the first line of the file is empty, sed deletes it.
You will find many tutorials on the web on both find and sed. You can find detailed information on either using man find or man sed.
You don't usegrep to edit files, you should use sed.
https://www.gnu.org/software/sed/manual/sed.html
I want to find specific text in a directory, and then delete the lines from the files that include the specific text.
Now I have two questions:
How can I achieve the task?
What is wrong with What I have tried? I have tried the methods below, but failed. the details are following:
grep -rnw "./" -e "webdesign"
This searches the current directory with pattern "webdesign", and I get the result:
.//pages/index.html:1:{% load webdesign %}
.//pages/pricing.html:1:{% load webdesign %}
.//prototypes.py:16: 'django.contrib.webdesign',
Then I use sed to remove the lines from those files, which doesn't work, only get blank file ( I mean it deletes all my file content):
sed -i "/webdesign/d" ./pages/index.html
or
sed "/webdesign/d" ./pages/index.html > ./pages/index.html
My software environment is: OS X Yosemite, Mac Terminal, Bash
A loop in bash will do the trick provided that there are no filenames with spaces (in which case other solutions are possible, but this is the simplest)
for i in `grep -lrnw "yourdirectory/" -e "webdesign"`
do
sed "/webdesign/d" $i > $i.tmp
# safety to avoid destroying the file if problem arises (disk full?)
if [ $? = 0 ] ; then
mv -f $i.tmp $i
fi
done
note that you should not locate this script in the current directory because it contains webdesign and it will be modified as well :)
Thanks to choroba, I know that -i option doesn't work like wished. But it has another meaning or it would be rejected by the opt parser. It has something to do with suffixes, well, it doesn't matter now, but it's difficult to see the problem at first.
Without -i you cannot work on a file in-place. And redirecting output to the input just destroys the input file (!). That's why your solution did not work.
You can install GNU sed that supports the -i option, then
sed -i '/webdesign/d' files
should work. Note that it's safer to use -i~ to create a backup.
You cannot write to the same file you're reading from, that's why
sed /webdesign/d file > file
doesn't work (it overwrites the file before you can read anything from it). Create a temporary file
sed /webdesign/d file > file.tmp
mv file.tmp file
I'm new to OSX command line tools.
I am trying to find a block of text in a file and append this text at the end of all lines in another text file. At run time I don't know what this text will be, I just know it will be located within "BEGINHMM" and "ENDHMM". Also, I don't know the makeup of the destination file, except for that it will not be an empty text file.
The command which finds the block of text of interest is:
sed -n '/<BEGINHMM>/,/<ENDHMM>/p' proto
where "proto" is a text file containing the text of interest.
I've been trying to pipe the output of the above command to another 'sed' command, in the following manner:
xargs -I '{}' sed -i .bak 's/$/{}/' monophones0.txt
but I am getting some bizarre results, I see the "{}" inserted in the text for example.
I've also tried piping to:
xargs -0 sed -i .bak 's/$/&/' monophones0.txt
but I just get the printout (similar to terminal echo) of the text I am trying to grab.
Ultimately I want to loop over several 'proto' files in multiple directories and copy the text between the "BEGINHMM", "ENDHMM" block in each directory, and append the selected text to that directory's monophones.txt lines.
I am running the commands in the terminal, bash, OSX 10.12.2
Any help would be appreciated.
(1) Your sed command is of the form sed -n '/A/,/B/p'; this will include the lines on which A and B occur, even if these strings do not appear at the beginning of the line. This form may have other surprises in store for you as well (what do expect will happen if B is missing or repeated?), but the remainder of this post assumes that's what you want.
(2) It's not clear how you intend to specify the "proto" files, but you do indicate they might be in several directories, so for the remainder of this post, I'll assume they are listed, one per line, in a file named proto.txt in each directory. This will ensure that you don't run into any limitations on command-line length, but the following can easily be modified if you don't want to create such a file.
(3) Here is a script which will use the sed command you've mentioned to copy segments from each of the "proto" files specified in a directory to monophones0.txt in the directory in which the script is executed.
#!/bin/bash
OUT=monophones0.txt
cat proto.txt | while read file
do
if [ -r "$file" ] ; then
sed -n '/<BEGINHMM>/,/<ENDHMM>/p' "$file" >> $OUT
elif [ -n "$file" ] ; then
echo "NOT FOUND: $file" >&2
fi
done
Just like what you did before. tmpfile=$(mktemp); sed -n '/<BEGINHMM>/,/<ENDHMM>/p' proto >$tmpfile; sed -i .bak "r $tmpfile" monophones0.txt; rm $tmpfile. This is the basic idea; there are other checks you need to perform to make this a robust script.
– 4ae1e1