I need to strip files out of a number of directories that all have the same file name a.txt. The difference comes from the parent folder so
example1\a.txt
example2\a.txt
...
so I am hoping to run a FIND command that will capture a.txt but not overwrite the file as it moves from folder to folder.
so the output would be
example1_a.txt
example2_a.txt
So from another post the FIND command I want is the following
find . -name "a.txt" -execdir echo cp -v {} /path/to/dest/ \;
So I want to modify in some way to append the source folder to the file. so my guess is to manipulate {} somehow to do it.
Thanks in advance
A one liner might be possible, but you could use this:
#!/bin/bash
targetprefix="targetdir"
find . -name "a.txt" -print0 | while read -r -d '' line
do
path=$(dirname "$line")
newpath=$(echo "${path#./}" | tr '/' '_')
target="$targetprefix/$newpath"
filename=$(basename "$line")
cp -v $line $target/$filename
done
change variable "targetprefix" to be the destination directory you desire.
this find with -print0 and while comes from https://mywiki.wooledge.org/BashFAQ/001
since results from find all start with "./", I use "${path#./}" to remove that prefix.
the tr replaces all subsequent "/" with an underscore. This will take care of sub directories
WARNING: I did not test all "weird" directory and filename formats for proper execution (like carriage returns in filenames!).
Related
I am using a shell script to remove the XML tags of a set of files in a folder. This is how my file looks like:
#!/bin/sh
find texts -type f -name '*.xml' -exec sh -c '
mkdir -p modified
file="$0"
sed "s/<[^>]*>//g" "$file" > modified/modified_texts
' {} ';'
This is supposed to take all the files(using $file) in the "texts" folder, remove their XML tags and place the files without the XML tags into the file "modified".
The problem is that, instead of taking all the files, it is using just one, and filling the file "modified_texts" with the content of one of the files(without XML tags, that part works).
I don't really understand what I'm doing wrong, so I would appreciate any help.
Instead of doing the output redirection (with truncation!) for every sed command, move it to the outer scope, so the output file is opened (and its prior contents are truncated) only once, before find is started at all.
#!/bin/sh
mkdir -p modified # this only needs to happen once, so move it outside
find texts -type f -name '*.xml' -exec sed 's/<[^>]*>//g' {} ';' > modified/modified_texts
I've a .txt file which contains names of various files.
When I simply use a while loop it works fine,
while read -r name
do
echo "$name"
done <fileNames.txt
But,
when I try to use find inside the loop, like this:
while read -r name
do
find ./ -iname "$name" -exec sed -i '1s/^/NEW LINE INSERTED \n/' '{}' ';'
done < fileNames.txt
nothing happens!
If i use the find outside the loop like with a specific file name it does what it's supposed to do, I can also use it on all files with a specific file-type but it doesn't work inside the loop.
What am I doing wrong over here?
I'm trying to read file names from a file, search for it inside a folder recursively and then append a line in the beginning using sed.
Use xargs instead to capture the results of find
while read -r name
do
find ./ -iname "$name" |xargs sed -i '1s/^/NEW LINE INSERTED \n/'
done <fileNames.txt
I have a bunch of javascript files and I need to add a comment on a new line at the beginning of every file that contains the filename e.g.
/* app.js */
I'd like to do this recursively on a folder from the terminal on my mac.
Can anyone help?
Thanks!
First, we have to locate those files:
$ find $folder_name -type f -name "*.js" -print
This will locate all of the files with the suffix of *.js. The type -f means only look for files (just incase you have a directory name that ends in .js). I will assume that your JavaScript file names don't contain whitespace, or control characters like <NL> characters in the name. Otherwise, we'll have to do a bit more work.
We can use that find statement to locate all of those files, then we can munge those files. Once we find those files, we have to figure out how to munge the file to get the comment line on top.
We'll take three step approach:
First, we'll create a new file with the comment line in it.
Next we'll concatenate the old file on the end of the new file.
We'll rename the new file to the old file.
For the first step, we'll use echo to create a new file with the old name. Then, we'll use the cat command to concatenate the old file on the new file. And, finally we'll use mv to rename the newly constructed file to the old name.
find $javascript_dir -type f -name "*.js" -print | while read file_name
do
basename=$(basename $file_name)
echo "/* $basename */" > "$basename.tmp"
cat "$file_name" >> "$basename.tmp"
mv "$basename.tmp" "$file_name"
done
Note that > redirects the output of the command to the named file, and that >> appends the output of the command to the named file. mv stands for move. In Unix, file are actually stored by inode identify. There's a table of contents that matches the inode to the full name of the file (including the directory). Thus, moving a file in Unix is the same as simply renaming it, so the directory name is changed. In Windows batch, there are separate move and rename commands. In Unix, they're both mv.
Addition
We could use sed to do an in place prepending of the line:
find . -name "*.js" -print | while read file_name
do
basename=$(basename $file_name)
sed -i "" "1i\\
/* $basename */
" "$file_name"
done
sed is a powerful line editor, and most people use only its substitution command mode. However, sed scripts can be way more complex and rich (and obscure which is why you rarely see them).
You can use cat - and a temp file:
# adds filename to top of each file in $DIR
DIR=. # set to whatever directory
FILES=$(find $DIR -type f)
for FILE in $FILES; do
FILE=$(echo $FILE | sed -e s/\.\\\///) # remove leading "./" if DIR is "."
echo "/* $FILE */" | cat - $FILE > tempfile.tmp # add the line using a temp file
mv tempfile.tmp $FILE # move the temp file back to the original file
done
Note that this will add the full path of the file relative to DIR if DIR is not '.'. To add only the filename itself you could use:
echo "/* $(basename $FILE) */" | cat - $FILE > tempfile.tmp # add the line using a temp file
You can use sed for this, try something like this:
#!/bin/bash
for i in $(find ./ -name *.js);
do
`sed -i 's/$i \*\//$i \*\/\n&/' "$i"`
done
Insert a newline with sed could be a problem since not all version works the same. To have an overview see this reference. You can for example replace the substitution part with G or {G;}. It depends on your sed version.
Similar Questions are already answered here and here
Another interesting solution:
If you want to append a newline after the first line you can also use this:
sed -i "1a\ " test.js
So change the script above into this:
#!/bin/bash
for i in $(find ./ -name *.js);
do
`sed -i "1a\ " $1`
done
How can I remove a directory if it contains only a specific file type (it should not contain any other files)?
For Example:
My Directory name is : AAA
AAA is contains a file abc.txt
In this case AAA should be deleted.
But if AAA contains abc.txt and def.pdf then in this case it should not be deleted.
In every case, you need find the directories containg file abc.txt. For this you can use the file command, like:
find $topdir -name abc.txt -print
this will print the full pathname of the files. Stripping the file part, you will get a directory name. For this exists the dirname command what for the:
dirname "./x/y/z/abc.txt"
produces
./x/y/z
or you can use the -printf functionality of the find command to print only the directory part, like:
find $topdir -name abc.txt -printf "%h\n"
Because the filenames can contain spaces and or newlines too, it is recommended to use null terminated strings. E.g. the above command could be
find $topdir -name abc.txt -printf "%h\0" #null terminated
The result of any command could be read in the while loop, for the null terminated strings use:
while read -d $'\0' -r line
do
#do something here
done < <(command)
The find command knows the -empty argument, what can help to find empty directories, or you can test the existence of any file with the
if [[ -f "$filename" ]]
then
# do something
else
# do something other
fi
or shortened
[[ -f "$filename ]] && dosomething
And finally you can remove a directory with the rmdir command or can recursively remove anything with the rm -r command. (dangerous!).
Now, you have probably all informations to make a script. Try create something and come back to ask if you meet some problems.
I want to make a search for all .fits files that contain a certain text in their name and then copy them to a directory.
I can use a command called fetchKeys to list the files that contain say 'foo'
The command looks like this : fetchKeys -t 'foo' -F | grep .fits
This returns a list of .fits files that contain 'foo'. Great! Now I want to copy all of these to a directory /path/to/dir. There are too many files to do individually , I need to copy them all using one command.
I'm thinking something like:
fetchKeys -t 'foo' -F | grep .fits > /path/to/dir
or
cp fetchKeys -t 'foo' -F | grep .fits /path/to/dir
but of course neither of these works. Any other ideas?
If this is on Linux/Unix, can you use the find command? That seems very much like fetchkeys.
$ find . -name "*foo*.fit" -type f -print0 | while read -r -d $'\0' file
do
basename=$(basename $file)
cp "$file" "$fits_dir/$basename"
done
The find command will find all files that match *foo*.fits in their name. The -type f says they have to be files and not directories. The -print0 means print out the files found, but separate them with the NUL character. Normally, the find command will simply return a file on each line, but what if the file name contains spaces, tabs, new lines, or even other strange characters?
The -print0 will separate out files with nulls (\0), and the read -d $'\0' file means to read in each file separating by these null characters. If your files don't contain whitespace or strange characters, you could do this:
$ find . -name "*foo*.fit" -type f | while read file
do
basename=$(basename $file)
cp "$file" "$fits_dir/$basename"
done
Basically, you read each file found with your find command into the shell variable file. Then, you can use that to copy that file into your $fits_dir or where ever you want.
Again, maybe there's a reason to use fetchKeys, and it is possible to replace that find with fetchKeys, but I don't know that fetchKeys command.
Copy all files with the name containing foo to a certain directory:
find . -name "*foo*.fit" -type f -exec cp {} "/path/to/dir/" \;
Copy all files themselves containing foo to a certain directory (solution without xargs):
for f in `find . -type f -exec grep -l foo {} \;`; do cp "$f" /path/to/dir/; done
The find command has very useful arguments -exec, -print, -delete. They are very robust and eliminate the need to manually process the file names. The syntax for -exec is: -exec (what to do) \;. The name of the file currently processed will be substituted instead of the placeholder {}.
Other commands that are very useful for such tasks are sed and awk.
The xargs tool can execute a command for every line what it gets from stdin. This time, we execute a cp command:
fetchkeys -t 'foo' -F | grep .fits | xargs -P 1 -n 500 --replace='{}' cp -vfa '{}' /path/to/dir
xargs is a very useful tool, although its parametrization is not really trivial. This command reads in 500 .fits files, and calls a single cp command for every group. I didn't tested it to deep, if it doesn't go, I'm waiting your comment.