I have multiple unique directories that each contain a file named filtered_feature_bc_matrix.h5. I want to paste the directory-name onto the filename to make the filename unique within each folder. Is this possible?
./sample_scRNA_unsorted_98433_Primary_bam/outs/filtered_feature_bc_matrix.h5
./sample_scRNA_unsorted_77570_Primary_bam/outs/filtered_feature_bc_matrix.h5
out:
./sample_scRNA_unsorted_98433_Primary_bam/outs/sample_scRNA_unsorted_98433_Primary_bam_filtered_feature_bc_matrix.h5
./sample_scRNA_unsorted_77570_Primary_bam/outs/sample_scRNA_unsorted_77570_Primary_bam_filtered_feature_bc_matrix.h5
For completeness here is a solution with only bash built-in:
shopt -s globstar nullglob
name="filtered_feature_bc_matrix.h5"
for f in */**/"$name"; do mv "$f" "${f%/*}/${f%%/*}_$name"; done
find . -type f -name 'filtered_feature_bc_matrix.h5' \
-exec sh -c \
'fp="$1"; d=$(echo "$fp"|cut -d/ -f2); echo $(dirname "$fp")/${d}_$(basename "$fp")' \
-- '{}' \;
find allows to iterate the files with relative path
-exec allows to do action on '{}' which is the file path
sh is used to simplify the action
dirname and basename allows to extract directories or filename.
cut is used to extract only the first directory (second field as the first one is .)
To rename:
find . -type f -name 'filtered_feature_bc_matrix.h5' \
-exec sh -c \
'fp="$1"; d=$(echo "$fp"|cut -d/ -f2); mv -v "$fp" "$(dirname "$fp")/${d}_$(basename "$fp")"' \
-- '{}' \;
Related
I would like to recursively go through all subdirectories and remove the oldest two PDFs in each subfolder named "bak":
Works:
find . -type d -name "bak" \
-exec bash -c "cd '{}' && pwd" \;
Does not work, as the double quotes are already in use:
find . -type d -name "bak" \
-exec bash -c "cd '{}' && rm "$(ls -t *.pdf | tail -2)"" \;
Any solution to the double quote conundrum?
In a double quoted string you can use backslashes to escape other double quotes, e.g.
find ... "rm \"\$(...)\""
If that is too convoluted use variables:
cmd='$(...)'
find ... "rm $cmd"
However, I think your find -exec has more problems than that.
Using {} inside the command string "cd '{}' ..." is risky. If there is a ' inside the file name things will break and might execcute unexpected commands.
$() will be expanded by bash before find even runs. So ls -t *.pdf | tail -2 will only be executed once in the top directory . instead of once for each found directory. rm will (try to) delete the same file for each found directory.
rm "$(ls -t *.pdf | tail -2)" will not work if ls lists more than one file. Because of the quotes both files would be listed in one argument. Therefore, rm would try to delete one file with the name first.pdf\nsecond.pdf.
I'd suggest
cmd='cd "$1" && ls -t *.pdf | tail -n2 | sed "s/./\\\\&/g" | xargs rm'
find . -type d -name bak -exec bash -c "$cmd" -- {} \;
You have a more fundamental problem; because you are using the weaker double quotes around the entire script, the $(...) command substitution will be interpreted by the shell which parses the find command, not by the bash shell you are starting, which will only receive a static string containing the result from the command substitution.
If you switch to single quotes around the script, you get most of it right; but that would still fail if the file name you find contains a double quote (just like your attempt would fail for file names with single quotes). The proper fix is to pass the matching files as command-line arguments to the bash subprocess.
But a better fix still is to use -execdir so that you don't have to pass the directory name to the subshell at all:
find . -type d -name "bak" \
-execdir bash -c 'ls -t *.pdf | tail -2 | xargs -r rm' \;
This could stll fail in funny ways because you are parsing ls which is inherently buggy.
You are explicitely asking for find -exec. Usually I would just concatenate find -exec find -delete but in your case only two files should be deleted. Therefore the only method is running subshell. Socowi already gave nice solution, however if your file names do not contain tabulator or newlines, another workaround is find while read loop.
This will sort files by mtime
find . -type d -iname 'bak' | \
while read -r dir;
do
find "$dir" -maxdepth 1 -type f -iname '*.pdf' -printf "%T+\t%p\n" | \
sort | head -n2 | \
cut -f2- | \
while read -r file;
do
rm "$file";
done;
done;
The above find while read loop as "one-liner"
find . -type d -iname 'bak' | while read -r dir; do find "$dir" -maxdepth 1 -type f -iname '*.pdf' -printf "%T+\t%p\n" | sort | head -n2 | cut -f2- | while read -r file; do rm "$file"; done; done;
find while read loop can also handle NUL terminated file names. However head can not handle this, so I did improve other answers and made it work with nontrivial file names (only GNU + bash)
replace 'realpath' with rm
#!/bin/bash
rm_old () {
find "$1" -maxdepth 1 -type f -iname \*.$2 -printf "%T+\t%p\0" | sort -z | sed -zn 's,\S*\t\(.*\),\1,p' | grep -zim$3 \.$2$ | xargs -0r realpath
}
export -f rm_old
find -type d -iname bak -execdir bash -c 'rm_old "{}" pdf 2' \;
However bash -c might still exploitable, to make it more secure let stat %N do the quoting
#!/bin/bash
rm_old () {
local dir="$1"
# we don't like eval
# eval "dir=$dir"
# this works like eval
dir="${dir#?}"
dir="${dir%?}"
dir="${dir//"'$'\t''"/$'\011'}"
dir="${dir//"'$'\n''"/$'\012'}"
dir="${dir//$'\047'\\$'\047'$'\047'/$'\047'}"
find "$dir" -maxdepth 1 -type f -iname \*.$2 -printf '%T+\t%p\0' | sort -z | sed -zn 's,\S*\t\(.*\),\1,p' | grep -zim$3 \.$2$ | xargs -0r realpath
}
find -type d -iname bak -exec stat -c'%N' {} + | while read -r dir; do rm_old "$dir" pdf 2; done
How do I modify the below command in a way that the search and the copy are performed based on a file's mimetype instead of using an extension such as (mp4, mkv, docx, etc)?
find . -name '*.wmv' -exec cp {} /media/backup/ \;
Like this, for only video files:
find . -type f -exec bash -c '
file -i "$1" | grep -q ": video/" && mv "$1" /media/backup/
' -- {} \;
file(1) — determine file type
Could you try that solution?
cp `find . -type f -exec file --mime-type {$1} \; | grep ": video/" | awk -F ":" '{print $1}'` /media/backup/
The cp command has to copy each item from the list (see the expression in the ``) to the folder "/media/backup/".
The find expression in the `` have to return the list of files whose MIME type is "video".
The file --mime-type {$1} expression has to return the MIME data for each file. (Example of output data: "./video.mkv: video/x-matroska")
The grep command is needed to leave only files that have the pattern ": video/". (I hope the target files do not contain that pattern in their paths and names).
The awk expression is needed to get only the path and file name (without MIME data).
I tried both
find . -type f -exec bash -c '
file -i "$1" | grep -q "video/" && mv "$1" /media/backup/
' -- {} \;
and
find . -type f -exec sh -c 'file -b --mime-type "$1" | grep -q "video" ' Find {} \; -a -exec mv -t /media/backup {} \;
The first code also moved some audio files whereas, the second one did not.
Consequently, the second code works better for me.
Anyway, thank you guys for your help.
I have a list of file names inside filenames.txt like the following:
1.sql
2.sql
3.sql
..
..
500.sql
I want to search for the file names in a directory and its sub-directories like the following:
Dir1/1.sql
Dir1/2.sql
Dir2/3.sql
Dir3/4.sql
Dir4/5.sql
Dir4/6.sql
..
..
etc
and copy the founded files to another directory.
I tried:
$ for i in `cat filenames.txt`; do `find ./* -type f -printf "%f\n"|grep -ie "$i" && cp -t "$i" /home/user/other_directory/"$i"`; done
but this doesn't work.
because the first argument of cp is "$i" (the matching pattern) and not the filename
find . -type f -iname "$i" -exec cp -t "{}" /home/user/other_directory \;
I would like to prepend a string of text to the beginning of each file in a directory. The string is uniwisc.
When I run the script:
#!/bin/sh
url="ftp://rammftp.cira.colostate.edu/Lindsey/spc/ir/"
wget -r -nd --no-parent -nc -P /awips2/edex/data/goes14/ $url
find /awips2/edex/data/goes14/ -type f -exec cp {} /awips2/edex/data/uniwisc/ \;
for f in /awips2/edex/data/uniwisc/*;
do
f="$(basename $f)"
mv "$f" "uniwisc.$f"
done;
find /awips2/edex/data/uniwisc/ -type f -mmin -6 -exec mv {} /awips2/edex/data/manual/ \;
exit 0
I get the error mv: cannot stat '<filenames>' "No such file or directory.
There are a number of different ways that you can do that.
Using paramater expansion, which is built into the bash shell:
for f in <dir path>/*; do
mv "$f" "${f%/*}/uniwisc.${f##*/}"
done
Using the rename command:
rename 's!^!uniwisc.!' *
Using the basename, as CodeGnome suggested:
for f in <dir path>/*; do
mv "$f" "$(dirname "$f")/uniwisc.$(basename "$f")"
done
I was going to write more methods, but there are a lot of them and they don't change significantly. Personally, I'd use the rename command in that situation.
I want to copy files found by find (with exec cp option) but, i'd like to change name of those files - e.g find ... -exec cp '{}' test_path/"test_"'{}' , which to my test_path should copy all files found by find but with prefix 'test'. but it ain't work.
I'd be glad if anyone could give me some ideas how to do it.
best regards
for i in `find . -name "FILES.EXT"`; do cp $i test_path/test_`basename $i`; done
It is assumed that you are in the directory that has the files to be copied and test_path is a subdir of it.
if you have Bash 4.0 and assuming you are find txt files
cd /path
for file in ./**/*.txt
do
echo cp "$file" "/test_path/test${file}"
done
of with GNU find
find /path -type f -iname "*.txt" | while read -r -d"" FILE
do
cp "$FILE" "test_${FILE}"
done
OR another version of GNU find+bash
find /path -type f -name "*txt" -printf "cp '%p' '/tmp/test_%f'\n" | bash
OR this ugly one if you don't have GNU find
$ find /path -name '*.txt' -type f -exec basename {} \; | xargs -I file echo cp /path/file /destination/test_file
You should put the entire test_path/"test_"'{}' in ""
Like:
find ... -exec cp "{}" "test_path/test_{}" \;
I would break it up a bit, like this;
for line in `find /tmp -type f`; do FULL=$line; name=`echo $line|rev|cut -d / -f -1|rev` ; echo cp $FULL "new/location/test_$name" ;done
Here's the output;
cp /tmp/gcc.version new/location/test_gcc.version
cp /tmp/gcc.version2 new/location/test_gcc.version2
Naturally remove the echo from the last part, so it's not just echo'ng what it woudl of done and running cp