I'm able to use the 'rename' command to add the missing character to all filenames in the current directory like this:
echo "Renaming files..."
rename -v "s/^abcd124(.+)/abcd1234$1/" *.wav.gz;
echo "Done."
However, I'd like to do this for the current directory and all its subdirectories. I tried this:
echo "Renaming files..."
for dir in $(find ./ -type d); do
rename -v "s/^$dir\/abcd124(.+)/$dir\/abcd1234$1/" *.wav.gz;
done;
echo "Done."
However, if the $dir variable contains any of these special characters: {}[]()^$.|*+?\ then they are not escaped properly with \ and my script fails.
What would be the best way to solve this problem? Also, what do you guys think of using awk to solve this problem (advantages/disadvantages?)
You can also try:
find . -name "*.wav.gz" | xargs rename -v "s/abcd124*/abcd1234$1/"
It works on newer Unix systems with "xargs" command available. Note that I edited the regular expression slightly.
Try:
find ./ -type d -execdir rename -v "s/^abcd124(.+)/abcd1234\1/" *.wav.gz ";"
Find does already provide an iterator over your files - you don't need for around it or xargs behind , which are often seen. Well - in rare cases, they might be helpful, but normally not.
Here, -execdir is useful. Gnu-find has it; I don't know if your find has it too.
But you need to make sure not to have a *.wav.gz-file in the dir you're starting this command, because else your shell will expand it, and hand the expanded names over to rename.
Note: I get an warning from rename, that I should replace \1 with $1 in the regex, but if I do so, the pattern isn't catched. I have to use \1 to make it work.
Here is another approach. Why at all search for directories, if we search for wav.gz-files?
find . -name "*.wav.gz" -exec rename -v "s/^abcd124(.+)/abcd1234\1/" {} ";"
In bash 4:
shopt -s globstar
rename -v "s/^$dir\/abcd124(.+)/$dir\/abcd1234$1/" **/*.wav.gz;
Just be aware that Gentoo Linux has its rename utility points to
http://developer.berlios.de/project/showfiles.php?group_id=413
by its ebuild
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sys-apps/rename/rename-1.3.ebuild?view=markup
for Debian or maybe Ubuntu,
rename is /usr/bin/prename, which is a perl script
See rename --help before your move.
Related
I can use the cp or mv command to copy/mv files to a new folder manually but while in a for loop, it fails.
I've tried various ways of doing this and none seem to work. The most frustrating part is it works when run locally.
A simple version of what I'm trying to do is shown below:
#!bin/bash
#Define path variables
source_dir=/home/me/loop
destination_dir=/home/me/loop/new
#Change working dir
cd "$source_dir"
#Step through source_dir for each .txt. file
for f in *.txt
do
# If the txt file was modified within the last 300 minutes...
if [[ $(find "$f" -mmin -300) ]]
then
# Add breaks for any spaces in filenames
f="${f// /\\ }"
# Copy file to destination
cp "$source_dir/$f $destination_dir/"
fi
done
Error message is:
cp: missing destination file operand after '/home/me/loop/first\ second.txt /home/me/loop/new/'
Try 'cp --help' for more information.
However, I can manually run:
mv /home/me/loop/first\ second.txt /home/me/loop/new/
and it works fine.
I get the same error using cp and similar errors using rsync so I'm not sure what I'm doing wrong...
cp "$source_dir/$f $destination_dir/"
When you surround both arguments with double quotes you turn them into one argument with an embedded space. Quote them separately.
cp "$source_dir/$f" "$destination_dir/"
There's no do anything special for spaces beforehand. The quoting already ensures files with whitespace are handled correctly.
# Add breaks for any spaces in filenames
f="${f// /\\ }"
Let's take a step back, though. Looping over all *.txt files and then checking each one with find is overly complicated. find already loops over multiple files and does arbitrary things to those files. You can do everything in this script in a single find command.
#!bin/bash
source_dir=/home/me/loop
destination_dir=/home/me/loop/new
find "$source_dir" -name '*.txt' -mmin -300 -exec cp -t "$destination_dir" {} +
You need to divide it in to two strings, like this:
cp "$source_dir/$f" "$destination_dir/"
by having as one you are basically telling cp that the entire line is the first parameter, where it is actually two (source and destination).
Edit: As #kamil-cuk and #aaron states there are better ways of doing what you try to do. Please read their comments
I have a directory with this structure:
root
|-dir1
| |-pred_20181231.csv
|
|-dir2
| |-pred_20181234.csv
...
|-dir84
|-pred_2018123256.csv
I want to run a command that will rename all the pred_XXX.csv files to pred.csv.
How can I easily achieve that?
I have looked into the rename facility but I do not understand the perl expression syntax.
EDIT: I tried with this code: rename -n 's/\training_*.csv$/\training_history.csv/' *.csv but it did not work
Try with this command:
find root -type f -name "*.csv" -exec perl-rename 's/_\d+(\.csv)/$1/g' '{}' \;
Options used:
-type f to specify file or directory.
-name "*.csv" to only match files with extension csv
-exec\-execdir to execute a command, in this case, perl-rename
's/_\d+(\.csv)/$1/g' search a string like _20181234.csv and replace it with .csv, $1 means first group found.
NOTE
Depending in your S.O. you could use just rename instead of perl-rename.
Use some shell looping:
for file in **/*.csv
do
echo mv "$(dirname "$file")/$(basename "$file")" "$(dirname "$file")/pred.csv"
done
On modern shells ** is a wildcard that matches multiple directories in a hierarchy, an alternative to find, which is a fine solution too. I'm not sure if this should instead be /**/*.csv or /root/**/*.csv based on tree you provided, so I've put echo before the 'mv' to see what it's about to do. After making sure this is going to do what you expect it to do, remove the echo.
From the current directory I have multiple sub directories:
subdir1/
001myfile001A.txt
002myfile002A.txt
subdir2/
001myfile001B.txt
002myfile002B.txt
where I want to strip every character from the filenames before myfile so I end up with
subdir1/
myfile001A.txt
myfile002A.txt
subdir2/
myfile001B.txt
myfile002B.txt
I have some code to do this...
#!/bin/bash
for d in `find . -type d -maxdepth 1`; do
cd "$d"
for f in `find . "*.txt"`; do
mv "$f" "$(echo "$f" | sed -r 's/^.*myfile/myfile/')"
done
done
however the newly renamed files end up in the parent directory
i.e.
myfile001A.txt
myfile002A.txt
myfile001B.txt
myfile002B.txt
subdir1/
subdir2/
In which the sub-directories are now empty.
How do I alter my script to rename the files and keep them in their respective sub-directories? As you can see the first loop changes directory to the sub directory so not sure why the files end up getting sent up a directory...
Your script has multiple problems. In the first place, your outer find command doesn't do quite what you expect: it outputs not only each of the subdirectories, but also the search root, ., which is itself a directory. You could have discovered this by running the command manually, among other ways. You don't really need to use find for this, but supposing that you do use it, this would be better:
for d in $(find * -maxdepth 0 -type d); do
Moreover, . is the first result of your original find command, and your problems continue there. Your initial cd is without meaningful effect, because you're just changing to the same directory you're already in. The find command in the inner loop is rooted there, and descends into both subdirectories. The path information for each file you choose to rename is therefore stripped by sed, which is why the results end up in the initial working directory (./subdir1/001myfile001A.txt --> myfile001A.txt). By the time you process the subdirectories, there are no files left in them to rename.
But that's not all: the find command in your inner loop is incorrect. Because you do not specify an option before it, find interprets "*.txt" as designating a second search root, in addition to .. You presumably wanted to use -name "*.txt" to filter the find results; without it, find outputs the name of every file in the tree. Presumably you're suppressing or ignoring the error messages that result.
But supposing that your subdirectories have no subdirectories of their own, as shown, and that you aren't concerned with dotfiles, even this corrected version ...
for f in `find . -name "*.txt"`;
... is an awfully heavyweight way of saying this ...
for f in *.txt;
... or even this ...
for f in *?myfile*.txt;
... the latter of which will avoid attempts to rename any files whose names do not, in fact, change.
Furthermore, launching a sed process for each file name is pretty wasteful and expensive when you could just use bash's built-in substitution feature:
mv "$f" "${f/#*myfile/myfile}"
And you will find also that your working directory gets messed up. The working directory is a characteristic of the overall shell environment, so it does not automatically reset on each loop iteration. You'll need to handle that manually in some way. pushd / popd would do that, as would running the outer loop's body in a subshell.
Overall, this will do the trick:
#!/bin/bash
for d in $(find * -maxdepth 0 -type d); do
pushd "$d"
for f in *.txt; do
mv "$f" "${f/#*myfile/myfile}"
done
popd
done
You can do it without find and sed:
$ for f in */*.txt; do echo mv "$f" "${f/\/*myfile/\/myfile}"; done
mv subdir1/001myfile001A.txt subdir1/myfile001A.txt
mv subdir1/002myfile002A.txt subdir1/myfile002A.txt
mv subdir2/001myfile001B.txt subdir2/myfile001B.txt
mv subdir2/002myfile002B.txt subdir2/myfile002B.txt
If you remove the echo, it'll actually rename the files.
This uses shell parameter expansion to replace a slash and anything up to myfile with just a slash and myfile.
Notice that this breaks if there is more than one level of subdirectories. In that case, you could use extended pattern matching (enabled with shopt -s extglob) and the globstar shell option (shopt -s globstar):
$ for f in **/*.txt; do echo mv "$f" "${f/\/*([!\/])myfile/\/myfile}"; done
mv subdir1/001myfile001A.txt subdir1/myfile001A.txt
mv subdir1/002myfile002A.txt subdir1/myfile002A.txt
mv subdir1/subdir3/001myfile001A.txt subdir1/subdir3/myfile001A.txt
mv subdir1/subdir3/002myfile002A.txt subdir1/subdir3/myfile002A.txt
mv subdir2/001myfile001B.txt subdir2/myfile001B.txt
mv subdir2/002myfile002B.txt subdir2/myfile002B.txt
This uses the *([!\/]) pattern ("zero or more characters that are not a forward slash"). The slash has to be escaped in the bracket expression because we're still inside of the pattern part of the ${parameter/pattern/string} expansion.
Maybe you want to use the following command instead:
rename 's#(.*/).*(myfile.*)#$1$2#' subdir*/*
You can use rename -n ... to check the outcome without actually renaming anything.
Regarding your actual question:
The find command from the outer loop returns 3 (!) directories:
.
./subdir1
./subdir2
The unwanted . is the reason why all files end up in the parent directory (that is .). You can exclude . by using the option -mindepth 1.
Unfortunately, this was onyl the reason for the files landing in the wrong place, but not the only problem. Since you already accepted one of the answers, there is no need to list them all.
a slight modification should fix your problem:
#!/bin/bash
for f in `find . -maxdepth 2 -name "*.txt"`; do
mv "$f" "$(echo "$f" | sed -r 's,[^/]+(myfile),\1,')"
done
note: this sed uses , instead of / as the delimiter.
however, there are much faster ways.
here is with the rename utility, available or easily installed wherever there is bash and perl:
find . -maxdepth 2 -name "*.txt" | rename 's,[^/]+(myfile),/$1,'
here are tests on 1000 files:
for `find`; do mv 9.176s
rename 0.099s
that's 100x as fast.
John Bollinger's accepted answer is twice as fast as the OPs, but 50x as slow as this rename solution:
for|for|mv "$f" "${f//}" 4.316s
also, it won't work if there is a directory with too many items for a shell glob. likewise any answers that use for f in *.txt or for f in */*.txt or find * or rename ... subdir*/*. answers that begin with find ., on the other hand, will also work on directories with any number of items.
I'm looping through certain files (all files starting with MOVIE) in a folder with this bash script code:
for i in MY-FOLDER/MOVIE*
do
which works fine when there are files in the folder. But when there aren't any, it somehow goes on with one file which it thinks is named MY-FOLDER/MOVIE*.
How can I avoid it to enter the things after
do
if there aren't any files in the folder?
With the nullglob option.
$ shopt -s nullglob
$ for i in zzz* ; do echo "$i" ; done
$
for i in $(find MY-FOLDER/MOVIE -type f); do
echo $i
done
The find utility is one of the Swiss Army knives of linux. It starts at the directory you give it and finds all files in all subdirectories, according to the options you give it.
-type f will find only regular files (not directories).
As I wrote it, the command will find files in subdirectories as well; you can prevent that by adding -maxdepth 1
Edit, 8 years later (thanks for the comment, #tadman!)
You can avoid the loop altogether with
find . -type f -exec echo "{}" \;
This tells find to echo the name of each file by substituting its name for {}. The escaped semicolon is necessary to terminate the command that's passed to -exec.
for file in MY-FOLDER/MOVIE*
do
# Skip if not a file
test -f "$file" || continue
# Now you know it's a file.
...
done
I run unsuccessfully in Mac
mv .* *
and
mv .* ./*
My files disappeared into thin air.
How can you convert dot-files to non-dotfiles safely?
for i in `ls -d .*`; do mv $i "`echo $i | sed 's/^.//'`"; done
or, much easier,
rename 's/^.//' `ls -d .*`
if your system have got it.
In zsh, you could just use .* safely, but in bash you'll have to use ls -d .*
You can't use mv to rename multiple files like that. What you want is mmv (get it here).
mmv .\* \#1
You have to escape the asterisk to prevent bash from expanding it. Use the -n flag to do a test run to make sure what will happen is what you want.
You could also do this in shell scripting but I much prefer mmv because the -n flag shows what it would do. You'd have to alter your script to echo instead of mv, which seems more dangerous than dropping the -n flag (especially when you get more complicated.
The tricky part about this is selecting dotfiles without selecting "." and "..".
ls .??* is sometimes used for this, since it forces the filenames to be three or more characters long. There is a risk though, of overlooking a dotfile with a short name, such as ".x"
ls -d .* prevents directories from being expanded, but it doesn't filter out "." or ".."
The find command could be used, as in find . -maxdepth 1 -type f -name '.*'. The maxdepth limits it to the current directory and not subdirectories. The -type f limits it to files, eliminating directories such as "." and "..". Then again, maybe you want to rename the .ssh directory to ssh.
Here's an alternative that selects dotfiles while avoiding "." and "..".
ls -A | sed -n 's/^\.\(.*\)/mv ".\1" "\1"/p' | bash
The -A lists all files and dotfiles, yet eliminates "." and ".." for us. Then the sed command selects only those lines with "." as the first character, and prints out appropriate "mv" commands, complete with quotes in case you have a bizarre dotfilename with a space in it.
Run it without the "| bash" first, to see what mv commands are generated.
i don't know what type of system you're on, but it looks unix like, i would do
ls -1 .?* | cut -b1- | xargs -i{} mv .{} {}
this lists, everything that starts with a ., but isn't . or .., then cut the first column off, then pipe that list to a move command
In Linux, there is usually a rename utility available (a perl script, if I am not mistaken):
rename 's/^.//' .*
It is available on a Mac. You can install it by following tips at here.
Even simpler:
for x in .*; do mv $x ${x/./}; done