usage error when moving files of one extension to a directory - bash

I have files with extension .mp3 that are in different folders within a directory, and I need to move them all to one directory to work with. I have looked at multiple tutorials and questions on SO, and no matter what I try, I either get
usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
or "No such file or directory".
There are way too many to into each individual folder, but for now I cd into one of the folders. With this:
mv *.mp3 /Users/myname/Volumes/LaCie/model/folder/media
I get the "usage" error above. I looked at Moving files to a directory and tried:
find . | grep ".mp3" | xargs mv /Users/myname/Volumes/LaCie/model/folder/media
and get same error.
What am I doing wrong? What is the correct syntax? Also, will I be able to extract the mp3 files and move them if I'm in a directory that contains the directory with the files, but not in that directory itself? I appreciate insights into this. Thanks.
EDIT: A major part of this issue was that the path when using /Volumes starts with /Volumes. I was doing /Users/myname/Volumes and that was one reason I had so much trouble.

The problem with the way you're using xargs is that by default, xargs will append the arguments to the end of the command string you've provided it. So you'll end up running a bunch of mv commands that look like this:
mv /Users/myname/Volumes/LaCie/model/folder/media foo.mp3
You can fix that by telling xargs where to place the arguments within the command:
<other commands> | xargs -I{} mv {} /Users/myname/Volumes/LaCie/model/folder/media
The -I option lets you provide any arbitrary string as a placeholder for where the args should go. I used {} just because that seems to be the conventional token that you see used in similar contexts (such as with the -exec option of find, as shown below).
But there's an easier way to do it, using the find command's -exec option:
find . -name '*.mp3' -exec mv {} /Users/myname/Volumes/LaCie/model/folder/media \;
Also note the -name '*.mp3' part, which lets you get rid of the | grep ".mp3" part.
Lastly, just to be safe, I'd personally put a / at the end of your destination path. If the media directory doesn't exist in /Users/myname/Volumes/LaCie/model/folder, or if a non-directory item (such as a regular file or a symlink) named media exists in that location, then the find command above will happily just move all your mp3 files, one at a time, to that folder, creating a file named media there each time. And you will have lost all of your mp3 files except for the last one, which will now be a file named media.
However, with a trailing /, if media is not a directory, the mv commands will fail with an error saying so. So the revised command would be:
find . -name '*.mp3' -exec mv {} /Users/myname/Volumes/LaCie/model/folder/media/ \;
Update: Per Gordon Davisson's comment below, you should also consider adding -i or -n to the mv command, to avoid accidentally overwriting files with duplicate names. For example, if you have a/foo.mp3 and b/foo.mp3, the above command will overwrite one with the other. The -i option will cause mv to prompt you to confirm each file move, whereas the -n option (a.k.a. --no-clobber) will prevent mv from overwriting a file if a file with the same name already exists.

There are different implementations and versions of mv. You can check the allowed syntax of your version using man mv.
If you have GNU mv you could use mv -t target/dir *.mp3.
Most implementations should support mv *.mp3 target/dir.
If your mv only supports the absolute minimum of mv source target with exactly one source and one target file you can use the following command which should always work if target/dir/ exists.
for i in *.mp3; do mv "$i" "target/dir/$i"; done

Related

search and rename files

I've found a bad habit file naming among the users of the QNAP Linux NAS system I'm administering.
We have a Mac OS network, and some folders and files are named using the "/" character that I know it's causing problems to Linux file system.
As a matter of fact, from the Mac OS side, the files containing "/" simply disappear, since QTS replaces automatically all instances with "/" with ":".
I'd like to search and rename filenames from ":" to "_".
Discussing on the web I've found that I can SSH connect to the Linux NAS from my Mac with Terminal and perform a script like this:
for f in $(find /share/Public/demofind -name "*:*"); do mv $f ${f/:/_}; done
assuming that the files are in /demofind folder.
I launched the script, but got this error:
[/share/Public] # for f in $(find /share/Public/demofind/ -name "*:*"); do mv $f ${f/:/_}; done
mv: unable to rename `/share/Public/demofind/Redazionali': No such file or directory
mv: unable to rename `01:03:19.pdf': No such file or directory
mv: unable to rename `/share/Public/demofind/Redazionali': No such file or directory
mv: unable to rename `06:09:19.pdf': No such file or directory
[/share/Public] # for f in $(find /share/Public/demofind/ -name "*:*"); do mv $f ${f/:/_}; done
mv: unable to rename `/share/Public/demofind/Redazionali': No such file or directory
mv: unable to rename `01:03:19.pdf': No such file or directory
mv: unable to rename `/share/Public/demofind/Redazionali': No such file or directory
mv: unable to rename `06:09:19.pdf': No such file or directory
By the way, the files to rename have a syntax like this: "Redazionali 06:09:19.pdf"
The NAS seems to be running BusyBox v1.01 (2021.12.12-04:24+0000) so I would need a solution which is compatible with this platform.
The immediate problem is that you have broken quoting but your code has other problems too.
The robust solution would be
find /share/Public/demofind -name "*:*" -depth -exec bash -c '
for f; do mv "$f" "${f//:/_}"; done' _ {} +
Putting the for loop inside find -exec works around pesky problems around file names with unusual characters in them, not just spaces (though your for loop would also inherently break on just single spaces, too; you would have to separately tell the shell to not perform whitespace tokenization if you used the output from find in a command substitution, but again, the proper fix is to simply not do that).
For details, please review https://mywiki.wooledge.org/BashFAQ/020
The -depth option to find says to traverse the directory tree depth-first; this is important when renaming, so that you don't end up renaming directories before you rename the files within them, which then leads to errors when the file you want to rename no longer exists where it was originally found.
The parameter expansion ${f//:/_} is a Bash feature; if you don't have Bash, you have to use an external tool (or a really hairy function around the more pedestrian parameter expansion facilities of POSIX sh; but let's not go there).
find /share/Public/demofind -name "*:*" -depth -exec sh -c '
for f; do mv "$f" "$(echo "$f" | tr : _)"; done' _ {} +
If your platform doesn't even support find -exec you can manually loop over the output from find but again, see the FAQ link above for a number of caveats.
find /share/Public/demofind -name "*:*" -depth -print |
while IFS='' read -r f; do
mv "$f" "$(echo "$f" | tr : _)"
done

Operating on multiple specific folders at once with cp and rm commands

I'm new to linux (using bash) and I wanted to ask about something that I do often while I work, I'll give two examples.
Deleting multiple specific folders inside a certain directory.
Copying multiple specific folders into a ceratin directory.
I succesfully done this with files, using find with some regex and then using -exec and -delete. But for folders I found it more problematic, because I had problem pipelining the list of folders I got to the cp/rm command succescfully, each time getting the "No such file or directory error".
Looking online I found the following command (in my case for copying all folders starting with a Z):
cp -r $(ls -A | grep "Z*") destination
But when I execute it it says nothing and the prompt won't show up again until I hit Ctrl+C and nothing is copied.
How can I achieve what I'm looking for? For both cp and rm.
Thanks in advance!
First of all, you are trying to grep "Z*" but it means you are looking for Z, ZZ, ZZZZ, ZZZZZ ?
also try to execute ls -A - you will get multiple columns. I think need at least ls -1A to print result one per line.
So for your command try something like:
cp -r $(ls -1A|grep "^p") destination
or
cp -r $(ls -1A|grep "^p") -t destination
But all the above is just to correct syntax of your example.
It is much better to use find. Just in case try to put target directory in quotas like:
find <PATH_FROM> -type d -exec cp -r \"{}\" -t target \;

bash: moving files to original directory based on filename?

I've got a bunch of subdirectories with a couple thousand PNG files that will be sent through Photoshop, creating PSD files. Photoshop can only output those to a single folder, and I want to move each one back to their original directory - so the new file foo_bar_0005.psd should go to where foo_bar_0005.png already is. Every filename only exists once.
Can somebody help me with this? I'm on OSX.
You might start from this minimal script:
#!/bin/bash
search_dir="search/png/from/this/directory/"
psd_dir="path/to/psd/directory/"
for psd_file in "$psd_dir"*.psd; do
file_name="$(echo $psd_file | sed 's/.*\/\(.*\).psd$/\1/g')"
png_dir="$(find $search_dir -name $file_name.png | grep -e '.*/' -o)"
mv $psd_file $png_dir
done
But note that this script doesn't include any error handlers e.g. file collision issue, file not found issue, etc.
Each file found with this find is piped to a Bash command that successively make the psd conversion and move the .psd to the .png original directory.
psd_dir=/psd_dir/
export psd_dir
find . -type f -name '*.png' | xargs -L 1 bash -c 'n=${1##*/}; echo photoshop "$1" && echo mv ${psd_dir}${n%.png}.psd ${1%/*}/; echo' \;
The echo are here to give you an overview of the result.
You should remove them to launch the real photoshop command.

Can I limit the recursion when copying using find (bash)

I have been given a list of folders which need to be found and copied to a new location.
I have basic knowledge of bash and have created a script to find and copy.
The basic command I am using is working, to a certain degree:
find ./ -iname "*searchString*" -type d -maxdepth 1 -exec cp -r {} /newPath/ \;
The problem I want to resolve is that each found folder contains the files that I want, but also contains subfolders which I do not want.
Is there any way to limit the recursion so that only the files at the root level of the found folder are copied: all subdirectories and files therein should be ignored.
Thanks in advance.
If you remove -R, cp doesn't copy directories:
cp *searchstring*/* /newpath
The command above copies dir1/file1 to /newpath/file1, but these commands copy it to /newpath/dir1/file1:
cp --parents *searchstring*/*(.) /newpath
for GNU cp and zsh
. is a qualifier for regular files in zsh
cp --parents dir1/file1 dir2 copies file1 to dir2/dir1 in GNU cp
t=/newpath;for d in *searchstring*/;do mkdir -p "$t/$d";cp "$d"* "$t/$d";done
find *searchstring*/ -type f -maxdepth 1 -exec rsync -R {} /newpath \;
-R (--relative) is like --parents in GNU cp
find . -ipath '*searchstring*/*' -type f -maxdepth 2 -exec ditto {} /newpath/{} \;
ditto is only available on OS X
ditto file dir/file creates dir if it doesn't exist
So ... you've been given a list of folders. Perhaps in a text file? You haven't provided an example, but you've said in comments that there will be no name collisions.
One option would be to use rsync, which is available as an add-on package for most versions of Unix and Linux. Rsync is basically an advanced copying tool -- you provide it with one or more sources, and a destination, and it makes sure things are synchronized. It knows how to copy things recursively, but it can't be told to limit its recursion to a particular depth, so the following will copy each item specified to your target, but it will do so recursively.
xargs -L 1 -J % rsync -vi -a % /path/to/target/ < sourcelist.txt
If sourcelist.txt contains a line with /foo/bar/slurm, then the slurm directory will be copied in its entiriety to /path/to/target/slurm/. But this would include directories contained within slurm.
This will work in pretty much any shell, not just bash. But it will fail if one of the lines in sourcelist.txt contains whitespace, or various special characters. So it's important to make sure that your sources (on the command line or in sourcelist.txt) are formatted correctly. Also, rsync has different behaviour if a source directory includes a trailing slash, and you should read the man page and decide which behaviour you want.
You can sanitize your input file fairly easily in sh, or bash. For example:
#!/bin/sh
# Avoid commented lines...
grep -v '^[[:space:]]*#' sourcelist.txt | while read line; do
# Remove any trailing slash, just in case
source=${line%%/}
# make sure source exist before we try to copy it
if [ -d "$source" ]; then
rsync -vi -a "$source" /path/to/target/
fi
done
But this still uses rsync's -a option, which copies things recursively.
I don't see a way to do this using rsync alone. Rsync has no -depth option, as find has. But I can see doing this in two passes -- once to copy all the directories, and once to copy the files from each directory.
So I'll make up an example, and assume further that folder names do not contain special characters like spaces or newlines. (This is important.)
First, let's do a single-pass copy of all the directories themselves, not recursing into them:
xargs -L 1 -J % rsync -vi -d % /path/to/target/ < sourcelist.txt
The -d option creates the directories that were specified in sourcelist.txt, if they exist.
Second, let's walk through the list of sources, copying each one:
# Basic sanity checking on input...
grep -v '^[[:space:]]*#' sourcelist.txt | while read line; do
if [ -d "$line" ]; then
# Strip trailing slashes, as before
source=${line%%/}
# Grab the directory name from the source path
target=${source##*/}
rsync -vi -a "$source/" "/path/to/target/$target/"
fi
done
Note the trailing slash after $source on the rsync line. This causes rsync to copy the contents of the directory, rather than the directory.
Does all this make sense? Does it match your requirements?
You can use find's ipath argument:
find . -maxdepth 2 -ipath './*searchString*/*' -type f -exec cp '{}' '/newPath/' ';'
Notice the path starts with ./ to match find's search directory, ends with /* in order to exclude files in the top level directory, and maxdepth is set to 2 to only recurse one level deep.
Edit:
Re-reading your comments, it seems like you want to preserve the directory you're copying from? E.g. when searching for foo*:
./foo1/* ---> copied to /newPath/foo1/* (not to /newPath/*)
./foo2/* ---> copied to /newPath/foo2/* (not to /newPath/*)
Also, the other requirement is to keep maxdepth at 1 for speed reasons.
(As pointed out in the comments, the following solution has security issues for specially crafted names)
Combining both, you could use this:
find . -maxdepth 1 -type d -iname 'searchString' -exec sh -c "mkdir -p '/newPath/{}'; cp "{}/*" '/newPath/{}/' 2>/dev/null" ';'
Edit 2:
Why not ditch find altogether and use a pure bash solution:
for d in *searchString*/; do mkdir -p "/newPath/$d"; cp "$d"* "/newPath/$d"; done
Note the / at the end of the search string, causing only directories to be considered for matching.

Copy all files with a certain extension from all subdirectories

Under unix, I want to copy all files with a certain extension (all excel files) from all subdirectories to another directory. I have the following command:
cp --parents `find -name \*.xls*` /target_directory/
The problems with this command are:
It copies the directory structure as well, and I only want the files (so all files should end up in /target_directory/)
It does not copy files with spaces in the filenames (which are quite a few)
Any solutions for these problems?
--parents is copying the directory structure, so you should get rid of that.
The way you've written this, the find executes, and the output is put onto the command line such that cp can't distinguish between the spaces separating the filenames, and the spaces within the filename. It's better to do something like
$ find . -name \*.xls -exec cp {} newDir \;
in which cp is executed for each filename that find finds, and passed the filename correctly. Here's more info on this technique.
Instead of all the above, you could use zsh and simply type
$ cp **/*.xls target_directory
zsh can expand wildcards to include subdirectories and makes this sort of thing very easy.
From all of the above, I came up with this version.
This version also works for me in the mac recovery terminal.
find ./ -name '*.xsl' -exec cp -prv '{}' '/path/to/targetDir/' ';'
It will look in the current directory and recursively in all of the sub directories for files with the xsl extension. It will copy them all to the target directory.
cp flags are:
p - preserve attributes of the file
r - recursive
v - verbose (shows you whats
being copied)
I had a similar problem. I solved it using:
find dir_name '*.mp3' -exec cp -vuni '{}' "../dest_dir" ";"
The '{}' and ";" executes the copy on each file.
I also had to do this myself. I did it via the --parents argument for cp:
find SOURCEPATH -name filename*.txt -exec cp --parents {} DESTPATH \;
In 2022 the zsh solution also works in Linux Bash:
cp **/*.extension /dest/dir
works as expected.
find [SOURCEPATH] -type f -name '[PATTERN]' |
while read P; do cp --parents "$P" [DEST]; done
you may remove the --parents but there is a risk of collision if multiple files bear the same name.
On macOS Ventura 13.1, on zsh, I saw the following error when there were too many files to copy, saw the following error:
zsh: argument list too long: cp
Had to use find command along with cp to get the files copied to my destination:
find ./module/*/src -name \*.java -print | while read filelocation; do cp $filelocation mydestinationlocation; done

Resources