In the process of posting this question I've been experimenting with my code, I've come up with something that works, but my restless intellect wants to be a better programmer, not just solve the problem at hand, so I'm posting Script 1.0 and Script 1.1 to ask the community, Why does the change to line 3 make it work?
I'm copying files from a server and renaming them. The copying is going fine; the renaming isn't. These filenames have spaces. cp handles them competently; mv seems throws an error. I expected putting "" around the variable name to resolve the issue, but it still gives an error: mv: rename /src/foo.wav to zzz - /dest/foo.wav - copied 201403261800.wav: No such file or directory. I'm trying to prepend "zzz" to the filename, but it's getting prepended to the destination path. Here's Script 1.0:
cd /src/
DATE=$(date +"%Y%m%d%H%M")
find . -type f -iname "*.wav" | while read file ; do
if [[ "$file" != *zzz* ]]; then
echo "Tranferring...";
cp "$file" /dest/
mv "$file" \zzz\ \-\ "$file"\ \-\ copied\ $DATE.wav
echo "Transfer complete.";
fi
done
(The if condition ensures that I don't re-copy anything I've previously copied and renamed.)
Changing the 3rd line makes it work. But why? Here's Script 1.1:
cd /src/
DATE=$(date +"%Y%m%d%H%M")
find *.* -iname "*.wav" | while read file ; do
if [[ "$file" != *zzz* ]]; then
echo "Tranferring...";
cp "$file" /dest/
mv "$file" \zzz\ \-\ "$file"\ \-\ copied\ $DATE.wav
echo "Transfer complete.";
fi
done
If you look at the output of the different find commands you'll see that find . -type f will return lines like this: ./passwd.
Now you can't use that as the target, as renaming ./passwd to zzz - ./passwd would try to move it to a zzz - . folder as passwd.
By the way, you can surround the whole target name with quotes, to make it look better (also you should not escape the first z anyway):
mv "$file" "zzz - $file - copied $DATE.wav"
Also you probably want to get rid of the .wav part in the middle of the file:
mv "$file" "zzz - ${file%.wav} - copied $DATE.wav"
The ${var%string} returns the value of $var with string removed from the end of it if present. You can find more about this looking for bash string manipulation.
Related
I am trying to write in shell script to run this..
I guess it requires for syntax or find syntax..
but I am stuck dealing with scan every folder..
I have tried with "find" -maxdepth 1 -name "*.jpg | mv " but failed...
for every jpg files in every dir (folder1, folder2, folder3...folder5...etc)
move files to target dir which is parent dir.
if file name duplicated, move to dup dir.
DIR tree looks like this
Something like
for f in folder*/*.jpg; do
if [ -e "$(basename "$f")" ]; then
mv "$f" dup/
else
mv "$f" .
fi
done
run from the parent directory. Just iterates over every jpg in the folder subdirectories, moving to one place or another depending on if a file with that name already exists or not.
Slightly more efficient bash version:
for f in folder*/*.jpg; do
if [[ -e ${f##*/} ]]; then
mv "$f" dup/
else
mv "$f" .
fi
done
Problem
I've got thousands of files with the format "^[[:digit:]]\{4\} - [[:alpha:]].*", for exampe: 7958 - a3ykof zyimeo3.txt. I'm trying to simply move them into folders alphabetically beginning with the first alpha-character after the hyphen.
I feel like I'm so close to getting this to happen the way I want but there's a (hopefully simple) problem.
I tested the commmand with echo first to make sure it grabs the correct information. Then I tried to execute it for real with mv. I've included some examples below based on this list of files:
1439 - a74389 josifj3oj.txt
3589 - Bfoei 839982 3il.txt
4719 - an38n8f n839mm20 mi02.txt
6398 - b39ji oij3o8 j2o.txt
9287 - A2984 j289jj9 oiw.txt
.... several thousand more files
Examples
This works
This lists all the files starting with the letter "a" (after the 4 digits-space-hyphen-space pattern in the beginning):
for i in "$(ls | grep -i "^[[:digit:]]\{4\} - a")"; do echo "$i"; done
This fails
This doesn't put all the files starting with the letter "a" (after the 4 digits-space-hyphen-space pattern) in the "A" folder:
for i in "$(ls | grep -i "^[[:digit:]]\{4\} - a")"; do mv "$i" A; done
I expected this second command to move each file named "#### - a*" or "#### - A*" to the folder named A. But it sees it as one big string/filename joined by "\n".
Here's an example error message:
mv: cannot stat '1439 - a74389 josifj3oj.txt\n9287 - A2984 j289jj9 oiw.txt\n2719 - an38n8f n839mm20 mi02.txt': No such file or directory
Does anybody know what I'm missing?
Edit
Between #alvits's answer and #chepner's and #courtlandj comments, what worked flawless for me was this:
for directory in {A..Z}; do
mkdir -p "$directory" &&
find . -iregex "./[0-9]* - ${directory}.*" -exec mv -t "$directory" {} +;
done
Here's the simplest way to do it.
for directory in {A..Z}; do
mkdir "$directory" &&
find . -iregex "./[0-9]* - ${directory}.*" -exec mv "{}" "$directory" \;
done
The for loop will query for filenames according to each directory they belong.
The find command will find the files and move them to the directory.
BASH has RE-like globbing, and sequence creation, built-in. You can make use of it something like this:
for i in {{A..Z},{a..z}}; do
mkdir "${i}" && mv [0-9][0-9][0-9][0-9]" - ${i}"*" "${i}"
done
You notice the four repetitions of the digits, and yeah it looks clumsier than a normal RE like [0-9]{4}.
Currently learning some bash scripting and having an issue with a question involving listing all files in a given directory and stating if they are a file or directory. The issue I am having is that I only get either my current directory or if a specify a directory it will just say that it is a directory eg. /home/user/shell_scripts will return shell_scipts is a directory rather than the files contained within it.
This is what I have so far:
dir=$dir
for file in $dir; do
if [[ -d $file ]]; then
echo "$file is a directory"
if [[ -f $file ]]; then
echo "$file is a regular file"
fi
done
Your line:
for file in $dir; do
will expand $dir just to a single directory string. What you need to do is expand that to a list of files in the directory. You could do this using the following:
for file in "${dir}/"* ; do
This will expand the "${dir}/"* section into a name-only list of the current directory. As Biffen points out, this should guarantee that the file list wont end up with split partial file names in file if any of them contain whitespace.
If you want to recurse into the directories in dir then using find might be a better approach. Simply use:
for file in $( find ${dir} ); do
Note that while simple, this will not handle files or directories with spaces in them. Because of this, I would be tempted to drop the loop and generate the output in one go. This might be slightly different than what you want, but is likely to be easier to read and a lot more efficient, especially with large numbers of files. For example, To list all the directories:
find ${dir} -maxdepth 1 -type d
and to list the files:
find ${dir} -maxdepth 1 -type f
if you want to iterate into directories below, then remove the -maxdepth 1
This is a good use for globbing:
for file in "$dir/"*
do
[[ -d "$file" ]] && echo "$file is a directory"
[[ -f "$file" ]] && echo "$file is a regular file"
done
This will work even if files in $dir have special characters in their names, such as spaces, asterisks and even newlines.
Also note that variables should be quoted ("$file"). But * must not be quoted. And I removed dir=$dir since it doesn't do anything (except break when $dir contains special characters).
ls -F ~ | \
sed 's#.*/$#/& is a Directory#;t quit;s#.*#/& is a File#;:quit;s/[*/=>#|] / /'
The -F "classify" switch appends a "/" if a file is a directory. The sed code prints the desired message, then removes the suffix.
for file in $(ls $dir)
do
[ -f $file ] && echo "$file is File"
[ -d $file ] && echo "$file is Directory"
done
or replace the
$(ls $dir)
with
`ls $`
If you want to list files that also start with . use:
for file in "${dir}/"* "${dir}/"/.[!.]* "${dir}/"/..?* ; do
I have a folder "test" in it there is 20 other folder with different names like A,B ....(actually they are name of people not A, B...) I want to write a shell script that go to each folder like test/A and rename all the .c files with A[1,2..] and copy them to "test" folder. I started like this but I have no idea how to complete it!
#!/bin/sh
for file in `find test/* -name '*.c'`; do mv $file $*; done
Can you help me please?
This code should get you close. I tried to document exactly what I was doing.
It does rely on BASH and the GNU version of find to handle spaces in file names. I tested it on a directory fill of .DOC files, so you'll want to change the extension as well.
#!/bin/bash
V=1
SRC="."
DEST="/tmp"
#The last path we saw -- make it garbage, but not blank. (Or it will break the '[' test command
LPATH="/////"
#Let us find the files we want
find $SRC -iname "*.doc" -print0 | while read -d $'\0' i
do
echo "We found the file name... $i";
#Now, we rip off the off just the file name.
FNAME=$(basename "$i" .doc)
echo "And the basename is $FNAME";
#Now we get the last chunk of the directory
ZPATH=$(dirname "$i" | awk -F'/' '{ print $NF}' )
echo "And the last chunk of the path is... $ZPATH"
# If we are down a new path, then reset our counter.
if [ $LPATH == $ZPATH ]; then
V=1
fi;
LPATH=$ZPATH
# Eat the error message
mkdir $DEST/$ZPATH 2> /dev/null
echo cp \"$i\" \"$DEST/${ZPATH}/${FNAME}${V}\"
cp "$i" "$DEST/${ZPATH}/${FNAME}${V}"
done
#!/bin/bash
## Find folders under test. This assumes you are already where test exists OR give PATH before "test"
folders="$(find test -maxdepth 1 -type d)"
## Look into each folder in $folders and find folder[0-9]*.c file n move them to test folder, right?
for folder in $folders;
do
##Find folder-named-.c files.
leaf_folder="${folder##*/}"
folder_named_c_files="$(find $folder -type f -name "*.c" | grep "${leaf_folder}[0-9]")"
## Move these folder_named_c_files to test folder. basename will hold just the file name.
## Don't know as you didn't mention what name the file to rename to, so tweak mv command acc..
for file in $folder_named_c_files; do basename=$file; mv $file test/$basename; done
done
I am trying to loop through all the files in a directory.
I want to do some stuff on each file (convert it to xml, not included in example), then write the file to a new directory structure.
for file in `find /home/devel/stuff/static/ -iname "*.pdf"`;
do
echo $file;
sed -e 's/static/changethis/' $file > newfile +".xml";
echo $newfile;
done
I want the results to be:
$file => /home/devel/stuff/static/2002/hello.txt
$newfile => /home/devel/stuff/changethis/2002/hello.txt.xml
How do I have to change my sed line?
If you need to rename multiple files, I would suggest to use rename command:
# remove "-n" after you verify it is what you need
rename -n 's/hello/hi/g' $(find /home/devel/stuff/static/ -type f)
or, if you don't have rename try this:
find /home/devel/stuff/static/ -type f | while read FILE
do
# modify line below to do what you need, then remove leading "echo"
echo mv $FILE $(echo $FILE | sed 's/hello/hi/g')
done
Are you trying to change the filename? Then
for file in /home/devel/stuff/static/*/*.txt
do
echo "Moving $file"
mv "$file" "${file/static/changethis}.xml"
done
Please make sure /home/devel/stuff/static/*/*.txt is what you want before using the script.
First, you have to create the name of the new file based on the name of the initial file. The obvious solution is:
newfile=${file/static/changethis}.xml
Second you have to make sure that the new directory exists or create it if not:
mkdir -p $(dirname $newfile)
Then you can do something with your file:
doSomething < $file > $newfile
I wouldn't do the for loop because of the possibility of overloading your command line. Command lines have a limited length, and if you overload it, it'll simply drop off the excess without giving you any warning. It might work if your find returns 100 file. It might work if it returns 1000 files, but it might fail if your find returns 1000 files and you'll never know.
The best way to handle this is to pipe the find into a while read statement as glenn jackman.
The sed command only works on STDIN and on files, but not on file names, so if you want to munge your file name, you'll have to do something like this:
$newname="$(echo $oldname | sed 's/old/new/')"
to get the new name of the file. The $() construct executes the command and puts the results of the command on STDOUT.
So, your script will look something like this:
find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
echo $file;
newfile="$(echo $file | sed -e 's/static/changethis/')"
newfile="$newfile.xml"
echo $newfile;
done
Now, since you're renaming the file directory, you'll have to make sure the directory exists before you do your move or copy:
find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
echo $file;
newfile="$(echo $file | sed -e 's/static/changethis/')"
newfile="$newfile.xml"
echo $newfile;
#Check for directory and create it if it doesn't exist
$dirname=$(dirname "$newfile")
if [ ! -d "$dirname" ]
then
mkdir -p "$dirname"
fi
#Directory now exists, so you can do the move
mv "$file" "$newfile"
done
Note the quotation marks to handle the case there's a space in the file name.
By the way, instead of doing this:
if [ ! -d "$dirname" ]
then
mkdir -p "$dirname"
fi
You can do this:
[ -d "$dirname"] || mkdir -p "$dirname"
The || means to execute the following command only if the test isn't true. Thus, if [ -d "$dirname" ] is a false statement (the directory doesn't exist), you run mkdir.
It's a fairly common shortcut when you see shell scripts.
find ... | while read file; do
newfile=$(basename "$file").xml;
do something to "$file" > "$somedir/$newfile"
done
OUTPUT="$(pwd)";
for file in `find . -iname "*.pdf"`;
do
echo $file;
cp $file $file.xml
echo "file created in directory = {$OUTPUT}"
done
This will create a new file with name whatyourfilename.xml, for hello.pdf the new file created would be hello.pdf.xml, basically it creates a new file with .xml appended at the end.
Remember the above script finds files in the directory /home/devel/stuff/static/ whose file names match the matcher string of the find command (in this case *.pdf), and copies it to your present working directory.
The find command in this particular script only finds files with filenames ending with .pdf If you wanted to run this script for files with file names ending with .txt, then you need to change the find command to this find /home/devel/stuff/static/ -iname "*.txt",
Once I wanted to remove trailing -min from my files. i.e. wanted alg-min.jpg to turn into alg.jpg. so after some struggle, managed to figure something like this:
for f in *; do echo $f; mv $f $(echo $f | sed 's/-min//g');done;
Hope this helps someone willing to REMOVE or SUBTITUDE some part of their file names.