Let's say I have files nested in subdirectories with potentially duplicate file names. I want to copy all files to a new directory but prevent overwriting AND preserve filenames (mostly).
The following does not work because it overwrites duplicate filenames:
find /SourceDir/. -type f -exec cp -pv \{\} /DestDir/ \;
Adding noclobber (cp -n) doesn't help either because duplicates are just skipped.
Current File Structure:
SourceDir
--SubdirA
----File1.gif
---- ...
----File1000.jpg
--SubdirB
----File1.gif
---- ...
----File1000.png
...
--SubdirZ
----SubdirAA
------File1.sh
------ ...
------File1000.jpg
Desired File Structure:
DestDir
--File1.gif
--File1_1.gif <-- original name was `File1.gif` but this already existed
--File2.jpg
--File2.gif <-- `File2.jpg` already exists, but not `File2.gif`
--File3.gif
--File3_1.gif
--File4.jpg
--File4_1.jpg
--File4_2.jpg <-- original name was `File4.jpg`, but `File4_1.jpg` already existed too.
-- ...
--File1000.png
I do not want to rename every file. And I don't want to give arbitrary hashes to those I need to duplicate. What do you recommend?
I'm on a Mac, so Linux commands are all fair game.
Here is one solution.
#!/bin/bash
SourceDir=".";
DestDir="../dest";
cd ${SourceDir}
find . -type f |
while read x
do
bn=`basename $x`;
if [ -f "${DestDir}/$bn" ]
then
for i in {1..9999}
do
if [ ! -f "${DestDir}/${bn%.*}_${i}.${bn##*.}" ]
then
echo "Next free file extension is no $i";
bn="${DestDir}/${bn%.*}_${i}.${bn##*.}"
break;
fi
done
fi
echo "copy file $x to ${DestDir}/$bn";
cp -p "$x" "${DestDir}/$bn";
done
Please let me know if that works for you.
Related
I have a large set of working files of the form a.mp4 b.txt c.avi d.doc etc (the extension is irrelevant to the question). I also have a set of files which include the same named files except with a common specific prefix "broken_" eg broken_a.mp4, broken_b.txt
If I have a.mp4 and broken_a.mp4, I want to move the broken_a.mp4 to a holding directory. If I have broken_d.mp4 but no matching d.mp4, then leave it alone.
I have some code successfully used to identify and move files with the same extension which I'd like to modify
This is the form of working example code for same extension files (kudos to the original author) which I'd like to modify if possible to do the job
#!/bin/bash
# Name of source directory
SOURCE_DIR=.
# Name of destination directory
DEST_DIR=already_converted_m4v
# Create the destination directory for the moved files, if it doesn't already exist.
[ ! -d $DEST_DIR ] && mkdir -p $DEST_DIR
find $SOURCE_DIR -maxdepth 1 -type f -iname "*.avi" | while read fin
do
#echo "m4v doing avi"
fm4v=${fin/.avi/.m4v}
[ -f "$fm4v" ] && gmv -v --backup=numbered "$fin" $DEST_DIR/
done
My garbage first attempt which clearly doesnt work looks horribly like:
#!/bin/bash
# Name of source directory
SOURCE_DIR=.
# Name of destination directory
DEST_DIR=Already_broken
# Create the destination directory for the moved files, if it doesn't already exist.
[ ! -d $DEST_DIR ] && mkdir -p $DEST_DIR
find $SOURCE_DIR -maxdepth 1 -type f -iname "*" | while read fin
do
#echo "working to find existing broken and unbroken files"
filetest_basename=$(basename "$fin" )
filetest_extension=$(extension "$fin" )
echo $filetest_basename
echo $filetest_extension
fileok=${filetest_basename/!broken_/broken_}
[ -f "$fileok" ] && gmv -v --backup=numbered "$fin" $DEST_DIR/
done
Grateful for help
find is irrelevant here, a simple shell loop would suffice:
SRCDIR='.'
DSTDIR='Already_broken'
if ! [ -d "$DSTDIR" ]; then
mkdir -p -- "$DSTDIR"
fi
for broken in "$SRCDIR"/broken_*; do
if [ -f "${broken%"${broken##*/}"}${broken##*/broken_}" ]; then
echo gmv -v --backup=numbered "$broken" -- "$DSTDIR"
fi
done
If its output looks good, remove echo.
Some notes:
We can't use ${broken/broken_} here for SRCDIR might contain broken_ in the future.
Nested PE (${broken##*/}) needs to be quoted for its result might contain metacharacters and that would bring about undesired results.
I have this find
find "$source_folder" -name "IMG_[0-9][0-9][0-9][0-9].JPG" -exec cp {} $destination_folder \;
i want only the IMG_[0-9][0-9][0-9][0-9].JPG, in the source folder there are diferent files with same name,and same files with same name, how can i copy everything and rename all the same name files with extra .JPG without deleting any unique files?
PS: noob, please could you explain so i can try and learn
I'll assume that your file names don't contain any whitespace. It makes things easier.
You can pipe the output of the find command to a loop where you can run some tests whether or not you want to copy the file.
I have to determine the name of the file and where it is copied to. In order to do that, I have to strip off the $source_folder from the name of the file I find, and prepend the name of the $dest_folder. This is where I want to actually copy the file.
Your directions are a bit confusing. I am assuming you're doing the copy if the $dest_file doesn't exist or it is different from the source. Once I determine that this is the file you want me to copy, I have to make sure the destination directory exists, and if it doesn't I create it. Now, I can do my copy.
I have two echo statements in here. This way, you can do a dry run of this script to make sure it's doing what you want it to do. If it looks good, you can remove the echo commands from the two lines and rerun the script.
find "$source_folder" -name "IMG_[0-9][0-9][0-9][0-9].JPG" | while read file_name
do
rootname=${file_name#$source_folder/} # Removes the source folder from the name
dest_name="${dest_folder}/$rootname"
if [ ! -e "$dest_name" ] || [ ! diff "$file_name" "$dest_name" > /dev/null 2>&1 ]
then
$dest_folder=$(basename $dest_name)
[ ! -d "$dest_folder" ] && echo mkdir -p "$dest_folder" #Remove echo if it works
echo cp "$file_name" "$dest_name" #Remove 'echo' if it works
fi
done
I'm writing a shell script as follows:
for file in `ls`
do
mkdir "$file"_folder
cp $file "$file"_folder
done
What I want to do is to make a folder for each file in the current directory with its name and then underscore folder as the name and then copy that file into it. My problem is that the file names contain spaces in them. How do I escape them?
There are many resources explaining how to do this for variables but none of them can be applied to this situation where I use a for loop to get the names.
Don't use ls there, use shell globbing. (In general, do not parse the output of ls.)
for file in *
do
# only consider files, not directories
if [ -f "$file" ] ; then
new_dir="$file"_folder
# create the directory
if [ ! -d "$new_dir" ] ; then
mkdir "$new_dir"
if [ $? -ne 0 ] ; then
# handle directory creation eror
fi
fi
# possibly check for the copied file existence here
# and deal with that appropriately (i.e. skip/error/copy anyway)
cp "$file" "$new_dir"
fi
done
How about
find . -type f -exec mkdir {}_folder \; -exec cp {} {}_folder \;
It finds all regular files in the current directory, creates the folder (first -exec), and copies the file into the new folder (second -exec).
You do not parse
ls for exactly this reason
for file in *
do
mkdir "${file}_folder"
cp "$file" "${file}_folder"
done
My directory structure is as follows
Directory1\file1.jpg
\file2.jpg
\file3.jpg
Directory2\anotherfile1.jpg
\anotherfile2.jpg
\anotherfile3.jpg
Directory3\yetanotherfile1.jpg
\yetanotherfile2.jpg
\yetanotherfile3.jpg
I'm trying to use the command line in a bash shell on ubuntu to take the first file from each directory and rename it to the directory name and move it up one level so it sits alongside the directory.
In the above example:
file1.jpg would be renamed to Directory1.jpg and placed alongside the folder Directory1
anotherfile1.jpg would be renamed to Directory2.jpg and placed alongside the folder Directory2
yetanotherfile1.jpg would be renamed to Directory3.jpg and placed alongside the folder Directory3
I've tried using:
find . -name "*.jpg"
but it does not list the files in sequential order (I need the first file).
This line:
find . -name "*.jpg" -type f -exec ls "{}" +;
lists the files in the correct order but how do I pick just the first file in each directory and move it up one level?
Any help would be appreciated!
Edit: When I refer to the first file what I mean is each jpg is numbered from 0 to however many files in that folder - for example: file1, file2...... file34, file35 etc... Another thing to mention is the format of the files is random, so the numbering might start at 0 or 1a or 1b etc...
You can go inside each dir and run:
$ mv `ls | head -n 1` ..
If first means whatever the shell glob finds first (lexical, but probably affected by LC_COLLATE), then this should work:
for dir in */; do
for file in "$dir"*.jpg; do
echo mv "$file" "${file%/*}.jpg" # If it does what you want, remove the echo
break 1
done
done
Proof of concept:
$ mkdir dir{1,2,3} && touch dir{1,2,3}/file{1,2,3}.jpg
$ for dir in */; do for file in "$dir"*.jpg; do echo mv "$file" "${file%/*}.jpg"; break 1; done; done
mv dir1/file1.jpg dir1.jpg
mv dir2/file1.jpg dir2.jpg
mv dir3/file1.jpg dir3.jpg
Look for all first level directories, identify first file in this directory and then move it one level up
find . -type d \! -name . -prune | while read d; do
f=$(ls $d | head -1)
mv $d/$f .
done
Building on the top answer, here is a general use bash function that simply returns the first path that resolves to a file within the given directory:
getFirstFile() {
for dir in "$1"; do
for file in "$dir"*; do
if [ -f "$file" ]; then
echo "$file"
break 1
fi
done
done
}
Usage:
# don't forget the trailing slash
getFirstFile ~/documents/
NOTE: it will silently return nothing if you pass it an invalid path.
I have a folder "test" in it there is 20 other folder with different names like A,B ....(actually they are name of people not A, B...) I want to write a shell script that go to each folder like test/A and rename all the .c files with A[1,2..] and copy them to "test" folder. I started like this but I have no idea how to complete it!
#!/bin/sh
for file in `find test/* -name '*.c'`; do mv $file $*; done
Can you help me please?
This code should get you close. I tried to document exactly what I was doing.
It does rely on BASH and the GNU version of find to handle spaces in file names. I tested it on a directory fill of .DOC files, so you'll want to change the extension as well.
#!/bin/bash
V=1
SRC="."
DEST="/tmp"
#The last path we saw -- make it garbage, but not blank. (Or it will break the '[' test command
LPATH="/////"
#Let us find the files we want
find $SRC -iname "*.doc" -print0 | while read -d $'\0' i
do
echo "We found the file name... $i";
#Now, we rip off the off just the file name.
FNAME=$(basename "$i" .doc)
echo "And the basename is $FNAME";
#Now we get the last chunk of the directory
ZPATH=$(dirname "$i" | awk -F'/' '{ print $NF}' )
echo "And the last chunk of the path is... $ZPATH"
# If we are down a new path, then reset our counter.
if [ $LPATH == $ZPATH ]; then
V=1
fi;
LPATH=$ZPATH
# Eat the error message
mkdir $DEST/$ZPATH 2> /dev/null
echo cp \"$i\" \"$DEST/${ZPATH}/${FNAME}${V}\"
cp "$i" "$DEST/${ZPATH}/${FNAME}${V}"
done
#!/bin/bash
## Find folders under test. This assumes you are already where test exists OR give PATH before "test"
folders="$(find test -maxdepth 1 -type d)"
## Look into each folder in $folders and find folder[0-9]*.c file n move them to test folder, right?
for folder in $folders;
do
##Find folder-named-.c files.
leaf_folder="${folder##*/}"
folder_named_c_files="$(find $folder -type f -name "*.c" | grep "${leaf_folder}[0-9]")"
## Move these folder_named_c_files to test folder. basename will hold just the file name.
## Don't know as you didn't mention what name the file to rename to, so tweak mv command acc..
for file in $folder_named_c_files; do basename=$file; mv $file test/$basename; done
done