I have two folders on my Mac's hard drive.
Directory A, and Directory B
Both directories contain image files.
I need to find matching filenames is Dir A.
If I find a match overwrite the file in Dir A with the matching one from Dir B.
How should I approach this?
rsync is THE best tool for synchronizing files. It has tons of options so be sure to read the man page in detail.
rsync -rv --inplace --existing /path/to/dir/b/* /path/to/dir/a
If you want to write a script then the this should do what you are looking for:
#!/bin/bash
DIR_A='/path/to/dir_a'
DIR_B='/path/to/dir_b'
for file in "$DIR_B"/*; do
name="${file##*/}"
if [[ -e $DIR_A/$name ]]; then
echo "Match found = $name";
cp "$file" "$DIR_A"
fi
done
What it does is, look for files in Directory B and extracts the file name (since we use absolute paths). It checks for that filename to see if it exists in Directory A. -e is the test that does that. If it is successful test then we print a message saying that Match found along with the filename. We then proceed to copy the file from Directory B to Directory A.
Now you may choose to remove the message that prints out to the screen and use mv instead of cp if you don't want the copy present in Directory B.
Related
I have a txt file with a list of files (approximately 500) for example:
file_0_hard.msOut
file_1_hard.msOut
file_10_hard.msOut
.
.
.
file_1000_hard.msOut
I want to delete all those files whose name is not in the txt file. All of these files are in the same directory. How can I do this using bash where I read the text file and then delete all those files in the directory that are not in the text file. Help would be appreciated.
Along the lines of user1934428
There is something to say for this solution. But since we have linux at our disposal with a strong filesystem in use I hope. we can make hardlinks; The only requirement for that the destination is on the same filesystem.
So along those lines:
make a directory to store the files you want to keep.
hardlink (ln {file} {target}) ; as this does not cost extra disk space, it only stores the inode number in the new directory file.
remove all files
move the files back from their origin.
And actually this would be about the same as:
mv {files} {save spot}
remove all files
mv {save spot}/{files} back
Which does pretty much the same thing. Then again; it is a nice way to learn about the power of a hardlink.
you may try this :
cd path/dir
for f in *; do
if ! grep -Fxq "$f" pathToFile/file.txt; then
rm -r "$f"
else
printf "exists-- %s \n" ${f}
fi
done
In case you are wondering (as I did) what -Fxq means in plain English:
F: Affects how PATTERN is interpreted (fixed string instead of a regex)
x: Match whole line
q: Shhhhh... minimal printing
Assuming the directory in question is mydir
set -e
cd mydir
tmpdir=/tmp/x$$ # adapt this to your taste
mv $(<list.txt) $tmpdir
cd ..
rm -r mydir
mkdir mydir
mv $tmpdir/* mydir
rm -r $tmpdir
Basically, instead to delete those files you want to keep, you safe them, then delete everything, and then restore them. For your case, this is probably faster than doing the other way around.
UPDATE:
As Michiel commented, it is advisable that you place your tmpdir in the same file system as mydir.
can you please tell me what is wrong with my simple code? (Mac Bash)
I am trying to copy all files that are called vol_0000.nii from one directory to another. I replaced the variable file name with '*'. All files (vol_0000.nii) have the same name but they are in different folders (Indicated by '*'). Im not sure whether when they are being copied they are replacing each other since they have the same name of the cp creates, for example, vol_00001.nii, vol_00002.nii and so on..?
cp /Users/dave/biomkr/dat/*/rs/orig/vol_0000.nii /Users/dave/Documents/MIT_Har_stu/rsfmri
Something like this should do the job, using a similar numbering behaviour to that which you mentioned in your question.
#!/bin/bash
i=0
for src in /Users/dave/biomkr/dat/*/rs/orig/vol_0000.nii
do
dest=$(basename "$src")
dest=${dest/.nii/_$i.nii}
cp "$src" "/Users/dave/Documents/MIT_Har_stu/rsfmri/$dest"
let i++
done
Another option is to create a subdirectory, based on the directory name substituted for * in the original glob, which we can get with a little string replacement.
#!/bin/bash
for src in /Users/dave/biomkr/dat/*/rs/orig/vol_0000.nii
do
srcdir=${src#/Users/dave/biomkr/dat/}
srcdir=${srcdir%/rs/orig/vol_0000.nii}
srcdir="/Users/dave/Documents/MIT_Har_stu/rsfmri/$srcdir/"
mkdir -p "$srcdir"
cp "$src" "$srcdir"
done
I have a few files with the format ReportsBackup-20140309-04-00 and I would like to send the files with same pattern to the files as the example to the 201403 file.
I can already create the files based on the filename; I would just like to move the files based on the name to their correct folder.
I use this to create the directories
old="directory where are the files" &&
year_month=`ls ${old} | cut -c 15-20`&&
for i in ${year_month}; do
if [ ! -d ${old}/$i ]
then
mkdir ${old}/$i
fi
done
you can use find
find /path/to/files -name "*201403*" -exec mv {} /path/to/destination/ \;
Here’s how I’d do it. It’s a little verbose, but hopefully it’s clear what the program is doing:
#!/bin/bash
SRCDIR=~/tmp
DSTDIR=~/backups
for bkfile in $SRCDIR/ReportsBackup*; do
# Get just the filename, and read the year/month variable
filename=$(basename $bkfile)
yearmonth=${filename:14:6}
# Create the folder for storing this year/month combination. The '-p' flag
# means that:
# 1) We create $DSTDIR if it doesn't already exist (this flag actually
# creates all intermediate directories).
# 2) If the folder already exists, continue silently.
mkdir -p $DSTDIR/$yearmonth
# Then we move the report backup to the directory. The '.' at the end of the
# mv command means that we keep the original filename
mv $bkfile $DSTDIR/$yearmonth/.
done
A few changes I’ve made to your original script:
I’m not trying to parse the output of ls. This is generally not a good idea. Parsing ls will make it difficult to get the individual files, which you need for copying them to their new directory.
I’ve simplified your if ... mkdir line: the -p flag is useful for “create this folder if it doesn’t exist, or carry on”.
I’ve slightly changed the slicing command which gets the year/month string from the filename.
I am trying to write a bash script that does the following:
Enumerates through list of files in a directory, that match a specified pattern
Creates a tar file containing the matching files
Removes (i.e. deletes) the matched files from their source directories
To keep things simple, I intend to use a hard coded list of directories and file patterns
This is what I have come up with so far:
#!/bin/bash
filenames[0]='/home/user1/*.foo'
filenames[1]='/some/otherpath/*.fbar'
for f in ${filenames[#]}
do
echo "$f"
done
However, I am unusure on how to proceed from this point onward. Specifically, I need help on:
How to glob the files matching the pattern $f
How to add the ENTIRE list of matching files (i.e. from all directories) to a tar file in one go
Regarding deleting the files, I am thinking of simply iterating through the ENTIRE list obtained in step 2 above, and 'rm' the actual file - is there a better/quicker/more elegant way?
PS:
I am running this on Ubuntu 10.0.4 LTS
If you want to use a loop because you have many directories, you can use the -r option to append to the tar file. You can also use --remove-files to remove files after adding them to the archive.
filenames[0]='/home/user1/*.foo'
filenames[1]='/some/otherpath/*.fbar'
for f in "${filenames[#]}"
do
tar -rvf --remove-files foo.tar $f
done
If you don't have the --remove-files option, use rm $f after the tar command.
tar(1) supports an --remove-files option that will remove the files after adding them to the archive.
Depending upon what you're trying to do with your shell globs, you might be able to ignore doing all that extra work there, too. Try this:
tar cf /dir/archive.tar --remove-files /home/user1/*.foo /some/otherpath/*.fbar
I am writing the following script to copy *.nzb files to a folder to queue them for Download.
I wrote the following script
#!/bin/bash
#This script copies NZB files from Downloads folder to HellaNZB queue folder.
${DOWN}="/home/user/Downloads/"
${QUEUE}="/home/user/.hellanzb/nzb/daemon.queue/"
for a in $(find ${DOWN} -name *.nzb)
do
cp ${a} ${QUEUE}
rm *.nzb
done
it gives me the following error saying:
HellaNZB.sh: line 5: =/home/user/Downloads/: No such file or directory
HellaNZB.sh: line 6: =/home/user/.hellanzb/nzb/daemon.queue/: No such file or directory
Thing is that those directories exsist, I do have right to access them.
Any help would be nice.
Please and thank you.
Variable names on the left side of an assignment should be bare.
foo="something"
echo "$foo"
Here are some more improvements to your script:
#!/bin/bash
#This script copies NZB files from Downloads folder to HellaNZB queue folder.
down="/home/myusuf3/Downloads/"
queue="/home/myusuf3/.hellanzb/nzb/daemon.queue/"
find "${down}" -name "*.nzb" | while read -r file
do
mv "${file}" "${queue}"
done
Using while instead of for and quoting variables that contain filenames protects against filenames that contain spaces from being interpreted as more than one filename. Removing the rm keeps it from repeatedly producing errors and failing to copy any but the first file. The file glob for -name needs to be quoted. Habitually using lowercase variable names reduces the chances of name collisions with shell variables.
If all your files are in one directory (and not in multiple subdirectories) your whole script could be reduced to the following, by the way:
mv /home/myusuf3/Downloads/*.nzb /home/myusuf3/.hellanzb/nzb/daemon.queue/
If you do have files in multiple subdirectories:
find /home/myusuf3/Downloads/ -name "*.nzb" -exec mv {} /home/myusuf3/.hellanzb/nzb/daemon.queue/ +
As you can see, there's no need for a loop.
The correct syntax is:
DOWN="/home/myusuf3/Downloads/"
QUEUE="/home/myusuf3/.hellanzb/nzb/daemon.queue/"
for a in $(find ${DOWN} -name *.nzb)
# escape the * or it will be expanded in the current directory
# let's just hope no file has blanks in its name
do
cp ${a} ${QUEUE} # ok, although I'd normally add a -p
rm *.nzb # again, this is expanded in the current directory
# when you fix that, it will remove ${a}s before they are copied
done
Why don't you just use rm $(a}?
Why use a combination of cp and rm anyway, instead of mv?
Do you realize all files will end up in the same directory, and files with the same name from different directories will overwrite each other?
What if the cp fails? You'll lose your file.