I need to compare if "dir1" has the same files as "dir2" and ideally remove the similar contents in "dir2".
So far i have tried using the find command:
$ find "$dir1" "$dir2/" "$dir2/" -printf '%P\n' | sort | uniq -u^C
But this doesn't work cause, while the filename are similar, the extension of the files are different in the two folders.
so how do i go about comparing filenames in bash?
Sounds like you just need to use a loop:
for path in "$dir1"/*; do
base=${path##*/} # remove everything up to and including the last / to get the name
if [ -e "$dir2/$base" ]; then
echo rm -r "$dir2/$base"
fi
done
Loop through everything in $dir1 and if $dir2 has a file with the same name, then remove it.
Remove the echo when you're happy that the script is going to remove the right files.
Related
I need to write a Bash script to check if there are any new folders in a path, if yes do something and if not then simply exit.
My thinking is to create a text file to keep track of all old folders and do a diff, if there something new then perform action. Please help me achieve this:
I've tried to use two file tracking but I don't think I've got this right.
The /tmp/ folder has multiple sub folders
#/bin/sh
BASEDIR=/tmp/
cd $BASEDIR
ls -A $BASEDIR >> newfiles.txt
DIRDIFF=$(diff oldfiles.txt newfiles.txt | cut -f 2 -d "")
for file in $DIRDIFF
do
if [ -e $BASEDIR/$file ]
then echo $file
fi
done
Generally don't use ls in scripts. Here is a simple refactoring which avoids it.
#!/bin/sh
printf '%s\n' /tmp/.[!.]* /tmp/* >newfiles.txt
if cmp -13 oldfiles.txt newfiles.txt | grep .; then
rc=0
rm newfiles.txt
else
rc=$?
mv newfiles.txt oldfiles.txt
fi
exit "$rc"
Using comm instead of diff simplifies processing somewhat (the wildcard should expand the files in sorted order, so the requirement for sorted input will be satisfied) and keeping the files in the current directory (instead of in /tmp) should avoid having the script trigger itself. The output from comm will be the new files, so there is no need to process it further. The grep . checks if there are any output lines, so that we can set the exit status to reflect whether or not there were new entries.
Your script looks for files, not directories. If you really want to look for new directories, add a slash after each wildcard expression:
printf '%s\n' /tmp/.[!.]*/ /tmp/*/ >newfiles.txt
This will not notice if an existing file or directory is modified. Probably switch to inotifywait if you need anything more sophisticated (or perhaps even if this is all you need).
Let's assume you are interested in sub-directories only. Let's assume too that you do not have directory names with newlines in them.
Do not process ls output, it is for humans only. Prefer find. Then, simply sort the old and new lists with sort and compare them with comm, keeping only the lines found in the new list but not in the old list (man comm will tell you why the -13 option does this). In the following the do something is just echo, replace by whatever is needed:
#!/bin/sh
BASEDIR=/tmp/
cd "$BASEDIR"
find . -type d | sort > new.txt
if [ -f old.txt ]; then
comm -13 old.txt new.txt | while IFS= read -r name; do
echo "$name"
done
fi
mv new.txt old.txt
This will explore the complete hierarchy under the starting point, and consider any directory. If you do not want to explore the complete hierarchy under the starting point but only the current level:
find . -maxdepth 1 -type d | sort > new.txt
I am using freeNAS and I need to move folders to another folder but only if there is a file within it that does not contain a special character [in my case a ( or )].
Is there any way this can be done? I have found ways to move files that have specific characters in filenames but not for this scenario.
destinationFolder="~/destination"
searchFolder=$PWD
for folder in `ls -lrth $searchFolder| grep "^d"| awk -F '{print $9}'`
do
fileCount=`ls $folder|wc -l`
findCount=`find ${searchFolder}/${folder} -type f | xargs egrep -v "\(|\)"`
if [[ $fileCount -eq $findCount ]]; then
cp -rp $folder $destinationFolder
fi
done
The script is based on one assumption that the folder which you are trying to copy has only files in it and has no subfolders in it. If there are subfolders in it, change the logic accordingly.
If the assumption is as per your expectation then go ahead and use this small script. Let me know if this met your need.
Problem
I've got thousands of files with the format "^[[:digit:]]\{4\} - [[:alpha:]].*", for exampe: 7958 - a3ykof zyimeo3.txt. I'm trying to simply move them into folders alphabetically beginning with the first alpha-character after the hyphen.
I feel like I'm so close to getting this to happen the way I want but there's a (hopefully simple) problem.
I tested the commmand with echo first to make sure it grabs the correct information. Then I tried to execute it for real with mv. I've included some examples below based on this list of files:
1439 - a74389 josifj3oj.txt
3589 - Bfoei 839982 3il.txt
4719 - an38n8f n839mm20 mi02.txt
6398 - b39ji oij3o8 j2o.txt
9287 - A2984 j289jj9 oiw.txt
.... several thousand more files
Examples
This works
This lists all the files starting with the letter "a" (after the 4 digits-space-hyphen-space pattern in the beginning):
for i in "$(ls | grep -i "^[[:digit:]]\{4\} - a")"; do echo "$i"; done
This fails
This doesn't put all the files starting with the letter "a" (after the 4 digits-space-hyphen-space pattern) in the "A" folder:
for i in "$(ls | grep -i "^[[:digit:]]\{4\} - a")"; do mv "$i" A; done
I expected this second command to move each file named "#### - a*" or "#### - A*" to the folder named A. But it sees it as one big string/filename joined by "\n".
Here's an example error message:
mv: cannot stat '1439 - a74389 josifj3oj.txt\n9287 - A2984 j289jj9 oiw.txt\n2719 - an38n8f n839mm20 mi02.txt': No such file or directory
Does anybody know what I'm missing?
Edit
Between #alvits's answer and #chepner's and #courtlandj comments, what worked flawless for me was this:
for directory in {A..Z}; do
mkdir -p "$directory" &&
find . -iregex "./[0-9]* - ${directory}.*" -exec mv -t "$directory" {} +;
done
Here's the simplest way to do it.
for directory in {A..Z}; do
mkdir "$directory" &&
find . -iregex "./[0-9]* - ${directory}.*" -exec mv "{}" "$directory" \;
done
The for loop will query for filenames according to each directory they belong.
The find command will find the files and move them to the directory.
BASH has RE-like globbing, and sequence creation, built-in. You can make use of it something like this:
for i in {{A..Z},{a..z}}; do
mkdir "${i}" && mv [0-9][0-9][0-9][0-9]" - ${i}"*" "${i}"
done
You notice the four repetitions of the digits, and yeah it looks clumsier than a normal RE like [0-9]{4}.
I am trying to write a bash command that will rename all the files in the current directory by omitting the last 3 characters. I am not sure if it is possible thats why I am asking here.
I have a lots of files named like this : 720-1458907789605.ts
I need to rename all of them by omitting last 3 characters to obtain from 720-1458907789605.ts ---> 720-1458907789.ts for all files in the current directory.
Is it possible using bash commands? I am new to bash scripts.
Thank you!
Native bash solution:
for f in *.ts; do
[[ -f "$f" ]] || continue # if you do not need to rename directories
mv "$f" "${f:: -6}.ts"
done
This solution is slow if you have really many files: star-expansion in for will take up memory and time.
Ref: bash substring extraction.
If you have a really large data set, a bit more complex but faster solution will be:
find . -type f -name '*.ts' -depth 1 -print0 | while read -d $\0 f; do
mv "$f" "${f%???.ts}.ts"
done
With Larry Wall's rename:
rename -n 's/...\.ts$/.ts/' *.ts
If everything looks okay remove dry run option -n.
I have many files named
001ac.jpg 002ae.jpg 003.ag.jpg ... 012gf.jpg
I need to change them to
001.jpg 002.jpg 003.jpg 004.jpg....012.jpg
i have some solutions now, but i think they are wired too. So any other good solutions?
now i have this:
#!/bin/sh
rename .jpg .89 *
for i in {a..z} do
rename $i '' *.jpg
rename $i '' *.jpg
done
rename .89 .jpg *
and this:
1 #!/bin/bash
2
3 for i in `find . -name "*.jpg"`
4 do
5 j=${i:0:5}
6 echo $j
7 mv $i $j.jpg
8 done
This might work for you:
find . -name "*.jpg" |
sed -n 's|^\(\./[0-9]\+\)\([^0-9]\+\)\(\.jpg\)$|mv -v & \1\3|p' | sh
You're almost there, I would use your loop to explcitily remove just the chars from the filename, i.e.
#!/bin/bash
for i in $( find . -name "*.jpg") ;do
new=$(echo "$i" | sed 's/[A-Za-z\.][A-Za-z\.]*//g')
echo $new
echo mv $i $new.jpg
done
Remove the echo in front of mv when you are satisfied this is working as needed. Any spaces in filenames will mess things up, might want to add -printf0 at the end of your fine command.
Also, Don't use backquotes for cmd-substitution. They've been deprecated at least since 1995 ;-)
I hope this helps.
I don't quite comprehend the first solution. I don't know of a rename command.
The second solution might be good if you can guarantee the exact format of the name, but why do you have the length as 5 in ${i:0:5} instead of 3? The examples you gave all have a number length of 3 digits. And, if you're doing a find, you shouldn't put it in a for loop. Just pipe it into a a read:
find . -name "*.jpg" | while read $name
do
newName=${name:0:3}
mv $name $newName.jpg
done
Here's another possible solution. It simply loops through all of your numbers. The printf formats the number to be zero filled and three digits. The if makes sure the file exists before you try to rename it.
for number in {1..100}
do
zf_number=$(printf "%03d", $number) #Zero fill number
if [ -e ${zf_number}* ]
then
mv ${zf_number}* $zf_number.jpg
fi
done
That will go sequentially through all the files in the directory and rename them. The printf zero fills the number to match the name on the files.
Because find gives filenames like this: ./001 ./002 ./003
You're right. However, you're probably better off removing the directory and basename of the file, then putting them back together. That way, you don't have issues if some of the files are in sub-directories.
find . -name "*.jpg" | while read $name
do
dirname=$(dirname $name)
basename=$(basename $name)
newName=${basename:0:3}
mv "$dirname/$basename" "$dirname/$newname.jpg"
done
Try the following script:
numerate.sh
This code snipped should do the job:
./numerate.sh -d <your image folder> -b <start number> -L 3 -s .jpg -o numerically -r