I've 2 directories dir1 and dir2 containing more than 8000 files each. I want to extract the files from dir1 that has same name in dir 2 to one directory and also the files in dir2 that has the same name in dir1 into another directory.
If I understand correctly, you are looking for something like this
#!/bin/bash
dir1list=(/path/to/dir1/*)
dir2list=(/path/to/dir2/*)
mkdir /tmp/dirlist
for(( n=0; n<${#dir1list}; n++)); do
echo "${dir1list[n]##/*/}" >> /tmp/dirlist/dir1.list
done
for(( n=0; n<${#dir2list}; n++)); do
echo "${dir2list[n]##/*/}" >> /tmp/dirlist/dir2.list
done
grep -Fx -f /tmp/dirlist/dir2.list /tmp/dirlist/dir1.list > /tmp/dirlist/difflist1.txt
grep -Fx -f /tmp/dirlist/dir1.list /tmp/dirlist/dir2.list > /tmp/dirlist/difflist2.txt
XIFS=$IFS
IFS=$'\n'
Difflist1=($(</tmp/dirlist/difflist1.txt))
Difflist2=($(</tmp/dirlist/difflist2.txt))
IFS=$XIFS
for(( n=0; n<${#Difflist1}; n++)); do
mv "/path/to/dir1/${Difflist1[n]}" /path/to/dir3
done
for(( n=0; n<${#Difflist2}; n++)); do
mv "/path/to/dir2/${Difflist2[n]}" /path/to/dir4
done
There may be a more elegant way to do this. I based this off of a much more complicated script I had to write a few months ago, so it may not be optimized for your specific job.
Related
I am using the following script to iterate over a folder structure, however folders contains space in between is skipped.
for i in $(echo $File_Directory | sed "s/\// /g")
do
if [ -d $i ]
then
echo "$i directory exists."
else
echo "Creating directory $i"
`mkdir $i`
fi
done
Appreciate help on this..
Looks to me like the whole loop ought to be mkdir -p "$File_Directory" ...
But assuming he meant to put them all in the same dir:
$: File_Directory=a/b/c/d/e/f/g
$: ( IFS=/; for d in $File_Directory; do mkdir -p "$d"; done; )
That does play a little fast and loose with the parameter parsing though.
in a directory, i have thousands of files.
i use the following bash snippet, in order to make these thousands of files, be put in folders, that have each 1000 files,
and i face two issues:
a) now each folder has a prefix of dir_ while i would like it to have a name, that will have 6 digits, if less than 6 in the folder name, leading zeros should be added appropriately.
b) current script, puts first folder, into the last one, for example, i have dir_400325 as the last folder, and in there,i find the bash script i have run, and the dir_1000 folder, which is the first folder created. How could i change this, so the first folder, is not stored into the last one?
#!/bin/bash
c=0; d=1000; mkdir -p dir_${d}
for file in *
do
if [ $c -eq 1000 ]
then
d=$(( d + 1000 )); c=0; mkdir -p dir_${d}
fi
mv "$file" dir_${d}/
c=$(( c + 1 ))
done
You can use printf and a format string to generate your 6-digit directory name with leading zeros (%06d), demonstrated in a shell:
bash-4.4$ d=1001
bash-4.4$ dir_name=$(printf "/path/to/%06d" $d)
bash-4.4$ echo $dir_name
/path/to/001001
Using an absolute path may help ensure the files end up where you're expecting them and not in some subfolder of your current working directory.
#!/bin/bash
c=0
d=1000
dir_name=$(printf "/path/to/%06d" $d)
mkdir -p $dir_name
for file in *
do
if [ $c -eq 1000 ]
then
c=0
d=$(( d + 1000 ))
dir_name=$(printf "/path/to/%06d" $d)
mkdir -p $dir_name
fi
if [[ -f "$file" ]]
then
mv "$file" $dir_name
c=$(( c + 1 ))
fi
done
I am able to rename files numerically, in place, in multiple folders. However, it is not the result I am looking for. My file structure looks as follows:
Pictures-
Vacation-
img.001.jpg
img.002.jpg
img.003.jpg
Holidays-
img.004.jpg
img.005.jpg
img.006.jpg
Fun-
img.007.jpg
What I'd like to achieve is:
Pictures-
Vacation-
img.001.jpg
img.002.jpg
img.003.jpg
Holidays-
img.001.jpg
img.002.jpg
img.003.jpg
Fun-
img.001.jpg
So far I have come up with the following:
a=1
for i in $vm/Holiday/*; do
new=$(printf "%03d.jpg" ${a})
mv ${i} $vm/Holiday/${new}
let a=a+1
done
How can I achieve my desired result without having to separately run this on every single directory within my pictures folder?
Take your version and make it iterate over the folders as well.
#!/bin/bash
for dir in ~/code/stack/Pictures/*; do
[ -d "${dir}" ] || continue
i=1
for img in "${dir}"/*.jpg; do
[ -e "${img}" ] || break
new="$(printf "%03d.jpg" "${i}")"
echo mv "${img}" "$(dirname "${img}")/${new}"
((i++))
done
done
Change the location of your Pictures folder and dryrun with the echo in place first. Is that what you wanted...?
try this
ls > folders #cat top folders into folders
while read folder
do
cd $folder #go to vacation folder for example
i=0
ls *.jpg > files
while read line
do
mv $line img.$i.jpg #rename files according to your need
i=$(($i+1))
done < "files"
rm files
cd ..
done < "folders"
rm folders
#!/bin/bash
function renameImages {
local i=1;
for oldFile in * ; do
if [ -d "$oldFile" ] ; then
pushd "$oldFile" >/dev/null
renameImages
popd >/dev/null
elif [ "${oldFile##*.}" == "jpg" ] ; then
newFile=$(printf img.%03d.jpg $i)
echo "$PWD/$oldFile -> $PWD/$newFile"
i=$((i+1))
fi
done
}
renameImages
This won't actually rename any files, but instead it will print on screen how it would.
If you're happy with what it does, change
echo "$PWD/$oldFile -> $PWD/$newFile"
to
mv "$PWD/$oldFile" "$PWD/$newFile"
NOTE: The first edit of this answer would rename all files, .jpg or not. This version only renames .jpgs.
#!/bin/bash
a=1
for i in $vm/Vacation/*; do
new=$(printf "%04d.jpg" ${a})
mv ${i} $vm/Vacation/${new}
let a=a+1
done
b=1
for i in $vm/Holiday/*; do
new=$(printf "%04d.jpg" ${b})
mv ${i} $vm/Holiday/${new}
let b=b+1
done
c=1
for i in $vm/Fun/*; do
new=$(printf "%04d.jpg" ${c})
mv ${i} $vm/Fun/${new}
let c=c+1
done
This is long and tedious, not ideal, but it works. With years of folders to sort and add pictures to as I come across them, a much shorter solution would have been accepted.
I've found most of the questions of this kind where the change in name has been same for the entire set of files in that directory.
But i'm here presented with a situation to give a different name to every file in that directory or just add a different prefix.
For Example, I have about 200 files in a directory, all of them with numbers in their filename. what i want to do is add a prefix of 1 to 200 for every file. Like 1_xxxxxxxx.png,2_xxxxxxxx.png...........200_xxxxxxxx.png
I'm trying this, but it doesnt increment my $i everytime, rather it gives a prefix of 1_ to every file.
echo "renaming files"
i=1 #initializing
j=ls -1 | wc -l #Count number of files in that dir
while [ "$i" -lt "$j" ] #looping
do
for FILE in * ; do NEWFILE=`echo $i_$FILE`; #swapping the file with variable $i
mv $FILE $NEWFILE #doing the actual rename
i=`expr $i+1` #increment $i
done
Thanks for any suggestion/help.
To increment with expr, you definitely need spaces( expr $i + 1 ), but you would probably be better off just doing:
echo "renaming files"
i=1
for FILE in * ; do
mv $FILE $((i++))_$FILE
done
i=1
for f in *; do
mv -- "$f" "${i}_$f"
i=$(($i + 1))
done
I need to convert a lot of PNG files in many folders and process crop files separately to make thumbnails 100x100 px for just the "crop" files.
File naming is:
????_thumb.png
????_snapshot.png
????_crop.png
where ???? is a number.
My script so far is working to do conversions just fine,
however I need to detect when a "crop" file is reached and
then call ImageMagick and create a 100x100px thumbnail from it named ????_crop_th.png
I can't seem to figure how to detect on a wildcard ????_crop.png.
My script so far:
#!/bin/bash
BASE64=/root/scripts/base64
logfile=/root/tester/convert_failed.txt
goodfile=/root/tester/goodfile.txt
proc_dir=/root/tester/testing
temp_file=/root/tester/temp.png
b64=/root/tester/b64.txt
cd $proc_dir
for i in *
do
if [ -d $i ]
then
for j in $i/*.png
do
if [ -f $j ]
then
#just get files name without extension
fname=`echo $j | cut -d'.' -f1`
#perform operations
cp $j ${fname}.b64
$BASE64/base64 -d $j $temp_file
if [ $ -eq 0 ]
then
cp $temp_file $j
echo $j >> $goodfile
rm -f ${fname}.b64
fi
fi
done
fi
done
`find $proc_dir -name *.b64 -print >$b64`
sort $logfile -o $logfile
sort $goodfile -o $goodfile
sort $b64 -o $b64
Any help is well appreciated.
There are a few things that your script does less-than-perfectly, as well as some redundancies.
I also don't see anything in your script that uses Imagemagick to generate thumbnails from *_crop.png files, which is theoretically what this question is about
I vote for a rewrite. I have no idea if the following will be directly applicable to your situation, but the techniques should at least let you write better shell scripts.
#!/bin/bash
base64=/root/scripts/base64/base64
logfile=/root/tester/convert_failed.log
goodfile=/root/tester/goodfile.txt
proc_dir=/root/tester/testing
# The `cd` command will fail, if it fails. (Really.)
if cd "$proc_dir"; then
# Find all the PNGs in all subdirectories one level under our WD
for file in */*.png; do
# Do stuff (I have no idea what this is for...)
if $base64 -d "$file" "${file%.png}".b64 && mv "${file%.png}".b64 "$file"; then
echo "$file" >> $goodfile
else
printf '[%s] FAILED: %s\n' "${date '+%Y-%m-%d %T')" "$file" >> $logfile
fi
# Only make thumbnails if we need them
if [[ $file =~ _crop.png$ ]] && [[ ! -f "${file%_crop.png}_thumb.png" ]]; then
convert "$file" -scale 100x100 "${file%_crop.png}_thumb.png"
fi
done
fi
You can use regex matching or trailing substring removal, such as:
if [[ "$j" =~ _crop.png$ ]]
or
if [[ "${j%_crop.png}" != "$j" ]]
Also note that chopping off the extension is similarly easy:
fname=${j%.*}
Another useful bash feature is recursive globbing, so you don't need the nested loops and the specialized directory handling:
shopt -s globstar
for j in **/*.png