Bash script to remove small images - image

I didn't write the script I'm about to paste, I found it and modified it slightly because I don't know how to do it myself. Here is the current code:
for F in *.jpg *.gif ; do
identify "$F"
done | awk '{ split($3, wh, /x/); } wh[1] < 800 && wh[2] < 800 { print $1 }'
This works and displays images with dimensions less than 800px in either dimension.
But if I try to change it to this:
for F in *.jpg *.gif ; do
identify "$F"
done | awk '{ split($3, wh, /x/); } wh[1] < 800 && wh[2] < 800 { rm -vfr $1 }'
Nothing happens, nothing is deleted, why not?

rm is a shell command, not an awk command.
One way to do what you want is to use awk just to print the file names (which it can do) and then pipe that into something that's capable of deleting:
for F in *.jpg *.gif ; do
identify "$F"
done | awk '{split($3,wh,/x/);} wh[1]<800 && wh[2]<800 {print $1}' | xargs rm
I haven't checked whether any of your other code is valid, I'm assuming it is because the script was working before you made your changes.
xargs will receive a list of things on its standard input and batch them up to have a specific command run against them. So, for example, the following sequence:
touch paxjunk001
touch paxjunk002
echo paxjunk* | xargs rm
will effectively be the same as:
touch paxjunk001
touch paxjunk002
rm paxjunk001 paxjunk002
Getting it to handle the "weird" file names such as those with spaces in them, is a matter of using NUL characters as the separator rather than white space.
That means your awk print command will turn into something like:
{ printf "%s\0", $1 }`
to get the filenames separated correctly, and using:
xargs --null
to ensure xargs understands the format.

If you're using ImageMagick, you can defer to identify the test about the size of your image. The -format switch to identify is very powerful and allows a lot of computations to be done.
For example:
identify -format '%[fx:w<800 && h<800]' image.jpg
will output to stdout 1 if the image has a width less than 800px and a height less than 800px, and outputs 0 otherwise. Using this is much safer than your solution of parsing the raw output of identify, since your solution breaks as soon as a filename contains a space.
I would suggest using something along these lines instead:
#!/bin/bash
shopt -s nullglob
for f in *.jpg *.gif; do
a=$(identify -format '%[fx:w<800 && h<800]' -- "$f") || continue
if [[ $a = 1 ]]; then
echo rm -v -- "$f"
fi
done
This will only echo the rm commands, and will not perform them. Remove the echo command in front of rm if you're happy with what you see.
For more information about these features of ImageMagick, please see these links:
ImageMagick's escapes,
ImageMagick's fx operator.

Related

Batch Renaming files to a sequence [duplicate]

I want to rename the files in a directory to sequential numbers. Based on creation date of the files.
For Example sadf.jpg to 0001.jpg, wrjr3.jpg to 0002.jpg and so on, the number of leading zeroes depending on the total amount of files (no need for extra zeroes if not needed).
Beauty in one line:
ls -v | cat -n | while read n f; do mv -n "$f" "$n.ext"; done
You can change .ext with .png, .jpg, etc.
Try to use a loop, let, and printf for the padding:
a=1
for i in *.jpg; do
new=$(printf "%04d.jpg" "$a") #04 pad to length of 4
mv -i -- "$i" "$new"
let a=a+1
done
using the -i flag prevents automatically overwriting existing files, and using -- prevents mv from interpreting filenames with dashes as options.
I like gauteh's solution for its simplicity, but it has an important drawback. When running on thousands of files, you can get "argument list too long" message (more on this), and second, the script can get really slow. In my case, running it on roughly 36.000 files, script moved approx. one item per second! I'm not really sure why this happens, but the rule I got from colleagues was "find is your friend".
find -name '*.jpg' | # find jpegs
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | # build mv command
bash # run that command
To count items and build command, gawk was used. Note the main difference, though. By default find searches for files in current directory and its subdirectories, so be sure to limit the search on current directory only, if necessary (use man find to see how).
A very simple bash one liner that keeps the original extensions, adds leading zeros, and also works in OSX:
num=0; for i in *; do mv "$i" "$(printf '%04d' $num).${i#*.}"; ((num++)); done
Simplified version of http://ubuntuforums.org/showthread.php?t=1355021
using Pero's solution on OSX required some modification. I used:
find . -name '*.jpg' \
| awk 'BEGIN{ a=0 }{ printf "mv \"%s\" %04d.jpg\n", $0, a++ }' \
| bash
note: the backslashes are there for line continuation
edit July 20, 2015:
incorporated #klaustopher's feedback to quote the \"%s\" argument of the mv command in order to support filenames with spaces.
with "rename" command
rename -N 0001 -X 's/.*/$N/' *.jpg
or
rename -N 0001 's/.*/$N.jpg/' *.jpg
To work in all situations, put a \" for files that have space in the name
find . -name '*.jpg' | gawk 'BEGIN{ a=1 }{ printf "mv \"%s\" %04d.jpg\n", $0, a++ }' | bash
On OSX, install the rename script from Homebrew:
brew install rename
Then you can do it really ridiculously easily:
rename -e 's/.*/$N.jpg/' *.jpg
Or to add a nice prefix:
rename -e 's/.*/photo-$N.jpg/' *.jpg
NOTE The rename commands here include -n which previews the rename. To actually perform the renaming, remove the -n
If your rename doesn't support -N, you can do something like this:
ls -1 --color=never -c | xargs rename -n 's/.*/our $i; sprintf("%04d.jpg", $i++)/e'
NOTE The rename commands here includes -n which previews the rename. To actually perform the renaming, remove the -n
Edit To start with a given number, you can use the (somewhat ugly-looking) code below, just replace 123 with the number you want:
ls -1 --color=never -c | xargs rename -n 's/.*/our $i; if(!$i) { $i=123; } sprintf("%04d.jpg", $i++)/e'
This lists files in order by creation time (newest first, add -r to ls to reverse sort), then sends this list of files to rename. Rename uses perl code in the regex to format and increment counter.
However, if you're dealing with JPEG images with EXIF information, I'd recommend exiftool
This is from the exiftool documentation, under "Renaming Examples"
exiftool '-FileName<CreateDate' -d %Y%m%d_%H%M%S%%-c.%%e dir
Rename all images in "dir" according to the "CreateDate" date and time, adding a copy number with leading '-' if the file already exists ("%-c"), and
preserving the original file extension (%e). Note the extra '%' necessary to escape the filename codes (%c and %e) in the date format string.
Follow command rename all files to sequence and also lowercase extension:
rename --counter-format 000001 --lower-case --keep-extension --expr='$_ = "$N" if #EXT' *
find . | grep 'avi' | nl -nrz -w3 -v1 | while read n f; do mv "$f" "$n.avi"; done
find . will display all file in folder and subfolders.
grep 'avi' will filter all files with avi extension.
nl -nrz -w3 -v1 will display sequence number starting 001 002 etc following by file name.
while read n f; do mv "$f" "$n.avi"; done will change file name to sequence numbers.
I spent 3-4 hours developing this solution for an article on this:
https://www.cloudsavvyit.com/8254/how-to-bulk-rename-files-to-numeric-file-names-in-linux/
if [ ! -r _e -a ! -r _c ]; then echo 'pdf' > _e; echo 1 > _c ;find . -name "*.$(cat _e)" -print0 | xargs -0 -t -I{} bash -c 'mv -n "{}" $(cat _c).$(cat _e);echo $[ $(cat _c) + 1 ] > _c'; rm -f _e _c; fi
This works for any type of filename (spaces, special chars) by using correct \0 escaping by both find and xargs, and you can set a start file naming offset by increasing echo 1 to any other number if you like.
Set extension at start (pdf in example here). It will also not overwrite any existing files.
Let us assume we have these files in a directory, listed in order of creation, the first being the oldest:
a.jpg
b.JPG
c.jpeg
d.tar.gz
e
then ls -1cr outputs exactly the list above. You can then use rename:
ls -1cr | xargs rename -n 's/^[^\.]*(\..*)?$/our $i; sprintf("%03d$1", $i++)/e'
which outputs
rename(a.jpg, 000.jpg)
rename(b.JPG, 001.JPG)
rename(c.jpeg, 002.jpeg)
rename(d.tar.gz, 003.tar.gz)
Use of uninitialized value $1 in concatenation (.) or string at (eval 4) line 1.
rename(e, 004)
The warning ”use of uninitialized value […]” is displayed for files without an extension; you can ignore it.
Remove -n from the rename command to actually apply the renaming.
This answer is inspired by Luke’s answer of April 2014. It ignores Gnutt’s requirement of setting the number of leading zeroes depending on the total amount of files.
I had a similar issue and wrote a shell script for that reason. I've decided to post it regardless that many good answers were already posted because I think it can be helpful for someone. Feel free to improve it!
numerate
#Gnutt The behavior you want can be achieved by typing the following:
./numerate.sh -d <path to directory> -o modtime -L 4 -b <startnumber> -r
If the option -r is left out the reaming will be only simulated (Should be helpful for testing).
The otion L describes the length of the target number (which will be filled with leading zeros)
it is also possible to add a prefix/suffix with the options -p <prefix> -s <suffix>.
In case somebody wants the files to be sorted numerically before they get numbered, just remove the -o modtime option.
a=1
for i in *.jpg; do
mv -- "$i" "$a.jpg"
a=`expr $a + 1`
done
Again using Pero's solution with little modifying, because find will be traversing the directory tree in the order items are stored within the directory entries. This will (mostly) be consistent from run to run, on the same machine and will essentially be "file/directory creation order" if there have been no deletes.
However, in some case you need to get some logical order, say, by name, which is used in this example.
find -name '*.jpg' | sort -n | # find jpegs
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | # build mv command
bash # run that command
The majority of the other solutions will overwrite existing files already named as a number. This is particularly a problem if running the script, adding more files, and then running the script again.
This script renames existing numerical files first:
#!/usr/bin/perl
use strict;
use warnings;
use File::Temp qw/tempfile/;
my $dir = $ARGV[0]
or die "Please specify directory as first argument";
opendir(my $dh, $dir) or die "can't opendir $dir: $!";
# First rename any files that are already numeric
while (my #files = grep { /^[0-9]+(\..*)?$/ } readdir($dh))
{
for my $old (#files) {
my $ext = $old =~ /(\.[^.]+)$/ ? $1 : '';
my ($fh, $new) = tempfile(DIR => $dir, SUFFIX => $ext);
close $fh;
rename "$dir/$old", $new;
}
}
rewinddir $dh;
my $i;
while (my $file = readdir($dh))
{
next if $file =~ /\A\.\.?\z/;
my $ext = $file =~ /(\.[^.]+)$/ ? $1 : '';
rename "$dir/$file", sprintf("%s/%04d%s", $dir, ++$i, $ext);
}
Sorted by time, limited to jpg, leading zeroes and a basename (in case you likely want one):
ls -t *.jpg | cat -n | \
while read n f; do mv "$f" "$(printf thumb_%04d.jpg $n)"; done
(all on one line, without the \)
Not related to creation date but numbered based on sorted names:
python3 -c \
'ext="jpg"
start_num=0
pad=4
import os,glob
files=glob.glob(f"*.{ext}")
files.sort()
renames=list(zip(files,range(start_num,len(files)+start_num)))
for r in renames:
oname=r[0]
nname=f"{r[1]:0{pad}}.{ext}"
print(oname,"->",nname)
os.rename(oname,nname)
'
This script will sort the files by creation date on Mac OS bash. I use it to mass rename videos. Just change the extension and the first part of the name.
ls -trU *.mp4| awk 'BEGIN{ a=0 }{ printf "mv %s lecture_%03d.mp4\n", $0, a++ }' | bash
ls -1tr | rename -vn 's/.*/our $i;if(!$i){$i=1;} sprintf("%04d.jpg", $i++)/e'
rename -vn - remove n for off test mode
{$i=1;} - control start number
"%04d.jpg" - control count zero 04 and set output extension .jpg
To me this combination of answers worked perfectly:
ls -v | gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | bash
ls -v helps with ordering 1 10 9 in correct: 1 9 10 order, avoiding filename extension problems with jpg JPG jpeg
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' renumbers with 4 characters and leading zeros. By avoiding mv I do not accidentally try to overwrite anything that is there already by accidentally having the same number.
bash executes
Be aware of what #xhienne said, piping unknown content to bash is a security risk. But this was not the case for me as I was using my scanned photos.
Here is what worked for me.
I Have used rename command so that if any file contains spaces in name of it then , mv command dont get confused between spaces and actual file.
Here i replaced spaces , ' ' in a file name with '_' for all jpg files
#! /bin/bash
rename 'y/ /_/' *jpg #replacing spaces with _
let x=0;
for i in *.jpg;do
let x=(x+1)
mv $i $x.jpg
done
Nowadays there is an option after you select multiple files for renaming (I have seen in thunar file manager).
select multiple files
check options
select rename
A prompt comes with all files in that particular dir
just check with the category section
Using sed :
ls -tr | sed "s/(.*)/mv '\1' \=printf('%04s',line('.').jpg)/" > rename.sh
bash rename.sh
This way you can check the script before executing it to avoid big mistakes
Here a another solution with "rename" command:
find -name 'access.log.*.gz' | sort -Vr | rename 's/(\d+)/$1+1/ge'
Pero's answer got me here :)
I wanted to rename files relative to time as the image viewers did not display images in time order.
ls -tr *.jpg | # list jpegs relative to time
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | # build mv command
bash # run that command
To renumber 6000, files in one folder you could use the 'Rename' option of the ACDsee program.
For defining a prefix use this format: ####"*"
Then set the start number and press Rename and the program will rename all 6000 files with sequential numbers.

Move files based of a comparison with a file

I have 1000 files with following names:
something-345-something.txt
something-5468-something.txt
something-100-something.txt
something-6200-something.txt
and a lot more...
And I have one txt file, with only numbers in it. f.e:
1000
500
5468
6200
699
usw...
Now I would like to move all files, which have a number in their filenames which is in my txt file.
So in my example above the following files should be moved only:
something-5468-something.txt
something-6200-something.txt
Is there an easy way to achieve this?
What about on the fly moving files by doing this:
for i in `cat you-file.txt`; do
find . -iname "*-$i-*" -exec mv '{}' /target/dir \;
; done
For every line in your text file, the find command will try to find only does matching the pattern *-$i-* (something-6200-something.txt) and move it to your target dir.
Naive implementation: for file in $(ls); do grep $(echo -n $file | sed -nr 's/[^-]*-([0-9]+).*/\1/p') my-one-txt.txt && mv $file /tmp/somewhere; done
In English: For every file in output of ls: parse number part of filename with sed and grep for it in your text file. grep returns a non-zero exit code if nothing is found, so mv is in evaluated in that case.
Script file named move (executable):
#!/bin/bash
TARGETDIR="$1"
FILES=`find . -type f` # build list of files
while read n # read numbers from standard input
do # n contains a number => filter list of files by that number:
echo "$FILES" | grep "\-$n-" | while read f
do # move file that passed the filter because its name matches n:
mv "$f" "$TARGETDIR"
done
done
Use it like this:
cd directory-with-files
./move target-directory < number-list.txt
Here's a crazy bit of bash hackery
shopt -s extglob nullglob
mv -t /target/dir *-#($(paste -sd "|" numbers.txt))-*
That uses paste to join all the lines in your numbers file with pipe characters, then uses bash extended pattern matching to find the files matching any one of the numbers.
I assume mv from GNU coreutils for the -t option.

Get index of argument with xargs?

In bash, I have list of files all named the same (in different sub directories) and I want to order them by creation/modified time, something like this:
ls -1t /tmp/tmp-*/my-file.txt | xargs ...
I would like to rename those files with some sort of index or something so I can move them all into the same folder. My result would ideally be something like:
my-file0.txt
my-file1.txt
my-file2.txt
Something like that. How would I go about doing this?
You can just loop through these files and keep appending an incrementing counter to desired file name:
for f in /tmp/tmp-*/my-file.txt; do
fname="${f##*/}"
fname="${fname%.*}"$((i++)).txt
mv "$f" "/dest/dir/$fname"
done
EDIT: In order to sort listed files my modification time as is the case with ls -1t you can use this script:
while IFS= read -d '' -r f; do
f="${f#* }"
fname="${f##*/}"
fname="${fname%.*}"$((i++)).txt
mv "$f" "/dest/dir/$fname"
done < <(find /tmp/tmp-* -name 'my-file.txt' -printf "%T# %p\0" | sort -zk1nr)
This handles filenames with all special characters like white spaces, newlines, glob characters etc since we are ending each filename with NUL or \0 character in -printf option. Note that we are also using sort -z to handle NUL terminated data.
So I found an answer to my own question, thoughts on this one?
ls -1t /tmp/tmp-*/my-file.txt | awk 'BEGIN{ a=0 }{ printf "cp %s /tmp/all-the-files/my-file_%03d.txt\n", $0, a++ }' | bash;
I found this from another stack overflow question looking for something similar that my search didn't find at first. I was impressed with the awk line, thought that was pretty neat.

Renaming files in a folder to sequential numbers

I want to rename the files in a directory to sequential numbers. Based on creation date of the files.
For Example sadf.jpg to 0001.jpg, wrjr3.jpg to 0002.jpg and so on, the number of leading zeroes depending on the total amount of files (no need for extra zeroes if not needed).
Beauty in one line:
ls -v | cat -n | while read n f; do mv -n "$f" "$n.ext"; done
You can change .ext with .png, .jpg, etc.
Try to use a loop, let, and printf for the padding:
a=1
for i in *.jpg; do
new=$(printf "%04d.jpg" "$a") #04 pad to length of 4
mv -i -- "$i" "$new"
let a=a+1
done
using the -i flag prevents automatically overwriting existing files, and using -- prevents mv from interpreting filenames with dashes as options.
I like gauteh's solution for its simplicity, but it has an important drawback. When running on thousands of files, you can get "argument list too long" message (more on this), and second, the script can get really slow. In my case, running it on roughly 36.000 files, script moved approx. one item per second! I'm not really sure why this happens, but the rule I got from colleagues was "find is your friend".
find -name '*.jpg' | # find jpegs
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | # build mv command
bash # run that command
To count items and build command, gawk was used. Note the main difference, though. By default find searches for files in current directory and its subdirectories, so be sure to limit the search on current directory only, if necessary (use man find to see how).
A very simple bash one liner that keeps the original extensions, adds leading zeros, and also works in OSX:
num=0; for i in *; do mv "$i" "$(printf '%04d' $num).${i#*.}"; ((num++)); done
Simplified version of http://ubuntuforums.org/showthread.php?t=1355021
using Pero's solution on OSX required some modification. I used:
find . -name '*.jpg' \
| awk 'BEGIN{ a=0 }{ printf "mv \"%s\" %04d.jpg\n", $0, a++ }' \
| bash
note: the backslashes are there for line continuation
edit July 20, 2015:
incorporated #klaustopher's feedback to quote the \"%s\" argument of the mv command in order to support filenames with spaces.
with "rename" command
rename -N 0001 -X 's/.*/$N/' *.jpg
or
rename -N 0001 's/.*/$N.jpg/' *.jpg
To work in all situations, put a \" for files that have space in the name
find . -name '*.jpg' | gawk 'BEGIN{ a=1 }{ printf "mv \"%s\" %04d.jpg\n", $0, a++ }' | bash
On OSX, install the rename script from Homebrew:
brew install rename
Then you can do it really ridiculously easily:
rename -e 's/.*/$N.jpg/' *.jpg
Or to add a nice prefix:
rename -e 's/.*/photo-$N.jpg/' *.jpg
NOTE The rename commands here include -n which previews the rename. To actually perform the renaming, remove the -n
If your rename doesn't support -N, you can do something like this:
ls -1 --color=never -c | xargs rename -n 's/.*/our $i; sprintf("%04d.jpg", $i++)/e'
NOTE The rename commands here includes -n which previews the rename. To actually perform the renaming, remove the -n
Edit To start with a given number, you can use the (somewhat ugly-looking) code below, just replace 123 with the number you want:
ls -1 --color=never -c | xargs rename -n 's/.*/our $i; if(!$i) { $i=123; } sprintf("%04d.jpg", $i++)/e'
This lists files in order by creation time (newest first, add -r to ls to reverse sort), then sends this list of files to rename. Rename uses perl code in the regex to format and increment counter.
However, if you're dealing with JPEG images with EXIF information, I'd recommend exiftool
This is from the exiftool documentation, under "Renaming Examples"
exiftool '-FileName<CreateDate' -d %Y%m%d_%H%M%S%%-c.%%e dir
Rename all images in "dir" according to the "CreateDate" date and time, adding a copy number with leading '-' if the file already exists ("%-c"), and
preserving the original file extension (%e). Note the extra '%' necessary to escape the filename codes (%c and %e) in the date format string.
Follow command rename all files to sequence and also lowercase extension:
rename --counter-format 000001 --lower-case --keep-extension --expr='$_ = "$N" if #EXT' *
find . | grep 'avi' | nl -nrz -w3 -v1 | while read n f; do mv "$f" "$n.avi"; done
find . will display all file in folder and subfolders.
grep 'avi' will filter all files with avi extension.
nl -nrz -w3 -v1 will display sequence number starting 001 002 etc following by file name.
while read n f; do mv "$f" "$n.avi"; done will change file name to sequence numbers.
I spent 3-4 hours developing this solution for an article on this:
https://www.cloudsavvyit.com/8254/how-to-bulk-rename-files-to-numeric-file-names-in-linux/
if [ ! -r _e -a ! -r _c ]; then echo 'pdf' > _e; echo 1 > _c ;find . -name "*.$(cat _e)" -print0 | xargs -0 -t -I{} bash -c 'mv -n "{}" $(cat _c).$(cat _e);echo $[ $(cat _c) + 1 ] > _c'; rm -f _e _c; fi
This works for any type of filename (spaces, special chars) by using correct \0 escaping by both find and xargs, and you can set a start file naming offset by increasing echo 1 to any other number if you like.
Set extension at start (pdf in example here). It will also not overwrite any existing files.
Let us assume we have these files in a directory, listed in order of creation, the first being the oldest:
a.jpg
b.JPG
c.jpeg
d.tar.gz
e
then ls -1cr outputs exactly the list above. You can then use rename:
ls -1cr | xargs rename -n 's/^[^\.]*(\..*)?$/our $i; sprintf("%03d$1", $i++)/e'
which outputs
rename(a.jpg, 000.jpg)
rename(b.JPG, 001.JPG)
rename(c.jpeg, 002.jpeg)
rename(d.tar.gz, 003.tar.gz)
Use of uninitialized value $1 in concatenation (.) or string at (eval 4) line 1.
rename(e, 004)
The warning ”use of uninitialized value […]” is displayed for files without an extension; you can ignore it.
Remove -n from the rename command to actually apply the renaming.
This answer is inspired by Luke’s answer of April 2014. It ignores Gnutt’s requirement of setting the number of leading zeroes depending on the total amount of files.
I had a similar issue and wrote a shell script for that reason. I've decided to post it regardless that many good answers were already posted because I think it can be helpful for someone. Feel free to improve it!
numerate
#Gnutt The behavior you want can be achieved by typing the following:
./numerate.sh -d <path to directory> -o modtime -L 4 -b <startnumber> -r
If the option -r is left out the reaming will be only simulated (Should be helpful for testing).
The otion L describes the length of the target number (which will be filled with leading zeros)
it is also possible to add a prefix/suffix with the options -p <prefix> -s <suffix>.
In case somebody wants the files to be sorted numerically before they get numbered, just remove the -o modtime option.
a=1
for i in *.jpg; do
mv -- "$i" "$a.jpg"
a=`expr $a + 1`
done
Again using Pero's solution with little modifying, because find will be traversing the directory tree in the order items are stored within the directory entries. This will (mostly) be consistent from run to run, on the same machine and will essentially be "file/directory creation order" if there have been no deletes.
However, in some case you need to get some logical order, say, by name, which is used in this example.
find -name '*.jpg' | sort -n | # find jpegs
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | # build mv command
bash # run that command
The majority of the other solutions will overwrite existing files already named as a number. This is particularly a problem if running the script, adding more files, and then running the script again.
This script renames existing numerical files first:
#!/usr/bin/perl
use strict;
use warnings;
use File::Temp qw/tempfile/;
my $dir = $ARGV[0]
or die "Please specify directory as first argument";
opendir(my $dh, $dir) or die "can't opendir $dir: $!";
# First rename any files that are already numeric
while (my #files = grep { /^[0-9]+(\..*)?$/ } readdir($dh))
{
for my $old (#files) {
my $ext = $old =~ /(\.[^.]+)$/ ? $1 : '';
my ($fh, $new) = tempfile(DIR => $dir, SUFFIX => $ext);
close $fh;
rename "$dir/$old", $new;
}
}
rewinddir $dh;
my $i;
while (my $file = readdir($dh))
{
next if $file =~ /\A\.\.?\z/;
my $ext = $file =~ /(\.[^.]+)$/ ? $1 : '';
rename "$dir/$file", sprintf("%s/%04d%s", $dir, ++$i, $ext);
}
Sorted by time, limited to jpg, leading zeroes and a basename (in case you likely want one):
ls -t *.jpg | cat -n | \
while read n f; do mv "$f" "$(printf thumb_%04d.jpg $n)"; done
(all on one line, without the \)
Not related to creation date but numbered based on sorted names:
python3 -c \
'ext="jpg"
start_num=0
pad=4
import os,glob
files=glob.glob(f"*.{ext}")
files.sort()
renames=list(zip(files,range(start_num,len(files)+start_num)))
for r in renames:
oname=r[0]
nname=f"{r[1]:0{pad}}.{ext}"
print(oname,"->",nname)
os.rename(oname,nname)
'
This script will sort the files by creation date on Mac OS bash. I use it to mass rename videos. Just change the extension and the first part of the name.
ls -trU *.mp4| awk 'BEGIN{ a=0 }{ printf "mv %s lecture_%03d.mp4\n", $0, a++ }' | bash
ls -1tr | rename -vn 's/.*/our $i;if(!$i){$i=1;} sprintf("%04d.jpg", $i++)/e'
rename -vn - remove n for off test mode
{$i=1;} - control start number
"%04d.jpg" - control count zero 04 and set output extension .jpg
To me this combination of answers worked perfectly:
ls -v | gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | bash
ls -v helps with ordering 1 10 9 in correct: 1 9 10 order, avoiding filename extension problems with jpg JPG jpeg
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' renumbers with 4 characters and leading zeros. By avoiding mv I do not accidentally try to overwrite anything that is there already by accidentally having the same number.
bash executes
Be aware of what #xhienne said, piping unknown content to bash is a security risk. But this was not the case for me as I was using my scanned photos.
Here is what worked for me.
I Have used rename command so that if any file contains spaces in name of it then , mv command dont get confused between spaces and actual file.
Here i replaced spaces , ' ' in a file name with '_' for all jpg files
#! /bin/bash
rename 'y/ /_/' *jpg #replacing spaces with _
let x=0;
for i in *.jpg;do
let x=(x+1)
mv $i $x.jpg
done
Nowadays there is an option after you select multiple files for renaming (I have seen in thunar file manager).
select multiple files
check options
select rename
A prompt comes with all files in that particular dir
just check with the category section
Using sed :
ls -tr | sed "s/(.*)/mv '\1' \=printf('%04s',line('.').jpg)/" > rename.sh
bash rename.sh
This way you can check the script before executing it to avoid big mistakes
Here a another solution with "rename" command:
find -name 'access.log.*.gz' | sort -Vr | rename 's/(\d+)/$1+1/ge'
Pero's answer got me here :)
I wanted to rename files relative to time as the image viewers did not display images in time order.
ls -tr *.jpg | # list jpegs relative to time
gawk 'BEGIN{ a=1 }{ printf "mv %s %04d.jpg\n", $0, a++ }' | # build mv command
bash # run that command
To renumber 6000, files in one folder you could use the 'Rename' option of the ACDsee program.
For defining a prefix use this format: ####"*"
Then set the start number and press Rename and the program will rename all 6000 files with sequential numbers.

Best way to choose a random file from a directory in a shell script

What is the best way to choose a random file from a directory in a shell script?
Here is my solution in Bash but I would be very interested for a more portable (non-GNU) version for use on Unix proper.
dir='some/directory'
file=`/bin/ls -1 "$dir" | sort --random-sort | head -1`
path=`readlink --canonicalize "$dir/$file"` # Converts to full path
echo "The randomly-selected file is: $path"
Anybody have any other ideas?
Edit: lhunath makes a good point about parsing ls. I guess it comes down to whether you want to be portable or not. If you have the GNU findutils and coreutils then you can do:
find "$dir" -maxdepth 1 -mindepth 1 -type f -print0 \
| sort --zero-terminated --random-sort \
| sed 's/\d000.*//g/'
Whew, that was fun! Also it matches my question better since I said "random file". Honsetly though, these days it's hard to imagine a Unix system deployed out there having GNU installed but not Perl 5.
files=(/my/dir/*)
printf "%s\n" "${files[RANDOM % ${#files[#]}]}"
And don't parse ls. Read http://mywiki.wooledge.org/ParsingLs
Edit: Good luck finding a non-bash solution that's reliable. Most will break for certain types of filenames, such as filenames with spaces or newlines or dashes (it's pretty much impossible in pure sh). To do it right without bash, you'd need to fully migrate to awk/perl/python/... without piping that output for further processing or such.
Is "shuf" not portable?
shuf -n1 -e /path/to/files/*
or find if files are deeper than one directory:
find /path/to/files/ -type f | shuf -n1
it's part of coreutils but you'll need 6.4 or newer to get it... so RH/CentOS does not include it.
# ******************************************************************
# ******************************************************************
function randomFile {
tmpFile=$(mktemp)
files=$(find . -type f > $tmpFile)
total=$(cat "$tmpFile"|wc -l)
randomNumber=$(($RANDOM%$total))
i=0
while read line; do
if [ "$i" -eq "$randomNumber" ];then
# Do stuff with file
amarok $line
break
fi
i=$[$i+1]
done < $tmpFile
rm $tmpFile
}
Something like:
let x="$RANDOM % ${#file}"
echo "The randomly-selected file is ${path[$x]}"
$RANDOM in bash is a special variable that returns a random number, then I use modulus division to get a valid index, then reference that index in the array.
This boils down to: How can I create a random number in a Unix script in a portable way?
Because if you have a random number between 1 and N, you can use head -$N | tail to cut somewhere in the middle. Unfortunately, I know no portable way to do this with the shell alone. If you have Python or Perl, you can easily use their random support but AFAIK, there is no standard rand(1) command.
I think Awk is a good tool to get a random number. According to the Advanced Bash Guide, Awk is a good random number replacement for $RANDOM.
Here's a version of your script that avoids Bash-isms and GNU tools.
#! /bin/sh
dir='some/directory'
n_files=`/bin/ls -1 "$dir" | wc -l | cut -f1`
rand_num=`awk "BEGIN{srand();print int($n_files * rand()) + 1;}"`
file=`/bin/ls -1 "$dir" | sed -ne "${rand_num}p"`
path=`cd $dir && echo "$PWD/$file"` # Converts to full path.
echo "The randomly-selected file is: $path"
It inherits the problems other answers have mentioned should files contain newlines.
Newlines in file-names can be avoided by doing the following in Bash:
#!/bin/sh
OLDIFS=$IFS
IFS=$(echo -en "\n\b")
DIR="/home/user"
for file in $(ls -1 $DIR)
do
echo $file
done
IFS=$OLDIFS
Here's a shell snippet that relies only on POSIX features and copes with arbitrary file names (but omits dot files from the selection). The random selection uses awk, because that's all you get in POSIX. It's a very poor random number generator, since awk's RNG is seeded with the current time in seconds (so it's easily predictable, and returns the same choice if you call it multiple times per second).
set -- *
n=$(echo $# | awk '{srand(); print int(rand()*$0) + 1}')
eval "file=\$$n"
echo "Processing $file"
If you don't want to ignore dot files, the file name generation code (set -- *) needs to be replaced by something more complicated.
set -- *; [ -e "$1" ] || shift
set .[!.]* "$#"; [ -e "$1" ] || shift
set ..?* "$#"; [ -e "$1" ] || shift
if [ $# -eq 0]; then echo 1>&2 "empty directory"; exit 1; fi
If you have OpenSSL available, you can use it to generate random bytes. If you don't but your system has /dev/urandom, replace the call to openssl by dd if=/dev/urandom bs=3 count=1 2>/dev/null. Here's a snippet that sets n to a random value between 1 and $#, taking care not to introduce a bias. This snippet assumes that $# is at most 2^23-1.
while
n=$(($(openssl rand 3 | od -An -t u4) + 1))
[ $n -gt $((16777216 / $# * $#)) ]
do :; done
n=$((n % $#))
BusyBox (used on embedded devices) is usually configured to support $RANDOM but it doesn't have bash-style arrays or sort --random-sort or shuf. Hence the following:
#!/bin/sh
FILES="/usr/bin/*"
for f in $FILES; do echo "$RANDOM $f" ; done | sort -n | head -n1 | cut -d' ' -f2-
Note trailing "-" in cut -f2-; this is required to avoid truncating files that contain spaces (or whatever separator you want to use).
It won't handle filenames with embedded newlines correctly.
Put each line of output from the command 'ls' into an associative array named line and then choose one of those like so...
ls | awk '{ line[NR]=$0 } END { print line[(int(rand()*NR+1))]}'
My 2 cents, with a version that should not break when filenames with special chars exist:
#!/bin/bash --
dir='some/directory'
let number_of_files=$(find "${dir}" -type f -print0 | grep -zc .)
let rand_index=$((1+(RANDOM % number_of_files)))
printf "the randomly-selected file is: "
find "${dir}" -type f -print0 | head -z -n "${rand_index}" | tail -z -n 1
printf "\n"

Resources