I have a bunch of files in the same directory with names like:
IMG_20160824_132614.jpg
IMG_20160824_132658.jpg
IMG_20160824_132738.jpg
The middle section is the date and last section is time the photo was taken. So if I were to sort these files by their name the result would be the same as sorting by date/time modified
I'd like to batch rename these files using bash to something of the form:
1-x-3.jpg
Where the x represents the place of the file in the sequential ordering (ordered by name/time modified)
So the 3 examples above would be renamed to:
1-1-3.jpg
1-2-3.jpg
1-3-3.jpg
Is there a bash command that can achieve this? Or is a script required?
Try:
i=1; for f in *.jpg; do mv "$f" "1-$((i++))-3.jpg"; done
For example, using your file names:
$ ls
IMG_20160824_132614.jpg IMG_20160824_132658.jpg IMG_20160824_132738.jpg
$ i=1; for f in *.jpg; do mv "$f" "1-$((i++))-3.jpg"; done
$ ls
1-1-3.jpg 1-2-3.jpg 1-3-3.jpg
Notes:
When expanding *.jpg, the shell lists the files in alphanumeric order. This seems to be what you want. Note, though, that alphanumeric order can depend on locale.
The sequential numbering is done with $((i++)). Here, $((...)) represents arithmetic expansion. ++ simply means increment the variable by 1.
Related
I have been trying to do this all afternoon and cannot figure out how to do this. I'm running MXLinux and from the commandline am trying (unsucessfully) to batch edit a bunch of filenames (I've about 500 so don't want to do this by hand) from:
2020-August-15.pdf
2021-October-15.pdf
To:
2020-08-15.pdf
2021-10-15.pdf
I cannot find anything that does this (in a way I understand) so am wondering. Is this possible or am I to do this by hand?
Admittedly I'm not very good with Bash but I can use sed, awk, rename, date, etc. I just can't seem to find a way to combine them to rename my files.
I cannot find anything on here that has been of any help in doing this.
Many thanks.
EDIT:
I'm looking for a way to combine commands and ideally not have to overtly for-loop through the files and the months. What I mean is I would prefer, and was trying to, pipe ls into a command combination to convert as specified above. Sorry for the confusion.
EDIT 2:
Thank you to everyone who came up with answers, and for you patience with my lack of ability. I don't think I'm qualified to make a decision as to the best answer however have settled, for my use-case on the following:
declare -A months=( [January]=01 [February]=02 [March]=03 [April]=04 [May]=05\
[June]=06 [July]=07 [August]=08 [September]=09 [October]=10 [November]=11 [December]=12 )
for oldname in 202[01]-[A-za-z]*-15.pdf
do
IFS=-. read y m d ext <<< "${oldname}"
mv "$oldname" "$y-${months[$m]}-$d.$ext"
done
I think this offer the best flexibility. I would have liked the date command but don't know how to not have the file extension hard coded. I was unaware of the read command or that you could use patterns in the for-loop.
I have learned a lot from this thread so again thank you all. Really my solution is a cross of most of the solutions below as I've taken from them all.
With just Bash built-ins, try
months=(\
January February March April May June \
July August September October November December)
for file in ./*; do
dst=$file
for ((i=1; i<=${#months[#]}; ++i)); do
((i<10)) && i=0$i
dst=${dst//${months[$i]}/$i}
done
mv -- "$file" "$dst"
done
This builds up an array of month names, and loops over it to find the correct substitution.
The line ((i<10)) && i=0$i adds zero padding for single-digit month numbers; remove it if that's undesired.
As an aside, you should basically never use ls in scripts.
The explicit loop could be avoided if you had a command which already knows how to rename files, but this implements that command. If you want to save it in a file, replace the hard-coded ./* with "$#", add a #!/bin/bash shebang up top, save it as monthrenamer somewhere in your PATH, and chmod a+x monthrenamer. Then you can run it like
monthrenamer ./*
to rename all the files in the current directory without an explicit loop, or a more restricted wildcard argument to only select a smaller number of files, like
monthrenamer /path/to/files/2020*.pdf
You could run date twelve times to populate the array, but it's not like hard-coding the month names is going to be a problem. We don't expect them to change (and calling twelve subprocesses at startup just to avoid that seems quite excessive in this context).
As an aside, probably try to fix the process which creates these files to produce machine-readable file names. It's fairly obvious to a human, too, that 2021-07 refers to the month of July, whereas going the other way is always cumbersome (you will need to work around it in every tool or piece of code which wants to order the files by name).
Assuming you have the GNU version of date(1), you could use date -d to map the month names to numbers:
for f in *.pdf; do
IFS=- read y m d <<<"${f%.pdf}"
mv "$f" "$(date -d "$m $d, $y" +%F.pdf)"
done
I doubt it's any more efficient than your sed -e 's/January/01/' -e 's/February/02/' etc, but it does feel less tedious to type. :)
Explanation:
Loop over the .pdf files, setting f to each filename in turn.
The read line is best explained right to left:
a.
"${f%.pdf}" expands to the filename without the .pdf part, e.g. "2020-August-15".
b. <<< turns that value into a here-string, which is a mechanism for feeding a string as standard input to some command. Essentially, x <<<y does the same thing as echo y | x, with the important difference that the x command is run in the current shell instead of a subshell, so it can have side effects like setting variables.
c. read is a shell builtin that by default reads a single line of input and assigns it to one or more shell variables.
d. IFS is a parameter that tells the shell how to split lines up into words. Here we're setting it – only for the duration of the read command – to -. That tells read to split the line it reads on hyphens instead of whitespace; IFS=- read y m d <<<"2020-August-15" assigns "2020" to y, "August" to m, and "15" to d.
The GNU version of date(1) has a -d parameter that tells it to display another date instead of the current one. It accepts a number of different formats itself, sadly not including "yyyy-Mon-dd", which is why I had to split the filename up with read. But it does accept "Mon dd, yyyy", so that's what I pass to it. +%F.pdf tells it that when it prints the date back out it should do so ISO-style as "yyyy-mm-dd", and append ".pdf" to the result. ("%F" is short for "%Y-%m-%d"; I could also have used -I instead of +anything and moved the .pdf outside the command expansion.)
f. The call to date is wrapped in $(...) to capture its output, and that result is used as the second parameter to mv to rename the files.
Another way with POSIX shell:
# Iterate over pattern that will exclude already renamed pdf files
for file in [0-9][0-9][0-9][0-9]-[^0-9]*.pdf
do
# Remove echo if result match expectations
echo mv -- "$file" "$(
# Set field separator to - or . to split filename components
IFS=-.
# Transfer filename components into arguments using IFS
set -- $file
# Format numeric date string
date --date "$3 $2 $1" '+%Y-%m-%d.pdf'
)"
done
If you are using GNU utilities and the Perl version of rename (not the util-linux version), you can build a one-liner quite easily:
rename "$(
seq -w 1 12 |
LC_ALL=C xargs -I# date -d 1970-#-01 +'s/^(\d{4}-)%B(-\d{2}\.pdf)$/$1%m$2/;'
)" *.pdf
You can shorten if you don't care about safety (or legibility)... :-)
rename "$(seq -f%.f/1 12|date -f- +'s/%B/%m/;')" *.pdf
What I mean is I would prefer, and was trying to, pipe ls into a command combination to convert as specified above.
Well, you may need to implement that command combination then. Here’s one consisting of a single “command” and in pure Bash without external processes. Pipe your ls output into that and, once satisfied with the output, remove the final echo…
#!/bin/bash
declare -Ar MONTHS=(
[January]=01
[February]=02
[March]=03
[April]=04
[May]=05
[June]=06
[July]=07
[August]=08
[September]=09
[October]=10
[November]=11
[December]=12)
while IFS= read -r path; do
IFS=- read -ra segments <<<"$path"
segments[-2]="${MONTHS["${segments[-2]}"]}"
IFS=- new_path="${segments[*]}"
echo mv "$path" "$new_path"
done
What is working for me in Mac OS 12.5 with GNU bash, version 3.2.57(1)-release (arm64-apple-darwin21)
is the following :
for f in *.pdf; do mv "$f" "$(echo $f |sed -e 's/Jan/-01-/gi' -e 's/Feb/-02-/gi' -e 's/Mar/-03-/gi' -e 's/Apr/-04-/gi' -e 's/May/-05-/gi' -e 's/jun/-06-/gi' -e 's/Jul/-07-/gi' -e 's/Aug/-08-/gi' -e 's/Sep/-09-/gi' -e 's/Oct/-10-/gi' -e 's/Nov/-11-/gi' -e 's/Dec/-12-/gi' )"; done
Note the original file had the month expressed in three litters in my case :
./04351XXX73435-2021Mar08-2021Apr08.pdf
i'm trying to count all the .txt files in the folders, the problem is that the main folder has more than one folder and inside everyone of them there are txt files , so in total i want to count the number of txt files . till now i've tried to build such a solution,but of course it's wrong:
#!/bin/bash
counter=0
for i in $(ls /Da) ; do
for j in $(ls i) ; do
$counter=$counter+1
done
done
echo $counter
the error i'm getting is :ls cannot access i ...
the problem is that i don't know how i'm supposed to build the inner for loop as it depends on the external for loop(schema) ?
This can work for you
find . -name "*.txt" | wc -l
In the first part find looks for the *.txt from this folder (.) and its subfolders. In the second part wc counts the returnes lines (-l) of find.
You want to avoid parsing ls and you want to quote your variables.
There is no need for repeated loops, either.
printf 'x\n' /Da/* /Da/*/* | wc -l
depending also on whether you expect the entries in /Da to be all files (in which case /Da/* will suffice), all directories (in which case /Da/*/* alone is enough), or both. Additionally, if you don't want to count directories at all, maybe switch to find /Da -type f -printf 'x\n' or similar.
There is no need to print the file names at all; this avoids getting the wrong result if a file name should ever contain a line feed (touch $'/Da/ick\npoo' to see this in action.)
More generally, a correct nested loop looks like
for i in list of things; do
for j in different items, perhaps involving "$i"; do
things with "$j" and perhaps also "$i"
done
done
i is a variable, so you need to reference it via $, i.e. the second loop should be
for j in $(ls "$i") ; do
Let's say I have 100 jpg files.
DSC_0001.jpg
DSC_0002.jpg
DSC_0003.jpg
....
DSC_0100.jpg
And I want to rename them like
summer_trip_1.jpg
summer_trip_2.jpg
summer_trip_3.jpg
.....
summer_trip_100.jpg
So I want these properties to be modified:
1. filename
2. order of the file(as the order by date files were created)
How could I achieve this by using bash? Like:
for file in *.jpg ; do mv blah blah blah ; done
Thank you!
It's very simple: have a variable and increment it at each step.
Example:
cur_number=1
prefix="summer_trip_"
suffix=""
for file in *.jpg ; do
echo mv "$file" "${prefix}${cur_number}${suffix}.jpg" ;
let cur_number=$cur_number+1 # or : cur_number=$(( $cur_number + 1 ))
done
and once you think it's ready, take out the echo to let the mv occur.
If you prefer them to be ordered by file date (usefull, for example, when mixing photos from several cameras, of if on yours the "numbers" rolled over):
change
for file in *.jpg ; do
into
for file in $( ls -1t *.jpg ) ; do
Note that that second example will only work if your original filenames don't have space (and other weird characters) in them, which is fine with almost all cameras I know about.
Finally, instead of ${cur_number} you could use $(printf '%04s' "${cur_number}") so that the new number has leading zeros, making sorting much easier.
How about using rename ?
rename DSC_ summer_trip_ *.jpg
See man page of rename
This works if your original numbers are all padded with zeroes to the same length:
i=0; for f in DSC_*.jpg; do mv "$f" "summer_trip_$((++i)).jpg"; done
If I understand your goal correctly:
So I want these properties to be modified: 1. filename 2. order of the file(as the order by date files were created)
if numbers of renamed files shall increment in order by file creation date, then use the following for loop:
for file in $(ls -t -r *.jpg); do
-t sorts by mtime (last modification time, not exactly creation time, newest first) and -r reverses the order listing oldest first. Just in case if original .jpg file numbers are not in the same order as pictures were taken.
As it was mentioned previously, this won't work if file names have whitespaces. If your files have spaces please try modifying IFS variable before for loop:
IFS=$'\n'
It will force split of 'ls' command results on newlines only (not whitespaces). Also it would fail if there is a newline in a file name (rather exotic IMHO :). Changing IFS may have some subtle effects further in your script, so you can save old one and restore after the loop.
I have a script where inotifywait is piped into a while loop that executes the following logic.
cp "$S3"/2/post2.png "$S3";
mv "$S3"/1/post1.png "$S3"/2/post2.png;
cp "$S3"/3/post3.png "$S3";
mv "S3"/post2.png "$S3"/3/post3.png;
so forth and so on..... then at the end of the script...
mv "$dir"/$file "$S3"/1/post1.png
That line represents a fresh post, the above is the rotation of older post.
I can can hand code the iterations all the way down to 100+, but I would like to program more efficiently and save time.
So, what's some correct ways to loop this?
I think a better mechanism would list the directories in "$S3" in reverse numeric order, and arrange to process them like that. It isn't clear if the 100 directories are all present or whether they need to be created. We'll assume that directories 1..100 might exist, and directory N will always and only contain postN.png.
I'm assuming that there are no spaces, newlines or other awkward characters in the file paths; this means that ls can be used without too much risk.
for dirnum in $(cd "$S3"; ls */*.png | sed 's%/.*%%' | sort -nr)
do
next=$(($dirnum + 1))
mv "$S3/$dirnum/post$dirnum.png" "$S3/$next/post$next.png"
done
The cd "$S3" means I don't get a possibly long pathname included in the output; the ls */*.png lists the files that exist; the sed removes the file name and slash, leaving just a list of directory numbers containing files; and the sort puts the directories in reverse numeric order.
The rest is straight-forward, given the assumption that the necessary directories already exist. It would not be hard to add [ -d "$S3/$next" ] || mkdir -p "$S3/$next" before moving the file. Clearly, after the loop you can use your final command:
mv "$dir/$file" "$S3/1/post1.png"
Note that I've enclosed complete names in double quotes; it generally leads to fewer nasty surprises if something acquires spaces unexpectedly.
Try this:
for i in $(ls -r1 "$3"); do
mkdir -p "$3/$((i+1))"
mv "$3/$i/post$i.png" "$3/$((i+1))/post$((i+1)).png"
done
mv "$dir"/$file "$S3"/1/post1.png
The loop will iterate through all directories in reverse order and move the files.
I'm after a little help with some Bash scripting (on OSX). I want to create a script that takes two parameters - source folder and target folder - and checks all files in the source hierarchy to see whether or not they exist in the target hierarchy. i.e. Given a data DVD check whether the files contained on it are already on the internal drive.
What I've come up with so far is
#!/bin/bash
if [ $# -ne 2 ]
then
echo "Usage is command sourcedir targetdir"
exit 0
fi
source="$1"
target="$2"
for f in "$( find $source -type f -name '*' -print )"
do
I'm now not sure how it's best to obtain the filename without its path and then see if it exists. I am really a beginner at scripting.
Edit: The answers given so far are all very efficient in terms of compact code. However I need to be able to look for files found within the total source hierarchy anywhere within the target hierarchy. If found I would like to compare checksums and last modified dates etc and comment or, if not found, I would like to note this. The purpose is to check whether files on external media have been uploaded to a file server.
This should give you some ideas:
#!/bin/bash
DIR1="tmpa"
DIR2="tmpb"
function sorted_contents
{
cd "$1"
find . -type f | sort
}
DIR1_CONTENTS=$(sorted_contents "$DIR1")
DIR2_CONTENTS=$(sorted_contents "$DIR2")
diff -y <(echo "$DIR1_CONTENTS") <(echo "$DIR2_CONTENTS")
In my test directories, the output was:
[user#host so]$ ./dirdiff.sh
./address-book.dat ./address-book.dat
./passwords.txt ./passwords.txt
./some-song.mp3 <
./the-holy-grail.info ./the-holy-grail.info
> ./victory.wav
./zzz.wad ./zzz.wad
If its not clear, "some-song.mp3" was only in the first directory while "victory.wav" was only in the second. The rest of the files were common.
Note that this only compares the file names, not the contents. If you like where this is headed, you could play with the diff options (maybe --suppress-common-lines if you want cleaner output).
But this is probably how I'd approach it -- offload a lot of the work onto diff.
EDIT: I should also point out that something as simple as:
[user#host so]$ diff tmpa tmpb
would also work:
Only in tmpa: some-song.mp3
Only in tmpb: victory.wav
... but not feel as satisfying as writing a script yourself. :-)
To list only files in $source_dir that do not exist in $target_dir:
comm -23 <(cd "$source_dir" && find .|sort) <(cd "$target_dir" && find .|sort)
You can limit it to just regular files with -f on the find commands, etc.
The comm command (short for "common") finds lines in common between two text files and outputs three columns: lines only in the first file, lines only in the second file, and lines common to both. The numbers suppress the corresponding column, so the output of comm -23 is only the lines from the first file that don't appear in the second.
The process substitution syntax <(command) is replaced by the pathname to a named pipe connected to the output of the given command, which lets you use a "pipe" anywhere you could put a filename, instead of only stdin and stdout.
The commands in this case generate lists of files under the two directories - the cd makes the output relative to the directories being compared, so that corresponding files come out as identical strings, and the sort ensures that comm won't be confused by the same files listed in different order in the two folders.
A few remarks about the line for f in "$( find $source -type f -name '*' -print )":
Make that "$source". Always use double quotes around variable substitutions. Otherwise the result is split into words that are treated as wildcard patterns (a historical oddity in the shell parsing rules); in particular, this would fail if the value of the variable contain spaces.
You can't iterate over the output of find that way. Because of the double quotes, there would be a single iteration through the loop, with $f containing the complete output from find. Without double quotes, file names containing spaces and other special characters would trip the script.
-name '*' is a no-op, it matches everything.
As far as I understand, you want to look for files by name independently of their location, i.e. you consider /dvd/path/to/somefile to be a match to /internal-drive/different/path-to/somefile. So make an list of files on each side indexed by name. You can do this by massaging the output of find a little. The code below can cope with any character in file names except newlines.
list_files () {
find . -type f -print |
sed 's:^\(.*\)/\(.*\)$:\2/\1/\2:' |
sort
}
source_files="$(cd "$1" && list_files)"
dest_files="$(cd "$2" && list_files)"
join -t / -v 1 <(echo "$source_files") <(echo "$dest_files") |
sed 's:^[^/]*/::'
The list_files function generates a list of file names with paths, and prepends the file name in front of the files, so e.g. /mnt/dvd/some/dir/filename.txt will appear as filename.txt/./some/dir/filename.txt. It then sorts the files.
The join command prints out lines like filename.txt/./some/dir/filename.txt when there is a file called filename.txt in the source hierarchy but not in the destination hierarchy. We finally massage its output a little since we no longer need the filename at the beginning of the line.