unix command to find most recent directory created - shell

I want to copy the files from the most recent directory created. How would I do so in unix?
For example, if I have the directories names as date stamp as such:
/20110311
/20110318
/20110325

This is the answer to the question I think you are asking.
When I deal with many directories that have date/time stamps in the name, I always take the approach that you have which is YYYYMMDD - the great thing about that is that the date order is then also the alphabetical order. In most shells (certainly in bash and I am 90% sure of the others), the '*' expansion is done alphabetically, and by default 'ls' return alphabetical order. Hence
ls | head -1
ls | tail -1
Give you the earliest and the latest dates in the directory.
This can be extended to only keep the last 5 entries etc.

lastdir=`ls -tr <parentdir> | tail -1`
I don't know how to make the backticks play nice with the commenting system here. Just replace those apostrophes with backticks.

After some experimenting, I came up with the following:
The unix stat command is useful here. The '-t' option causes stat to print its output in terse mode (all in one line), and the 13th element of that terse output is the unix timestamp (seconds since epoch) for the last-modified time. This command will list all directories (and sub-directories) in order from newest-modified to oldest-modified:
find -type d -exec stat -t {} \; | sort -r -n -k 13,13
Hopefully the "terse" mode of stat will remain consistent in future releases of stat !
Here's some explanation of the command-line options used:
find -type d # only find directories
find -exec [command] {} \; # execute given command against each *found* file.
sort -r # reverse the sort
sort -n # numeric sort (100 should not appear before 2!)
sort -k M,N # only sort the line using elements M through N.
Returning to your original request, to copy files, maybe try the following. To output just a single directory (the most recent), append this to the command (notice the initial pipe), and feed it all into your 'cp' command with backticks.
| head --lines=1 | sed 's/\ .*$//'

The trouble with the ls based solutions is that they are not filtering just for directories. I think this:
cp `find . -mindepth 1 -maxdepth 1 -type d -exec stat -c "%Y %n" {} \; |sort -n -r |head -1 |awk '{print $2}'`/* /target-directory/.
might do the trick, though note that that will only copy files in the immediate directory. If you want a more general answer for copying anything below your newest directory over to a new directory I think you would be better off using rsync like:
rsync -av `find . -mindepth 1 -maxdepth 1 -type d -exec stat -c "%Y %n" {} \; |sort -n -r |head -1 |awk '{print $2}'`/ /target-directory/
but it depends a bit which behaviour you want. The explanation of the stuff in the backticks is:
. - the current directory (you may want to specify an absolute path here)
-mindepth/-maxdepth - restrict the find command only to the immediate children of the current directory
-type d - only directories
-exec stat .. - outputs the modified time and the name of the directory from find
sort -n -r |head -1 | awk '{print $2}' - date orders the directory and outputs the name of the most recently modified

If your directories are named YYYYMMDD like your question suggests, take advantage of the alphabetic globbing.
Put all directories in an array, and then pick the first one:
dirs=(*/); first_dir="$dirs";
(This is actually a shortcut for first_dir="${dirs[0]}";.)
Similarly, for the last one:
dirs=(*/); last_dir="${dirs[$((${#dirs[#]} - 1))]}";
Ugly syntax, but this is what it breaks down to:
# Create an array of all directories inside the working directory.
dirs=(*/);
# Get the number of entries in the array.
num_dirs=${#dirs[#]};
# Calculate the index of the last entry.
last_index=$(($num_dirs - 1));
# Get the value at the last index.
last_dir="${dirs[$last_index]}";
I know this is an old question with an accepted answer, but I think this method is preferable as it does everything in Bash. No reason to spawn extra processes, let alone parse the output of ls. (Which, admittedly, should be fine in this particular case of YYYYMMDD names.)

please try with following command
ls -1tr | tail -1

find ~ -type d | ls -ltra
This one is simple and useful which I learned recently.
This command will show the results in reverse chronological order.

I wrote a command that can be used to identify which folder or files are created in a folder as a newest. That's seems pure :)
#/bin/sh
path=/var/folder_name
newest=`find $path -maxdepth 1 -exec stat -t {} \; |sed 1d |sort -r -k 14 | head -1 |awk {'print $1'} | sed 's/\.\///g'`
find $path -maxdepth 1| sed 1d |grep -v $newest

Related

How to get full path of last modified file in directory including nested directories?

How would I modify this code to give me the full file path of the last modified file in the code directory, including nested sub-directories?
# Gets the last modified file in the code directory.
get_filename(){
cd "$code_directory" || no_code_directory_error # Stop script if directory doesn't exist.
last_modified=$(ls -t | head -n1)
echo "$last_modified"
}
Use find instead of ls, because the use of ls is an anti-pattern.
Use a Schwartzian transform to prefix your data with a sort key.
Sort the data.
Take what you need.
Remove the sort key.
Post process the data.
find "$code_directory" -type f -printf '%T# %p\n' |
sort -rn |
head -1 |
sed 's/^[0-9.]\+ //' |
xargs readlink -f
You can use the realpath utility.
# Gets the last modified file in the code directory.
get_filename(){
cd "$code_directory" || no_code_directory_error # Stop script if directory doesn't exist.
last_modified=$(ls -t | head -1)
echo "$last_modified"
realpath "$last_modified"
}
Output:
blah.txt
/full/path/to/blah.txt
ls -t sort by modification time and if you want first one you can add | head -1, R helps you recursively sort files, I think the only tips here is ls -tR doesn't stack all files then sort them, so you can use
find . -type f -printf "%T# %f\n" | sort -rn > out.txt

Batch rename all files in a directory to basename-sequentialnumber.extention

I have a directory containing .jpg files, currently named photo-1.jpg, photo-2.jpg etc. There are about 20,000 of these files, sequentially numbered.
Sometimes I delete some of these files, which creates gaps in the file naming convention.
Can you guys help me with a bash script that would sequentially rename all the files in the directory to eliminate the gaps? I have found many posts about renaming files and tried a bunch of things, but can't quite get exactly what I'm looking for.
For example:
photo-1.jpg
photo-2.jpg
photo-3.jpg
Delete photo-2.jpg
photo-1.jpg
photo-3.jpg
run script to sequentially rename all files
photo-1.jpg
photo-2.jpg
done
With find and sort.
First check the output of
find directory -type f -name '*.jpg' | sort -nk2 -t-
If the output is not what you expected it to be, meaning the order of sorting is not correct, then It might have something to do with your locale. Add the LC_ALL=C before the sort.
find directory -type f -name '*.jpg' | LC_ALL=C sort -nk2 -t-
Redirect it to a file so it can be recorded, add a | tee output.txt after the sort
Add the LC_ALL=C before the sort in the code below if it is needed.
#!/bin/sh
counter=1
find directory -type f -name '*.jpg' |
sort -nk2 -t- | while read -r file; do
ext=${file##*[0-9]} filename=${file%-*}
[ ! -e "$filename-$counter$ext" ] &&
echo mv -v "$file" "$filename-$counter$ext"
counter=$((counter+1))
done # 2>&1 | tee log.txt
Change the directory to the actual name of your directory that contains the files that you need to rename.
If your sort has the -V flag/option then that should work too.
sort -nk2 -t- The -n means numerically sort. -k2 means the second field and the -t- means the delimiter/separator is a dash -, can be written as -t -, caveat, if the directory name has a dash - as well, sorting failure is expected. Adjust the value of -k and it should work.
ext=${file##*[0-9]} is a Parameter Expansion, will remain only the .jpg
filename=${file%-*} also a Parameter Expansion, will remain only the photo plus the directory name before it.
[ ! -e "$filename-$counter$ext" ] will trigger the mv ONLY if the file does not exists.
If you want some record or log, remove the comment # after the done
Remove the echo if you think that the output is correct

Can we store the creation date of a folder (not file) using bash script?

Actually I’m a newbie at Bash and I’m learning with some hands on.. I used the following stat command:
find "$DIRECTORY"/ -exec stat \{} --printf="%w\n" \; | sort -n -r | head -n 1 > timestamp.txt
where DIRECTORY is any path say, c:/some/path . It contains a lot of folders. I need to extract the creation date of the latest created folder and store it in a variable for further use. Here I started by storing it in a txt file. But the script never completes. It stays stuck at the point it reaches this command line. Please help. I'm using cygwin. I had used --printf="%y\n" to extract last Modified date of the latest folder and it had worked fine.
The command is okay (save for escaped \{} which I believe is a mistake in the post). It only seems so that it never finishes, but given enough time, it'll finish.
Direct approach - getting the path
The main bottleneck lies in executing stat for each file. Spawning process under Cygwin is extremely slow, and executing one for each of possibly thousands of files is totally infeasible. The only way to circumvent this is not spawning processes like this.
That said, I see few areas for improvement:
If you need only directories like the title of your post suggests, you can pass -type d to your find command to filter out any files.
If you need only modification time (see what means directory modification time on Linux here, I guess this may be similar in Cygwin), you can use find's built in facilities rather than stat's like this:
find "$DIRECTORY"/ -type d -printf '%TY-%Tm-%Td %TH:%TM:%TS %Tz %p\n' \
| sort -nr \
| head -n1 \
| cut -f4 -d' '
Example line before we cut the path with cut - most of stuff in -printf is used to format the date:
2014-09-25 09:41:50.3907590000 +0200 ./software/sqldeveloper/dataminer/demos/obe
After cut:
./software/sqldeveloper/dataminer/demos/obe
It took 0.7s to scan 560 directories and 2300 files.
The original command from your post took 28s without -type d trick, and 6s with -type d trick when ran on the same directory.
Last but not least, if $DIRECTORY is empty, your command will prune whole directory tree, which will take massive amount of time.
Another approach - getting just the date
If you only need creation date of a subdirectory within a directory (e.g. not the path to the directory), you can probably just use stat:
stat --printf '%Y' "$DIRECTORY"/
I'm not sure whether this includes file creations as well, though.
Alternative approaches
Since getting the last created folder is clearly expensive, you could also either:
Save the directory name somewhere when creating said directory, or
Use naming convention such as ddddyymm-name-of-directory which doesn't require any extra syscalls - just find -type d|....
You could do with a -type d option to include only the directories from the current folder, and as discussed in the comments section if you need the output from the stat in just yyyy-mm-dd format, use awk as below.
find "$DIRECTORY"/ -type d -exec stat \{} --printf="%w\n" \; | sort -n -r | head -n 1 | awk '{print $1}'
To store the value in a bash variable:-
$ myvar=$(find "$DIRECTORY"/ -type d -exec stat \{} --printf="%w\n" \; | sort -n -r | head -n 1 | awk '{print $1}')
$ echo $myvar
2016-05-20

Get the newest directory to a variable in Bash

I would like to find the newest sub directory in a directory and save the result to variable in bash.
Something like this:
ls -t /backups | head -1 > $BACKUPDIR
Can anyone help?
BACKUPDIR=$(ls -td /backups/*/ | head -1)
$(...) evaluates the statement in a subshell and returns the output.
There is a simple solution to this using only ls:
BACKUPDIR=$(ls -td /backups/*/ | head -1)
-t orders by time (latest first)
-d only lists items from this folder
*/ only lists directories
head -1 returns the first item
I didn't know about */ until I found Listing only directories using ls in bash: An examination.
This ia a pure Bash solution:
topdir=/backups
BACKUPDIR=
# Handle subdirectories beginning with '.', and empty $topdir
shopt -s dotglob nullglob
for file in "$topdir"/* ; do
[[ -L $file || ! -d $file ]] && continue
[[ -z $BACKUPDIR || $file -nt $BACKUPDIR ]] && BACKUPDIR=$file
done
printf 'BACKUPDIR=%q\n' "$BACKUPDIR"
It skips symlinks, including symlinks to directories, which may or may not be the right thing to do. It skips other non-directories. It handles directories whose names contain any characters, including newlines and leading dots.
Well, I think this solution is the most efficient:
path="/my/dir/structure/*"
backupdir=$(find $path -type d -prune | tail -n 1)
Explanation why this is a little better:
We do not need sub-shells (aside from the one for getting the result into the bash variable).
We do not need a useless -exec ls -d at the end of the find command, it already prints the directory listing.
We can easily alter this, e.g. to exclude certain patterns. For example, if you want the second newest directory, because backup files are first written to a tmp dir in the same path:
backupdir=$(find $path -type -d -prune -not -name "*temp_dir" | tail -n 1)
The above solution doesn't take into account things like files being written and removed from the directory resulting in the upper directory being returned instead of the newest subdirectory.
The other issue is that this solution assumes that the directory only contains other directories and not files being written.
Let's say I create a file called "test.txt" and then run this command again:
echo "test" > test.txt
ls -t /backups | head -1
test.txt
The result is test.txt showing up instead of the last modified directory.
The proposed solution "works" but only in the best case scenario.
Assuming you have a maximum of 1 directory depth, a better solution is to use:
find /backups/* -type d -prune -exec ls -d {} \; |tail -1
Just swap the "/backups/" portion for your actual path.
If you want to avoid showing an absolute path in a bash script, you could always use something like this:
LOCALPATH=/backups
DIRECTORY=$(cd $LOCALPATH; find * -type d -prune -exec ls -d {} \; |tail -1)
With GNU find you can get list of directories with modification timestamps, sort that list and output the newest:
find . -mindepth 1 -maxdepth 1 -type d -printf "%T#\t%p\0" | sort -z -n | cut -z -f2- | tail -z -n1
or newline separated
find . -mindepth 1 -maxdepth 1 -type d -printf "%T#\t%p\n" | sort -n | cut -f2- | tail -n1
With POSIX find (that does not have -printf) you may, if you have it, run stat to get file modification timestamp:
find . -mindepth 1 -maxdepth 1 -type d -exec stat -c '%Y %n' {} \; | sort -n | cut -d' ' -f2- | tail -n1
Without stat a pure shell solution may be used by replacing [[ bash extension with [ as in this answer.
Your "something like this" was almost a hit:
BACKUPDIR=$(ls -t ./backups | head -1)
Combining what you wrote with what I have learned solved my problem too. Thank you for rising this question.
Note: I run the line above from GitBash within Windows environment in file called ./something.bash.

remove old backup files

# find /home/shantanu -name 'my_stops*' | xargs ls -lt | head -2
The command mentioned above will list the latest 2 files having my_stops in it's name. I want to keep these 2 files. But I want to delete all other files starting with "my_stops" from the current directory.
If you create backups on a regular basis, it may be useful to use the -atime option of find so only files older than your last two backups can be selected for deletion.
For daily backups you might use
$ find /home/shantanu -atime +2 -name 'my_stops*' -exec rm {} \;
but a different expression (other than -atime) may suit you better.
In the example I used +2 to mean more than 2 days.
Here is a non-recursive solution:
ls -t my_stops* | awk 'NR>2 {system("rm \"" $0 "\"")}'
Explanation:
The ls command lists files with the latest 2 on top
The awk command states that for those lines (NR = number of records, i.e. lines) greater than 2, delete them
The quote characters are needed just in case the file names have embedded spaces
See here
(ls -t|head -n 2;ls)|sort|uniq -u|xargs rm
That will show you from the second line forward ;)
find /home/shantanu -name 'my_stops*' | xargs ls -lt | tail -n +2
Just keep in mind that find is recursive ;)
Without recursive approach:
find /home/folder/ -maxdepth 1 -name "*.jpg" -mtime +2

Resources