I need to list out the files which are created in a specific year and then to delete the files. year should be the input.
i tried with date it is working for me. but not able to covert that date to year for comparison in loop to get the list of files.
Below code is giving 05/07 files. but want to list out the files which are created in 2022,2021,etc.,
for file in /tmp/abc*txt ; do
[ "$(date -I -r "$file")" == "2022-05-07" ] && ls -lstr "$file"
done
If you end up doing ls -l anyway, you might just parse the date information from the output. (However, generally don't use ls in scripts.)
ls -ltr | awk '$8 ~ /^202[01]$/'
date -r is not portable, though if you have it, you could do
for file in /tmp/abc*txt ; do
case $(date -I -r "$file") in
2020-* | 2021-* ) ls -l "$file";;
esac
done
(The -t and -r flags to ls have no meaning when you are listing a single file anyway.)
If you don't, the tool of choice would be stat, but it too has portability issues; the precise options to get the information you want will vary between platforms. On Linux, try
for file in /tmp/abc*txt ; do
case $(LC_ALL=C stat -c %y "$file") in
2020-* | 2021-* ) ls -l "$file";;
esac
done
On BSD (including MacOS) try stat -f %Sm -t %Y "$file" to get just the year.
If you need proper portability, perhaps look for a scripting language with wide support, such as Perl or Python. The stat() system call is the fundamental resource for getting metainformation about a file. The find command also has some features for finding files by age, though its default behavior is to traverse subdirectories, too (you can inhibit that with -maxdepth 1; but then the options to select files by age are again not entirely POSIX portable).
To list out files which were last modified in a specific year and then to delete those files, you could use a combination of the find -newer and touch commands:
# given a year as input
year=2022
stampdir=$(mktemp -d)
touch -t ${year}01010000 "$stampdir"/beginning
touch -t $((year+1))01010000 "$stampdir"/end
find /tmp -name 'abc*txt' -type f -newer "$stampdir/beginning" ! -newer "$stampdir/end" -print -delete
rm -r "$stampdir"
First, create a temporary working directory to store the timestamp files; we don't want the find command to accidentally find them. Be careful here; mktemp will probably create a directory in /tmp; this use-case is safe only because we're naming the timestamp files such that they don't match the "abc*txt" pattern from the question.
Next, create bordering timestamp files with the touch command: one that is the newest date in the year, named "beginning", and another for the newest date of the next year, named "end".
Then run the find command; here's the breakdown:
start in /tmp (from the question)
files named with the 'abc*txt' pattern (from the question)
only files (not directories, etc -- from the question)
newer than the beginning timestamp file
not newer (i.e. older) than the end timestamp file
if found, print the filename and then delete it
Finally, clean up the temporary working directory that we created.
Try this:
For checking which files are picked up:
echo -e "Give Year :"
read yr
ls -ltr /tmp | grep "^-" |grep -v ":" | grep $yr | awk -F " " '{ print $9;}'
** You can replace { print $9 ;} with { rm $9; } in the above command for deleting the picked files
I need to write a Bash script to check if there are any new folders in a path, if yes do something and if not then simply exit.
My thinking is to create a text file to keep track of all old folders and do a diff, if there something new then perform action. Please help me achieve this:
I've tried to use two file tracking but I don't think I've got this right.
The /tmp/ folder has multiple sub folders
#/bin/sh
BASEDIR=/tmp/
cd $BASEDIR
ls -A $BASEDIR >> newfiles.txt
DIRDIFF=$(diff oldfiles.txt newfiles.txt | cut -f 2 -d "")
for file in $DIRDIFF
do
if [ -e $BASEDIR/$file ]
then echo $file
fi
done
Generally don't use ls in scripts. Here is a simple refactoring which avoids it.
#!/bin/sh
printf '%s\n' /tmp/.[!.]* /tmp/* >newfiles.txt
if cmp -13 oldfiles.txt newfiles.txt | grep .; then
rc=0
rm newfiles.txt
else
rc=$?
mv newfiles.txt oldfiles.txt
fi
exit "$rc"
Using comm instead of diff simplifies processing somewhat (the wildcard should expand the files in sorted order, so the requirement for sorted input will be satisfied) and keeping the files in the current directory (instead of in /tmp) should avoid having the script trigger itself. The output from comm will be the new files, so there is no need to process it further. The grep . checks if there are any output lines, so that we can set the exit status to reflect whether or not there were new entries.
Your script looks for files, not directories. If you really want to look for new directories, add a slash after each wildcard expression:
printf '%s\n' /tmp/.[!.]*/ /tmp/*/ >newfiles.txt
This will not notice if an existing file or directory is modified. Probably switch to inotifywait if you need anything more sophisticated (or perhaps even if this is all you need).
Let's assume you are interested in sub-directories only. Let's assume too that you do not have directory names with newlines in them.
Do not process ls output, it is for humans only. Prefer find. Then, simply sort the old and new lists with sort and compare them with comm, keeping only the lines found in the new list but not in the old list (man comm will tell you why the -13 option does this). In the following the do something is just echo, replace by whatever is needed:
#!/bin/sh
BASEDIR=/tmp/
cd "$BASEDIR"
find . -type d | sort > new.txt
if [ -f old.txt ]; then
comm -13 old.txt new.txt | while IFS= read -r name; do
echo "$name"
done
fi
mv new.txt old.txt
This will explore the complete hierarchy under the starting point, and consider any directory. If you do not want to explore the complete hierarchy under the starting point but only the current level:
find . -maxdepth 1 -type d | sort > new.txt
I have a folder, e.g. named 'folder'. There are 50000 txt files under it, e.g, '00001.txt, 00002.txt, etc'.
Now I want to use one command line to show the head 10 lines in '00001.txt'. I have tried:
ls folder | head -1
which will show the filename of the first:
00001.txt
But I want to show the contents of folder/00001.txt
So, how do I do something like os.path.join(folder, xx) and show its head -10?
The better way to do this is not to use ls at all; see Why you shouldn't parse the output of ls, and the corresponding UNIX & Linux question Why not parse ls (and what to do instead?).
On a shell with arrays, you can glob into an array, and refer to items it contains by index.
#!/usr/bin/env bash
# ^^^^- bash, NOT sh; sh does not support arrays
# make array files contain entries like folder/0001.txt, folder/0002.txt, etc
files=( folder/* ) # note: if no files found, it will be files=( "folder/*" )
# make sure the first item in that array exists; if it does't, that means
# the glob failed to expand because no files matching the string exist.
if [[ -e ${files[0]} || -L ${files[0]} ]]; then
# file exists; pass the name to head
head -n 10 <"${files[0]}"
else
# file does not exist; spit out an error
echo "No files found in folder/" >&2
fi
If you wanted more control, I'd probably use find. For example, to skip directories, the -type f predicate can be used (with -maxdepth 1 to turn off recursion):
IFS= read -r -d '' file < <(find folder -maxdepth 1 -type f -print0 | sort -z)
head -10 -- "$file"
Although hard to understand what you are asking but I think something like this will work:
head -10 $(ls | head -1)
Basically, you get the file from $(ls | head -1) and then print the content.
If you invoke the ls command as ls "$PWD"/folder, it will include the absolute path of the file in the output.
I am taking one of important folder every day backup by using cron. That folder name it will store with the current date.
Now my requirement is i need to keep only the current day and last two days backup.
i.e I want to keep only:
test_2016-11-04.tgz
test_2016-11-03.tgz
test_2016-11-02.tgz
Remaining folder it has to delete automatically. Please let us know how to do in shell script.
Below is my backup folder structure.
test_2016-10-30.tgz test_2016-11-01.tgz test_2016-11-03.tgz
test_2016-10-31.tgz test_2016-11-02.tgz test_2016-11-04.tgz
With ls -lrt | head -n -3 | awk '{print $9}
you can print all but the last 3 files in your directory.
Passing this output into rm you obtain the result desidered.
you could append end of backup script;
find ./backupFolder -name "test_*.tgz" -mtime +3 -type f -delete
also use this;
ls -1 test_*.tgz | sort -r | awk 'NR > 3 { print }' | xargs -d '\n' rm -f --
Generate an array on files you want to keep:
names=()
for d in {0..2}; do
names+=( "test_"$(date -d"$d days ago" "+%Y-%m-%d")".tgz" )
done
so that it looks like this:
$ printf "%s\n" "${names[#]}"
test_2016-11-04.tgz
test_2016-11-03.tgz
test_2016-11-02.tgz
Then, loop through the files and keep those that are not in the array:
for file in test_*.tgz; do
[[ ! ${names[*]} =~ "$file" ]] && echo "remove $file" || echo "keep $file"
done
If ran on your directory, this would result on an output like:
remove test_2016-10-30.tgz
remove test_2016-10-31.tgz
remove test_2016-11-01.tgz
keep test_2016-11-02.tgz
keep test_2016-11-03.tgz
keep test_2016-11-04.tgz
So now it is just a matter or replacing those echo with something more meaningful like rm.
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 28 days ago.
The community reviewed whether to reopen this question 28 days ago and left it closed:
Original close reason(s) were not resolved
Improve this question
This command lists directories in the current path:
ls -d */
What exactly does the pattern */ do?
And how can we give the absolute path in the above command (e.g. ls -d /home/alice/Documents) for listing only directories in that path?
*/ is a pattern that matches all of the subdirectories in the current directory (* would match all files and subdirectories; the / restricts it to directories). Similarly, to list all subdirectories under /home/alice/Documents, use ls -d /home/alice/Documents/*/
Four ways to get this done, each with a different output format
1. Using echo
Example: echo */, echo */*/
Here is what I got:
cs/ draft/ files/ hacks/ masters/ static/
cs/code/ files/images/ static/images/ static/stylesheets/
2. Using ls only
Example: ls -d */
Here is exactly what I got:
cs/ files/ masters/
draft/ hacks/ static/
Or as list (with detail info): ls -dl */
3. Using ls and grep
Example: ls -l | grep "^d"
Here is what I got:
drwxr-xr-x 24 h staff 816 Jun 8 10:55 cs
drwxr-xr-x 6 h staff 204 Jun 8 10:55 draft
drwxr-xr-x 9 h staff 306 Jun 8 10:55 files
drwxr-xr-x 2 h staff 68 Jun 9 13:19 hacks
drwxr-xr-x 6 h staff 204 Jun 8 10:55 masters
drwxr-xr-x 4 h staff 136 Jun 8 10:55 static
4. Bash Script (Not recommended for filename containing spaces)
Example: for i in $(ls -d */); do echo ${i%%/}; done
Here is what I got:
cs
draft
files
hacks
masters
static
If you like to have '/' as ending character, the command will be: for i in $(ls -d */); do echo ${i}; done
cs/
draft/
files/
hacks/
masters/
static/
I use:
ls -d */ | cut -f1 -d'/'
This creates a single column without a trailing slash - useful in scripts.
For all folders without subfolders:
find /home/alice/Documents -maxdepth 1 -type d
For all folders with subfolders:
find /home/alice/Documents -type d
Four (more) Reliable Options.
An unquoted asterisk * will be interpreted as a pattern (glob) by the shell. The shell will use it in pathname expansion. It will then generate a list of filenames that match the pattern.
A simple asterisk will match all filenames in the PWD (present working directory). A more complex pattern as */ will match all filenames that end in /. Thus, all directories. That is why the command:
1.- echo.
echo */
echo ./*/ ### Avoid misinterpreting filenames like "-e dir"
will be expanded (by the shell) to echo all directories in the PWD.
To test this: Create a directory (mkdir) named like test-dir, and cd into it:
mkdir test-dir; cd test-dir
Create some directories:
mkdir {cs,files,masters,draft,static} # Safe directories.
mkdir {*,-,--,-v\ var,-h,-n,dir\ with\ spaces} # Some a bit less secure.
touch -- 'file with spaces' '-a' '-l' 'filename' # And some files:
The command echo ./*/ will remain reliable even with odd named files:
./--/ ./-/ ./*/ ./cs/ ./dir with spaces/ ./draft/ ./files/ ./-h/
./masters/ ./-n/ ./static/ ./-v var/
But the spaces in filenames make reading a bit confusing.
If instead of echo, we use ls. The shell is still what is expanding the list of filenames. The shell is the reason to get a list of directories in the PWD. The -d option to ls makes it list the present directory entry instead of the contents of each directory (as presented by default).
ls -d */
However, this command is (somewhat) less reliable. It will fail with the odd named files listed above. It will choke with several names. You need to erase one by one till you find the ones with problems.
2.- ls
The GNU ls will accept the "end of options" (--) key.
ls -d ./*/ ### More reliable BSD ls
ls -d -- */ ### More reliable GNU ls
3.-printf
To list each directory in its own line (in one column, similar to ls -1), use:
$ printf "%s\n" */ ### Correct even with "-", spaces or newlines.
And, even better, we could remove the trailing /:
$ set -- */; printf "%s\n" "${#%/}" ### Correct with spaces and newlines.
An attempt like
$ for i in $(ls -d */); do echo ${i%%/}; done
will fail on:
some names (ls -d */) as already shown above.
will be affected by the value of IFS.
will split names on spaces and tabs (with default IFS).
each newline in the name will start a new echo command.
4.- Function
Finally, using the argument list inside a function will not affect the arguments list of the present running shell. Simply
$ listdirs(){ set -- */; printf "%s\n" "${#%/}"; }
$ listdirs
presents this list:
--
-
*
cs
dir with spaces
draft
files
-h
masters
-n
static
-v var
These options are safe with several types of odd filenames.
The tree command is also pretty useful here. By default it will show all files and directories to a complete depth, with some ASCII characters showing the directory tree.
$ tree
.
├── config.dat
├── data
│ ├── data1.bin
│ ├── data2.inf
│ └── sql
| │ └── data3.sql
├── images
│ ├── background.jpg
│ ├── icon.gif
│ └── logo.jpg
├── program.exe
└── readme.txt
But if we wanted to get just the directories, without the ASCII tree, and with the full path from the current directory, you could do:
$ tree -dfi
.
./data
./data/sql
./images
The arguments being:
-d List directories only.
-f Prints the full path prefix for each file.
-i Makes tree not print the indentation lines, useful when used in conjunction with the -f option.
And if you then want the absolute path, you could start by specifying the full path to the current directory:
$ tree -dfi "$(pwd)"
/home/alice/Documents
/home/alice/Documents/data
/home/alice/Documents/data/sql
/home/alice/Documents/images
And to limit the number of subdirectories, you can set the max level of subdirectories with -L level, e.g.:
$ tree -dfi -L 1 "$(pwd)"
/home/alice/Documents
/home/alice/Documents/data
/home/alice/Documents/images
More arguments can be seen with man tree.
In case you're wondering why output from 'ls -d */' gives you two trailing slashes, like:
[prompt]$ ls -d */
app// cgi-bin// lib// pub//
it's probably because somewhere your shell or session configuration files alias the ls command to a version of ls that includes the -F flag. That flag appends a character to each output name (that's not a plain file) indicating the kind of thing it is. So one slash is from matching the pattern '*/', and the other slash is the appended type indicator.
To get rid of this issue, you could of course define a different alias for ls. However, to temporarily not invoke the alias, you can prepend the command with backslash:
\ls -d */
Actual ls solution, including symlinks to directories
Many answers here don't actually use ls (or only use it in the trivial sense of ls -d, while using wildcards for the actual subdirectory matching. A true ls solution is useful, since it allows the use of ls options for sorting order, etc.
Excluding symlinks
One solution using ls has been given, but it does something different from the other solutions in that it excludes symlinks to directories:
ls -l | grep '^d'
(possibly piping through sed or awk to isolate the file names)
Including symlinks
In the (probably more common) case that symlinks to directories should be included, we can use the -p option of ls, which makes it append a slash character to names of directories (including symlinked ones):
ls -1p | grep '/$'
or, getting rid of the trailing slashes:
ls -1p | grep '/$' | sed 's/\/$//'
We can add options to ls as needed (if a long listing is used, the -1 is no longer required).
Note: if we want trailing slashes, but don't want them highlighted by grep, we can hackishly remove the highlighting by making the actual matched portion of the line empty:
ls -1p | grep -P '(?=/$)'
A plain list of the current directory, it'd be:
ls -1d */
If you want it sorted and clean:
ls -1d */ | cut -c 1- | rev | cut -c 2- | rev | sort
Remember: capitalized characters have different behavior in the sort
I just add this to my .bashrc file (you could also just type it on the command line if you only need/want it for one session):
alias lsd='ls -ld */'
Then lsd will produce the desired result.
Here is what I am using
ls -d1 /Directory/Path/*;
If a hidden directory is not needed to be listed, I offer:
ls -l | grep "^d" | awk -F" " '{print $9}'
And if hidden directories are needed to be listed, use:
ls -Al | grep "^d" | awk -F" " '{print $9}'
Or
find -maxdepth 1 -type d | awk -F"./" '{print $2}'
For listing only directories:
ls -l | grep ^d
For listing only files:
ls -l | grep -v ^d
Or also you can do as:
ls -ld */
Try this one. It works for all Linux distribution.
ls -ltr | grep drw
ls and awk (without grep)
No need to use grep since awk can perform regularexpressino check so it is enough to do this:
ls -l | awk '/^d/ {print $9}'
where ls -l list files with permisions
awk filter output
'/^d/' regularexpresion that search only for lines starting with letter d (as directory) looking at first line - permisions
{print} would prints all columns
{print $9} will print only 9th column (name) from ls -l output
Very simple and clean
To show folder lists without /:
ls -d */|sed 's|[/]||g'
I found this solution the most comfortable, I add to the list:
find * -maxdepth 0 -type d
The difference is that it has no ./ at the beginning, and the folder names are ready to use.
Test whether the item is a directory with test -d:
for i in $(ls); do test -d $i && echo $i ; done
FYI, if you want to print all the files in multi-line, you can do a ls -1 which will print each file in a separate line.
file1
file2
file3
*/ is a filename matching pattern that matches directories in the current directory.
To list directories only, I like this function:
# Long list only directories
llod () {
ls -l --color=always "$#" | grep --color=never '^d'
}
Put it in your .bashrc file.
Usage examples:
llod # Long listing of all directories in current directory
llod -tr # Same but in chronological order oldest first
llod -d a* # Limit to directories beginning with letter 'a'
llod -d .* # Limit to hidden directories
Note: it will break if you use the -i option. Here is a fix for that:
# Long list only directories
llod () {
ls -l --color=always "$#" | egrep --color=never '^d|^[[:digit:]]+ d'
}
file * | grep directory
Output (on my machine) --
[root#rhel6 ~]# file * | grep directory
mongo-example-master: directory
nostarch: directory
scriptzz: directory
splunk: directory
testdir: directory
The above output can be refined more by using cut:
file * | grep directory | cut -d':' -f1
mongo-example-master
nostarch
scriptzz
splunk
testdir
* could be replaced with any path that's permitted
file - determine file type
grep - searches for string named directory
-d - to specify a field delimiter
-f1 - denotes field 1
One-liner to list directories only from "here".
With file count.
for i in `ls -d */`; do g=`find ./$i -type f -print| wc -l`; echo "Directory $i contains $g files."; done
Using Perl:
ls | perl -nle 'print if -d;'
I partially solved it with:
cd "/path/to/pricipal/folder"
for i in $(ls -d .*/); do sudo ln -s "$PWD"/${i%%/} /home/inukaze/${i%%/}; done
ln: «/home/inukaze/./.»: can't overwrite a directory
ln: «/home/inukaze/../..»: can't overwrite a directory
ln: accesing to «/home/inukaze/.config»: too much symbolics links levels
ln: accesing to «/home/inukaze/.disruptive»: too much symbolics links levels
ln: accesing to «/home/inukaze/innovations»: too much symbolics links levels
ln: accesing to «/home/inukaze/sarl»: too much symbolics links levels
ln: accesing to «/home/inukaze/.e_old»: too much symbolics links levels
ln: accesing to «/home/inukaze/.gnome2_private»: too much symbolics links levels
ln: accesing to «/home/inukaze/.gvfs»: too much symbolics links levels
ln: accesing to «/home/inukaze/.kde»: too much symbolics links levels
ln: accesing to «/home/inukaze/.local»: too much symbolics links levels
ln: accesing to «/home/inukaze/.xVideoServiceThief»: too much symbolics links levels
Well, this reduce to me, the major part :)
Here is a variation using tree which outputs directory names only on separate lines, yes it's ugly, but hey, it works.
tree -d | grep -E '^[├|└]' | cut -d ' ' -f2
or with awk
tree -d | grep -E '^[├|└]' | awk '{print $2}'
This is probably better however and will retain the / after directory name.
ls -l | awk '/^d/{print $9}'
if you have space in your folder name $9 print wont work try below command
ls -l yourfolder/alldata/ | grep '^d' | awk '{print $9" " $10}'
output
ls -l yourfolder/alldata/ | grep '^d' | awk '{print $9" " $10}'
testing_Data
Folder 1
To answer the original question, */ has nothing to do with ls per se; it is done by the shell/Bash, in a process known as globbing.
This is why echo */ and ls -d */ output the same elements. (The -d flag makes ls output the directory names and not contents of the directories.)
Adding on to make it full circle, to retrieve the path of every folder, use a combination of Albert's answer as well as Gordans. That should be pretty useful.
for i in $(ls -d /pathto/parent/folder/*/); do echo ${i%%/}; done
Output:
/pathto/parent/folder/childfolder1/
/pathto/parent/folder/childfolder2/
/pathto/parent/folder/childfolder3/
/pathto/parent/folder/childfolder4/
/pathto/parent/folder/childfolder5/
/pathto/parent/folder/childfolder6/
/pathto/parent/folder/childfolder7/
/pathto/parent/folder/childfolder8/
Here is what I use for listing only directory names:
ls -1d /some/folder/*/ | awk -F "/" "{print \$(NF-1)}"