I have the following find command that searches all subdirectories and lists those folders that contain a *.RAR AND a *.MKV file.
find -type d -exec sh -c '[ -f "$0"/*.rar ] && [ -f "$0"/*.mkv ]' '{}' \; -print | sort
What I want to do now is to delete the *.MKV file from those directories.
For example, the above command finds FILEA.RAR and FILEB.MKV and lists the directory as DIRECTORY_CHARLIE. I would like to be able to have the above code, also delete the FILEB.MKV file, well delete the found MKV file from each directory that had both file types.
To start, I've created "_TestDir" with the following subfolders:
#useroneserver ~/files/_TestDir $ ls -all
drwxr-xr-x 2 userone userone 50 Feb 27 19:31 Folder01
drwxr-xr-x 2 userone userone 50 Feb 27 19:32 Folder02
drwxr-xr-x 2 userone userone 50 Feb 27 19:32 Folder03
drwxr-xr-x 2 userone userone 50 Feb 27 19:33 Folder04
drwxr-xr-x 2 userone userone 50 Feb 27 19:34 Folder05
Each folder has 2 files, except Folder03, which only has one file.
userone#remoteserver ~/files/_TestDir $ ls Folder01 -all
drwxr-xr-x 2 userone userone 50 Mar 1 20:03 .
drwxr-xr-x 7 userone userone 105 Feb 27 19:30 ..
-rw-r--r-- 1 userone userone 0 Feb 27 19:31 File1.rar
-rw-r--r-- 1 userone userone 0 Feb 27 19:30 FileA.mkv
userone#remoteserver ~/files/_TestDir $ ls Folder02 -all
drwxr-xr-x 2 userone userone 50 Mar 1 20:04 .
drwxr-xr-x 7 userone userone 105 Feb 27 19:30 ..
-rw-r--r-- 1 userone userone 0 Feb 27 19:32 File2.rar
-rw-r--r-- 1 userone userone 0 Feb 27 19:31 FileB.mkv
userone#remoteserver ~/files/_TestDir $ ls Folder03 -all
drwxr-xr-x 2 userone userone 30 Mar 1 20:04 .
drwxr-xr-x 7 userone userone 105 Feb 27 19:30 ..
-rw-r--r-- 1 userone userone 0 Feb 27 19:32 FileC.mkv
userone#remoteserver ~/files/_TestDir $ ls Folder04 -all
drwxr-xr-x 2 userone userone 50 Mar 1 20:04 .
drwxr-xr-x 7 userone userone 105 Feb 27 19:30 ..
-rw-r--r-- 1 userone userone 0 Feb 27 19:33 File4.rar
-rw-r--r-- 1 userone userone 0 Feb 27 19:33 FileD.mkv
userone#remoteserver ~/files/_TestDir $ ls Folder05 -all
drwxr-xr-x 2 userone userone 50 Mar 1 20:05 .
drwxr-xr-x 7 userone userone 105 Feb 27 19:30 ..
-rw-r--r-- 1 userone userone 0 Feb 27 19:34 File5.rar
-rw-r--r-- 1 userone userone 0 Feb 27 19:33 FileE.mkv
When I run the command in the original post, I get this:
userone#remoteserver ~/files/_TestDir $ find -type d -exec sh -c '[ -f "$0"/*.rar ] && [ -f "$0"/*.mkv ]' '{}' \; -print | sort
./Folder01
./Folder02
./Folder04
./Folder05
That is what I expect to see as the results, since folders 1, 2, 4 & 5 have a file of each of the extensions that I am looking for (*.rar & *.mkv), where as folder 3 only has one *.mkv file.
I have not tried to add any delete function since I have no clue where to start.
What I would like to happen is to be able to remove/delete the following
files:
FileA.mkv from Folder01
FileB.mkv from Folder02
FileD.mkv from Folder04
FileE.mkv from Folder05
Nothing gets deleted from Folder03 since it does not have a .RAR AND a .MKV file, it only has the .MKV. Hope this helps clarify.
Thank you for your assistance.
Regards.
You want to use xargs to run rm which will delete the files.
And define the replacement string, using the -I option of xargs.
Using your directory structure, you can do:
find -type d -exec sh -c '[ -f "$0"/*.rar ] && [ -f "$0"/*.mkv ]' '{}' \; -print | sort | xargs -I % sh -c "rm -f %/File?.mkv"
I took your exact find command, and added this:
| xargs -I % sh -c "rm -f %/File?.mkv"
Explanation
when find runs, it will output
./Folder01
./Folder02
./Folder04
./Folder05
Since xargs is used with the -I % option, it will run the command, replacing % with each directory (like in a loop, one by one). You could use another character than %, but avoid wildcard characters.
The command that xargs will run is sh -c "rm -f %/File?.mkv"
It will therefore do the following commands, in succession:
sh -c "rm -f Folder01/File?.mkv"
sh -c "rm -f Folder02/File?.mkv"
sh -c "rm -f Folder04/File?.mkv"
sh -c "rm -f Folder05/File?.mkv"
Obviously, you can adjust as required.
Related
Is there any way to take away the owner's permission to read a file in macOS? I know there's no reason to do this but I have to for school and I can't find an answer anywhere. Removing my write permission works fine but when I try to remove my read permission it automatically give me my read and write permissions back. As you can see in the console when I use chmod -v -v (extra verbose) it shows the correct permissions it should be changed to but then when checking afterwards they havent changed into that...
thijs#Thijss-MacBook-Air-2 week6 % ls -l
total 16
-rw----r-- 1 thijs staff 12 Oct 11 21:10 greeting.txt
-rw-r--r-- 1 thijs staff 0 Oct 11 21:10 hello.txt
-rw------- 1 thijs staff 15 Oct 11 21:11 weather.txt
thijs#Thijss-MacBook-Air-2 week6 % chmod -v -v u-w weather.txt
weather.txt: 0100600 [-rw------- ] -> 0100400 [-r-------- ]
thijs#Thijss-MacBook-Air-2 week6 % ls -l
total 16
-rw----r-- 1 thijs staff 12 Oct 11 21:10 greeting.txt
-rw-r--r-- 1 thijs staff 0 Oct 11 21:10 hello.txt
-r-------- 1 thijs staff 15 Oct 11 21:11 weather.txt
thijs#Thijss-MacBook-Air-2 week6 % chmod -v -v u-r weather.txt
weather.txt: 0100400 [-r-------- ] -> 0100000 [---------- ]
thijs#Thijss-MacBook-Air-2 week6 % ls -l
total 16
-rw----r-- 1 thijs staff 12 Oct 11 21:10 greeting.txt
-rw-r--r-- 1 thijs staff 0 Oct 11 21:10 hello.txt
-rw------- 1 thijs staff 15 Oct 11 21:11 weather.txt
I am trying to grep some pattern in a file set under a folder like below
Where on the output I have to perform remaining operation.
The output main.log is coming so huge almost 50k lines ,as the files starting with server02.log are almost 30 to 40 in number . The script based on this output is taking forever to complete.
Is there a way that I can only take files name starting with server02.log. and generated between time
20:00:00 and 21:00:00
ls -lrth server02.log.*
-rw-r--r-- 1 user user 1.9M Apr 15 20:20 server02.log.2020
-rw-r--r-- 1 user user 1.7M Apr 15 20:30 server02.log.2030
-rw-r--r-- 1 user user 1.6M Apr 15 20:41 server02.log.2041
-rw-r--r-- 1 user user 1.9M Apr 15 20:50 server02.log.2050
-rw-r--r-- 1 user user 2.1M Apr 15 21:00 server02.log.2100
-rw-r--r-- 1 user user 1.4M Apr 15 21:10 server02.log.2110
-rw-r--r-- 1 user user 1.9M Apr 15 21:20 server02.log.2120
-rw-r--r-- 1 user user 656K Apr 15 21:29 server02.log.2129
-rw-r--r-- 1 user user 4.6M Apr 15 21:40 server02.log.2140
-rw-r--r-- 1 user user 1.9M Apr 15 21:50 server02.log.2150
-rw-r--r-- 1 user user 1.7M Apr 15 21:59 server02.log.2159
-rw-r--r-- 1 user user 724K Apr 15 22:09 server02.log.2209
-rw-r--r-- 1 user user 1.3M Apr 15 22:20 server02.log.2220
-rw-r--r-- 1 user user 1.1M Apr 15 22:29 server02.log.2229
-rw-r--r-- 1 user user 1.7M Apr 15 22:41 server02.log.2241
-rw-r--r-- 1 user user 1.5M Apr 15 22:49 server02.log.2249
-rw-r--r-- 1 user user 2.4M Apr 15 23:01 server02.log.2301
-rw-r--r-- 1 user user 1.4M Apr 15 23:10 server02.log.2310
-rw-r--r-- 1 user user 585K Apr 15 23:19 server02.log.2319
-rw-r--r-- 1 user user 858K Apr 15 23:30 server02.log.2330
-rw-r--r-- 1 user user 892K Apr 15 23:40 server02.log.2340
-rw-r--r-- 1 user user 698K Apr 15 23:49 server02.log.2349
grep -E "###Update |###Initiate |###Re-Initiate " server02.log.* >> main.log
from the comments I made the change to my code as below
#!/bin/bash
DIR="."
d=$(date +%Y-%m-%d);
log_dir="logs/$d"
PREFIX="$log_dir/srv_02.log"
#PREFIX="srv_02.log"
echo "prefix value is $PREFIX"
START_HOUR="06"
for F in "$( find "$DIR" -name "${PREFIX}*" -printf '%Tc %p\n' | grep "\ ${START_HOUR}:" )"; do
echo "F value is $F"
grep -E "###Update |###Initiate |###Re-Initiate" "$F" >> main.log
done
error:
prefix value is logs/2021-04-16/srv_02.log
find: warning: Unix filenames usually don't contain slashes (though pathnames do). That means that '-name `logs/2021-04-16/srv_02.log*'' will probably evaluate to false all the time on this system. You might find the '-wholename' test more useful, or perhaps '-samefile'. Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `logs/2021-04-16/osbpd_srv_02.log*''.
F value is
grep: : No such file or directory
This solution looks for files in the given directory, created during the specified hour with names matching the given prefix.
#!/bin/bash
d=$(date +%Y-%m-%d)
DIR="logs/$d/$log_dir"
PREFIX="srv_02.log"
#PREFIX=server02.log
echo "prefix value is $PREFIX"
START_HOUR="06"
for F in "$( find "$DIR" -name "${PREFIX}*" -printf '%TY-%Tm-%Td\n' | grep "\ ${START_HOUR}:" )"; do
echo "$F"
# grep -E "###Update |###Initiate |###Re-Initiate Assignment Milestone|###Complete Assignment Milestone|###Cancel Assignment Milestone|###Suspend Assignment Milestone|###Resume Assignment Milestone" "$F" >> main.log
done
When I execute the ls -l -h command, I get an output as show by the image below.
How can the number of the items in a folder be included in the output?
Update
The current output looks like this
total 41M
-rw-r--r-- 1 root root 41M Dec 20 09:56 completed_projects.bson
-rw-r--r-- 1 root root 213 Dec 20 09:57 completed_projects.metadata.json
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:22 contents
-rw-rw-r-- 1 adipster adipster 13 Jun 16 13:20 file.py
drwxrwxr-x 4 adipster adipster 4.0K Jun 16 13:22 folder
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:21 items
But I'll like to have another column indicating the number of items in a folder like this
total 41M
-rw-r--r-- 1 root root 41M Dec 20 09:56 completed_projects.bson
-rw-r--r-- 1 root root 213 Dec 20 09:57 completed_projects.metadata.json
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:22 contents 235
-rw-rw-r-- 1 adipster adipster 13 Jun 16 13:20 file.py
drwxrwxr-x 4 adipster adipster 4.0K Jun 16 13:22 folder 19
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:21 items 5
where the numbers at the extreme right represents the number of items in a folder
You can do something like this:
echo -n "Number of files in folder is: " && ls | wc -l && ls -l
ouptut should be something like this:
umber of files in folder is: 3
Total 279K
-rwxr-xr-x 1 user users 19K Jun 16 00:17 a
-rwxr-xr-x 1 user users 5K Jun 16 00:17 b
-rwxr-xr-x 1 user users 255K Jun 16 00:17 c
You can omit echo statement, just as a note -n is no new line flag.
sed has an option to execute the constructed replacement with /e.
We only count subdirs, looking at the first character
ls -l | sed -r 's/d(.*) ([^ ]*)/printf "d%s %-20s%s\n" "\1" \2 $(ls \2| wc -l)/e'
EDIT: Solution for directories with spaces in their name.
Parsing ls should be avoided. When you try to fix above cmmand for directory names with spaces, you might try
# Don't do this
ls -l | sed -r 's/d(.{,48}) (.*)/printf "d%s %-20s%s\n" "\1" "\2" $(ls "\2"| wc -l)/e'
It is time to write a script. Perhaps with find or something like
#/bin/bash
for i in *; do
printf "%-70s %s\n" "$(/bin/ls -ld "$i")" "$(/bin/ls -d "$i"/* 2>/dev/null| wc -l)"
done
The wc in the subdir will count wrong when filenames have newlines.
ls() { command ls "$#" | tee >(echo "$(wc -l) items"); }
That uses an output process substitution to run the little "echo" script on its stdin while also displaying stdin (thanks to tee). This way, you don't have to run ls twice.
Usual caveat: output will be incorrect when there's a file with a newline in the name.
I have a directory like this:
-rw-r--r-- 1 root root 0 Jan 7 15:04 tmp_FILE2015_123_1_3123.LOG
-rw-r--r-- 1 root root 0 Jan 7 15:04 tmp_FILE2015_133_1_3123.LOG
-rw-r--r-- 1 root root 0 Jan 7 16:04 tmp_FILE2015_133_1_3125.LOG
-rw-r--r-- 1 root root 0 Jan 7 16:04 tmp_FILE2015_133_1__3223125.LOG
-rw-r--r-- 1 root root 0 Jan 7 16:04 tmp_FILE2015_133_1_3223125.LOG
I need to remove tmp_ and I can do like this:
for i in *; do s=$(sed -r 's/^(tmp_)(.*.LOG)/\2/' <<< $i); if [[ "$i" != "$s" ]]; then mv "$i" "$s"; fi; done;
But I need to do this for just older than 1 hour (modified time) files:
For example ( now: Jan 7 16:10 ):
-rw-r--r-- 1 root root 0 Jan 7 13:00 FILE2015_123_1_3123.LOG
-rw-r--r-- 1 root root 0 Jan 7 15:04 FILE2015_133_1_3123.LOG
-rw-r--r-- 1 root root 0 Jan 7 15:01 FILE2015_133_1_3125.LOG
-rw-r--r-- 1 root root 0 Jan 7 16:04 tmp_FILE2015_133_1__3223125.LOG
-rw-r--r-- 1 root root 0 Jan 7 16:10 tmp_FILE2015_133_1_3223125.LOG
How can I do that?
This will operate on all files modified in the last hour:
for orig_file in $(find . -type f -depth 1 -mtime -60m); do
new_file="${orig_file#./tmp_}"
if [[ "$new_file" != "$orig_file" ]]; then
mv "$orig_file" "$new_file";
fi;
done
If you want to operate on files older than one hour use +60m instead
I've changed the use of sed to use some built in bash functionality.
Using only grep and sed, is there a way I can tranform the output of ls -l * into this :
-rw-r--r-- agenda.txt
-rw-r--r-- annuaire.txt
Thanks!
seeing that you have already got your "answer", here's one of the simpler solution
ls -l | tr -s " "| cut -d" " -f1,8-
#OP, sed is "powerful", but sometimes, simplicity is more powerful.
Side note: Don't parse file names like that.
ls -l | sed 's/[ ]+//g' | sed 's/ [0-9].*:.[0-9]/ /g'
ls -altrh| sed -E 's/ +.+ / / g'
Or you can go with ssed which supports Perl Regular Expressions.
I solved your problem using the ssed program you can install it in any Posix system, ssed stands for super sed.
so i did a ls -latrh in my home directory.
telsa:~ mahmoh$ ls -altrh
total 136
drwxr-xr-x 5 root admin 170B Jun 24 00:27 ../
drwx------+ 4 mahmoh staff 136B Jun 24 00:27 Pictures/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Music/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Movies/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Desktop/
-rw------- 1 mahmoh staff 3B Jun 24 00:27 .CFUserTextEncoding
drwxr-xr-x+ 5 mahmoh staff 170B Jun 24 00:27 Public/
drwx------+ 5 mahmoh staff 170B Jun 24 02:19 Documents/
-rw-r--r--# 1 mahmoh staff 15K Jun 24 02:19 .DS_Store
drwx------# 36 mahmoh staff 1.2K Jun 24 14:48 Library/
-rw-r--r-- 1 mahmoh staff 279B Jun 24 15:27 .profile~
-rw-r--r--# 1 mahmoh staff 14K Jun 24 15:29 .vimrc
-rw-r--r-- 1 mahmoh staff 279B Jun 24 15:30 .profile
drwx------ 2 mahmoh staff 68B Jun 24 15:46 .Trash/
drwxr-xr-x 3 mahmoh staff 102B Jun 24 20:26 .mplayer/
-rw------- 1 mahmoh staff 3.5K Jun 24 22:11 .bash_history
-rw------- 1 mahmoh staff 42B Jun 24 23:25 .lesshst
-rw-r--r-- 1 mahmoh staff 3.6K Jun 24 23:39 temp
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:43 rtorrent.rc~
drwxr-xr-x 5 mahmoh staff 170B Jun 24 23:52 torrents/
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:56 .rtorrent.rc~
-rw------- 1 mahmoh staff 3.7K Jun 24 23:56 .viminfo
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:56 .rtorrent.rc
drwxr-xr-x+ 25 mahmoh staff 850B Jun 24 23:56 ./
drwx------+ 10 mahmoh staff 340B Jun 24 23:58 Downloads/
Now watch.
telsa:~ mahmoh$ ls -altrh| ssed -R -e 's/ +.+ / / g'
total 136
drwxr-xr-x ../
drwx------+ Pictures/
drwx------+ Music/
drwx------+ Movies/
drwx------+ Desktop/
-rw------- .CFUserTextEncoding
drwxr-xr-x+ Public/
drwx------+ Documents/
-rw-r--r--# .DS_Store
drwx------# Library/
-rw-r--r-- .profile~
-rw-r--r--# .vimrc
-rw-r--r-- .profile
drwx------ .Trash/
drwxr-xr-x .mplayer/
-rw------- .bash_history
-rw------- .lesshst
-rw-r--r-- temp
-rw-r--r-- rtorrent.rc~
drwxr-xr-x torrents/
-rw-r--r-- .rtorrent.rc~
-rw------- .viminfo
-rw-r--r-- .rtorrent.rc
drwxr-xr-x+ ./
drwx------+ Downloads/
ls -l | sed 's/^\([^\t ]\+\)\(.*:.[^ \t]\+\)\(.\+\)/\1 \3/'
Here is a working command. The slightly tricky thing is that ls -l will print the year for files that are older than some time (6 months) and hh:mm for newer files.
ls -l | sed 's/ .*[0-9]* .*[A-Z][a-z][a-z] [ 0-9][0-9] \{1,2\}[0-9][0-9]:*[0-9][0-9] / /'
For the following example
drwxr-xr-x 39 root root 1024 Feb 19 08:58 /
the starting .* will match 39 root root 1024 and then the rest of the regular expression matches month name (so you might restrict a-z to fewer characters) followed by year or hh:mm.
why not use awk instead of sed? awk is built for stuff like this.
see this manual page for more about fields in awk.
Like this?
ls -l | sed 's/ [0-9].*:.[0-9] / /' | less
Transforms
-rw-r--r-- 1 tomislav tomislav 609 2009-11-26 10:32 Test.class
-rw-r--r-- 1 tomislav tomislav 46 2009-12-14 12:16 test.groovy
into
-rw-r--r-- Test.class
-rw-r--r-- test.groovy