Unix scripting for finding files & deleting them based on a given size - shell

I'm working on a unix script that has 2 input parameters - path and size.
The script will check all the files in the given path with the given size and deletes them. If the delete operation fails, the respective file-name is recorded into a file. For any other case, the file is rendered without any action.
I have written a short code (don't know whether it works).
find $path -type f -size +${byte_size}c -print | xargs -I {}
if $?=1;
then
rm -rf {};
else
echo {} >> Error_log_list.txt'
where
$path is the path where we search for the files.
size is the input size.
Error_log_list.txt is the file where we send the non-deletable filenames.
Can anyone please help me verify whether it is correct?

GNU find has a -delete option for this exact use case. More information (and a number of different approaches) in the find documentation.
find $path -type f -size +${byte_size}c -delete

Executing your script results in the following syntax error:
./test.sh: line 9: unexpected EOF while looking for matching `''
./test.sh: line 11: syntax error: unexpected end of file
Moreover the condition of the if statement seems not correct.
If I am not wrong it tests the return code of the "rm" command before to
execute the command.
I am not familiar with xargs and I tried to rewrite your script
using a while loop construct. Here my script
#!/bin/bash
path=$1
byte_size=$2
find $path -type f -size +${byte_size}c -print | while read file_name
do
rm -f $file_name
if [ ! $? -eq 0 ]; then
echo $file_name >> Error_log_list.txt
fi
done
I tested it trying to delete files without the right permission and it works.

I wrote a script, please check this functionality
a=`find . -type f -size +{$size}c -print`
#check if $a is empty
if [ -z "$a" ]
then
echo $a > error_log.txt
#if a is not empty then remove them
else
rm $a
fi
Let me explain what we are doing here.
First assigning the file_names in current directory (which satisfy
size requirement) to a variable 'a'
Checking if that variable is
empty (empty means there is no file with your size requirement) if a
has some values then delete them

Related

Find command in for loop

I have bash script that intents to find all files older then "X" minutes and to redirect the output into a file. The logic is a have a for loop and i want to do a find through all files, but for some reason it prints and redirect in the output file just the file from the last directory(TESTS[3]="/tmp/test/"). So i want all the files from the directories to be redirected there. Thank you for the help :D
Here is the sh:
#!/bin/bash
set -x
if [ ! -d $TEST ]
then
echo "The directory does not exist (${TEST})!"
echo "Aborted."
exit 1
fi
TESTS[0]="/tmp/t1/"
TESTS[1]="/tmp/t2/"
TESTS[2]="/tmp/t3/"
TESTS[3]="/tmp/test/"
for TEST in "${TESTS[#]}"
do
find $TEST -type f -mmin +1 -exec ls -ltrah {} \; > /root/alex/out
done
You are using > inside the loop to redirect the output of the latest command to the file each time, overwriting the previous contents of the file. If you used >> it would open the file in "append" mode each time instead, but...
A better way to fix your issue would be by moving the redirection to outside the loop:
done > /root/alex/out
And an even better way than that would be to avoid a loop entirely and just use:
find "${TESTS[#]}" -type f -mmin +1 -exec ls -ltrah {} \; > /root/alex/out
Since find accepts multiple paths.
I think you can use {} + instead of {} \; to call the minimum number of ls required to process all arguments, and you might want to check -printf in man find because you can probably get a similar output using built-in format specifiers without calling ls at all.

Rename all files of a certain name within all second-level sub-directories

My goal is to automate the following process: search within all second-level sub-directories, and for all files called "Test.pptx" in said sub-directories, rename to "Test - Appended.pptx". Based on responses I have seen to other questions on StackOverflow I have attempted the following code:
for D in *; do
if [ -d "${D}" ]; then
echo "${D}"
for E in "${D}"; do
if [ -d "${E} ]; then
echo "${E}"
for f in "Test.pptx"; do mv "$f" "Test - Appended.pptx"; done
fi
done
fi
done
I set the script executable (using chmod +x) and run it, but get the following errors:
line 7: unexpected EOF while looking for matching `"'
line 12: syntax error: unexpected end of file
I am a relative newcomer to Bash scripts, so I would appreciate any help in diagnosing the errors and achieving the initial goal. Thank you!
use find:
while read -r pptx
do
mv -n "${pptx}" "${pptx%.pptx} - Appended.pptx"
done < <( find . -mindepth 3 -maxdepth 3 -type f -name "*.pptx" )
Be aware, that I did not test it and it might need adaptions on your special case.
As long as the -n option is set in mv it will not overwrite anything.
sub-loops not really needed.
for f in */*/Test.pptx; do mv "$f" "${f%/*}/Test - Appended.pptx"; done
${f%/*} is the current file's full path with everything from the last slash (/) forward stripped off, so if the file is a/b/Test.pptx then ${f%/*} is a/b.

Bash script gives 'Permission denied' when trying piping path to file into sed

I'm trying to create a bash script that finds all the files in my dotfiles directory, and symlinks them into ~. If the directory a file is in does not exist, it shall be created.
The script as it is now is just trying to find files and create the two paths, when that works "real" and "symlink" will be used with ln -s. However when trying to save the strings in "real" and "symlink" all i get is line 12: ./.zshrc: Permission denied
What am I doing wrong?
#!/bin/bash
dotfiles=()
readarray -d '' dotfiles < <(find . -type f ! -path '*/.git/*' ! -name "README.md" -type f -print0)
for file in ${dotfiles[#]}; do
dir=$(dirname $file | sed s#.#$HOME#1)
[ ! -d $dir ] && echo "directory $dir not exist!" && mkdir -p $dir
# Create path strings
real=$("$file" | sed s#.#$PWD#1)
symlink=$("$file" | sed s#.#$HOME#1)
echo "Real path: $cur"
echo "Symbolic link path: $new"
done
exit
P.S, I'm a bash noob and am mostly doing this script as a learning experience.
Here is a refactoring which attempts to fix the obvious problems. See comments with # ... for details. Probably also check with http://shellcheck.net/ before you ask for human assistance.
The immediate reason for the error message is that $("$file" ...) does indeed attempt to run $file as a command and capture its output. You probably meant $(echo "$file" ...) but that can often be avoided.
#!/bin/bash
dotfiles=()
readarray -d '' dotfiles < <(find . -type f ! -path '*/.git/*' ! -name "README.md" -type f -print0)
# ... Fix broken quoting throughout
for file in "${dotfiles[#]}"; do
dir=$(dirname "$file" | sed "s#^\.#$HOME#") # ... 1 is not required or useful
# ... Use standard error for diagnostics
[ ! -d "$dir" ] && echo "$0: directory $dir does not exist!" >&2 && mkdir -p "$dir"
# Create path strings
# ... Use parameter substitution
real=$PWD/${file#./}
symlink=$HOME/${$file#./}
# ... You mean $real, not $cur?
echo "Real path: $real"
# ... You mean $symlink, not $new?
echo "Symbolic link path: $symlink"
done
# ... No need to explicitly exit; trust me, you will anyway
#exit
These are just syntactic fixes; I would probably avoid storing the results from find in an array and just loop over them directly, and I haven't checked if the logic actually does what (we might guess) you are perhaps trying to do.
See also Looping over pairs of values in bash for a similar topic, and When to wrap quotes around a shell variable?.
The script still has a pesky assumption that it will be run in the dotfiles directory. Probably a better design would be to explicitly run find in that directory, and refactor accordingly.
sed 's/x/y/' will replace the first occurrence of x with y on each line by definition and by design; there is no reason or need to explicitly add 1 after the final delimiter. (Some sed variants allow a number there to select a different match than the first, but this is not portable; and of course specifying the first match when that's already the default is simply silly. There is a g flag to replace all matches instead of just the first which many beginners want to use everywhere, but of course that's not necessary or useful here either.)

How to loop through all the files located under a certain path in zsh?

Here's what I have so far:
for file in $(find /path/to/directory -type f); echo $file; done
but I get this error:
zsh: parse error near `done'
There is no need to use find. You could try the following:
for file in /path/to/directory/**/*(.); do echo $file; done
or
for file in /path/to/directory/**/*(.); echo $file
the ** pattern matches multiple directories recursively. So a/**/b matches any b somewhere below a. It is essentially matches the list find a -name b produces.
(.) is a glob qualifier and tells zsh to only match plain files. It is the equivalent to the -type f option from find.
you do not really need double quotes around $file because zsh does not split variables into words on substitution.
the first version is the regular form of the for-loop; the second one is the short form without do and done
The reason for the error you get is due to the last point: when running a single command in the loop you need either both do and done or none of them. If you want to run more than one command in the loop, you must use them.
Can you check if adding "" around $file solves this problem like so:
for file in $(find /path/to/directory -type f); echo "$file"; done
Edit:
add do before echo and let me know if it solves the problem:
for file in $(find /path/to/directory -type f); do echo "$file"; done

List directories not containing certain files?

I used this command to find all the directories containing .mp3 in the current directory, and filtered out only the directory names:
find . -iname "*.mp3" | sed -e 's!/[^/]*$!!' -e 's!^\./!!' | sort -u
I now want the opposite, but I found it a little harder. I can't just add a '!' to the find command since it'll only exclude .mp3 when printing them not find directories that do not contain .mp3.
I googled this and searched on stackoverflow and unix.stackexchange.com.
I have tried this script so far and it returns this error:
#!/bin/bash
find . -type d | while read dir
do
if [[! -f $dir/*.mp3 ]]
then
echo $dir
fi
done
/home/user/bin/try.sh: line 5: [[!: command not found
#!/bin/bash
find . -type d | while read dir
do
if [! -f $dir/*.mp3 ]
then
echo $dir
fi
done
/home/user/bin/try.sh: line 5: [!: command not found
#!/bin/bash
find . -type d | while read dir
do
if [[! -f "$dir/*.mp3" ]]
then
echo $dir
fi
done
/home/user/bin/try.sh: line 5: [!: command not found
I'm thinking it has to do with multiple arguments for the test command.
Since I'm testing all the directories the variable is going to change, and I use a wildcard for the filenames.
Any help is much appreciated. Thank You.
[ "$(echo $dir/*.mp3)" = "$dir/*.mp3" ]
should work.
Or simply add a space between '[' and '!'
A method that is probably significantly faster is
if find "$dir" -name '*.mp3' -quit ; then
: # there are mp3-files in there.
else
; # no mp3:s
fi
Okay, I solved my own answer by using a counter.
I don't know how efficient it is, but it works. I know it can be made better. Please feel free to critique.
find . -type d | while read dir
do
count=`ls -1 "$dir"/*.mp3 2>/dev/null | wc -l`
if [ $count = 0 ]
then
echo $dir
fi
done
This prints all directories not containing MP3s It also shows sub-directories thanks to the find command printing directories recursively.
I ran a script to automatically download cover art for my mp3 collection. It put a file called "cover.jpg" in the directory for each album for which it could retrieve the art. I needed to check for which albums the script had failed - i.e. which CDs (directories) did not contain a file called cover.jpg. This was my effort:
find . -maxdepth 1 -mindepth 1 -type d | while read dir; do [[ ! -f $dir/cover.jpg ]] && echo "$dir has no cover art"; done
The maxdepth 1 stops the find command from descending into a hidden directory which my WD My Cloud NAS server had created for each album and placed a default generic disc image. (This got cleared during the next scan.)
Edit: cd to the MP3 directory and run it from there, or change the . in the command above to the path to point to it.

Resources