I am a bit new to the bash scripting and have a scenario on which i need to know the efficient way to complete.
I need to check the size of all the files inside a directory and if the size of any files exceeds certain limit i need to empty those files and then send email to a mail ID with details like name of the files which are emptied, the original size, the current size after emptying.
Found lots of blog on the same topic but just wanted to check the most efficient way i can complete the above scenario.
Thanks
Was able complete the above scenario with the help of find command.
rm -f logFile*
find . -type f -size +20k -exec du -sh {} \; >> logFile.txt
cat logFile*
find . -type f -size +20k -exec truncate -s 0 {} \;
Related
I would like to write a Bash script that selects files within a given folder with file size between 10kB and 100kB. This list of files should be written to a new file. Something like:
fileSelector ~/my-folder-containing-files ~/my-report-file
Can you help me develop such a script using bash?
If you just want to get a list of files within a specific size range, you can try:
$ find . -type f -size +10k -size -101k -exec ls {} \;
notice that since the size is calculated to be rounded up to the next unit, you would actually get a size range between 10241 to 102400 bytes from the above expression
if you want to make the limits to be byte-precise, you can use:
$ find . -type f -size +10239c -size -102401c -exec ls {} \;
I have created a vi file and I want to check the files in my home directory to see their size. If the size of the regular file is greater than 1kb I want to back it up as a compressed file with .bak extension. I have started with the command du -h --max-depth=0 * | sort -r which list the files like...
10K archive.tar
1.0K activity48
1.0K activity47
1.0K activity46
1.0K activity45
1.0k activity44
1.0K activity43
1.0K activity42
1.0K activity41
1.0K activity40
1.0K activity39
1.0K activity38
These are some of the files listed but my thought is I need to cut field 1 and somehow create an if statement and compare the field something like if [ $x -ge 1.0 ] ; do something. Any thoughts on how I should go about the problem...?
I'd use find:
find . -maxdepth 1 -type f -size +1k -exec gzip -k -S .bak '{}' \;
I'd probably not use a custom extension for the compressed file, though; that's just asking for future confusion.
find searches a directory (. in this case) for files that pass a filter. Complex filters can be constructed; in this relatively simple case, several primitive filters are chained to select
Files that are no more than one level deep into . (i.e., subdirectories are not searched),
are regular files,
1KB or larger, and
for which gzip -k S .bak filename exits with a status code of 0.
The -exec filter is special in that it is considered an action (other actions include -delete and -print). If a filter does not contain an action, an implicit -print action is appended to the filter so that the names of all files that fit the filter are printed. Since our filter contains an action, that does not happen.
In any case, we're not really interested in the result of the -exec filter in this case, just in its side effect of running the specified command. It is useful to know that -exec is also a filter, however, in case you want to chain commands. For example, if you wanted to copy the backup files to another folder after packing them, you could write
find . -maxdepth 1 -type f -size +1k -exec gzip -k -S .bak '{}' \; -exec cp '{}.bak' /some/where/else/ \;
Then cp filename.bak /some/where/else/ would be executed only if gzip -k -S .bak filename returned with an exit status of 0 (that is, if it indicated successful completion).
find . -maxdepth 1 -type f -size +1k -exec gzip {} \;
That ought to do it. Well it produces compressed .gz files.
I'm trying to look for a file from a database. I'm getting the data from a php file just fine. It's just this one line I'm having issues with:
directory=`find ./ -type f -name "*$thismodelnormal*" -exec ls -la {} \;`
$thismodelnormal is just a string, but it's dynamic based on data from the database. Can anyone enlighten me on how to get this done? I've done a good bit of research already and couldn't find a solution.. surely somebody has done this before though.
Adding set -x at the top of my script allowed me to view the command that are actually being run. In this case my command needed to be
directory=`find ./ -type f -name "*"$thismodelnormal"*" -exec ls -la {} \;`
NOTE the two sets of double quotes. One is for the find itself and the other extracts the variable.
This question already has answers here:
Argument list too long error for rm, cp, mv commands
(31 answers)
Closed 3 years ago.
If I run the command mv folder2/*.* folder, I get "argument list too long" error.
I find some example of ls and rm, dealing with this error, using find folder2 -name "*.*". But I have trouble applying them to mv.
find folder2 -name '*.*' -exec mv {} folder \;
-exec runs any command, {} inserts the filename found, \; marks the end of the exec command.
The other find answers work, but are horribly slow for a large number of files, since they execute one command for each file. A much more efficient approach is either to use + at the end of find, or use xargs:
# Using find ... -exec +
find folder2 -name '*.*' -exec mv --target-directory=folder '{}' +
# Using xargs
find folder2 -name '*.*' | xargs mv --target-directory=folder
find folder2 -name '*.*' -exec mv \{\} /dest/directory/ \;
First, thanks to Karl's answer. I have only minor correction to this.
My scenario:
Millions of folders inside /source/directory, containing subfolders and files inside. Goal is to copy it keeping the same directory structure.
To do that I use such command:
find /source/directory -mindepth 1 -maxdepth 1 -name '*' -exec mv {} /target/directory \;
Here:
-mindepth 1 : makes sure you don't move root folder
-maxdepth 1 : makes sure you search only for first level children. So all it's content is going to be moved too, but you don't need to search for it.
Commands suggested in answers above made result directory structure flat - and it was not what I looked for, so decided to share my approach.
This one-liner command should work for you.
Yes, it is quite slow, but works even with millions of files.
for i in /folder1/*; do mv "$i" /folder2; done
It will move all the files from folder /folder1 to /folder2.
find doesn't work with really long lists of files, it will give you the same error "Argument list too long". Using a combination of ls, grep and xargs worked for me:
$ ls|grep RadF|xargs mv -t ../fd/
It did the trick moving about 50,000 files where mv and find alone failed.
I have a directory with roughly 100000 files in it, and I want to perform some function on all files beginning with a specified string, which may match tens of thousands of files.
I have tried
ls mystring*
but this returns with the bash error 'Too many arguments'. My next plan was to use
find ./mystring* -type f
but this has the same issue.
The code needs to look something like
for FILE in `find ./mystring* -type f`
do
#Some function on the file
done
Use find with a wildcard:
find . -name 'mystring*'
ls | grep "^abc"
will give you all files beginning (which is what the OP specifically required) with the substringabc.
It operates only on the current directory whereas find operates recursively into sub folders.
To use find for only files starting with your string try
find . -name 'abc'*
If you want to restrict your search only to files you should consider to use -type f in your search
try to use also -iname for case-insensitive search
Example:
find /path -iname 'yourstring*' -type f
You could also perform some operations on results without pipe sign or xargs
Example:
Search for files and show their size in MB
find /path -iname 'yourstring*' -type f -exec du -sm {} \;