Would like to create a shell script that will read the contents of a folder that contains many folders on a server- it should output a list of these folders with their size and date modified if possible.
If you want to do it recursively (it's not clear to me from the question whether or not you do), you can do:
$ find /path/to/dir -type d -exec stat -c '%n: %s: %y' {} \;
(If you have a find which supports the feature, you can replace '\;' with '+')
Note that the %s gives the size of the directory, which is not the number of files in the directory, nor is it the disk usage of the files in the directory.
ls -l /path/to/folder | grep ^d
Try this find command to list sub-directories and their size (since stat command doesn't run same way on mac and Linux):
#!/bin/bash
find /your/base/dir -type d -exec du -k {} \; > sizes.txt
Related
Is it possible to write a bash command that would write to a file a (key, value) structure that would represent every file within given directory and its corresponding file permissions as octal number (i.e. 664)? I know this command returns an octal value:
stat -c '%a' /path/to/file/
but I don't know how to combine it with walking through a directory and writing it out to a file. What might be useful is also this command that creates my_md5.txt file with key, value like structure of hash codes...
find /path/to/file/ -type f -exec md5sum {} \; > /tmp/my_md5.txt
but I don't know how to combine the two bits of code to do what I want.
Any ideas?
You mean something like that?
find -type f -exec stat -c "%n: %a" {} \; | cut -b 3- > output.txt
explanation
find all files in working directory
print name and permissions
ignore first two characters from filename "./"
write to outputfile
I need a Script that writes the directory and subdirectory in a text-file.
For example the script lies in /Mainfolder and in this folder are four other folders. Each contains several files.
Now I would like the script to write the path of each file in the textfile.
Subfolder1/File1.dat
Subfolder1/File2.dat
Subfolder2/File1.dat
Subfolder3/File1.dat
Subfolder4/File1.dat
Subfolder4/File2.dat
Important is that there is no slash in front of the listing.
Use the find command:
find Mainfolder > outputfile
and if you only want the files listed, do
find Mainfolder -type f > outputfile
You can also strip the leading ./ if you search the current directory, with the %P format option:
find . -type f -printf '%P\n' > outputfile
If your bash version is high enough, you can do it like that:
#!/bin/bash
shopt -s globstar
echo ** > yourtextfile
This solution assumes that the subdirectories contain only files -- they do not contain any directory in turn.
find . -type f -print | sed 's|^.*/S|S|'
I have created a single file in each of the four subdirectories. The original output is:
./Subfolder1/File1.dat
./Subfolder4/File4.dat
./Subfolder2/File2.dat
./Subfolder3/File3.dat
The filtered output is:
Subfolder1/File1.dat
Subfolder4/File4.dat
Subfolder2/File2.dat
Subfolder3/File3.dat
You can use this find with -exec:
find . -type f -exec bash -c 'f="{}"; echo "${f:2}"' \;
This will print all files starting from current paths by removing ./ from front.
So I have some folder
|-Folder1
||-SubFolder1
||-SubFolder2
|-Folder2
||-SubFolder3
||-SubFolder4
Each subfolder contains several jpg I want to zip to the root folder...
I'm a little bit stuck on "How to enter each folder"
Here is my code:
find ./ -type f -name '*.jpg' | while IFS= read i
do
foldName=${PWD##*/}
zip ../../foldName *
done
The better would be to store FolderName+SubFolderName and give it to the zip command as name...
Zipping JPEGs (for Compression) is Usually Wasted Effort
First of all, attempting to compress already-compressed formats like JPEG files is usually a waste of time, and can sometimes result in archives that are larger than the original files. However, it is sometimes useful to do so for the convenience of having a bunch of files in a single package.
Just something to keep in mind. YMMV.
Use Find's -execdir Flag
What you need is the find utility's -execdir flag. The GNU find man page says:
-execdir command {} +
Like -exec, but the specified command is run from the subdirec‐
tory containing the matched file, which is not normally the
directory in which you started find.
For example, given the following test corpus:
cd /tmp
mkdir -p foo/bar/baz
touch foo/bar/1.jpg
touch foo/bar/baz/2.jpg
you can zip the entire set of files with find while excluding the path information with a single invocation. For example:
find /tmp/foo -name \*jpg -execdir zip /tmp/my.zip {} +
Use Zip's --junk-paths Flag
The zip utility on many systems supports a --junk-paths flag. The man page for zip says:
--junk-paths
Store just the name of a saved file (junk the path), and do not
store directory names.
So, if your find utility doesn't support -execdir, but you do have a zip that supports junking paths, you could do this instead:
find /tmp/foo -name \*jpg -print0 | xargs -0 zip --junk-paths /tmp/my.zip
You can use dirname to get the directory name of a file/directory it is located in.
You can also simplify the find command to search only for directories by using -type d. Then you should use basename to get only the name of the subdirs:
find ./*/* -type d | while read line; do
zip --junk-paths "$(basename $line)" $line/*.jpg
done
Explanation
find ./*/* -type d
will print out all directories located in ./*/* which will result in all subdirs of directories located in the current dir
while read line reads each line from the stream and stores it in the variable "line". Thus $line will be the relative path to the subdir, e.g. "Folder1/Subdir2"
"$(basename $line)" returns the only the name of the subdir, e.g. "Subdir2"
Update: add --junk-paths to the zip command if you do not want the directy paths to be stored in the zip filde
So a little check, I finally got something working:
find ./*/* -type d | while read line; do
#printf '%s\n' "$line"
zip ./"$line" "$line"/*.jpg
done
But this create un archive containing:
Subfolder.zip
Folder
|-Subfolder
||-File1.jpg
||-File2.jpg
||-File3.jpg
Instead I fold like it to be:
Subfolder.zip
|-File1.jpg
|-File2.jpg
|-File3.jpg
So I tried using basename and dirname in differnet combination...Always got some error...
And just to learn how to: what if I would like the new archive to be created in the same root directory as "Folder"?
Ok finally got it!
find ./* -name \*.zip -type f -print0 | xargs -0 rm -rf
find ./*/* -type d | while read line; do
#printf '%s\n' "$line"
zip --junk-paths ./"$line" "$line"/*.jpg
done
find . -name \*.zip -type f -mindepth 2 -exec mv -- '{}' . \;
In first row I simply remove all .zip files,
Then I zip all and in the final row I move all zip to the root directory!
Thanks everbody for your help!
I have a script using a for loop that would rename folders and files. The script would take the list of files and folders and rename them conditionally. I would invoke the file using the command:
find test/* -exec ./replace.sh {} \;
My replace.sh script would contain something similar to:
for i in $#
mv $OLDFILE $NEWFILE
done
$OLDFILE and $NEWFILE has been set previously and I don't believe any problems will arise from them.
My problem arises when I hit upon subdirectories. Originally, I would have folders like:
folder_1
-file1
-file2
When my script changes folder_1 into folderX1, the next argument, folder_1/file1 woudl be invalid as the changed path would be folderX1/file1. I figured I could create a stack with a list of folders that is being changed and pop them out later to rename the files but this seems hard on bash. Is there a better method that I am missing?
P.S I could run the program several times to go through all the subdirectories but this doesn't seem efficient.
You can add -depth to the find command. This will process the directory's files before the directory itself. See man find for details.
Your find usage is problematic. The first option is the start location for the search, so you don't want to use a glob there. If you want only the files in test/ and not any of its subdirectories, use the -depth option, as Olaf suggested.
You don't really need to use a separate script to handle this rename. It can be done within the find command line, if you don't mind a little mess.
To handle just the top-level of files, you could do this:
$ touch foo.txt bar.txt baz.ext
$ find test -depth 1 -type f -name \*.txt -exec bash -c 'f="{}"; mv -v "{}" "${f/.txt/.csv}"' \;
./foo.txt -> ./foo.csv
./bar.txt -> ./bar.csv
$
But your concern is valid -- find will build a list of matches, and if your -exec changes the list out from under find, some renames will fail.
I suspect your quickest solution is to do this in TWO stages (not several): one for files, followed by one for directories. (Or change the order, I don't think it should matter.)
$ mkdir foo_1; touch red_2 foo_1/blue_3
$ find . -type f -name \*_\* -exec bash -c 'f="{}"; mv -v "{}" "${f%_?}X${f##*_}"' \;
./foo_1/blue_3 -> ./foo_1/blueX3
./red_2 -> ./redX2
$ find . -type d -name \*_\* -exec bash -c 'f="{}"; mv -v "{}" "${f%_?}X${f##*_}"' \;
./foo_1 -> ./fooX1
Bash parameter expansion will get you a long way.
Another option, depending on your implementation of find, is the -d option:
-d Cause find to perform a depth-first traversal, i.e., directories
are visited in post-order and all entries in a directory will be
acted on before the directory itself. By default, find visits
directories in pre-order, i.e., before their contents. Note, the
default is not a breadth-first traversal.
So:
$ mkdir -p foo_1/bar_2; touch red_3 foo_1/blue_4 foo_1/bar_2/green_5
$ find . -d -name \*_\* -exec bash -c 'f="{}"; mv -v "{}" "${f%_?}X${f##*_}"' \;
./foo_1/bar_2/green_5 -> ./foo_1/bar_2/greenX5
./foo_1/bar_2 -> ./foo_1/barX2
./foo_1/blue_4 -> ./foo_1/blueX4
./foo_1 -> ./fooX1
./red_3 -> ./redX3
$
I have a folder with 200 files in it. We can say that the files are named "abc0" to "abc199". Five of these files contain the string "ez123" but I don't know which ones. My current attempt to find the file names of the files that contain the string is:
#!/bin/sh
while read FILES
do
cat $FILES | egrep "ez123"
done
I have a file that contains the filenames of all files in the directory. So I then execute:
./script < filenames
This is verifies for me that the files containing the string exist but I still don't have the name of the files. Are there any ideas concerning the best way to accomplish this?
Thanks
you can try
grep -l "ez123" abc*
find /directory -maxdepth 1 -type f -exec fgrep -l 'ez123' \{\} \;
(-maxdepth 1 is only necessary if you only want to search the directory and not the tree recursively (if there's any)).
fgrep is a bit faster than grep. -l lists the matched filenames only.
Try
find -type f -exec grep -qs "ez123" {} \; -print
This will use find to find all real files in current directory (and subdirectories), execute grep on them ({} will be replaced by file name, -qs tells it to be silent and just set an exit code), -print will print out the names of the files that grep found a matching line in.
What about:
xargs egrep -l ez123
That reads filenames from stdin and prints out the filenames with matches.