I'm using command find to recursively browse through directory tree, counting files, sizes, etc...
Now I need to get directory depth of each file.
Is there any portable way for both FreeBSD and CentOS?
I know that find is able to prinf actual directory depth but sadly this works only on CentOS, not FreeBSD.
Additionaly - I need to keep standard find output OR put directory depth on the beginning of output and cut it from there.
You can count the / in path :
$ find . -type f -exec bash -c 'echo '{}' | grep -o / | wc -l' \;
Or with file names :
$ mkdir -p one/two/three four/five && touch file one/two/file one/two/three/file
$ find . -type f -exec bash -c 'echo -n '{}' :; echo '{}' | grep -o / | wc -l' \;
./file :1
./one/two/file :3
./one/two/three/file :4
Try this:
find . -type d -exec bash -c 'echo $(tr -cd / <<< "$1"|wc -c):$1' -- {} \; | sort -n | tail -n 1 | awk -F: '{print $1, $2}'
Related
Can anyone guide, how to implement this linux command into a bash script
df --local -P | awk {'if (NR!=1) print $6'} | xargs -I '{}' find '{}' -xdev -type d -perm -0002 2>/dev/null | xargs chmod a+t
Simply
Put this in a file, add a shell shebang on 1st line:
#!/bin/sh
df --local -P |
awk '{if (NR!=1) print $6}' |
xargs -I '{}' find '{}' -xdev -type d -perm -0002 2>/dev/null |
xargs --no-run-if-empty chmod a+t
But as this question is tagged bash:
Under bash, I would write this something like:
#!/bin/bash
{ read foo; mapfile -t mpoints;} < <(df --local -P)
TEXTDOMAIN=libc
exec 2> >(exec grep -v '^find: .*'$"Permission denied"\$ >&2)
find "${mpoints[#]#*% }" -xdev -type d -perm -0002 -exec chmod a+t {} +
Short, quick and efficient!
Explanation:
read foo just whipe header line of df output
mapfile -t take rest of df output into $mpoint array, (-t remove a trailing newline).
exec ... {} ... + will do approx same job than using xargs.
exec 2>... >&2 will silently delete lines matching regex in STDERR.
TEXTDOMAIN=libc and $"Permission denied" will localize message to make this script work in many languages.
I have managed to do this separately using
grep -r "zone 19" path
mkdir zone19
find . -name "ListOfFilesfromGrep" -exec mv -i {} zone19 \;
I just don't know how to combine the two, that is, how to input the list of files I get from grep into the find command.
You should use grep from within find:
find /path/to/dir -type f -exec grep -q "zone 19" {} \; -exec mv -i {} zone19 \;
You could try
grep -lr "zone 19" path | while read in ; do mv -i "$in" zone19; done
-l prints the filenames with matched string; while ... done move the files one by one.
Using GNU versions of the standard tools:
grep -l will give you the filenames.
mv -t will move to a given directory.
xargs -r will invoke a command using arguments from stdin, but only if there's at least one.
Combine them like this:
grep -l -r -e 'zone 19' path | xargs -r mv -i -t 'zone19'
Or (if your filenames might contain newlines etc):
grep -lZr -e 'zone 19' path | xargs -0r mv -it 'zone19'
You can pipe the result from grep and use xargs:
grep -lr "zone 19" path | xargs <command>
<command> will be applied on each result of grep. Note thta -o flag tells grep to show only matching parts.
Below is the command to move all files containing string "Hello" to folder zone19.
grep Hello * |cut -f1 -d":"|sort -u|xargs -I {} mv {} zone19
I'm looking for a way to traverse directories recursively to find a specific file, then stop the search and pipe the filename path to an awk function or something similar. I asked a question earlier that was similar, but after testing on machines other than mine it turns out the locate command isn't going to work since not everyone uses it on their system.
Code that I used with locate:
dir="/path/to/destination/";
mkdir "$dir";
locate -l 1 target_file.txt | \
awk -v dir="$dir" '{printf "cp \"%s\" \"%s\"\n", $1, dir}' | \
sh
The find(1) command will do it. To only get one line, use head(1).
dir="/path/to/destination/";
mkdir "$dir";
find /path/location -name target_file.txt |
head -n 1 |
awk -v dir="$dir" '{printf "cp \"%s\" \"%s\"\n", $1, dir}' |
sh
If you know only one file exists then you can use
find ./ -name "target_file.txt" -exec cp -r {} $dir \;
And if you are not sure, use head to limit 1 and use xargs
find ./ -name "target_file.txt" | head -1 | xargs -I {} cp -r {} $dir/
I need to count the number of files on a large number of directories. Is there an easy way to do this with a shell script (using find, wc, sed, awk or similar)? Just to avoid having to write a proper script in python.
The output would be something like this:
$ <magic_command>
dir1 2
dir2 12
dir3 5
The number after the dir name would be the number of files. A plus would be able to turn counting of dot/hidden files on and off.
Thanks!
Try the below one:
du -a | cut -d/ -f2 | sort | uniq -c | sort -nr
from http://www.linuxquestions.org/questions/linux-newbie-8/how-to-find-the-total-number-of-files-in-a-folder-510009/#post3466477
find <dir> -type f | wc -l
find -type f will list all files in the specified directory one at each line, wc -l count the amount of newlines seen from stdin.
Also for future reference: answers like this are a google away.
More or less what I was looking for:
find . -type d -exec sh -c 'echo "{}" `ls "{}" |wc -l`' \;
try ls | wc it list the file in your directory and gives list of file output to wc as input
One way like this:
$ for dir in $(find . -type d )
> do
> echo $dir $(ls -A $dir | wc -l )
> done
Just remove the -A option if you do not want the hidden file count
find . -type d | xargs ls -1 | perl -lne 'if(/^\./ || eof){print $a." ".$count;$a=$_;$count=-1}else{$count++}'
below is the test:
> find . -type d
.
./SunWS_cache
./wicked
./wicked/segvhandler
./test
./test/test2
./test/tempdir.
./signal_handlers
./signal_handlers/part2
> find . -type d | xargs ls -1 | perl -lne 'if(/^\./ || eof){print $a." ".$count;$a=$_;$count=-1}else{$count++}'
.: 79
./SunWS_cache: 4
./signal_handlers: 6
./signal_handlers/part2: 5
./test: 6
./test/tempdir.: 0
./test/test2: 0
./wicked: 4
./wicked/segvhandler: 9
A generic version of Mehdi Karamosly's solution to list folders of any directory without changing current directory
DIR=~/test/ sh -c 'cd $DIR; du -a | cut -d/ -f2 | sort | uniq -c | sort -nr'
Explanation:
Extract directory into variable
Start new shell
Change directory in that shell so that current shell's directory stays same
Process
I use these functions:
nf()(for d;do echo $(ls -A -- "$d"|wc -l) "$d";done)
nfr()(for d;do echo $(find "$d" -mindepth 1|wc -l) "$d";done)
Both assume that filenames don't contain newlines.
Here's bash-only versions:
nf()(shopt -s nullglob dotglob;for d;do a=("$d"/*);echo "${#a[#]} $d";done)
nfr()(shopt -s nullglob dotglob globstar;for d;do a=("$d"/**);echo "${#a[#]} $d";done)
I liked the output from the du based answer, but when I was looking at a large filesystem it was taking ages, so I put together a small ls based script which gives the same output, but much quicker:
for dir in `ls -1A ~/test/`;
do
echo "$dir `ls -R1Ap ~/test/$dir | grep -Ev "[/:]|^\s*$" | wc -l`"
done
You can try out copying the output of ls command in a text file and then count the number of lines in that file.
ls $LOCATION > outText.txt; NUM_FILES=$(wc -w outText.txt); echo $NUM_FILES
find -type f -printf '%h\n' | sort | uniq -c | sort -n
I want to find a bash command that will let me grep every file in a directory and write the output of that grep to a separate file. My guess would have been to do something like this
ls -1 | xargs -I{} "grep ABC '{}' > '{}'.out"
but, as far as I know, xargs doesn't like the double-quotes. If I remove the double-quotes, however, then the command redirects the output of the entire command to a single file called '{}'.out instead of to a series of individual files.
Does anyone know of a way to do this using xargs? I just used this grep scenario as an example to illustrate my problem with xargs so any solutions that don't use xargs aren't as applicable for me.
Do not make the mistake of doing this:
sh -c "grep ABC {} > {}.out"
This will break under a lot of conditions, including funky filenames and is impossible to quote right. Your {} must always be a single completely separate argument to the command to avoid code injection bugs. What you need to do, is this:
xargs -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}
Applies to xargs as well as find.
By the way, never use xargs without the -0 option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).
Also don't parse ls. Ever. Use globbing or find instead: http://mywiki.wooledge.org/ParsingLs
Use find for everything that needs recursion and a simple loop with a glob for everything else:
find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;
or non-recursive:
for file in *; do grep "$file" > "$file.out"; done
Notice the proper use of quotes.
A solution without xargs is the following:
find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;
...and the same can be done with xargs, it turns out:
ls -1 | xargs -I {} sh -c "grep ABC '{}' > '{}.out'"
Edit: single quotes added after remark by lhunath.
I assume your example is just an example and that you may need > for other things. GNU Parallel http://www.gnu.org/software/parallel/ may be your rescue. It does not need additional quoting as long as your filenames do not contain \n:
ls | parallel "grep ABC {} > {}.out"
If you have filenames with \n in it:
find . -print0 | parallel -0 "grep ABC {} > {}.out"
As an added bonus you get the jobs run in parallel.
Watch the intro videos to learn more: http://pi.dk/1
The 10 seconds installation will try to do a full installation; if that fails, a personal installation; if that fails, a minimal installation:
$ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
fetch -o - http://pi.dk/3 ) > install.sh
$ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
12345678 883c667e 01eed62f 975ad28b 6d50e22a
$ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
cc21b4c9 43fd03e9 3ae1ae49 e28573c0
$ sha512sum install.sh | grep da012ec113b49a54e705f86d51e784ebced224fdf
79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
$ bash install.sh
If you need to move it to a server, that does not have GNU Parallel installed, try parallel --embed.
Actually, most of the answers here do not work with all filenames (if they contain double and single quotes), including the answer by lhunath and Stephan202.
This solution works with filenames with single and double quotes:
find . -mindepth 1 -print0 | xargs -0 -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}
Here's a test with filename with both single and double quotes:
echo ABC > "I'm here.txt"
# lhunath solution (hangs waiting for input)
$ find . -exec sh -c 'grep "$1" > "$1.out"' -- {} \;
# Stephan202 solutions
$ find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;
grep: ./Im: No such file or directory
grep: here.txt > ./Im here.txt.out: No such file or directory
$ ls -1 | xargs -I {} sh -c "grep ABC '{}' > '{}.out'"
xargs: unterminated quote
# this solution
$ find . -mindepth 1 -print0 | xargs -0 -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}
$ ls -1
"I'm here.txt"
"I'm here.txt.out"