I'm making a bash script and it's like this:
#!/bin/bash
DNUM=$(ls -lAR / 2> /dev/null | grep '^d' | wc -l)
echo there are $DNUM directories.
the problem is, that when I run this line directly on the terminal:
ls -lAR / 2> /dev/null | grep '^d' | wc -l
I get a number.
But when I run the script it displays me a greater number, like 30 to 50 more.
What is the problem here?
Why is the "wc" command counting more lines when running it from a script?
You may have different directory roots for the two runs. Instead of ls to find the directories only you can use this
find parent_directory -type d
and pipe to wc -l to count.
The /proc directory will have processes and treated as directories and will change from run to run. To exclude it from the count use
find / -path /proc -prune -o -type d | wc -l
To find the differences in your exact case I would suggest to run
#!/bin/bash
for r in 1 2; do
ls -lAR / 2> /dev/null | grep '^d' > run${r}.txt 1> out${r}.txt
done
diff -Nura out1.txt out2.txt
rm -f out1.txt out2.txt
But as the most ppl. already said it would make sense to exclude directories like sys,proc ...
Related
I want to make a bash script that counts how many files there are in specific folders.
Example:
#!/usr/bin/env bash
sum=0
find /home/user/Downloads | wc -l
find /home/user/Documents | wc -l
find /home/user/videos | wc -l
echo "files are $sum "
Downloads folder has 5 files, Documents has 10 files and videos has 10 files.
I want to add all files from above directories and print the number of files.
echo "files are $sum "
Please I would like to use "only" find command, because my script delete some files. My goal is how many files I deleted.
Anything that's piping to wc -l isn't counting how many files you have, it's counting how many newlines are present, and so it'll fail if any of your file names contain newlines (a very real possibility, especially given the specific directories you're searching). You could do this instead using GNU tools:
find \
/home/user/Downloads \
/home/user/Documents \
/home/user/videos \
-print0 |
awk -v RS='\0' 'END{print NR}'
I'm not sure about this, but you can try this code.
#!/usr/bin/env bash
sum=0
downloads=$(find /home/user/Downloads | wc -l)
((sum += downloads))
documents=$(find /home/user/Documents | wc -l)
((sum += documents))
videos=$(find /home/user/videos | wc -l)
((sum += videos))
echo "files are $sum "
You can save the individual results and sum them as as per FS-GSW's answer.
That's what I would do in Java or C, but shell scripts have their own patterns. Whenever you find yourself with lots of variables or explicit looping, take a step back. That imperative style is not super idiomatic in shell scripts.
Can you pass multiple file names to the same command?
Can that loop become a pipe?
In this case, I would pass all three directories to a single find command. Then you only need to call wc -l once on the combined listing:
find /home/user/Downloads /home/user/Documents /home/user/videos | wc -l
Or using tilde and brace expansion:
find ~user/{Downloads,Documents,videos} | wc -l
And if the list of files is getting long you could store it in an array:
files=(
/home/user/Downloads
/home/user/Documents
/home/user/videos
)
find "${files[#]}" | wc -l
Just use ..
find /home/user/ -type f | wc -l
.. to count the files inside the user directory recursively.
With tree you could also find/count (or whatever else you want to do) hidden files.
e.g. tree -a /home/user/ - for which the output would be: XXX directories, XXX files - which then wouldn't apply to your question though.
This question already has an answer here:
Getting all files which have changed in specific time range
(1 answer)
Closed 3 years ago.
I'm trying to build a one-line command in bash which counts the number of files of type *.lu in the current directory that were modified between 15:17 and 15:47 (date does not matter here). I'm not allowed to use find (otherwise it would been easy). I'm allowed to use basic commands like ls, grep, cut, wc and so on.
What I tried to do:
ls -la *.lu | grep <MISSING> | wc -l
First of all, I'll find all files *.lu, than I need to check with grep the date (which I'm not sure how to do) and than we need to count the number of lines. I think we need to insert also cut to get to the date and check it, but how? Also if current directory does not have *.lu files it will fail rather than returning 0.
How to solve it?
ls -l *.lu | grep -E '15:[2-3][0-9]|15:1[7-9]|15:4[0-7]' | wc -l
Should do it.
With awk:
ls -al *.lu | awk 'BEGIN{count=0} {if((substr($8,0,2) == "15") && (int(substr($8,4)) >=17 && int(substr($8,4)) <= 47 )){count++}} END{print count}'
UPDATE:
Without -E
ls -l *.lu | grep '15:[2-3][0-9]\|15:1[7-9]\|15:4[0-7]' | wc -l
Redirect error in case of zeros files:
ls -l *.lu 2> /dev/null | grep '15:[2-3][0-9]\|15:1[7-9]\|15:4[0-7]' | wc -l
This is pretty ugly and can probably be done better. However, I wanted to challenge myself to do this without regexes (excepting the sed one). I don't guarantee it'll handle all of your use cases (directories might be an issue).
ls -l --time-style="+%H%M" *.lu 2>/dev/null |
sed '/total/d' |
tr -s ' ' |
cut -d ' ' -f6 |
xargs -r -n 1 -I ARG bash -c '(( 10#ARG >= 1517 && 10#ARG <= 1547)) && echo ARG' |
wc -l
There is probably a way to avoid parsing ls via stat --format=%Y.
I'm using command find to recursively browse through directory tree, counting files, sizes, etc...
Now I need to get directory depth of each file.
Is there any portable way for both FreeBSD and CentOS?
I know that find is able to prinf actual directory depth but sadly this works only on CentOS, not FreeBSD.
Additionaly - I need to keep standard find output OR put directory depth on the beginning of output and cut it from there.
You can count the / in path :
$ find . -type f -exec bash -c 'echo '{}' | grep -o / | wc -l' \;
Or with file names :
$ mkdir -p one/two/three four/five && touch file one/two/file one/two/three/file
$ find . -type f -exec bash -c 'echo -n '{}' :; echo '{}' | grep -o / | wc -l' \;
./file :1
./one/two/file :3
./one/two/three/file :4
Try this:
find . -type d -exec bash -c 'echo $(tr -cd / <<< "$1"|wc -c):$1' -- {} \; | sort -n | tail -n 1 | awk -F: '{print $1, $2}'
I need to count the number of files on a large number of directories. Is there an easy way to do this with a shell script (using find, wc, sed, awk or similar)? Just to avoid having to write a proper script in python.
The output would be something like this:
$ <magic_command>
dir1 2
dir2 12
dir3 5
The number after the dir name would be the number of files. A plus would be able to turn counting of dot/hidden files on and off.
Thanks!
Try the below one:
du -a | cut -d/ -f2 | sort | uniq -c | sort -nr
from http://www.linuxquestions.org/questions/linux-newbie-8/how-to-find-the-total-number-of-files-in-a-folder-510009/#post3466477
find <dir> -type f | wc -l
find -type f will list all files in the specified directory one at each line, wc -l count the amount of newlines seen from stdin.
Also for future reference: answers like this are a google away.
More or less what I was looking for:
find . -type d -exec sh -c 'echo "{}" `ls "{}" |wc -l`' \;
try ls | wc it list the file in your directory and gives list of file output to wc as input
One way like this:
$ for dir in $(find . -type d )
> do
> echo $dir $(ls -A $dir | wc -l )
> done
Just remove the -A option if you do not want the hidden file count
find . -type d | xargs ls -1 | perl -lne 'if(/^\./ || eof){print $a." ".$count;$a=$_;$count=-1}else{$count++}'
below is the test:
> find . -type d
.
./SunWS_cache
./wicked
./wicked/segvhandler
./test
./test/test2
./test/tempdir.
./signal_handlers
./signal_handlers/part2
> find . -type d | xargs ls -1 | perl -lne 'if(/^\./ || eof){print $a." ".$count;$a=$_;$count=-1}else{$count++}'
.: 79
./SunWS_cache: 4
./signal_handlers: 6
./signal_handlers/part2: 5
./test: 6
./test/tempdir.: 0
./test/test2: 0
./wicked: 4
./wicked/segvhandler: 9
A generic version of Mehdi Karamosly's solution to list folders of any directory without changing current directory
DIR=~/test/ sh -c 'cd $DIR; du -a | cut -d/ -f2 | sort | uniq -c | sort -nr'
Explanation:
Extract directory into variable
Start new shell
Change directory in that shell so that current shell's directory stays same
Process
I use these functions:
nf()(for d;do echo $(ls -A -- "$d"|wc -l) "$d";done)
nfr()(for d;do echo $(find "$d" -mindepth 1|wc -l) "$d";done)
Both assume that filenames don't contain newlines.
Here's bash-only versions:
nf()(shopt -s nullglob dotglob;for d;do a=("$d"/*);echo "${#a[#]} $d";done)
nfr()(shopt -s nullglob dotglob globstar;for d;do a=("$d"/**);echo "${#a[#]} $d";done)
I liked the output from the du based answer, but when I was looking at a large filesystem it was taking ages, so I put together a small ls based script which gives the same output, but much quicker:
for dir in `ls -1A ~/test/`;
do
echo "$dir `ls -R1Ap ~/test/$dir | grep -Ev "[/:]|^\s*$" | wc -l`"
done
You can try out copying the output of ls command in a text file and then count the number of lines in that file.
ls $LOCATION > outText.txt; NUM_FILES=$(wc -w outText.txt); echo $NUM_FILES
find -type f -printf '%h\n' | sort | uniq -c | sort -n
How do I pipe commands and their results in Ubuntu when writing them in the terminal. I would write the following commands in sequence -
$ ls | grep ab
abc.pdf
cde.pdf
$ cp abc.pdf cde.pdf files/
I would like to pipe the results of the first command into the second command, and write them all in the same line. How do I do that ?
something like
$ cp "ls | grep ab" files/
(the above is a contrived example and can be written as cp *.pdf files/)
Use the following:
cp `ls | grep ab` files/
Well, since the xargs person gave up, I'll offer my xargs solution:
ls | grep ab | xargs echo | while read f; do cp $f files/; done
Of course, this solution suffers from an obvious flaw: files with spaces in them will cause chaos.
An xargs solution without this flaw? Hmm...
ls | grep ab | xargs '-d\n' bash -c 'docp() { cp "$#" files/; }; docp "$#"'
Seems a bit klunky, but it works. Unless you have files with returns in them I mean. However, anyone who does that deserves what they get. Even that is solvable:
find . -mindepth 1 -maxdepth 1 -name '*ab*' -print0 | xargs -0 bash -c 'docp() { cp "$#" files/; }; docp "$#"'
To use xargs, you need to ensure that the filename arguments are the last arguments passed to the cp command. You can accomplish this with the -t option to cp to specify the target directory:
ls | grep ab | xargs cp -t files/
Of course, even though this is a contrived example, you should not parse the output of ls.