I am trying to find the number of files in a directory with two different patterns in the filenames. I don't want the combined count, but display the combined result.
Command 1: find | grep ".coded" | wc -l | Output : 4533
Command 2: find | grep ".read" | wc -l | Output: 654
Output sought: 4533 | 654 in one line
Any suggestions? Thanks!
With the bash shell using process substitution and pr
pr -mts' | ' <(find | grep "\.coded" | wc -l) <(find | grep "\.read" | wc -l)
With GNU find, you can use -printf to print whatever you want, for example a c for each file matching .coded and an "r" for each file matching .read, and then use awk to count how many of each you have:
find -type f \
\( -name '*.coded*' -printf 'c\n' \) \
-o \
\( -name '*.read*' -printf 'r\n' \) \
| awk '{ ++a[$0] } END{ printf "%d | %d\n", a["c"], a["r"] }'
By the way, your grep patterns match Xcoded and Yread, or really anything for your period; if it is a literal period, it has to be escaped, as in '\.coded' and '\.read'. Also, if your filenames contain linebreaks, your count is off.
Related
I have a large number of file names. I need to create a bash script that gets all of the unique dates from the file names.
Example:
input:
opencomposition_dxxx_20201123.csv.gz
opencomposition_dxxv_20201123.csv.gz
opencomposition_dxxu_20201123.csv.gz
opencomposition_sxxv_20201123.csv.gz
opencomposition_sxxe_20211223.csv.gz
opencomposition_sxxe_20211224.csv.gz
opencomposition_sxxe_20211227.csv.gz
opencomposition_sxxesgp_20230106.csv.gz
output:
20201123 20211224 20211227 20230106
Code:
for asof_dt in `find -H ./ -maxdepth 1 -nowarn -type f -name *open*.gz
| sort -r | cut -f3 -d "_" | cut -f1 -d"." | uniq`; do
echo $asof_dt
done
Error:
line 20: /bin/find: Argument list too long
Like this (GNU grep):
You need to add quotes on the glob: '*open*.gz', if not, the shell try to expand the wildcard *.
find -H ./ -maxdepth 1 -nowarn -type f -name '*open*.gz' |
grep -oP '_\K\d{8}(?=\.csv)' |
sort -u
Output
20201123
20211223
20211224
20211227
20230106
The regular expression matches as follows:
Node
Explanation
_
_
\K
resets the start of the match (what is Kept) as a shorter alternative to using a look-behind assertion: perlmonks look arounds and Support of K in regex
\d{8}
digits (0-9) (8 times)
(?=
look ahead to see if there is:
\.
.
csv
'csv'
)
end of look-ahead
Using tr:
find -H ./ -maxdepth 1 -nowarn -type f -name '*open*.gz' | tr -d 'a-z_.' | sort -u
If filenames don't contain newline characters, a quick-and-dirty method, similar to your attempt, might be
printf '%s\n' open*.gz | cut -d_ -f3 | cut -d. -f1 | sort -u
Note that printf is a bash builtin command and argument list too long is not applied to bash builtins.
So I do this:
find . -name '*.md' -type f -exec wc -w {} \; | awk '{ print $1 }'
And get a column of numbers (truncated):
...
2829
3619
828
1195
2406
2857
1480
1846
23
But then when I pipe all of that into a sum, I get an incorrect amount:
find . -name '*.md' -type f -exec wc -w {} \; | awk '{ print $1 }' | sum
9658 2
I thought awk would strip the white space out of wc -w output. But am I missing something?
(End result: I want to take a weekly word count and compare it previous weeks.)
The issue with your code is that sum does not count the sup of the output of the previous command.
Here is the sum help manual
Usage: sum [OPTION]... [FILE]...
Print checksum and block counts for each FILE.
Here is what you can do
find . -name '*.md' -type f -exec wc -w {} \; | awk '{s+=$1} END {printf "%.0f", s}'
Where the awk increments the s on each step with the value and prints it as an integer (to 0 decimal places) when done.
Concatenate all the files and pipe the result to wc -w, this way you don't need to sum word counts of individual files.
find . -name '*.md' -type f -exec awk 1 {} + | wc -w
awk 1 is for making sure each file's content is separated from that of the other with a newline, if that's not necessary, you can use cat instead.
Hi hoping someone can help, I have some directories on disk and I want to count the number of files in them (as well as dir size if possible) and then strip info from the output. So far I have this
find . -type d -name "*,d" -print0 | xargs -0 -I {} sh -c 'echo -e $(find "{}" | wc -l) "{}"' | sort -n
This gets me all the dir's that match my pattern as well as the number of files - great!
This gives me something like
2 ./bob/sourceimages/psd/dzv_body.psd,d
2 ./bob/sourceimages/psd/dzv_body_nrm.psd,d
2 ./bob/sourceimages/psd/dzv_body_prm.psd,d
2 ./bob/sourceimages/psd/dzv_eyeball.psd,d
2 ./bob/sourceimages/psd/t_zbody.psd,d
2 ./bob/sourceimages/psd/t_gear.psd,d
2 ./bob/sourceimages/psd/t_pupil.psd,d
2 ./bob/sourceimages/z_vehicles_diff.tga,d
2 ./bob/sourceimages/zvehiclesa_diff.tga,d
5 ./bob/sourceimages/zvehicleswheel_diff.jpg,d
From that I would like to filter based on max number of files so > 4 for example, I would like to capture filetype as a variable for each remaining result e.g ./bob/sourceimages/zvehicleswheel_diff.jpg,d
I guess I could use awk for this?
Then finally I would like like to remove all the results from disk, with find I normally just do something like -exec rm -rf {} \; but I'm not clear how it would work here
Thanks a lot
EDITED
While this is clearly not the answer, these commands get me the info I want in the form I want it. I just need a way to put it all together and not search multiple times as that's total rubbish
filetype=$(find . -type d -name "*,d" -print0 | awk 'BEGIN { FS = "." }; {
print $3 }' | cut -d',' -f1)
filesize=$(find . -type d -name "*,d" -print0 | xargs -0 -I {} sh -c 'du -h
{};' | awk '{ print $1 }')
filenumbers=$(find . -type d -name "*,d" -print0 | xargs -0 -I {} sh -c
'echo -e $(find "{}" | wc -l);')
files_count=`ls -keys | nl`
For instance:
ls | nl
nl printed numbers of lines
I'm trying to calculate the number of words written in a project. There are a few levels of folders and lots of text files within them.
Can anyone help me find out a quick way to do this?
bash or vim would be good!
Thanks
use find the scan the dir tree and wc will do the rest
$ find path -type f | xargs wc -w | tail -1
last line gives the totals.
tldr;
$ find . -type f -exec wc -w {} + | awk '/total/{print $1}' | paste -sd+ | bc
Explanation:
The find . -type f -exec wc -w {} + will run wc -w on all the files (recursively) contained by . (the current working directory). find will execute wc as few times as possible but as many times as is necessary to comply with ARG_MAX --- the system command length limit. When the quantity of files (and/or their constituent lengths) exceeds ARG_MAX, then find invokes wc -w more than once, giving multiple total lines:
$ find . -type f -exec wc -w {} + | awk '/total/{print $0}'
8264577 total
654892 total
1109527 total
149522 total
174922 total
181897 total
1229726 total
2305504 total
1196390 total
5509702 total
9886665 total
Isolate these partial sums by printing only the first whitespace-delimited field of each total line:
$ find . -type f -exec wc -w {} + | awk '/total/{print $1}'
8264577
654892
1109527
149522
174922
181897
1229726
2305504
1196390
5509702
9886665
paste the partial sums with a + delimiter to give an infix summation:
$ find . -type f -exec wc -w {} + | awk '/total/{print $1}' | paste -sd+
8264577+654892+1109527+149522+174922+181897+1229726+2305504+1196390+5509702+9886665
Evaluate the infix summation using bc, which supports both infix expressions and arbitrary precision:
$ find . -type f -exec wc -w {} + | awk '/total/{print $1}' | paste -sd+ | bc
30663324
References:
https://www.cyberciti.biz/faq/argument-list-too-long-error-solution/
https://www.in-ulm.de/~mascheck/various/argmax/
https://linux.die.net/man/1/find
https://linux.die.net/man/1/wc
https://linux.die.net/man/1/awk
https://linux.die.net/man/1/paste
https://linux.die.net/man/1/bc
You could find and print all the content and pipe to wc:
find path -type f -exec cat {} \; -exec echo \; | wc -w
Note: the -exec echo \; is needed in case a file doesn't end with a newline character, in which case the last word of one file and the first word of the next will not be separated.
Or you could find and wc and use awk to aggregate the counts:
find . -type f -exec wc -w {} \; | awk '{ sum += $1 } END { print sum }'
If there's one thing I've learned from all the bash questions on SO, it's that a filename with a space will mess you up. This script will work even if you have whitespace in the file names.
#!/usr/bin/env bash
shopt -s globstar
count=0
for f in **/*.txt
do
words=$(wc -w "$f" | awk '{print $1}')
count=$(($count + $words))
done
echo $count
Assuming you don't need to recursively count the words and that you want to include all the files in the current directory , you can use a simple approach such as:
wc -l *
10 000292_0
500 000297_0
510 total
If you want to count the words for only a specific extension in the current directory , you could try :
cat *.txt | wc -l
I'm trying to do a script that counts all the files in the system that have a name with a odd number of characters, only the name not the extension.
somebody can help me?
I've done this but it doesn't work
find /usr/lib -type f | cut -f 1 -d '.' | rev | cut -f 4 -d '/' | rev | wc -m
with this I count all the characters of all file, but how do I count the number of character of one file ?
The following awk command will print out the number of files with an odd number of characters in their name.
find /usr/lib -type f | awk -F/ '{gsub(/\.[^\.]*$/,"",$NF);if(length($NF)%2!=0)i++}END{print i}'
Print all the file names with an odd number of characters,
find /usr/lib -type f | xargs -i basename {} | cut -d . -f 1 | grep -Pv '^(..)+$'
pipe to wc to count.