So i have a log file that display err|error messages that i wanna count every day
cat user.log | grep 'err|error' | wc -l
gimme almos all that i need whoever there are other log files that are zip so
zcat user.log.* | grep 'err|error' | wc -l
also almost there
so here is where im stuck, i need to check every log.zip file check if has any error message of today date in multiple files and also on user.log that is my current file collecting the errors
so i need to check over 50 user.log.Z files count all line that has today date
Oct 8 00:00:00 until 23:59:59
and my user.log
gratz in advance
EDIT-------------
solverd with
DATE=`date "+%b %e"` ;find /var/adm/ ! -path /var/adm/ -prune -name "user.log*" -prune -mtime -1 -exec zgrep "$DATE" {} \; |grep "user:err|error" |wc -l
If the date is part of the filename, append it (whereever it's added by logrotate):
zcat -f user.log.* user.log-$(date +%Y-%m-%d).* | grep 'err|error' | wc -l
zcat -f user.log user.log.$(date +%Y-%m-%d).* | grep 'err|error' | wc -l
grep can also count, zgrep can also process compressed files, so a bit shorter:
zgrep -c 'err|error' user.log user.log.$(date +%Y-%m-%d)*
If the date is not part of the filename, you need to process all files and filter out all lines with a different date:
zgrep $(date +%Y-%m-%d) file ... | grep -c 'err|error'
zgrep $(date "+%b %d") file ... | grep -c 'err|error'
Or using awk:
# Assumes the date is in the first field, like "2018-10-09 ... err|error ..."
zcat ... | awk 'BEGIN{d=strftime("%Y-%m-%d")}/err\|error/&&$01==d{c++}END{print c}'
Related
macOS Majove
I can find the oldest Created file in the directory:
cd /path/ && ls | xargs mdls -name kMDItemContentCreationDate | awk '{print $3, $4}' | sort -n | head -n1
Outputs:
2020-02-04 08:24:46
But I struggle to get its path displayed alongside. I would want something like:
2020-02-04 08:24:46 /Path/Filename
I can easily do it with last access time but not when it comes to creation time. Any help would be much appreciated. Pretty sure it is something simple, and I am just overthinking it :(
Use a while loop instead of xargs. Then you can print the filename along with the metadata.
cd /path/ && ls | while read f; do
printf '%s %s %s\n' $(mdls -name kMDItemContentCreationDate "$f" | awk '{print $3, $4}') "$f"
done | sort -n | head -n1
Hi, i'm looking for a way to output a filesize in bytes. Whatever i try i will get either 96 or 96k instead of 96000.
if [[ -d $1 ]]; then
largestN=$(find $1 -depth -type f | tr '\n' '\0' | du -s --files0-from=- | sort | tail -n 1 | awk '{print $2}')
largestS=$(find $1 -depth -type f | tr '\n' '\0' | du -h --files0-from=- | sort | tail -n 1 | awk '{print $1}')
echo "The largest file is $largestN which is $largestS bytes."
else
echo "$1 is not a directory..."
fi
This prints "The largest file [file] is 96k bytes"
there is -b option for this
$ du -b ...
Looks like you're trying to find the largest file in a given directory. It's more efficient (and shorter) to let find do the heavy lifting for you:
find $1 -type f -printf '%s %p\n' | sort -n | tail -n1
Here, %s expands to the size in bytes of the file, and %p expands to the name of the file.
Given a text file containing some list of files, e.g.
$ cat file_list.txt
/var/x/file1.txt
/var/y/file2.txt
<etc>
How can I sort this list of files by some criteria - like their last accessed time, or last changed time?
Thanks in advance.
You can use stat command with sort like this:
while read -r line; do
stat -c '%Y %n' "$line"
done < file_list.txt | sort -n | cut -d ' ' -f2
stat -c '%Y %n' lists time of last modification, seconds since Epoch followed by a space and file name
sort -n sorts timestamps and their filename numerically
cut -d ' ' -f2 prints only file names from sort's output
Try one liner (by modification time):
ls -t $(cat file_list.txt)
OR
ls -t `cat file_list.txt`
You can get the most recently changed file with
cat file_list.txt | xargs stat -c '%Y %n' | sort | tail -1 | cut -c 12-
You can get the most recent timestamp with
cat file_list.txt | xargs stat -c '%Y %n' | sort | tail -1 | cut -c -10
I have used this code
#!/bin/bash
ls -l
echo -n "Number of simple files : "
ls -l | egrep '^-' | wc -l
echo -n "Number of directories : "
ls -l | egrep '^d' | wc -l
echo -n "Number of hidden files : "
ls -la | egrep '^.*\.$' | wc -l
echo -n "Number of hidden directories : "
ls -la | egrep '^d.*\.$' | wc -l
echo " End"
While I can understand how the first two egrep work I can't figure out how the last
two work. More specific what does this mean '^.*\.$' ?
I want a file that starts with . (hidden file) and then how I should shape my regular expression?
You shouldn't be using grep (or ls) for this task at all. See http://mywiki.wooledge.org/ParsingLs for an in-depth discussion of how ls should only ever be used for interactive display to humans.
all_files=( * ) # includes directories
directories=( */ ) # directories only
hidden_files=( .* ) # includes directories
hidden_directories=( .*/ ) # directories only
echo "Number of files: $(( ${#all_files[#]} - ${#all_directories[#]} ))"
echo "Number of directories: ${#directories[#]}"
echo "Number of hidden files: $(( ${#hidden_files[#]} - ${#hidden_directories[#]} ))"
echo "Number of hidden directories: $(( ${#hidden_directories[#]} - 2 ))"
The - 2 in the last calculation is to remove . and .., which will always be present.
Note that your approach of parsing ls output for this purpose is wrong. See #Dharles Duffy's answer for a better alternative. To answer your question though and explain the regex a little bit:
'^.*\.$' means
^ // From the beginning of the string
.* // match zero or more of any character
\. // match one literal full-stop
$ // end of the string
I am not sure what you mean with a "secret" file but if you mean a hidden file, i.e. one that begins with . and then a filename, the way to regex that would be
'^\..*$'
Note that this is not when parsing ls output and is just for a file or directory name and does not discern between the two.
last two work incorrect, but
ls -la | egrep '^.*\.$' | wc -l
ls -la | egrep '^d.*\.$' | wc -l
return 2
ls -la | egrep '^.*\.$'
ls -la | egrep '^d.*\.$'
return
drwxr-xr-x 7 root root 4096 date time .
drwxr-xr-x 31 root root 4096 date time ..
variant:
secret files:
ls -la | grep '^-' |awk '{print $9}' |egrep '^\.[^\.]' |wc -l
secret dirs:
ls -la | grep '^d' |awk '{print $9}' |egrep '^\.[^\.]' |wc -l
The regex doesn't work, because '^.*\.$' matches the dot at the end of the line. Use these commands instead to count hidden files and directories:
ls -ld .* | egrep '^-' | wc -l
ls -ld .* | egrep '^d' | wc -l
Note that egrep '^d' matches . and .. too, so you need subtract 2 from the result:
ls -ld .* | egrep '^d' | wc -l | awk '{print $1 - 2}'
Alternatives:
ls -ld .* | egrep '^d' | tail -n +3 | wc -l
echo $(($(ls -ld .* | egrep '^d' | wc -l) - 2))
find $THE_DIRECTORY -maxdepth 1 -type f -name '.*' | wc --lines
Should work, you may want to use -L if you want to find symlinks as well.
I've build this little script for finding files which was modified in a specified month.
My script worked fine for this last four years, but from a recent update, now with bash V4, this won't work anymore and I don't understand why.
There is the script (fmonth.sh):
#!/bin/sh
month=$1
shift
printf "now\n%s-01\n%s-01 00:00 +1 month\n" $month $month |
date -f - +%s |
xargs printf "n=%s/60-.5;s=%s/60;e=%s/60;n-s;if (n-e <0) 0 else n-e\n" |
bc -l |
xargs printf "find $# -mmin -%.0f -mmin +%.0f -print0\n" |
sh |
xargs -0 /bin/ls -ltrd |
less -S
And this could by used as following:
fmonth.sh 2012-09 /etc /home/user /root /var/lib
Ok, I found the answer: $# could be replaced by $*:
#!/bin/sh
month=$1
shift
printf "now\n%s-01\n%s-01 00:00 +1 month\n" $month $month |
date -f - +%s |
xargs printf "n=%s/60-.5;s=%s/60;e=%s/60;n-s;if (n-e <0) 0 else n-e\n" |
bc -l |
xargs printf "find $* -mmin -%.0f -mmin +%.0f -print0\n" |
sh |
xargs -0 /bin/ls -ltrd |
less -S
This work fine now!
I'm not sure to really understand why... for now... I would search for the reason later...