I'd like to fetch result, let's say from 2017-12-19 19:14 till the entire day from a log file that looks like this -
/var/opt/MarkLogic/Logs/ErrorLog_1.txt:2017-12-19 19:14:00.723 Info: Saving /var/opt/MarkLogic/Forests/Meters/00001829
/var/opt/MarkLogic/Logs/ErrorLog_1.txt:2017-12-19 19:14:01.134 Info: Saved 9 MB at 22 MB/sec to /var/opt/MarkLogic/Forests/Meters/00001829
/var/opt/MarkLogic/Logs/ErrorLog_1.txt:2017-12-19 19:14:01.376 Info: Merging 19 MB from /var/opt/MarkLogic/Forests/Meters/0000182a and /var/opt/MarkLogic/Forests/Meters/00001829 to /var/opt/MarkLogic/Forests/Meters/0000182c, timestamp=15137318408510140
/var/opt/MarkLogic/Logs/ErrorLog_1.txt:2017-12-19 19:14:02.585 Info: Merged 18 MB in 1 sec at 15 MB/sec to /var/opt/MarkLogic/Forests/Meters/0000182c
/var/opt/MarkLogic/Logs/ErrorLog_1.txt:2017-12-19 19:14:05.200 Info: Deleted 15 MB at 337 MB/sec /var/opt/MarkLogic/Forests/Meters/0000182a
/var/opt/MarkLogic/Logs/ErrorLog_1.txt:2017-12-19 19:14:05.202 Info: Deleted 9 MB at 4274 MB/sec /var/opt/MarkLogic/Forests/Meters/00001829
I am new to Unix and familiar with grep command. I tried the below command
date="2017-12-19 [19-23]:[14-59]"
echo "$date"
grep "$date" $root_path_values
but it throws invalid range end error. Any solution ? The date is going to be coming from a variable so it will be unpredictable. Therefore, don't make a command just keeping the example in mind. $root_path_values is a sequence of error files like errorLog.txt, errorLog_1.txt, errorLog_2.txt and so on.
I'd like to fetch result, let's say from 2017-12-19 19:14 till the entire day … The date is going to be coming from a variable …
This is not a job for regular expressions. Since the timestamp has a sensible form, we can simply compare it as a whole, e. g.:
start='2017-12-19 19:14'
end='2017-12-20'
awk -vstart="$start" -vend=$end 'start <= $0 && $0 < end' ErrorLog_1.txt
egrep '2017-12-19 (19|2[0-3])\:(1[4-9]|[2-5][0-9])\:*\.*' path/to/your/file Try this regexp.
In the case if you need pattern in variable:
#!/bin/bash
date="2017-12-19 (19|2[0-3])\:(1[4-9]|[2-5][0-9])\:*\.*"
egrep ${date} path/to/your/file
Related
I want a simple way to add 2 numbers taken from a text file. Details below:
Daily, I run clamscan against my /home/ folder, which generates a simple log along the lines of this:
Scanning 851M in /home/.
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.000 sec (0 m 0 s)
Start Date: 2021:11:27 06:25:02
End Date: 2021:11:27 06:25:02
Weekly, I scan both my /home/ folder and an external drive, so I get twice as much in the log:
Scanning 851M in /home/.
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.000 sec (0 m 0 s)
Start Date: 2021:11:28 06:25:02
End Date: 2021:11:28 06:25:02
Scanning 2.8T in /mnt/ext/.
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.005 sec (0 m 0 s)
Start Date: 2021:11:28 06:26:30
End Date: 2021:11:28 06:26:30
I don't email the log to myself, I just have a bash script that sends an email that (for the daily scan) reads the number that comes after "Infected files:" and says either "No infected files found" or "Infected files found, check log." (And, to be honest, once I'm 100% comfortable that it all works the way I want it to, I'll skip the "No infected files found" email.) The problem is, I don't know how to make that work for the weekly scan of multiple folders, because the summary I get doesn't combine those numbers.
I'd like the script to find both lines that start "Infected files:", get the numbers that follow, and add them. I guess the ideal solution use a loop in case I ever need to scan more than two folders. I've taken a couple of stabs at it with grep and cut, but I'm just not experienced enough a coder to make it all work.
Thanks!
This bash script will print out the sum of infected files:
#!/bin/bash
n=$(sed -n 's/^Infected files://p' logfile)
echo $((${n//$'\n'/+}))
or a one-liner:
echo $(( $(sed -n 's/^Infected files: \(.*\)/+\1/p' logfile) ))
The task is to write a bash scenario to delete in a directory those files whose last access time exceeds N days.
If there are no such files, the message should be displayed.
I'm trying to delete 1 file and find the last access time for it:
stat -c%x test.txt
The command returns a date: 2018-12-01 16:45:30.390000000 +0000
Then I'm writing a current date command:
date '+%Y-%m-%d %H:%M:%S.%N'
The command returns a date: 2018-12-01 18:39:16.873718766
Now I have no idea how to subtract this dates and apply to all files of the directory.
Use the command find with a parameter -atime.
find /path/to/dir -atime +100 -delete
deletes files accessed 101 or more days ago.
I'm checking an alternative to 'find' command in shell scripting so as to eliminate the discrepancy of Accessed date of sub directories.
According to my observation, when find command is executed to list all the files in a directory, the accessed date of sub-directories is getting changed.
I want to post genuine statistics in one of the junk platforms, So I have been looking at some forums and got the alternative with 'ls' command. But that doesn't completely fulfill my request.
Below is the answer given by #ghostdog74.
ls -R %path% | awk '/:$/&&f{s=$0;f=0} /:$/&&!f{sub(/:$/,"");s=$0;f=1;next} NF&&f{ print s"/"$0 }'.
But this finds only the files inside the sub directories. I need all the files and sub-directories' files to be listed.
For example:
bash-3.2# pwd
/Users/manojkapalavai/Desktop/SleepTimeReport
bash-3.2# ls
**6th floor** manoj17 manoj26.txt manoj36 manoj45.txt manoj55 manoj70.txt manoj80 manoj9.txt **test1**
manoj14 manoj23.txt manoj33 manoj42.txt manoj52 manoj61.txt manoj71 manoj80.txt manoj90 **test2**.
The highlighted ones are sub-directories inside "SleepTimeReport" directory and remaining are just files. So, when I execute the above command, I get only the below output.
bash-3.2# ls -R ~/Desktop/SleepTimeReport | awk '/:$/&&f{s=$0;f=0} /:$/&&!f{sub(/:$/,"");s=$0;f=1;next} NF&&f{ print s"/"$0 }'.
~/Desktop/SleepTimeReport/6th floor/Script to increase the Sleep Time.numbers.
~/Desktop/SleepTimeReport/6th floor/Zone1Sleep.pages.
~/Desktop/SleepTimeReport/test1/New_folder.
~/Desktop/SleepTimeReport/test1/manoj.txt.
~/Desktop/SleepTimeReport/test1/sathish.txt.
~/Desktop/SleepTimeReport/test1/vara.txt.
~/Desktop/SleepTimeReport/test1/New_folder/Script to increase the Sleep Time.numbers.
~/Desktop/SleepTimeReport/test1/New_folder/Zone1Sleep.pages.
i.e.; only those files inside sub-directories are listed.
Brief explanation of what issue I'm facing, please see below
Manojs-MacBook-Pro:SleepTimeReport manojkapalavai$ ls -l
total 16
drwxr-xr-x 8 manojkapalavai staff 272 Sep 14 15:07 6th floor
-rwxr-xr-x 1 manojkapalavai staff 59 Nov 13 10:41 AltrFind.sh
-rw-r--r-- 1 manojkapalavai staff 0 Nov 2 15:15 manoj%.txt
-rw-r--r-- 1 manojkapalavai staff 0 Nov 2 18:23 manoj1
When I try finding Created time and Accessed Time of the folder 6th floor before using 'find' command, the below is output.
Manojs-MacBook-Pro:SleepTimeReport manojkapalavai$ stat -f '%N, %SB, %Sa' 6th\ floor/
6th floor/, Sep 13 10:34:55 2017, **Nov 13 11:21:33 2017**
Manojs-MacBook-Pro:SleepTimeReport manojkapalavai$ find /Users/manojkapalavai/Desktop/SleepTimeReport/
/Users/manojkapalavai/Desktop/SleepTimeReport/
/Users/manojkapalavai/Desktop/SleepTimeReport//6th floor
/Users/manojkapalavai/Desktop/SleepTimeReport//6th floor/.DS_Store
/Users/manojkapalavai/Desktop/SleepTimeReport//6th floor/Script to increase the Sleep Time.numbers
/Users/manojkapalavai/Desktop/SleepTimeReport//6th floor/Zone1Sleep.pages
Now, after finding all the files inside a directory, below is the output of atime. you can notice the change
Manojs-MacBook-Pro:SleepTimeReport manojkapalavai$ stat -f '%N, %SB, %Sa' 6th\ floor/
6th floor/, Sep 13 10:34:55 2017, **Nov 13 14:26:03 2017**
All tha I have done is just find the files, and atime of sub-folders inside a folder when we find is getting changed to that current time.
Is there any way to solve this?
ls is the wrong tool for programmatic use. Generally, you should be able to fix your find usage to not have an effect on atimes (actually, it's pretty rare for folks to even have atimes enabled at the filesystem level on modern production systems), but if you really want to avoid it, consider the bash globstar option:
shopt -s globstar
for file in **/*; do
echo "Doing whatever with $file"
done
I need to run curl using following URL and compare current system time with the time showing in the output. If current time is grater than 5 minutes than the output time then I need an email alert.
# curl -g http://new.abc.com/arcfilesync/cachejob?jobType=CACHEBUILD
null('{"status":"RUNNING","lastRunTime":"Mon Jun 13 2016 08:58:13 AM","lastRunStatus":"SUCCESS"}')
Can anyone guide me to write this?
Try this script:
#!/bin/bash
time1="Mon Jun 13 2016 08:58:13 AM";
timestamp1=$(date --date="$time1" +%s)
# create current date
time2=`date +%Y-%m-%d\ %H:%M:%S`
# Compute the timestamp
timestamp2=`date --date="$time2" +%s`
# get the difference between dates in seconds
let "diff=$timestamp2-$timestamp1"
# Compute the diff in minutes
let "diffInMin=$diff/60"
echo "diff $time1 <--> $time2"
echo "$diffInMin"
which will output the diff in minutes:
./getTimeDiff.sh
diff Mon Jun 13 2016 08:58:13 AM <--> 2016-06-13 11:13:36
135
If will create the timestamp from the time in your string and compare that to the current time. Now you only have to parse your curl request and get the time from there. I would use awk
I have a large number of downloaded radio programs that consist of 4 mp3 files each. The files are named like so:
Show Name - Nov 28 2011 - Hour 1.mp3
Show Name - Nov 28 2011 - Hour 2.mp3
Show Name - Nov 28 2011 - Hour 3.mp3
Show Name - Nov 28 2011 - Hour 4.mp3
Show Name - Nov 29 2011 - Hour 1.mp3
Show Name - Nov 29 2011 - Hour 2.mp3
Show Name - Nov 29 2011 - Hour 3.mp3
Show Name - Nov 29 2011 - Hour 4.mp3
Show Name - Nov 30 2011 - Hour 1.mp3 and so on...
I have used the cat command to join the files with great success by moving the four files of the same date into a folder and using the wildcard:
cat *.mp3 > example.mp3
The files are all the same bitrate, sampling rate, etc.
What I would like to do is run a script that will look at the file name and combine hours 1-4 of each date and name the file accordingly. Just the show name, the date and drop the 'Hour 1'.
I looked around and found a number of scripts that can be used to move files around based on their names but I'm not adept enough at bash scripting to be able to understand the methods used and adapt them to my needs.
I'm using Ubuntu 14.04.
Many thanks in advance
You can use a bash for loop to find each distinct date name and then construct the expected mp3 names from that.
Because your files have spaces in their names and my solution uses globbing, you'll also have to edit your Internal Field Separator to ignore spaces for the duration of the script.
SAVEIFS=$IFS
IFS=$'\n\b'
for mdy in `/bin/ls *mp3 | cut -d' ' -f'4,5,6' | sort -u`; do
cat *${mdy}*.mp3 > "showName_${mdy}_full.mp3"
done
IFS=$SAVEIFS
This won't alert you if some hours are missing for some particular date. It'll just join together whatever's there for that date.
Note: The comment pointing out that cat probably won't work for these files is spot on. The resulting file will probably be corrupted. You probably want to use something like mencoder or ffmpeg instead. (Check out this thread.)