I have a list of files with the substring YYYYMMDDHH in them (example: 2016112200 means 2016 November 22th at 00 hours). These files are: temp_2016102200.data, temp_2016102212.data, temp_2016102300.data, temp_2016102312.data, ..., temp_20170301.data. And I also have other family of files substituting temp by wind.
For each string YYYYMMDDHH I want to create a tar with the temp and its correspondent wind file. I don't want this process to stop if one or both files are missing.
My idea was to loop from 12 hours to 12 hours, but I am having some problems because to specify the date I did: b=$(date -d '2016111400' +'%Y%m%d%H') but bash informs me that that is not a valid date...
Thanks.
It's not bash telling you the date format is wrong: date is telling you. Not everything you type is a bash command.
As Kamil comments, you have to split it up so that date can parse it. The YYYY-mm-dd HH:MM:SS format is parsable. Using bash parameter expansion to extract the relevant substrings:
$ d=2016111400
$ date -d "${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00"
Mon Nov 14 00:00:00 EST 2016
Now, when you want to add 12 hours, you have to be careful to do it in the right place in the datetime string: if you add a + character after the time, it will be parsed as a timezone offset, so put the relative part either first or between the date and the time.
$ date -d "+12 hours ${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00"
Mon Nov 14 12:00:00 EST 2016
As a loop, you could do:
d=2016111400
for ((i=1; i<=10; i++)); do
# print this datetime
date -d "${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00"
# add 12 hours
d=$( date -d "+12 hours ${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00" "+%Y%m%d%H" )
done
outputs:
Mon Nov 14 00:00:00 EST 2016
Mon Nov 14 12:00:00 EST 2016
Tue Nov 15 00:00:00 EST 2016
Tue Nov 15 12:00:00 EST 2016
Wed Nov 16 00:00:00 EST 2016
Wed Nov 16 12:00:00 EST 2016
Thu Nov 17 00:00:00 EST 2016
Thu Nov 17 12:00:00 EST 2016
Fri Nov 18 00:00:00 EST 2016
Fri Nov 18 12:00:00 EST 2016
OK, a "nicer" way to loop
start=2019043000
end=2019050300
plus12hours() {
local d=$1
date -d "+12 hours ${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00" "+%Y%m%d%H"
}
for (( d = start; d <= end; d = $(plus12hours "$d") )); do
printf "%d\t%s\n" "$d" "$(date -d "${d:0:4}-${d:4:2}-${d:6:2} ${d:8:2}:00:00")"
done
2019043000 Tue Apr 30 00:00:00 EDT 2019
2019043012 Tue Apr 30 12:00:00 EDT 2019
2019050100 Wed May 1 00:00:00 EDT 2019
2019050112 Wed May 1 12:00:00 EDT 2019
2019050200 Thu May 2 00:00:00 EDT 2019
2019050212 Thu May 2 12:00:00 EDT 2019
2019050300 Fri May 3 00:00:00 EDT 2019
I need to parse all a file into a better format to produce an outcome with columns delimited by a comma, thinking of being able to export the content in CSV file.
This is an example of my input;
. D 0 Mon Dec 10 11:07:46 2018
.. D 0 Mon Feb 19 11:38:06 2018
RJ9-5 D 0 Fri Nov 30 10:34:24 2018
WorkingOnClass D 0 Wed Feb 28 09:37:52 2018
ML-Test001 D 0 Fri Dec 7 16:38:56 2018
TestML4Testing D 0 Wed Aug 22 08:58:42 2018
ML-NewDataSE SetCases1.xlsx A 1415577 Wed Aug 29 14:00:16 2018
DR0001-Dum01 D 0 Thu Aug 16 08:24:25 2018
DR0002-Dum02 D 0 Thu Aug 16 09:04:50 2018
Readme File for Documentation And Data Description.docx A 16136 Wed Aug 29 14:00:24 2018
ML Database Prototype D 0 Thu Dec 6 15:11:11 2018
OneNote D 0 Mon Dec 3 09:39:20 2018
Data A 0 Mon Dec 10 11:07:46 2018
\RJ9-5
. D 0 Fri Nov 30 10:34:24 2018
.. D 0 Mon Dec 10 11:07:46 2018
KLR0151_Set023_Files_RJ9_05.xlsx A 182462 Wed Apr 4 02:48:55 2018
KLR0152_Set023_Files_RJ9_05.xlsx A 525309 Wed Apr 4 02:53:57 2018
\ML-Test001
. D 0 Wed Feb 28 09:37:52 2018
.. D 0 Mon Dec 10 11:07:46 2018
WT_Conforming_Format1_1.docx A 500914 Mon Feb 26 08:50:55 2018
Conforming_Format_1_1.xlsx A 130647 Mon Feb 26 08:52:33 2018
DR0135_Dum01_text.xls A 974848 Mon Feb 12 08:11:11 2018
DR0139_Dum02_body.xls A 1061888 Tue Jun 19 13:43:54 2018
DataSet_File_mod0874953.xlsx A 149835 Mon Feb 26 14:17:02 2018
File Path For Dataset-2018.07.11.xlsx A 34661 Mon Feb 12 09:27:17
This is script right here can make the job:
#!/bin/bash
awk -v OFS=, '
BEGIN { print "PATH, FILENAME, SIZE, TIMESTAMP" }
/[\\]/ { path=$0 }
$2 ~ /A/ {print path"\\"$1,$3,$4 " " $5 " " $6 " " $7 " "$8 }
' "$#"
But is ignoring the names with spaces on it, so I need to validate them with something like:
awk -v FS="\t" '{print $1}'
But I could't integrate into the shell script, because the way the shell script is working, so I was thinking on make AWK to start reading by the end, since the end is always the same, and leave the rest.
The output should something like this:
/RJ9-5/KLR0151_Set023_Files_RJ9_05.xlsx,182462,Wed Apr 4 02:48:55 2018
/RJ9-5/KLR0152_Set023_Files_RJ9_05.xlsx,25309,Wed Apr 4 02:53:57 2018
/ML-Test001/WT_Conforming_Format1_1.docx,500914,Mon Feb 26 08:50:55 2018
/ML-Test001/Format_1_1.xlsx,130647,Mon Feb 26 08:52:33 2018
/ML-Test001/DR0135_Dum01_text.xls,974848,Mon Feb 12 08:11:11 2018
/ML-Test001/DR0139_Dum02_body.xls,1061888,Tue Jun 19 13:43:54 2018
/ML-Test001/DataSet_File_mod0874953.xlsx,149835,Mon Feb 26 14:17:02 2018
/ML-Test001/File Path For Dataset-2018.07.11.xlsx,34661,Mon Feb 12 09:27:17 2018
With GNU awk for the 3rd arg to match() (and far less importantly \s shorthand for [[:space:]]):
$ cat tst.awk
BEGIN { OFS="," }
{ gsub(/^\s+|\s+$/,"") }
sub(/^\\/,"/") { path = $0; next }
path == "" { next }
match($0,/^(.*[^ ]) +A +([^ ]+) +(.*)/,a) { print path "/" a[1], a[2], a[3] }
$ awk -f tst.awk file
/RJ9-5/KLR0151_Set023_Files_RJ9_05.xlsx,182462,Wed Apr 4 02:48:55 2018
/RJ9-5/KLR0152_Set023_Files_RJ9_05.xlsx,525309,Wed Apr 4 02:53:57 2018
/ML-Test001/WT_Conforming_Format1_1.docx,500914,Mon Feb 26 08:50:55 2018
/ML-Test001/Conforming_Format_1_1.xlsx,130647,Mon Feb 26 08:52:33 2018
/ML-Test001/DR0135_Dum01_text.xls,974848,Mon Feb 12 08:11:11 2018
/ML-Test001/DR0139_Dum02_body.xls,1061888,Tue Jun 19 13:43:54 2018
/ML-Test001/DataSet_File_mod0874953.xlsx,149835,Mon Feb 26 14:17:02 2018
/ML-Test001/File Path For Dataset-2018.07.11.xlsx,34661,Mon Feb 12 09:27:17
Try this Perl solution:
$ perl -lane ' if(/^\s*$/) { $x=0;$y=0} if(/^\\/) {$x=1 ;($a=$_)=~s/\s*$//g;$a=~s/\\/\//g; } $y++ if $x==1 ; if($y>3) { s/^\s*//g; $_=~s/(.+?)\s+\S+\s+((\d+)\s+.+)/$1 $2/g;print "$a/$_" } ' essparaq.txt
/RJ9-5/KLR0151_Set023_Files_RJ9_05.xlsx 182462 Wed Apr 4 02:48:55 2018
/RJ9-5/KLR0152_Set023_Files_RJ9_05.xlsx 525309 Wed Apr 4 02:53:57 2018
/ML-Test001/WT_Conforming_Format1_1.docx 500914 Mon Feb 26 08:50:55 2018
/ML-Test001/Conforming_Format_1_1.xlsx 130647 Mon Feb 26 08:52:33 2018
/ML-Test001/DR0135_Dum01_text.xls 974848 Mon Feb 12 08:11:11 2018
/ML-Test001/DR0139_Dum02_body.xls 1061888 Tue Jun 19 13:43:54 2018
/ML-Test001/DataSet_File_mod0874953.xlsx 149835 Mon Feb 26 14:17:02 2018
/ML-Test001/File Path For Dataset-2018.07.11.xlsx 34661 Mon Feb 12 09:27:17
$ cat essparaq.txt
. D 0 Mon Dec 10 11:07:46 2018
.. D 0 Mon Feb 19 11:38:06 2018
RJ9-5 D 0 Fri Nov 30 10:34:24 2018
WorkingOnClass D 0 Wed Feb 28 09:37:52 2018
ML-Test001 D 0 Fri Dec 7 16:38:56 2018
TestML4Testing D 0 Wed Aug 22 08:58:42 2018
ML-NewDataSE SetCases1.xlsx A 1415577 Wed Aug 29 14:00:16 2018
DR0001-Dum01 D 0 Thu Aug 16 08:24:25 2018
DR0002-Dum02 D 0 Thu Aug 16 09:04:50 2018
Readme File for Documentation And Data Description.docx A 16136 Wed Aug 29 14 :00:24 2018
ML Database Prototype D 0 Thu Dec 6 15:11:11 2018
OneNote D 0 Mon Dec 3 09:39:20 2018
Data A 0 Mon Dec 10 11:07:46 2018
\RJ9-5
. D 0 Fri Nov 30 10:34:24 2018
.. D 0 Mon Dec 10 11:07:46 2018
KLR0151_Set023_Files_RJ9_05.xlsx A 182462 Wed Apr 4 02:48:55 2018
KLR0152_Set023_Files_RJ9_05.xlsx A 525309 Wed Apr 4 02:53:57 2018
\ML-Test001
. D 0 Wed Feb 28 09:37:52 2018
.. D 0 Mon Dec 10 11:07:46 2018
WT_Conforming_Format1_1.docx A 500914 Mon Feb 26 08:50:55 2018
Conforming_Format_1_1.xlsx A 130647 Mon Feb 26 08:52:33 2018
DR0135_Dum01_text.xls A 974848 Mon Feb 12 08:11:11 2018
DR0139_Dum02_body.xls A 1061888 Tue Jun 19 13:43:54 2018
DataSet_File_mod0874953.xlsx A 149835 Mon Feb 26 14:17:02 2018
File Path For Dataset-2018.07.11.xlsx A 34661 Mon Feb 12 09:27:17
This question already has answers here:
Bash script/command to print out date 5 min before/after
(4 answers)
Closed 5 years ago.
I want to add 10 seconds 10 times. But I don't know well how to add times to the value.
This is my code.
./time.sh
time=$(date)
counter=1
while [ $counter -le 10 ]
do
echo "$time"
time=$('$time + 10 seconds') //error occurred.
((counter++))
done
echo All done
Using GNU Date
Assuming GNU date, replace:
time=$('$time + 10 seconds')
with:
time=$(date -d "$time + 10 seconds")
Putting it all together, try:
$ cat a.sh
t=$(date)
counter=1
while [ "$counter" -le 10 ]
do
echo "$t"
t=$(date -d "$t + 10 seconds")
((counter++))
done
echo All done
(I renamed time to t because time is also a bash built-in command and it is best to avoid potential confusion.)
When run, the output looks like:
$ bash a.sh
Tue Jan 16 19:19:44 PST 2018
Tue Jan 16 19:19:54 PST 2018
Tue Jan 16 19:20:04 PST 2018
Tue Jan 16 19:20:14 PST 2018
Tue Jan 16 19:20:24 PST 2018
Tue Jan 16 19:20:34 PST 2018
Tue Jan 16 19:20:44 PST 2018
Tue Jan 16 19:20:54 PST 2018
Tue Jan 16 19:21:04 PST 2018
Tue Jan 16 19:21:14 PST 2018
All done
Using Bash (>4.2)
Recent versions of bash support date calculations without external utilities. Try:
$ cat b.sh
#!/bin/bash
printf -v t '%(%s)T' -1
counter=1
while [ "$counter" -le 10 ]
do
((t=t+10))
printf '%(%c)T\n' "$t"
((counter++))
done
echo All done
Here, t is time since epoch in seconds.
When run, the output looks like:
$ bash b.sh
Tue 16 Jan 2018 07:31:44 PM PST
Tue 16 Jan 2018 07:31:54 PM PST
Tue 16 Jan 2018 07:32:04 PM PST
Tue 16 Jan 2018 07:32:14 PM PST
Tue 16 Jan 2018 07:32:24 PM PST
Tue 16 Jan 2018 07:32:34 PM PST
Tue 16 Jan 2018 07:32:44 PM PST
Tue 16 Jan 2018 07:32:54 PM PST
Tue 16 Jan 2018 07:33:04 PM PST
Tue 16 Jan 2018 07:33:14 PM PST
All done
I have this specific file:
Client 1: MAC 00:03:52:49:99:55
First : Fri Nov 7 09:50:11 2014
Last : Fri Nov 7 09:51:06 2014
--
Client 1: MAC 00:03:52:04:88:55
First : Fri Nov 7 09:51:44 2014
Last : Fri Nov 7 09:51:44 2014
--
Client 2: MAC 00:03:52:49:99:55
First : Fri Nov 7 10:50:10 2014
Last : Fri Nov 7 10:50:10 2014
--
Client 3: MAC 00:03:52:04:66:55
First : Fri Nov 7 09:51:30 2014
Last : Fri Nov 7 09:51:30 2014
--
From this file with many duplicate items like to create a new file like this:
00:03:52:49:99:55
First : Fri Nov 7 09:50:11 2014
Last : Fri Nov 7 09:51:06 2014
First : Fri Nov 7 09:50:11 2014
Last : Fri Nov 7 09:51:06 2014
00:03:52:04:88:55
First : Fri Nov 7 09:51:44 2014
Last : Fri Nov 7 09:51:44 2014
00:03:52:04:66:55
First : Fri Nov 7 09:51:30 2014
Last : Fri Nov 7 09:51:30 2014
How i can search with Bash Script the File with a For-Loop ? Important that the loop don't make more then 1 entry for the MAC-address. The MAC should be unique.
yes i have tried all this day :/
#!/bin/bash
array=$(cat Kismet-20141107-09-48-19-1.nettxt | grep Client -A 3 | grep -v Manuf)
echo "Array size: ${#array[#]}"
echo "Array items:"
for item in ${array[*]}
do
if [ $item -eq 3 ]; then
echo "$array[$item]"
fi
done
no it's not a requirement to use bash.. if you have other tools i will try it!
Try this out:
Shell> cat test1
#!/bin/bash
MACS=(`grep Client file|awk '{print $4}'|sort|uniq|xargs`)
for i in `echo ${MACS[*]}`; do
echo $i
grep $i file -A 2 | grep -vE 'MAC|--'
done
Shell> cat file
Client 1: MAC 00:03:52:49:99:55
First : Fri Nov 7 09:50:11 2014
Last : Fri Nov 7 09:51:06 2014
--
Client 1: MAC 00:03:52:04:88:55
First : Fri Nov 7 09:51:44 2014
Last : Fri Nov 7 09:51:44 2014
--
Client 2: MAC 00:03:52:49:99:55
First : Fri Nov 7 10:50:10 2014
Last : Fri Nov 7 10:50:10 2014
--
Client 3: MAC 00:03:52:04:66:55
First : Fri Nov 7 09:51:30 2014
Last : Fri Nov 7 09:51:30 2014
--
Output:
Shell> ./test1
00:03:52:04:66:55
First : Fri Nov 7 09:51:30 2014
Last : Fri Nov 7 09:51:30 2014
00:03:52:04:88:55
First : Fri Nov 7 09:51:44 2014
Last : Fri Nov 7 09:51:44 2014
00:03:52:49:99:55
First : Fri Nov 7 09:50:11 2014
Last : Fri Nov 7 09:51:06 2014
First : Fri Nov 7 10:50:10 2014
Last : Fri Nov 7 10:50:10 2014
Right now, I am running the following command:
rpm -qa --queryformat '%{name}\t%{installtime:date}\n' | sort -nr
and getting some output like this:
dhclient Fri 07 Feb 2014 01:37:47 PM EST
device-mapper-persistent-data Fri 07 Feb 2014 01:27:37 PM EST
device-mapper-libs Fri 07 Feb 2014 01:34:44 PM EST
device-mapper Fri 07 Feb 2014 01:34:46 PM EST
device-mapper-event-libs Fri 07 Feb 2014 01:34:48 PM EST
device-mapper-event Fri 07 Feb 2014 01:34:50 PM EST
dbus-libs Fri 07 Feb 2014 01:25:28 PM EST
dbus-glib Fri 07 Feb 2014 01:33:48 PM EST
db4-utils Fri 07 Feb 2014 01:30:05 PM EST
db4 Fri 07 Feb 2014 01:24:58 PM EST
dash Fri 07 Feb 2014 01:30:19 PM EST
cyrus-sasl-lib Fri 07 Feb 2014 01:25:48 PM EST
(note the odd tabs)
How do I tell the command I want it to output it into a table with common spacing instead of specifying the number of tabs?
Extra Question:
What I'm trying to do is just find out what has been installed and when so I can uninstall everything that I installed recently. How do I do that better than what I'm doing?
rpm -qa --queryformat '%-40{name} %{installtime:date}\n' | sort -nr
^^^
This will left-align the name and pad it to 40 characters.
If you want to order by time, you could print the numeric time first so it's easy to sort by.
$ rpm -qa --queryformat '%-10{installtime} %{installtime:date} %{name}\n' | sort -n
...
1375369678 Thu 01 Aug 2013 11:07:58 AM EDT xorg-x11-util-macros
1375886901 Wed 07 Aug 2013 10:48:21 AM EDT libdc1394
1378148462 Mon 02 Sep 2013 03:01:02 PM EDT gnome-system-monitor
1384526666 Fri 15 Nov 2013 09:44:26 AM EST perl-File-Next
1384526667 Fri 15 Nov 2013 09:44:27 AM EST ack
1385065567 Thu 21 Nov 2013 03:26:07 PM EST trousers
1385065568 Thu 21 Nov 2013 03:26:08 PM EST tpm-tools
1387405750 Wed 18 Dec 2013 05:29:10 PM EST libusb1