Currently I have a csv file like this:
11:00 p.m.
11:00 p.m.
03:00 p.m.
03:00 p.m.
05:00 a.m.
05:00 a.m.
07:00 a.m.
12:00 p.m.
07:00 a.m.
05:00 a.m.
I want to delete the duplicates that are in sequential rows so the output will be this:
11:00 p.m.
03:00 p.m.
05:00 a.m.
07:00 a.m.
12:00 p.m.
07:00 a.m.
05:00 a.m.
I do not want to delete all duplicates, just duplicates that are in sequential rows, for example if the 4th and 5th row match, delete one of the duplicate rows. Is there an easy way to do this without having to run a for-loop?
Try uniq.
It can do what exactly you want to do.
With awk
awk '$0 != prev; {prev=$0}' file.txt
Related
I've been using splitmuxsink to split the recordings of a live video stream by length or by the "split now" cmd. The resulting video files are all timestamped with the "global time" from when the pipeline started.
File1 00:00 - 05:00
File2 05:00 - 10:00
File3 10:00 - 15:00
This is causing issues when playing back the files in certain video players that are expected a time stamp starting at 0.
What I'd like to do is reset the time stamping every time the recordings are split and a new file is started.
File1 00:00 - 05:00
File2 00:00 - 05:00
File3 00:00 - 05:00
I'm trying to create a shell script in Linux to accept only 2 arguments for the date and one for the time (while at the same time accepting am/pm)
Here's one of the scripts I attempted to write up.
#!/bin/bash
date=$1
space=" "
time=$2
ampm=$3
timeampm=$2$space$3
print("date $1")
print("time $2")
print("ampm $3")
print("timeampm $timeampm")
Heres the file I'm going off
0310 02:00:00 AM Abigale Rich
0310 05:00:00 AM Billy Jones
0310 08:00:00 AM Billy Jones
0310 11:00:00 AM Summer-Louise Hammond
0310 02:00:00 PM Billy Jones
0310 05:00:00 PM Rahima Figueroa
0310 08:00:00 PM Billy Jones
0310 11:00:00 PM Billy Jones
0312 02:00:00 AM Abigale Rich
0312 05:00:00 AM Billy Jones
0312 08:00:00 AM Billy Jones
0312 11:00:00 AM Summer-Louise Hammond
0312 02:00:00 PM Billy Jones
0312 05:00:00 PM Rahima Figueroa
0312 08:00:00 PM Billy Jones
0312 11:00:00 PM Billy Jones
0315 02:00:00 AM Abigale Rich
0315 05:00:00 AM Billy Jones
0315 08:00:00 AM Billy Jones
0315 02:00:00 PM Billy Jones
0315 05:00:00 PM Rahima Figueroa
0315 08:00:00 PM Billy Jones
For example, on line 3, I want to be able to ./scriptname.sh 0310 08:00:00 AM and it pulls out the name "Billy Jones"
I have been mostly trying to use grep, awk and sed. If you want to see my other codes I've written up, I'll add them.
Your script contains several syntax errors and other oddities. Perhaps you are looking for
#!/bin/bash
date=$1
time=$2
ampm=$3
timeampm="$2 $3"
echo "date $1"
echo "time $2"
echo "ampm $3"
echo "timeampm $timeampm"
sed -n "s/^$date $timeampm //p" file
which of course can be reduced to just
#!/bin/sh
sed -n "s/^$1 $2 $3 //p" file
and you can run it with sh -x if you want to see exactly what it's doing.
For robustness, maybe add some validation for the arguments, or maybe switch to Awk which has less bewildering failure modes when you pass in things in unexpected formats. Plain standard grep doesn't easily let you remove the matching text and print the rest (though if you have GNU grep, waybe try grep -Po "^$1 $2 $3 \K.*" file).
More fundamentally, perhaps your file should contain dates in a standard format. On Linux, you can ask date -d to convert a large number of date formats and other time expressions to whichever format your file uses.
When I run this command I get what you'd expect:
date -d "2018-06-07 + 1 days"
Fri Jun 8 00:00:00 CEST 2018
1 day is added to the day provided (using midnight as starting point).
However when I try to work in a time (17:00:00), two things happen.
date -d "2018-06-07 17:00:00 + 28 days"
Up to 25 days, the output is wrong: wrong dates/wrong time (I have run this in a loop).
Above 25 days, it starts spitting out "date: invalid date ‘2018-06-07 17:00:00 +25 days’"
The manpage says about -d /--date that is pretty much free format. But I'm starting to think the plus sign is incorrectly interpreted (maybe as a timezone offset?) when you use the time (hours:minutes:seconds)?
So how can I add days FROM a timestamped date?
For increment on the days with timestamp to work, the timestamp needs to be in the standard format returned by default by the date command. So sanitize the date to a format in which it accepts minute arithmetic and do the processing.
date -d "2018-06-07 17:00:00"
Thu, Jun 07, 2018 5:00:00 PM
Now put it in a variable, e.g. putting your string in the example below
dateStr=$(date -d "2018-06-07 17:00:00")
date -d "$dateStr + 28 days"
returns
Thu, Jul 05, 2018 5:00:00 PM
The example uses timezones from IST.
Totally new to BASH. Apologies in advance.
Problem
I'd like to add X days to a specific date.
Code
I figured out that date in BASH retrieves the current date.
I also figured out that I can add X days to the current date in the following way,
expiration_date=$ date -v +1d
which gives,
Tue Sep 26 20:28:13 CEST 2017
which is indeed the date of writing plus X=1 days.
Question
In stead of date in the command line above, I'd like to insert a particular date to which X days will be added, e.g. 20/09/2017.
Don't care about the format of the particular date.
In other words: How do I make the following work,
expiration_date=$ '20/09/2017' -v +1d
Tried this answer, but doesn't do what I want.
Edit: Did not know things are different for OSX.
You can do this way:
dt='2017-09-20'
date -d "$dt +1 day"
Thu Sep 21 00:00:00 EDT 2017
date -d "$dt +2 day"
Fri Sep 22 00:00:00 EDT 2017
It seems OP is using OSX. You can use date addition this way:
s='20/09/2017'
date -j -v +1d -f "%d/%m/%Y" "$s"
Thu Sep 21 14:49:51 EDT 2017
You can do something like this:
date -d "Sun Sep 6 02:00:00 IST 2012+10 days"
I am trying to convert a UTC time to GMT time in my small script, but it doesn't work:
TimestampUTC=$(date +"%s")
echo $TimestampUTC
dates=$(date -d #$TimestampUTC)
echo $dates
## 2 hours difference between UTC and GMT
Hours2=120
TimestampGMT=$((TimestampUTC - Hours2))
echo $TimestampGMT
diff=$((TimestampUTC - TimestampGMT))
echo $diff
dateGMT=$(date -d #$TimestampGMT)
echo $dateGMT
The displayed result for $dateGMT is the same as $dates.
Thanks in advance.
error in script.
Unix timestaps are given in seconds.
Hours2=120 means 120 seconds.
So your 2 timestaps are diverging by 2 minutes, not 2 hours.
This code is correct:
Hours2=7200
Also you claim having 2 hours between GMT and UTC, I'm sure you mean CET (central european time)
Note: there is nothing like a CET timestamp. It's just the normal unix timestamp displayed with a timezone offset. So independently of world location, the unix timestamp is always, worldwide, the same at the same instant.
You can replace all your code by just this
# get the timestamp 2 hours in the future from now
date2h=$(date -d "2 hours" +%s)
Which gives you the unix timestamp from the future. It is NOT the current timestamp in CET. The current CET timestamp is always the same as UTC.
How to get the time from UTC and CET? Set the environment variable TZ before the command.
$ TZ=UTC date
Mon Aug 17 11:44:05 UTC 2015
$ TZ=CET date
Mon Aug 17 13:44:05 CEST 2015
$ TZ=GMT date
Mon Aug 17 11:44:05 GMT 2015
but the timestap is always the same
$ TZ=UTC date +%s
1439812072
$ TZ=CET date +%s
1439812072
$ TZ=GMT date +%s
1439812072
GMT and UTC do not differ by 2 hours. In fact they don't differ at all. So displaying the dates of GMT and UTC will always show exactly the same number.
Also I don't know bash but I find it hard to believe that 2 hours is represented by 120 minutes. Normally when doing math with dates milliseconds are used.
In your favourite terminal use the following sequence
export TZ=GMT; date
date_format='+%d %B %Y %H:%M'
datatest="2021-11-21 12:00:00 UTC"
echo $(date -d "$datatest" "$date_format")
datatest="2021-11-21 12:00:00 CET"
echo $(date -d "$datatest" "$date_format")
datatest="2021-11-21 12:00:00 GMT"
echo $(date -d "$datatest" "$date_format")
Out:
21 November 2021 13:00
21 November 2021 12:00
21 November 2021 13:00