I would like to automate a download of an image from a third party server with a CRON job and then upload the image to my website.
I have 2 issues:
First, the third party site changes the image name every day using the following logic:
http://thirdpartysite.com/ImageFinder.aspx?ReportID=FILENAME
where FILENAME is 26601 +14 for each day after 6 Oct 2014 (so 7 Oct would be 26615, 8 Oct would be 26629 etc).
How do I build this into a simple Linux bash script for use with wget?
Second, how do I upload this to my site via FTP (or similar) with the same script.
NOTE: I have permission to host the file on my site and have linked the original site / placed credit for the image.
Following the suggestions of #Abhay, first get the timestamp of Oct 6, let's store it in the variable $d0:
d0=$(date +%s -d 20141006)
Then store the timestamp of a target date, say Oct 8 and store it in $d1:
d1=$(date +%s -d 20141008)
Then you can calculate the difference and apply the required arithmetic operations in $((...)), like this:
echo $((26601 + 14 * (d1 - d0) / 60 / 60 / 24))
# outputs: 26629
The date command has one very good format: %s, which prints number of seconds since "epoch", which is the fixed date 1 January 1970, 00:00 UTC. I'll call it "timestamp". In combination with this, you can use the -d date-string, so that it prints the given date as number of seconds. Now you can take today's timestamp, subtract the timestamp of "6 Oct 2014" from it, and you get number of seconds between the two times. Now you can divide it by (60 * 60 * 24) to get it in number of days, and do further arithmetic to get the desired number, and make a file name out of it.
The date string formats that -d option takes are flexible, but as of now I am not sure whether it takes "6 Oct 2014" as is. Try a few permutations, or better, check the "info" page.
Related
I have two timestamps in formats :
Timestamp 1 (Variable - RunStartDate): Thu May 3 14:12:54 CDT 2018
Timestamp 2 (Variable - RunEndDate): Thu May 3 18:11:46 CDT 2018
I want the difference of number of hours between these two timestamps in UNIX shell. (I.e. RunEndDate - RunStartDate in hours)
Please help, I am new to UNIX and it is throwing me errors when I just try to subtract the two.
You have a few options here, such as calling out to Perl or Python and using a date/time library to do the math for you. Another option is to use the date program to convert the dates to seconds, subtract the values, and then convert back to hours. Unfortunately, you can't do floating-point math in Bash, so we'll have to call out to a helper program to do that, too.
START=$(date -d "$RunStartDate" +"%s")
END=$(date -d "$RunEndDate" +"%s")
HOURS=$(bc -l <<< "($END - $START) / 3600")
Note that this will only work on GNU systems (e.g. Linux).
oldest_year_month_temp=201602
NUM_PART_RETAIN=20
oldest_year_month=`date --date="$(oldest_year_month_temp +%Y%m) - $NUM_PART_RETAIN month" "+%Y%m"`
Date is not coming as expected.
One easy way to do it would be to simply append a 01 to your input of yymm to provide a format date -d could read as the starting date, then simply subtract 20 months and output the resulting date in %y%m format. For example, if you provide the date 9910 (Oct. 1999), you can do:
$ date -d "991001 - 20 months" +%y%m
9802
Which returns Feb. 1998 (20 months earlier)
(note: the $ above just indicates a command by a normal user as opposed to # indicating a command by the super user (e.g. root))
Inside the $(...) there must be a command, e.g. $(date ...).
This should have been obvious from the error message you got, which was probably oldest_year_month_temp: no such command.
When reading from a variable, you must write a $ before its name.
I have a time-stamp like 7:00:00, which means 7am.
I would like to write a short command that returns 06:45:00, or simply 06:45, preferably using date command so that I can avoid long shell script. Do you have any elegant solution?
I'm also looking for a 24h format. For example, 12:00:00 - 15 minutes = 11:45:00.
With GNU date, use 7:00:00 AM - 15 minutes as d (--date) string :
% date -d '7:00:00 AM - 15 minutes' '+%H:%M'
06:45
+%H:%M sets the output format as HH:MM.
On BSD variants Date has a -v flag which can be used to take the current timestamp and display the result of a positive or negative adjustment.
This will subtract 15mins from the current timestamp:
date -v -15M
We have a little issue with a time calculation in Bash.
Lets give you a little explaination about the situation up here.
We download every 5 minutes one file from an FTP server. This file contains time information about the data in this file. But the timeformat of the file is in UTC, and our local time is UTC+2. The files contains information about the past 5 minutes from local time. Now we have the following code:
TIMESTAMP=$(echo "$(TZ=UTC date "+%Y%m%d%H%M") - ($(date +%M)%5)-5" | bc)
That works well for several hours but after 55 minutes it becomes a problem. So we dont able to get the files with the 55 minutes, 60 minutes.
So if local time is: 19:47
The file with time 17:40 (utc) is available on the server at 19:45 local time
The time the files are available on the server are not constant too bad...
19:00, 19:05, 19:10 etc... but sometimes the file is one minute later....
This is my crontab file:
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/kbroeren/cronscripts
*/1 * * * * /home/kbroeren/cronscripts/radar >> /home/kbroeren/radar_log.txt 2>&1
*/5 * * * * sudo /usr/bin/python /home/kbroeren/cronscripts/radar_plot.py >>/home/kbroeren/out.txt 2>&1
Is there a better and correct way to do this ?
You seem to be looking for %s:
date +%s
This returns:
%s seconds since 1970-01-01 00:00:00 UTC
You'd need to change the arithmetic a bit, though.
The date command can convert to and from a number of date and time formats. If you want to do arithmetic with dates and times, it's probably easiest to convert to “seconds since the unix epoch”, do the arithmetic, and convert back to the time format of your choice.
You might consider something like:
date --date="#$(echo $(TZ=UTC date +%s) - $(date +%s)'%(5*60)-(5*60)' | bc)"
I need to convert Julian timestamp to Regular timestamp in UNIX using Bash.
On Tandem OS, conversion is pretty straightforward -
Example:
212186319010244541
$OLSAPP SYSTST 1> #interprettimestamp 212186319010244541
#interprettimestamp 212186319010244541 expanded to:
2455860 2011 10 25 16 10 10 244 541
I wish to do the same on UNIX environment. The conversion will be a part of a parser script. So one-liners would be greatly appreciated.
UPDATE:
INTERPRETTIMESTAMP inbuilt function on Tandem returns a space-separated list of nine numbers, consisting of the Julian day number, year, month, day, hour, minute, second, millisecond, and microsecond.
Assuming the number is as #blahdiblah says
"a value representing the number of microseconds since January 1, 4713 B.C."
Then you first need to know the Julian timestamp for 01-JAN-1970 which is the epoch for unix time. So a cludgy oracle query gives
210866803200000000
Then you could in theory just have a shell command to compute the number of seconds since 1-Jan-1970.
unixtime=$(( ( 212186319010244541 - 210866803200000000 ) / 1000000 ))
The problems with this are:
you still need to format it
your bash may not like integer arithmatic with 18 digit numbers. (think its OK in 64 bit, but not 32 bit).
Now if you have perl installed you can solve these using the bigint and POSIX modules. As a shell "one" liner it looks like
perl -mbigint -mPOSIX -e 'print( POSIX::strftime("%Y-%m-%d %T",localtime( ($ARGV[0]-210866803200000000)/1000000 ) )."\n")' 212186319010244541
Which gives
2011-10-25 15:10:10
The 1 hour difference is probably due to daylight savings differences. It could be either in the perl, or more likely the value I used for 01-Jan-1970 could be an hour out. So you may need to check both of them to be sure its right for your system.
this is MJD converter
MJD is -3506716800 than epoch
# date -d '1858-11-17 UTC' +%s
-3506716800
example:
MDJ 57153 is Mon May 11 00:00:00 UTC 2015
# date -d #`echo 57153*86400-3506716800|bc`
Mon May 11 00:00:00 UTC 2015