I have a text file (Markdown, actually, but close enough) with dates in a slightly annoying US format %m/%d/%Y. I'd like it to be in ISO-8601 format everywhere throughout.
I could easily write a 5-10 line script to do this specific conversion, but it feels like the sort of thing someone has written a more complete version of already. I just don't know what this hypothetical tool is called.
My imagined tool would take arbitrary formats for input and output (but from the world of datetime formats), although maybe with shortcuts for common ones. Basically, a datetime-aware sed is what I would want. So I might run, e.g.:
$ datecvt -i US -o ISO8601 < myfile.txt > gooddates.txt
$ # ... or ...
$ datecvt -i '%m/%d/%Y' -o '%Y-%m-%d' --inplace myfile.txt
Perhaps slightly different usage syntax, but that sort of thing. Does it exist? (for OSX, but almost all Linux-oriented tools can be compiled, and have been with Homebrew).
It appears GNU date interprets 01/02/2018 as January 2nd
$ date -d '01/02/2018'
Tue Jan 2 00:00:00 EST 2018
So you can do this:
$ date -d '01/02/2018' -I
2018-01-02
Or, if you want to be explicit about the input format, perl is useful:
$ perl -MTime::Piece -slE 'say Time::Piece->strptime($date, "%m/%d/%Y")->ymd' -- -date=01/02/2018
2018-01-02
The date utility does a part of what I want. But only for one single string, not to process a file or stream.
I know how to do the task in various programming languages. What I hoped for was a command-line utility to do it. I decided to try to write a "moderately good" version in Julia. Not because it's necessarily the best language to choose (startup time is terrible), but simply because I wanted to play with the language. I'm sure this early attempt is not idiomatic or optimal, but it does basically the goal:
gist.github.com/DavidMertz/447661d67f7b3f5ffa3c23d56f3b7294
Related
date +'%A %B %d' | sed -e 's/\(^\|[^[:digit:]]\+\)0\+\([[:digit:]]\)/\1\2/g
I like the output of the above command, which strips leading zeroes off days of the month produced by the date command, in the case of numerals less than 10. It's the only way I've thus far found of producing single digit dates from the date command's output for the day of the month, which otherwise would be 01, 02, 03, etc.
A couple of questions in this regard. Is there a more elegant way of accomplishing the stated goal of stripping off zeroes? I do know about date's %e switch and would like to use it, but with numerals 10 and greater it has the undesirable effect of losing the space between the month name and the date (so, July 2 but July10).
The second question regards the larger intended goal of arriving at such an incantation. I'm putting together a script that will scrape some data from a web page. The best way of locating the target data on the page is by searching on the current date. But the site uses only single digits for the first 9 days of the month, thus the need to strip off leading zeroes. So what's the best way of getting this complex command into a variable so I can call it within my script? Would a variable within a variable be called for here?
RESOLUTION
I'll sort of answer my own question here, though it is really input from Renaud Pacalett (below) that enabled me to resolve the matter. His input revealed to me that I'd not understood very well the man page, particularly the part where is says "date pads numeric fields with zeroes," and below that where it is written "- (hyphen) do not pad the field." Had I understood better those statements, I would have realized that there is no need for the complex sed line through which I piped the date output in the title of this posting: had I used there %-d instead of just %d there would have been no leading zeroes in front of numerals less than 10 and so no need to call sed (or tr, as suggested below by LMC) to strip them off. In light of that, the answer to the second question about putting that incantation into a variable becomes elementary: var=$(date +'%A %B %-d') is all that is needed.
I may go ahead and mark Renaud Pacalet's response as the solution since, even though I did not implement all of his suggestions into the latest incarnation of my script, it proved crucial in clarifying key requirements of the task.
If your date utility supports it (the one from GNU coreutils does) you can use:
date +'%A %B %-d'
The - tells date to not pad the numeric field. Demo:
$ date -d"2021/07/01" +'%A %B %-d'
Thursday July 1
Not sure I understand your second question but if you want to pass this command to a shell script (I do not really understand why you would do that), you can use the eval shell command:
$ cat foo.sh
#!/usr/bin/env bash
foo="$(eval "$1")"
echo "$foo"
$ ./foo.sh 'date -d"2021/07/01" +"%A %B %-d"'
Thursday July 1
Please pay attention to the double (") and simple (') quotes usage. And of course, you will have to add to this example script what is needed to handle errors, avoid misuses...
Note that many string comparison utilities support one form or another of extended regular expressions. So getting rid of these leading zeros or spaces can be as easy as:
grep -E 'Thursday\s+July\s+0*1' foo.txt
This would match any line of foo.txt containing
Thursday<1 or more spaces>July<1 or more spaces><0 or more zeros>1
I‘m aware of date +%u to get the day of the week for today.
I‘d like to get that integer for any arbitrary date i input - if possible in the format I choose (e.g. %YYmmdd)
ok, found it finally:
date -j -f %Y%m%d +%u 20200910
this is, because date on macOS doesn't take a switch for putting in custom date (fyi for those folks, how try to make -v work, like me^^)
in addition, -f affects only input format (it's literally the second word in the man page, but I managed to overlook more than once)
-j is needed to use -f without setting the date.
hope this will spare someone time in the future ;)
edit:
it seems to be important, to specify input format before output format (see comment from #chepner below)
(also be careful with quotes)
$ date +%u -d "2020-09-10"
4
I am trying to get the last time the date a file was modified. I used a variable for date and a variable for time.
This will get the date and time but I want to use -r using the date command to make a reflection of when the date was last modified. Just not sure how I go about using it in my variables.
How would I go about doing this?
Here are my variables:
DATE="$(date +'%m/%d/%Y')"
TIME="$(date +'%H:%M')"
I tried putting the -r after and before the time and date.
Though people might tell you, you should not parse the output of ls, simply that can easily break if your file name contains tabs, spaces, line breaks, your user decides to simply specify a different set of ls options, the ls version you find is not behaving like you expected...
Use stat instead:
stat -c '%Y'
will give you the seconds since epoch.
Try
date -d "#$(stat -c '%Y' $myfile)" "+%m/%d/%Y"
to get the date, and read through man date to get the time in the format you want to, replacing '%F' in the command line above:
date -d "#$(stat -c '%Y' $myfile)" "+%H:%M"
EDIT: used your formats.
EDIT2: I really don't think your date format is wise, because it's just so ambiguous for anyone not from the US, and also it's not easily sortable. But it's a cultural thing, so this is more of a hint: If you want to make your usable for people from abroad, either use Year-month-day as format, or get the current locale's setting to format dates.
I think you are looking for
ls -lt myfile.txt
Here in 6th column you will see when file was modified.
Or you could use stat myfile.txt to check the modified time of a file.
I know this is a very old question, but, for the sake of completeness, I'm including an additional answer here.
The original question does not specify the specific operating system. stat differs significantly from SysV-inspired Unixes (e.g. Linux) and BSD-inspired ones (e.g. Free/Open/NetBSD, macOS/Darwin).
Under macOS Big Sur (11.5), you can get the date of a file with a single stat command:
stat -t '%m/%d/%Y %H:%M' -f "%Sm" myfile.txt
will output
04/10/2021 23:22
for April 10, 2021.
You can easily put that in two commands, one for the date, another for the time, of course, to comply with the original question as formulated.
Use GNU stat.
mtime=$(stat --format=%y filename)
sample code :
modified_time=`ls -lt core* | head -1 | awk '{print $6,$7,$8}'`
echo modified time = $modified_time
I am trying to convert the last modified time for a file in seconds with the help of below command on aix box
t2=`date +'%s' -d "$modified_time"`
echo t2 = $t2
Note : the code i have posted is working on cygwin on bash. However its giving error on AIX ( ksh ).
I am getting below error :
egdev04{stc}[/home/stc]% t2=`date +'%s' -d "$modified_time"`
Invalid character in date/time specification.
Usage: date [-u] [+Field Descriptors]
Could someone please let me know what part of the code is wrong and suggest what needs to be used instead.
Unfortunately, date(1) is really poorly covered by standards, especially on dated systems, such as AIX (no pun intended).
Even on modern GNU/Linux vs BSD systems, there are different keys to achieve the behavior that you try to invoke:
GNU date has one key:
-d, --date=STRING
display time described by STRING, not 'now'
BSD date would use two keys and special invocation:
date [-jnRu] -f input_fmt new_date [+output_fmt]
-j Do not try to set the date. This allows you to use the -f flag
in addition to the + option to convert one date format to
another.
-f Use input_fmt as the format string to parse the new_date provided
rather than using the default [[[[[cc]yy]mm]dd]HH]MM[.ss] format.
Parsing is done using strptime(3).
AIX doesn't seem to include either one of these facilities. So, ultimately, if you really need that you'll have to execute a micro-script in some scripting language, such as Perl/Ruby/Python/etc.
Going a step backwards, parsing results of ls(1) is always a very bad idea, as they tend to vary wildly based on particular OS implementation, locale, output format, "human-readable" defaults, etc. If you really just want to get some file modification time, why don't you use stat(1)? May be it's available on AIX? Something like
stat -c '%Y' "$file"
seems to solve your task.
I have a string with a custom date format written in Japanese: 2013年1月8日 20時19分. With osx's date command, I can convert this to some other format with the following command:
timestamp="2013年1月8日 20時19分"
date -j -f "%Y年%m月%d日 %H時%M分" "$timestamp" +"%F %R"
While searching I found this question helpful, but it ultimately did not help when it came to gnu date. The command gdate -d "2013年1月8日 20時19分" +"%F %R" fails saying that it does not understand the date format. The -d flag allows some simple formats, but how I can apply a more radical custom format and convert the date? Am I stuck with parsing the string myself with string manipulation in shell?
Any help would be greatly appreciated.
You probably will have to tinker with some environment variables (ex: TZ, LC_ALL, etc).
See this page showing you most of the common environnement variables, and their meanings
To try some: you can force the value to change just for the duration of the following command by putting them on the same line, before the command itself:
TZ=.... LC_LANG=..... date -d .......
will invoke date -d .... with the 2 environment variables TZ and LC_LANG set to a temporary value.
Some interresting pointers (I can't right now tell if there is a program that will take as input any locale's date and translate that to the relevant Epoch or Unix Timestamp... BUt there seems to be hope following that (looking quite standard) trail of online docs:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/date.html
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02
which talks, amongst many other, about:
LC_TIME
This variable shall determine the locale category for date and time formatting information. It affects the behavior of the time functions in strftime(). Additional semantics of this variable, if any, are implementation-defined.
Which points to: http://pubs.opengroup.org/onlinepubs/9699919799/functions/getdate.html
which says in the middle:
The match between the template and input specification performed by getdate() shall be case-insensitive.
The month and weekday names can consist of any combination of upper and lowercase letters. The process can request that the input date or time specification be in a specific language by setting the LC_TIME category (see setlocale ).
and points to: http://pubs.opengroup.org/onlinepubs/9699919799/functions/setlocale.html
... I wish you an happy reading ! Let us know what you find!
I finally figured this out with the aid of the coreutils mailing list. However, the example they give there uses perl. They specifically rely on the POSIX::strptime module, which does not come with a standard installation of perl. Therefore, I solved this with python, which has the time module. This module should be available in most installations of python2 and python3.
Here's how to use it programmatically:
Python solution:
$ timestamp='2013年1月8日 20時19分'
$ time_format='%Y年%m月%d日 %H時%M分'
$ gdate -u -R -d "$(python -c 'import sys; from time import strptime; t=strptime(sys.argv[-1],"'$time_format'"); print("%d-%d-%d %d:%d"%(t.tm_year,t.tm_mon,t.tm_mday,t.tm_hour,t.tm_min))' $timestamp)"
Tue, 08 Jan 2013 20:19:00 +0000
This works with both python2 and python3. You can substitute any timestamp and format as you like.
Perl solution
To document the answer given to me on coreutils, the perl solution is this (requires POSIX::strptime)
$ gdate -u -R -d "$(perl -MPOSIX::strptime -le 'my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday) = POSIX::strptime("$ARGV[0]","%Y年%m月%d日 %H時%M分");$year+=1900;$mon+=1;printf("%04d-%02d-%02d %0d:%02d\n",$year,$mon,$mday,$hour,$min);' "2013年1月8日 20時19分")"