I have a program (jhead) that compiles with very few tweaks for both Windows and generic Unix variants. From time to time, windows users ask if it can be modified to also set the "creation date/time" of the files, but I don't see a way to do this with the POSIX api. What I'm currently doing is:
{
struct utimbuf mtime;
mtime.actime = NewUnixTime;
mtime.modtime = NewUnixTime;
utime(FileName, &mtime);
}
Ideally, struct utimebuf would just have a creation time, but it doesn't. It strikes me it would take a lot of windows specific, non-portable code to change the creation time under Windows. Is there another POSIX way of doing this? Any suggestions?
POSIX only recognizes three different file times:
atime (access time): The last time the file was read
mtime (modification time): The last time the file was written
ctime (attribute change time): The last time the file's metadata was modified
Any other file times that may exist in the underlying OS require OS-specific API calls in order to be modified.
And don't worry about creating non-portable code; only these times really exist under most *nix variants.
The Win32 API for this isn't really all that bad, as Windows APIs go: https://msdn.microsoft.com/en-us/library/windows/desktop/ms724933%28v=vs.85%29.aspx . The trickiest thing is to work out how many seconds Windows thinks there were between 1st January 1601 and 1st January 1970; the rest is straightforward.
Related
In my Synology NAS, I have an APFS share with files that have been transferred back and forth for decades across different OSes.
original systems: probably ext4 filesystem and Synology-hosted NFS mount, years ago (various systems, Linux/Windows)
current system: EXT4 filesystem, with Synology-hosted AFP mounts (to a macOS 10.15 system, though I doubt that matters)
For files that were copied via NFS originally, and now hosted via AFP, all the file dates seem to be offset by some amount. I can sort of eyeball the datetime offset, but is there a definitive number I can use? (And a simple way to parse/modify the times using something like GetFileInfo?)
For reference, I have a copy of iTerm2-3_2_6.zip, dated "2039-01-22 08:25:17". I would probably map that to 2019-01-21 (release date for 3.2.7), implying a 20-year offset.
The closest thing I can think of is macOS epoch starting on 2001-01-01 instead of UNIX 1970-01-01, but that's a 30-year offset.
There's also the "year 2038 problem", and some software might be doing something clever with 32-bit overflows to support post-2038 datetimes, but I have at least one file dated "2031-08-10", so that seems unlikely.
This seems to be something related to 32-bit and 64-bit overflows, somewhere in this complicated storage stack; the way the datetimes + error offsets add up is consistently close to 2^31, though I wasn't able to find any clear pattern.
Also, I noticed strange behavior from my Synology system's use of eCryptFS, which seems to lag metadata updates when done in batch. (In particular, I suspect that some eCryptFS/Synology metadata is getting translated incorrectly, or just never written to disk.)
Anyway, I basically wrote a Python script that does the following:
check if os.stat() reports an atime/mtime newer than 2030
check that both atime and mtime are newer; error out if they differ
adjust both times back by 632599096 seconds (offset based on comparing copies of the one file I found in common between two systems).
with the following caveats to watch out for:
macOS's GetFileInfo/SetFile utilities only support 32-bit datetimes, so you should generally avoid using them (even just to verify the metadata updates).
something in the Synology/eCryptFS encryption gets very slow, so after a few dozen metadata updates, the updates will no longer be visible (even after calling sync from the shell). But if you give it some time, you'll see the corrected atime/mtime changes.
The OS-specific os.stat field, ctime, does literally track metadata update times. And there doesn't seem to be a way to manually set it (nor a need to, since this isn't visible through any GUI).
The combination of slow metadata updates + GetFileInfo reporting the wrong time made this incredibly frustrating, until I figured both out. In practice, this means you have to test metadata updates on a few files at a time, then your large batch execution can only be verified a few hours later (I waited a day).
This should have been a blog post, good riddance.
Does Windows offer anything like POSIX stat's st_ctime field, which gives the timestamp of last metadata change?
Background: I have some data files that I'd like to check if they've been modified since a particular timestamp. Checking the modified timestamp (mtime) is easy and takes care of "normal" modifications, but if the user copies over older versions of the data files, then the modified timestamp will show that they're older. On a POSIX system, copying over an older file would result in ctime being newer, even if mtime is older.
As far as I can tell, Windows does provide a "last changed" field, but I haven't found any Win32 API for accessing it, so you have to use the Native API.
Specifically: (I haven't actually tried this.)
Call NtOpenFile to get a handle.
Call NtQueryInformationFile with a FileInformationClass parameter of FileNetworkOpenInformation to get a FILE_NETWORK_OPEN_INFORMATION struct.
The ChangeTime member of the FILE_NETWORK_OPEN_INFORMATION struct is equivalent to the POSIX ctime.
Using the Native API isn't exactly straightforward. This question and answer describe how to do it.
Cygwin takes advantage of this to provide POSIX semantics under Windows, as discussed on their mailing list. I'm getting my information from their implementation.
And although it's only tangentially related, this article has a good description of the "created" timestamp that Windows does present (and unfortunately refers to as ctime). The created timestamp can be newer than the modified timestamp if a file is copied to a new location (since created then refers to when the copy was created, while modified refers to when the original was last modified), but it otherwise is not updated when metadata is changed.
I'm trying to set the filesystem creation time for a file on Mac OS using a ruby script.
On Mac OS X the 'ctime' represents the last time of inode modification rather than the file creation time, thus using ruby's File.utime() to set ctime will not help.
Using this hint [ http://inessential.com/2008/12/18/file_creation_date_in_ruby_on_macs ] I can retrieve a file's creation time:
Time.parse(`mdls -name kMDItemContentCreationDate -raw "#{filename}"`)
...but any idea on how to set it using ruby?
-- UPDATE --
Okay, I think I can actually do this with File.utime in ruby.
Even though the ctime is technically not used by Mac OS to track file creation time, when you use utime to update the ctime (along with the mtime, which must be simultaneously set) the filesystem appears to magically also update the creation time as per kMDItemContentCreationDate.
So to set filename to a ctime of 1st Oct 2010 and a mtime of 2nd Oct 2010:
File.utime(Time.strptime('011010', '%d%m%y'), Time.strptime('021010', '%d%m%y'), filename)
There is a Ruby solution with the method utime. But you will have to set modification time (mtime) and access time (atime) at once. If you want to keep access time you could use:
File.utime(File.atime(path), modification_time, path)
See Ruby core documentation as well.
So you've definitely got a pure Ruby solution working, but since this is OS X, are you opposed to exec() or system() and just using touch? In your case, I'd almost prefer:
system "touch -t YYYYMMDDhhmm /what/ever"
if for no other reason than clarity.
This works for me to update creation time on OS X 10.11.1:
system "SetFile -d '#{time.strftime "%m/%d/%Y %H:%M:%S"}' #{file}"
No claims of portability - SetFile is an OS X command (and the man page says it's deprecated with XCode 6, so may not work for very long) - couldn't find another way to do it though, Time.utime didn't update creation time, but only modified and accessed time.
See: https://apple.stackexchange.com/q/99536/65787
Ruby uses the utimes system call to change the file-times.
Reading the man-page for utimes explains what happens:
int
utimes(const char *path, const struct timeval *times);
..
If times is non-NULL, it is assumed to point to an array of two timeval
structures. The access time is set to the value of the first element,
and the modification time is set to the value of the second element. For
file systems that support file birth (creation) times (such as UFS2), the
birth time will be set to the value of the second element if the second
element is older than the currently set birth time. To set both a birth
time and a modification time, two calls are required; the first to set
the birth time and the second to set the (presumably newer) modification
time. ...
So ctime only get updated backwards in time.
For an assignment I need to determine the creation time of a random file. As far as I know, Unix does not store the creation time (in contrast to *-BSD). I think I've read somewhere that you should ask for the modification time instead but I don't know where and asking Google doesn't give me a non-ambigious answser either.
Any ideas?
You cannot get the creation time of files in this context. That makes the assignment easy enough: it reasonably cannot be completed.
If someone is talking about creation time in Unix, they are confused. Modification time is completely different from creation time (obviously).
For the record, there are exactly three timestamps in Unix files: ctime, atime and mtime.
Try stat. It will give you all the times associated with a file.
stat filename
To get just the modification date and time:
stat --format=%y filename
Or in seconds since the Epoch:
stat --format=%Y filename
I have a program that uses save files. It needs to load the newest save file, but fall back on the next newest if that one is unavailable or corrupted. Can I use the windows file creation timestamp to tell the order of when they were created, or is this unreliable? I am asking because the "changed" timestamps seem unreliable. I can embed the creation time/date in the name if I have to, but it would be easier to use the file system dates if possible.
If you have a directory full of arbitrary and randomly named files and 'time' is the only factor, it may be more pointful to establish a filename that matches the timestamp to eliminate need for using tools to view it.
2008_12_31_24_60_60_1000
Would be my recommendation for a flatfile system.
Sometimes if you have a lot of files, you may want to group them, ie:
2008/
2008/12/
2008/12/31
2008/12/31/00-12/
2008/12/31/13-24/24_60_60_1000
or something larger
2008/
2008/12_31/
etc etc etc.
( Moreover, if you're not embedding the time, what is your other distinguishing characteritics, you cant have a null file name, and creating monotonically increasing sequences is way harder ? need info )
What do you mean by "reliable"? When you create a file, it gets a timestamp, and that works. Now, the resolution of that timestamp is not necessarily high -- on FAT16 it was 2 seconds, I think. On FAT32 and NTFS it probably is 1 second. So if you are saving your files at a rate of less then one per second, you should be good there. Keep in mind, that user can change the timestamp value arbitrarily. If you are concerned about that, you'll have to embed the timestamp into the file itself (although in my opinion that would be ovekill)
Of course if the user of the machine is an administrator, they can set the current time to whatever they want it to be, and the system will happily timestamp files with that time.
So it all depends on what you're trying to do with the information.
Windows timestamps are in UTC. So if your timezone changes (ie. when daylight savings starts or ends) the timestamp will move forward/back an hour. Apart from that, and the accuracy of about 2 seconds, there is no reason to think that the timestamps are invalid, and its certainly ok to use them. But I think its bad practice, when you can simply put the timestamp in the name, or in the file itself even.
What if the system time is changed for some reason? It seems handy, but perhaps some other version number counting up would be better.
Added: A similar question, but with databases, here.
I faced some issues with created time of a file after deletion and recreation under same name.
Something similar to this comment in GetFileInfoEx docs
Problem getting correct Creation Time after file was recreated
I tried to use GetFileAttributesEx and then get ftCreationTime field of
the resulting WIN32_FILE_ATTRIBUTE_DATA structure. It works just fine
at first, but after I delete file and recreate again, it keeps giving
me the original already incorrect value until I restart the process
again. The same problem happens for FindFirstFile API, as well. I use
Window 2003.
this is said to be related to something called tunnelling
try usining this when you want to rename the file
Path.Combine(ArchivedPath, currentDate + " " + fileInfo.Name))