Is there really no way to copy a local file by
using some portable way
using built in golang calls ...
... which in turn use the native API of the operating system ?
All ways I found make use of some kind of Reader / Writer or use buffers to copy the content of the source to the target, either bytewise or blockwise or - even worse - spawn commands like cp.
Even add on libraries like [gut/yos][1] that I found by searching awesomego internally do the same
The downside of just reading the source file and writing the content to a new file is that you have to take care that these "properties" (depending on the OS you are using) of the source and destination match:
timestamps
permissions
setuid bits
(soft/hard) links
alternate data streams
I know there are specialized libraries that for example expose the Windows API and enable one to call CopyFileEx. But this approach would be limited to Windows and I need at least to support Linux.
Related
I have a library that has a file based interface (it only accept file names/paths).
Since the operations are time critical i would like to load the file to memory.
Is there a way to create create in-memory files that have a file path that is resolvable with the normal file access routines?
For example many programs still work if they get some win32 device path or an UNC path instead of a 'classic' windows path.
or maybe if that is not the case maybe a softlink can be used to map it into 'normal' filesystem.
One obvious way would be to use some ramdisk software but this would require further manual steps and also requires to tell the program the ramdisks letter.
The title pretty much says it, on Windows, can I somehow simulate multiple working directories in a multithreaded application by using something similar to openat?
See also this question.
Yes there is, and it's called NtCreateFile() (https://msdn.microsoft.com/en-us/library/bb432380(v=vs.85).aspx) :)
openat() takes an open fd to a base directory from which path operations start. Similarly, you can supply a HANDLE to the ObjectAttributes.RootDirectory of NtCreateFile() to have whatever that directory's path is to be used as a base for relative path operations.
If going direct to the NT kernel API is too much for you and you want to remain within Win32, you can retrieve the current path of the base directory HANDLE easily enough (see https://msdn.microsoft.com/en-us/library/windows/desktop/aa366789(v=vs.85).aspx). If you open the directory without FILE_SHARE_DELETE permissions, nobody will be able to rename it and therefore the path retrieved will never move so long as you keep the HANDLE open.
You then stitch together the retrieved path with the relative path using normal string concatenation.
The NT kernel API approach is the only solution on Windows which allows the base directory HANDLE to be arbitrarily renamed by third party processes and that not cause races in your code. i.e. it's the only equivalent to POSIX openat(). I agree it's very unfortunate that Win32 doesn't make available this facility - in fact, Win32 doesn't make available atomic renames which is a very useful POSIX facility indeed to let you avoid lock files and is something the NT kernel API provides too.
Is there an API or straight forward way to simulate any data as a windows explorer drive/file-hierarchy? I don't want to create actual files I want to have a view on my data as though it were.
E.g. if my real data was in a database but I want it to look like a drive/folder/files?
You can create shell namespace extension or a virtual file system.
Shell namespace extension has many shortcomings but if you need files to be only exposed and copied, it can work.
Virtual file system or disk can be created only with help of third-party solutions such as our Callback File System or CallbackDisk. This approach fully emulates the local disk and the filesystem up to possibility to manipulate individual sectors (but this depends on the way used, i.e. whether you emulate the disk or the filesystem).
I have a big collection of folders for projects I'm working on. I've been trying to find a better way to sort them all for a long time and I want to write an app that creates groups based on whatever criteria I say, such as "folders from 2011" or "folders containing a x type of file" etc.
This is fairly straightforward, and wouldn't present much of a problem to code using its own UI in winForms or WPF or something. But I think it would be far better if I could make these folders appear to be part of the filesystem, so other apps (like existing file explorers) can see them.
Is this possible? Would it cause problems I haven't considered? How do I go about doing it if it is possible?
One way I thought of doing it would be to have the app monitor the filesystem and create folder shortcuts every time there's a change, but I'm curious about whether its possible to actually present a fake filesystem to explorer through a 'gateway' folder
EDIT: Ok it's obviously possible since http://www.virtualfolder.net/ can do it, and now that I think of it so can TrueCrypt, although it would be nice if it didn't have to appear as a separate drive. So the question becomes, how do I implement it?
You can create a Shell Namespace Extension that gathers the file information you want and displays it within Windows Explorer any way you wish. You can choose where your extension is located, whether as its own top-level node, a child of another system virtual folder/extension, or as a child of a file system folder.
Writing a SNE is not trivial, but it is a lot easier then writing a lower-level file system driver, and it does not require special driver-oriented compilers. Any compiler that supports developing COM objects will work.
This is accomplished using filesystem drivers or filesystem filter drivers. First let you create a virtual filesystem and mount it to a drive letter and also to a folder on NTFS drive (folder must exist but its contents are "replaced" with a virtual filesystem directory tree). Filesystem filter drivers let you introduce virtual files and folders in existing folders without replacing them.
VirtualFolder uses filesystem driver as it creates a drive letter.
Both types of drivers are written in C and work in kernel-mode. Writing them requires deep knowledge of Windows internals and experience with driver development (since filesystem drivers are one of the most complicated driver types).
We offer several products related to virtual storage. One of them, Callback File System, is a filesystem driver. It calls your user-mode code to perform actual filesystem functions. Another product, CallbackFilter, is an FS filter driver (and it also calls your user-mode code). However, current version of CallbackFilter doesn't let you introduce virtual files and folders (this would be implemented in the next release).
There's also Pismo File Mount product available, they use filter driver techniques. You can check with them if what you need can be accomplished.
From what I gather you are looking for a way to present the results of predefined file queries to appear as though they are located at a specific location in the file system. If that is correct you may want to look into Hard Links and Junctions. There are limits on what you can do with these file system services. However it is really straight forward to implement.
Using the raw Windows programming API from C/C++ and a file handle or a path to a file, folder, link, etc; how can I programmatically decide whether the file (etc) supports ADS (Alternate Data Streams)?
I assume one thing I have to know is whether the file is on an NTFS partition, but then again for all I know it might be possible to mount some kind of Mac or *nix filesystems which support data forks or alternate data streams of some kind, and all such cases might be covered by a single API call or data structure.
Secondly I'm not sure whether every kind of object that can exist on an NTFS partition can have ADSs - such as folders, symlinks, hardlinks, anything else?
What API etc can handle all cases to tell me whether a given file etc has the ability to have ADSs?
(For this question I'm not looking for whether files have ADSs, just whether its possible for files to have them. It could include a file I've just created for instance.)
ADS is a feature of NTFS. You can use GetVolumeInformation() to detect if a given path is on an NTFS file system, and even if that volume supports ADS at all. AFAIK, only a real file can have an ADS attached to it. You can use GetFileAttributes() to detect if a path is a file, directory, symbolic link, etc.
Like any other file, Directories can also host other ADS! Any file object on NTFS can store more than one DATA Stream. The 'visible' one is named, any additional data stream is 'invisible' as far as Explorer is concerned. Actually, at the prompt now one can display ADS using the /R switch when invoking dir.