I’m designing a custom file format. It will be either a monolith file or a folder with smaller files. It’s a rather large file in total and there is no need to load everything into memory at once. It would make it also slower than necessary. One of the file(s) may or may not be database file. Running SQL queries would be useful.
The user can have many such files. The user might want to share files with others even if it takes some time to up/download it.
Conceptually I run into issues with shared network folders, Dropbox, iCloud, etc. Such services can lead to sync issues if the file is not loaded entirely in memory or the database file can get corrupted.
One solution is to prohibit storing the file on such services. Either by using a user/library folder or forcing the user to pick a local folder.
Using a folder in library means recreating a file navigation system like Finder. It limits the choice of the user as well in where the files end up. Limiting the location to a local folder seems the better choice.
Is there a way to programmatically detect if a folder is local?
Related
I'm creating a macOS FileProviderExtension for the remote Document Storage System (kind of like GoogleDrive), where it is possible to share a single document with multiple folders.
For example, Document1.pdf can simultaneously exist in Folder A and Folder B because it's shared with both folders. In my FileProvider extension, this would mean that file should be accessible in both folders:
Folder A/Document1.pdf
Folder B/Document1.pdf
But the file provider extension will treat those as two completely separate files. I.e., if you download one of them, and then try to open the other one, it will redownload the other one, effectively doubling the used space on user's disk and consuming network connection.
I'm looking for a way to tell the FileProviderItem what is the backing data for the given file, and thus solve problems such as:
If user downloads a file in one location, ideally I would tell the FileProvider extension that the same document in all the other locations is also now downloaded (cloud icon should disappear from all files).
Some approaches I considered:
I thought of using symbolic links as part of solution, but I don't really think that's possible
When user tries to open non-downloaded file, fetchContents(for itemIdentifier) callback is invoked. Once file is downloaded, I would ideally now notify all the other files of the same document that they are downloaded, i.e. by updating the isDownloaded property in NSFileProviderItem, but that doesn't seem to work. Also, even if I do that, I still can't say to file, what his backing data file should be.
By turning off the Sandbox capability, I guess I could, when user tries to download/open the file which has already been downloaded in other location, immediately report that file has been downloaded and provide the copy of already downloaded file as data for the requested file, but there are two drawbacks here:
3.1. I would have to turn off the Sandbox capability because I want to access the file in FileProvider path directly
3.2 System would still use disk space for each file. So, if I have same document in multiple folders, extension would keep all those copies in the system, without the option to tell it that for all those files, there is same backing data file somewhere in extension's Container.
Is it possible to append/remove a ressource file to a binary at execution time?
I have an application written with go, which saves/searches data from a database file, and i would like this database file to be embedded to the binary, and updated by the application itself.
This way the application would be self contained with its database.
Modifying the executable, this is generally a very bad idea.
Several issues pop right into my head, such as:
Does the current user have sufficient permissions?
Is the file locked during execution?
What about multiple running instances of the application?
Even if you manage to do just that, think of what anti-virus and firewall applications will say to it: most when they detect the change will flag the executable and/or contain it, or deny running it, or some may even delete it. Rightfully, as this is what many viruses do: modify existing executables.
Also virus scanner databases maintain reports where files (their contents) are identified based on the hash of their content. Modifying the executable will naturally change the file content hash, thus render the file unknown / suspicious to these databases.
As mentioned, just write / cache data in separate file(s), preferably in user's home folder or in the application folder (next to the executable, optionally in sub-folders). Or make the cache file / folder a changeable option (command line flags).
Technically, this is possible, but this is a bad idea. Your application could be run by users not having write permissions to your binary.
If you're talking about a portable app, your best option might be using a file in the same directory the binary is located, otherwise - use the user's home directory according to the conventions of the OS you're running on. You can use the os/user package to find the home directory.
How do I decide which of my application's files go in Program Files (FOLDERID_ProgramFilesX64) and which go in ProgramData? (FOLDERID_ProgramData)? I don't understand what the reason is for splitting up my application's fixed files into these two categories or how I should decide which file goes in what.
For example - image files which my application displays, are they "program" or "data"?
Is there any problem with just putting everything under one or the other?
The application is installed for All Users and has no user-specific configuration files or data.
Program Files is for executables and other static files that came as part of the installation. ProgramData is for user-agnostic data generated during execution such as shared cache, shared databases, shared settings, shared preferences, etc. User-specific data goes in the AppData folder. Note that these are for non-user-visible data. User-visible data belongs in the documents folder (or music, video, custom sibling folder, etc.).
Please see Special Folders and Custom Folders for a detailed explanation. Note that the terminology used varies slightly between the name used in the documentation here, the name of the folder, and the name used by various enumerations used to get these paths from the system.
Hi I am trying to find out what is the best location to save a cache file.
I have an Windows form application that updates user's data from the server using a custom tool.
I want to write the timestamp of the latest updates done on user's machine in the cache file.
Where is the best location for keeping this file:
1. in application directory (c:\program files..)
2. in a temp location e.g. Users profile folder or c:\windows\temp
3. in any location (e.g. c:\dataupdates) where user has full access to read/write to.
Not in the application directory. That much is clear. :) The application directory shouldn't even be writable by the program (or actually by the user account that runs the program). Although some applications still use this location, it has actually been deprecated since Windows 95, I believe, and it has become a real pain since the more rigid UAC applied in Windows Vista and 7.
So the most obvious options are:
The temp folder, which is for temporary files. Note however, that you will need to clean those files up. Temp folder is not automatically cleared by default, so adding new files all the time will consume increasingly much space on the hard drive. On the other hand, some users do clear their temp folders, or may have scripts installed that do that for them, so you cannot trust such files to remain. Also, this is not always C:\Temp of whatever. You'll have to ask Windows what the location is.
You can pick 'any' location. Note that you cannot write anywhere. You cannot even expect the C drive to exist. If you choose this, then you have to make it a configurable setting.
The %app data% directory, my personal favorite, which is a special directory for applications to store their data in. The advantage is, that you can ask Windows for this location, and you can make up a relative path based on that directory, so you don't really have to make it an application setting. For more info on how to get it, see also this question: C# getting the path of %AppData%
I would definitely choose the App Data path for this purpose.
I'm working on an application that generates a set of bitmaps and then loads them into a form for a user to pick from.
The bitmaps are generated from a small vector library which the user can add to. The code now creates the files and then deletes them immediatelyafter use, only to have to generate them again (making the UI take seconds to load) next time the user opens the UI.
So what I'm wondering is, is it okay to leave my bitmaps in the user temp folder "forever", and regenerate them if they are not in the folder? I can't expect to be able to store the images in the application directory, due to possible permission issues, and like I said, I can't prepopulate the files since the user can add more.
Ideally you should generate any temporary data into the RAM rather to the file system.
It is acceptable to depend on temporary files if you can make sure that your application is storing only a limited amount of such files per user. Any temporary files can be left behind on unexpected crashes/power offs no matter what your code does. You therefore need to implement a mechanism that will delete any stale files created by the same application in a previous session - presumably during its next start up.
Assuming such a safety mechanism, intentionally leaving behind temporary files when the application exits sounds like a non-standard but reasonable "cache".
Caveat: the next version of your application may need a slightly different file format, and should detect, delete and regenerate any files in a mismatched format based on some simple versioning scheme to avoid cross-build dependences.