I've a problem with a document-based project in Cocoa. I've searched for a while but I didn't find anything which seems to resemble my goal. What I want to do is a (computation intensive) simulation program which generates a lot of data (probably in the order of GBs) and store them to the disk for a future visualization (so I cannot write/read the files all at once).
I created a document-based project (I don't know if it is the way to go...) with the idea to save all the data in many binary-files within a package, so the user can see it as a single file. I have already tried that part and I was able to save the document with NSFileWrapper. But the simulation-files are generated as the simulation is running. And here comes the problem.
There is a way to force the user to save the document and retrieve the path so I can put there all the files generated? Or it's best to save the simulation-files in a temporary location and then save the document periodically so that it saves all the files ready for saving? Or what can I do? It's not clear to me the usage of the nsdocument architecture in this case and what it's a good way to achieve my goal.
The document has also another couple of files in which there are the simulation parameters and the initial state, so I can resume the simulation at a later time.
Related
I'd like to store some of my data in relative big files (a few GBs per file). I'd like to use event sourcing and save events related to these files, e.g. FileCreated: title, description, timestamp, author, personal, encryptionkey, etc. After a while some of the files won't be needed any longer, and they take up a lot of space. So in order to free up space, I need to delete them. Doing so is problematic, because I will have the history in the event storage, but not the file in the filesystem. Is there any way to keep integrity and somehow delete both? Or is there a best practice for this problem?
Since I did not get an answer, I try to answer this myself.
It is possible to remove an event from the history, you need to create a new event storage and filter the events for the same aggregate id you want to get rid of. After you are done, you can switch to the new event storage and remove the old one. Probably you need to replay projections as well. So it is very similar to a whole migration, it takes a lot of time. In the current case it is not problem if I need to do this only once every year or so. Another problem with storing this data in the event storage that either I stream it from there or I need to duplicate it in order to serve it. The latter one is not always a good solution, because sometimes it takes too much time to copy and in order to save the data you need to stream it anyways, otherwise you will be out of memory very fast. So the event storage should support streaming attachments.
Another solution to keep the relative big data in the files and display something like 404 not found, or file was removed because this and that. I see this frequently. In this case it is ok to keep the event in the storage and for example you can add a ContentRemoved event, where you can select the cause. Another option to hide the removed file, so it won't be listed by the app, this is usual I guess too. This solution has drawbacks too. Migration is more complex with this approach, because you need to move both the event storage and the files. If you remove a file by accident, you cannot undo it later unless you have the file in the backup. This can be fixed by delaying the actual file removal with a few days, so you can undo it if you change your mind. Another option to make a trash and files will be deleted only by emptying the trash.
I think both solutions are worth to consider and probably it depends on the actual project which one is better suited.
The documentation for NSDocument states:
Subclasses can override this method and use it to restore any
information that would be needed to restore the document’s window to
its current state. For example, you could use this method to record
references to the data currently managed by the document and displayed
by the window. (Do not store the actual data itself. Store only
references to the data so that you can load it later from disk.) You
must store enough data to reconfigure the document and its window to
their current state during a subsequent launch of the app.
What does "Do not store the actual data itself." actually mean? Is this a hard and fast rule? Or is it more of a guideline?
In particular, I'm wondering about the case of documents with unsaved changes in them. Is it "permissible" to store the unsaved changes (which may be everything if this is a new document)? Or, do I need to save the data off in a file somewhere... and if so, where is the preferred location?
I don't want to restore a bunch of identical (blank) documents if I had multiple unsaved new documents when the application was shut down.
Thanks for any hints on the proper way to handle this.
Never mind. It hit me in the shower this morning (where I make most of my tech breakthroughs).
I am pretty sure now that the key is to get autosaving working with my application.
In a Mac OS X app (Cocoa), I'm copying some images from my app to others using a NSDraggingSession. The NSDraggingItem makes use of an object that implements the protocol NSPasteboardItemDataProvider, to provide the data when the user drops it.
As I'm dealing with images, the types involved are: NSPasteboardTypePNG, kPasteboardTypeFileURLPromise, kUTTypeFileURL, com.adobe.photoshop-image and public.svg-image. These images are in a remote location, so before I can provide them to the pasteboard, I have to download them from the Internet.
I implement the method - pasteboard(pasteboard:item:provideDataForType:) doing something like this:
If the type requested is kPasteboardTypeFileURLPromise, I get the paste location and build and set in the pasteboard the URL string with the location where the file is supposed to be written in the future.
If the type requested is kUTTypeFileURL, I download the file, specify a temporal location and write the downloaded file to that location. Then, I set in the pasteboard the URL string of the location.
If the type requested is one of the others, I download the file and set the plain NSData in the pasteboard.
All these operations are performed on the main thread, producing some lags that I want to get rid of.
I've tried to perform these operations on a background thread, and come back to the main thread to set the final data in the pasteboard, but this doesn't work because the method finishes before.
Does anyone know a way to achieve it?
Promises of pasteboard types are usually meant to be an alternative format of data that you already have, where you want to avoid the expense in time and memory of converting before it's necessary. I don't think it's really appropriate to use it to defer downloading any of the data, at all. For one thing, the download could fail when it's ultimately requested. For another, it could take an arbitrarily long time, as you're struggling with now.
So, I think you should download the data in advance. Either keep it in memory or save it to a temporary file. Use promised types, if appropriate, to deliver it in different forms, but have it on hand in advance.
I have almost zero experience coding in Visual Studio, MFC, etc. But I've got several data files that were created in a now-defunct MFC application, which I need to migrate to another format.
Unfortunately there's really no good way, within the application itself, to extract the data (short of copy-pasting hundreds or even thousands of records individually). And viewing the files themselves, i.e. in a Hex Editor, has proven fruitless; even though the raw data stored by the app is text-based, the database files are encoded in some cryptic binary format.
So far I've been able to determine that the app was written using MFC and that it uses the CDocument class (or a simple derivative thereof) to store the files. I understand that CDocument-based data files have something to do with serializing the data, but I'm not sure how to make sense of the encoding.
Does anyone know enough about MFC to explain to me how CDocument actually works?
Does anyone have any ideas on how I might be able to decode these files to extract the text?
I once faced an almost identical scenario. I eventually worked out the code to deserialize the data, but it wasn't easy.
Write a small MFC application to do the work, that way you can leverage the same serialization code that the original app used. The topic of reverse engineering a data format is way too complex to answer here. It's probably not encrypted; more likely compressed.
If you're an experienced programmer you should be able to read the MFC source code, then apply that knowledge to the raw data. Not everything can be heuristically determined just by observing the raw data, but if you have an independent way of determining the actual content, it's certainly possible with sufficient work.
We have a small team running StarTeam. A constant source of frustration and problems is the handling of deleted files in StarTeam. It is obvious that Starteam keeps track of deleted files internally, but it does not seem to be possible to get any information about a file deletion.
So far, my only solution to find the timing of a delete is to perform a manual binary search using the 'compare' views. Is there any better way (the query for 'delete time' never seems to pick up any files).
The Audit tab (just to the right of File, ChangeRequest, etc.) is probably your best bet if you're just looking for who deleted what and when. The Audit tab also provides information about when items and folders were created, shared, or moved, as well as when View labels are attached/detached. Whenever someone has files unexpectedly appear or disappear, I direct them to the Audit tab first.
There is a server-side configuration setting for the length of time the audit data is retained (30 days by default, I believe). Since it is not retained forever, it isn't a good option for historical data. The number of audits can be quite large in active views.
If you're looking for something more than that or older than your audit retention time, go with Bubbafat's suggestion of the SDK and getDeletedTime/getDeletedUserID.
Comparing views (or rolling back a view to see the item again) is the only way I know how to do this in StarTeam without writing code.
If you are willing to write a little code the StarTeam API provides the Item.getDeletedTime and Item.getDeletedUserId methods (I believe these showed up in 2006).