Is there a Go implementation of fs.ReadDir for Google Cloud Storage? - go

I am creating a web application in Go.
I have modified my working code so that it can read and write files on both a local filesystem and a bucket of Google Cloud Storage based on a flag.
Basically I included a small package in the middle, and I implemented my-own-pkg.readFile or my-own-pkg.WriteFile and so on...
I have replaced all calls in my code where I read or save files from the local filesystem with calls to my methods.
Finally these methods include a simple switch case that runs the standard code to read/write locally or the code to read/wrote from/to a gcp bucket.
My current problem
In some parts I need to perform a ReadDir to get the list of DirEntries and then cycle though them. I do not want to change my code except for replacing os.readDir with my-own-pkg.ReadDir.
So far I understand that there is not a native function in the gcp module. So I suppose (but here I need your help because I am just guessing) that I would need an implementation of fs.FS for the gcp. It being a new feature of go 1.6 I guess it's too early to find one.
So I am trying to create simply a my-own-pkg.ReadDir(folderpath) function that does the following:
case "local": { }
case "gcp": {
<Use gcp code sample to list objects in my bucket with Query.Prefix = folderpath and
Query.Delimiter="/"
Then create a slice of my-own-pkg.DirEntry (because fs.DkrEntry is just an interface and so it needs to be implemented... :-( ) and return them.
In order to do so I need to implement also the interface fs.DirEntry (which requires the implementation of interface for FileInfo and maybe something else...)
Question 1) is this the right path to follow to solve my issue or is there a better way?
Question 2) (only) if so, does the gcp method that lists object with a prefix and a delimiter return just files? I can't see a method that returns also the list of prefixes found
(If I have prefix/file1.txt and prefix/a/file2.txt I would like to get both "file1.txt" and "a" as files and prefixes...)
I hope I was enough clear... This time I can't include code because it's incomplete... But in case it helps I can paste what I can.
NOTE: by the way go 1.6 allowed me to solve elegantly a similar issue when dealing with assets either embedded or on the filesystem thanks to the existing implementation of fs.FS and the related ReadDirFS. So good if I could follow the same route 🙂
By the way I am going on studying and experimenting so in case I am successful I will contribute as well :-)

I think your abstraction layer is good but you need to know something on Cloud Storage: The directory doesn't exist.
In fact, all the object are put at the root of the bucket / and the fully qualified name of the object is /path/to/object.file. You can filter on a prefix, that return all the object (i.e. file because directory doesn't exist) with the same path prefix.
It's not a full answer to your question but I'm sure that you can think and redesign the rest of your code with this particularity in mind.

Related

Ruby viewpoint with EWS

I am trying to get started using viewpoint against EWS within Ruby, and it's not making a lot of sense at the moment. I am wondering where I can get some good example code, or some pointers? I am using 1.0.0-beta.
For example: I know the name of the calendar folder I want to use, so I could search for it, but how to access methods in that folder once I find it? What are the appropriate parameters, etc...
Any advice?
If you haven't read it yet I would recommend the README file in the repository. It has a couple of examples that should put you on the right path. Also, the generated API documentation should give you enough to work with.
http://rubydoc.info/github/WinRb/Viewpoint/frames
At a very basic level you can get all of your calendar events with the following code:
calendar = client.get_folder :calendar
events = calendar.items
I hope that gives you a little more to get started with.
Follow-up:
Again, I would point you to the API docs for concrete methods like #items. There are however dynamically added methods depending on the type that you can fetch with obj.ews_methods. In the case of CalendarItem one of those methods is #name so you can call obj.name to get the folder name. The dynamic methods are all backed by a formatted Hash based on the returned SOAP packet. You can see it in its raw format by issuing obj.ews_item
Cheers,
Dan

How can Customizing the IMFByteStream?

. Hello All?
I want to customize IMFByteStream interface, but i'm facing some problems.
Before explains my problems, describes how to create from what i've got.
First, customized IMFByteStream has IMFByteStream's instance that created using MFCreateFile method. Therefore, we need to implement the necessary ones (For example, BeginRead, Read, etc..)
Second, we need to decrypt to the received data. Because the file was encrypted.
As a result, the read sequence was following.
CustomByteStream::BeginRead() -> CustomByteStream::Read() -> IMFByteStream::Read() -> CustomByteStream::Decrypt() -> Passes the decrypted data.
But, I don't know how to pass the data. AsyncResult or AsyncCallback should I use? I don't know how.
Please help me. Thank you.
If you implement IMFByteStream, you have to implement IMFAsyncCallback too.
I can't explain here. But when i will update my project with Mpeg2 source: MFNode
you will see an implementation of IMFByteStream. I use it because original IMFByteStream fails on some video files with no reason. My implementation works well with all files i've tested. (For now my implementation does not handle big file.)
Edit: i checked my code. I don't implement IMFByteStream. i created a class that acts like IMFByteStream. I implement IMFAsyncCallback for BeginRead/EndRead.

distinguish use cases in NSAutosaveElsewhereOperation

I try to add AutoSave support to the Core Data File Wrapper example
Now if i have a new/untitled document writeSafelyToURL is called with the NSAutosaveElsewhereOperation type.
The bad thing is, I get this type in both typical use cases
- new file: which store a complete new document by creating the file wrapper and the persistent store file
- save diff: where the file wrapper already exists and only an update is required.
Does somebody else already handled this topic or did somebody already migrated this?
The original sample use the originalStoreURL to distinguish those two use cases, which solution worked best for you?
Thanks

Loading a file from filesystem in SPRING

I have been strugling with this for a long while.
I am using an outer API and I need to pass file's path directly. I cannot modify it.
I looks like: functionmethod(String path);
So i cannot use Resource because I need to pass just path.
Is it possible in SPRING?
Maybe you could use:
(new File("")).getAbsolutePath() that gives you the current path (application).
or (I think this one will fit better for you)
getResource("fileName").getFile()

Implementing resource list in WP7

I am porting a JAVA game to WP7. We have lots of images in our game and for loading them in JAVA use a function something like this "Resources.getImage(IMG_BULLETS);" where IMG_BULLETS is ID(an int) of the image.
But in WP7 we have to pass the path(a string) of image in order to load that.
Now my question is :
How to achieve a int-String mapping? so that I dont have to manually change the Id into path.
One possible solution comes in my mind is to have a .txt file having image path and its Id and parse that. But I am sure their is much better solution for this.
Note : we also have a multi-level folder structure for images and other files.
If you really want to refer to resources by int, you are going to have to map them some how to their path. As far as I know, there is no way to crawl your assets, and even if you could, i some how doubt you will get the same index order and you would have to redo your enum.
There are going to be a few hurdles you hit with trying to port and code changes. Another one, (one you will still have even with a txt file work around) will be that the contentManager.Load takes a type, eg, contentManager.Load<Texture2D>('path') returning a Texture2D.
One option would be to create your own singleton 'Resources' class that has get methods that take int to get the appropriate asset, however you will still need a mapping via xml or txt file of somekind. However this will require all your assets being loaded from the start which are more problems of their own (90mb memory limit and really long load time).
My advice would be look at the game development section of the App Hub and have a look at the game state example to help you see how you might be able to structure your game to work well with XNA.

Resources