I am working on a GAE/J based project. One of the requirements is to let the users upload files (doc,ppt,pdf,xls etc).
What options do I have for storing files besides http://code.google.com/p/gae-filestore/
Is it possible to make these files "searchable"?
Blobstore service. It stores files up to 2GB, and its API was intended for user uploaded files. See: http://code.google.com/appengine/docs/java/blobstore/overview.html It isn't indexed, but I believe that some people have been able use Map/Reduce with it: http://ikaisays.com/2010/08/11/using-the-app-engine-mapper-for-bulk-data-import/
Datastore. You can store up to 1 MB as a "blob property".
Related
My goal is to download a large zip file (15 GB) and extract it to Google Cloud using Laravel Storage (https://laravel.com/docs/8.x/filesystem) and https://github.com/spatie/laravel-google-cloud-storage.
My "wish" is to sort of stream the file to Cloud Storage, so I do not need to store the file locally on my server (because it is running in multiple instances, and I want to have the disk size as small as possible).
Currently, there does not seem to be a way to do this without having to save the zip file on the server. Which is not ideal in my situation.
Another idea is to use a Google Cloud Function (eg with Python) to download, extract and store the file. However, it seems like Google Cloud Functions are limited to a max timeout of 9 mins (540 seconds). I don't think that will be enough time to download and extract 15GB...
Any ideas on how to approach this?
You should be able to use streams for uploading big files. Here’s the example code to achieve it:
$disk = Storage::disk('gcs');
$disk->put($destFile, fopen($sourceZipFile, 'r+'));
I'm creating a web application using flask that takes 3 input from the user: name, picture, grades.
I want to store these information temporary depending on the user's session.
and as a beginner I read that sessions are not for storing files, what other secure way you recommend me to use?
I would recommend to write the files to disk.
If this is really temporary, e.g. you have a two-step-sign-up-form, you could write the files to temporary files or into a temporary directory.
Please see the excellent documentation at https://docs.python.org/3/library/tempfile.html
Maybe this should not be this temporary? It sounds like a user picture is something more permanent.
Then I would recommend e.g. to create a directory for each user and store the files there.
This is done with standard Python io, e.g with the open function.
More info about reading and writing files also can be found in the official Python documentation:
https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files
I saved a file to storage using:
$request->file('avatar')->store('avatars');
Which saved it to:
storage/app/avatars/avatar.png
How can I get the path to this file/folder (not the URL)? What is the correct way to do this using Laravel's Filesystem?
There is no correct way to do this; because it should not be done. The Storage is an opaque system to talk to different storage systems; as such there is no api to get the backing file path. As an example, that wouldn't work with Amazon S3. The only path your application knows about is the string you send to the Storage facade to work with the file, there are no guarantees that this string is used to generate the filename when the storage system stores the file.
There are some hacks you can use that works for the local disk, but those are not available for the Storage system in general. Using these solutions means that you'll limit yourself to only use the local disk; this will cause you troubles when you need to scale out and add another server. You'll then have two servers with two separate local disks, with separate content.
The correct way to work with the files, that will work for all configurations, is to get the file content (Storage::get), do the modifications (including storing them in a temporary file) and then write back the new file content (Storage::set).
If you're really sure that you will only ever use the local filesystem, use the File facade instead of the Storage facade. I'm unable to find any documentation for this, only the interface it exposes.
Reference: https://github.com/laravel/framework/issues/13610
Try this
storage_path('app/avatars/avatar.png');
you can only get the storage folder path from laravel function, you can give nested folder name after it, it will bind the base url as well
storage_path(folder1/folder2/.../file.png);
We are new to Windows azure and have used Windows azure storage for blob objects while developing sitefinity application but the blob files which are uploaded to this storage via publishing to azure from Visual Studio uploads files with only the file names and do not maintain the prefix folder name and slash. Hence we have to rename all files manually on the windows azure management portal and put the folder name and slash in the beginning of each file name so that the page which is accessing these images can show the images properly otherwise the images are not shown due to incorrect path.
Though in sitefinity admin panel , when we upload these images/blob files in those pages , we upload them inside a folder and we have configured to leverage sitefinity to use azure storage instead of database.
Please check the file attached to see the screenshot.
Please help me to solve this.
A few things I would like to mention first:
Windows Azure does not support rename functionality. Rename blob functionality = copy blob followed by delete blob.
Copy blob operation is asynchronous so you must wait for copy operation to finish before deleting the blob.
Blob storage does not support folder hierarchy natively. As you may have already discovered, you create an illusion of a folder by prepending a blob name (say logo.png) with the name of folder you want (say images) and separate them with slash (/) so your blob name becomes images/logo.png.
Now coming to your problem. Needless to say that manually renaming the blobs would be a cumbersome exercise. I would recommend using a storage management tool to do that. One such example would be Azure Management Studio from Cerebrata. If you use that tool, essentially what you can do is create an empty folder in the container and then move the files into that folder. That to me would be the fastest way to achieve your objective.
If you wish to write some code to do that, here are the steps you will take:
First you will list all blobs in a blob container.
Next you will loop over this list.
For each blob (let's call it source blob), you would get its name and prepend the folder name that you want and create an instance of a CloudBlockBlob object.
Next you would initiate a copy blob operation on that blob using StartCopyFromBlob on this new blob where source is your source blob.
You would need to wait for the copy operation to finish. Once the copy operation is finished, you can safely delete the source blob.
P.S. I would have written some code but unfortunately I'm stuck with something else. I might write something later on (but please don't hold your breath for that :)).
What's the standard path on MacOS X for storing application data that is to be shared by different users? I am not talking about temporary data, but data which is used by one particular program on a regular basis and belongs to no particular user. For example a game highscore table.
Thanks,
Adrian
I believe you're talking about Support files - a file that supports the application but is not required to run (your highscore table for example).
These files should be put in ~/Library/Application Support/YourApp or /Library/Application Support/YourApp for shared users.
The Library Directory Stores App-Specific Files
To get the directory, you can use the function "NSSearchPathForDirectoriesInDomains", with the directory parameter being "NSApplicationSupportDirectory", and the domainMask parameter being "NSLocalDomainMask".
(NSApplicationSupportDirectory is the "Location of application support files", while NSLocalDomainMask means "Local to the current machine—the place to install items available to everyone on this machine.")
Some applications put files into the /Users/Shared-directory. I know it's the standard way to share files between users, but I'm not 100% sure it's thought for application data storage.
The there's the /Library*-folder which is thought for systemwide common data, similiar to the /Users/Usernames/Library.
But you certainly shouldn't write data to the Application.app-directory. Users without admin rights won't even have the right to write to these directories.
* = Or /System/Library. Need to verify.