I know Spring has MultipartFile component.
I am wondering if there is any API to unzip files or read zip files to do some processing?
I have a zip file that following a certain format.
photos\
audio\
report.xml
when the user upload it via web, I wish to scan the zip file and do some processing.
Is there a solution for this issue?
I do not know spring have any such type of API,
but you can use other API for ZIP or UNZIP files.
1) http://commons.apache.org/compress/
2) java.util.zip
and also see
What is a good Java library to zip/unzip files?
There are a couple of Java SE APIs for reading ZIP files:
java.util.zip.ZipInputStream - gives you a one-pass reader
java.util.zip.ZipFile - gives you a reader that allows you to read the entries and the files in any order.
You should be able to use one or the other of these, depending on the nature of your processing.
If the processing requires the images to be in actual files, you would have to create the directories and write the files yourself. In this case, it would probably be simpler to use an external command to do the ZIP extraction.
Related
My goal is to download a large zip file (15 GB) and extract it to Google Cloud using Laravel Storage (https://laravel.com/docs/8.x/filesystem) and https://github.com/spatie/laravel-google-cloud-storage.
My "wish" is to sort of stream the file to Cloud Storage, so I do not need to store the file locally on my server (because it is running in multiple instances, and I want to have the disk size as small as possible).
Currently, there does not seem to be a way to do this without having to save the zip file on the server. Which is not ideal in my situation.
Another idea is to use a Google Cloud Function (eg with Python) to download, extract and store the file. However, it seems like Google Cloud Functions are limited to a max timeout of 9 mins (540 seconds). I don't think that will be enough time to download and extract 15GB...
Any ideas on how to approach this?
You should be able to use streams for uploading big files. Here’s the example code to achieve it:
$disk = Storage::disk('gcs');
$disk->put($destFile, fopen($sourceZipFile, 'r+'));
I am new to spring batch. My requirement is, I have a folder say D:\xyzfolder\source which is having 25 flat files. Using spring batch I need to read and implement some business logic and write all 25 files with the same name into a different folder say D:\xyzfolder\destination
Currently, I am using MultiResourceItemReader and reading all the 25 files from the source folder and I am able to write into a single file using FlatFileItemWriter with setResource(outputResource) but my requirement is to write as 25 different files. Please suggest how to achieve the above requirement
For a similar use case this answer https://stackoverflow.com/a/20356050/4767829 suggests using a MultiResourceItemWriter in combination with an ItemWriteListener that dynamically sets the output resource for each item.
I have a third-party executable that takes a directory path as an argument and in turn looks there for a collection of .db files. I have said collection of files stored in a Google Cloud Storage bucket and would like to stream the content of those files into some local named pipes that can be used as input to the executable.
I'm writing an application to perform the above in Go and am using the "cloud.google.com/go/storage" package to work with cloud storage objects.
As a note, I need all pipes/files to be available for reading at the time I run the executable.
What is the best way to go about this? I'm looking to essentially used the named pipe as a proxy of sorts to make remote files look local to this executable. Possible?
I want to make the file to zip . I can make the file to zip file but I don't want to create temporary zip file. Is any way to make the file to zip and to byte[] ?
Thanks
zip files are typically generated with streams anyway, so there's no need to temporarily store them in a file - might as well be in memory or streamed directly to a remote recipient (with only a small memory buffer to avoid a large memory footprint).
Ref Sample helperclass in the accepted answer section How can I generate zip file without saving to the disk with Java?
I want to use Spring Batch to perform the upload of files from my server to e.g. Google Drive.
So the steps are:
get files from a specified folder
upload them to Google Drive
update each entry in my DB corresponding to this file (i.e. update path)
My question is: do I necessarily have to do it with tasklet? If so, do I have to split the job into chunks myself and there will be no restart-on-failure support?