Saving Images Using Spring Boot and MySQL - spring

I am creating a web app using Spring Boot and JPA. I want to upload many images. I want those images to save in a storage and save the location of the file into database. I am not understanding how to achieve this?

After researching a lot, I have come to a solution. I have used Amazon AWS S3 for storing the media data. Amazon AWS S3 comes with its own library. It gives you rich api to upload the media files and having control over them. I have just uploaded the file into the S3 Bucket and saved the corresponding URL into the MYSQL database. Please comment if you need the corresponding codes.

I have encountered this problem in Django application.
What i did was just used amazon S3 bucket and store all the images. I get the location of the image which is in s3 and store it in mysql DB. when i need the image i just call the image from s3 and used it.

Accept Images as a zipstream(Faster)/binarystream or as a MultipartFile
Load the stream ZipInputStream zis = new ZipInputStream(inputStream);
load the stream to File using Java.IO.File class
Push the file data to file system(Same server machine or any non-sql Database)
save the Location in a DB using Spring data JPA

This is how I DID it.
I store my images as a LONGBLOB in my MySQL database so use columnDefinition = "LONGBLOB". In your HTML code for your upload button, make sure you include the multiple = "true" in your tag if you are using HTML.
In your Controller class make a multipart file that will get the images if you are using binding models and also, you need to have a Set/List of images so you can go through them all.
MultipartFile[] images = bindingModel.getImages();
Set<Image> newImages = new HashSet<>();
I have this neat for loop so I can go through all of my images:
for (MultipartFile multipartFileFOREACH : images)
{
Image newImage = new Image(multipartFileFOREACH.getBytes(), postEntity);//Image is an Entity here, postEntity is an object
newImages.add(newImage);
this.imageRepository.saveAndFlush(newImage);
}
this.articleRepository.saveAndFlush(articleEntity);
So, here I store my bytes into the LONGBLOB in the SQL database
Then for the actual view
Set<Image> imagesBytes = post.getImages();
List<String> images = new ArrayList<>();
for (Image image : imagesBytes)
{
images.add(Base64.getEncoder().encodeToString(image.getLink()));
}
My Image Entity has a link variable of type byte[]
Then I simply add the view for the HTML:
model.addAttribute("images", images);

use s3 bucket for storing image and my sql for storing the address of the image

Using MultipartFile you can upload the file.
Save the File in your server using Files.Copy through BufferedInputStream
Save the saved-location of your file in your database through Spring-Data repository.

There are three options to store your files.
a) in your local file system
b) store in db as bytes
c) use cloud service such as AWS S3 bucket
This is a good tutorial for setting up S3 bucket with SpringBoot applications.
https://medium.com/oril/uploading-files-to-aws-s3-bucket-using-spring-boot-483fcb6f8646

Related

Azure Storage Blob Container Vituval Folder Image Download IN JAVA

I want to download a image from Azure Storage Blob Container have virtual folder, it contain Image, need to download that image in java code.
Asure Storage Stucture:
"Blob-Container" -> "Blob-Folder" -> "Sample.jpg"
Below code is direct download image from Container, i need to download image from fodler inside the container
BlobServiceClient storageClient = new BlobServiceClientBuilder()
.endpoint(endpoint.toString())
.credential(credential)
.buildClient();
BlobContainerClient blobContainer = storageClient.getBlobContainerClient(azureContainer);
BlockBlobClient blobClient = blobContainer.getBlobClient(fileName).getBlockBlobClient();
blobClient.download(response.getOutputStream());
First of all, you need to know that the storage of azure blob storage is actually flat, it actually has no so-called folders at all.
What I mean is that you have to combine the path and file name as the filename passing method. If you list the files in the current container, you will find that they are in a form similar to this:
folder1/folder2/filename.suffix

Firefox web extension - read local file (last downloaded file)

Im creating a web extension and porting from XUL. I used to be able to easily read files with
var dJsm = Components.utils.import("resource://gre/modules/Downloads.jsm").Downloads;
var tJsm = Components.utils.import("resource://gre/modules/Task.jsm").Task;
var fuJsm = Components.utils.import("resource://gre/modules/FileUtils.jsm").FileUtils;
var nsiPromptService = Components.classes["#mozilla.org/embedcomp/prompt-service;1"].getService(Components.interfaces.nsIPromptService);
....
NetUtil.asyncFetch(file, function(inputStream, status) {
if (!Components.isSuccessCode(status)) {
return;
}
var data = NetUtil.readInputStreamToString(inputStream, inputStream.available());
var data = window.btoa(data);
var encoded_data_to_send_via_xmlhttp = encodeURIComponent(data);
...
});
This above will be deprecated.
I can use the downloads.download() to know what was the last download but I can NOT read the file and then get the equivalent for encoded_data_to_send_via_xmlhttp
Also in Firefox 57 onwards, means that I have to try to fake a user action by a button click or something, or upload a file.
Access to file:// URLs or reading files without any explicit user input
isnt there an easy way to read the last downloaded file?
The WebExtension API won't allow extensions to read local files anymore. You could let the extension get CORS privilege and read the content directly from the URL via fetch() or XMLHttpRequest() as blob and store directly to IndexedDB or memory, then encode and send to server. This comes with many restrictions and limitations such as to which origin you can read from and so forth.
Also, this would add potentially many unneeded steps. If the purpose is, as it seem to be in the question at the moment, to share the downloaded file with a server, I would instead suggest that you obtain the last DownloadItem object, extract the URL (.url) from that object and send the URL back to server.
This way the server can load directly from that URL (and encode it on server if needed). The network load will be about the same (a little less actually since there is no Base64 encoding involved which adds 33% to the size), and much less load on the client. The server would read the data as a binary/byte data stream; about the same as if the data was sent directly from the extension.
To obtain the last downloaded file you would do the following from a privileged script:
browser.downloads.search({
limit: 1,
orderBy: ["-startTime"]
})
.then(getLastDownload);
function getLastDownload(downloads) {
if (downloads.length) {
var url = downloads[0].url;
// ... send url to the server and let server fetch the data from it directly
}
}
According to this support mozilla question.
(2) Local file security
Firefox limits access from pages on web servers to pages on local disk or UNC paths. [...]).
Which solution ?
Use local-filesystem-links firefox addon (not tested)
and/or
run a small local webserver on client side, supposing server was run with sufficient privileges, you may finally access any local content via http:// (but still cannot with file:///)

Image doesn’t load consistently from Azure blob storage

I am having an erratic problem using Azure blob storage where my images do not load consistently. The problem is that sometimes when I load a web page, the browser will not show the image, but if I refresh it will load correctly.
When the image doesn't load, the browser shows the default image placeholder. Here is an example:
If I check the hyperlink for the image placeholder, I find that it is the same as the when the image loads successfully, except the Shared Access Signature is different.
Sometimes the same image will fail to load for one link but load successfully for another link even in the same page and same page load. The only difference in the URL is the Shared Access Signature.
Here is my code to build the URL with the shared signature
// Get reference to blob (file) that is to be downloaded
blob = blobContainer.GetBlobReference(blobURL.ToString());
// Get shared access signature to download file from azure blob (valid upto "active duration" minutes) from now
signature = blob.GetSharedAccessSignature(new SharedAccessPolicy()
{
SharedAccessStartTime = null,
SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(60),
Permissions = SharedAccessPermissions.Read
});
// Append signature query string to blob / file that is to be downloaded
downloadURL = string.Format("{0}{1}", blob.Uri.AbsoluteUri, signature);
This is the final HTML image link on the web page, i.e. if I show source on the web page in the browser:
<img alt="Profile Picture" src="https://mystorageaccount2.blob.core.windows.net/abcdefg1-hi23-40b5-86de-a20b568f5626/1601/1234d664d1b74ce1aebf4403e5b74af7.jpg?se=2015-10-31T11%3A38%3A39Z&sr=b&sp=r&sig=SaiUToJg%5Ab3zcdef8EeOq84urHf6HQqS%2BAFt1dEQMNI%3D">
Has anyone else seen this problem? Any recommendations on what I might be doing wrong?
I suspect this could be related to the expiry period which you have set on your image blob's shared access signature (SAS). Is there any good reason where you need to set the SAS to 1 minute when you have set it's permission to read-only?

Is it possible to upload image or file to SkyDrive fom Metro Style App?

Is it possible to upload image or file to SkyDrive fom Metro Style App?
I have already found how to browse the file from SkyDrive. But I haven't found regarding uploading file to SkyDrive. If you reply me, it will be very thankful..
I don't think the file picker method works unless the user has the desktop app installed.
You should use a Sharing contract. If you add a data file (Storage Item) to share, then SkyDrive will be listed as a share target and the user gets a UI where they can choose where in their SkyDrive they want to save. This is how I implemented it in my app.
For more info...
http://msdn.microsoft.com/en-us/library/windows/apps/hh771179.aspx
You can use FileSavePicker to save files. This will of course give the user a chance to select where he wants to save to local documents folder or sky drive. The user is in control.
FileSavePicker savePicker = new FileSavePicker();
savePicker.SuggestedStartLocation = PickerLocationId.DocumentsLibrary;
savePicker.DefaultFileExtension = ".YourExtension";
savePicker.SuggestedFileName = "SampleFileName";
savePicker.FileTypeChoices[".YourExtension"] = new List<string>() { ".YourExtension"};
StorageFile file = await savePicker.PickSaveFileAsync();
if (file != null)
{
await FileIO.WriteTextAsync(file, "A bunch of text to save to the file");
}
Please note that in the sample code I am creating the content of the file in code. If you want the user to select an existing file from the computer then you will have to first use FileOpenPicker, get the file and then use FileSavePicker to save the contents of the selected file to the SkyDrive
Assuming that you are using XAML/JavaScript, the suggested solution is to use FilePicker.
The following link may help you.
http://msdn.microsoft.com/en-us/library/windows/apps/jj150595.aspx
Thanks Mamta Dalal and Dangling Neuron, but there is problem. But it looks like I can't use FileSavePicker. I have to upload file(documnet, photo) not only text file. I have to copy from one path to another. If I use FileSavePicker, I have to write every file content (text, png, pdf, etc) and can't copy. Currently I am using FolderPicker. But unfortunately, FolderPicker doesn't support SkyDrive.My Code is As follow:
>FolderPicker saveFolder = new FolderPicker();
>saveFolder.ViewMode = PickerViewMode.Thumbnail;
>saveFolder.SuggestedStartLocation = PickerLocationId.Desktop;
>saveFolder.FileTypeFilter.Add("*");
>StorageFolder storagefolderSave = await saveFolder.PickSingleFolderAsync();
>StorageFile storagefileSave = [Selected storagefile with file picker];
>await storagefileSave.CopyAsync(storagefolderSave,storagefileSave.Name,NameCollisionOption.ReplaceExisting);
It will be greate that if FolderPicker supports SkyDrive or can copy file using FileSavePicker.

How do I upload some file into Azure blob storage without writing my own program?

I created an Azure Storage account. I have a 400 megabytes .zip file that I want to put into blob storage for later use.
How can I do that without writing code? Is there some interface for that?
Free tools:
Visual Studio 2010 -- install Azure tools and you can find the blobs in the Server Explorer
Cloud Berry Lab's CloudBerry Explorer for Azure Blob Storage
ClumpsyLeaf CloudXplorer
Azure Storage Explorer from CodePlex (try version 4 beta)
There was an old program called Azure Blob Explorer or something that no longer works with the new Azure SDK.
Out of these, I personally like CloudBerry Explorer the best.
The easiest way is to use Azure Storage PowerShell. It provided many commands to manage your storage container/blob/table/queue.
For your mentioned case, you could use Set-AzureStorageBlobContent which could upload a local file into azure storage as a block blob or page blob.
Set-AzureStorageBlobContent -Container containerName -File .\filename -Blob blobname
For details, please refer to http://msdn.microsoft.com/en-us/library/dn408487.aspx.
If you're looking for a tool to do so, may I suggest that you take a look at our tool Cloud Storage Studio (http://www.cerebrata.com/Products/CloudStorageStudio). It's a commercial tool for managing Windows Azure Storage and Hosted Service. You can also find a comprehensive list of Windows Azure Storage Management tools here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/04/17/windows-azure-storage-explorers.aspx
Hope this helps.
The StorageClient has this built into it. No need to write really anything:
var account = new CloudStorageAccount(creds, false);
var client = account.CreateCloudBlobClient();
var blob = client.GetBlobReference("/somecontainer/hugefile.zip");
//1MB seems to be a pretty good all purpose size
client.WriteBlockSizeInBytes = 1024;
//this sets # of parallel uploads for blocks
client.ParallelOperationThreadCount = 4; //normally set to one per CPU core
//this will break blobs up automatically after this size
client.SingleBlobUploadThresholdInBytes = 4096;
blob.UploadFile("somehugefile.zip");
I use Cyberduck to manage my blob storage.
It is free and very easy to use. It works with other cloud storage solutions as well.
I recently found this one as well: CloudXplorer
Hope it helps.
There is a new OpenSource tool provided by Microsoft :
Project Deco - Crossplatform Microsoft Azure Storage Account Explorer.
Please, check those links:
Download binaries: http://storageexplorer.com/
Source Code: https://github.com/Azure/deco
You can use Cloud Combine for reliable and quick file upload to Azure blob storage.
A simple batch file using Microsoft's AzCopy utility will do the trick. You can drag-and-drop your files on the following batch file to upload into your blob storage container:
upload.bat
#ECHO OFF
SET BLOB_URL=https://<<<account name>>>.blob.core.windows.net/<<<container name>>>
SET BLOB_KEY=<<<your access key>>>
:AGAIN
IF "%~1" == "" GOTO DONE
AzCopy /Source:"%~d1%~p1" /Dest:%BLOB_URL% /DestKey:%BLOB_KEY% /Pattern:"%~n1%~x1" /destType:blob
SHIFT
GOTO AGAIN
:DONE
PAUSE
Note that the above technique only uploads one or more files individually (since the Pattern flag is specified) instead of uploading an entire directory.
You can upload files to Azure Storage Account Blob using Command Prompt.
Install Microsoft Azure Storage tools.
And then Upload it to your account blob will CLI command:
AzCopy /Source:"filepath" /Dest:bloburl /DestKey:accesskey /destType:blob
Hope it Helps.. :)
You can upload large files directly to the Azure Blob Storage directly using the HTTP PUT verb, the biggest file I have tried with the code below is 4,6 Gb. You can do this in C# like this:
// write up to ChunkSize of data to the web request
void WriteToStreamCallback(IAsyncResult asynchronousResult)
{
var webRequest = (HttpWebRequest)asynchronousResult.AsyncState;
var requestStream = webRequest.EndGetRequestStream(asynchronousResult);
var buffer = new Byte[4096];
int bytesRead;
var tempTotal = 0;
File.FileStream.Position = DataSent;
while ((bytesRead = File.FileStream.Read(buffer, 0, buffer.Length)) != 0
&& tempTotal + bytesRead < CHUNK_SIZE
&& !File.IsDeleted
&& File.State != Constants.FileStates.Error)
{
requestStream.Write(buffer, 0, bytesRead);
requestStream.Flush();
DataSent += bytesRead;
tempTotal += bytesRead;
File.UiDispatcher.BeginInvoke(OnProgressChanged);
}
requestStream.Close();
if (!AbortRequested) webRequest.BeginGetResponse(ReadHttpResponseCallback, webRequest);
}
void StartUpload()
{
var uriBuilder = new UriBuilder(UploadUrl);
if (UseBlocks)
{
// encode the block name and add it to the query string
CurrentBlockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(Guid.NewGuid().ToString()));
uriBuilder.Query = uriBuilder.Query.TrimStart('?') + string.Format("&comp=block&blockid={0}", CurrentBlockId);
}
// with or without using blocks, we'll make a PUT request with the data
var webRequest = (HttpWebRequest)WebRequestCreator.ClientHttp.Create(uriBuilder.Uri);
webRequest.Method = "PUT";
webRequest.BeginGetRequestStream(WriteToStreamCallback, webRequest);
}
The UploadUrl is generated by Azure itself and contains a Shared Access Signature, this SAS URL says where the blob is to be uploaded to and how long time the security access (write access in your case) is given. You can generate a SAS URL like this:
readonly CloudBlobClient BlobClient;
readonly CloudBlobContainer BlobContainer;
public UploadService()
{
// Setup the connection to Windows Azure Storage
var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
BlobClient = storageAccount.CreateCloudBlobClient();
// Get and create the container
BlobContainer = BlobClient.GetContainerReference("publicfiles");
}
string JsonSerializeData(string url)
{
var serializer = new DataContractJsonSerializer(url.GetType());
var memoryStream = new MemoryStream();
serializer.WriteObject(memoryStream, url);
return Encoding.Default.GetString(memoryStream.ToArray());
}
public string GetUploadUrl()
{
var sasWithIdentifier = BlobContainer.GetSharedAccessSignature(new SharedAccessPolicy
{
Permissions = SharedAccessPermissions.Write,
SharedAccessExpiryTime =
DateTime.UtcNow.AddMinutes(60)
});
return JsonSerializeData(BlobContainer.Uri.AbsoluteUri + "/" + Guid.NewGuid() + sasWithIdentifier);
}
I also have a thread on the subject where you can find more information here How to upload huge files to the Azure blob from a web page
The new Azure Portal has an 'Editor' menu option in preview when in the container view. Allows you to upload a file directly to the container from the Portal UI
I've used all the tools mentioned in post, and all work moderately well with block blobs. My favorite however is BlobTransferUtility
By default BlobTransferUtility only does block blobs. However changing just 2 lines of code and you can upload page blobs as well. If you, like me, need to upload a virtual machine image it needs to be a page blob.
(for the difference please see this MSDN article.)
To upload page blobs just change lines 53 and 62 of BlobTransferHelper.cs from
new Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob
to
new Microsoft.WindowsAzure.Storage.Blob.CloudPageBlob
The only other thing to know about this app is to uncheck HELP when you first run the program to see the actual UI.
Check out this post Uploading to Azure Storage where it is explained how to easily upload any file via PowerShell to Azure Blob Storage.
You can use Azcopy tool to upload the required files to the azure default storage is block blob u can change pattern according to your requirement
Syntax
AzCopy /Source : /Destination /s
Try the Blob Service API
http://msdn.microsoft.com/en-us/library/dd135733.aspx
However, 400mb is a large file and I am not sure a single API call will deal with something of this size, you may need to split it and reconstruct using custom code.

Resources