I am new to Azure and I am moving my on-prem table to Azure blob as csv format.
But in the Azure storage explorer, I am seeing the 'content type' column is showing as 'application/octet-stream' instead of 'csv'? . I am wondering why it is not showing as csv file ? .Can you explain, please ?
Regards
When you upload files in your Azure Storage account usually all files are uploaded with application/octet-stream as ContentType. If you just upload .csv files to Azure blob, it doesn't matter. You could change it to text/csv that you would like.
Modify ContentType:
private static CloudBlockBlob TrySetContentType(CloudBlockBlob blob, string contentType)
{
if (blob.Properties.ContentType.ToLower() != contentType)
{
blob.Properties.ContentType = contentType;
return blob;
}
return null;
}
For more details about ContentType property, see here.
In the latest (12.x+) New Azure SDK for .NET, setting content type needs to be done via BlobHttpHeaders object.
BlobClient blob = _container.GetBlobClient(fileName);
var blobHttpHeader = new BlobHttpHeaders();
string extension = Path.GetExtension(blob.Uri.AbsoluteUri);
switch (extension.ToLower())
{
case ".jpg":
case ".jpeg":
blobHttpHeader.ContentType = "image/jpeg";
break;
case ".png":
blobHttpHeader.ContentType = "image/png";
break;
case ".gif":
blobHttpHeader.ContentType = "image/gif";
break;
case ".csv":
blobHttpHeader.ContentType = "text/csv";
break;
default:
break;
}
await using (var fileStream = new MemoryStream(fileBytes))
{
var uploadedBlob = await blob.UploadAsync(fileStream, blobHttpHeader);
}
Related
I have following function which works fine when saving to disk. I am executing the code from an Azure function. Is there anyway to to write to a blob storage instead without saving to disk?
private void ExportDataSet(DataTable ds, string destination)
{
using (var workbook = SpreadsheetDocument.Create(destination, DocumentFormat.OpenXml.SpreadsheetDocumentType.Workbook))
{
var workbookPart = workbook.AddWorkbookPart();
workbook.WorkbookPart.Workbook = new DocumentFormat.OpenXml.Spreadsheet.Workbook();
workbook.WorkbookPart.Workbook.Sheets = new DocumentFormat.OpenXml.Spreadsheet.Sheets();
var sheetPart = workbook.WorkbookPart.AddNewPart<WorksheetPart>();
var sheetData = new DocumentFormat.OpenXml.Spreadsheet.SheetData();
sheetPart.Worksheet = new DocumentFormat.OpenXml.Spreadsheet.Worksheet(sheetData);
DocumentFormat.OpenXml.Spreadsheet.Sheets sheets = workbook.WorkbookPart.Workbook.GetFirstChild<DocumentFormat.OpenXml.Spreadsheet.Sheets>();
string relationshipId = workbook.WorkbookPart.GetIdOfPart(sheetPart);
uint sheetId = 1;
if (sheets.Elements<DocumentFormat.OpenXml.Spreadsheet.Sheet>().Count() > 0)
{
sheetId =
sheets.Elements<DocumentFormat.OpenXml.Spreadsheet.Sheet>().Select(s => s.SheetId.Value).Max() + 1;
}
DocumentFormat.OpenXml.Spreadsheet.Sheet sheet = new DocumentFormat.OpenXml.Spreadsheet.Sheet() { Id = relationshipId, SheetId = sheetId, Name = "Sites" };
sheets.Append(sheet);
DocumentFormat.OpenXml.Spreadsheet.Row headerRow = new DocumentFormat.OpenXml.Spreadsheet.Row();
List<String> columns = new List<string>();
foreach (System.Data.DataColumn column in ds.Columns)
{
columns.Add(column.ColumnName);
DocumentFormat.OpenXml.Spreadsheet.Cell cell = new DocumentFormat.OpenXml.Spreadsheet.Cell();
cell.DataType = DocumentFormat.OpenXml.Spreadsheet.CellValues.String;
cell.CellValue = new DocumentFormat.OpenXml.Spreadsheet.CellValue(column.ColumnName);
headerRow.AppendChild(cell);
}
sheetData.AppendChild(headerRow);
foreach (System.Data.DataRow dsrow in ds.Rows)
{
DocumentFormat.OpenXml.Spreadsheet.Row newRow = new DocumentFormat.OpenXml.Spreadsheet.Row();
foreach (String col in columns)
{
DocumentFormat.OpenXml.Spreadsheet.Cell cell = new DocumentFormat.OpenXml.Spreadsheet.Cell();
cell.DataType = DocumentFormat.OpenXml.Spreadsheet.CellValues.String;
cell.CellValue = new DocumentFormat.OpenXml.Spreadsheet.CellValue(dsrow[col].ToString()); //
newRow.AppendChild(cell);
}
sheetData.AppendChild(newRow);
}
}
}
I would expect you maybe could save to a Stream?
Save to stream is the solution if you don't like to save it to a disk(In azure function, you can save it to a disk in azure function kudu like D:\home etc.).
If you choose to save to stream, just a few changes to your code, like below:
private void ExportDataSet(DataTable ds, MemoryStream memoryStream)
{
using (var workbook = SpreadsheetDocument.Create(memoryStream, DocumentFormat.OpenXml.SpreadsheetDocumentType.Workbook))
{
//your code logic here
}
//here, the code to upload to azure blob storage.
CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("test1");
CloudBlockBlob myblob = cloudBlobContainer.GetBlockBlobReference("myexcel.xlsx");
//upload to blob storage
memoryStream.Position = 0;
myblob.UploadFromStream(memoryStream)
//or you can use Asnyc mehtod like myblob.UploadFromStreamAsync(memoryStream)
}
Note: if you're using the latest azure blob storage sdk Microsoft.Azure.Storage.Blob, version 9.4.0 or later, you can use either UploadFromStreamAsync or UploadFromStream method in azure function v2.
Azure Storage Java SDK CloudBlockBlob.downloadText() operation failing with error: "Blob hash mismatch (integrity check failed), Expected value is xxxx, retrieved xxx".
I'm using azure storage sdk version: azure-storage:8.0.0. Here is the code snippet.
List<BlockEntry> blockList = new ArrayList<BlockEntry>();
File parent = new File(System.getProperty("java.io.tmpdir"));
File temp = new File(parent, "sample-file.txt");
String blockText="hellow world";
if (temp.exists()) {
temp.delete();
}
try {
temp.createNewFile();
CloudBlockBlob blob = container.getBlockBlobReference(temp.getName());
blob.uploadFromFile(temp.getAbsolutePath());
//Add Block to the created blob
String blockId1 = Base64.encode("B1".getBytes());
InputStream inputStream1 = new ByteArrayInputStream(blockText.getBytes());
long len = -1;
BlockEntry block1 = new BlockEntry(blockId1);
blockList.add(block1);
blockBlob.uploadBlock(blockId1, inputStream1, len);
blockBlob.commitBlockList(blockList);
//Print the blob content
System.out.println("Printing the block content" + blob.downloadText());
} catch (IOException ex) {
ex.printStackTrace();
}
I am trying to upload a .mp4 file, selected from the user's iOS or Android device, to my Azure Media Services account.
This code works for small files ( less than ~95MB):
public static async Task<string> UploadBlob(string blobContainerSasUri, string blobName, byte[] blobContent, string path)
{
string responseString;
int contentLength = blobContent.Length;
string queryString = (new Uri(blobContainerSasUri)).Query;
string blobContainerUri = blobContainerSasUri.Split('?')[0];
string requestUri = string.Format(System.Globalization.CultureInfo.InvariantCulture, "{0}/{1}{2}", blobContainerUri, blobName, queryString);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);
request.Method = "PUT";
request.AllowWriteStreamBuffering = false;
request.Headers.Add("x-ms-blob-type", "BlockBlob");
request.ContentLength = contentLength;
request.Timeout = Int32.MaxValue;
request.KeepAlive = true;
int bufferLength = 1048576; //upload 1MB at time, useful for a simple progress bar.
Stream requestStream = request.GetRequestStream();
requestStream.WriteTimeout = Int32.MaxValue;
ProgressViewModel progressViewModel = App.Locator.GetProgressBar(App.Locator.MainViewModel.currentModuleItemId);
MyVideosPage myVideosPage = App.Locator.GetVideosPage(App.Locator.MainViewModel.currentModuleItemId);
FileStream fileStream = new FileStream(path, FileMode.Open, FileAccess.Read);
int nRead = 0;
int currentPos = 0;
while ((nRead = fileStream.Read(blobContent, currentPos, bufferLength)) > 0)
{
await requestStream.WriteAsync(blobContent, currentPos, nRead);
currentPos += nRead;
}
fileStream.Close();
requestStream.Close();
HttpWebResponse objHttpWebResponse = null;
try
{
// this is where it fails for large files
objHttpWebResponse = (HttpWebResponse)request.GetResponse();
Stream responseStream = objHttpWebResponse.GetResponseStream();
StreamReader stream = new StreamReader(responseStream);
responseString = stream.ReadToEnd();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
if (objHttpWebResponse != null)
objHttpWebResponse.Close();
}
return responseString;
}
An exception is thrown after this line is called:
(HttpWebResponse)request.GetResponse();
The exception message is "The request body is too large and exceeds the maximum permissible limit."
The exception StatusCode is "RequestEntityTooLarge".
How can I upload large files? Is this a problem with HttpWebRequest, or Azure Media Services?
Azure Storage supports one shot upload (aka PutBlob API) up to 256MB if you are using the new REST API versions. But since you are not specifying the REST API version, you're defaulting to a very old version where the maximum supported size of one shot upload is 100MB.
Use x-ms-version: 2018-03-28 header to be able to upload up to 256MB in one HTTP request.
If you have to deal with larger files, you will need to use block & commit upload. You will need to use PutBlock API to stage blocks from the source file. Blocks can be up to 100MB each. Then you need to commit all the blocks using the PutBlockList API. If you don't have to deal with this logic yourself, simply use the Azure Storage SDK for .NET (supports Xamarin) and use the uploadFromFile method. It is simple, and resilient.
What is the best way to resize an uploaded image in MVC 6? I'd like to store multiple variants of an image (such as small, large, etc.) to be able to choose which to display later.
Here's my code for the action.
[HttpPost]
public async Task<IActionResult> UploadPhoto()
{
if (Request.Form.Files.Count != 1)
return new HttpStatusCodeResult((int)HttpStatusCode.BadRequest);
IFormFile file = Request.Form.Files[0];
// calculate hash
var sha = System.Security.Cryptography.SHA256.Create();
byte[] hash = sha.ComputeHash(file.OpenReadStream());
// calculate name and patch where to store the file
string extention = ExtentionFromContentType(file.ContentType);
if (String.IsNullOrEmpty(extention))
return HttpBadRequest("File type not supported");
string name = WebEncoders.Base64UrlEncode(hash) + extention;
string path = "uploads/photo/" + name;
// save the file
await file.SaveAsAsync(this.HostingEnvironment.MapPath(path));
}
I would suggest using Image Processor library.
http://imageprocessor.org/imageprocessor/
Then you can just do something along the lines of:
using (var imageFactory = new ImageFactory())
using (var fileStream = new FileStream(path))
{
file.Value.Seek(0, SeekOrigin.Begin);
imageFactory.FixGamma = false;
imageFactory.Load(file.Value)
.Resize(new ResizeLayer(new Size(264, 176)))
.Format(new JpegFormat
{
Quality = 100
})
.Quality(100)
.Save(fileStream);
}
Where file.Value is your file that was uploaded (the stream) (I don't know what it is in MVC, this is code I use in a Nancy project)
In Windows Phone 7 how can I save a BitmapImage to local storage? I need to save the image for caching and reload if it is requested again in the next few days.
If you save the file into IsolatedStorage you can set a relative path to view it from there.
Here's a quick example saving a file that was included in the XAP (as a resource) into Isolated Storage.
using (IsolatedStorageFile isoStore = IsolatedStorageFile.GetUserStoreForApplication())
{
if (!isoStore.FileExists(fileName)
{
var sr = Application.GetResourceStream(new Uri(fileName, UriKind.Relative));
using (var br = new BinaryReader(sr.Stream))
{
byte[] data = br.ReadBytes((int)sr.Stream.Length);
string strBaseDir = string.Empty;
const string DelimStr = "/";
char[] delimiter = DelimStr.ToCharArray();
string[] dirsPath = fileName.Split(delimiter);
// Recreate the directory structure
for (int i = 0; i < dirsPath.Length - 1; i++)
{
strBaseDir = Path.Combine(strBaseDir, dirsPath[i]);
isoStore.CreateDirectory(strBaseDir);
}
using (BinaryWriter bw = new BinaryWriter(isoStore.CreateFile(fileName)))
{
bw.Write(data);
}
}
}
}
You may also be interested in the image caching converters created by Ben Gracewood and Peter Nowaks. They both show saving images into isolated storage and loading them from there.
Another approach I've used is to pass the stream you retrieve for the image in your xap straight into an isolated storage file. Not a lot of moving parts.
using (var isoStore = IsolatedStorageFile.GetUserStoreForApplication()) {
var bi = new BitmapImage();
bi.SetSource(picStreamFromXap);
var wb = new WriteableBitmap(bi);
using (var isoFileStream = isoStore.CreateFile("pic.jpg"))
Extensions.SaveJpeg(wb, isoFileStream, wb.PixelWidth, wb.PixelHeight, 0, 100);
}