Serve large file async and then delete it - async-await

Using Web API 2, I have a process that generates a temporary file for the purpose of writing it to the output stream for client consumption. The process can be somewhat long running, taking a few minutes to complete.
What I'd like to do is serve the file asynchronously and then delete it upon completion or cancellation (connection timeout).
Would something like this be close? I haven't tested yet, but I'm wondering of the continue will delete the file before it's served entirely.
public class FileCreationResult
{
public String FilePathOut { get; set; }
}
public class MyFileCreationProcess{
const string NaiveDefaultTempFilePath = #"c:\temp\mytempfile.bin";
public FileCreationResult Execute(){
// do stuff that makes a file
File.WriteAllBytes(NaiveDefaultTempFilePath, NaiveDefaultTempFilePath);
return new FileCreationResult{
FilePathOut = NaiveDefaultTempFilePath
};
}
}
public class Controller : ApiController
{
private readonly MyFileCreationProcess process;
public Controller(MyFileCreationProcess process)
{
this.process = process;
}
public async Task<HttpResponse> GetFile(CancellationToken cancellationToken){
return await Task.Run(()=>{
var fileCreationResult = process.Execute();
using( var stream = File.OpenRead(fileCreationResult.FilePathOut)){
var response = Request.CreateResponse();
response.Content = new StreamContent( stream);
return response;
//
// I would like to delete the file at path fileCreationResult.FilePathOut
// after it has been written to the output stream..
// how would I do this ?
}
});
}
}

You can probably just get away with a try finally block
public async Task<HttpResponse> GetFile(CancellationToken cancellationToken){
return await Task.Run(()=>{
var fileCreationResult = process.Execute();
try{
using( var stream = File.OpenRead(fileCreationResult.FilePathOut)){
var response = Request.CreateResponse();
response.Content = new StreamContent( stream);
return response;
}
}finally{
if(File.Exists(fileCreationResult.FilePathOut))
{
File.Delete(fileCreationResult.FilePathOut);
}
}
});
}
or if you want to use continueWith:
public Task<HttpResponse> GetFile(CancellationToken cancellationToken){
string path;
return Task.Run(()=>{
var fileCreationResult = process.Execute();
path = fileCreationResult.FilePathOut;
using( var stream = File.OpenRead(path)){
var response = Request.CreateResponse();
response.Content = new StreamContent( stream);
return response;
}
}).ContinueWith(x=>{
if(File.Exists(fileCreationResult.FilePathOut))
{
File.Delete(fileCreationResult.FilePathOut);
}
});
}
I should note I don't have a C# compiler handy so it may not compile but I think you get the point.

Related

How check byte count (size) of existing blob (file) on Azure

I cannot find a decent example of how to use the latest version of Azure BlobClient to get the byte count of an existing blob.
Here is the code I am working with so far. I cannot figure out how to filter the blob I need to find. I can get them all, but it takes ages.
protected BlobContainerClient AzureBlobContainer
{
get
{
if (!isConfigurationLoaded) { throw new Exception("AzureCloud currently has no configuration loaded"); }
if (_azureBlobContainer == null)
{
if (!string.IsNullOrEmpty(_configuration.StorageEndpointConnection))
{
BlobServiceClient blobClient = new BlobServiceClient(_configuration.StorageEndpointConnection);
BlobContainerClient container = blobClient.GetBlobContainerClient(_configuration.StorageContainer);
container.CreateIfNotExists();
_azureBlobContainer = container;
}
}
return _azureBlobContainer;
}
}
public async Task<Response<BlobProperties>> GetAzureFileSize(string fileName)
{
BlobClient cloudFile = AzureBlobContainer.GetBlobClient(fileName);
await foreach (BlobItem blobItem in AzureBlobContainer.GetBlobsAsync())
{
Console.WriteLine("\t" + blobItem.Name);
}
// Am i supposed to just iterate every single blob on there? how do I filter?
return blobProps;
}
Thoughts?
Ended up doing it like this... I get that it isn't the proper way, but it works.
public async Task<(BlobClient cloudFile, CopyFromUriOperation result)> MigrateFileToAzureBlobAsync(Uri url, string fileName, long? bytesCount)
{
BlobClient cloudFile = AzureBlobContainer.GetBlobClient(fileName);
cloudFile.DeleteIfExists();
BlobCopyFromUriOptions options = new BlobCopyFromUriOptions();
options.Metadata = new Dictionary<string, string>();
options.Metadata.Add("confexbytecount", bytesCount.ToString());
CopyFromUriOperation result = await cloudFile.StartCopyFromUriAsync(url, options);
Response x = await result.UpdateStatusAsync();
await result.WaitForCompletionAsync();
return (cloudFile, result);
}
public async Task<long> GetAzureFileSize(string fileName)
{
long retVal = 0;
await foreach (BlobItem blobItem in AzureBlobContainer.GetBlobsAsync(BlobTraits.All, BlobStates.All, fileName))
{
if (blobItem.Metadata.TryGetValue("confexbytecount", out string confexByteCount))
{
long.TryParse(confexByteCount, out retVal);
}
}
return retVal;
}

Receive data and file in method POST

I have a WebService that is working and receiving files using the POST method, but in which I also need to receive data, simultaneously.
ASP.NET WebApi code:
public Task<HttpResponseMessage> Post()
{
HttpRequestMessage request = this.Request;
if (!request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = System.Web.HttpContext.Current.Server.MapPath("~/App_Data/uploads");
var provider = new MultipartFormDataStreamProvider(root);
var task = request.Content.ReadAsMultipartAsync(provider).
ContinueWith<HttpResponseMessage>(o =>
{
string file1 = provider.FileData.First().LocalFileName;
return new HttpResponseMessage()
{
Content = new StringContent("File uploaded.")
};
}
);
return task;
}
And the client, developed for Android, is sending the file and the data like this (the send of the file is tested and working, the sending of the data is still not tested, as I need it to be working in the server side):
OkHttpClient client = new OkHttpClient();
RequestBody requestBody = new MultipartBuilder()
.type(MultipartBuilder.FORM)
.addPart(
Headers.of("Content-Disposition", "form-data; name=\"title\""),
RequestBody.create(null, "Sample Text Content"))
.addPart(
Headers.of("Content-Disposition", "form-data; name=\"file\"; filename=\"" + fileName + ".png\""),
RequestBody.create(MEDIA_TYPE_PNG, bitmapdata))
.addFormDataPart("fullpath", "test")
.build();
final com.squareup.okhttp.Request request = new com.squareup.okhttp.Request.Builder()
.url(url)
.post(requestBody)
.build();
How can I change the server to read not only the file but also the data?
Can any one help?
Thanks in advance.
The client in this case android is sending additional values in the body like media_type_png. I had to do something similar however the client was angular and not a mobile app, after some searching back then I found code from the following stackoverflow. Which resulted in the code below.
First receive the incoming message and check that you can process it i.e. [IsMimeMultipartContent][1]()
[HttpPost]
public async Task<HttpResponseMessage> Upload()
{
// Here we just check if we can support this
if (!Request.Content.IsMimeMultipartContent())
{
this.Request.CreateResponse(HttpStatusCode.UnsupportedMediaType);
}
// This is where we unpack the values
var provider = new MultipartFormDataMemoryStreamProvider();
var result = await Request.Content.ReadAsMultipartAsync(provider);
// From the form data we can extract any additional information Here the DTO is any object you want to define
AttachmentInformationDto attachmentInformation = (AttachmentInformationDto)GetFormData(result);
// For each file uploaded
foreach (KeyValuePair<string, Stream> file in provider.FileStreams)
{
string fileName = file.Key;
// Read the data from the file
byte[] data = ReadFully(file.Value);
// Save the file or do something with it
}
}
I used this to unpack the data:
// Extracts Request FormatData as a strongly typed model
private object GetFormData(MultipartFormDataMemoryStreamProvider result)
{
if (result.FormData.HasKeys())
{
// Here you can read the keys sent in ie
result.FormData["your key"]
AttachmentInformationDto data = AttachmentInformationDto();
data.ContentType = Uri.UnescapeDataString(result.FormData["ContentType"]); // Additional Keys
data.Description = Uri.UnescapeDataString(result.FormData["Description"]); // Another example
data.Name = Uri.UnescapeDataString(result.FormData["Name"]); // Another example
if (result.FormData["attType"] != null)
{
data.AttachmentType = Uri.UnescapeDataString(result.FormData["attType"]);
}
return data;
}
return null;
}
The MultipartFormDataMemoryStreamProvider is defined as follows:
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using System.Web;
namespace YOURNAMESPACE
{
public class MultipartFormDataMemoryStreamProvider : MultipartMemoryStreamProvider
{
private readonly Collection<bool> _isFormData = new Collection<bool>();
private readonly NameValueCollection _formData = new NameValueCollection(StringComparer.OrdinalIgnoreCase);
private readonly Dictionary<string, Stream> _fileStreams = new Dictionary<string, Stream>();
public NameValueCollection FormData
{
get { return _formData; }
}
public Dictionary<string, Stream> FileStreams
{
get { return _fileStreams; }
}
public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
{
if (parent == null)
{
throw new ArgumentNullException("parent");
}
if (headers == null)
{
throw new ArgumentNullException("headers");
}
var contentDisposition = headers.ContentDisposition;
if (contentDisposition == null)
{
throw new InvalidOperationException("Did not find required 'Content-Disposition' header field in MIME multipart body part.");
}
_isFormData.Add(String.IsNullOrEmpty(contentDisposition.FileName));
return base.GetStream(parent, headers);
}
public override async Task ExecutePostProcessingAsync()
{
for (var index = 0; index < Contents.Count; index++)
{
HttpContent formContent = Contents[index];
if (_isFormData[index])
{
// Field
string formFieldName = UnquoteToken(formContent.Headers.ContentDisposition.Name) ?? string.Empty;
string formFieldValue = await formContent.ReadAsStringAsync();
FormData.Add(formFieldName, formFieldValue);
}
else
{
// File
string fileName = UnquoteToken(formContent.Headers.ContentDisposition.FileName);
Stream stream = await formContent.ReadAsStreamAsync();
FileStreams.Add(fileName, stream);
}
}
}
private static string UnquoteToken(string token)
{
if (string.IsNullOrWhiteSpace(token))
{
return token;
}
if (token.StartsWith("\"", StringComparison.Ordinal) && token.EndsWith("\"", StringComparison.Ordinal) && token.Length > 1)
{
return token.Substring(1, token.Length - 2);
}
return token;
}
}
}

Error "Must set UnitOfWorkManager before use it"

I'm developing the service within ASP.NET Boilerplate engine and getting the error from the subject. The nature of the error is not clear, as I inheriting from ApplicationService, as documentation suggests. The code:
namespace MyAbilities.Api.Blob
{
public class BlobService : ApplicationService, IBlobService
{
public readonly IRepository<UserMedia, int> _blobRepository;
public BlobService(IRepository<UserMedia, int> blobRepository)
{
_blobRepository = blobRepository;
}
public async Task<List<BlobDto>> UploadBlobs(HttpContent httpContent)
{
var blobUploadProvider = new BlobStorageUploadProvider();
var list = await httpContent.ReadAsMultipartAsync(blobUploadProvider)
.ContinueWith(task =>
{
if (task.IsFaulted || task.IsCanceled)
{
if (task.Exception != null) throw task.Exception;
}
var provider = task.Result;
return provider.Uploads.ToList();
});
// store blob info in the database
foreach (var blobDto in list)
{
SaveBlobData(blobDto);
}
return list;
}
public void SaveBlobData(BlobDto blobData)
{
UserMedia um = blobData.MapTo<UserMedia>();
_blobRepository.InsertOrUpdateAndGetId(um);
CurrentUnitOfWork.SaveChanges();
}
public async Task<BlobDto> DownloadBlob(int blobId)
{
// TODO: Implement this helper method. It should retrieve blob info
// from the database, based on the blobId. The record should contain the
// blobName, which should be returned as the result of this helper method.
var blobName = GetBlobName(blobId);
if (!String.IsNullOrEmpty(blobName))
{
var container = BlobHelper.GetBlobContainer();
var blob = container.GetBlockBlobReference(blobName);
// Download the blob into a memory stream. Notice that we're not putting the memory
// stream in a using statement. This is because we need the stream to be open for the
// API controller in order for the file to actually be downloadable. The closing and
// disposing of the stream is handled by the Web API framework.
var ms = new MemoryStream();
await blob.DownloadToStreamAsync(ms);
// Strip off any folder structure so the file name is just the file name
var lastPos = blob.Name.LastIndexOf('/');
var fileName = blob.Name.Substring(lastPos + 1, blob.Name.Length - lastPos - 1);
// Build and return the download model with the blob stream and its relevant info
var download = new BlobDto
{
FileName = fileName,
FileUrl = Convert.ToString(blob.Uri),
FileSizeInBytes = blob.Properties.Length,
ContentType = blob.Properties.ContentType
};
return download;
}
// Otherwise
return null;
}
//Retrieve blob info from the database
private string GetBlobName(int blobId)
{
throw new NotImplementedException();
}
}
}
The error appears even before the app flow jumps to 'SaveBlobData' method. Am I missed something?
Hate to answer my own questions, but here it is... after a while, I found out that if UnitOfWorkManager is not available for some reason, I can instantiate it in the code, by initializing IUnitOfWorkManager in the constructor. Then, you can simply use the following construction in your Save method:
using (var unitOfWork = _unitOfWorkManager.Begin())
{
//Save logic...
unitOfWork.Complete();
}

Best practice to send a lot of files (images) from your device to your server

I have a list of files to send from my device to my server.
List<myFiles> list = new List<myFiles>() { "[long list of files...]" };
For that I want to create a list of tasks: for each file I should invoke a function that sends to my webapi that file via PUT
public async Task<bool> UploadPhoto(byte[] photoBytes, int PropertyId, string fileName)
{
bool rtn = false;
if (CrossConnectivity.Current.IsConnected)
{
var content = new MultipartFormDataContent();
var fileContent = new ByteArrayContent(photoBytes);
fileContent.Headers.ContentType =
MediaTypeHeaderValue.Parse("multipart/form-data");
fileContent.Headers.ContentDisposition =
new ContentDispositionHeaderValue("attachment")
{
FileName = fileName + ".jpg"
};
content.Add(fileContent);
string url = RestURL() + "InventoriesPicture/Put";
try
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("authenticationToken", SyncData.Token);
HttpResponseMessage response = await client.PutAsync(url, content);
if (response.IsSuccessStatusCode)
{
rtn = true;
Debug.WriteLine($"UploadPhoto response {response.ReasonPhrase}");
}
else
{
Debug.WriteLine($"UploadPhoto response {response.ReasonPhrase}");
}
}
}
catch (Exception ex)
{
Debug.WriteLine($"UploadPhoto exception {ex.Message}");
}
Debug.WriteLine($"UploadPhoto ends {fileName}");
}
return rtn;
}
In a function I have a foreach that calls UploadPhoto. I think there are too many tasks at the same time then I want to send a file, wait the result from the webapi and then send next file and so on.
What can I do? What is the best practice for that? Or in any case how can I resolve my problem? :)
Thank you in advance

How to make async call to api (http). in c#. inside a Task

I am developeing a chatbot using microsoftbotframmwok where I have some requirement to make a call from my task to an api(httpclient). but it is not working. when i test the api from an stand alone console application in side main method it works. but in my application it doesn't work.
I tried to call an api from an simple method without task but when it makes a cal its basically halts or stucked somewhere, i converted my function into task and while making an api call i used await keyword to call it asynchronously but it is returning error, while reading it not the result.
here is the my code which make an api call
private async Task<String> getProblem(IDialogContext context)
{
var response = "Thannks for contacting..";
//here some code logix..
SnowApiClient client = new SnowApiClient(Url, UserId, ApiPassword);
IncidentRequestPayload payload = new IncidentRequestPayload();
payload.caller_id = "tet111";
payload.assignment_group = "it";
payload.category = "complaint";
payload.u_feedback_type = "Praise";
payload.service_offering = "Application Management";
payload.priority = 2;
payload.short_description = "computer battery is dead";
payload.comments = String.Empty;
ApiResponse objResponse = await client.CreateIncident(payload);
//objResponse.payload.number;
return response;
}
//code for CreateIncident...in Api project librarary
public async Task<ApiResponse> CreateIncident(IncidentRequestPayload payload)
{
var incidentRequest = new ApiRequest { method = CreateIncidentMethod, payload = payload };
var createResult = await ExecuteRequest(incidentRequest);
return await ReadIncident(createResult.payload.incNumber);
}
public async Task<ApiResponse> ReadIncident(string number)
{
var incidentRequest = new ApiRequest { method = ReadIncidentMethod, payload = new RequestPayload { number = number } };
return await ExecuteRequest(incidentRequest);
}
private async Task<ApiResponse> ExecuteRequest(ApiRequest requestObject)
{
HttpResponseMessage response = await _client.PostAsJsonAsync("/SRintegratedAPI.rest", requestObject);
ApiResponse responseObject = null;
if (response.IsSuccessStatusCode)
{
responseObject = await response.Content.ReadAsAsync<ApiResponse>();
}
else
{
throw new System.Net.WebException(response.ReasonPhrase);
}
if (responseObject.result != "ok")
{
throw new System.Net.WebException(responseObject.message);
}
return responseObject;
}
I don't understand how and where do i used async/await here in basicalaly in my getProblem function.
please help

Resources