is returning stream considered anti pattern in web api? - asp.net-web-api

I am from the old world that think webapi should return a strong typed object and let json serialization return data.
However, recently we got this requirement:
We have a sql table which has more than 500 columns.
The customer always want to return all the columns.
Our c# code does nothing other than reading the SqlDatareader, convert the reader to a c# object and return result.
In this case, wouldn't better to do this (example copied from another stackoverflow post). Basically just return a stream? Does returning a stream still considered to be anti-pattern?
public HttpResponseMessage SomeMethod(List<string> someIds)
{
HttpResponseMessage resp = new HttpResponseMessage();
resp.Content = new PushStreamContent(async (responseStream, content, context) =>
{
await CopyBinaryValueToResponseStream(responseStream, someIds);
});
return resp;
}
private static async Task CopyBinaryValueToResponseStream(Stream responseStream, int imageId)
{
// PushStreamContent requires the responseStream to be closed
// for signaling it that you have finished writing the response.
using (responseStream)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
await connection.OpenAsync();
using (SqlCommand command = new SqlCommand("SELECT 500 columns FROM [StupidWideTable] WHERE ....", connection))
{
.....
using (SqlDataReader reader = await command.ExecuteReaderAsync(CommandBehavior.SequentialAccess))
{
if (await reader.ReadAsync())
{
if (!(await reader.IsDBNullAsync(0)))
{
using (Stream data = reader.GetStream(0))
{
// Asynchronously copy the stream from the server to the response stream
await data.CopyToAsync(responseStream);
}
}
}
}
}
}
}// close response stream
}

Does returning a stream still considered to be anti-pattern?
Well, that depends on what you want to do. For example, if you want to return a 500 if the SQL server fails partway through, then you shouldn't return a stream.
Streaming results works fine on ASP.NET, but it's important to note that all headers (including the response status code) are sent before the stream begins. So you'll send an immediate 200 when you start streaming the result, and if there's an error later on there's no way to go back in time and change that to a 500. Or add some kind of Continue header.
In other words, yes it's supported; but you lose all the benefits of model binding, content negotiation, exception handlers, etc., because you're bypassing that whole pipeline.

Related

Twilio Base64 Media Payload for Google Speech To Text API not Responding

I have a need to do some real time transcriptions from twilio phone calls using Google speech-to-text api and I've followed a few demo apps showing how to set this up. My application is in .net core 3.1 and I am using webhooks with a Twilio defined callback method. Upon retrieving the media from Twilio through the callback it is passed as Raw audio in encoded in base64 as you can see here.
https://www.twilio.com/docs/voice/twiml/stream
I've referenced this demo on Live Transcribing as well and am trying to mimic the case statement in the c#. Everything connects correctly and the media and payload is passed into my app just fine from Twilio.
The audio string is then converted to a byte[] to pass to the Task that needs to transcribe the audio
byte[] audioBytes = Convert.FromBase64String(info);
I am following the examples based of the Google docs that either stream from a file or an audio input (such as a microphone.) Where my use case is different is, I already have the bytes for each chunk of audio. The examples I referenced can be seen here. Transcribing audio from streaming input
Below is my implementation of the latter although using the raw audio bytes. This Task below is hit when the Twilio websocket connection hits the media event. I pass the payload directly into it. From my console logging I am getting to the Print Responses hit... console log, but it will NOT get into the while (await responseStream.MoveNextAsync()) block and log the transcript to the console. I do not get any errors back (that break the application.) Is this possible to even do? I have also tried loading the bytes into a memorystream object and passing them in as the Google doc examples do as well.
static async Task<object> StreamingRecognizeAsync(byte[] audioBytes)
{
var speech = SpeechClient.Create();
var streamingCall = speech.StreamingRecognize();
// Write the initial request with the config.
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
StreamingConfig = new StreamingRecognitionConfig()
{
Config = new RecognitionConfig()
{
Encoding =
RecognitionConfig.Types.AudioEncoding.Mulaw,
SampleRateHertz = 8000,
LanguageCode = "en",
},
InterimResults = true,
SingleUtterance = true
}
}); ;
// Print responses as they arrive.
Task printResponses = Task.Run(async () =>
{
Console.WriteLine("Print Responses hit...");
var responseStream = streamingCall.GetResponseStream();
while (await responseStream.MoveNextAsync())
{
StreamingRecognizeResponse response = responseStream.Current;
Console.WriteLine("Response stream moveNextAsync Hit...");
foreach (StreamingRecognitionResult result in response.Results)
{
foreach (SpeechRecognitionAlternative alternative in result.Alternatives)
{
Console.WriteLine("Google transcript " + alternative.Transcript);
}
}
}
});
//using (MemoryStream memStream = new MemoryStream(audioBytes))
//{
// var buffer = new byte[32 * 1024];
// int bytesRead;
// while ((bytesRead = await memStream.ReadAsync(audioBytes, 0, audioBytes.Length)) > 0)
// {
// await streamingCall.WriteAsync(
// new StreamingRecognizeRequest()
// {
// AudioContent = Google.Protobuf.ByteString
// .CopyFrom(buffer, 0, bytesRead),
// });
// }
//}
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
AudioContent = Google.Protobuf.ByteString
.CopyFrom(audioBytes),
});
await streamingCall.WriteCompleteAsync();
await printResponses;
return 0;
}
After all this, I discovered that this code works fine, just needs to be broken up and called in different events in the Twilio stream lifecycle.
The config section needs to be placed during the connected event.
The print messages task needs to be placed in the media event.
Then, the WriteCompleteAsync needs to be placed in the stop event when the websocket is closed from Twilio.
One other important item to consider are the number of requests being sent to Google STT to ensure that too many requests aren't overloading the quota which seems to be (for now) 300 requests / minute.

Using WebAPI to stream MJPEG always buffers

I'm using HttpSelfHostConfiguration to create a WebAPI (service). My goal is to have one route stream the mjpeg video from a security feed and have other routes available for configuration and the web interface.
The issue I'm having is that every example I've come across expects a known quantity of images in order to set the content-length for the main response. I don't have this, and flushing the stream doesn't do anything.
Here's the current code for the response. If I use this same code with raw sockets instead of through an ApiController, I can stream it just fine, but creating a webserver from scratch for everything else I'm needing doesn't seem like a lot of fun.
[HttpGet]
public HttpResponseMessage Stream(int channel)
{
var response = Request.CreateResponse();
response.Content = new PushStreamContent((outputStream, content, context) =>
{
StreamWriter writer = new StreamWriter(outputStream);
while (true)
{
using (MemoryStream ms = new MemoryStream())
{
ReadMemoryMappedFile(channel);
ms.SetLength(0);
this.Image.Bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] buffer = ms.GetBuffer();
writer.WriteLine("--boundary");
writer.WriteLine("Content-Type: image/jpeg");
writer.WriteLine(string.Format("Content-length: {0}", buffer.Length));
writer.WriteLine();
writer.Write(buffer);
writer.Flush();
}
}
});
response.Content.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("multipart/x-mixed-replace; boundary=--boundary");
return response;
}
I hope my late answer will help, since I ran into the same issue recently and it took me some time to figure it out...
My solution was to specify the boudary int the ContentType without the "--" (but you need to keep them while writing in the stream).
Try to configure the Headers like that :
response.Content.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("multipart/x-mixed-replace; boundary=boundary");
And write boundaries in the stream like that :
writer.WriteLine("--boundary");
Like this it works for me.
I couldn't find anywhere that explicitly states this, but I'm going to assume that HttpSelfHostConfiguration does not support the functionality I'm looking for, and always requires the stream to be closed before it will release the buffer.
I swapped HttpSelfHostConfiguration with OWIN.SelfHost and it works as expected.

GZip .NET Compact Framework 3.5

I am trying to send and receive process gzip-ed data from server on my client device application (not web).
I am sending gzip-ed content and on client side, I have following method that returns WebResponse:
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse res = base.GetWebResponse(request);
if (((System.Net.HttpWebResponse)(res)).ContentEncoding.Contains("gzip"))
{
Stream responseStream = res.GetResponseStream();
responseStream = new GZipStream(responseStream, CompressionMode.Decompress);
}
//This returns g-ziped content as WebResponse, but I need to return
//above decompressed responseStream as WebResponse, how do I do that?
return res;
}
I am new to this but I am thinking that intercepting every response comming to my app in GetWebResponse is excellent centralized spot to decompress all responses. But the problem is how to pass the decompressed stream as response back?
Much appreciated

OWIN OnSendingHeaders Callback - Reading Response Body

This question is related to the excellent answer by Youssef. I love OnSendingHeaders callback. I can now add the response headers without worrying about switching streams. Anyways, here is my question. Is it possible to read the response body inside the callback, like so.
public override async Task Invoke(OwinRequest request, OwinResponse response)
{
request.OnSendingHeaders(state =>
{
var resp = (OwinResponse)state;
// Here, I want to convert resp, which is OwinResponse
// to HttpResponseMessage so that when Content.ReadAsStringAsync
// is called off this HttpResponseMessage object, I want the
// response body as string.
var responseMessage = new HttpResponseMessage();
responseMessage.Content = new StreamContent(resp.Body);
// Here I would like to call
// responseMessage.Content.ReadAsStringAsync()
}, response);
await Next.Invoke(request, response);
}
The methods I want to call from the callback are part of classes that depend on HttpResponseMessage and do not want to change them.
If I set the response body to memory stream before the pipeline processing starts (as was initially suggested by Youssef in the linked answer), I'm able to get this working. Is there a better way to do this here in the callback instead of that?
EDIT:
Is this okay?
public override async Task Invoke(OwinRequest request, OwinResponse response)
{
// Do something with request
Stream originalStream = response.Body;
var buffer = new MemoryStream();
response.Body = buffer;
await Next.Invoke(request, response);
var responseMessage = new HttpResponseMessage();
response.Body.Seek(0, SeekOrigin.Begin);
responseMessage.Content = new StreamContent(response.Body);
// Pass responseMessage to other classes for the
// response body to be read like this
// responseMessage.Content.ReadAsStringAsyn()
// Add more response headers
if (buffer != null && buffer.Length > 0)
{
buffer.Seek(0, SeekOrigin.Begin);
await buffer.CopyToAsync(originalStream);
}
}
What do you want to do with the response body?
This callback is invoked on first write, so it's too late to replace the stream. You also can't read from the response stream as there is nothing stored in it normally. This is normally a write-only stream that goes out to the network.
Replacing the response stream earlier is the correct approach here.

WebService ASP.NET MVC 3 Send and Receive

I've been racking my brain for a couple of days now on how to approach a new requirement.
I have two websites. The first one lets the user fill out an application. The second website is an internal website use to manage the users applications. I need to develop a "web service" that sends the application data from website 1 to website 2 and return a response to website 2 of success or failure. I have never done a web service before and I'm a bit confused on where to start. I've been reading various examples online but they all seem to be just a starting point for building a webservice... no specific examples.
So for posting the data website 1, what would my controller method look like? Do I use Json to post the data to website 2? What would and example of that look like? Is there some form of redirect in the method that points to website 2?
So for posting the response back to website 2 what would that controller method look like? I assume I would use Json again to send the response back to website 1? Is there some form of redirect in the method that points back to website 1?
I would use JSON and POST the application to the web service.
First I am assuming the application data is contained in some type of object. Use JSON.Net to serialize the object into JSON. It will look something like the following code.
var application = new Application();
string serializedApplication = JsonConvert.Serialize(application);
Second is to POST the code your endpoint(webservice, mvc action). To this you'll need to make a HTTPRequest to the endpoint. The following code is what I use to make to POST the code.
public bool Post(string url, string body)
{
//Make the post
ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, errors) => true;
var bytes = Encoding.Default.GetBytes(body);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
Stream stream = null;
try
{
request.KeepAlive = false;
request.ContentLength = bytes.Length;
request.ContentType = "application/x-www-form-urlencoded";
request.Timeout = -1;
request.Method = "POST";
stream = request.GetRequestStream();
stream.Write(bytes, 0, bytes.Length);
}
finally
{
if (stream != null)
{
stream.Flush();
stream.Close();
}
}
bool success = GetResponse(request);
return success;
}
public bool GetResponse(HttpWebRequest request)
{
bool success;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
if (response.StatusCode != HttpStatusCode.OK && response.StatusCode != HttpStatusCode.Created)
{
throw new HttpException((int)response.StatusCode, response.StatusDescription);
}
var end = string.Empty;
using (StreamReader reader = new StreamReader(responseStream))
{
end = reader.ReadToEnd();
reader.Close();
success = JsonConvert.DeserializeObject<bool>(end);
}
response.Close();
}
}
return success;
}
So now you have can POST JSON to an endpoint and receive a response the next step is to create the endpoint. The following code will get you started on an endpoint in mvc that will receive an application and process it.
[HttpPost]
public ActionResult SubmitApplication()
{
//Retrieve the POSTed payload
string body;
using (StreamReader reader = new StreamReader(Request.InputStream))
{
body = reader.ReadToEnd();
reader.Close();
}
var application = JsonConvert.Deserialize<Application>(body);
//Save the application
bool success = SaveApplication(application);
//Send the server a response of success or failure.
return Json(success);
}
The above code is a good start. Please note, I have not tested this code.
You have obviously more than one client for the data & operations. so a service is what you are looking for.
ASP.NET MVC is a good candidate for developing RESTful services. If you (and your Manager) are ready to use beta version, Then Checkout ASP.NET-Web API.
If you want to stay with a stable product, Go for MVC3. you may need to write some custom code to return the data in XML as well as JSON to server different kind of clients. There are some tutorials out there.
So create a Service (ASP.NET MVC / WCF Service) .You may then create 2 client apps, one for the external clients and another for the Internal users. Both of this apps can call methods in the Service to Create/ Read the user accounts / or whatever operation you want to do.
To make the apps more interactive and lively , you may conside including a wonderful thing called SiganalR, which helps you to get some real time data without continuosly polling the data base/ middle tier very in every n seconds !

Resources