I have an OWIN Selfhosted WebAPI with a Controller that accepts a large file as input. The code works in PC without any problems or errors. However in MONO on OSX, the message stays in hung mode in Chrome. When I supply a small size file using Base64 string, the call gets through. This helped me conclude that my file which is of size 1.5 MB is not an acceptable Base64 message size for POST on MONO with OSX.
However I tried using maxContentSize to a gig on HttpRunTime and Also tried OWIN Middleware implementation with setting MaxRequestSizeInBytes to a Gig. Both of these did not let me POST the file still to Web API Controller and request stays in pending status.
Please let me know, if you have any other ideas for setting up Max Message Size or know if there is something on MONO preventing file of size 1.5 MB.
I found a workaround for MAC using OwinMiddleware. Even if I increased the message size, it didn't work, however below solution let me bypass the message size limitation (65K max).
public override async Task Invoke(IOwinContext context)
{
try
{
IOwinRequest request = context.Request;
if (request.Path.HasValue && request.Path.Value.Contains("METHODNAMEFORLARGEFILE"))
{
//TODO: Remove Workaround for MAC to explicitly download whole stream to avoid issues with message size. PC works without this workaround.
LogHelper.Logger.Info("OWIN METHODNAMEFORLARGEFILE Request");
var stream = request.Body;
string requestBody = await (new StreamReader(stream)).ReadToEndAsync();
var requestData = Encoding.UTF8.GetBytes(requestBody);
context.Request.Body = new MemoryStream(requestData);
}
await Next.Invoke(context);
}
catch(Exception ex)
{
LogHelper.Logger.Error(ex.Message, ex);
throw ex;
}
}
Related
I'm currently working on a bot for my Discord server and I was wondering how to implement various image commands (for example, !cat, !meme) to make the bot send a random image each time the command is called.
Almost every bot I've seen has a feature like that but for some reason I can't seem to find a working way to do this in JDA. And any JDA example I found was either outdated or it simply didn't work, so I really hope someone can give me a hand here.
Here's a (very basic) example I already did, but the problem is that the pictures don't randomize with each call and just stay the same until I restart discord
public void sendCatImage() {
EmbedBuilder result= new EmbedBuilder();
result.setTitle("Here's a cat!");
result.setImage("http://thecatapi.com/api/images/get?format=src&type=png");
event.getChannel().sendMessage(result.build()).queue();
}
I'm using JDA Version 4.1.0_100, if it helps
Any help will be greatly appreciated!
Discord will cache the image based on the URL. You can append a random number as a query to prevent this:
public String randomize(String url) {
ThreadLocalRandom random = ThreadLocalRandom.current();
return url + "&" + random.nextInt() + "=" + random.nextInt();
}
...
result.setImage(randomize(url));
...
Furthermore, you can avoid discord updating the image by also uploading it alongside the embed. For that you first need to download the image and then upload it:
// Use same HTTP client that jda uses
OkHttpClient http = jda.getHttpClient();
// Make an HTTP request to download the image
Request request = new Request.Builder().url(imageUrl).build();
Response response = http.newCall(request).execute();
try {
InputStream body = response.body().byteStream();
result.setImage("attachment://image.png"); // Use same file name from attachment
channel.sendMessage(result.build())
.addFile(body, "image.png") // Specify file name as "image.png" for embed (this must be the same, its a reference which attachment belongs to which image in the embed)
.queue(m -> response.close(), error -> { // Send message and close response when done
response.close();
RestAction.getDefaultFailure().accept(error);
});
} catch (Throwable ex) {
// Something happened, close response just in case
response.close();
// Rethrow the throwable
if (ex instanceof Error) throw (Error) ex;
else throw (RuntimeException) ex;
}
I'm using HttpSelfHostConfiguration to create a WebAPI (service). My goal is to have one route stream the mjpeg video from a security feed and have other routes available for configuration and the web interface.
The issue I'm having is that every example I've come across expects a known quantity of images in order to set the content-length for the main response. I don't have this, and flushing the stream doesn't do anything.
Here's the current code for the response. If I use this same code with raw sockets instead of through an ApiController, I can stream it just fine, but creating a webserver from scratch for everything else I'm needing doesn't seem like a lot of fun.
[HttpGet]
public HttpResponseMessage Stream(int channel)
{
var response = Request.CreateResponse();
response.Content = new PushStreamContent((outputStream, content, context) =>
{
StreamWriter writer = new StreamWriter(outputStream);
while (true)
{
using (MemoryStream ms = new MemoryStream())
{
ReadMemoryMappedFile(channel);
ms.SetLength(0);
this.Image.Bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] buffer = ms.GetBuffer();
writer.WriteLine("--boundary");
writer.WriteLine("Content-Type: image/jpeg");
writer.WriteLine(string.Format("Content-length: {0}", buffer.Length));
writer.WriteLine();
writer.Write(buffer);
writer.Flush();
}
}
});
response.Content.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("multipart/x-mixed-replace; boundary=--boundary");
return response;
}
I hope my late answer will help, since I ran into the same issue recently and it took me some time to figure it out...
My solution was to specify the boudary int the ContentType without the "--" (but you need to keep them while writing in the stream).
Try to configure the Headers like that :
response.Content.Headers.ContentType = System.Net.Http.Headers.MediaTypeHeaderValue.Parse("multipart/x-mixed-replace; boundary=boundary");
And write boundaries in the stream like that :
writer.WriteLine("--boundary");
Like this it works for me.
I couldn't find anywhere that explicitly states this, but I'm going to assume that HttpSelfHostConfiguration does not support the functionality I'm looking for, and always requires the stream to be closed before it will release the buffer.
I swapped HttpSelfHostConfiguration with OWIN.SelfHost and it works as expected.
I've written a web API for sending and receiving data between a server and an app I'm working on.
It works fine for everything, but I've now found it won't let me send large strings. The strings are base64 strings which represent images, typically around 100kb in size.
The method I'm attempting is a multipart post, breaking the base64 string into chunks that can be sent successfully.
When I use this method, I get an error:
stream was reset: PROTOCOL_ERROR
Upon checking the database it seems the first string chunk sends successfully, but nothing more after that.
Can anyone shed some light on what's causing this to happen?
Relevant code is here:
First is the process for breaking the image into manageable chunks and posting it:
HTTPRequest req = new HTTPRequest();
IEnumerable<string> imgStrSplit = Split(img1byteArrayStr, 1000);
foreach (string s in imgStrSplit)
{
response = await req.SubmitImage("Test", s, "1");
}
And below is the SubmitImage() method of the HTTPRequest class:
public async Task<int> SubmitImage(string name, string imageString, string imgNum)
{
using (System.Net.Http.HttpClient client = new System.Net.Http.HttpClient(new NativeMessageHandler()))
{
client.DefaultRequestHeaders.Add("Accept", "application/json");
int successResponse;
string address = $"https://myURL/SubmitImage?name=" + name + "&imageStr=" + imageString + "&imgNum=" + imgNum + "&curl=AYZYBAYZE143";
HttpResponseMessage response = await client.GetAsync(address);
successResponse = JsonConvert.DeserializeObject<int>(response.Content.ReadAsStringAsync().Result);
return successResponse;
}
Thanks.
Found a solution via this discussion.
The problem was in the fact that I was using the ModernHTTPClint NuGet package, as seen in this line of the SubmitImage method:
using (System.Net.Http.HttpClient client = new System.Net.Http.HttpClient(new NativeMessageHandler()))
The NativeMessageHandler() is said to make connections work much faster, but apparently in some cases results in the PROTOCOL_ERROR I had encountered. By removing this part, I fixed the problem.
I am using the Background Transfer to upload Photographs to my Web Service. As the Photograph uploads can consume significant time and memory, I thought it might be a nice idea to use the background transfer request to accomplish this. After the photo is uploaded, I want to obtain the Id of the uploaded photo and then use it for post-processing. However, it turns out I can't do that in a background transfer request.
Per my understanding, Background Transfer works using the following logic ONLY:
You have to obtain the file you want to upload and then save/copy it to your app's Isolated Storage under the folder: shared/transfers. This is extremely important. Apparently, using file in a different location didn't work for me. Maybe it isn't the shared/transfers as much as it is a 'relative' path. But I would stick to the same conventions.
After you have saved the file in that location, your background request can be created based on that. It doesn't look like you can pass POST CONTENT other than the file contents, so any other parameters like file name, mime type etc. will need to be passed as QUERY String parameters only. I can understand this, but it would've been nice if I could pass both as POST Content. I don't think HTTP has a limitation on how this works.
Here is some code for creating a request using Hammock:
string url = App.ZineServiceAuthority + "articles/save-blob?ContainerName={0}&MimeType={1}&ZineId={2}&Notes={3}&IsPrivate={4}&FileName={5}";
url = String.Format(url, userId, "image/jpg", ZineId, txtStatus.Text, true, UploadFileName);
var btr = new BackgroundTransferRequest(new Uri(url, UriKind.Absolute));
btr.TransferPreferences = TransferPreferences.AllowCellularAndBattery;
btr.Method = "POST";
btr.Headers.Add("token", IsolatedStorageHelper.GetTravzineToken());
btr.UploadLocation = new Uri(#"/shared\transfers/" + UploadFileName, UriKind.Relative);
btr.TransferStatusChanged += new EventHandler<BackgroundTransferEventArgs>(btr_TransferStatusChanged);
btr.TransferProgressChanged += new EventHandler<BackgroundTransferEventArgs>(btr_TransferProgressChanged);
BackgroundTransferService.Add(btr);
In my case, I am literally passing all the necessary parameters using the query string. On a successful save, my Web Service returns back the Id of the Photo I just uploaded. However:
There is NO way (or at least I know of) to obtain and evaluate the RESPONSE. The Background Transfer Request Event handlers do not expose a RESPONSE.
Here are my event handlers:
void btr_TransferProgressChanged(object sender, BackgroundTransferEventArgs e)
{
bool isUploading = e.Request.TotalBytesToSend > 0 ? true : false;
lblStatus.Text = isUploading ? "Uploading" + e.Request.BytesSent.ToString() + " sent" : "Done";
}
void btr_TransferStatusChanged(object sender, BackgroundTransferEventArgs e)
{
if (e.Request.TransferStatus == TransferStatus.Completed)
{
using (IsolatedStorageFile iso =
IsolatedStorageFile.GetUserStoreForApplication())
{
if (iso.FileExists(e.Request.UploadLocation.OriginalString))
iso.DeleteFile(e.Request.UploadLocation.OriginalString);
}
BackgroundTransferService.Remove(e.Request);
if (null != e.Request.TransferError)
{
MessageBox.Show(e.Request.TransferError.Message);
}
else
{
lblStatus.Text = "Done baby done";
}
}
}
So now my question is, how does anyone do any sort of POST Processing in such scenarios?
Can anyone please tell me the line of thought behind designing such an inflexible class?
Any thoughts on how I could get around this issue would be appreciated.
Also, does anyone have any working examples of a homegrown BackgroundTransfer?
Haven't tried it but why not set a download location like this:
btr.DownloadLocation = "myDownloadFile.html";
btr.UploadLocation = "myUploadFile.jpg";
...
If the request is completed read the file "myDownloadFile.html" where your response has been stored and delete it afterwards.
I use in my code calls to HttpWebRequest.BeginGetResponse() method to get data from my server. The server produces content that may range from few KB to few GB.
My problem is that HttpWebRequest.BeginGetResponse completes too late. It should complete immediately after the connection to the server is established and the HTTP header is received.
Here is sample code using GET method:
public bool StartDownload()
{
try
{
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(m_getUrl);
myHttpWebRequest.Method = "GET";
// Start the asynchronous request.
m_requestState = new RequestState();
m_requestState.request = myHttpWebRequest;
myHttpWebRequest.BeginGetResponse(new AsyncCallback(ResponseCompleted), m_requestState);
}
catch (Exception)
{
m_requestState = null;
}
return m_requestState != null;
}
private void ResponseCompleted(IAsyncResult result)
{
RequestState myRequestState = (RequestState)result.AsyncState;
HttpWebRequest myHttpWebRequest = myRequestState.request;
m_logger.LogMessage("ResponseCompleted notification received!");
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)myHttpWebRequest.EndGetResponse(result);
}
catch (Exception)
{
}
.......
}
I run the code using "http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.39.1.tar.bz2" for example and the result looks like:
hh:mm:ss.ms
12:51:30.9440000 - Download started!
12:53:04.8520000 - ResponseCompleted notification received!
12:53:04.8560000 - Header received!
12:53:04.8570000 - DataReceived: 524288 bytes
.........................................
12:53:04.8940000 - DataReceived: 78818 bytes
12:53:04.8940000 - Request data received!
12:53:04.8940000 - Received bytes: 76100578
The problem can be easily detected in the log. It is not possible to spend more that one minute to connect and 38 ms to download about 72.5 MB.
It seems that the data is downloaded somewhere on the phone and the RequestComplete notification is sent to the application only when the full content is available locally. This is not OK for me because I need to show progress for the operation.
I get the same result on the device and emulator for WP7 (also on WP7.1).
I run same code on Windows desktop and it run correctly: the request completes within one second and the rest of the download takes about 1-2 minutes.
Is there any solution on WP7 or WP 7.1?
The newly introduced WP 7.1 API "Background File Transfers" does not help because I need full control over the HTTP headers and content. Not all HTTP requests that I make to the server produce files as output.
Thank you!
Mihai
You need to disable response buffering if you want to stream the data down. You can do this by setting AllowReadStreamBuffering to false.
HttpWebRequest myHttpWebRequest = WebRequest.CreateHttp(m_getUrl);
myHttpWebRequest.Method = "GET";
myHttpWebRequest.AllowReadStreamBuffering = false;