Get Large JSON from web API - ajax

I am using .net Web Api to get data for my data grid. The call is made via ajax call like this
$.ajax({
type: 'GET',
dataType: 'json',
contentType: "application/json; charset=utf-8",
url: ReportURL, // "api/AppData/InvoiceReport/10"
success: function (mydata) {
console.log(mydata);
createReportGrid(myData); // this function creates a KENDO grid
},
error: function (error) {
alert(error);
}
});
The Web API method Looks like this
[HttpGet]
public HttpResponseMessage InvoiceReport(int Id)
{
// some llogic of data retrieving
// objReportDataList is of Type List<vmReport>
// thisstructure contains a DataTable, and 2 more list type
return Request.CreateResponse(HttpStatusCode.OK, objReportDataList);
}
These calls work perfectly for Rows Count approx 100K rows
Web Api serializes perfectly. But when the Row Count exceeds 200K i get 500 internal server error
The stack traces tells "System-OutOfMemoryException-occured-in-mscorlib-dll"
NOTE - I cant use server pagination to get little data only. This Million rows data is working on a ASP.NET Webforms application. We have migrated to MVC pattern and used WebApi for fetching Data but this error is occuring.
PS - I have tried many many solutions, but helpless
Please Guide me to Get this error removed and my reports get going

you should stream the response. then you can return millions of rows which are sent in chunks.
[HttpGet]
public HttpResponseMessage PushStreamContent()
{
var response = Request.CreateResponse();
response.Content =
new PushStreamContent((stream, content, context) =>
{
foreach (var staffMember in _staffMembers)
{
var serializer = new JsonSerializer();
using (var writer = new StreamWriter(stream))
{
serializer.Serialize(writer, staffMember);
stream.Flush();
}
}
});
return response;
}
more info here:
http://dblv.github.io/2014/07/02/streaming-web-api/

you can add below in web.config file
<configuration>
<system.web.extensions>
<scripting>
<webServices>
<jsonSerialization maxJsonLength="50000000"/>
</webServices>
</scripting>
</system.web.extensions>
</configuration>

Related

Server-side method does not write to database unless client-side caller is paused/interrupted (via alert)

I am using an Ajax post method to pass a JSON string to a server-side MVC action. The IActionResult method parses the JSON string into an array which is uploaded into SQL Server via Microsoft.Data.SqlClient methods. The IActionResult returns an Ok() result to the caller upon completion.
The anomaly I have observed is that the database upload (server-side) only completes if I pause/interrupt the browser by placing an alert just after the Ajax method (client-side). My code is as follows:
Client-side:
function ExportJSON() {
var myJson = "some JSON stuff goes here";
$.ajax({
type: "POST",
url: "/Dailies/UploadJson/",
dataType: 'json',
data: { jsonString: myJson },
success: function (data) {
console.log(data);
}
});
alert("Your data has been saved.");
}
Server-side action:
[HttpPost]
[ProducesResponseType(StatusCodes.Status200OK)]
public async Task<IActionResult> UploadJson(string jsonString)
{
if (jsonString != null) {
var myJArray = JsonConvert.DeserializeObject<JArray>(jsonString);
TimePunches[] timePunches = myJArray.ToObject<TimePunches[]>();
try
{
string constr = _configuration.GetConnectionString("MYSQLSERVER");
using (SqlConnection sqlConnection = new SqlConnection(constr)) {
await sqlConnection.OpenAsync();
foreach (TimePunches timePunch in timePunches) {
string query = "INSERT INTO TimePunches([Projectid], [CrewLeaderId]) ";
query += "VALUES(#Projectid, #CrewLeaderId) ";
using (SqlCommand cmd = new SqlCommand(query)) {
cmd.Connection = sqlConnection;
cmd.Parameters.AddWithValue("#Projectid", timePunch.Projectid);
cmd.Parameters.AddWithValue("#CrewLeaderId", timePunch.CrewLeaderId);
await cmd.ExecuteNonQueryAsync();
}
}
sqlConnection.Close();
}
}
catch (Exception ex) {
TempData["msg"] = ex.Message;
}
}
return Ok();
}
To reiterate, the server-side action uploads data to the database (as expected) so long as the alert is present in the client-side ExportJSON() method. Conversely, removing the alert causes the database upload to fail.
Any assistance would be greatly appreciated.
I found an answer to this issue. The following link provides an excellent article explaining the nuances of asynchronous JavaScript calls and, in particular, how to manage synchronization issues when using Ajax: https://stackify.com/return-ajax-response-asynchronous-javascript-call/
In my particular case the solution was as simple as adding an async: false qualifier to my Ajax "post" method.

XML Parsing Error: no root element found in ASP.NET Core 2.0 API

I've encountered this problem, and I couldn't figure it out. I'm using ASP.NET Core 2 and Ajax.
This is what the JavaScript debugger says:
XML Parsing Error: no root element found Location:
http://localhost:52617/api/favorites/ Line Number 1, Column 1:
This is my JavaScript code:
$(".js-toggle-fav").click(function (e) {
function sendPost() {
console.log("inside post send");
var button = $(e.target);
$.ajax({
type: 'POST',
url: "http://localhost:52617/api/Favorites/",
data: {"EventId": #Model.Event.EventId},
contentType: "application/json; charset=utf-8"
});
}
$.getJSON("http://localhost:52617/api/favorites/#Model.Event.EventId", function (data) {
if (data == null) {
console.log("fav is null");
sendPost();
fav.addClass(toggling);
fav.text("unfav");
}
else {
console.log("fav is NOT null");
sendPost();
fav.removeClass(toggling);
fav.text("fav");
}
);
});
And my API:
[HttpPost]
public async Task<IActionResult> PostFavorite([FromBody] FavoriteDto favorite)
{
if (!ModelState.IsValid)
{
Console.WriteLine(ModelState.ValidationState.ToString());
return BadRequest(ModelState);
}
var uid = _userManager.GetUserId(HttpContext.User);
var fav = await _context.Favourites.SingleOrDefaultAsync(x => x.EventId == favorite.EventId && x.UserId == uid);
if (fav == null)
{
_context.Favourites.Add(new Favorite { EventId = favorite.EventId, UserId=uid });
}
else
{
_context.Favourites.Remove(fav);
}
try
{
await _context.SaveChangesAsync();
}
catch (DbUpdateException)
{
if (FavoriteExists(favorite.EventId))
{
return new StatusCodeResult(StatusCodes.Status409Conflict);
}
else
{
throw;
}
}
return Ok(favorite);
}
When I do this with Postman or any restclient, everything works like a charm! With Ajax, that's not the case.
NOTES:
In the same .cshtml file, there's more jQuery and JavaScript code which does something like this, and it's just working! All the solutions
I've checked on the internet didn't work, unfortunately.
The Get methods (for returning List, or single element are working!)
The problem is coming from
data: {"EventId": #Model.Event.EventId},
Instead of passing it in directly, use JSON.Stringify
var payload = {EventId: #Model.Event.EventId};
$.ajax({
type: 'POST',
url: "http://localhost:52617/api/Favorites/",
data: JSON.stringify(payload),
contentType: "application/json; charset=utf-8"
});
I'm assuming your FavoriteDto class looks something like this
public class FavoriteDto
{
public int EventId { get; set; }
}
The reason why you were getting an xml error is that the controller action
public async Task<IActionResult> PostFavorite([FromBody] FavoriteDto favorite)
could not parse 'favorite' so it was never initialised and was null. You then return ok(null) which caused a parsing error on your client side ajax when it recieved the response back from your server.
I encountered this error during a DELETE call to an ASP.Net Core 2.1 Web API. I removed the [FromBody] from the model that was expected from the DELETEcall, such as
public async Task<IActionResult> Delete(MyModel body)
{
// Just an example
await DoSomeWork(body);
return StatusCode(200);
}
but that did not solve the problem. However, after I returned some content with StatusCode, in this example just a simple string, the error was gone. So the following code should avoid the error in Firefox:
public async Task<IActionResult> Delete(MyModel body)
{
// Just an example
await DoSomeWork(body);
return StatusCode(200, "Deletion Successfull");
}
To add to the other answers, this error can also happen if jQuery ajax gets an empty string response where it expects valid JSON, or no content at all.
Check the value of the data in the done callback, if it's an empty string "" rather than undefined, you'll have to fix your server to return status code 204 (No Content) instead of an 200 (OK) with no content.
In WebAPI2 this would mean returning StatusCode(HttpStatusCode.NoContent) instead of Ok().

Why is asp.net webAPI always returning text/html?

I would like to create webservices returning json. However, I'm always getting 'text/html' as the responses content type.
First shot:
public StringContent Get()
{
List<Cell> list = new List<Cell>();
Cell c = new Cell("Cell1");
Cell c2 = new Cell("Cell2");
list.Add(c);
list.Add(c2);
return new StringContent(
Newtonsoft.Json.JsonConvert.SerializeObject(list),
Encoding.UTF8,
"application/json");
}
Responsecontent: System.Net.Http.StringContent
second shot:
public List<Cell> Get()
{
Cell c = new Models.Cell("Cell1");
List<Cell> list = new List<Cell>();
list.Add(c);
return list;
}
Responsecontent: System.Collections.Generic.List`1[TestApp.Models.Cell]
This is how I access the endpoint:
$.ajax({
url: "http://localhost:54787/Cell/Get",
type: "GET",
contentType:"application/json",
accepts: {
text: "application/json"
},
success: function (response) {
$("#result").html(JSON.parse(response));
},
error: function (xhr, status) {
alert("error");
}
});
If you have no good reason to do serialization manually, you should use Web API default mechanism by returning object instead of StringContent. For example, you can change your method to return List<Cell> directly.
public List<Cell> Get()
{
// return List<Cell> just like you write a typical method
}
This way, you will not get text/html anymore. However, you will still get XML in Chrome. It is because Chrome's default HTTP Accept header contains application/xml, and it is supported by default in Web API. If you have no need to support XML result, so you can remove it by the following code during startup (maybe in Global.asax)
GlobalConfiguration.Configuration.Formatters.XmlFormatter.SupportedMediaTypes.Clear();
PS: If you don't know whether you need XML or not, then you don't need it.

Pending Requests When Posting Data to ASP.NET web API Controller from ASP.NET MVC Controller

Im working on a SOA project using, ASP.NET MVC and ASP.NET WEB API. Both projects are inside different solutions, so we have a mvc site and a web api site and the mvc site calls the web api site using http methods operations.
With that cenario in mind, im facing a strange behaviour when POSTING some data to the web API from my mvc controller. When I do GET operations everything works fine. Here's what Im doing:
Inside my cshtml view, i have a js code that do an ajax call to my MVC controller:
> JS Code inside Create.cshtml:
var create = (function () {
var _init = function (container, starter, url) {
$(starter).autocomplete({
minLength: 3,
focus: function () {
return false;
},
select: function (event, ui) {
var terms = this.value.replace(" ", "").split(',');
terms.pop();
terms.push(ui.item.label);
terms.push("");
this.value = terms.join(",");
return false;
},
search: function (event, ui) {
if (_extractLast(this.value).length < 3)
return false;
return true;
},
source: function (request, response) {
_execute(request, response, url);
}
});
};
var _extractLast = function (term) {
var termArray = term.split(',');
var lastTerm = termArray[termArray.length - 1];
return lastTerm.replace(' ', '');
};
var _execute = function (request, response, url) {
var responseCallback = response;
$.ajax({
url: '\Comunication\Contacts',
async: true,
type: 'POST',
dataType: 'json',
data: { termo: _extractLast(request.term) },
success: function (data) {
responseCallback($.map(data, function (item) {
return { label: item.Name, value: item.Id }
}));
}
});
}
return { init: _init };})();
And from inside my controller I call a method from a specialized Service layer to do a POST request to my WEB API SITE doing the SOA thing, as you can see bellow:
> The MVC CONTROLLER CODE:
public class ComunicationController : BaseController{
[HttpPost]
public async Task<JsonResult> Contacts(string termo)
{
var contacts = new ContactServices.ConsumeAsync(new ContactParameter{Term: termo});
return Json(contacts, JsonRequestBehavior.AllowGet);
}
}
> The Layer SERVICE CODE:
public class ContactServices{
public async Task<List<ContactsDTO>> ConsumeAsync(ContactParameter parameter)
{
using (System.Net.Http.HttpClient httpClient = new System.Net.Http.HttpClient())
{
httpClient.BaseAddress = new Uri("http://localhost:123456/api/comunication/contacts");
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await httpClient.PostAsJsonAsync(new Uri("http://localhost:123456/api/comunication/contacts"), parameter);
return JsonConvert.DeserializeObjectAsync<ResponseProxy<List<ContactsDTO>>>(response.Content.ReadAsStringAsync().Result).Result.data;
}
}
}
My WEB API SITE accept the request, process the request like expected, no concerns here. But when I return the HttpResponseMessage from my WEB API CONTROLLER to the MVC Site under a POST operation, the request response hangs on forever pending status according to my network monitor....
So basically my request is hanging, the Protocol status is Pending and I never get the request response, heres is a pic:
Pending Status from My Post Request
As I said, when I do GET requests to my web api, everything works fine but when I do POST requests this happens...
Your problem lies in this line:
return JsonConvert.DeserializeObjectAsync<ResponseProxy<List<ContactsDTO>>>(response.Content.ReadAsStringAsync().Result).Result.data;
The use of the Result property of your Task generates a deadlock on you MVC controller. If an API is asynchronous, then always await the task.
var content = await response.Content.ReadAsStringAsync();
var deserialized = await JsonConvert.DeserializeObjectAsync<ResponseProxy<List<ContactsDTO>>>(content);
return deserialized.data;
To learn more about why you should never block on async methods check this great post:
Don't Block on Async Code

Deserialize HttpError ModelState

Wit the latest webapi bits I now have
return Request.CreateErrorResponse(HttpStatusCode.BadRequest, ModelState);
My client is an MVC website that reads the value from the response. I finally got it to read the HttpError object from the response, but loading the ModelState is... not intuitive to say the least.
Is there a cleaner way to write this?
var httpError = response.Read<HttpError>();
var errors = httpError["ModelState"] as JObject;
foreach (var error in errors)
foreach (var message in error.Value.Values<string>())
{
modelState.AddModelError(error.Key, message);
}
While an error response, as you've identified, can readily be deserialised into an HttpError, the ModelState value within it only ends up as a JObject.
You've probably already tried something like:
var errors = ((JObject)errorMsg["ModelState"])
.ToObject<ModelStateDictionary>();
Or maybe:
var errors = ((JObject)errorMsg["ModelState"])
.ToObject<Dictionary<string, ICollection<ModelError>>>();
And found that it won't convert. The best alternative I could find was:
var errors = ((JObject)errorMsg["ModelState"])
.ToObject<Dictionary<string, IList<string>>>();
Which makes your iteration marginally neater:
foreach (var err in errors)
foreach (var msg in err.Value)
{
modelStateDictionary.AddModelError(err.Key, msg);
}
The problem, as I see it, is a string won't deserialise to a ModelError instance, and is further compounded by ModelErrorCollection unfortunately hiding its base class constructor of Collection(IList list).
How are you making the API calls? Client-side? If so, I noticed Brad Wilson using the below format for receiving AJAX responses via jQuery in a recent demo. This may not be what you're looking for but it's been working well for me:
$.ajax({
url: "/apicontorller/action/",
contentType: "application/json",
type: "POST",
data: jsonData,
statusCode: {
200: function (data) {
},
404: function () {
alert('404');
},
400: function () {
alert('invalid');
},
500: function () {
alert('error');
},
// ...
}
});
The nice thing about this is you can handle the actual HTTP status codes. In my experience, error messages/data can be accessed by providing a parameter to the predicate function. I'm still learning about REST and the WebAPI platform so if anyone can provide a better implementation, I certainly welcome it!

Resources