Indy doesn't get response text returned with error 404 - httpresponse

Question

My HTTP server returns custom 404 error text when REST route is not found:
{"sessionIdent":"051F-dUen7-tetW-kNf82-WxT","Details":[{"messageCode":60,"messageCategory":"","messageText":"No matching route for \"POST \/Warehouse\/A1\/Orders\/execute\""}]}
Following JavaScript code displays this response text in browser just fine:
function httpReq(method, url, headers, jsonStr, userName, password) {
try
{
var xmlhttp = new XMLHttpRequest();
xmlhttp.open(method, url, true);
xmlhttp.onreadystatechange = function () {
console.log("onreadystatechange");
if (xmlhttp.readyState == 4) {
console.log("ready");
console.log(xmlhttp.status);
console.log(xmlhttp.responseText);
}
}
// Send the request
xmlhttp.setRequestHeader('Cache-Control', 'no-cache, max-age=0');
xmlhttp.setRequestHeader('Content-Type', 'application/json; charset=utf-8');
xmlhttp.setRequestHeader('Session-Ident', '051F-dUen7-tetW-kNf82-WxT');
xmlhttp.setRequestHeader('Accept', 'application/json');
if (headers) {
var headerKeys = Object.keys(headers);
Object.keys(headers).forEach(key => {
xmlhttp.setRequestHeader(key, headers[key]);
});
}
if ((userName !== "") && (password !== ""))
{
xmlhttp.setRequestHeader("Authorization", "Basic " + btoa(userName + ":" + password));
}
console.log("before send");
xmlhttp.send(jsonStr);
console.log("after send");
}
catch (ex)
{
console.log(ex);
}
}
Indy's TIdHTTP raises an EIdHTTPProtocolException exception with message HTTP/1.1 404 Not Found instead of my response text inside.
When I use the hoNoProtocolErrorException option:
_client.HTTPOptions := _client.HTTPOptions + [hoNoProtocolErrorException];
exception is not raised any more, but response text is empty.
procedure TFormRestTest._httpSend(AMethod, APath, AHeaders, ABody: string);
var
queryData, replyData: TStream;
resultText: string;
begin
queryData := TStringStream.Create(ABody, TEncoding.UTF8);
try
replyData := TMemoryStream.Create;
try
_client.Request.ContentType := 'application/json';
_client.Request.CharSet := 'UTF-8';
_client.Request.BasicAuthentication := True;
_client.Request.Username := 'Username';
_client.Request.Password := 'Password';
_client.Request.CustomHeaders.Clear;
_client.Request.CustomHeaders.Text := AHeaders;
_client.DoRequest(AMethod, APath, queryData, replyData, []);
replyData.Position := 0;
resultText = ReadStringAsCharset(replyData, _client.Response.CharSet)]);
_log(resultText); //resultText is empty
finally
replyData.Free();
end;
finally
queryData.Free();
end;
end;
How can I retrieve my response body?

Answer 1

When I use the hoNoProtocolErrorException option ... exception is not raised any more, but response text is empty.
That is by design. When disabling the exception, you need to also enable the hoWantProtocolErrorContent option to actually receive the response's body data into your replyData stream, eg:
_client.HTTPOptions := _client.HTTPOptions + [hoNoProtocolErrorException, hoWantProtocolErrorContent];
This is explained in more detail on the following article on Indy's Changelog Blog:
New TIdHTTP flags and OnChunkReceived event
Three new flag have been added to the TIdHTTP.HTTPOptions property:
...
hoWantProtocolErrorContent: when an HTTP error response is received, TIdHTTP normally reads and discards the response’s message body and then raises EIdHTTPProtocolException (the message body is available in the EIdHTTPProtocolException.ErrorMessage property). If the hoNoProtocolErrorException flag is enabled, or the ResponseCode number is specified in the request’s AIgnoreReplies parameter, then no EIdHTTPProtocolException is raised, as the caller would like to process the ResponseCode manually. Normally TIdHTTP would still discard the message body, though. If this new flag is enabled, the message body will no longer be discarded, it will be saved in the caller’s target TStream like a successful response would be. This flag is disabled by default to preserve existing behavior to discard error message bodies.
...
Based on your earlier comment to another question, you seem to not have the hoWantProtocolErrorContent option available in your version of Indy. In which case, you are using a very outdated version of Indy and should upgrade to the latest version from Indy's GitHub repository.
UPDATE: If that is not an option for you, for whatever reason, then you have no choice but to catch the EIdHTTPProtocolException and read the body content from its ErrorMessage property, eg:
try
_client.DoRequest(AMethod, APath, queryData, replyData, []);
replyData.Position := 0;
resultText := ReadStringAsCharset(replyData, _client.Response.CharSet)]);
except
on E: EIdHTTPProtocolException do
resultText := E.ErrorMessage;
end;