Lazarus 1.8.4
Indy 10 (March 2020)
Windows Server 2016
I'm trying to build a little "Dialer" module for a contact list I've built in Lazarus. After fiddling for months, I was finally able to translate all the RingCentral API settings into Postman, then into Indy HTTP settings, to accomplish obtaining a token. That part works reliably. However, I am unable to do a RingOut API. I always get 404 not found. The examples in other programming languages given by RingCentral in their docs don't translate well into Indy.
if htpCall.Connected then
htpCall.Request.Connection := 'close'; // Close the socket after using it to obtain the token
htpCall.Request.Clear;
htpCall.Request.Accept := 'application/json';
htpCall.Request.ContentType := 'application/json';
htpCall.Request.Username := '<redacted>'; // Client ID
htpCall.Request.Password := '<redacted>'; // Client Secret
htpCall.Request.CustomHeaders.Clear;
htpCall.Request.CustomHeaders.FoldLines := False;
htpCall.Request.BasicAuthentication := False;
htpCall.Request.CustomHeaders.AddValue('Authorization', 'Bearer ' + acc_tkn);
prms_lst.Add('from=<redacted>');
prms_lst.Add('to=<redacted>');
prms_lst.Add('callerId=<redacted>');
prms_lst.Add('country=Canada');
prms_lst.Add('playPrompt=true');
// Remove the text ".devtest" for the production version of your code, and replace the phone account with the real main phone number
htpCall.Post('https://platform.devtest.ringcentral.com/restapi/v1.0/account/<redacted>/extension/101/ring-out', prms_lst);
UPDATE:
If I leave the TIdHTTP open, then run the code above, I get a different error.
Sent:
POST /restapi/v1.0/account//extension/101/ring-out HTTP/1.1
Content-Type: application/json
Content-Length: 82
Authorization: Basic
Host: platform.devtest.ringcentral.com
Accept: application/json
User-Agent: Mozilla/3.0 (compatible; Indy Library)
from=&to=&callerId=&country=Canada&playPrompt=true
Rcvd:
HTTP/1.1 401 Unauthorized
Server: nginx
Date: Tue, 25 Aug 2020 01:50:24 GMT
Content-Type: application/json
Content-Length: 175
Connection: keep-alive
WWW-Authenticate: Bearer realm="RingCentral REST API"
RCRequestId: 57c97008-e675-11ea-87cf-005056bb81b6
{
"errorCode" : "AGW-402",
"message" : "Invalid Authorization header",
"errors" : [ {
"errorCode" : "AGW-402",
"message" : "Invalid Authorization header"
} ]
}
In the Odata 4.01 URL conventions it says that for GET requests with extremely long filter expressions you can append /$query to the resource path of the URL, use the POST verb instead of GET, and pass the query options part of the URL in the request body. If I try that with my service I get back a 404.
Does the /$query endpoint need to be manually created in the back end or is this something odata is supposed to take care of transparently? I've been searching like crazy but I'm having trouble finding anything about how to implement this.
To support this you add app.UseODataQueryRequest() to your startup somewhere before app.UseRouting()
The framework then transforms ~/$query POST requests into GET requests which are handled by the HttpGet action methods on your controller (source).
Documentation is here (although currently not up to date)
For a complete sample have a look here
One way to avoid this is wrapping the request in a batch request
You can resolve long url with $batch query.
Good post from Hassan Habib https://devblogs.microsoft.com/odata/all-in-one-with-odata-batch//
All what you should do is:
Allow batching in Startup.cs
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseODataBatching(); <---- (1)
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.Select().Filter().Expand().OrderBy();
endpoints.MapODataRoute(
routeName: "api",
routePrefix: "api",
model: GetEdmModel(),
batchHandler: new DefaultODataBatchHandler()); <---- (2)
});
}
Request batch query with body, that contains long url request
POST http://localhost/api/$batch
Content-Type: multipart/mixed; boundary=batch_mybatch
body:
--batch_mybatch
Content-Type: application/http
Content-Transfer-Encoding: binary
GET http://localhost/api/students/735b6ae6-485e-4ad8-a993-36227ac82851 HTTP/1.1 <--long url requst
OData-Version: 4.0
OData-MaxVersion: 4.0
Accept: application/json;odata.metadata=minimal
Accept-Charset: UTF-8
User-Agent: Microsoft ADO.NET Data Services
--batch_mybatch
I have a web api method:
[HttpPost]
public IActionResult Post([FromBody]string entryInJson)
{ .... }
This method is hit by a remote domain, so CORS is needed. As i wasn't successful in finding the relevant cs file in my mvc6 app, i used this solution in web.config:
<httpProtocol>
<customHeaders>
<add name="Access-Control-Allow-Origin" value="*" />
</customHeaders>
</httpProtocol>
When I do the post, using online tools like http://requestmaker.com/, i am getting this:
Request Headers Sent:
POST /api/Entries/ HTTP/1.1
Host: justalk.tukuoro.com
Accept: */*
Content-Type: text/plain
Content-Length: 179
Response Headers:
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 0
Expires: -1
Location: https://justalk.tukuoro.com/Account/Login?ReturnUrl=%2FHome%2FError
X-Powered-By: ASP.NET
Access-Control-Allow-Origin: *
Date: Mon, 16 May 2016 05:25:53 GMT
Now, it seems like it is trying to redirect me (302) but to the error page, and it is not sending anything back from the post method, which returns:
return new ObjectResult(true);
I am not sure what is going on:
Am I getting to the right controller? i have code that is supposed to log the incoming string but it does not create the log.
If it is not working? why? is the response header telling me something I don't understand?
Am I doing it right?
thank you for your time!
if you want send json post request this entryInJson must be object type not string
for example
[HttpPost]
public IActionResult Post([FromBody]EntryInJson entryInJson)
{ .... }
chat comment
the code doing the http request to my post method - how does it convert this class "RemoteEntry" to JSON data
http://www.newtonsoft.com/json/help/html/serializingjson.htm
Update :
302 is not an error but a redircet to a tempoaray location. you could resend the request to the new location received in the response header.
and it's propably a server misconfiguration issue then.
modSecurity firewall blocking your requests an making automatics redirections (302).
I'm currently working on site that uses various Ajax-requests to save, load and autocomplete data. It is build using C#, MVC and JQuery. All actions on the MVC controllers require the users to be authorized, and we use IdentityServer3 for authentication. It was installed using NuGet, and the current version is 2.3.0.
When I open the page and push buttons, everything is working just fine. The problem seem to occur when a certain session expires. If I stay idle for a while, and try to use an Ajax-function, it generates the following error:
XMLHttpRequest cannot load https://identityserver.domain.com/connect/authorize?client_id=Bar&redirect_uri=http%3a%2f%2flocalhost%3a12345&response_mode=form_post&response_type=id_token+token&scope=openid+profile+email+phone+roles [...]. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:12345' is therefore not allowed access.
From what I know about Ajax, the problem itself is pretty simple. The MVC site has lost track of the current session, and it is asking the client to authenticate again. The response I get from the Ajax-request is a "302 Found", with a Location-header that points to our IdentityServer. The IdentityServer happens to be on another domain, and while this works fine when you are performing regular HTTP-requests, it does not work particularly well for Ajax-requests. The "Same Origin Policy" is straight up blocking the Ajax-function from authenticating. If I refresh the page, I will be redirected to the IdentityServer and authenticate normally. Things will then go back to normal for a few minutes.
The solution is probably to add an extra header in the response message from the IdentityServer, that explicitly states that cross-origin requests are allowed for this service.
I am currently not getting this header from the IdentityServer (checked in Fiddler).
According to the docs, it should be enabled by default. I have checked that we have indeed enabled CORS this way:
factory.CorsPolicyService = new Registration<ICorsPolicyService>(new DefaultCorsPolicyService { AllowAll = true });
This is one of my clients:
new Client
{
Enabled = true,
ClientName = "Foo",
ClientId = "Bar",
ClientSecrets = new List<Secret>
{
new Secret("Cosmic")
},
Flow = Flows.Implicit,
RequireConsent = false,
AllowRememberConsent = true,
AccessTokenType = AccessTokenType.Jwt,
PostLogoutRedirectUris = new List<string>
{
"http://localhost:12345/",
"https://my.domain.com"
},
RedirectUris = new List<string>
{
"http://localhost:12345/",
"https://my.domain.com"
},
AllowAccessToAllScopes = true
}
These settings do not work. I am noticing that I have an extra forward slash in the URIs here, but if I remove them, I get the default IdentityServer-error that states that the client is not authorized (wrong URI). If I deploy the site (instead of running a localhost debug), I use the domain name without a trailing slash, and I get the exact same behaviour as I do in debug. I do notice that there is no trailing slash in the error message above, and I figured this could be the problem until I saw the same thing in the deployed version of the site.
I also made my own policy provider, like this:
public class MyCorsPolicyService : ICorsPolicyService
{
public Task<bool> IsOriginAllowedAsync(string origin)
{
return Task.FromResult(true);
}
}
... and I plugged it into the IdentityServerServiceFactory like this:
factory.CorsPolicyService = new Registration<ICorsPolicyService>(new MyCorsPolicyService());
The idea is for it to return true regardless of origin. This did not work either; exactly the same results as before.
I've read about a dozen other threads on this particular subject, but I'm getting nowhere. To my knowledge, we are not doing anything unusual when it comes to the setup of the different sites. It's all pretty much out-of-the-box. Any advice?
----- UPDATE -----
The problem persists. I have now tried some fresh tactics. I read somewhere that cookie authentication was bad for Ajax-requests, and that I should be using bearer tokens instead. I set this up in Ajax like this:
$(function () {
$(document).ajaxSend(function (event, request, settings) {
console.log("Setting bearer token.");
request.setRequestHeader("Authorization", "Bearer " + $bearerToken);
});
});
Both the console in Chrome and Fiddler confirms that the token is indeed present and sent by JQuery. The token I use comes from the access_token-property on claims principal object from HttpContext.GetOwinContext().Authentication.User.
This didn't do much. I still get a 302-response from the server, and Fiddler reveals that the token is not sent on the following Ajax-request (which is a GET-request) to the IdentityServer.
From there, I read this thread:
Handling CORS Preflight requests to ASP.NET MVC actions
I tried to put this code in to the startup.cs of the IdentityServer, but there does not appear to be a "preflight" request going in. All I see in Fiddler is this (from the beginning):
1 - The initial Ajax-request from the client to the MVC controller:
POST http://localhost:12345/my/url HTTP/1.1
Host: localhost:12345
Connection: keep-alive
Content-Length: pretty long
Authorization: Bearer <insert long token here>
Origin: http://localhost:12345
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Accept: application/json, text/javascript, */*; q=0.01
X-Requested-With: XMLHttpRequest
Referer: http://localhost:12345/my/url
Accept-Encoding: gzip, deflate
Accept-Language: nb-NO,nb;q=0.8,no;q=0.6,nn;q=0.4,en-US;q=0.2,en;q=0.2
Cookie: OpenIdConnect.nonce.<insert 30 000 lbs of hashed text here>
param=fish&morestuff=salmon&crossDomain=true
2 - The redirect response from the MVC controller:
HTTP/1.1 302 Found
Cache-Control: private
Location: https://identityserver.domain.com/connect/authorize?client_id=Bar&redirect_uri=http%3a%2f%2flocalhost%3a12345%2f&response_mode=form_post&response_type=id_token+token&scope=openid+profile+email [...]
Server: Microsoft-IIS/10.0
X-AspNetMvc-Version: 5.2
X-AspNet-Version: 4.0.30319
Set-Cookie: OpenIdConnect.nonce.<lots of hashed text>
X-SourceFiles: <more hashed text>
X-Powered-By: ASP.NET
Date: Fri, 15 Jan 2016 12:23:08 GMT
Content-Length: 0
3 - The Ajax-request to the IdentityServer:
GET https://identityserver.domain.com/connect/authorize?client_id=Bar&redirect_uri=http%3a%2f%2flocalhost%3a12345%2f&response_mode=form_post&response_type=id_token+token&scope=openid+profile+email [...]
Host: identityserver.domain.com
Connection: keep-alive
Accept: application/json, text/javascript, */*; q=0.01
Origin: http://localhost:12345
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://localhost:12345/my/url
Accept-Encoding: gzip, deflate, sdch
Accept-Language: nb-NO,nb;q=0.8,no;q=0.6,nn;q=0.4,en-US;q=0.2,en;q=0.2
4 - The response from IdentityServer3
HTTP/1.1 302 Found
Content-Length: 0
Location: https://identityserver.domain.com/login?signin=<some hexadecimal id>
Server: Microsoft-IIS/8.5
Set-Cookie: SignInMessage.<many, many, many hashed bytes>; path=/; secure; HttpOnly
X-Powered-By: ASP.NET
Date: Fri, 15 Jan 2016 12:23:11 GMT
5 - The meltdown of Chrome
XMLHttpRequest cannot load https://identityserver.domain.com/connect/authorize?client_id=Bar&blahblahblah. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:12345' is therefore not allowed access.
I was having a similar issue using OWIN Middleware for OpenIDConnect with a different identity provider. However, the behavior occurred after 1 hour instead of 5 minutes. The solution was to check if the request was an AJAX request, and if so, force it to return 401 instead of 302. Here is the code that performed this:
app.UseOpenIdConnectAuthentication(new OpenIdConnectAuthenticationOptions
{
ClientId = oktaOAuthClientId,
Authority = oidcAuthority,
RedirectUri = oidcRedirectUri,
ResponseType = oidcResponseType,
Scope = oauthScopes,
SignInAsAuthenticationType = "Cookies",
UseTokenLifetime = true,
Notifications = new OpenIdConnectAuthenticationNotifications
{
AuthorizationCodeReceived = async n =>
{
//...
},
RedirectToIdentityProvider = n => //token expired!
{
if (IsAjaxRequest(n.Request))
{
n.Response.StatusCode = 401;//for web api only!
n.Response.Headers.Remove("Set-Cookie");
n.State = NotificationResultState.HandledResponse;
}
return Task.CompletedTask;
},
}
});
Then, I used an Angular interceptor to detect a statusCode of 401, and redirected to the authentication page.
I came across this problem as well and UseTokenLifetime = false was not solving the problem since you loose the token validity on STS.
When I tried to reach the authorized api method, I still got 401 even if I was valid on Owin.
The solution I found is keeping UseTokenLifetime = true as default but to write a global ajax error handler (or angular http interceptor) something like this:
$.ajaxSetup({
global: true,
error: function(xhr, status, err) {
if (xhr.status == -1) {
alert("You were idle too long, redirecting to STS") //or something like that
window.location.reload();
}
}});
to trigger the authentication workflow.
I had this issue recently, it was caused by the header X-Requested-With being sent with the AJAX request. Removing this header or intercepting it and handling it with a 401 will put you on the right track.
If you don't have this header, the issue is most likely being caused by a different header triggering the Access-Control-Allow-Origin response.
As you found, nothing you do in Identity Server regarding CORS will solve this.
As it turns out, the problem was in the client configuration in MVC. I was missing the UseTokenLifetime property, which should have been set to false.
app.UseOpenIdConnectAuthentication(
new OpenIdConnectAuthenticationOptions
{
ClientId = "Bar",
Scope = "openid profile email phone roles",
UseTokenLifetime = false,
SignInAsAuthenticationType = "Cookies"
[...]
For some reason, IdentityServer sets all these cookies to expire within 5 minutes of them being distributed. This particular setting will override IdentityServer's tiny expiration time, and instead use aprox. 10 hours, or whatever the default is in your client application.
One could say that this is good enough for solving the problem. It will however inevitably return if the user decides to spend 10 hours idling on the site, clicking nothing but Ajax-buttons.
https://github.com/IdentityServer/IdentityServer3/issues/2424
Assumptions:
.NET Framework 4.8 WebForms
OWIN-based auth lib i.e. Microsoft.Owin.Security.OpenIdConnect v4.2.2.0
UseOpenIdConnectAuthentication() with Azure AD endpoint
UseTokenLifetime=true
In Layout.Master:
$.ajaxSetup({
global: true,
error: function (xhr, status, err) {
if (xhr.status == 401) {
window.location.reload();
}
}
});
In startup.cs:
app.UseOpenIdConnectAuthentication(new OpenIdConnectAuthenticationOptions
{
...
Notifications = new OpenIdConnectAuthenticationNotifications()
{
...
RedirectToIdentityProvider = RedirectToIdentityProvider
}
});
...
public Task RedirectToIdentityProvider(RedirectToIdentityProviderNotification<OpenIdConnectMessage, OpenIdConnectAuthenticationOptions> context)
{
if (IsAjaxRequest(context.Request))
{
context.Response.StatusCode = 401;
context.Response.Headers.Remove("Set-Cookie");
context.State = NotificationResultState.HandledResponse;
}
}
public bool IsAjaxRequest(this IOwinRequest request)
{
if (request == null)
{
throw new ArgumentNullException("Woopsie!");
}
var context = HttpContext.Current;
var isCallbackRequest = false;
if (context != null && context.CurrentHandler != null && context.CurrentHandler is System.Web.UI.Page page)
{
isCallbackRequest = page.IsCallback;
}
return isCallbackRequest || (request.Cookies["X-Requested-With"] == "XMLHttpRequest") || (request.Headers["X-Requested-With"] == "XMLHttpRequest");
}
I am building a nodeJS server which allows users to login using an AJAX post on a client application. The server responds with a cookie that, after successful login, keeps track of the user. I can't however seem to get the client application to send the cookie back to the server. It never seems to respond to the server including the cookie it just recieved.
On the fist call that's made to the server, I login with my credentials. the server responds with this:
In the response headers:
Set-Cookie:SID=0xtW36rYCiV; path=/; expires=Tue, 24-Jun-2014 14:14:51 GMT; secure
In the subsequent request headers:
No cookie is sent back to the server
In my client application I am using the following code:
jQuery.ajax( {
url: this.domain + this.url,
async: this.async,
type: 'POST',
dataType: this.dataType,
crossDomain: true,
cache: true,
accepts: 'text/plain',
data: this.postVars,
success: this.onData.bind( this ),
error: this.onError.bind( this ),
xhrFields: {
withCredentials: true
}
});
Note I am using the xhrFields.
And in my node server I am responding with this: (notice I am including all the CORRS variables)
if ( request.headers.origin )
{
if ( request.headers.origin.match( /mydomain\.net/ ) || request.headers.origin.match( /appname\.mydomain\.com/ ) || request.headers.origin.match( /localhost/ )
|| request.headers.origin.match( /localhost\.com/ )
|| request.headers.origin.match( /localhost\.local/ )
|| request.headers.origin.match( /localtest\.com/ ) )
{
response.setHeader( 'Access-Control-Allow-Origin', request.headers.origin );
response.setHeader( 'Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE,OPTIONS' );
response.setHeader( 'Access-Control-Allow-Headers', 'Content-Type, Authorization, Content-Length, X-Requested-With' );
response.setHeader( "Access-Control-Allow-Credentials", "true" );
};
}
response.writeHead( 200, { "Content-Type": "application/json" });
response.end( JSON.stringify( json ) );
}
I have also edited my windows hosts file to include these test domains so that I don't have to use an IP or localhost iteself:
127.0.0.1 localhost
127.0.0.1 tooltest.com
127.0.0.1 localhost.com
127.0.0.1 localhost.local
But no matter what I do, or which of the above hosts I use, it never seems to work. It seems to only be related to localhost via ajax because if I go directly to the server url in question - it works.
EDIT 1
So for example - I open the client application and try to login to the server at foo.com/user/log-in. The headers for the request and response are as follows:
Request:
Remote Address:188.xxx.xxx.xx:7000
Request URL:https://foo.com:7000/user/log-in
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:undefined
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Connection:keep-alive
Content-Length:45
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Host:foo.com:7000
Origin:http://localhost
Referer:http://localhost/animate-ts/trunk/bin/
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
Form Dataview sourceview URL encoded
user:mat
password:testpassword
rememberMe:true
Response:
Response Headersview source
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:Content-Type, Authorization, Content-Length, X-Requested-With
Access-Control-Allow-Methods:GET,PUT,POST,DELETE,OPTIONS
Access-Control-Allow-Origin:http://localhost
Connection:keep-alive
Content-Type:application/json
Date:Tue, 24 Jun 2014 14:14:21 GMT
Set-Cookie:SID=0xtW36rYCiV; path=/; expires=Tue, 24-Jun-2014 14:14:51 GMT; secure
Transfer-Encoding:chunked
As I undertand it, Foo.com has told the browser to store the cookie 0xtW36rYCiV. However when I make the very next request (foo.com/user/is-logged-in) to see if the user is logged in, no cookies are sent to foo. Foo.com looks for the cookie with the ID 0xtW36rYCiV but can't find anything. When I look at the 2nd request in dev tools I can see:
Does anyone have any other ideas? I thought I covered everything in the above, but im starting to think it just wont work :(
Thanks
Mat
My cookie isn't marked secure, and I'm facing this problem trying to send a GET request to my server (https://auth.mywebsite.com) from http://localhost:3000/.
I have the same auth flow as OP here.
First, sign in to auth.website.com, and receive back cookie in the response header:
set-cookie: Authorization=Bearer%20blahblah.blahblah.blahblah; Path=/; Expires=Wed, 05 Aug 2020 02:07:57 GMT; HttpOnly
Note, I don't have the Secure flag in set-cookie.
I noticed in Chrome devtools there was this warning that popped up when hovering over the set-cookie line:
"This Set-Cookie didn't specify a "SameSite" attribute and was defaulted to "SameSite=Lax," and was blocked because it came from a cross-site response which was not the response to a top-level navigation. The Set-Cookie had to have been set with "SameSite=None" to enable cross-site usage."
This might be the answer to my problem. But, I don't understand why this was working yesterday, but not today. No code has changed in between.
This issue occurs on both on Chrome and Safari.
The MDN docs has a good description of all the flags for Set-Cookie.
This is not a full answer, but perhaps some extra context for others facing similar issues.
I had this problem, but #aspillers commment lead to the solution for me. The cookie was marked secure, but I was doing the request over HTTP instead of HTTPS. Once I got my debugger switched over to HTTPS it started working.
For chrome:
Go to chrome://flags/
Disable "SameSite by default cookies"
Restart Chrome