Custom spring-based rest API with Apache Knox gateway - spring

I'm trying to do a POC in which I need to protect my cluster through Apache Knox, and my home-grown rest API will sit behind the Knox. How should user authentication work with Knox Ranger, and how will I protect my data access?

Let's start with the service.xml file. It should probably look like the much simpler version below. You would only need the more complex forms if you need to apply specific rules to specific parts of requests or responses. Note that ideally only one route would be required but the ** in Knox means one or more path levels (not zero or more). So without the first route Knox wouldn't send requests to the root /Test_Web_App path to the service.
<service role="TEST_WEB_APP" name="Test_Web_App" version="0.0.1">
<routes>
<route path="/Test_Web_App/?**">
<route path="/Test_Web_App/**?**">
</routes>
</service>
If you wanted to be very specific about it this would be the equivalent. Here the <rewrite apply=""> identifies a specific named rule in rewrite.xml and <rewrite to=""> identifies what part of the request and or response the rule should be applied to.
<service role="TEST_WEB_APP" name="Test_Web_App" version="0.0.1">
<routes>
<route path="/Test_Web_App/?**">
<rewrite apply="TEST_WEB_APP/Test_Web_App" to="request.url"/>
</route>
<route path="/Test_Web_App/**?**">
<rewrite apply="TEST_WEB_APP/Test_Web_App/query" to="request.url"/>
</route>
</routes>
</service>
Now for the rewrite.xml. Yours was pretty close. All I added was a name (i.e. path) resulting in {path=**} to both the pattern and template of the second rule. In the <rule pattern=""> this is used to give a name to the values that are extracted from the matched pattern. In <rewrite template=""> the name is used to populate part of the URL being created with named values extracted from the matched pattern.
<rules>
<rule dir="IN" name="TEST_WEB_APP/Test_Web_App" pattern="*://*:*/**/Test_Web_App/?{**}">
<rewrite template="{$serviceUrl[TEST_WEB_APP]}/?{**}"/>
</rule>
<rule dir="IN" name="TEST_WEB_APP/Test_Web_App/query" pattern="*://*:*/**/Test_Web_App/{path=**}?{**}">
<rewrite template="{$serviceUrl[TEST_WEB_APP]/{path=**}?{**}"/>
</rule>
</rules>

You will need to provide a service definition for your custom REST API to Apache Knox. This will allow Knox to:
Recognize the URLs of the APIs incoming requests to your spring based service and be able to route requests to it
Know how to rewrite specific content such as URLs or other sensitive content from the responses to redirect the client back through gateway when appropriate.
See: http://knox.apache.org/books/knox-0-7-0/dev-guide.html#Service+Definition+Files within the developers guide for how to provide a service definition.
Once a service definition is in place, you just need to add a <service> element within a Knox topology to indicate where the actual spring based service is running. It would be something like this:
<service>
<role>SERVICE</role>
<url>http://url.to.your.service/v1/...</url>
</service>
This is also described in the same section of the developers guide.
You may also find the users guide helpful for additional examples of service declarations within the topology.
You can look at those examples and compare them to the service definition files for those existing services to see how service roles map. This will help you do the same for your own service.
For authentication details, see http://knox.apache.org/books/knox-0-7-0/user-guide.html#Authentication within the users guide for instructions in setting up LDAP based authentication. There are other authentication and federation provider implementations that may be of interest to you as well. You can find them within the users guide too.
Feel free to engage the dev or user email lists for Apache Knox as well.

These are my 2 xml file please look in to it if i have mad any mistake or what else i need to do
**rewrite.xml
<rule dir="IN" name="TEST_WEB_APP/Test_Web_App" pattern="*://*:*/**/Test_Web_App/?{**}">
<rewrite template="{$serviceUrl[TEST_WEB_APP]}/?{**}"/>
</rule>
<rule dir="IN" name="TEST_WEB_APP/Test_Web_App/query" pattern="*://*:*/**/Test_Web_App/{**}?{**}">
<rewrite template="{$serviceUrl[TEST_WEB_APP]/{**}?{**}"/>
</rule>
</rules>
**service.xml
<service role="TEST_WEB_APP" name="Test_Web_App" version="0.0.1">
<routes>
<route path="/Test_Web_App/**">
<rewrite apply="TEST_WEB_APP/Test_Web_App/**" to="response.body" />
</route>
<route path="/Test_Web_App/**?**">
<rewrite apply="TEST_WEB_APP/Test_Web_App/**?**" to="response.body"/>
</routes>
</service>

Related

Substitute WCF Endpoint Address in VSTS Release Management

I'm using VSTS to Build and Deploy a Windows Service to several different servers. Within the App.config for my Windows Service I have a reference to a WCF Service that I'm calling:
<system.serviceModel>
<bindings>
<wsHttpBinding>
<binding name="WSHttpBinding_IMobileSyncService">
<security mode="TransportWithMessageCredential">
<transport clientCredentialType="None" />
<message clientCredentialType="UserName" />
</security>
</binding>
</wsHttpBinding>
</bindings>
<client>
<endpoint address="https://192.111.11.111/1.0.0.18/MobileService.svc"
binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_IMobileService"
contract="MSSRef.IMobileService" name="WSHttpBinding_IMobileService" />
</client>
I'm currently using the same Build for all servers but using the Release Management stage of VSTS to substitute values in my App.config (for example, Connection Strings and server specific values). This is working perfectly.
With reference to the above code example however, how would I substitute the endpoint address out for values specific to each server? I'm not sure how to target this specific block in the config file.
There are some extensions that can do it, for example:
Release Management Utility tasks, related article: Using Tokenization (Token Replacement) for Builds/Releases in vNext/TFS 2015
Replace Tokens
You also can do it programming through PowerShell.

Apache Camel REST DSL 405 Method Not Allowed

I'm currently developing a REST Application with Apache Camel (using camel-spring), and getting some confusing behaviour. I have a set of endpoints defined in REST DSL, some are simply proxy requests to another server, and some others are passed on to routes I've defined for data aggregation. The DSL looks as follows:
<rest>
<!-- Specific DSL requests -->
<get uri="/v1/aggregate/data" consumes="application/json" produces="application/json">
<to uri="direct:dataEnrichment" />
</get>
<get uri="/v1/customer/{custId}/devices" consumes="application/json" produces="application/json">
<to uri="direct:getCustomerDevices" />
</get>
<get uri="/v1/login/{custId}" consumes="application/json" produces="application/json">
<to uri="direct:getCustomer" />
</get>
<get uri="/v1/status" consumes="application/json" produces="application/json">
<to uri="direct:statusInfo" />
</get>
<!-- Proxy requests -->
<post uri="/v1?matchOnUriPrefix=true&chunked=false">
<to uri="direct:proxyOut" />
</post>
<get uri="/v1?matchOnUriPrefix=true&chunked=false">
<to uri="direct:proxyOut" />
</get>
<put uri="/v1?matchOnUriPrefix=true&chunked=false">
<to uri="direct:proxyOut" />
</put>
<delete uri="/v1?matchOnUriPrefix=true&chunked=false">
<to uri="direct:proxyOut" />
</delete>
</rest>
The idea is that any requests with no exact matching URI get proxied through to another system (which is not Apache Camel). This saves me having to write a definition for every REST API on the other system (there are a lot).
All was working well, until I added the two requests with {custId} in the URI. Those requests work fine, but every time I try a URI that should get proxied through, I get 405 Method Not Allowed.
EDIT:
I should also have mentioned that I'm using Jetty as the REST component. Camel is running stand-alone, using org.apache.camel.spring.Main to start it up. I'm calling it with Postman at this stage, and the 405 response appears to be coming from Jetty/Camel.
The REST configuration looks like this (The security handler is using the Jetty BasicAuthenticator):
<restConfiguration contextPath="/mobilegateway/api" scheme="http"
host="0.0.0.0" bindingMode="off"
enableCORS="true"
component="jetty" port="8079">
<endpointProperty key="handlers" value="securityHandler" />
<endpointProperty key="sessionSupport" value="true" />
<endpointProperty key="httpClient.idleTimeout" value="30000" />
</restConfiguration>
The Proxy requests all send to the direct:proxyOut route, which looks like this:
<route id="proxyOutbound">
<description>A simple outbound route to proxy REST requests.</description>
<from uri="direct:proxyOut" />
<removeHeaders pattern="Authorization" />
<to
uri="http://{{remoteAddress}}/data/json?bridgeEndpoint=true&throwExceptionOnFailure=false" />
</route>
The intention is that everything after the /v1 in the URI is passed over in the proxied request. I've checked using Wireshark, and the request is not getting proxied. If I remove the routes with {custId} in the path, everything works fine.
Am I doing something wrong or is this a bug in camel/camel-spring?
Camel bug, the details of which can be found here:
https://issues.apache.org/jira/browse/CAMEL-11951
Its not clear whom is returning 405, is it the proxied backend you call, or does it not call that backend at all.
However when you proxy HTTP via Camel, then you may need to remove some CamelHttp* headers that can interfere.
So try adding
<removeHeaders pattern="CamelHttp*" />
The similar bug is still in place, I've filed one here:
https://issues.apache.org/jira/browse/CAMEL-15363
I've managed to make a workaround for my case, see my comment in the bugtracker

Custom JwtSecurityTokenHandler not being invoked

I have a .NET MVC application that uses Azure Active Directory for Auth.
I'm trying to add a custom JWTSecurityTokenHandler to authenticate a console app that performs some basic GET requests against the app. However every request just gets redirected to the Azure AD login page instead of being passed to the JWT handler (my breakpoints and logging statements in the handler are not being hit). Any ideas?
Web.config:
<system.identityModel>
<identityConfiguration>
<audienceUris>
<add value="https://localhost:44300/" />
</audienceUris>
<securityTokenHandlers>
<add type="QS.Admin.Infrastructure.MyJwtHandler, QS.Admin" />
<securityTokenHandlerConfiguration>
<certificateValidation certificateValidationMode="None" />
</securityTokenHandlerConfiguration>
</securityTokenHandlers>
<issuerNameRegistry type="System.IdentityModel.Tokens.ValidatingIssuerNameRegistry, System.IdentityModel.Tokens.ValidatingIssuerNameRegistry">
<authority name="https://[myaccessdomain].accesscontrol.windows.net/">
<keys>
<add thumbprint="[thumbprint]" />
</keys>
<validIssuers>
<add name="https://[myaccessdomain].accesscontrol.windows.net/" />
</validIssuers>
</authority>
</issuerNameRegistry>
<!--certificationValidationMode set to "None" by the the Identity and Access Tool for Visual Studio. For development purposes.-->
<certificateValidation certificateValidationMode="None" />
</identityConfiguration>
</system.identityModel>
<system.identityModel.services>
<federationConfiguration>
<cookieHandler requireSsl="false" />
<wsFederation passiveRedirectEnabled="true" issuer="https://[myaccessdomain].accesscontrol.windows.net/v2/wsfederation" realm="https://localhost:44300/" requireHttps="false" />
</federationConfiguration>
</system.identityModel.services>
in addition to the above advice,
the jwtsecuritytokenhandlers responsibility is to validate a jwt and serve claims upstream. I don't see session management anywhere in your config, if that is missing, each call to the host will require obtaining a new token from ACS.
The settings in web.config look right.
Couple of things to check :
Make sure ACS is configured to issue JWT tokens for your realm.
If you plug in the JwtSecurityTokenHandler from MS - is it getting hit? This will help in isolating the issue to your custom handler versus settings in ACS or web.config.

Cross domain work around with IIS7.5

I am building an application that is ready for release however I have come up against the cross domain scripting problem within my ajax calls.
I am using IIS7.5 and I have the following applications setup
Marketing Application - 'www.mydomain.com' - Used as the marketing website and for registering for a new business.I have a rule that forces any non www. to be www
Business Application - 'https://newbusiness.mydomain.com' - Used to access their services they have subscribed to
API Application - 'https://newbusiness.mydomain.com/api/.......' - Used for business application to do crud operations
API application is an application running under the business application. Containing a number of secured methods and some anonymous ones.
With jquery I am having trouble from my marketing application trying to call the api application. Currently if I want to call the api from my marketing site I am doing something like 'https://api.mydomain.com/api/whatevermethod' because it is cross domain it fails.
Is there anyway that I can somehow map 'https://api.mydomain.com/api/whatevermethod' to be just 'https://mydomain.com/api/whatevermethod' as this would get me around the cross domain problem.
You need to allow cross domain calls in the IIS. You can do this by setting the Access-Control-Allow-Origin and Access-Control-Allow-Headers
ex:
in root web.config
<system.webserver>
<httpprotocol>
<customheaders>
<add name="access-control-allow-origin" value="*" />
<add name="access-control-allow-headers" value="content-type" />
</customheaders>
</httpprotocol>
</system.webserver>
taken from here
Just in case you also got an error 500 from Amila's answer, here's what the whole web.config file content should look like
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<httpProtocol>
<customHeaders>
<add name="access-control-allow-origin" value="*" />
<add name="access-control-allow-headers" value="content-type" />
</customHeaders>
</httpProtocol>
</system.webServer>
</configuration>

Why use a Url Rewrite Filter in Spring?

A Spring app I'm using declares a Tuckey UrlReWrite Filter and then sets up a rewrite rule as the following:
<rule>
<from>^/(.*)$</from>
<to last="true">/app/$1</to>
</rule>
Why do this?
Will Spring not be able to recognize requests that do not go to the /app/ url?
Otherwise what is the advantage of this redirect?
Imagine that you want Spring MVC's DispatcherServlet to handle all URLs in your application excluding the URLs of static content. If you try to do it directly with <url-pattern>/</url-pattern>, this mapping will take precedence over the static content.
With rewrite filter you can specify exclusions for the static content, like this:
<urlrewrite default-match-type="wildcard">
<rule>
<from>/staticContentHere/**</from>
<to>/staticContentHere/$1</to>
</rule>
<rule>
<from>/**</from>
<to>/app/$1</to>
</rule>
</urlrewrite>
EDIT:
Note that since Spring 3.0.4 there is a <mvc:resources /> declaration. With this declaration, DispatcherServlet will serve static content from the /resources subfolder of your webapp, so rewriting will not be needed (see http://blog.springsource.com/2010/07/22/spring-mvc-3-showcase/).

Resources