Shibboleth setup - reverse proxy wtih Nginx to Jetty 9.4.9 - shibboleth

I'm setting up Shibboleth to use SAML 2.0. This setup contains reverse proxy using Nginx to Jetty 9.49 which points to shibboleth idp.war file.
For testing, I'm using the django Service provider from this example here.
I'm using self-sign certificates.
I can access both https://idp.localhost/idp/shibboleth and https://idp.localhost/idp/status, but not https://idp.localhost/idp/profile/SAML2/POST/SSO. From shibboleth log file, I'm getting this error each time I browse https://idp.localhost/idp/profile/SAML2/POST/SSO:
2018-04-25 18:20:47,746 - ERROR
[org.opensaml.saml.common.binding.security.impl.ReceivedEndpointSecurityHandler:200]
- Message Handler: SAML message intended destination endpoint 'https://idp.localhost/idp/profile/SAML2/POST/SSO' did not match the
recipient endpoint 'http://idp.localhost/idp/profile/SAML2/POST/SSO'
==> idp-warn.log <== 2018-04-25 18:20:47,746 - ERROR [org.opensaml.saml.common.binding.security.impl.ReceivedEndpointSecurityHandler:200]
- Message Handler: SAML message intended destination endpoint 'https://idp.localhost/idp/profile/SAML2/POST/SSO' did not match the
recipient endpoint 'http://idp.localhost/idp/profile/SAML2/POST/SSO'
==> idp-process.log <== 2018-04-25 18:20:47,748 - WARN [net.shibboleth.idp.profile.impl.WebFlowMessageHandlerAdaptor:202] -
Profile Action WebFlowMessageHandlerAdaptor: Exception handling
message org.opensaml.messaging.handler.MessageHandlerException: SAML
message failed received endpoint check at
org.opensaml.saml.common.binding.security.impl.ReceivedEndpointSecurityHandler.checkEndpointURI(ReceivedEndpointSecurityHandler.java:202)
==> idp-warn.log <== 2018-04-25 18:20:47,748 - WARN [net.shibboleth.idp.profile.impl.WebFlowMessageHandlerAdaptor:202] -
Profile Action WebFlowMessageHandlerAdaptor: Exception handling
message org.opensaml.messaging.handler.MessageHandlerException: SAML
message failed received endpoint check at
org.opensaml.saml.common.binding.security.impl.ReceivedEndpointSecurityHandler.checkEndpointURI(ReceivedEndpointSecurityHandler.java:202)
==> idp-process.log <== 2018-04-25 18:20:47,749 - WARN [org.opensaml.profile.action.impl.LogEvent:105] - A non-proceed event
occurred while processing the request: MessageAuthenticationError
==> idp-warn.log <== 2018-04-25 18:20:47,749 - WARN [org.opensaml.profile.action.impl.LogEvent:105] - A non-proceed event
occurred while processing the request: MessageAuthenticationError
Where did I go wrong?

I think that a request was done in HTTPS and a response returned in clear HTTP.
This kind of problem sometimes happen when jetty was non installed from source or with other corrupted packages. It's difficult to diagnose in this situation.
Try to compile jetty from source, clean first your old installation.
I've also build up a complete IDP/SP setup procedure with apache2 or nginx/ tomcat or jetty setup here, with ansible:
https://github.com/peppelinux/Ansible-Shibboleth-IDP-SP-Debian9
It takes 4 minutes to give you what you expect, It needs an existing LDAP installation.

Related

Springboot parsing HTTP HTTPS

I have found such error message in my Springboot app logs:
Oct 20 14:22:14 ip-10-0-78-113 web: 2020-10-20 14:22:14.388 INFO 3802 — [nio-8081-exec-5] o.apache.coyote.http11.Http11Processor : Error parsing HTTP request header
Oct 20 14:22:14 ip-10-0-78-113 web: Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
Oct 20 14:22:14 ip-10-0-78-113 web: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986
...
I got to know, that its probably because somebody / some app is calling my endpoint with use of HTTPS instead of HTTP. I was able to log those requests by including in properties:
logging.level.org.springframework.web: trace
logging.level.org.apache: trace
Now I have received unparsed text, something like this:
Received [ ÷ ó÷=EVq&GŵH¡à}C¶vÓÂíÿ#,¾ )L /ð^ w» e<Ú5~ñ¨7ëÌaÒ*l×e $À/À+À0À,̨̩À ÀÀ
À / 5
localhost ÿ
3 & $ ¬·lä¹]Ö·a6ððòvcd»[7Ûj V - + ]
Is it possible to decode/decrypt it, and find out what the actual API call was?
I had same problem & make it working by using http:// instead of https://

unable to uprade marklogic data hub framework using gradle

I am trying to follow the recommendation to upgrade the DHF using gradle but I am running through an issue that I cannot het my head around.
The build succeeds but the redeployment fails
Any idea on how to fix this ?
note that the login info is provided properly in the gradle.properties
> Task :hubDeploySecurity FAILED
Task ':hubDeploySecurity' is not up-to-date because:
Task has not declared any outputs despite executing actions.
Deploying app DHF with config dirs: [/src/main/hub-internal-config, /src/main/ml-config]
Executing command [com.marklogic.appdeployer.command.security.DeployPrivilegesCommand] with sort order [5]
Will read and merge resource files in each config path before saving any resources
Processing files in directory: /src/main/hub-internal-config/security/privileges
Checking to see if Configuration Management API is available at: /manage/v3
Sending JSON POST request as user 'tkadmin' (who should have the 'manage-admin' and 'security' roles) to path: /manage/v3
Error occurred while sending POST request to /manage/v3; logging request body to assist with debugging: {}
Processing file: /src/main/hub-internal-config/security/privileges/dhf-internal-data-hub.json
Processing file: /src/main/hub-internal-config/security/privileges/dhf-internal-entities.json
Processing file: /src/main/hub-internal-config/security/privileges/dhf-internal-mappings.json
Processing file: /src/main/hub-internal-config/security/privileges/dhf-internal-trace-ui.json
Processing files in directory: /src/main/ml-config/security/privileges
Checking to see if Configuration Management API is available at: /manage/v3
Sending JSON POST request as user 'tkadmin' (who should have the 'manage-admin' and 'security' roles) to path: /manage/v3
Error occurred while sending POST request to /manage/v3; logging request body to assist with debugging: {}
Merging payloads that reference the same resource
Checking to see if Configuration Management API is available at: /manage/v3
Sending JSON POST request as user 'tkadmin' (who should have the 'manage-admin' and 'security' roles) to path: /manage/v3
Error occurred while sending POST request to /manage/v3; logging request body to assist with debugging: {}
Checking for existence of resource: dhf-internal-data-hub
Sending XML GET request as user 'tkadmin' (who should have the 'manage-admin' and 'security' roles) to path: /manage/v2/privileges
Logging HTTP response body to assist with debugging: {"errorResponse": {"statusCode":401,
"status":"Unauthorized",
"message":"401 Unauthorized"
}
}
:hubDeploySecurity (Thread[Execution worker for ':',5,main]) completed. Took 0.01 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':hubDeploySecurity'.
> 401 Unauthorized: [{"errorResponse": {"statusCode":401,
"status":"Unauthorized",
"message":"401 Unauthorized"
}
}]
Assuming you have followed the DHF upgrade matrix:
https://docs.marklogic.com/datahub/5.2/upgrade.html
You probably ran the Gradle with an incorrect Admin interface & Security user. As such, the hubUpdate REST API requests will fail.
Try below see if it works:
Step 2
gradle hubUpdate -i -PmlUsername=admin -PmlPassword={admin-password} -Penvironment={env-name}
Step 4
gradle mlRedeploy -i -PmlUsername=admin -PmlPassword={admin-password} -Penvironment={env-name}

Spinnaker & Okta integration failing

Scenerio:
Upgraded Spinnaker to 1.12.0. No other config changes that would impact this integration (we had to modify an s3 IAM because it quit working). Okta integration stopped working. Public key was reissued during install process for the ingress, may be relevant?
SAML-TRACE shows payload getting to okta and back
Spinnaker throws two different errors depending on browser and how I get there.
Direct link to deck url: (500) No IDP was configured, please update included metadata with at least one IDP (seen in browser and gate)
Okta "chicklet" in okta dashboard: (401) Authentication Failed: Incoming SAML message is invalid
Config details (again none of this changed):
Downloading metadata directly
JKS is being leveraged and is valid
service url is confirmed
alias for JKS is confirmed
I had this issue as well when upgrading from 1.10.13 to 1.12.2. I found lots of these error messages in Gate's logs:
2019-02-19 05:31:30.421 ERROR 1 --- [.0-8084-exec-10] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw e
xception [org.opensaml.saml2.metadata.provider.MetadataProviderException: No IDP was configured, please update included metadata with at least one IDP] with root cause
org.opensaml.saml2.metadata.provider.MetadataProviderException: No IDP was configured, please update included metadata with at least one IDP
at org.springframework.security.saml.metadata.MetadataManager.getDefaultIDP(MetadataManager.java:795) ~[spring-security-saml2-core-1.0.2.RELEASE.jar:1.0.2.RELEASE]
at org.springframework.security.saml.context.SAMLContextProviderImpl.populatePeerEntityId(SAMLContextProviderImpl.java:157) ~[spring-security-saml2-core-1.0.2.RELEASE.jar
:1.0.2.RELEASE]
at org.springframework.security.saml.context.SAMLContextProviderImpl.getLocalAndPeerEntity(SAMLContextProviderImpl.java:127) ~[spring-security-saml2-core-1.0.2.RELEASE.ja
r:1.0.2.RELEASE]
at org.springframework.security.saml.SAMLEntryPoint.commence(SAMLEntryPoint.java:146) ~[spring-security-saml2-core-1.0.2.RELEASE.jar:1.0.2.RELEASE]
at org.springframework.security.web.access.ExceptionTranslationFilter.sendStartAuthentication(ExceptionTranslationFilter.java:203) ~[spring-security-web-4.2.9.RELEASE.jar
:4.2.9.RELEASE]
...
After downgrading back to 1.10.13, I upgraded to the next version, 1.11.0, and found that's when the issue started. Eventually, I looked at Gate's logs from the launch of the Container and found:
2019-02-20 22:31:40.132 ERROR 1 --- [0.0-8084-exec-3] o.o.s.m.provider.HTTPMetadataProvider : Error retrieving metadata from https://000000000000.okta.com/app/00000000000000000/sso/saml/metadata
javax.net.ssl.SSLException: Error in hostname verification
at org.opensaml.ws.soap.client.http.TLSProtocolSocketFactory.verifyHostname(TLSProtocolSocketFactory.java:241) ~[openws-1.5.4.jar:na]
at org.opensaml.ws.soap.client.http.TLSProtocolSocketFactory.createSocket(TLSProtocolSocketFactory.java:186) ~[openws-1.5.4.jar:na]
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) ~[commons-httpclient-3.1.jar:na]
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) ~[commons-httpclient-3.1.jar:na]
...
This lead me to realize that the TLS Certificate was being rejected by Gate. Not sure why it suddenly started failing the check. Up to this point, I had it configured as:
$ hal config security authn saml edit --metadata https://000000000000.okta.com/app/00000000000000000/sso/saml/metadata
I ended up downloading the metadata file and redeploying with halyard.
$ wget https://000000000000.okta.com/app/00000000000000000/sso/saml/metadata
$ hal config security authn saml edit --metadata "${PWD}/metadata"
$ hal config version edit --version 1.12.2
$ hal deploy apply
Opened up a private browser window as suggested by the Spinnaker documentation and Gate started redirecting to Okta correctly again.
Issue filed, https://github.com/spinnaker/spinnaker/issues/4017.
So I ended up finding the answer. The tomcat config changed apparently in spinnaker in later versions for gate.
I created this snippet in ~/.hal/default/profiles/gate-local.yml
server:
tomcat:
protocolHeader: X-Forwarded-Proto
remoteIpHeader: X-Forwarded-For
internalProxies: .*
Deployed spinnaker and it was back to working.

"Failed to call HTTP server" error after upgrading SonarQube from 6.0 to 6.7.2

I have upgraded a SonarQube server from 6.0 to 6.7.2. Everything is fine, except that, after logging in as user "admin" and navigating to tab Administration, clicking on subtab System yields an error message
An error has occurred: please contact your administrator.
In file web.log:
2018.03.06 11:38:24 ERROR web[AWH6436/ICo3FeIjAAAz][o.s.s.w.WebServiceEngine] Fail to process request http://<HOST NAME>:20295/sonarqube/api/system/info
java.lang.IllegalStateException: Failed to call HTTP server of process [key='ce', ipcIndex=3, logFilenamePrefix=ce]
at org.sonar.ce.http.CeHttpClientImpl.call(CeHttpClientImpl.java:171)
at org.sonar.ce.http.CeHttpClientImpl.retrieveSystemInfo(CeHttpClientImpl.java:60)
at org.sonar.server.platform.ws.StandaloneSystemInfoWriter.write(StandaloneSystemInfoWriter.java:53)
at org.sonar.server.platform.ws.InfoAction.handle(InfoAction.java:64)
at org.sonar.server.ws.WebServiceEngine.execute(WebServiceEngine.java:103)
at org.sonar.server.ws.WebServiceFilter.doFilter(WebServiceFilter.java:86)
at org.sonar.server.platform.web.MasterServletFilter$GodFilterChain.doFilter(MasterServletFilter.java:126)
...
Caused by: java.io.IOException: Server returned HTTP response code: 503 for URL: http://10.145.61.55
The IP address is that of the server. There is no mention of any port in the error message, I don't know if this has any relation with the error.
It seems that the Web component cannot access the Compute Engine component. However nothing particular was found in file ce.log which ends with a trace Compute Engine is operational.
I followed the short migration guide in https://docs.sonarqube.org/display/SONAR/Upgrading.
Environment: Linux, JDK 8, Oracle DB
Does anyone as a clue? Many thanks in advance.
For whoever who might stumble on the same issue: it was an error in my proxy configuration. Increasing the logging level by setting sonar.log.level=DEBUG in file sonar.properties yielded the following new traces in file web.log:
2018.03.07 11:52:04 DEBUG web[AWIAFTpArQRjrTzMAAAg][s.n.w.p.h.HttpURLConnection] sun.net.www.MessageHeader#668ee27b5 pairs: {GET http://127.0.0.1:40413/systemInfo HTTP/1.1: null}{User-Agent: SonarQube 6.7.2.37468 # AWHmcYEU8aGLjZ5sPfjj Java/1.8.0_60}{Host: 127.0.0.1:40413}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Proxy-Connection: keep-alive}
2018.03.07 11:52:04 DEBUG web[AWIAFTpArQRjrTzMAAAg][s.n.w.p.h.HttpURLConnection] sun.net.www.MessageHeader#527214044 pairs: {null: HTTP/1.1 301 Moved Permanently}{Server: BlueCoat-Security-Appliance}{Location: http://10.145.61.55}{Connection: Close}
2018.03.07 11:52:04 DEBUG web[AWIAFTpArQRjrTzMAAAg][s.n.w.p.h.HttpURLConnection] Redirected from http://127.0.0.1:40413/systemInfo to http://10.145.61.55
2018.03.07 11:52:04 ERROR web[AWIAFTpArQRjrTzMAAAg][o.s.s.w.WebServiceEngine] Fail to process request http://<HOST NAME>:20295/sonarqube/api/system/info
The proxy was redirecting the Compute engine URL http://127.0.0.1:40413/systemInfo to http://10.145.61.55.
The solution was to add 127.0.0.1 to http.nonProxyHosts in file sonar.properties.

Apache Directory Authentication Failed?

I am trying to connect the OpenLdap through Apache Directory Studio and its working upto Network Parameter and check Network Parameter working fine.
But Next step i am trying to Authentication and its failed with below message.
The authentication failed
- The response queue has been emptied, no response was found. org.apache.directory.api.ldap.model.exception.LdapException: The
response queue has been emptied, no response was found. at
org.apache.directory.ldap.client.api.LdapNetworkConnection.bind(LdapNetworkConnection.java:1327)
at
org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper$2.run(DirectoryApiConnectionWrapper.java:368)
at
org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.runAndMonitor(DirectoryApiConnectionWrapper.java:1269)
at
org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.doBind(DirectoryApiConnectionWrapper.java:460)
at
org.apache.directory.studio.connection.core.io.api.DirectoryApiConnectionWrapper.bind(DirectoryApiConnectionWrapper.java:306)
at
org.apache.directory.studio.connection.core.jobs.CheckBindRunnable.run(CheckBindRunnable.java:79)
at
org.apache.directory.studio.connection.ui.RunnableContextRunner$1.run(RunnableContextRunner.java:127)
at
org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:119)
Caused by:
org.apache.directory.api.ldap.model.exception.LdapException: TimeOut
occurred at
org.apache.directory.ldap.client.api.LdapNetworkConnection.bind(LdapNetworkConnection.java:1305)
... 7 more
The response queue has been emptied, no response was found.

Resources