Spinnaker & Okta integration failing - okta

Question

Scenerio:
Upgraded Spinnaker to 1.12.0. No other config changes that would impact this integration (we had to modify an s3 IAM because it quit working). Okta integration stopped working. Public key was reissued during install process for the ingress, may be relevant?
SAML-TRACE shows payload getting to okta and back
Spinnaker throws two different errors depending on browser and how I get there.
Direct link to deck url: (500) No IDP was configured, please update included metadata with at least one IDP (seen in browser and gate)
Okta "chicklet" in okta dashboard: (401) Authentication Failed: Incoming SAML message is invalid
Config details (again none of this changed):
Downloading metadata directly
JKS is being leveraged and is valid
service url is confirmed
alias for JKS is confirmed

Answer 1

I had this issue as well when upgrading from 1.10.13 to 1.12.2. I found lots of these error messages in Gate's logs:
2019-02-19 05:31:30.421 ERROR 1 --- [.0-8084-exec-10] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw e
xception [org.opensaml.saml2.metadata.provider.MetadataProviderException: No IDP was configured, please update included metadata with at least one IDP] with root cause
org.opensaml.saml2.metadata.provider.MetadataProviderException: No IDP was configured, please update included metadata with at least one IDP
at org.springframework.security.saml.metadata.MetadataManager.getDefaultIDP(MetadataManager.java:795) ~[spring-security-saml2-core-1.0.2.RELEASE.jar:1.0.2.RELEASE]
at org.springframework.security.saml.context.SAMLContextProviderImpl.populatePeerEntityId(SAMLContextProviderImpl.java:157) ~[spring-security-saml2-core-1.0.2.RELEASE.jar
:1.0.2.RELEASE]
at org.springframework.security.saml.context.SAMLContextProviderImpl.getLocalAndPeerEntity(SAMLContextProviderImpl.java:127) ~[spring-security-saml2-core-1.0.2.RELEASE.ja
r:1.0.2.RELEASE]
at org.springframework.security.saml.SAMLEntryPoint.commence(SAMLEntryPoint.java:146) ~[spring-security-saml2-core-1.0.2.RELEASE.jar:1.0.2.RELEASE]
at org.springframework.security.web.access.ExceptionTranslationFilter.sendStartAuthentication(ExceptionTranslationFilter.java:203) ~[spring-security-web-4.2.9.RELEASE.jar
:4.2.9.RELEASE]
...
After downgrading back to 1.10.13, I upgraded to the next version, 1.11.0, and found that's when the issue started. Eventually, I looked at Gate's logs from the launch of the Container and found:
2019-02-20 22:31:40.132 ERROR 1 --- [0.0-8084-exec-3] o.o.s.m.provider.HTTPMetadataProvider : Error retrieving metadata from https://000000000000.okta.com/app/00000000000000000/sso/saml/metadata
javax.net.ssl.SSLException: Error in hostname verification
at org.opensaml.ws.soap.client.http.TLSProtocolSocketFactory.verifyHostname(TLSProtocolSocketFactory.java:241) ~[openws-1.5.4.jar:na]
at org.opensaml.ws.soap.client.http.TLSProtocolSocketFactory.createSocket(TLSProtocolSocketFactory.java:186) ~[openws-1.5.4.jar:na]
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) ~[commons-httpclient-3.1.jar:na]
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) ~[commons-httpclient-3.1.jar:na]
...
This lead me to realize that the TLS Certificate was being rejected by Gate. Not sure why it suddenly started failing the check. Up to this point, I had it configured as:
$ hal config security authn saml edit --metadata https://000000000000.okta.com/app/00000000000000000/sso/saml/metadata
I ended up downloading the metadata file and redeploying with halyard.
$ wget https://000000000000.okta.com/app/00000000000000000/sso/saml/metadata
$ hal config security authn saml edit --metadata "${PWD}/metadata"
$ hal config version edit --version 1.12.2
$ hal deploy apply
Opened up a private browser window as suggested by the Spinnaker documentation and Gate started redirecting to Okta correctly again.
Issue filed, https://github.com/spinnaker/spinnaker/issues/4017.

Answer 2

So I ended up finding the answer. The tomcat config changed apparently in spinnaker in later versions for gate.
I created this snippet in ~/.hal/default/profiles/gate-local.yml
server:
tomcat:
protocolHeader: X-Forwarded-Proto
remoteIpHeader: X-Forwarded-For
internalProxies: .*
Deployed spinnaker and it was back to working.