HMTLUNIT + Algorithm constraints check failed on signature algorithm: MD2withRSA - htmlunit

Question

I am spider a site with MD2 certificate ... and it's throwing the following exception
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificates do not conform to algorithm constraints
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) ~[na:1.8.0_191]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) ~[na:1.8.0_191]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) ~[na:1.8.0_191]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) ~[na:1.8.0_191]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[na:1.8.0_191]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[na:1.8.0_191]
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[na:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[na:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[na:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[na:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[na:1.8.0_191]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[httpclient-4.5.6.jar:4.5.6]
at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:184) ~[htmlunit-2.33.jar:2.33]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:394) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) ~[httpclient-4.5.6.jar:4.5.6]
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:192) ~[htmlunit-2.33.jar:2.33]
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1402) ~[htmlunit-2.33.jar:2.33]
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1321) ~[htmlunit-2.33.jar:2.33]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:394) ~[htmlunit-2.33.jar:2.33]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:315) ~[htmlunit-2.33.jar:2.33]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:466) ~[htmlunit-2.33.jar:2.33]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:448) ~[htmlunit-2.33.jar:2.33]
at ...
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:73) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:83) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) [spring-test-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.junit.runner.JUnitCore.run(JUnitCore.java:137) [junit-4.12.jar:4.12]
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) [junit-rt.jar:na]
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) [junit-rt.jar:na]
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) [junit-rt.jar:na]
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) [junit-rt.jar:na]
Caused by: java.security.cert.CertificateException: Certificates do not conform to algorithm constraints
at sun.security.ssl.AbstractTrustManagerWrapper.checkAlgorithmConstraints(SSLContextImpl.java:1127) ~[na:1.8.0_191]
at sun.security.ssl.AbstractTrustManagerWrapper.checkAdditionalTrust(SSLContextImpl.java:1051) ~[na:1.8.0_191]
at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:993) ~[na:1.8.0_191]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ~[na:1.8.0_191]
... 58 common frames omitted
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: MD2withRSA
at sun.security.provider.certpath.AlgorithmChecker.check(AlgorithmChecker.java:278) ~[na:1.8.0_191]
at sun.security.ssl.AbstractTrustManagerWrapper.checkAlgorithmConstraints(SSLContextImpl.java:1123) ~[na:1.8.0_191]
... 61 common frames omitted
I am running htmlunit 2.33 ... with Oracle JDK 1.8.0 update 191
this are my options for web client
try {
try (final WebClient webClient = new WebClient(BrowserVersion.getDefault())) {
webClient.setJavaScriptTimeout(5000);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setJavaScriptEnabled(false);
webClient.waitForBackgroundJavaScript(30000);
webClient.getOptions().setActiveXNative(true);
webClient.getOptions().setAppletEnabled(true);
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setUseInsecureSSL(true);
I already try to
add website certificate to trust store (cacerts)
remove from java.security lines:
jdk.jar.disabledAlgorithms
jdk.tls.disabledAlgorithms
and is still blowing, any thoughts on this ?

Answer 1

This is reproducible and related to
webClient.getOptions().setUseInsecureSSL(true);
For our implementation of InsecureTrustManager we have to switch from X509TrustManager to X509ExtendedTrustManager. Have done the code change (github commit) and will made a new snapshot build available soon. Have a look at HtmlUnit#twitter; will tweet if a new snapshot build is available.