Error while sending mails using EWSJavaAPI - exchange-server

I am trying to send mails from my java web application by using EWS Java api. But ia m getting an error when sending mails. We have Exchange 2013 server installed.
Here is my code
ExchangeService service = new ExchangeService();
ExchangeCredentials credentials = new WebCredentials("username","password");
service.setCredentials(credentials);
try {
service.setUrl(new URI("host/ews/Exchange.asmx"));
service.setTraceEnabled(true);
EmailMessage msg = new EmailMessage(service);
msg.setSubject("Hello world!");
msg.setBody(MessageBody
.getMessageBodyFromText("Sent using the EWS Managed API."));
msg.getToRecipients().add("emailAddress");
msg.send();
}
catch (URISyntaxException e) {
e.printStackTrace();
}
}
And here is the trace report and error log
<Trace Tag="EwsRequestHttpHeaders" Tid="1" Time="2014-08-14 05:58:04Z">
POST /ews/Exchange.asmx HTTP/1.1
Content-type : text/xml; charset=utf-8
Accept-Encoding : gzip,deflate
Keep-Alive : 300
User-Agent : ExchangeServicesClient/0.0.0.0
Connection : Keep-Alive
Accept : text/xml
</Trace>
<Trace Tag="EwsRequest" Tid="1" Time="2014-08-14 05:58:04Z">
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><soap:Header><t:RequestServerVersion Version="Exchange2010_SP1"></t:RequestServerVersion></soap:Header><soap:Body><m:CreateItem MessageDisposition="SendOnly"><m:Items><t:Message><t:Subject>Hello world!</t:Subject><t:Body BodyType="HTML">Sent using the EWS Managed API.</t:Body><t:ToRecipients><t:Mailbox><t:EmailAddress>emailAddress</t:EmailAddress></t:Mailbox></t:ToRecipients></t:Message></m:Items></m:CreateItem></soap:Body></soap:Envelope>
Exception in thread "main" microsoft.exchange.webservices.data.ServiceRequestException: The request failed. sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at microsoft.exchange.webservices.data.ServiceRequestBase.validateAndEmitRequest(ServiceRequestBase.java:719)
at microsoft.exchange.webservices.data.SimpleServiceRequestBase.internalExecute(SimpleServiceRequestBase.java:36)
at microsoft.exchange.webservices.data.MultiResponseServiceRequest.execute(MultiResponseServiceRequest.java:140)
at microsoft.exchange.webservices.data.ExchangeService.internalCreateItems(ExchangeService.java:461)
at microsoft.exchange.webservices.data.ExchangeService.createItem(ExchangeService.java:530)
at microsoft.exchange.webservices.data.Item.internalCreate(Item.java:214)
at microsoft.exchange.webservices.data.EmailMessage.internalSend(EmailMessage.java:125)
at microsoft.exchange.webservices.data.EmailMessage.send(EmailMessage.java:253)
at SendEmail.main(SendEmail.java:28)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1747)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1209)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:135)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:943)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1188)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:506)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at microsoft.exchange.webservices.data.HttpClientWebRequest.executeRequest(HttpClientWebRequest.java:300)
at microsoft.exchange.webservices.data.ServiceRequestBase.emit(ServiceRequestBase.java:326)
at microsoft.exchange.webservices.data.ServiceRequestBase.validateAndEmitRequest(ServiceRequestBase.java:714)
... 8 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:323)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:217)
at sun.security.validator.Validator.validate(Validator.java:218)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:126)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:209)
at microsoft.exchange.webservices.data.EwsX509TrustManager.checkServerTrusted(EwsX509TrustManager.java:62)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1201)
... 27 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:318)
... 33 more

i got this problem solved. Added the certificate for my exchange server in java. This site helped- https://blogs.oracle.com/gc/entry/unable_to_find_valid_certification

Related

Connect to jenkins using Jsoup connect

I am trying to retrieve the HTML of the console logs of a Jenkins build using Jsoup connect method, but I am getting the following error
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:641)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:460)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:732)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:759)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:707)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:297)
at org.jsoup.helper.HttpConnection.get(HttpConnection.java:286)
I tried Document doc = Jsoup.connect(jobURL).validateTLSCertificates(false).get();
but my Intellij shows Cannot resolve method 'validateTLSCertificates' in 'Connection'
I am also not sure how can I pass my Jenkins username and usertoken (password) to retrieve the html.
This is the code I have for connection. Not sure if it is correct.
String username = "username";
String password = "password";
String login = username + ":" + password;
String base64login = new String(Base64.encodeBase64(login.getBytes()));
doc = Jsoup.connect(jobURL)
.header("Authorization", "Basic " + base64login)
.get();
Can anyone help figure out the issue?
Thanks!

How to consume a local HTTPS ASP.NET Core 2.1 API from Android and iOS simulators

I'm working on a Xamarin.Forms app that requires to consume some data exposed in an ASP.NET Core 2.1 API, I'm developing the service and the Xamarin app so everything is running local on MAC OS.
The problem I'm facing is happening when sending a request to the ASP.NET Core API, it seems to be related to the HTTPS protocol enabled by default in ASP.NET Core 2.1, when I run the api it's enabling by default the URL https://localhost:5001 so that is the URL I'm targeting in the Xamarin App.
In Android I can send the request to the API but once the response is sent back, the xamarin app throws the following exception:
Error in AppLoginViewModel.Login Javax.Net.Ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. ---> Java.Security.Cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. ---> Java.Security.Cert.CertPathValidatorException: Trust anchor for certification path not found.
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Java.Interop.JniEnvironment+InstanceMethods.CallVoidMethod (Java.Interop.JniObjectReference instance, Java.Interop.JniMethodInfo method, Java.Interop.JniArgumentValue* args) [0x00069] in <fb2a9245146c43afbafe6628542c6726>:0
at Java.Interop.JniPeerMembers+JniInstanceMethods.InvokeAbstractVoidMethod (System.String encodedMember, Java.Interop.IJavaPeerable self, Java.Interop.JniArgumentValue* parameters) [0x00014] in <fb2a9245146c43afbafe6628542c6726>:0
at Javax.Net.Ssl.HttpsURLConnectionInvoker.Connect () [0x0000a] in <9c89490198b4401e89a0f0bf9e68c7be>:0
at Xamarin.Android.Net.AndroidClientHandler+<>c__DisplayClass43_0.<ConnectAsync>b__0 () [0x0005a] in <9c89490198b4401e89a0f0bf9e68c7be>:0
at System.Threading.Tasks.Task.InnerInvoke () [0x0000f] in <5b7c58d953774179ac771c086a8fa093>:0
at System.Threading.Tasks.Task.Execute () [0x00000] in <5b7c58d953774179ac771c086a8fa093>:0
--- End of stack trace from previous location where exception was thrown ---
at Xamarin.Android.Net.AndroidClientHandler+<DoProcessRequest>d__45.MoveNext () [0x0012e] in <9c89490198b4401e89a0f0bf9e68c7be>:0
--- End of stack trace from previous location where exception was thrown ---
at Xamarin.Android.Net.AndroidClientHandler+<SendAsync>d__40.MoveNext () [0x00230] in <9c89490198b4401e89a0f0bf9e68c7be>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.HttpClient+<SendAsyncWorker>d__49.MoveNext () [0x000ca] in <006b95df38b04983b2ec7f1bf1c32608>:0
--- End of stack trace from previous location where exception was thrown ---
at AuditoriasCiudadanas.Mobile.Core.Services.RestService+<Seguridad_ValidarEmailAuditorAsync>d__16.MoveNext () [0x00184] in /Users/jorgeramirez/VSProjects/AuditoriasCiudadanas.Mobile/src/AuditoriasCiudadanas.Mobile.Core/Services/RestService.cs:242
--- End of stack trace from previous location where exception was thrown ---
at AuditoriasCiudadanas.Mobile.Core.Services.AuthService+<ValidarEmailAuditor>d__6.MoveNext () [0x0002b] in /Users/jorgeramirez/VSProjects/AuditoriasCiudadanas.Mobile/src/AuditoriasCiudadanas.Mobile.Core/Services/AuthService.cs:28
--- End of stack trace from previous location where exception was thrown ---
at AuditoriasCiudadanas.Mobile.Core.ViewModels.AppForgotPasswordViewModel+<ValidarEmail>d__27.MoveNext () [0x0006f] in /Users/jorgeramirez/VSProjects/AuditoriasCiudadanas.Mobile/src/AuditoriasCiudadanas.Mobile.Core/ViewModels/AppForgotPasswordViewModel.cs:82
--- End of managed Javax.Net.Ssl.SSLHandshakeException stack trace ---
javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:229)
at com.android.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:192)
at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:149)
at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:112)
at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:461)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:127)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.connect(DelegatingHttpsURLConnection.java:89)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:26)
Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:661)
at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:539)
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:495)
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:418)
at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:339)
at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:94)
at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:88)
at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:208)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.verifyCertificateChain(ConscryptFileDescriptorSocket.java:404)
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:375)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:224)
... 12 more
Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
... 24 more
In iOS the behavior is the same, the xamarin app throws an exception when receiving the response
Error in AppLoginViewModel.Login System.Net.WebException: An SSL error has occurred and a secure connection to the server cannot be made. ---> Foundation.NSErrorException: Error Domain=NSURLErrorDomain Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, NSErrorPeerCertificateChainKey=(
"<cert(0x7fce57040e00) s: localhost i: localhost>"
), NSErrorClientCertificateStateKey=0, NSErrorFailingURLKey=https://localhost:5001/api/auth, NSErrorFailingURLStringKey=https://localhost:5001/api/auth NSUnderlyingError=0x600001fccc60 {Error Domain=kCFErrorDomainCFNetwork Code=-1200 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, kCFStreamPropertySSLPeerTrust=<SecTrustRef: 0x6000023ce400>, _kCFNetworkCFStreamSSLErrorOriginalValue=-9802, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9802, kCFStreamPropertySSLPeerCertificates=(
"<cert(0x7fce57040e00) s: localhost i: localhost>"
)}}, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalDataTask <31AF1DE2-4C56-4480-9F1D-0E9B7C7B8D9A>.<1>"
), _kCFStreamErrorCodeKey=-9802, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <31AF1DE2-4C56-4480-9F1D-0E9B7C7B8D9A>.<1>, NSURLErrorFailingURLPeerTrustErrorKey=<SecTrustRef: 0x6000023ce400>, NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made.}
--- End of inner exception stack trace ---
at System.Net.Http.NSUrlSessionHandler+<SendAsync>d__29.MoveNext () [0x001c3] in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/src/Foundation/NSUrlSessionHandler.cs:202
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.HttpClient+<SendAsyncWorker>d__48.MoveNext () [0x00080] in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/external/mono/mcs/class/System.Net.Http/System.Net.Http/HttpClient.cs:276
--- End of stack trace from previous location where exception was thrown ---
at AuditoriasCiudadanas.Mobile.Core.Services.RestService+<Seguridad_ValidarEmailAuditorAsync>d__16.MoveNext () [0x00184] in /Users/me/RestService.cs:242
--- End of stack trace from previous location where exception was thrown ---
at AuditoriasCiudadanas.Mobile.Core.Services.AuthService+<ValidarEmailAuditor>d__6.MoveNext () [0x0002b] in /Users/me/AuthService.cs:28
--- End of stack trace from previous location where exception was thrown ---
at AuditoriasCiudadanas.Mobile.Core.ViewModels.AppForgotPasswordViewModel+<ValidarEmail>d__27.MoveNext () [0x0006f] in /Users/me/AppForgotPasswordViewModel.cs:82
Is not possible to debug locally using HTTPS? What I need to change to be able to debug the mobile app and the api at the same time?
I will appreciate your help
I had the same problem.
I solved with the Conveyor extension (https://keyoti.com/products/conveyor/index.html?utm_source=conveyor&utm_medium=extension_moreinfo&utm_campaign=conveyor).
In netcore, disable SSL first (project property => debug)

Nutch with solr on https

Good morning,
I come to you because I have a problem with Nutch (1.14) and Solr (7.2)
so it works fine until I put SSL in place.
With Solr in http, once the crawl is finished I execute this command
bin/nutch index -Dsolr.server.url=http://127.0.0.1:8983/solr/CORENAME crawltest/crawldb/ -linkdb crawltest/linkdb/ crawltest/segments/* -filter -normalize -deleteGone
And it works very well
However, once SSL is activated and the solr server in HTTPS, it is impossible to send the data to Solr.
I have added in nutch site the following properties
<name>solr.auth</name>
<value>true</value>
<property>
<name>solr.auth.username</name>
<value>xxxx</value>
<property>
<name>solr.auth.password</name>
<value>xxxx</value>
property>
<name>solr.server.type</name>
<value>https</value>
property>
<name>solr.server.url</name>
<value>https://127.0.0.1:8983/solr/CORENAME</value>
But when I execute the previous command I get an error of this type
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:8983/solr/CORENAME
&
caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
&
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Did you succeed in sending the data to a HTTPS solr?
Thanks
EDIT
To fix this errors following SSL procedure https://lucene.apache.org/solr/guide/7_0/enabling-ssl.html
And at the end execute this
keytool -import -file /path/to/solr/solr-ssl.pem -alias solr_cert -keystore /path/to/java-cacert (jre/lib/security/cacerts)
the default password is changeit
it advances a little, once the certificate import in cacerts I do not have any more this error.
Still in the same context, after activating SSL and authentication on the solr server. I use Nutch to Crawl the urls and send the data to solr.
Since the implementation of SSL I can no longer send data to SOLR.
When i execute this bin/nutch index -Dsolr.server.url=https://localhost:8983/solr/CORE -Dsolr.auth=true -Dsolr.auth.username='solr' -Dsolr.auth.password='xxxx' crawltest/crawldb/ -linkdb crawltest/linkdb/ crawltest/segments/* -filter -normalize -deleteGone
I have the following two errors:
java.lang.Exception: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://localhost:8983/solr/CORE: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /solr/CORE/update. Reason:
<pre> Unauthorized</pre></p>
</body>
</html>
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://localhost:8983/solr/CORE: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /solr/CORE/update. Reason:
<pre> Unauthorized</pre></p>
</body>
</html>
EDIT :
The first error is due to an authentication error.
After filling in the right values, I have a new error that I don't understand.. Do you have any ideas?
2018-06-20 09:47:18,116 INFO regex.RegexURLNormalizer - can't find rules for scope 'indexer', using default
2018-06-20 09:47:19,151 INFO indexer.IndexWriters - Adding org.apache.nutch.indexwriter.solr.SolrIndexWriter
2018-06-20 09:47:19,194 INFO solr.SolrMappingReader - source: content dest: content
2018-06-20 09:47:19,194 INFO solr.SolrMappingReader - source: title dest: title
2018-06-20 09:47:19,194 INFO solr.SolrMappingReader - source: host dest: host
2018-06-20 09:47:19,194 INFO solr.SolrMappingReader - source: segment dest: segment
2018-06-20 09:47:19,194 INFO solr.SolrMappingReader - source: boost dest: boost
2018-06-20 09:47:19,195 INFO solr.SolrMappingReader - source: digest dest: digest
2018-06-20 09:47:19,195 INFO solr.SolrMappingReader - source: tstamp dest: tstamp
2018-06-20 09:47:19,525 INFO solr.SolrIndexWriter - Indexing 250/250 documents
2018-06-20 09:47:19,525 INFO solr.SolrIndexWriter - Deleting 0 documents
2018-06-20 09:47:19,808 INFO solr.SolrIndexWriter - Indexing 250/250 documents
2018-06-20 09:47:19,809 INFO solr.SolrIndexWriter - Deleting 0 documents
2018-06-20 09:47:19,951 WARN mapred.LocalJobRunner - job_local146539832_0001
java.lang.Exception: java.io.IOException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: java.io.IOException
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.makeIOException(SolrIndexWriter.java:234)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.push(SolrIndexWriter.java:213)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.write(SolrIndexWriter.java:174)
at org.apache.nutch.indexer.IndexWriters.write(IndexWriters.java:87)
at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:50)
at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:41)
at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.write(ReduceTask.java:493)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:422)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:369)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:57)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://localhost:8983/solr/ESRF-EXTERNAL
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:589)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.push(SolrIndexWriter.java:210)
... 16 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:886)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:857)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:181)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:115)
at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:146)
at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:96)
at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:112)
at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:117)
at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:265)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.sendRequestEntity(ManagedClientConnectionImpl.java:203)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
... 20 more
2018-06-20 09:47:20,873 ERROR indexer.IndexingJob - Indexer: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:873)
at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:147)
at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:230)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:239)
Stacktrace
name cpuTime / userTime
process reaper (37)
java.util.concurrent.SynchronousQueue$TransferStack#24197386
1.8587ms
0.0000ms
process reaper (36)
java.util.concurrent.SynchronousQueue$TransferStack#24197386
1.2672ms
0.0000ms
Scheduler-201556483 (31) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#6202c0c1
1.1534ms
0.0000ms
searcherExecutor-7-thread-1 (30) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#663d5859
63.2030ms
50.0000ms
DestroyJavaVM (27)
1164.4748ms
1040.0000ms
Thread-12 (25)
java.lang.Object#233fcafa
0.1211ms
0.0000ms
Connection evictor (23)
0.9319ms
0.0000ms
Connection evictor (22)
2.0995ms
0.0000ms
org.eclipse.jetty.server.session.HashSessionManager#1a052a00Timer (21) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#6626f9cf
4.2127ms
0.0000ms
qtp2012232625-20 (20) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#12d62902
56.7955ms
50.0000ms
qtp2012232625-19 (19)
47.6864ms
40.0000ms
qtp2012232625-18 (18) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#12d62902
79.3320ms
70.0000ms
qtp2012232625-17 (17) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#12d62902
100.9593ms
90.0000ms
qtp2012232625-16-acceptor-0#2d033cc4-ServerConnector#23c4c714{SSL,[ssl, http/1.1]}{0.0.0.0:8983} (16)
4.5898ms
0.0000ms
qtp2012232625-15 (15) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#12d62902
73.3096ms
60.0000ms
qtp2012232625-14 (14) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#12d62902
18.7950ms
10.0000ms
qtp2012232625-13 (13)
79.7804ms
70.0000ms
qtp2012232625-12 (12)
70.2385ms
60.0000ms
qtp2012232625-11 (11) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#12d62902
22.1012ms
10.0000ms
ShutdownMonitor (10)
0.3055ms
0.0000ms
Signal Dispatcher (5)
0.0873ms
0.0000ms
Finalizer (3)
java.lang.ref.ReferenceQueue$Lock#1e254491
8.2575ms
0.0000ms
Reference Handler (2)
java.lang.ref.Reference$Lock#431035b5
6.3846ms
0.0000ms
EDIT2
To test I disabled authentication to see if the problem did not come from https. Without authentication it works!
I tried to change the file and include it in jetty-https.xml rather than jetty.xml.
I have 2 account configure like this
<security-constraint>
<web-resource-collection>
<web-resource-name>Solr authenticated application</web-resource-name>
<url-pattern>/</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>admin</role-name>
</auth-constraint>
</security-constraint>
<login-config>
<auth-method>BASIC</auth-method>
<realm-name>Test Realm</realm-name>
</login-config>
security.json
{
"authentication":{
"blockUnknown": true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"xxxx"}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}],
"user-role":{"solr":"admin"}
}}
when i execute the following command
bin/nutch index -Dsolr.server.url=https://localhost:8983/solr/MYCORE -Dsolr.auth=true -Dsolr.auth.username='admin' -Dsolr.auth.password='xxxx' crawltest/crawldb/ -linkdb crawltest/linkdb/ crawltest/segments/* -filter -normalize -deleteGone
and I get this error
java.lang.Exception: java.io.IOException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: java.io.IOException
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.makeIOException(SolrIndexWriter.java:234)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.push(SolrIndexWriter.java:213)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.write(SolrIndexWriter.java:174)
at org.apache.nutch.indexer.IndexWriters.write(IndexWriters.java:87)
at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:50)
at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:41)
at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.write(ReduceTask.java:493)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:422)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:369)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:57)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://localhost:8983/solr/MYCORE
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:589)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.push(SolrIndexWriter.java:210)
... 16 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:886)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:857)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:181)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:115)
at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:146)
at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:96)
at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:112)
at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:117)
at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:265)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.sendRequestEntity(ManagedClientConnectionImpl.java:203)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:481)
... 20 more
2018-06-25 09:38:41,870 ERROR indexer.IndexingJob - Indexer: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:873)
at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:147)
at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:230)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:239)
And when i execute this
bin/nutch index -Dsolr.server.url=https://localhost:8983/solr/MYCORE -Dsolr.auth=true -Dsolr.auth.username='solr' -Dsolr.auth.password='xxxxx' crawltest/crawldb/ -linkdb crawltest/linkdb/ crawltest/segments/* -filter -normalize -deleteGone
I get this error
java.lang.Exception: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://localhost:8983/solr/MYCORE: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /solr/MYCORE/update. Reason:
<pre> Unauthorized</pre></p>
</body>
</html>
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://localhost:8983/solr/MYCORE: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /solr/MYCORE/update. Reason:
<pre> Unauthorized</pre></p>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:544)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.push(SolrIndexWriter.java:210)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.write(SolrIndexWriter.java:174)
at org.apache.nutch.indexer.IndexWriters.write(IndexWriters.java:87)
at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:50)
at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:41)
at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.write(ReduceTask.java:493)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:422)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:369)
at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:57)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-06-25 09:45:20,106 ERROR indexer.IndexingJob - Indexer: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:873)
at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:147)
at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:230)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:239)
or now this :
java.lang.Exception: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:8983/solr/MYCORE: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 503 </title>
</head>
<body>
<h2>HTTP ERROR: 503</h2>
<p>Problem accessing /solr/MYCORE/update. Reason:
<pre> Service Unavailable</pre></p>
<hr />
</body>
</html>
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:8983/solr/MYCORE: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 503 </title>
</head>
<body>
<h2>HTTP ERROR: 503</h2>
<p>Problem accessing /solr/MYCORE/update. Reason:
<pre> Service Unavailable</pre></p>
<hr />
</body>
</html>
Solr logs :
2018-06-25 14:18:44.352 INFO (main) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
2018-06-25 14:18:44.597 WARN (main) [ ] o.e.j.w.WebAppContext Failed startup of context o.e.j.w.WebAppContext#5891e32e{/solr,file:///app/solr-7.2.1/server/solr-webapp/webapp/,UNAVAILABLE}{/app/solr-7.2.1/server/solr-webapp/webapp}
java.lang.IllegalStateException: No LoginService for org.eclipse.jetty.security.authentication.BasicAuthenticator#64c87930 in org.eclipse.jetty.security.ConstraintSecurityHandler#400cff1a
at org.eclipse.jetty.security.authentication.LoginAuthenticator.setConfiguration(LoginAuthenticator.java:76)
at org.eclipse.jetty.security.SecurityHandler.doStart(SecurityHandler.java:354)
at org.eclipse.jetty.security.ConstraintSecurityHandler.doStart(ConstraintSecurityHandler.java:448)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.handler.ScopedHandler.doStart(ScopedHandler.java:120)
at org.eclipse.jetty.server.session.SessionHandler.doStart(SessionHandler.java:116)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.handler.ScopedHandler.doStart(ScopedHandler.java:120)
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:809)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1406)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1368)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:522)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:41)
at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188)
at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:499)
at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:147)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:180)
at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:458)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:64)
at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:610)
at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:529)
at org.eclipse.jetty.util.Scanner.scan(Scanner.java:392)
at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:313)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:150)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:561)
at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.server.Server.start(Server.java:422)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
at org.eclipse.jetty.server.Server.doStart(Server.java:389)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1520)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:215)
at org.eclipse.jetty.start.Main.start(Main.java:458)
at org.eclipse.jetty.start.Main.main(Main.java:76)
2018-06-25 14:18:44.745 INFO (main) [ ] o.e.j.s.Server Started #799ms
I Solved "No LoginService" by move the security.json on /var/solr/data/ (SOLR_HOME)
EDIT3:
now I only get an error message "No allow" when I want to send nutch data to solr. Also I can no longer connect to the admin interface I get the same error. I think it came from the security.json file
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"xxxxxx"}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin"
"permissions":[{"name":"security-edit","role":"adminRole"},{"name":"collection-admin-edit","role":"adminRole"},{"name":"update","role":"adminRole"},{"name":"all","role":"adminRole"},{"name":"core-admin-edit","role":"adminRole"},{"name":"read","role":"adminRole"},{"name":"config-edit","role":"adminRole"},{"name":"core-admin-read","role":"adminRole"},{"name":"core-admin-read","role":"adminRole"}]
"user-role":{"solr":"adminRole"}
}}
What did I do wrong? thanks
I added a new answer because the previous was too long
SOLVED API AUTHENTICATION BUT NOT WITH NUTCH:
So to make authentication work via the API I deleted the configuration made in jetty-https.xml, webdefault.xml and I deleted the realm.properties file as well as the basic auth options in solr.in.sh
I only work on the security.json file in SOLR HOME
In fact the biggest problem was that I did not use an encrypted password to test the connection while without it impossible to connect. On the other hand I still have the problem via nutch which is not allowed.
Here is the security.json file
`{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{
"solr":"hzMjhfgN4b9X8KR0QgLB2Um3cUzqDzJygtEBL/O7g5E= CkP7HyXjYvqKNF3F4hBjnVvKGQOkLc/ta4FaNIkqgII="
}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[
{
"name":"security-edit",
"role":"adminRole"
},
{
"name":"collection-admin-edit",
"role":"adminRole"
},
{
"name":"update",
"role":"adminRole"
},
{
"name":"config-edit",
"role":"adminRole"
},
{
"name":"core-admin-edit",
"role":"adminRole"
},
{
"name":"core-admin-read",
"role":"adminRole"
{
"name":"schema-edit",
"role":"adminRole"
},
{
"name":"all",
"role":"adminRole"
}
],
"user-role":{
"solr":"adminRole"
}
}
}
`
The encrypted password represented value "test"
To test the file code i suggest this http://json.parser.online.fr/
What could I have missed to be able to update solr with nutch?
SOLVED
Add in update role path for dataimport
{
"name":"update",
"path":"/dataimport",
"role":"adminRole"
},
But now i can index nutch to solr but i have a new error when i'm crawling...
`Thu Jun 28 09:21:03 CEST 2018 : Iteration 2 of 5
Generating a new segment
/app/nutch-external/bin/nutch generate -D mapreduce.job.reduces=2 -D mapred.child.java.opts=-Xmx1000m -D mapreduce.reduce.speculative=false -D mapreduce.map.speculative=false -D mapreduce.map.output.compress=true crawl//crawldb crawl//segments -topN 50000 -numFetchers 1 -noFilter
Generator: starting at 2018-06-28 09:21:04
Generator: Selecting best-scoring urls due for fetch.
Generator: filtering: false
Generator: normalizing: true
Generator: topN: 50000
Generator: 0 records selected for fetching, exiting ...
Generate returned 1 (no new segments created)
Escaping loop: no more URLs to fetch now
`
And during the first Iteration i have this errors
`Authorization challenge processed
No form element found with 'id' = adminRole, trying 'name'.
No form element found with 'id' = adminRole, trying 'name'.
No form element found with 'name' = adminRole
No form element found with 'name' = adminRole
Supported authentication schemes in the order of preference: [ntlm, digest, basic]
Supported authentication schemes in the order of preference: [ntlm, digest, basic]
Challenge for ntlm authentication scheme not available
Challenge for ntlm authentication scheme not available
Challenge for digest authentication scheme not available
basic authentication scheme selected
Using authentication scheme: basic
Authorization challenge processed
No form element found with 'id' = adminRole, trying 'name'.
No form element found with 'name' = adminRole
Failed to get protocol output
java.lang.RuntimeException: java.lang.IllegalArgumentException: No form exists: adminRole
at org.apache.nutch.protocol.httpclient.Http.resolveCredentials(Http.java:506)
at org.apache.nutch.protocol.httpclient.Http.getResponse(Http.java:183)
at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:276)
at org.apache.nutch.fetcher.FetcherThread.run(FetcherThread.java:342)
Caused by: java.lang.IllegalArgumentException: No form exists: adminRole
at org.apache.nutch.protocol.httpclient.HttpFormAuthentication.getLoginFormParams(HttpFormAuthentication.java:219)
at org.apache.nutch.protocol.httpclient.HttpFormAuthentication.login(HttpFormAuthentication.java:95)
at org.apache.nutch.protocol.httpclient.Http.resolveCredentials(Http.java:504)
... 3 more
Challenge for digest authentication scheme not available
basic authentication scheme selected
Using authentication scheme: basic
Authorization challenge processed
No form element found with 'id' = adminRole, trying 'name'.
`
It's still security.json file. Have you any ideas ? thanks
SOLVED
I solved this by configure httpclient-auth.xml in /nutch/conf/
<auth-configuration>
<credentials username="solr" password="xxxxx">
<authscope host="localhost" port="8983"/>
</credentials>
</auth-configuration>
Thanks for your help

Spring Integration RSS Feed - 403 Error

I am trying to use Spring Integration to poll RSS feeds and I'm using SO user feed URL as an example. However when the application runs I get the following error:
Disconnected from the target VM, address: '127.0.0.1:58603', transport: 'socket'
2016-03-04 19:34:36.252 ERROR 2345 --- [ask-scheduler-4] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Failed to retrieve feed at url 'http://stackoverflow.com/feeds/user/813852'; nested exception is com.rometools.fetcher.FetcherException: Authentication required for that resource. HTTP Response code was:403
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:216)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.populateEntryList(FeedEntryMessageSource.java:182)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.doReceive(FeedEntryMessageSource.java:157)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.receive(FeedEntryMessageSource.java:122)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:175)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:224)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:57)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:176)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:173)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:330)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:324)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.rometools.fetcher.FetcherException: Authentication required for that resource. HTTP Response code was:403
at com.rometools.fetcher.impl.AbstractFeedFetcher.throwAuthenticationError(AbstractFeedFetcher.java:183)
at com.rometools.fetcher.impl.AbstractFeedFetcher.handleErrorCodes(AbstractFeedFetcher.java:170)
at com.rometools.fetcher.impl.HttpURLFeedFetcher.retrieveAndCacheFeed(HttpURLFeedFetcher.java:186)
at com.rometools.fetcher.impl.HttpURLFeedFetcher.retrieveFeed(HttpURLFeedFetcher.java:140)
at com.rometools.fetcher.impl.HttpURLFeedFetcher.retrieveFeed(HttpURLFeedFetcher.java:99)
at org.springframework.integration.feed.inbound.FeedEntryMessageSource.getFeed(FeedEntryMessageSource.java:204)
... 22 more
The URL loads fine through the browser (even in incognito mode so the error message is probably misleading). I downloaded an RSS reader and the feed worked fine.
This is how I configured the feed:
<int-feed:inbound-channel-adapter id="stackOverflow"
channel="log"
url="http://stackoverflow.com/feeds/user/813852">
<int:poller fixed-rate="10000" max-messages-per-poll="100" />
</int-feed:inbound-channel-adapter>
<int:logging-channel-adapter id="log" level="DEBUG" />
I also tried another RSS feed URL and it worked. Any ideas what could be wrong?
I just ran a wireshark trace and it looks like StackOverflow doesn't like the java user agent:
User-Agent: Java/1.8.0_66\r\n
...
<p>The owner of this website (stackoverflow.com) has banned your access based
on your browser's signature (27e797571e1923de-ua21).</p>\n
Setting a custom user agent goes around this restriction:
public class CustomFeedFetcher extends HttpURLFeedFetcher {
#Override
public void setUserAgent(String s) {
super.setUserAgent("AgentName");
}
}

WSO2 data service (DS) in ESB 4.0.3 returns wrong fault message

I work at Apple and we are working on the prototype using WSO2 ESB with Data Services Feature. I am using WSO2 ESB 4.0.3 with DSS feature. I have deployed data service and calling the data service from proxy service in ESB. I am using MAC OS X 10.7.4 and java version details are as below.
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04-415-11M3635)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01-415, mixed mode)
My DS service is expecting certain parameter say parameter A and I do not pass that this parameter in the Input XML I see following error on the WSO2 ESB console.
[2012-06-07 09:49:11,348] ERROR - DataService DS Fault Message: Error in 'CallQuery.extractParams', cannot find parameter with type:query-param name:Hw_Serial_Nr
DS Code: INCOMPATIBLE_PARAMETERS_ERROR
Source Data Service:-
Name: OWCHECK
Location: /TQS_OWCHECK.dbs
Description: N/A
Default Namespace: http://ws.wso2.org/dataservice
Current Request Name: OWCHECK_OP
Current Params: {COUNTRYORIGIN=US, SUBDT=20090428}
DS Fault Message: Error in 'CallQuery.extractParams', cannot find parameter with type:query-param name:Hw_Serial_Nr
DS Code: INCOMPATIBLE_PARAMETERS_ERROR
Source Data Service:-
Name: OWCHECK
Location: /TQS_OWCHECK.dbs
Description: N/A
Default Namespace: http://ws.wso2.org/dataservice
Current Request Name: OWCHECK_OP
Current Params: {COUNTRYORIGIN=US, SUBDT=20090428}
at org.wso2.carbon.dataservices.core.engine.CallQuery.extractParams(CallQuery.java:214)
at org.wso2.carbon.dataservices.core.engine.CallQuery.execute(CallQuery.java:183)
at org.wso2.carbon.dataservices.core.engine.CallQueryGroup.execute(CallQueryGroup.java:110)
at org.wso2.carbon.dataservices.core.description.operation.Operation.execute(Operation.java:71)
at org.wso2.carbon.dataservices.core.engine.DataService.invoke(DataService.java:592)
at org.wso2.carbon.dataservices.core.engine.DSOMDataSource.execute(DSOMDataSource.java:99)
at org.wso2.carbon.dataservices.core.engine.DSOMDataSource.serialize(DSOMDataSource.java:110)
at org.wso2.carbon.dataservices.core.engine.DSOMDataSource.getReader(DSOMDataSource.java:116)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.getDirectReader(OMSourcedElementImpl.java:225)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.forceExpand(OMSourcedElementImpl.java:254)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.getFirstOMChild(OMSourcedElementImpl.java:867)
at org.apache.axiom.om.impl.OMNavigator._getFirstChild(OMNavigator.java:196)
at org.apache.axiom.om.impl.OMNavigator.updateNextNode(OMNavigator.java:140)
at org.apache.axiom.om.impl.OMNavigator.getNext(OMNavigator.java:112)
at org.apache.axiom.om.impl.SwitchingWrapper.updateNextNode(SwitchingWrapper.java:1113)
at org.apache.axiom.om.impl.SwitchingWrapper.updateLastNode(SwitchingWrapper.java:1104)
at org.apache.axiom.om.impl.SwitchingWrapper.next(SwitchingWrapper.java:1041)
at javax.xml.stream.util.StreamReaderDelegate.next(StreamReaderDelegate.java:47)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.parserNext(StAXOMBuilder.java:681)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:214)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.build(OMSerializableImpl.java:78)
at org.apache.axiom.om.impl.llom.OMElementImpl.build(OMElementImpl.java:722)
at org.apache.axiom.om.impl.llom.OMElementImpl.cloneOMElement(OMElementImpl.java:1065)
at org.wso2.carbon.tracer.module.handler.AbstractTracingHandler.storeMessage(AbstractTracingHandler.java:81)
at org.wso2.carbon.tracer.module.handler.TracingMessageOutObservationHandler.invoke(TracingMessageOutObservationHandler.java:94)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:262)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:427)
at org.apache.axis2.receivers.AbstractInOutSyncMessageReceiver.invokeBusinessLogic(AbstractInOutSyncMessageReceiver.java:45)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:181)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.synapse.transport.nhttp.ServerWorker.processEntityEnclosingMethod(ServerWorker.java:409)
at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:261)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:173)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
[2012-06-07 09:49:11,352] ERROR - OMSourcedElementImpl Could not get parser from data source for element {http://ws.wso2.org/dataservice}EVENT
javax.xml.stream.XMLStreamException: DS Fault Message: Error in 'CallQuery.extractParams', cannot find parameter with type:query-param name:Hw_Serial_Nr
DS Code: INCOMPATIBLE_PARAMETERS_ERROR
Source Data Service:-
Name: OWCHECK
Location: /TQS_OWCHECK.dbs
Description: N/A
Default Namespace: http://ws.wso2.org/dataservice
Current Request Name: OWCHECK_OP
Current Params: {COUNTRYORIGIN=US, SUBDT=20090428}
at org.wso2.carbon.dataservices.core.engine.DSOMDataSource.execute(DSOMDataSource.java:105)
at org.wso2.carbon.dataservices.core.engine.DSOMDataSource.serialize(DSOMDataSource.java:110)
at org.wso2.carbon.dataservices.core.engine.DSOMDataSource.getReader(DSOMDataSource.java:116)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.getDirectReader(OMSourcedElementImpl.java:225)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.forceExpand(OMSourcedElementImpl.java:254)
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.getFirstOMChild(OMSourcedElementImpl.java:867)
at org.apache.axiom.om.impl.OMNavigator._getFirstChild(OMNavigator.java:196)
at org.apache.axiom.om.impl.OMNavigator.updateNextNode(OMNavigator.java:140)
at org.apache.axiom.om.impl.OMNavigator.getNext(OMNavigator.java:112)
at org.apache.axiom.om.impl.SwitchingWrapper.updateNextNode(SwitchingWrapper.java:1113)
at org.apache.axiom.om.impl.SwitchingWrapper.updateLastNode(SwitchingWrapper.java:1104)
at org.apache.axiom.om.impl.SwitchingWrapper.next(SwitchingWrapper.java:1041)
at javax.xml.stream.util.StreamReaderDelegate.next(StreamReaderDelegate.java:47)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.parserNext(StAXOMBuilder.java:681)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:214)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.build(OMSerializableImpl.java:78)
at org.apache.axiom.om.impl.llom.OMElementImpl.build(OMElementImpl.java:722)
at org.apache.axiom.om.impl.llom.OMElementImpl.cloneOMElement(OMElementImpl.java:1065)
at org.wso2.carbon.tracer.module.handler.AbstractTracingHandler.storeMessage(AbstractTracingHandler.java:81)
at org.wso2.carbon.tracer.module.handler.TracingMessageOutObservationHandler.invoke(TracingMessageOutObservationHandler.java:94)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:262)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:427)
at org.apache.axis2.receivers.AbstractInOutSyncMessageReceiver.invokeBusinessLogic(AbstractInOutSyncMessageReceiver.java:45)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:181)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
at org.apache.synapse.transport.nhttp.ServerWorker.processEntityEnclosingMethod(ServerWorker.java:409)
at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:261)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:173)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
[2012-06-07 09:49:11,355] ERROR - ServerWorker Error processing POST request
org.apache.axis2.AxisFault: javax.xml.stream.XMLStreamException
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:180)
at org.apache.synapse.transport.nhttp.ServerWorker.processEntityEnclosingMethod(ServerWorker.java:409)
at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:261)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:173)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.axiom.om.OMException: javax.xml.stream.XMLStreamException
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:296)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.build(OMSerializableImpl.java:78)
at org.apache.axiom.om.impl.llom.OMElementImpl.build(OMElementImpl.java:722)
at org.apache.axiom.om.impl.llom.OMElementImpl.cloneOMElement(OMElementImpl.java:1065)
at org.wso2.carbon.tracer.module.handler.AbstractTracingHandler.storeMessage(AbstractTracingHandler.java:81)
at org.wso2.carbon.tracer.module.handler.TracingMessageOutObservationHandler.invoke(TracingMessageOutObservationHandler.java:94)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:262)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:427)
at org.apache.axis2.receivers.AbstractInOutSyncMessageReceiver.invokeBusinessLogic(AbstractInOutSyncMessageReceiver.java:45)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:181)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
... 6 more
Caused by: javax.xml.stream.XMLStreamException
at org.apache.axiom.om.impl.SwitchingWrapper.updateLastNode(SwitchingWrapper.java:1106)
at org.apache.axiom.om.impl.SwitchingWrapper.next(SwitchingWrapper.java:1041)
at javax.xml.stream.util.StreamReaderDelegate.next(StreamReaderDelegate.java:47)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.parserNext(StAXOMBuilder.java:681)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:214)
... 19 more
[2012-06-07 09:49:11,375] WARN - ClientHandler Received an internal server error : Internal Server Error For : 127.0.0.1:8280 For Request : Axis2Request [Message ID : urn:uuid:55a4d59a-2be2-43ee-99b5-8943b1ff2b91] [Status Completed : true] [Status SendingCompleted : true]
How ever the fault message it returned to the proxy service is as below
2012-06-07 09:49:11,081 [-] [HttpServerWorker-1] TRACE OWCHECK_RPOXY To: /services/OWCHECK_RPOXY.OWCHECK_RPOXYHttpSoap12Endpoint, WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:1f8cb9bf-ab10-40a4-b75d-6b3d4805322c, Direction: request, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body><OWCHECK_OP spec="IDL:com/apple/vitria/idl/sapamr_rfcevents/RfcCallEvents:1.0#Z_WARR_VALIDATION_GROUP"><TAB_SERNHDR><SERIALNR>YM6191TJSCH</SERIALNR><SUBDT>20090428</SUBDT><COUNTRYORIGIN>US</COUNTRYORIGIN></TAB_SERNHDR></OWCHECK_OP></soapenv:Body></soapenv:Envelope>
2012-06-07 09:49:11,398 [-] [HttpClientWorker-1] TRACE OWCHECK_RPOXY To: http://www.w3.org/2005/08/addressing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:55a4d59a-2be2-43ee-99b5-8943b1ff2b91, Direction: response, MESSAGE = In-to-OutSequence !!!!!!!!!!!!!!!!!!!!!!!!!!!!, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body><soapenv:Fault><soapenv:Code><soapenv:Value>soapenv:Receiver</soapenv:Value></soapenv:Code><soapenv:Reason><soapenv:Text xml:lang="en-US">javax.xml.stream.XMLStreamException</soapenv:Text></soapenv:Reason><soapenv:Detail><Exception>org.apache.axis2.AxisFault: javax.xml.stream.XMLStreamException
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:180)
at org.apache.synapse.transport.nhttp.ServerWorker.processEntityEnclosingMethod(ServerWorker.java:409)
at org.apache.synapse.transport.nhttp.ServerWorker.run(ServerWorker.java:261)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:173)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.axiom.om.OMException: javax.xml.stream.XMLStreamException
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:296)
at org.apache.axiom.om.impl.llom.OMSerializableImpl.build(OMSerializableImpl.java:78)
at org.apache.axiom.om.impl.llom.OMElementImpl.build(OMElementImpl.java:722)
at org.apache.axiom.om.impl.llom.OMElementImpl.cloneOMElement(OMElementImpl.java:1065)
at org.wso2.carbon.tracer.module.handler.AbstractTracingHandler.storeMessage(AbstractTracingHandler.java:81)
at org.wso2.carbon.tracer.module.handler.TracingMessageOutObservationHandler.invoke(TracingMessageOutObservationHandler.java:94)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:262)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:427)
at org.apache.axis2.receivers.AbstractInOutSyncMessageReceiver.invokeBusinessLogic(AbstractInOutSyncMessageReceiver.java:45)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:181)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
... 6 more
Caused by: javax.xml.stream.XMLStreamException
at org.apache.axiom.om.impl.SwitchingWrapper.updateLastNode(SwitchingWrapper.java:1106)
at org.apache.axiom.om.impl.SwitchingWrapper.next(SwitchingWrapper.java:1041)
at javax.xml.stream.util.StreamReaderDelegate.next(StreamReaderDelegate.java:47)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.parserNext(StAXOMBuilder.java:681)
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:214)
... 19 more
</Exception></soapenv:Detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
Looking at the SOAP fault I have to return the error message to client but even thought the correct error message must "Parameter Missing" SOAP fault's reason text is "javax.xml.stream.XMLStreamException"
Is there any thing wrong I am doing or is it BUG in DSS?
By looking at the stacktrace, I can see that you've enabled the SOAP tracer in invoking the service, disable the SOAP tracer and you should get the proper fault message.
Cheers,
Anjana.
As error message explains itself, it is expecting input parameters, which is not coming int the request. It can be fixed by two ways:
Passing empty tag from ESB without value.
Set default value in DSS.
First option works with all versions, here is the example for setting default value.
<payloadFactory media-type="xml">
<format>
<ds:DATA_SERVICE_REQUST xmlns:ds="http://ws.wso2.org/dataservice">
<ds:INPUT_PARAM_1>$1</ds:INPUT_PARAM_1>
</ds:DATA_SERVICE_REQUST>
</format>
<args>
<arg xmlns:ds="http://ws.wso2.org/dataservice" evaluator="xml" expression="get-property('INPUT_PARAMETER')"/>
</args>
</payloadFactory>
<call>
<endpoint>
<address uri="DSS_END_POINT"/>
</endpoint>
</call>
For 2nd option, please check your DSS version if this supports null values
Latest Version of DSS Supports default parameter values in the query param list.
Could you please give a try by adding defaultValue="#{NULL}" in query param list in your data service (dbs) file?
<query id="QUERY-ID" useConfig="DATA-SOURCE-ID">
<sql>-------------SQL QUERY HERE -----</sql>
<result ----------- >
<element ------------ />
</result>
<param defaultValue="#{NULL}" name="PARAM-NAME" ordinal="1" sqlType="INTEGER"/>
</query>
If this issue is still not resolved then it would be good if you can provide your API details.

Resources