Caused by: org.apache.commons.net.MalformedServerReplyException: Could not parse response code - spring-boot

I'm developing a spring boot application which reads data from an ftp connection. Have been referring this article. http://docs.spring.io/spring-integration/reference/html/ftp.html I've added below dependency to pom.xml:
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-ftp</artifactId>
<version>4.3.2.RELEASE</version>
</dependency>
Here is my Spring boot application:
#SpringBootApplication
public class FtpApplication {
public static void main(String[] args) {
SpringApplication.run(FtpApplication.class, args);
}
#Bean
public SessionFactory<FTPFile> ftpSessionFactory() {
DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
sf.setHost("localhost");
sf.setPort(14147);
sf.setUsername("root");
sf.setPassword("root");
return new CachingSessionFactory<FTPFile>(sf);
}
#Bean
public FtpInboundFileSynchronizer ftpInboundFileSynchronizer() {
FtpInboundFileSynchronizer fileSynchronizer = new FtpInboundFileSynchronizer(ftpSessionFactory());
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.setRemoteDirectory("/");
fileSynchronizer.setFilter(new FtpSimplePatternFileListFilter("*.xml"));
return fileSynchronizer;
}
#Bean
#InboundChannelAdapter(channel = "ftpChannel")
public MessageSource<File> ftpMessageSource() {
FtpInboundFileSynchronizingMessageSource source = new FtpInboundFileSynchronizingMessageSource(
ftpInboundFileSynchronizer());
source.setLocalDirectory(new File("ftp-inbound"));
source.setAutoCreateLocalDirectory(true);
source.setLocalFilter(new AcceptOnceFileListFilter<File>());
return source;
}
#Bean
#ServiceActivator(inputChannel = "ftpChannel")
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
File file = (File) message.getPayload();
BufferedReader br;
String sCurrentLine;
try {
br = new BufferedReader(new FileReader(file.getPath()));
while ((sCurrentLine = br.readLine()) != null) {
System.out.println(sCurrentLine);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(message.getPayload());
}
};
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
pollerMetadata.setTrigger(new PeriodicTrigger(10));
return pollerMetadata;
}
}
From the windows explorer I'm adding a file. Now when the control comes to the MessageHandler function, I see the below error. But I can neatly get the file and I see the contents correctly when I read it. But I'm unable to figure out what is the error all about:
2016-09-27 08:25:07.548 ERROR 10292 --- [ask-scheduler-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Problem occurred while synchronizing remote to local directory; nested exception is org.springframework.messaging.MessagingException: Failed to obtain pooled item; nested exception is java.lang.IllegalStateException: failed to create FTPClient
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:274)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.doReceive(AbstractInboundFileSynchronizingMessageSource.java:193)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.doReceive(AbstractInboundFileSynchronizingMessageSource.java:59)
at org.springframework.integration.endpoint.AbstractMessageSource.receive(AbstractMessageSource.java:134)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:209)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:245)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:58)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:190)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:186)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:353)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:344)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.messaging.MessagingException: Failed to obtain pooled item; nested exception is java.lang.IllegalStateException: failed to create FTPClient
at org.springframework.integration.util.SimplePool.getItem(SimplePool.java:178)
at org.springframework.integration.file.remote.session.CachingSessionFactory.getSession(CachingSessionFactory.java:123)
at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:433)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:232)
... 22 more
Caused by: java.lang.IllegalStateException: failed to create FTPClient
at org.springframework.integration.ftp.session.AbstractFtpSessionFactory.getSession(AbstractFtpSessionFactory.java:169)
at org.springframework.integration.ftp.session.AbstractFtpSessionFactory.getSession(AbstractFtpSessionFactory.java:41)
at org.springframework.integration.file.remote.session.CachingSessionFactory$1.createForPool(CachingSessionFactory.java:81)
at org.springframework.integration.file.remote.session.CachingSessionFactory$1.createForPool(CachingSessionFactory.java:78)
at org.springframework.integration.util.SimplePool.doGetItem(SimplePool.java:188)
at org.springframework.integration.util.SimplePool.getItem(SimplePool.java:169)
... 25 more
Caused by: org.apache.commons.net.MalformedServerReplyException: Could not parse response code.
Server Reply: FZS ..... some speacial characters here.....
at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:336)
at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:292)
at org.apache.commons.net.ftp.FTP._connectAction_(FTP.java:418)
at org.apache.commons.net.ftp.FTPClient._connectAction_(FTPClient.java:966)
at org.apache.commons.net.ftp.FTPClient._connectAction_(FTPClient.java:954)
at org.apache.commons.net.SocketClient.connect(SocketClient.java:189)
at org.apache.commons.net.SocketClient.connect(SocketClient.java:209)
at org.springframework.integration.ftp.session.AbstractFtpSessionFactory.createClient(AbstractFtpSessionFactory.java:191)
at org.springframework.integration.ftp.session.AbstractFtpSessionFactory.getSession(AbstractFtpSessionFactory.java:166)
... 30 more
I'm new to spring integration, please help. Let me know the concepts that I should still consider preparing.

You are most probably connecting to the FileZilla FTP server administrative port (14147).
That port uses a proprietary protocol used by a "FileZilla Server Interface", not FTP, and you are not supposed to connect to it with your application.
Connect to the FTP port instead. By default that is 21. It is configured in "FileZilla Server Interface" on "General Settings" page of the "FileZilla Server Options" as "Listen to these ports".

Related

How to fix ListenerExecutionFailedException: Listener threw exception amqp.AmqpRejectAndDontRequeueException: Reply received after timeout

I set up rabbbitMQ on my java spring-boot application and it works properly (it seems), but after running for a while and somehow with same time interval It throws below exception.
org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener threw exception
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.wrapToListenerExecutionFailedExceptionIfNeeded(AbstractMessageListenerContainer.java:1646) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1550) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1473) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1461) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1456) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1405) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer$SimpleConsumer.callExecuteListener(DirectMessageListenerContainer.java:995) [spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer$SimpleConsumer.handleDelivery(DirectMessageListenerContainer.java:955) [spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at com.rabbitmq.client.impl.ConsumerDispatcher$5.run(ConsumerDispatcher.java:149) [amqp-client-5.4.3.jar!/:5.4.3]
at com.rabbitmq.client.impl.ConsumerWorkService$WorkPoolRunnable.run(ConsumerWorkService.java:104) [amqp-client-5.4.3.jar!/:5.4.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_201]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_201]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201]
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: Reply received after timeout
at org.springframework.amqp.rabbit.core.RabbitTemplate.onMessage(RabbitTemplate.java:2523) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectReplyToMessageListenerContainer.lambda$setMessageListener$1(DirectReplyToMessageListenerContainer.java:115) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1547) ~[spring-rabbit-2.1.4.RELEASE.jar!/:2.1.4.RELEASE]
... 11 common frames omitted
below you can find the consumer code for rabbit configuration
#Bean
public DirectExchange exchange() {
return new DirectExchange("rpc");
}
#Bean
#Qualifier("Consumer")
public Queue queue() {
return new Queue(RoutingEngine.class.getSimpleName()+"_"+config.getDatasetName());
}
#Bean
public Binding binding(Queue queue, DirectExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(Consumer.class.getSimpleName()+"_"+config.getDatasetName());
}
#Bean
#Qualifier("ConsumerExport")
public AmqpInvokerServiceExporter exporter(RabbitTemplate template, Consumer service) {
AmqpInvokerServiceExporter exporter = new AmqpInvokerServiceExporter();
exporter.setAmqpTemplate(template);
exporter.setService(service);
exporter.setServiceInterface(Consumer.class);
return exporter;
}
#Bean
public SimpleMessageListenerContainer container(ConnectionFactory connectionFactory,#Qualifier("consumer") Queue queue,
#Qualifier("RoutingEngineExport") AmqpInvokerServiceExporter exporter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
container.setPrefetchCount(5);
container.setQueues(queue);
container.setMessageListener(exporter);
logger.info("initialize rabbitmq with {} Consumers",config.getCount());
container.setConcurrency(1+"-"+config.getCount());
return container;
}
#Bean
public FanoutExchange fanoutExchange(){
return new FanoutExchange("event");
}
#Bean
#Qualifier("reinitialize")
public Queue reInitQueue() {
return new Queue("bus."+config.getConsumerName(),false,true,true);
}
#Bean
public Binding topicBinding(#Qualifier("reinitialize") Queue queue, FanoutExchange fanoutExchangee) {
return BindingBuilder
.bind(queue)
.to(fanoutExchangee);
}
#Bean
public MessageListener<Consumer> messageListener(RabbitTemplate rabbitTemplate,Consumer target){
return new MessageListener<>(rabbitTemplate, target, "engine", config.getConsumerName());
}
and also producer configuration code is
#Bean
public AmqpProxyFactoryBean rerouteProxy(RabbitTemplate template) {
AmqpProxyFactoryBean proxy = new AmqpProxyFactoryBean();
proxy.setAmqpTemplate(template);
proxy.setServiceInterface(ConsumerService.class);
proxy.setRoutingKey(ConsumerService.class.getSimpleName());
return proxy;
}
#Bean
public Map<String,Consumer> consumerEngines( RabbitTemplate template){
Map<String,Consumer> ret= new ConcurrentHashMap<>();
//FIXme read from config
List<String> lst = Arrays.asList(config.getEngines());
lst.parallelStream().forEach(k->{
AmqpProxyFactoryBean proxy = new AmqpProxyFactoryBean();
template.setReceiveTimeout(400);
template.setReplyTimeout(400);
proxy.setAmqpTemplate(template);
proxy.setServiceInterface(Consumer.class);
proxy.setRoutingKey(Consumer.class.getSimpleName() + "_" + k);
proxy.afterPropertiesSet();
ret.put(k, (Consumer) proxy.getObject());
});
return ret;
}
what causes this problem and how to fix it?
NOTE 1: I have 3 producers and 3 consumers on different servers, and rabbit is running on another server
ٔNOTE 2: Consumers are very fast, their response time is less than 100 miliseconds
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: Reply received after timeout
This is caused by one of two reasons
the reply took too long to arrive (in which case the send and receive operation would have returned null earlier).
a consumer sent more than one reply for the same request

kafka embedded : java.io.FileNotFoundException: /tmp/kafka-7785736914220873149/replication-offset-checkpoint.tmp

I use kafkaEmbedded in integration test and I get FileNotFoundException :
java.io.FileNotFoundException: /tmp/kafka-7785736914220873149/replication-offset-checkpoint.tmp
at java.io.FileOutputStream.open0(Native Method) ~[na:1.8.0_141]
at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[na:1.8.0_141]
at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[na:1.8.0_141]
at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[na:1.8.0_141]
at kafka.server.checkpoints.CheckpointFile.write(CheckpointFile.scala:43) ~[kafka_2.11-0.11.0.0.jar:na]
at kafka.server.checkpoints.OffsetCheckpointFile.write(OffsetCheckpointFile.scala:58) ~[kafka_2.11-0.11.0.0.jar:na]
at kafka.server.ReplicaManager$$anonfun$checkpointHighWatermarks$2.apply(ReplicaManager.scala:1118) [kafka_2.11-0.11.0.0.jar:na]
at kafka.server.ReplicaManager$$anonfun$checkpointHighWatermarks$2.apply(ReplicaManager.scala:1115) [kafka_2.11-0.11.0.0.jar:na]
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) [scala-library-2.11.11.jar:na]
at scala.collection.immutable.Map$Map1.foreach(Map.scala:116) [scala-library-2.11.11.jar:na]
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732) [scala-library-2.11.11.jar:na]
at kafka.server.ReplicaManager.checkpointHighWatermarks(ReplicaManager.scala:1115) [kafka_2.11-0.11.0.0.jar:na]
at kafka.server.ReplicaManager$$anonfun$1.apply$mcV$sp(ReplicaManager.scala:211) [kafka_2.11-0.11.0.0.jar:na]
at kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110) [kafka_2.11-0.11.0.0.jar:na]
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:57) [kafka_2.11-0.11.0.0.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_141]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_141]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_141]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_141]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]
My tests pass with success but I get this error in the end of my build
After many hours of research I found this :
kafka TestUtils.tempDirectory method is used to create temporary directory for embedded kafka broker. It also registers shutdown hook which deletes this directory when JVM exits.
when unit test finishes execution it calls System.exit, which in turn executes all registered shutdown hooks
If kafka broker runs at the end of unit test it will attempt to write/read data in a dir which is deleted and produces different FileNotFound exceptions.
My config class :
#Configuration
public class KafkaEmbeddedConfiguration {
private final KafkaEmbedded kafkaEmbedded;
public KafkaEmbeddedListenerConfigurationIT() throws Exception {
kafkaEmbedded = new KafkaEmbedded(1, true, "topic1");
kafkaEmbedded.before();
}
#Bean
public KafkaTemplate<String, Message> sender(ProtobufSerializer protobufSerializer,
KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry) throws Exception {
KafkaTemplate<String, Message> sender = KafkaTestUtils.newTemplate(kafkaEmbedded, new StringSerializer(),
protobufSerializer);
for (MessageListenerContainer listenerContainer :
registry.getListenerContainers()) {
ContainerTestUtils.waitForAssignment(listenerContainer,
kafkaEmbedded.getPartitionsPerTopic());
}
return sender;
}
Test class :
#RunWith(SpringRunner.class)
public class DeviceEnergyKafkaListenerIT {
...
#Autowired
private KafkaTemplate<String, Message> sender;
#Test
public void test (){
...
sender.send(topic, msg);
sender.flush();
}
Any ideas how to resolve this please ?
With a #ClassRule broker, add an #AfterClass method...
#AfterClass
public static void tearDown() {
embeddedKafka.getKafkaServers().forEach(b -> b.shutdown());
embeddedKafka.getKafkaServers().forEach(b -> b.awaitShutdown());
}
For a #Rule or bean, use an #After method.
final KafkaServer server =
embeddedKafka.getKafkaServers().stream().findFirst().orElse(null);
if(server != null) {
server.replicaManager().shutdown(false);
final Field replicaManagerField = server.getClass().getDeclaredField("replicaManager");
if(replicaManagerField != null) {
replicaManagerField.setAccessible(true);
replicaManagerField.set(server, null);
}
}
embeddedKafka.after();
For a more detail discussion you can refer this thread
Embedded kafka issue with multiple tests using the same context
The following solution provided by mhyeon-lee has worked for me:
import org.apache.kafka.common.utils.Exit
class SomeTest {
static {
Exit.setHaltProcedure((statusCode, message) -> {
if (statusCode != 1) {
Runtime.getRuntime().halt(statusCode);
}
});
}
#Test
void test1() {
}
#Test
void test2() {
}
}
When JVM shutdown Hook is running, kafka log file is deleted and
Exit.halt (1) is called when other shutdown hook accesses kafka log
file at the same time.
Since halt is called here and status is 1, i only defend against 1.
https://github.com/a0x8o/kafka/blob/master/core/src/main/scala/kafka/log/LogManager.scala#L193
If you encounter a situation where the test fails with a different
status value, you can add defense code.
An error log may occur, but the test will not fail because the command
is not propagated to Runtime.halt.
References:
https://github.com/spring-projects/spring-kafka/issues/194#issuecomment-612875646
https://github.com/spring-projects/spring-kafka/issues/194#issuecomment-613548108

Reaching a HTTPS Rest api using Spring RestTemplate by-passing host verification

I've read many things about this problem, and I thought I had found the most simple workaround (last code example from here : http://www.baeldung.com/httpclient-ssl), but it doesn't work.
Here is how I declare my RestTemplate with hostname verification turned off (and a proxy setting) :
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.setProxy(new HttpHost("10.xx.xx.xx", 3128, "http"))
.build();
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
return builder.requestFactory(requestFactory).build();
//return builder.build();
}
And here is the code to create my POST request :
LoginResponse loginResponse = restTemplate.postForObject("https://interflex.svc.suezsmartsolutions.com/path/to/my/api", loginRequest, LoginResponse.class);
And here is the exception I get (just like if I had not turned off hostname verifier) :
16:15:27 ERROR org.springframework.boot.SpringApplication:771 - Application startup failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:735)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:716)
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:703)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:304)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
at hello.Application.main(Application.java:23)
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://interflex.svc.suezsmartsolutions.com/path/to/api": sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:673)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:620)
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:387)
at hello.Application.lambda$0(Application.java:45)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:732)
... 6 common frames omitted
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.upgrade(DefaultHttpClientConnectionOperator.java:193)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.upgrade(PoolingHttpClientConnectionManager.java:375)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:416)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:89)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:659)
... 10 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
... 32 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
... 38 common frames omitted
Could someone help me get rid of this exception ?
Also I don't know why this exception appears in the first place, since the root CA used to generate the site certificate (VeriSign) is present in my truststore (cacerts) (the intermediate authority is not present though, could it be the reason ?).
Simple Fix, just skip the certificate
TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(
new SSLContextBuilder().loadTrustMaterial(null, acceptingTrustStrategy).build();
Your issue seems more related to a certificate issue than to proxy configuration.
Anyway in my projects I'm using this configuration:
#Bean
#Autowired
public RestTemplate restTemplate(ClientHttpRequestFactory factory)
{
RestTemplate result = new RestTemplate(factory);
return result;
}
#Bean
public ClientHttpRequestFactory requestFactory() throws Exception
{
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setHttpClient(httpClient());
return factory;
}
#Bean
public HttpClient httpClient() throws Exception
{
int timeout = new Integer(env.getProperty("web.http.client.timeout"));
CloseableHttpClient httpClient = null;
//I load a JSON where I specify the name and the PWD of keystores I want to use
String keystores = "keyStoreInfo.json";
PoolingHttpClientConnectionManager pcm = null;
if(StringUtils.hasText(keystores))
{
Resource jsonRes = new ClassPathResource(keystores);
if( jsonRes.exists() )
{
List<KeyStoreInfo> ksInfo = objectMapper().readValue(jsonRes.getInputStream(), new TypeReference<List<KeyStoreInfo>>()
{
});
SSLContext sslCtx = SSLContext.getInstance("TLS");
List<KeyManager> keymanagers = new ArrayList<KeyManager>();
for (KeyStoreInfo ksi : ksInfo)
{
String keystoreName = ksi.getNomeKeyStore();
String keyStorePwd = ksi.getPasswordKeyStore();
if( StringUtils.hasText(keystoreName) )
{
Resource keystoreRes = new ClassPathResource(keystoreName);
KeyMaterial km = new KeyMaterial(keystoreRes.getInputStream(), keyStorePwd.toCharArray());
KeyStore clientStore = km.getKeyStore();
KeyManagerFactory kmfactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmfactory.init(clientStore, keyStorePwd != null ? keyStorePwd.toCharArray() : null);
keymanagers.addAll(Arrays.asList(kmfactory.getKeyManagers()));
}
}
if( !keymanagers.isEmpty() )
{
X509TrustManager tm = new X509TrustManager() {
#Override
public void checkClientTrusted(java.security.cert.X509Certificate[] arg0, String arg1)
throws CertificateException {
}
#Override
public void checkServerTrusted(java.security.cert.X509Certificate[] arg0, String arg1)
throws CertificateException {
}
#Override
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
};
sslCtx.init(keymanagers.toArray(new KeyManager[keymanagers.size()]), new TrustManager[]{tm}, null);
SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(sslCtx);
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create().register("https", sslConnectionFactory).register("http", new PlainConnectionSocketFactory()).build();
pcm = new PoolingHttpClientConnectionManager(registry);
}
else
{
if( logger.isInfoEnabled() )
{
logger.info("Nessun keystore presente nel JSON di configurazione {}. Creo un PoolingHttpClientConnectionManager di default",keystores);
}
pcm = new PoolingHttpClientConnectionManager();
}
}
}
else
{
if( logger.isInfoEnabled() )
{
logger.info("Nessun keystore da caricare. Creo un PoolingHttpClientConnectionManager di default");
}
pcm = new PoolingHttpClientConnectionManager();
}
HttpClientBuilder hcb = HttpClientBuilder.create();
pcm.closeIdleConnections(timeout, TimeUnit.MILLISECONDS);
RequestConfig config = RequestConfig.custom().setConnectionRequestTimeout(timeout).setSocketTimeout(timeout).setConnectTimeout(timeout).build();
hcb.setDefaultRequestConfig(config);
hcb.setConnectionManager(pcm).setConnectionManagerShared(true);
boolean proxyEnable = new Boolean(env.getProperty("web.http.client.proxyEnable"));
if (proxyEnable)
{
int proxyPort = new Integer(env.getProperty("web.http.client.portProxy"));
String proxyHost = env.getProperty("web.http.client.hostProxy");
BasicCredentialsProvider credentialProvider = new BasicCredentialsProvider();
AuthScope scope = new AuthScope(proxyHost, proxyPort);
String usernameProxy = env.getProperty("web.http.client.usernameProxy");
String passwordProxy = env.getProperty("web.http.client.passwordProxy");
if (StringUtils.hasText(usernameProxy) && StringUtils.hasText(passwordProxy))
{
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(usernameProxy, passwordProxy);
credentialProvider.setCredentials(scope, credentials);
}
ProxyRoutePlanner proxyRoutPlanner = new ProxyRoutePlanner(new HttpHost(proxyHost, proxyPort), env.getProperty("web.http.client.urlNotProxy"));
hcb.setDefaultCredentialsProvider(credentialProvider).setRoutePlanner(proxyRoutPlanner);
}
WsKeepAliveStrategy cas = new WsKeepAliveStrategy();
cas.setTimeout(new Long(timeout));
hcb.setKeepAliveStrategy(cas);
httpClient = hcb.build();
return httpClient;
}
Where WsKeepAliveStrategy is:
public class WsKeepAliveStrategy implements ConnectionKeepAliveStrategy
{
private Long timeout;
#Override
public long getKeepAliveDuration(HttpResponse response, HttpContext context)
{
return timeout;
}
public void setTimeout(Long timeout)
{
this.timeout = timeout;
}
}
and ProxyRoutePlanner is:
public class ProxyRoutePlanner extends DefaultProxyRoutePlanner
{
private List<String> urlsNotProxy = null;
private boolean useAlwaysSuper = false;
public ProxyRoutePlanner(HttpHost proxy, String urlNotProxy)
{
super(proxy);
if (!StringUtils.hasText(urlNotProxy))
this.useAlwaysSuper = true;
else
{
this.urlsNotProxy = Arrays.asList(urlNotProxy.split(","));
}
}
#Override
public HttpRoute determineRoute(HttpHost host, HttpRequest request, HttpContext context) throws HttpException
{
String hostname = host.getHostName();
if (this.useAlwaysSuper || this.urlsNotProxy.contains(hostname) == false)
return super.determineRoute(host, request, context);// Super method
// with proxy
if ("http".equals(host.getSchemeName()))
return new HttpRoute(host);// Direct Route
HttpClientContext clientContext = HttpClientContext.adapt(context);
RequestConfig config = clientContext.getRequestConfig();
InetAddress local = config.getLocalAddress();
return new HttpRoute(host, local, true);
}
}
I'm using this configuration and I'm having no issue
In any case you should check what kind of certificate is necessary to use in your rest invocation
I hope it's useful
Angelo
The short answer is "yes you need the intermediate authority in your truststore, not only the root CA".

How to override the thread pool in Spring Boot app

I am writing below code to override the thread pool. But it is not working properly. What is the correct way to set override the thread pool in spring boot app startup. Note that i don't have a control in my code over the Server instance, so instantiating a Server is not a solution for my need.
#Bean
public EmbeddedServletContainerCustomizer getContainerCustomizer() {
return (configurableEmbeddedServletContainer) -> {
if (configurableEmbeddedServletContainer instanceof JettyEmbeddedServletContainerFactory) {
((JettyEmbeddedServletContainerFactory)configurableEmbeddedServletContainer).addServerCustomizers((server) -> {
QueuedThreadPool newPool = new QueuedThreadPool(10);
QueuedThreadPool oldPool = server.getBean(QueuedThreadPool.class);
server.updateBean(oldPool, newPool);
});
}
};
}
When i execute the code, it was throwing below error
Exception in thread "main"
java.util.concurrent.RejectedExecutionException: org.eclipse.jetty.io.ManagedSelector#1ee0005 id=0 keys=0 selected=0
at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:377)
at org.eclipse.jetty.io.SelectorManager.execute(SelectorManager.java:125)
at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:255)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:260)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:384)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
I tried by sample main code also, and it is also giving same error.
public class FileServer
{
public static void main(String[] args) throws Exception
{
Server server = new Server(9090);
QueuedThreadPool newPool = new QueuedThreadPool(10);
QueuedThreadPool oldPool = server.getBean(QueuedThreadPool.class);
server.updateBean(oldPool, newPool);
ResourceHandler resource_handler = new ResourceHandler();
resource_handler.setDirectoriesListed(true);
resource_handler.setWelcomeFiles(new String[]{ "index.html" });
resource_handler.setResourceBase(".");
HandlerList handlers = new HandlerList();
handlers.setHandlers(new Handler[] { resource_handler, new DefaultHandler() });
server.setHandler(handlers);
server.start();
server.join();
}
}
Here is an example of configuring a thread pool in Jetty with properties and a different type of thread pool. The thread pool in my case is InstrumentedQueuedThreadPool.
#Configuration
public class CustomJettyConfiguration {
#Bean
public JettyEmbeddedServletContainerFactory jettyEmbeddedServletContainerFactory(
#Value("${server.port:8080}") final String port,
#Value("${jetty.threadPool.maxThreads:600}") final String maxThreads,
#Value("${jetty.threadPool.minThreads:10}") final String minThreads,
#Value("${jetty.threadPool.idleTimeout:5000}") final String idleTimeout) {
final JettyEmbeddedServletContainerFactory factory =
new JettyEmbeddedServletContainerFactory(Integer.valueOf(port));
// Tweak the connection pool used by Jetty to handle incoming HTTP connections
InstrumentedQueuedThreadPool instThreadPool =
new InstrumentedQueuedThreadPool(registry);
instThreadPool.setPrefix("jetty");
instThreadPool.setMaxThreads(Integer.valueOf(maxThreads));
instThreadPool.setMinThreads(Integer.valueOf(minThreads));
instThreadPool.setIdleTimeout(Integer.valueOf(idleTimeout));
factory.setThreadPool(instThreadPool);
...
return factory;
}
}
In spring boot 2.x, the JettyEmbeddedServletContainerFactory has been replaced by org.springframework.boot.web.embedded.jetty.JettyServletWebServerFactory.
You need to customize JettyServletWebServerFactory and set your own thread pool.
#Component
public class JettyConfiguration implements WebServerFactoryCustomizer<JettyServletWebServerFactory> {
#Override
public void customize(JettyServletWebServerFactory factory) {
// customize your thread pool here
QueuedThreadPool qtp = new QueuedThreadPool(80, 500);
qtp.setName("jettyThreadPool");
factory.setThreadPool(qtp);
}
}

Camel lookup RemoteConnectionFactory on Wildfly

I'm very new to Apache Camel and completely new to Spring. I'm tryin to send some JMS messages to the embedded hornetq in Wildfly (ver.8.1.0). Here is my code:
public class CamelJMS {
public static void main(String[] args) throws Exception {
try {
CamelContext cc = new DefaultCamelContext();
Properties prop = new Properties();
prop.put(Context.INITIAL_CONTEXT_FACTORY,
"org.jboss.naming.remote.client.InitialContextFactory");
prop.put(Context.URL_PKG_PREFIXES,
"org.jboss.jms.jndi.JNDIProviderAdapter");
prop.put(Context.PROVIDER_URL, "http-remoting://localhost:8080");
prop.put(Context.SECURITY_PRINCIPAL,
System.getProperty("username", "usr"));
prop.put(Context.SECURITY_CREDENTIALS,
System.getProperty("password", "pwd"));
JndiTemplate jndiT = new JndiTemplate(prop);
jndiT.bind("ccf", "jms/RemoteConnectionFactory");
JndiObjectFactoryBean jndiCFB = new JndiObjectFactoryBean();
jndiCFB.setJndiTemplate(jndiT);
JmsComponent jmsC = JmsComponent.jmsComponent((ConnectionFactory)jndiCFB.getObject());
cc.addComponent("jmsrc", jmsC);
cc.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
System.out.println("Go!");
onException(Throwable.class)
.handled(true)
.process(new Processor() {
#Override
public void process(Exchange arg0) throws Exception {
System.out.println("error.");
((Exception) arg0.getProperty("CamelExceptionCaught", Exception.class)).printStackTrace();
}
});
from("file:///Users/Foo/Desktop/IN")
.process(new Processor() {
#Override
public void process(Exchange arg0) throws Exception {
System.out.println(arg0.getIn().getHeader("CamelFileAbsolutePath", String.class));
System.out.println(arg0.getIn().getBody(String.class));
}
})
.to("jms:jms/generatoreQueue?connectionFactory=ccf");
}
});
cc.start();
Thread.sleep(10000);
cc.stop();
} catch (Exception e) {
e.printStackTrace();
}
}
}
I'm sure about my Wildfly's configuration because I can access the same queue using a non Camel client. When I launch my client I got this error:
org.jboss.naming.remote.protocol.NamingIOException: Failed to bind [Root exception is java.io.IOException: Internal server error.]
at org.jboss.naming.remote.client.ClientUtil.namingException(ClientUtil.java:49)
at org.jboss.naming.remote.protocol.v1.Protocol$2.execute(Protocol.java:220)
at org.jboss.naming.remote.protocol.v1.Protocol$2.execute(Protocol.java:179)
at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1.bind(RemoteNamingStoreV1.java:108)
at org.jboss.naming.remote.client.HaRemoteNamingStore$2.operation(HaRemoteNamingStore.java:288)
at org.jboss.naming.remote.client.HaRemoteNamingStore$2.operation(HaRemoteNamingStore.java:285)
at org.jboss.naming.remote.client.HaRemoteNamingStore.namingOperation(HaRemoteNamingStore.java:137)
at org.jboss.naming.remote.client.HaRemoteNamingStore.bind(HaRemoteNamingStore.java:284)
at org.jboss.naming.remote.client.RemoteContext.bind(RemoteContext.java:133)
at org.jboss.naming.remote.client.RemoteContext.bind(RemoteContext.java:137)
at javax.naming.InitialContext.bind(InitialContext.java:419)
at org.springframework.jndi.JndiTemplate$2.doInContext(JndiTemplate.java:198)
at org.springframework.jndi.JndiTemplate.execute(JndiTemplate.java:87)
at org.springframework.jndi.JndiTemplate.bind(JndiTemplate.java:196)
at edu.pezzati.camel.jms.broker.CamelJMSBroker.main(CamelJMSBroker.java:38)
Caused by: java.io.IOException: Internal server error.
at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Looking to the server log, I found this:
...
09:12:02,585 INFO [org.jboss.as.naming] (default task-5) JBAS011806: Channel end notification received, closing channel Channel ID 793d9f9e (inbound) of Remoting c
onnection 4406e6f5 to /127.0.0.1:49289
09:12:20,121 ERROR [org.jboss.as.naming] (pool-1-thread-1) JBAS011807: Unexpected internal error: java.lang.UnsupportedOperationException: JBAS011859: Naming context is read-only
at org.jboss.as.naming.WritableServiceBasedNamingStore.requireOwner(WritableServiceBasedNamingStore.java:161)
at org.jboss.as.naming.WritableServiceBasedNamingStore.bind(WritableServiceBasedNamingStore.java:66)
at org.jboss.as.naming.NamingContext.bind(NamingContext.java:253)
at org.jboss.naming.remote.protocol.v1.Protocol$2.handleServerMessage(Protocol.java:249)
at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
...
Of course I'm misconfiguring something in Spring's JndiTemplate but I can't figure out what.
I'm not familiar with Wildfly and only a little familiar with JBoss, but you said the configuration parameters should be correct. So based off of my experience configuring the Camel JmsComponent...
Try this:
Properties prop = new Properties();
prop.put(Context.INITIAL_CONTEXT_FACTORY,
"org.jboss.naming.remote.client.InitialContextFactory");
prop.put(Context.URL_PKG_PREFIXES,
"org.jboss.jms.jndi.JNDIProviderAdapter");
prop.put(Context.PROVIDER_URL, "http-remoting://localhost:8080");
prop.put(Context.SECURITY_PRINCIPAL,
System.getProperty("username", "usr"));
prop.put(Context.SECURITY_CREDENTIALS,
System.getProperty("password", "pwd"));
Context context = new InitialContext(prop);
ConnectionFactory connectionFactory = (ConnectionFactory) context.lookup("jms/RemoteConnectionFactory");
JmsComponent jmsC = new JmsComponent(connectionFactory);
cc.addComponent("jms", jmsC);
And change your endpoint to:
.to("jms:queue:generatoreQueue");
You shouldn't need the "jms/" in front of the queue name, and the component name should be the same as what you bound it to above in the addComponent method.

Resources