Spring Boot elasticsearch produce warnings: Transport response handler not found of id - spring-boot

I setuped a web app with org.springframework.boot:spring-boot-starter-data-elasticsearch. Everything work well - I can populate indexes with my stand alone Elasticsearch 5. But I continue to receive some weird warnings:
2018-05-08 03:07:57.940 WARN 32053 --- [ient_boss][T#7]] o.e.transport.TransportService : Transport response handler not found of id [5]
2018-05-08 03:08:02.949 WARN 32053 --- [ient_boss][T#8]] o.e.transport.TransportService : Transport response handler not found of id [7]
2018-05-08 03:08:07.958 WARN 32053 --- [ient_boss][T#1]] o.e.transport.TransportService : Transport response handler not found of id [9]
2018-05-08 03:08:12.970 WARN 32053 --- [ient_boss][T#2]] o.e.transport.TransportService : Transport response handler not found of id [11]
...
Simple app to reproduce:
#SpringBootApplication
public class App {
#Configuration
#EnableElasticsearchRepositories(basePackages = "com.test")
public class EsConfig {
#Value("${elasticsearch.host}")
private String esHost;
#Value("${elasticsearch.port}")
private int esPort;
#Value("${elasticsearch.clustername}")
private String esClusterName;
#Bean
public TransportClient client() throws Exception {
Settings esSettings = Settings.builder().put("cluster.name", esClusterName).build();
InetSocketTransportAddress socketAddress = new InetSocketTransportAddress(
InetAddress.getByName(esHost), esPort);
return new PreBuiltTransportClient(esSettings).addTransportAddress(socketAddress);
}
#Bean
public ElasticsearchOperations elasticsearchTemplate(Client client) throws Exception {
return new ElasticsearchTemplate(client);
}
}
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
My compose file for ES
version: "2.3"
services:
elasticsearch:
image: 'docker.elastic.co/elasticsearch/elasticsearch:5.6.8
ports:
- "9200:9200"
- "9300:9300"
environment:
- xpack.security.enabled=false
- ES_JAVA_OPTS=-Xms700m -Xmx700m
- PATH_LOGS="/tmp/el-log"
- cluster.name=dou
cpu_shares: 1024
mem_limit: 1024MB
As a project transitive dependency I have org.elasticsearch.client:transport:5.6.8. Looks like versions of ES instance & library are the same.
So, what does this warning mean & how should we deal with it?

Related

Spring data cassandra - error while opening new channel

I have a problem with Cassandra's connection with spring-data. When Cassandra is running locally I have no problem with connecting, however when I ran my spring-boot app in k8s with external Cassandra I am stuck on WARN:
2020-07-24 10:26:32.398 WARN 6 --- [ s0-admin-0] c.d.o.d.internal.core.pool.ChannelPool : [s0|/127.0.0.1:9042] Error while opening new channel (ConnectionInitException: [s0|connecting...] Protocol initialization request, step 1 (STARTUP {CQL_VERSION=3.0.0, DRIVER_NAME=DataStax Java driver for Apache Cassandra(R), DRIVER_VERSION=4.7.2, CLIENT_ID=9679ee85-ff39-45b6-8573-62a8d827ec9e}): failed to send request (java.nio.channels.ClosedChannelException))
I don't understand why in the log I have [s0|/127.0.0.1:9042] instead of the IP of my contact points.
Spring configuration:
spring:
data:
cassandra:
keyspace-name: event_store
local-datacenter: datacenter1
contact-points: host1:9042,host2:9042
Also this WARN is not causing that spring-boot won't start however if I do query in service I have error:
{ error: "Query; CQL [com.datastax.oss.driver.internal.core.cql.DefaultSimpleStatement#9463dccc]; No node was available to execute the query; nested exception is com.datastax.oss.driver.api.core.NoNodeAvailableException: No node was available to execute the query" }
Option 1: test your yml file like this. (Have you tried with ip address?)
data:
cassandra:
keyspace-name: event_store
local-datacenter: datacenter1
port:9042
contact-points: host1,host2
username: cassandra
password: cassandra
Option 2: Create new properties on your yml and than a configuration class
cassandra:
database:
keyspace-name: event_store
contact-points: host1, host2
port: 9042
username: cassandra
password: cassandra
#Configuration
public class CassandraConfig extends AbstractCassandraConfiguration {
#Value("${cassandra.database.keyspace-name}")
private String keySpace;
#Value("${cassandra.database.contact-points}")
private String contactPoints;
#Value("${cassandra.database.port}")
private int port;
#Value("${cassandra.database.username}")
private String userName;
#Value("${cassandra.database.password}")
private String password;
#Override
protected String getKeyspaceName() {
return keySpace;
}
#Bean
public CassandraMappingContext cassandraMapping() throws ClassNotFoundException {
CassandraMappingContext context = new CassandraMappingContext();
context.setUserTypeResolver(new SimpleUserTypeResolver(cluster().getObject(), keySpace));
return context;
}
#Bean
public CassandraClusterFactoryBean cluster() {
CassandraClusterFactoryBean cluster = super.cluster();
cluster.setUsername(userName);
cluster.setPassword(password);
cluster.setContactPoints(contactPoints);
cluster.setPort(port);
return cluster;
}
#Override
protected boolean getMetricsEnabled() {
return false;
}
}

Spring boot rabbitmq no exchange '"xxxxxxx"' in vhost '/'

I'm writing a simple rabbitmq producer with spring boot 2.2.7.
On the broker side I've setup a direct exchange samples , a queue named samples.default and binded them together adding a samples.default bindkey key.
when running the application I get the following error
Attempting to connect to: [127.0.0.1:5672]
2020-05-14 15:13:39.232 INFO 28393 --- [nio-8080-exec-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory#2f860823:0/SimpleConnection#3946e760 [delegate=amqp://open-si#127.0.0.1:5672/, localPort= 34710]
2020-05-14 15:13:39.267 ERROR 28393 --- [ 127.0.0.1:5672] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange '"samples"' in vhost '/', class-id=60, method-id=40)
The rabbitmq server configuration is correct as I've a python producer that already puts messages succesfully in the "samples.default" queue.
in Spring boot I'm using jackson serialization, but that's not the prolem here I think as I've tested the code without the Jakson serialization configuration and the problem is still the same.
My broker configuration is set both in the application.properties :
#spring.rabbitmq.host=localhost
spring.rabbitmq.addresses=127.0.0.1
spring.rabbitmq.port=5672
spring.rabbitmq.username=xxxx
spring.rabbitmq.password=xxxx
broker.exchange = "samples"
broker.routingKey = "samples.default"
note that using spring.rabbitmq.host doesn't work as it results in using my internet provider address !
and in a BrokerConf configuration class :
#Configuration
public class BrokerConf {
#Bean("publisher")
MessagePublisher<BaseSample> baseSamplePublisher(RabbitTemplate rabbitTemplate) {
return new MessagePublisher<BaseSample>(rabbitTemplate);
}
#Bean
public RabbitTemplate rabbitTemplate(final ConnectionFactory connectionFactory) {
final var rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(producerJackson2MessageConverter());
return rabbitTemplate;
}
#Bean
public MessageConverter producerJackson2MessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
The publisher base class is as :
#Component
public class MessagePublisher<T> {
private static final Logger log = LoggerFactory.getLogger(MessagePublisher.class);
private final RabbitTemplate rabbitTemplate;
public MessagePublisher(RabbitTemplate r) {
rabbitTemplate = r;
}
public void publish(List<BaseSample> messages, String exchange, String routingKey) {
for (BaseSample message: messages) {
rabbitTemplate.convertAndSend(exchange, routingKey, message);
}
}
}
that I use in a rest controller
private static final Logger logger = LoggerFactory.getLogger(SamplesController.class);
#Autowired
private MessagePublisher<BaseSample> publisher;
#Value("${broker.exchange}")
private String exchange;
#Value("${broker.routingKey}")
private String routingKey;
#PutMapping(value = "/new", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<SampleAck> add(#RequestBody List<BaseSample> samples) {
publisher.publish(samples, exchange, routingKey);
return ResponseEntity.ok(new SampleAck(samples.size(), new Date()));
}
So the broker connection is OK but the exchange is not found
and rabbitmq resources exists
xxxxxx#xxxxxxx:~/factory/udc-collector$ sudo rabbitmqctl list_exchanges
Listing exchanges for vhost / ...
name type
amq.topic topic
amq.rabbitmq.trace topic
amq.match headers
amq.direct direct
amq.fanout fanout
direct
amq.rabbitmq.log topic
amq.headers headers
samples direct
xxxx#xxxxx:~/factory/udc-collector$ sudo rabbitmqctl list_queues
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
name messages
samples.default 2
Any idea ?
thanks in advance.
The error seems quite obvious:
no exchange '"samples"' in vhost
broker.exchange = "samples"
broker.routingKey = "samples.default"
Remove the quotes
broker.exchange=samples
broker.routingKey=samples.default

route FROM and route TO with spring cloud stream and functions

I have some issues with the new routing feature in spring cloud stream
I tried to implement a simple scenario, I want to send a message with a header spring.cloud.function.definition = consume1 or consume2
I expect that consume1 or consume2 should be called based on what is sent on the header but the methods are called randomly.
I send the message to the exchange consumer using the rabbit admin console
I'm having the following logs:
2020-02-27 14:48:25.896 INFO 22132 --- [ consumer.app-1] com.example.demo.TestConsumer : ==============>consume1 messge [[payload=ok, headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=#, amqp_receivedExchange=consumer, amqp_deliveryTag=1, deliveryAttempt=1, amqp_consumerQueue=consumer.app, amqp_redelivered=false, id=9a4dff25-88ef-4d76-93e2-c8719cda122d, spring.cloud.function.definition=consume1, amqp_consumerTag=amq.ctag-gGChFNCKIVd25yyR9H6-fQ, sourceData=(Body:'[B#3a92faa7(byte[2])' MessageProperties [headers={spring.cloud.function.definition=consume1}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=consumer, receivedRoutingKey=#, deliveryTag=1, consumerTag=amq.ctag-gGChFNCKIVd25yyR9H6-fQ, consumerQueue=consumer.app]), timestamp=1582811303347}]]
2020-02-27 14:48:25.984 INFO 22132 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2020-02-27 14:48:25.984 INFO 22132 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2020-02-27 14:48:25.991 INFO 22132 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 7 ms
2020-02-27 14:48:26.037 INFO 22132 --- [oundedElastic-1] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel customer-1
2020-02-27 14:48:26.111 INFO 22132 --- [oundedElastic-1] o.s.c.s.m.DirectWithAttributesChannel : Channel 'application.customer-1' has 1 subscriber(s).
2020-02-27 14:48:26.116 INFO 22132 --- [oundedElastic-1] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:5672]
2020-02-27 14:48:26.123 INFO 22132 --- [oundedElastic-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory.publisher#32438e24:0/SimpleConnection#3e58666d [delegate=amqp://guest#127.0.0.1:5672/, localPort= 62514]
2020-02-27 14:48:26.139 INFO 22132 --- [-1.customer-1-1] o.s.i.h.s.MessagingMethodInvokerHelper : Overriding default instance of MessageHandlerMethodFactory with provided one.
2020-02-27 14:48:26.140 INFO 22132 --- [-1.customer-1-1] com.example.demo.TestSink : Data received customer-1...body
2020-02-27 14:49:14.185 INFO 22132 --- [ consumer.app-1] o.s.i.h.s.MessagingMethodInvokerHelper : Overriding default instance of MessageHandlerMethodFactory with provided one.
2020-02-27 14:49:14.194 INFO 22132 --- [ consumer.app-1] com.example.demo.TestConsumer : ==============>consume2 messge [[payload=ok, headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=#, amqp_receivedExchange=consumer, amqp_deliveryTag=1, deliveryAttempt=1, amqp_consumerQueue=consumer.app, amqp_redelivered=false, id=33581edb-2832-1c92-b765-a05794512b34, spring.cloud.function.definition=consume1, amqp_consumerTag=amq.ctag-RIp2nZdcG2a0hNQeImwtBw, sourceData=(Body:'[B#8159793(byte[2])' MessageProperties [headers={spring.cloud.function.definition=consume1}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=consumer, receivedRoutingKey=#, deliveryTag=1, consumerTag=amq.ctag-RIp2nZdcG2a0hNQeImwtBw, consumerQueue=consumer.app]), timestamp=1582811354186}]]
2020-02-27 14:49:14.203 INFO 22132 --- [oundedElastic-1] o.s.i.monitor.IntegrationMBeanExporter : Registering MessageChannel customer-2
2020-02-27 14:49:14.213 INFO 22132 --- [oundedElastic-1] o.s.c.s.m.DirectWithAttributesChannel : Channel 'application.customer-2' has 1 subscriber(s).
2020-02-27 14:49:14.216 INFO 22132 --- [-2.customer-2-1] o.s.i.h.s.MessagingMethodInvokerHelper : Overriding default instance of MessageHandlerMethodFactory with provided one.
2020-02-27 14:49:14.216 INFO 22132 --- [-2.customer-2-1] com.example.demo.TestSink : Data received customer-2...body
application.yml
spring:
main:
allow-bean-definition-overriding: true
spring.cloud.stream:
function.definition: supplier;receive1;receive2;consume1;consume2
function.routing:
enabled: true
bindings:
consume1-in-0.destination: consumer
consume1-in-0.group: app
consume2-in-0.destination: consumer
consume2-in-0.group: app
receive1-in-0.destination: customer-1
receive1-in-0.group: customer-1
receive2-in-0.destination: customer-2
receive2-in-0.group: customer-2
DemoApplication.java
import com.fasterxml.jackson.databind.ObjectMapper
import org.apache.commons.logging.Log
import org.apache.commons.logging.LogFactory
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.context.annotation.Bean
import org.springframework.http.HttpStatus
import org.springframework.messaging.Message
import org.springframework.messaging.support.MessageBuilder
import org.springframework.stereotype.Component
import org.springframework.web.bind.annotation.PathVariable
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RequestMethod.GET
import org.springframework.web.bind.annotation.ResponseStatus
import org.springframework.web.bind.annotation.RestController
import org.springframework.web.client.RestTemplate
import reactor.core.publisher.EmitterProcessor
import reactor.core.publisher.Flux
import java.util.function.Consumer
import java.util.function.Supplier
#SpringBootApplication
class DemoApplication
fun main(args: Array<String>) {
runApplication<DemoApplication>(*args)
}
#RestController
class DynamicDestinationController(private val jsonMapper: ObjectMapper) {
private val processor: EmitterProcessor<Message<String>> = EmitterProcessor.create<Message<String>>()
#RequestMapping(path = ["/api/dest/{destName}"], method = [GET], consumes = ["*/*"])
#ResponseStatus(HttpStatus.ACCEPTED)
fun handleRequest(#PathVariable destName:String) {
val message: Message<String> = MessageBuilder.withPayload("body")
.setHeader("spring.cloud.stream.sendto.destination", destName).build()
processor.onNext(message)
}
#Bean
fun supplier(): Supplier<Flux<Message<String>>> {
return Supplier { processor }
}
}
const val destResourceUrl = "http://localhost:8080/api/dest"
#Component
class TestConsumer() {
private val restTemplate: RestTemplate = RestTemplate()
private val logger: Log = LogFactory.getLog(javaClass)
#Bean
fun consume1(): Consumer<Message<String>> = Consumer {
logger.info("==============>consume1 messge [[payload=${it.payload}, headers=${it.headers}]]")
restTemplate.getForEntity("$destResourceUrl/customer-1", String::class.java)
}
#Bean
fun consume2(): Consumer<Message<String>> = Consumer {
logger.info("==============>consume2 messge [[payload=${it.payload}, headers=${it.headers}]]")
restTemplate.getForEntity("$destResourceUrl/customer-2", String::class.java)
}
}
#Component
class TestSink {
private val logger: Log = LogFactory.getLog(javaClass)
#Bean
fun receive1(): Consumer<String> = Consumer {
logger.info("Data received customer-1..." + it);
}
#Bean
fun receive2(): Consumer<String> = Consumer {
logger.info("Data received customer-2..." + it);
}
}
Any idea how to fix the route to consumer?
thanks in advance.
demo-repo
Actually I am a bit confused, so let's do one step at the time. Here is the functioning (modelled after yours) app which uses sendto feature allowing you to send messages to the specific (existing and/or dynamically resolved) destinations.
(in java but you can rework it to Kotlin)
#Controller
public class WebSourceApplication {
public static void main(String[] args) {
SpringApplication.run(WebSourceApplication.class,
"--spring.cloud.function.definition=supplier;consA;consB",
"--spring.cloud.stream.bindings.consA-in-0.destination=consumerA",
"--spring.cloud.stream.bindings.consA-in-0.group=consumerA-grp",
"--spring.cloud.stream.bindings.consB-in-0.destination=consumerB",
"--spring.cloud.stream.bindings.consB-in-0.group=consumerB-grp"
);
}
EmitterProcessor<Message<String>> processor = EmitterProcessor.create();
#RequestMapping(path = "/api/dest/{destName}", consumes = "*/*")
#ResponseStatus(HttpStatus.ACCEPTED)
public void delegateToSupplier(#RequestBody String body, #PathVariable String destName) {
Message<String> message = MessageBuilder.withPayload(body)
.setHeader("spring.cloud.stream.sendto.destination", destName)
.build();
processor.onNext(message);
}
#Bean
public Supplier<Flux<Message<String>>> supplier() {
return () -> processor;
}
#Bean
public Consumer<String> consA() {
return v -> {
System.out.println("Consuming from consA: " + v);
};
}
#Bean
public Consumer<String> consB() {
return v -> {
System.out.println("Consuming from consB: " + v);
};
}
}
And when i curl it i get consistent invocation pr the appropriate consumer:
curl -H "Content-Type: application/json" -X POST -d "Hello Spring Cloud Stream" http://localhost:8080/api/dest/consumerA
log: Consuming from consA: Hello Spring Cloud Stream
. . .
curl -H "Content-Type: application/json" -X POST -d "Hello Spring Cloud Stream" http://localhost:8080/api/dest/consumerB
log: Consuming from consB: Hello Spring Cloud Stream
Notice: There is no enable routing property. That feature is mainly aimed to always call one function functionRouter and have it call other functions on your behalf. It is a feature of spring-cloud-function which means it works outside of spring-cloud-srteam and channels/destinations etc.
Isn't that what you are trying to accomplish? Send message to a different destination based on some oath variable in your HTTP request?
Here is an example of a different microservice which receives on routing function which hen routes to different functions
public class FunctionRoutingApplication {
public static void main(String[] args) {
SpringApplication.run(FunctionRoutingApplication.class,
"--spring.cloud.stream.function.routing.enabled=true"
);
}
#Bean
public Consumer<String> consA() {
return v -> {
System.out.println("Consuming from consA: " + v);
};
}
#Bean
public Consumer<String> consB() {
return v -> {
System.out.println("Consuming from consB: " + v);
};
}
}
And that's pretty much it. Go to your broker and send data to functionRouter-in-0 exchange while providing spring.cloud.function.definition=consA/consB headers and you will see consistent invocations.
Am I still missing something?

Problem with #PropertySource and Map binding

I have this specific problem with my yml configuration file.
I have a multi-module maven project as follows:
app
|-- core
|-- web
|-- app
I have this configuration file in core project
#Configuration
#PropertySource("core-properties.yml")
public class CoreConfig {
}
And this mapping:
#Component
#ConfigurationProperties(prefix = "some.key.providers.by")
#Getter
#Setter
public class ProvidersByMarket {
private Map<String, List<String>> market;
}
Here are my core-properties.yml
some.key.providers:
p1: 'NAME1'
p2: 'NAME2'
some.key.providers.by.market:
de:
- ${some.key.providers.p1}
- ${some.key.providers.p2}
gb:
- ${some.key.providers.p1}
When I load the file via profile activation, for example, rename the file to application-core-properties.yml and then -Dspring.profiles.active=core-propertiesit does work however if when I try to load the file via #PropertySource("core-properties.yml") it does not and I get the following error:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-03-27 10:07:36.397 -ERROR 13474|| --- [ restartedMain] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under 'some.key.providers.by.market' to java.util.Map<java.lang.String, java.util.List<java.lang.String>>:
Reason: No converter found capable of converting from type [java.lang.String] to type [java.util.Map<java.lang.String, java.util.List<java.lang.String>>]
Action:
Update your application's configuration
Process finished with exit code 1
Bacouse you don't have equivalent properties stracture,
example
spring:
profiles: test
name: test-YAML
environment: test
servers:
- www.abc.test.com
- www.xyz.test.com
---
spring:
profiles: prod
name: prod-YAML
environment: production
servers:
- www.abc.com
- www.xyz.com
And config class should be
#Configuration
#EnableConfigurationProperties
#ConfigurationProperties
public class YAMLConfig {
private String name;
private String environment;
private List<String> servers = new ArrayList<>();
// standard getters and setters
I have resolved the issue implementing the following PropertySourceFactory detailed described in here
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Properties;
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.core.env.PropertySource;
import org.springframework.core.io.support.EncodedResource;
import org.springframework.core.io.support.PropertySourceFactory;
import org.springframework.lang.Nullable;
public class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(#Nullable String name, EncodedResource resource) throws IOException {
Properties propertiesFromYaml = loadYamlIntoProperties(resource);
String sourceName = name != null ? name : resource.getResource().getFilename();
return new PropertiesPropertySource(sourceName, propertiesFromYaml);
}
private Properties loadYamlIntoProperties(EncodedResource resource) throws FileNotFoundException {
try {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
factory.afterPropertiesSet();
return factory.getObject();
} catch (IllegalStateException e) {
// for ignoreResourceNotFound
Throwable cause = e.getCause();
if (cause instanceof FileNotFoundException)
throw (FileNotFoundException) e.getCause();
throw e;
}
}
}
I had a similar problem and found a workaround like this:
diacritic:
isEnabled: true
chars: -> I wanted this to be parsed to map but it didn't work
ą: a
ł: l
ę: e
And my solution so far:
diacritic:
isEnabled: true
chars[ą]: a -> these ones could be parsed to Map<String, String>
chars[ł]: l
chars[ę]: e

Remove file from remote using streaming inbound channel adapter spring boot implementation

I am trying to remove file from remote by implementing streaming inbound but connection is closing before adviceChain implementing.
CODE:
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
factory.setPassword(sftpPwd);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
#Bean
#InboundChannelAdapter(channel = "stream", poller = #Poller(cron = "2 * * * * ?"))
public MessageSource<InputStream> sftpMessageSource() {
SftpStreamingMessageSource messageSource = new SftpStreamingMessageSource(template());
messageSource.setRemoteDirectory(remoteDirecotry);
messageSource.setFilter(new AcceptAllFileListFilter<>());
return messageSource;
}
#Bean
public SftpRemoteFileTemplate template() {
return new SftpRemoteFileTemplate(sftpSessionFactory());
}
#Bean
#Transformer(inputChannel = "stream", outputChannel = "data")
public org.springframework.integration.transformer.Transformer transformer() {
return new StreamTransformer("UTF-8");
}
#ServiceActivator(inputChannel = "data" ,adviceChain = "afterChain")
#Bean
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
String fileName = message.getHeaders().get("file_remoteFile").toString();
if (!StringUtils.isEmpty(message.toString())) {
else{
log.info("No file found in the Remote location");
}
}
};
}
#Bean
public ExpressionEvaluatingRequestHandlerAdvice afterChain() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnSuccessExpression(
"#template.remove(headers['file_remoteDirectory'] + headers['file_remoteFile'])");
//advice.setOnSuccessExpressionString("#template.remove(headers['file_remoteFile'])");
advice.setPropagateEvaluationFailures(true);
return advice;
}
wherever i search every one is suggesting to implement ExpressionEvaluatingRequestHandlerAdvice but it is throwing me below error.
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=starsBatchJob]] completed with the following parameters: [{JobID=1522168322277}] and the following status: [COMPLETED]
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Job Status Completed
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Total time tokk for Stars Batch execution: 0 seconds.
2018-03-27 12:32:02.618 INFO 23216 --- [ask-scheduler-1] c.f.u.config.ParentBatchConfiguration : Batch Job lock is released
2018-03-27 12:32:02.633 INFO 23216 --- [ask-scheduler-1] com.jcraft.jsch : Disconnecting from hpchd1e.hpc.ford.com port 22
2018-03-27 12:32:02.633 ERROR 23216 --- [ask-scheduler-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to remove file: 2: No such file; nested exception is 2
I had this problem. My path to the remote file was incorrect. I needed a trailing /. It is a little difficult to see since the path is being created inside a Spel Expression. You can see the path using the following in the handleMessage() method.
String remoteDirectory = (String) message.getHeaders().get("file_remoteDirectory");
String remoteFile = (String) message.getHeaders().get("file_remoteFile");
I did have to use the advice.setOnSuccessExpressionString("#template.remove(headers['file_remoteFile'])"); that is commented out above instead of advice.setOnSuccessExpression"#template.remove(headers['file_remoteDirectory'] + headers['file_remoteFile'])");
It is incorrect in the documentation https://docs.spring.io/spring-integration/reference/html/sftp.html#sftp-streaming which is why I believe people who struggle with this lose faith in the doc. But this seems to be the only error.

Resources