Sftp Outbound gateway. Fail to list file using regex - ftp

Question

I want to lists files recursively as describe through a regex. This is the code that works
#Bean(value = "listFiles")
public IntegrationFlow listFiles() {
IntegrationFlow flow = IntegrationFlows
.fromSupplier(() -> "/", e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(5))))
.handle(Ftp.outboundGateway(ftp(), Command.LS, null)
.regexFileNameFilter("(test|.*txt)")
.options(AbstractRemoteFileOutboundGateway.Option.RECURSIVE,AbstractRemoteFileOutboundGateway.Option.NAME_ONLY))
.handle(p -> display(p))
.get();
return flow;
}
This ftp server is simulated using a local ftp server on my computer and it works okay.
However, when I switched the ftp() to an actual remote one and its a sftp, it does not work.
#Bean(value = "listFiles")
public IntegrationFlow listFiles() {
IntegrationFlow flow = IntegrationFlows
.fromSupplier(() -> "/", e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(5))))
.handle(Sftp.outboundGateway(sftp(), Command.LS, null)
.regexFileNameFilter("(test|.*txt)")
.options(AbstractRemoteFileOutboundGateway.Option.RECURSIVE,AbstractRemoteFileOutboundGateway.Option.NAME_ONLY))
.handle(p -> display(p))
.get();
return flow;
}
private SessionFactory<LsEntry> sftp() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
factory.setHost(REMOTE_ADR);
factory.setPort(REMOTE_PORT);
factory.setAllowUnknownKeys(true);
factory.setUser(REMOTE_USER);
factory.setPassword(REMOTE_PASS);
return factory;
}
I get this error
2022-04-07 13:45:20.260 ERROR 11136 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessageHandlingException: error occurred in message handler [bean 'listFiles.sftp:outbound-gateway#0' for component 'listFiles.org.springframework.integration.config.ConsumerEndpointFactoryBean#0'; defined in: 'class path resource [my/com/company/integration/background/IntegrationConfiguration.class]'; from source: 'bean method listFiles']; nested exception is org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to list files; nested exception is 4: , failedMessage=GenericMessage [payload=/, headers={id=132641aa-0f7e-9dcb-5738-146f4b0baf64, timestamp=1649310313647}]
at org.springframework.integration.support.utils.IntegrationUtils.wrapInHandlingExceptionIfNecessary(IntegrationUtils.java:191)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:65)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:317)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:272)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.handleMessage(SourcePollingChannelAdapter.java:196)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.messageReceived(AbstractPollingEndpoint.java:475)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:461)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.pollForMessage(AbstractPollingEndpoint.java:413)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$4(AbstractPollingEndpoint.java:348)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.lambda$execute$0(ErrorHandlingTaskExecutor.java:57)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$5(AbstractPollingEndpoint.java:341)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to list files; nested exception is 4: at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:461) at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.doLs(AbstractRemoteFileOutboundGateway.java:607) at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.handleRequestMessage(AbstractRemoteFileOutboundGateway.java:584) at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:136) at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:56) ... 27 more Caused by: org.springframework.core.NestedIOException: Failed to list files; nested exception is 4: at org.springframework.integration.sftp.session.SftpSession.list(SftpSession.java:123) at org.springframework.integration.sftp.session.SftpSession.list(SftpSession.java:54) at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.listFilesInRemoteDir(AbstractRemoteFileOutboundGateway.java:992) at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.ls(AbstractRemoteFileOutboundGateway.java:957) at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.lambda$doLs$6(AbstractRemoteFileOutboundGateway.java:608) at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:452) ... 31 more Caused by: 4: at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1747) at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1553) at org.springframework.integration.sftp.session.SftpSession.list(SftpSession.java:111) ... 36 more Caused by: java.lang.NullPointerException at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2943) at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1572) ... 38 more
At this point, I'm not even sure if it's a programming problem or an ftp or sftp problem.

Answer 1

You are getting an NPE because you are not providing an expression for the directory; you were just lucky it works with FTP, the SFTP client is not so forgiving.
.handle(Sftp.outboundGateway(sftp(), Command.LS, null)
Use
.handle(Sftp.outboundGateway(sftp(), Command.LS, "payload")