Mule faile to download file from SFTP location - ftp

I am trying to connect to an SFTP location and download a .zip file using Mule SFTP connector. It looks very straightforward configuration, but I am not sure what is missing in my configuration. I am not able to make out why it is not working for me. Can someone please look at it and suggest what should I change to make work?
In below flows configurations, I am starting with an HTTP end point (http://localhost:8181/invoice) then “ftpconnectivityFlow1” is called and it checks the value of "ftp" variable and based on its value it either goes to my FTP location or SFTP location. when I set the variable "ftp" to true it works as expected, as I can see file from FTP location is downloaded in my output folder and deleted from FTP location as expected. When I set it to false it is not giving any error but file at SFTP location is still there meaning it is not able to read file (I am guessing) and it is not downloaded to my output folder. So for some debugging I added a custom transformer so that I can inspect payload. In my custom transformer I notice that when it connects to FTP location it has some binary data (all number), I am guessing it is my .zip file, but when variable "ftp" is set to false, meaning it is trying to connect to SFTP location in that case payload contains "/invoice" which is my http relative path. So my output folder contains a file with name “null” and all it contains is "/invoice"
Any help is greatly appreciated.
<flow name="ftpconnectivityFlow1">
<logger message="ftp:#[message.outboundProperties['ftp']]" doc:name="Logger" level="INFO"/>
<choice doc:name="Choice">
<when expression="#[message.outboundProperties['ftp']==true]">
<flow-ref name="FTPConnection" doc:name="FTPFileDownloadConnection"/>
</when>
<otherwise>
<flow-ref name="SFTPConnection" doc:name="SFTPFileDownloadConnection"/>
</otherwise>
</choice>
</flow>
<flow name="FTPConnection">
<ftp:inbound-endpoint host="host" port="22" path="abc" user="user" password="password" responseTimeout="10000" doc:name="FTP"/>
<custom-transformer class="abc.transformer.CustomeFileTransformer" />
<logger message="connected to FTP" level="INFO" doc:name="Logger"/>
<file:outbound-endpoint path="output" outputPattern="#[message.inboundProperties['originalFilename']]" responseTimeout="10000" doc:name="File"/>
</flow>
<flow name="SFTPConnection">
<sftp:inbound-endpoint connector-ref="sftp-default" doc:name="SFTP" responseTimeout="10000" host="host" password="password" path="/Inbound" port="21" user="user"/>
<custom-transformer class="abc.transformer.CustomeFileTransformer" />
<logger level="INFO" doc:name="Logger"/>
<file:outbound-endpoint path="output" outputPattern="#[message.inboundProperties['originalFilename']]" responseTimeout="10000" doc:name="File"/>
</flow>

<ftp:inbound-endpoint host="host" port="22" ... doc:name="FTP"/>
...
<sftp:inbound-endpoint ... port="21" user="user"/>
You might have those port numbers backwards. FTP normally runs on port 21 and SFTP (SSH) normally uses port 22.

Related

Mule ESB: Streaming a large files/Multipart/form-data

I a batch job that picks the file (larger than 200 MB) from FTP folder and we are doing a multipart/form data upload to an external service. The issue we are running into is out of memory and mule logs suggest enable streaming for FTP connector. If I enable streaming on the FTP connector then I am getting a ‘no object DCH for mime type application/xml’ error. How do I set the content type for FTP file stream?
Below is my config:
<ftp:inbound-endpoint host="${ftp.host}" port="${ftp.port}" path="${ftp.path}" user="${ftp.user}" password="${ftp.password}" connector-ref="FTP" responseTimeout="10000" doc:name="FTP"/>
<set-attachment attachmentName="#[message.inboundProperties.originalFilename]" value="#[message.payload]" contentType="application/xml" doc:name="Set File"/>
<set-payload value="#[null]" doc:name="Set Payload"/>
<http:request config-ref="HTTP_Request_Configuration" path="${api.importPath}//${api.files}" method="POST" doc:name="HTTP-Invoke Upload ">
<http:request-builder>
<http:header headerName="${access.token}" value="${access.token.value}"/>
<http:header headerName="${client.secret}" value="${client.secret.value}"/>
<http:header headerName="${authorization.mode}" value="${authorization.mode.value}"/>
</http:request-builder>
</http:request>
<ftp:connector name="FTP" pollingFrequency="1000" validateConnections="true" doc:name="FTP" streaming="true"/>
Thanks,
ROA
You have error message that application/xml is not supported.
Try another mime type like text/plain here
<set-attachment attachmentName="#[message.inboundProperties.originalFilename]" value="#[message.payload]" contentType="text/plain" doc:name="Set File"/>
Try to use the mime type as below:
<set-attachment attachmentName="#[message.inboundProperties.originalFilename]" value="#[message.payload]" contentType="**/**" doc:name="Set File"/>

how to achieve retry mechanism for ftp outbound end point using vm transaction?

We have tried like using vm as outbound in flow1 and inbound in flow2. In flow2 we are using FTP as outbound end point and we have enabled the vm transaction even then also its not working. Do we need to enable transaction for retrying? As per below question we tried using transaction how to make until successful as synchronous to retry mechanism for FTP Outbound in mule 3.4.2 could you please help me out in resolving this issue??
<flow name="FTPFlow1" doc:name="FTPFlow1">
<set-payload doc:name="Set Payload" value="#[payload]"/>
<vm:outbound-endpoint exchange-pattern="one-way" doc:name="VM" path="doProcess">
<vm:transaction action="ALWAYS_BEGIN"/>
</vm:outbound-endpoint>
</flow>
<flow name="FTPFlow2" doc:name="FTPFlow2">
<vm:inbound-endpoint exchange-pattern="one-way" path="doProcessMessage" doc:name="VM">
<vm:transaction action="JOIN_IF_POSSIBLE"/>
</vm:inbound-endpoint>
<ftp:outbound-endpoint host="localhost" port="21" path="/data/mule/ftp" user="admin" password="admin" responseTimeout="10000" doc:name="FTP"/>
</flow>
You've got the transactions wrong: the VM outbound doesn't need to be transacted, it's the VM inbound that needs to in order to trigger redeliveries in case of FTP failures.
<flow name="FTPFlow1" doc:name="FTPFlow1">
<set-payload doc:name="Set Payload" value="#[payload]"/>
<vm:outbound-endpoint exchange-pattern="one-way" doc:name="VM" path="doProcess" />
</flow>
<flow name="FTPFlow2" doc:name="FTPFlow2">
<vm:inbound-endpoint exchange-pattern="one-way" path="doProcessMessage" doc:name="VM">
<vm:transaction action="ALWAYS_BEGIN"/>
</vm:inbound-endpoint>
<ftp:outbound-endpoint host="localhost" port="21" path="/data/mule/ftp" user="admin" password="admin" responseTimeout="10000" doc:name="FTP"/>
</flow>

Can we use JMS in the middle of proxy flow in mule

Actually we have produced JAX-WS web services, and it is working fine. But now we want to use Mule ESB's JMS. But i'm unable to configure that.
I have tried Mule's Proxy for webservices, and it is working fine. But we are trying to put JMS in between HTTP Endpoints. But the body of the soap cannot be transferred to the other end (i.e. to our services)
JMS Server is ActiveMQ.
Thanks in advance,
Copied the flow from comment --
<flow name="finalFlow1" doc:name="finalFlow1">
<http:inbound-endpoint exchange-pattern="request-response"
host="localhost" port="8888" contentType="text/xml" doc:name="HTTP" />
<jms:outbound-endpoint exchange-pattern="request-response"
queue="servicesQueue" doc:name="JMS" connector-ref="Active_MQ" />
<http:outbound-endpoint exchange-pattern="request-response"
method="POST" address="localhost:5050/MyServices" ; mimeType="text/xml"
contentType="text/xml" doc:name="HTTP" />
</flow>
In the flow that you posted you are consuming the message via HTTP based inbound endpoint. If you just want to consume this message from JMS and send it to another HTTP endpoint, you need to use JMS inbound
<flow name="finalFlow1" doc:name="finalFlow1">
<jms:inbound-endpoint exchange-pattern="request-response"
queue="servicesQueue" doc:name="JMS" />
<logger level="INFO" doc:name="Logger" />
<http:outbound-endpoint exchange-pattern="request-response"
host="localhost" port="5050" method="POST" doc:name="HTTP" path="MyServices"
mimeType="text/xml" />
</flow>
However, this will just send the payload as is and not going to convert it into SOAP payload. If you want to convert the message consumed from JMS to SOAP payload, you need to use CXF
<flow name="finalFlow1" doc:name="finalFlow1">
<jms:inbound-endpoint exchange-pattern="request-response"
queue="servicesQueue" doc:name="JMS" />
<logger level="INFO" doc:name="Logger" message="#[payload]" />
<logger message="SOAP call started" level="INFO" doc:name="Logger"/>
<http:outbound-endpoint
mimeType="text/xml" doc:name="HTTP" exchange-pattern="request-response" method="POST" path="MyServices" host="localhost" port="5050">
<cxf:proxy-client payload="body"
enableMuleSoapHeaders="false">
<cxf:inInterceptors>
<spring:bean class="org.apache.cxf.interceptor.LoggingInInterceptor" />
</cxf:inInterceptors>
<cxf:outInterceptors>
<spring:bean class="org.apache.cxf.interceptor.LoggingOutInterceptor" />
</cxf:outInterceptors>
</cxf:proxy-client>
</http:outbound-endpoint>
<logger message="SOAP call completed" level="INFO" doc:name="Logger"/>
</flow>
If you just want to kickoff the JMS consumption via HTTP you can go with what Seba suggested using MuleRequester - https://github.com/mulesoft/mule-module-requester/blob/master/mulerequesterdemo/src/main/app/MuleRequesterDemo.xml
If you need to consume messages from a queue in the middle of a flow, you should take a look at this: http://blogs.mulesoft.org/introducing-the-mule-requester-module

Mule FTP transfer problems

I've got a problem with transfering files with Mule trough FTP.
The mule console says Failed to retrieve file 40525600_100492.xml. Ftp error: 550
but all files but one is transfered but when they end up in the other FTP they
are renamed to a 0aa00a0-00aaa0-00akahd-sakka-aaa.dat-file.
What is wrong? Do I need to have a delay?
<flow name="XR-importFlow2" doc:name="XR-importFlow2">
<ftp:inbound-endpoint host="XX.XX.XX.77" port="21" path="XRRDF/wrkcd" user="user" password="password" responseTimeout="10000" doc:name="FTP" disableTransportTransformer="true" passive="true"/>
<ftp:outbound-endpoint host="xx.xx.xx.xx" port="21" path="x-path" user="anonymous" responseTimeout="10000" doc:name="FTP"/>
</flow>
To overwrite the default name mule is using for copying files, specify the outputPattern attribute in ftp:outbound-endpoint. To copy files with original name, you can use outputPattern="#[header:originalFilename]", like this:
<ftp:outbound-endpoint host="xx.xx.xx.xx" port="21" path="x-path" user="anonymous" responseTimeout="10000" doc:name="FTP" outputPattern="#[header:originalFilename]"/>
Try specifying a pollingFrequency attribute on ftp:inbound-endpoint with a value of may be 60000 (1min), to see if it helps you

Mule FTP: how to move up one directory?

I need to configure a FTP inbound endpoint in Mule, what I got so far is this:
<ftp:connector name="ftpConnector" pollingFrequency="1000"
validateConnections="true"
moveToDirectory="C:\Users\jonbrynjar.FRETT\Documents\national_registry"
moveToPattern="*.txt"/>
<ftp:inbound-endpoint host="ftp1.xxxx.is" port="21"
user="xxxx" password="xxxx" binary="false"
pollingFrequency="5000" responseTimeout="10000"
connector-ref="ftpConnector">
<file:filename-wildcard-filter pattern="../einst.txt" />
</ftp:inbound-endpoint>
I can access this server in command prompt this way:
user:xxxx
pass:xxx
cd ..
get K0274K.N4503.EIN.E32 einst.txt
get K0274K.N301.F300 fyrirt.txt
bye
I think the problem is I am not able to move up one directory as implied in the command text!
How would I implement this action Mule?
I would suggest using the mule's composite source to use multiple source(The path of the folder being different in each ) .
<flow name="MuleRunnerFlow1" doc:name="MuleRunnerFlow1">
<composite-source doc:name="Composite Source">
<ftp:inbound-endpoint host="ftp1.xxxx.is" port="21" user="xxxx" password="xxxx" binary="false" pollingFrequency="5000" responseTimeout="10000" connector-ref="ftpConnector" doc:name="FTP" path="/parent">
<file:filename-wildcard-filter pattern="einst.txt" />
</ftp:inbound-endpoint>
<ftp:inbound-endpoint host="ftp1.xxxx.is" port="21" user="xxxx" password="xxxx" binary="false" pollingFrequency="5000" responseTimeout="10000" connector-ref="ftpConnector2" doc:name="FTP" path="/parent/children">
<file:filename-wildcard-filter pattern="einst.txt" />
</ftp:inbound-endpoint>
</composite-source>
<logger level="INFO" doc:name="Logger" />
</flow>
You May use two connectors or same connector based on your requirement and you can take the path and other properties from the property file if necessary
Hope this helps

Resources