Why is CF FTP transfer speed multiple times slower than standard FTP? - ftp

Question

I have a ColdFusion application that I use to transfer files between our development and production servers. The code that actually sends the files is as follows:
ftp = new Ftp();
ftp.setUsername(username);
ftp.setPassword(password);
ftp.setServer(server);
ftp.setTimeout(1800);
ftp.setConnection('dev');
ftp.open();
ftp.putFile(transferMode="binary",localFile=localpath,remoteFile=remotepath);
ftp.close(connection='dev');
When sending a file using the above code, I cap out at just under 100KB/s(monitored via FileZilla on the receiving server). If I send the exact same file using the Windows command-line FTP tool, my speeds are upwards of 1000KB/s.
I created a brand new file with nothing but the code above and that has no effect on the transfer speed, so I know it has nothing to do with the surrounding code in the original application.
So, what could be causing these abysmally low speeds?
Edit: All tests are being done transferring files from my production server to my development server. I also tried using the <cfftp> tag instead of cfscript, and I have the same results.
Edit #2: I ended up using cfexecute, the code is as follows:
From my FTP script:
public function sendFiles(required string localpath, required string remotepath) {
this.writeFtpInstructions(localpath);
exe = "C:\Windows\system32\ftp.exe";
params = "-s:" & request.localapproot & "/" & "upload.txt";
outputfile = request.localapproot & '/ftp.log';
timeout = 120;
Request.cliExec(exe,params,outputfile,timeout);
}
public function writeFtpInstructions(required string localpath) {
instructions = request.localapproot & "/" & "upload.txt";
crlf = chr(13) & chr(10);
data = "";
data &= "open " & this.server & crlf;
data &= this.username & crlf;
data &= this.password & crlf;
data &= "cd " & request.remoteapproot & crlf;
data &= "put " & localpath & crlf;
data &= "quit";
FileWrite(instructions, data);
}
The cliExec() function(necessary to create a wrapper since there is no equivalent of cfexecute in cfscript):
<cffunction name="cliExec">
<cfargument name="name">
<cfargument name="arguments">
<cfargument name="outputfile">
<cfargument name="timeout">
<cfexecute
name="#name#"
arguments="#arguments#"
outputFile="#outputfile#"
timeout="#timeout#" />
</cffunction>

Answer 1

With my experience using cfftp on CF9, it was impossible to transfer larger files. If you view the active transfers on the FTP server side, you will notice that the transfers start out at top speed, but as more & more data is transmitted the speeds keep dropping. After 100 MB had been transfered, the reduction started to become very drastic, until they eventually reached a single digit crawl. Eventually the transfer timed out & failed. I was trying to work with a 330 MB file & found it impossible to transfer using cfftp. The cfexecute was not an option for me using the standard windows ftp.exe, because it doesn’t seem to support SFTP.
I ended up seeking out an external java class to use through coldfusion & settled on JSch (http://www.jcraft.com/jsch/). Ironically, CF9 appears to use a variation of this class for CFFTP (jsch-0.1.41m.jar), but the results are much different using this latest downloaded version (jsch-0.1.45.jar).
Here is the code that I put together for a proof of concept:
<cfscript>
stAppPrefs = {
stISOFtp = {
server = 'sftp.server.com',
port = '22',
username = 'youser',
password = 'pa$$w0rd'
}
};
/* Side-Load JSch Java Class (http://www.jcraft.com/jsch/) */
try {
// Load Class Using Mark Mandel's JavaLoader (http://www.compoundtheory.com/?action=javaloader.index)
/*
Add Mark's LoaderClass To The ColdFusion Class Path Under CF Admin:
Java and JVM : ColdFusion Class Path : C:\inetpub\wwwroot\javaloader\lib\classloader-20100119110136.jar
Then Restart The Coldfusion Application Service
*/
loader = CreateObject("component", "javaloader.JavaLoader").init([expandPath("jsch-0.1.45.jar")]);
// Initiate Instance
jsch = loader.create("com.jcraft.jsch.JSch").init();
}
catch(any excpt) {
WriteOutput("Error loading ""jsch-0.1.45.jar"" java class: " & excpt.Message & "<br>");
abort;
}
/* SFTP Session & Channel */
try {
// Create SFTP Session
session = jsch.getSession(stAppPrefs.stISOFtp.username, stAppPrefs.stISOFtp.server, stAppPrefs.stISOFtp.port);
// Turn Off & Use Username/Password
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword(stAppPrefs.stISOFtp.password);
session.connect();
// Create Channel To Transfer File(s) On
channel = session.openChannel("sftp");
channel.connect();
}
catch(any excpt) {
WriteOutput("Error connecting to FTP server: " & excpt.Message & "<br>");
abort;
}
// Returns Array Of Java Objects, One For Each Zip Compressed File Listed In Root DIR
// WriteDump(channel.ls('*.zip'));
// Get First Zip File Listed For Transfer From SFTP To CF Server
serverFile = channel.ls('*.zip')[1].getFilename();
/* Debug */
startTime = Now();
WriteOutput("Transfer Started: " & TimeFormat(startTime, 'hh:mm:ss') & "<br>");
/* // Debug */
/* Transfer File From SFTP Server To CF Server */
try {
// Create File On Server To Write Byte Stream To
transferFile = CreateObject("java", "java.io.File").init(expandPath(serverFile));
channel.get(serverFile, CreateObject("java", "java.io.FileOutputStream").init(transferFile));
// Close The File Output Stream
transferFile = '';
}
catch(any excpt) {
WriteOutput("Error transfering file """ & expandPath(serverFile) & """: " & excpt.Message & "<br>");
abort;
}
/* Debug */
finishTime = Now();
WriteOutput("Transfer Finished: " & TimeFormat(finishTime, 'hh:mm:ss') & "<br>");
expiredTime = (finishTime - startTime);
WriteOutput("Duration: " & TimeFormat(expiredTime, 'HH:MM:SS') & "<br>");
WriteOutput("File Size: " & NumberFormat(Evaluate(GetFileInfo(ExpandPath(serverFile)).size / 1024), '_,___._') & " KB<br>");
WriteOutput("Transfer Rate: " & NumberFormat(Evaluate(Evaluate(GetFileInfo(ExpandPath(serverFile)).size / 1024) / Evaluate(((TimeFormat(expiredTime, 'H') * 60 * 60) + (TimeFormat(expiredTime, 'M') * 60) + TimeFormat(expiredTime, 'S')))), '_,___._') & " KB/Sec <br>");
/* // Debug */
channel.disconnect();
session.disconnect();
</cfscript>
Results:
Transfer Started: 09:37:57
Transfer Finished: 09:42:01
Duration: 00:04:04
File Size: 331,770.8 KB
Transfer Rate: 1,359.7 KB/Sec
The transfer speed that was achieved is on par with what I was getting using the FileZilla FTP Client & manually downloading. I found this method to be a viable solution for the inadequacy of cfftp.

Answer 2

I have been looking and I dont have an answer about why it is slower. But, in theory, you should be able to use cfexecute to do this through the windows command line. You might could even create a batch file and do it in one call.