I need to copy all the files of an FTP folder to my local Windows folder, but without replacing the files that already exist. This would need to be a job/task that runs unattended every hour.
This is what the job would need to do:
1. Connect to FTP server.
2. In ftp, move to folder /var/MyFolder.
3. In local PC, move to c:\MyDestination.
4. Copy all files in /var/MyFolder that do not exist in c:\MyDestination.
5. Disconnect.
I had previously tried the following script using MGET * (that runs from a .bat), but it copies and overwrites everything. Which means that even if 1000 files were previously copied, it will copy them again.
open MyFtpServer.com
UserName
Password
lcd c:\MyDestination
cd /var/MyFolder
binary
mget *
Any help is appreciated.
Thanks.
Use wget for Windows.
If you want to include subdirectories (adjust the cut-dirs number according to the depth of your actual remote path):
cd /d C:\MyDestination
wget.exe --mirror -np -nH --cut-dirs=2 ftp://UserName:Password#MyFtpServer.com/var/MyFolder
If you don't want subdirectories:
cd /d C:\MyDestination
wget.exe -nc ftp://UserName:Password#MyFtpServer.com/var/MyFolder/*
The "magic" bit (for this second form) is the -nc option, which tells wget not to overwrite files that are already there locally. Do keep in mind that old files are also left alone, so if a file on your FTP server gets edited or updated, it won't get re-downloaded. If you want to also update files, use -N instead of -nc.
(Note that you can also type wget instead of wget.exe, I just included the extension to point out that these are Windows batch file commands)
Related
I have to get and delete multiple files via FTP so I wrote this script:
open ftp.myftpserver.org
user
pass
cd folder
lcd E:\localdir
mget *
mdel *
bye
This works but is not safe since the folder is being fed from other sources and the mdel * step may delete files uploaded in the meanwhile.
I guess a solution could be moving the files remotely to a different folder, building a filelist at the beginning of the process, but i have no idea how to make it.
Is it possibile?
FTR I followed the nice hint and I managed to succesfully made something working, maybe not elegant but works:
First step to get the file list:
getfilelist.bat
open ftp.myserver.it
myuser
pass1234
cd ftpfolder
prompt n
lcd E:\localdir
ls *.??? filelist.txt
bye
Second step to download and delete the above files
movefiles.bat
#echo off
setlocal enableextensions
setlocal enabledelayedexpansion
echo open ftp.myserver.it>>myscript
echo user myuser pass1234>>myscript
echo cd ftpfolder>>myscript
echo prompt n>>myscript
echo ascii>>myscript
echo lcd E:\downloaddir>>myscript
for /F "usebackq tokens=1,2* delims=," %%G IN ("E:\localdir\filelist.txt") DO ECHO rename %%G %%G_TMP>>myscript
echo mget *_TMP>>myscript
echo mdelete *_TMP>>myscript
echo bye>>myscript
ftp -n -s:myscript
del filelist.txt
del myscript
e:
cd E:\downloaddir
ren *.???_TMP *.???
A bat file to recall the above steps:
E:
cd E:\localdir
ftp -i -s:E:\localdir\getfilelist.bat
E:\localdir\movefiles.bat
hope it helps
I had a similar problem. It looks a lot of people struggle here ;-)
I have to download multiple files and remove them after successfully downloading them on the remote Server to avoid double processing. There is a very small chance that while doing an mget the remote System adds further files. So a mdelete might delete untransferred files.
My approach was not to use anything else than FTP commands. Here we go:
cd <dir on remote server>
mkdir Transfer
mv *.* Transfer
mget Transfer\*.*
del Transfer\*.*
rmdir Transfer
At a glance I move all files to an extra directory on the remote server. Should be very fast as this happens locally. Once the files are shifted to the transfer dir, I perform the real mget download. This might take longer and meanwhile further files can be safely uploaded on the remote Server main dir.
After the download is done, I delete all files in the transfer dir and remove the dir itself.
Don't know if you can do that with a script from with ftp client.
Might be better to do it as a program or scipt using a language of your choice with an FTP library so you have much more control of the FTP operations. e.g. perl with Net::ftp, java etc.
You could then implement a algorithum like:
remote cd to required folder
localcd to required folder
list current files in remote folder
for each file in list that matchs required pattern
get file
if get ok then
delete file
else
log error,exit or whatever error handling you want
endif
endfor
Also need to make sure you don't try to get a file that is in the process of being written, depending on o/s this might be handled for you with file locks, but you need to make sure files are written to the remote dir in a two stage process. First too a temporay directory or file name that does not match the file pattern you are checking for, then renamed or moved into the correct location and name that you will detect.
ok, then i think you can implement the algorithum i've described as a batch script using two seperate ftp calls and scripts. The first to list the files to be transfered from the remote dir, the second to get and delete a single file from the list.
The file get script would have to be created dynamically on each loop iteration to get the next file.
cd to working dir
Run FTP with script to:
cd to remote folder
list files matching required pattern and store in local file
(e.g: ls files_*.txt filelist.txt)
for each file in file list created above (e.g. use 'for /f ...' command to loop through filelist.txt)
create temp ftp script to:
cd to remote dir
lcd to local dir
get file
del file
run FTP with temp script
endfor
This site has an example of doing similer (note script as shown dosn't work, comments give details of corrections needed).
http://www.computing.net/answers/programming/batch-ftp-script-list-rename-get-delete/25728.html
basically i have a 2 files on my company FTP, i need the users to be able to click a script to swap the 2 files when they want.
FTP tree look like this:
/file1.vxml
/swap/file1.vxml
And here i need to connect to my ftp (on Windows), and rename /file1.vxml to /file2.vxml before moving then move it to /swap/ (to not overwrite /swap/file1.vxml). Then doing the opposite on /swap/file2.vxml and move it to the root.
i already have the connection :
open host.myhost.com
user myusername
mypassword
cd /
bye
but the thing i'm lacking is how to move and rename properly files on the ftp.
Use "REN" to rename files in FTP.
ftp> REN FileA FileB
Renaming and moving are the same thing essentially in FTP, you can move a file to a different folder by renaming it.
ftp> REN Here ../There
You'll probably have to download the first file back to your local server to save a copy, then rename the second file, and copy the file you downloaded back up to the server afterwards. I don't know of any FTP servers that support copying files within themselves.
Hi I want to LOOP through files on FTP and copy one by one. Every thing is fine with FTP connection and accessing folders.
My question is How can loop through all files on FTP. It looks like there is no "For" type of functionality available to access FTP files because each line is considered as complete command.
open MyServerName 21
MyUserName
MyPassword
lcd E:\LocalDirectory
cd /FTPDirectory/upload
I WANT TO LOOP THROGH ALL FILES AND COPY ONE BY ONE TO LOCAL DIRECTORY
disconnect
bye
Why i want to loop through all files in FTP is, I want to copy only those files which are not locked and available for copy.
Use a different FTP client: wget.
With the -m option (for --mirror), use the following in a script
cd mylocaldirectory
wget -m ftp://username:password#hostname/theremotedirectory
I used SSH to connect to a server and navigate to the folder where I want to store some files from my Mac. I think what I need to do is use SCP to do the copy but I'm not sure exactly about the terminology in the command parameters. And so far everything I've tried gets some sort of "not found" error.
Before logging on to the server the prompt is :
Apples-MacBook-Pro-2:~ neiltayl$
After logging in and navigating to the folder I want to store things in it is :
[neiltayl#cs136 Tracer]$
I need to copy several files from the Tracer folder on my local computer to the Tracer folder on cs136 and cannot fathom the correct parts of the respective FROM and TO parts of SCP to make it work.
This is the nearest I got so far;
Apples-MacBook-Pro-2:~ neiltayl$ ls
Applications Downloads Music Tracer
Desktop Library Pictures c151
Documents Movies Public dwhelper
Apples-MacBook-Pro-2:~ neiltayl$ scp ./Tracer/*.* neiltayl#cs136.cs.iusb.edu:Tracer
neiltayl#cs136.cs.iusb.edu's password:
./Tracer/*.*: No such file or directory
The scp command is -
$ scp File1 username#someting:DIRNAME
Here File 1 is the file that you are sending over to the other computer.
DIRNAME is the path to the directory where you want the file to be stored.
In your case the command would be
scp -r Tracer neiltayl#cs136:New_Tracer
Here Tracer is the folder that contains all the files that you want to copy.
I'm trying to find a way to see if a file exists on an ftp site via DOS. I tried a get command on the file hoping that if it didn't exist it wouldn't download it to my local directory. However it seams that it still does, but it's an empty file. This doesn't work for me however because the file I'm looking for is just a empty trigger file so I can't tell the difference.
I would like to dump a listing ls of the ftp directory to a text file on my local drive and so I try
ls > listing.txt.
It creates the listing.txt file locally but it's always empty even though there are files on the ftp site.
What are my options with this?
I have used dir > listing.txt and ls > listing.txt and every time listing.txt is empty even though there are files in the directories I'm running those commands on.
Sorry if I didn't make this clear, but I'm trying to get the listing for an automated process and not simply for my visual when manually doing this.
Unless you're on FreeDOS, you're probably not using DOS. Perhaps you're using ftp.exe in the windows console? If that's the case, don't use a normal file redirect. Instead check here the syntax for ls in the standard Windows ftp client:
ls [RemoteDirectory] [LocalFile]
So you can do a ls . listing.txt to get a list of files in the current remote directory. The listing.txt file will appear in your user directory, e.g. c:\Users\user.