I'm trying to write a batch script (CMD # Windows XP Pro) that will automatically download and unzip packages with the help of 7zip and putty/psftp
If I have a URL to a package to download http://somesite.org/packages/package.zip how do I download it on command line using putty?
Also if you have a better way to do this that would be helpful too.
wget is of course an obvious solution, but I also suggest to have a look at cURL. From their website:
curl is a command line tool for
transferring files with URL syntax,
supporting FTP, FTPS, HTTP, HTTPS,
SCP, SFTP, TFTP, TELNET, DICT, LDAP,
LDAPS and FILE. curl supports SSL
certificates, HTTP POST, HTTP PUT, FTP
uploading, HTTP form based upload,
proxies, cookies, user+password
authentication (Basic, Digest, NTLM,
Negotiate, kerberos...), file transfer
resume, proxy tunneling and a busload
of other useful tricks.
Of course free and open source, and despite its huge list of supported protocols it's as simple to use as wget, so to use your example
curl -O http://somesite.org/packages/package.zip
downloads package.zip to a local file with the same name
curl -o myname.zip http://somesite.org/packages/package.zip
downloads package.zip as myname.zip
curl http://somesite.org/packages/package.zip > package.zip
redirects curl's stdout to package.zip
EDIT - example corrected, with thanks to #PrabhakarKasi
win32 version of wget:
http://pages.interlog.com/~tcharron/wgetwin.html
Putty isn't really a download tool. Unless you want to download something via SCP/SFTP. So yes, wget is more helpful here.
I don't know putty, but certainly wget can do. If you are in Windows, you can get it by cygwin or just google a win32 version.
pscp.exe -pw yourpassword you#somesite.org:/packages/package.zip .\
The path to /packages/package.zip should be whatever the path to the public web files are on the server. So, for example, on some old apache server, it might be:
pscp.exe -pw yourpassword you#somesite.org:/users/httpd/vhosts/default/packages/package.zip .\
Use pscp, which comes with PuTTY:
pscp user#host:/path/to/file.7z .
7z e file.7z
If you set this up with SSH keys, pscp won't have to ask you for a password.
Related
I need to connect to a host with username, password, implicit TLS encryption and port number to download files to a folder daily on windows server standard. Is there a third party command-line application that I could download, install and use for this (preferably free)? I'm not absolutely sure if this could be done with Windows ftp and if it can, could it be done in batch file?
I am trying NcFTP but I'm not sure if it supports encryption either.
I was given specific credentials, I have no control over the server. I have only instructions on how to access and download the files with FileZilla client over TLS. I need to schedule a routine that does this job for me since I don't want to manually do this every day. I can manage myself on this I only need a tool that could do this job over command-line.
You can use WinSCP, it supports both scripting and TLS/SSL.
See automating file transfers to FTP server.
A simple batch file to download files over an explicit TLS/SSL (note the ftpes://) with WinSCP looks like:
winscp.com /log=c:\path\ftp.log /command ^
"open ftpes://user:password#example.com/" ^
"get /home/user/* c:\destination\" ^
"exit"
You can have the batch file generated by WinSCP GUI for you.
For scheduling, simply use the Windows Scheduler.
For details see scheduling file transfers to FTP server.
(I'm the author of WinSCP)
Similarly for an upload: Schedule an automatic FTP upload on Windows with WinSCP
You can use sftp to get files. But you need to have an SSH server installed on remote host (OpenSSH is free).
Then you can use putty tools (free) psftp.exe or pscp.exe to upload/download files in encripted format.
Link for putty tools download: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
Link for OpenSSH download: http://www.openssh.com/
I am using FileZilla (GUI) to download files from an FTP server. Can any one tell me a command line argument to download file from FTP server to local file system?
FileZilla does not have any command line arguments (nor any other way) that allow automatic transfer.
See:
FileZilla Client command-line arguments
https://trac.filezilla-project.org/ticket/2317
Though you can use any other FTP client that allows automation.
Built-it Windows ftp.exe (use its get command).
Though Windows ftp.exe does not support a passive mode, what makes it useless nowadays, when connecting over Internet due to ubiquitous firewalls and NATs.
WinSCP: You can automatically make it download a file both in GUI and scripting mode
GUI:
winscp.exe ftp://user:password#host/path/file.ext
You will get a dialog, where you select a directory to download the file to.
See https://winscp.net/eng/docs/commandline
Scripting (put the command to a .bat file):
winscp.com /command ^
"open ftp://user:password#host/" ^
"get /path/file.ext c:\" ^
"exit"
See https://winscp.net/eng/docs/guide_automation
The WinSCP can even generate a script from an imported FileZilla session.
For details, see a guide to FileZilla automation.
(I'm the author of WinSCP)
FileZilla does support some basic command line options for the upload and download of files. Not all options are documented.
You can download a file with the option "-d sourcefile targetdir"
Example (download):
filezilla.exe -d ftp://user:pass#ftp.myserver.com/folder/* "c:\downloads\" /overwrite /close
Example (upload):
filezilla.exe -u "c:\temp\samplefile.exe" ftp://user:pass#ftp.myserver.com/folder /overwrite /close
/close -- will close the GUI after the download/upload completes
/overwrite -- will overwrite the files in target without prompting the user
You can also connect to a site from the site manager e.g. --site="0/site1".
Below are reference sites with further examples including download links to FileZilla versions (2.32 / 3.1) that supported the feature.
FAQ Link
FileZilla Download link
I need to connect to a host with username, password, implicit TLS encryption and port number to download files to a folder daily on windows server standard. Is there a third party command-line application that I could download, install and use for this (preferably free)? I'm not absolutely sure if this could be done with Windows ftp and if it can, could it be done in batch file?
I am trying NcFTP but I'm not sure if it supports encryption either.
I was given specific credentials, I have no control over the server. I have only instructions on how to access and download the files with FileZilla client over TLS. I need to schedule a routine that does this job for me since I don't want to manually do this every day. I can manage myself on this I only need a tool that could do this job over command-line.
You can use WinSCP, it supports both scripting and TLS/SSL.
See automating file transfers to FTP server.
A simple batch file to download files over an explicit TLS/SSL (note the ftpes://) with WinSCP looks like:
winscp.com /log=c:\path\ftp.log /command ^
"open ftpes://user:password#example.com/" ^
"get /home/user/* c:\destination\" ^
"exit"
You can have the batch file generated by WinSCP GUI for you.
For scheduling, simply use the Windows Scheduler.
For details see scheduling file transfers to FTP server.
(I'm the author of WinSCP)
Similarly for an upload: Schedule an automatic FTP upload on Windows with WinSCP
You can use sftp to get files. But you need to have an SSH server installed on remote host (OpenSSH is free).
Then you can use putty tools (free) psftp.exe or pscp.exe to upload/download files in encripted format.
Link for putty tools download: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
Link for OpenSSH download: http://www.openssh.com/
In my terminal:
url='http://58.30.207.171/youku/69764FC8BC2447992487A2488/030002010051001478A6FA0109ACBF22B0F614-2746-1AE5-C9EF-2266A1CC83DB.flv'
curl $url -o test1.flv #can download
wget -c $url -O test2.flv #can not download
1.Why wget can not download it?
2.How can i make wget download it?
The site may be blocking wget specifically. You can override this by setting the user-agent to an empty string:
I found this at http://www.gnu.org/software/wget/manual/wget.html#Option-Syntax
‘-U agent-string’
‘--user-agent=agent-string’
Identify as agent-string to the http server.
The http protocol allows the clients to identify themselves using a User-Agent header field. This enables distinguishing the www software, usually for statistical purposes or for tracing of protocol violations. Wget normally identifies as ‘Wget/version’, version being the current version number of Wget.
However, some sites have been known to impose the policy of tailoring the output according to the User-Agent-supplied information. While this is not such a bad idea in theory, it has been abused by servers denying information to clients other than (historically) Netscape or, more frequently, Microsoft Internet Explorer. This option allows you to change the User-Agent line issued by Wget. Use of this option is discouraged, unless you really know what you are doing.
Specifying empty user agent with ‘--user-agent=""’ instructs Wget not to send the User-Agent header in http requests.
I have connected to a server via SFTP using FileZilla and accepted adding the server's SSH key to the key cache in FileZilla.
How can I extract this cached key to a keyfile so that may use it through other SFTP applications that require a keyfile be made available?
I have not been able to find anything in the FileZilla documentation related to this.
If you use the standard openssh console client (cygwin or from linux), host keys are stored, one-per-line, in ~/.ssh/known_hosts. From there, it's a simple matter of figuring out which bit of that host key is needed for your library.
Putty also stores host keys, but it appears to encode them in hex. Those can be found at HKCUR\Software\SimonTatham\PuTTY\SshHostKeys
Thomas was correct. FileZilla piggybacks on PuTTY's PSFTP program and stores the saved keys encoded in a hex format at the registry key he listed (HKCUR\Software\SimonTatham\PuTTY\SshHostKeys). I needed the key in known_hosts format, so I has able to install a windows version of openssh at his recommendation and used the ssh-keyscan tool to hit the server and save the key info out in the correct format:
ssh-keyscan -t rsa <my_ftp_ip_address> > c:\known_hosts
ssh-keyscan -t dsa <my_ftp_ip_address> > c:\known_hosts
Thank you Thomas and SO!
If you'd rather use a GUI, you can snag the host key from the log window or the first-time connection popup using WinSCP FTP client: https://winscp.net/eng/docs/ssh_verifying_the_host_key
Thanks Dougman for the tip!
To further help any newcomers reading your answer.
Prior to running the ssh-keyscan, assuming the openssh is install by default, there is a few commands that needs to be run (read the quickstart/readme install for details).
Here are my commands which allow me to obtain the host key.
C:\Program Files\OpenSSH\bin>mkgroup -l >> ..\etc\group
C:\Program Files\OpenSSH\bin>mkpasswd -l >> ..\etc\passwd
C:\Program Files\OpenSSH\bin>net start opensshd
The OpenSSH Server service is starting.
The OpenSSH Server service was started successfully.
C:\Program Files\OpenSSH\bin>ssh-keyscan -t rsa vivo.sg.m.com > c:\known_hosts
vivo.sg.m.com SSH-2.0-Sun_SSH_1.1
Unless I am misunderstanding you: you don't need to.
If you connect to the server with another application (ie: PuTTY) and it has not seen the server before then you will be prompted to accept the key.
I see why you might want to do this, but each application could have it's own way to store keys.