So I am using the FTP protocols to download an MP3 from our news center to our computer in the radio station. Its all setup with scheduled tasks and is automated. The news mp3 is coming in "garbled" sounding, almost like its playing backwards or something, I am having no problems using the same code to download from our weather center.
Also, it should be noted that I can download the News with an ftp client and the mp3 plays perfectly, its only when using the ftp protocol that it becomes an issue
The code is simple
open site.site.com
user username password
mget News-2.mp3
bye
I have messed with it and messed with it and tried multiple clients and other ways of downloading and its only using the FTP protocols that it screws up the file.
I suspect you need to set binary mode before doing the transfers so that music that happens to contain the code for linefeed and carriage returns is not converted.
Try using:
open site...
user username password
bi
mget ...
Also, try running DIR New*mp3 in the FTP session and then running ls -l *mp3 locally after GETTING the file, the sizes should be identical, but if they are different by a few bytes it is probably the line endings being dynamically corrected during the transfer.
Related
Using pureftpd I uploaded 1% of a 1276541542 byte file or about 15 megs. Then I killed the network connection abnormally to simulate a client getting kicked off their ISP. Then I waited an hour. Then I re-connected and issued an APPE (append) command and uploaded the rest of the file. The final size of the file on the server after the upload finished was 1292326238. i.e. about 15 megs MORE than it should be. Corrupt file. What is correct way for an FTP server to prevent corrupted uploaded files because of late append?
What is correct way for an FTP server to prevent corrupted uploaded files because of late append?
There is no way for the FTP server to prevent corrupted uploaded files because the server does not know what the file should be.
But the server can help the client to do a proper upload by implementing the SIZE command. Using this command the client can determine the current file size at the server and thus the position in the file where the upload should be continued. Of course this logic has to be implemented at the client.
i have pure-ftpd answers about it’s upload-script
i’m running pure-uploadscript --run /home/aa/done.rb —daemonize
and my done.rb program is
#!/usr/bin/env ruby
puts "done"
f=File.open("/home/aa/ddd.txt", "w")
f << "test"
f.close
and when I run pure-ftpd —uploadscript and upload a file, sure enough the done.rb program is run.
(I know it’s run cuz there is a new file called ddd.txt)
BUT when I’m uploading a big file and kill the ftp client in middle of upload done.rb is STILL run. (Yes I deleted ddd.txt first.)
Therefore, the answer to the question is, EVEN pureftpd can't handle this because of the limits of FTP protocol.
I am trying to setup a "dropbox" on a Win7 workstation we will use to process simulation jobs. My plan was to pull ownership from file (do a simple dir /q "filename") so I can use the owner information during the simulation (send them an email when done for example).
The problem I have is when the user drops the simulation file on the share I setup, the ownership is set to BUILTIN\Administrators. I have tried tweak the share settings but so far nothing seems to work.
I do have a work around where users can embed their email address in the simulation file and I could pull that. But trying to make it easier as I know somer user will forget to do that... Any ideas how to preserver the ownership inforamation?
Quite possibly, you could embed the owner's email as an alternate data stream into the file. Read this link here.
And with a few powershell scripts, you could write the job owner into the file at submit time and extract it out on the remote machine at run time.
I believe the alternate data stream survives the command line copy command as long as NTFS is used everywhere as the filesystem.
This question already has answers here:
How to confirm SFTP file delivery?
(3 answers)
Closed 1 year ago.
I'm uploading the file through Sftp to destination server using bash scripts.
How I can be sure that the file which is uploaded is complete upload in the case sftp will not return anything or network connection could be broken?
I see that I can get the size of the file before uploading to the server and then I can compare it with the existing size for the file on the server.
Perhaps you can mention about other better options?
Thank you.
I think getting the size is a good option.
What I could imagine :
Client side :
- Put the size of the file, and its md5 in a file, like ".fileinfo"
- Send the fileinfo to the server
- Send the (interesting) File to the server
Server side :
- Check periodically files of a folder (with "watch ls" command for example)
- If a ".fileinfo" exists, read it, and check if the size corresponds to an existing file of the same name (without ".filefome"). If the size corresponds, do an "md5sum" of the file, and check if it corresponds. If yes, move your file into your final destination folder, and delete the ".fileinfo" file. If not reiterate.
Many sites for downloading softwares will provide both the software and its checksum.
we can use the same technique to check our uploading file.
upload the file together with its checksum, on the server side compare the file's checksum with uploaded checksum,
if the two don't match, you will know
The file uploaded is corrupted, or
The checksum uploaded is corrupted, or
Both the checksum and file uploaded are corrupted.
Test exit code of sftp. If it returns 0 you can be pretty sure that everything is ok (assuming you are using OpenSSH sftp). This works only when you use -b switch (what I assume you are doing).
SFTP protocol allows checksum calculation, but I suppose you are stuck with OpenSSH (or either or both sides) that does not support this.
To be 100% sure, you can download the file back and compare with original.
I guess this is kind of a programming question, because I'm going to write a program if this doesn't exist.
So I found a very cheap web-host (I don't really care about the actual web hosting). They will give me a domain name and ftp server with a ton of storage space. Anyway, I want to backup a few hundred gigs of data (mostly family photos and scans of important documents). I also want to backup any future family photos / documents. I don't care if everything on my local NAS dies in a fire, I just want to have the photos and important documents backed up off-site.
So I want some program that lets me select folders locally and schedules them to be backed up to the ftp server. I'm a bit of a security nut, so i'd like the files to be encrypted locally before being transferred up onto the server.
I know I can do this with truecrypt volumes, but I don't want to transfer an entire encrypted volume blob up to the server ever time I change a file in it. So I could do multiple true crypt volumes but that will be a pain to manage
Also this must be mac/linux compatible although I'll primarily be on linux.
I basically need rsync + truecrypt + cron + sftp all rolled into a cryptographically secure program.
I've been searching for days with no luck. Any ideas?
mozyBackup does this - it doesn't use FTP, it has a custom uploader.
ps. Remember a typical home ADSL connection only does about 1Gb/day upstream
Linux option.
Out of the box option probably duplicity ( for example see http://www.howtoforge.com/creating-encrypted-ftp-backups-with-duplicity-and-ftplicity-on-debian-lenny )
Otherwise if these are basically rarely changed archive copies of files - I would roll my own gnupg (or dpad) individual file encryption, a file changed script, and ftp or rsync.
I need to set up some sort of infrastructure to automatically FTP some files from one remote server to another. The FTP transaction will occur on a scheduled basis. Both these servers are Windows boxes and the location of the files that need to be FTP'ed will depend on the current date (the folder they sit in will be named the current day's date).
I would really hate to have to write something like that from scratch, so are there tools/utilities/the likes out there?
Windows command line ftp will read input from a text file. For example:
ftp -ni < ftpscript.txt
Where these options are used:
-n - Suppresses auto-login upon initial connection.
-i - Turns off interactive prompting during multiple file transfers.
The ftpscript.txt would look something like this:
open ftp.mycompany.com
user someuser password
binary
get /somedir/myfile.dat
It should be reasonably straightforward to write a script that outputs an FTP script containing the correct file name.
You would think that this would be an easy task. It isn't.
The best tool I've found is SyncBack and I think there is a free/lite version available.
If you're in a Windows environment consider using PowerShell and the System.Net.WebClient .NET Framework class like this post. This is a download example, but the WebClient object also provides an UploadFile method.