First this isn't an assignment or for employment. I want to be a better/more secure programmer.
I have read RFC 959, I am at a loss as to how you actually use this to make a program that is compliant. Do I just send the commands as strings and make the server interpret them as the command?
For instance if I sent PWD from the client to the server would I just make the server parse this and send back the current dir name?
For instance if I sent PWD from the client to the server would I just make the server parse this and send back the current dir name?
Yes. That's correct.
The FTP protocol is text-based.
An exchange between the client and the server looks literally like this:
Server: 220 ProFTPD 1.3.5rc3 Server (Debian)
Client: USER user
Server: 331 Password required for user
Client: PASS password
Server: 230 User user logged in
Client: SYST
Server: 215 UNIX Type: L8
Client: PWD
Server: 257 "/home/user" is the current directory
Related
I'm currently using github.com/jlaffaye/ftp on Go to send files.
I'm trying to connect to an FTP and upload a zipped file of about 700MB.
I connect to server properly and change the working dir but when I'm about to call the Stor function, it responds with "connection refused" and drops.
This is the code:
ftpFile="hola.txt"
fileOpen, err := os.Open(ftpFile) // For read access.
if err != nil {
fmt.Println("error "+ftpFile)
panic(err)
} else {
fmt.Println("abierto "+ftpFile)
}
reader:=bufio.NewReader(fileOpen)
err = client.Stor(ftpDir+ftpFile,reader)
if err != nil {
fmt.Println("error en stor "+ftpDir+ftpFile)
panic(err)
} else {
fmt.Println("subiendo "+ftpFile)
}
defer fileOpen.Close()
I'm opening a file with os.Open then read it with bufio.NewReader and pass it to the Stor function but it disconnects. The FTP is good as I was connected to it via FileZilla, the zipfile or txt file (in this code example) are both good and I believe I'm missing the shot when it comes to the bufio.NewReader but I can't find a working example of reading a file and using this goftp library.
Update:
Here's the logfile
220 Microsoft FTP Service
USER *user goes here*
331 Password required
PASS *plain text password goes here*
230 User logged in.
FEAT
211-Extended features supported:
LANG EN*
UTF8
AUTH TLS;TLS-C;SSL;TLS-P;
PBSZ
PROT C;P;
CCC
HOST
SIZE
MDTM
REST STREAM
211 END
TYPE I
200 Type set to I.
OPTS UTF8 ON
200 OPTS UTF8 command successful - UTF8 encoding now ON.
CWD /web-import/pre/
250 CWD command successful.
PWD
257 "/web-import/pre" is current directory.
PASV
227 Entering Passive Mode (x,x,x,x,x,x).
Update: Looks like someone had the same problem some weeks ago: goftp - 229 Entering Extended Passive Mode now I'll be looking for solutions and will post here.
Update: Changed the script to connect to a CentOS server and got the same error. Now that the FTP server is discarded then there are 2 culprits: goftp package and the script itself.
Update: Tried again with a simpler script following example at documentation and failed again at the same spot. Had to report issue on developer's Github https://github.com/jlaffaye/ftp/issues/272
Update: I connected via terminal and opened FTP from command line. I typed "ls" by mistake and got this error
227 Entering Passive Mode (x,x,x,x,x,x).
ftp: connect: Connection refused
Last Update: After coding and debugging in the end it wasn't the goftp package neither the server neither the client. It was my firewall that blocked my ftp from going PASV. I whitelisted the ip and it worked perfectly.
i've tried to create a script that will upload some file to a ftp server using lftp, but without any luck so far. If I used build in ftp command in debian I manage to succsessfully connected and put the file.
Here is the debug output from lftp command:
lftp xxx.xxx.xxx.xxxx -e "put -O /out/ some_file_name" -d
---- using user `user01' and password from ~/.netrc
---- Resolving host address...
---- 1 address found:xxx.xxx.xxx.xxxx
---- Connecting to xxx.xxx.xxx.xxxx (xxx.xxx.xxx.xxxx) port 21
<--- 220 (vsFTPd 2.0.7)
---> FEAT
<--- 211-Features:
<--- EPRT
<--- EPSV
<--- MDTM
<--- PASV
<--- REST STREAM
<--- SIZE
<--- TVFS
<--- UTF8
<--- 211 End
---> OPTS UTF8 ON
<--- 200 Always in UTF8 mode.
---> USER user01
<--- 331 Please specify the password.
---> PASS XXXX
<--- 230 Login successful.
---> PWD
<--- 257 "/"
---> TYPE I
<--- 200 Switching to Binary mode.
---> EPSV
<--- 550 Permission denied.
---- Switching passive mode off
---- Closing data socket
---- Closing control socket
As you can see I'm using stored user name and password from .netrc file. I have another script that connect to the same server but uploads files and rename them inside the remote ftp folder using lftp again. Can someone help and explain why I cannot put with lfpt but can do it using ftp.
after add set ftp:passive-mode true and set ftp:prefer-epsv false to /etc/lftp.conf the error is changed
<--- 230 Login successful.
---> PWD
<--- 257 "/"
---> TYPE I
<--- 200 Switching to Binary mode.
---> PASV
<--- 227 Entering Passive Mode (xxx.xxx.xxx.xxx,76,92)
---- Connecting data socket to (xxx.xxx.xxx.xxx) port 19548
---- Data connection established
---> ALLO 710
<--- 550 Permission denied.
---> STOR out/my_file_name
---> ABOR
put: Access failed: 550 Permission denied. (/out/my_file_name)
---- Closing aborted data socket
---- Closing control socket
Okay I've understand what ALLO means
The ALLO command may be sent to a server that requires the necessary space for an uploaded to be reserved before the transfer takes place
so after a quick search in ftp man page, I've found a command to shut it down. After adding set ftp:use-allo false and with epsv false everything is fine now. Thanks alot :)
lftp -e "set ftp:use-allo false; set ftp:passive-mode true; set ftp:prefer-epsv false; mirror -R {local dir} {remote dir}" -u {username},{password} {host}
Use this single command to sync your file from local to server without 550 permission error.
While 550 Permission denied. is a strange response to the EPSV command it means that the server or some middlebox in between does not understand the EPSV command (likely a middlebox since the response to FEAT shows EPSV as supported). If you use the builtin ftp command instead of lftp it will probably use the older PASV command (IPv4 only) instead of the newer EPSV command (IPv4+IPv6 capable).
According to the man page there is a setting ftp:prefer-epsv which should default to false. Maybe some configuration is setting this value to true so that lftp will use EPSV instead of PASV. Check your settings (set -a inside lftp) and if it is true (expected) set it to false and try again, in the hope that it will then use PASV instead of EPSV.
After successful FTP file transfer, the the response is used to be "226 File send OK", but suddenly, it has changed to be "226 Transfer complete"
I have below questions:
Does FTP response code has any standard?
Can we customize FTP output response for a specific status code?
Find the standard FTP response for file transfer
$ ftp canopus
Connected to canopus.austin.century.com.
220 canopus.austin.century.com FTP server (Version 4.1 Sat Nov 23 12:52:09 CST 1991) ready.
Name (canopus:eric): dee
331 Password required for dee.
Password:
230 User dee logged in.
ftp> pwd
257 "/home/dee" is current directory.
ftp> cd desktop
250 CWD command successful.
ftp> type ascii
200 Type set to A.
ftp> send typescript
200 PORT command successful.
150 Opening data connection for typescript (128.114.4.99,1412).
226 File send OK.
ftp> cdup
250 CWD command successful.
ftp> bye
221 Goodbye.
Note: suddenly the response text 226 File send OK has changed to 226 Transfer complete
Find the details about FTP responses on wikipedia
RFC 959, 4.2. FTP REPLIES:
An FTP reply consists of a three digit number (transmitted as
three alphanumeric characters) followed by some text. The number
is intended for use by automata to determine what state to enter
next; the text is intended for the human user. It is intended
that the three digits contain enough encoded information that the
user-process (the User-PI) will not need to examine the text and
may either discard it or pass it on to the user, as appropriate.
In particular, the text may be server-dependent, so there are
likely to be varying texts for each reply code.
I am using LFTP to transfer files from a server, which unfortunately does not recognize the PORT command. I do not have control over the server (do not know in detail what server is) and I have to use the active mode.
This is the command line as:
lftp -e 'debug 10;set ftp:passive-mode off; set ftp:auto-passive-mode no; ls; bye;' -u user,password ftp://ftp.site.com
This is the debug output:
<--- 200 Using default language en_US
---> OPTS UTF8 ON
<--- 200 UTF8 set to on
---> OPTS MLST modify;perm;size;type;UNIX.group;UNIX.mode;UNIX.owner;
<--- 200 OPTS MLST modify;perm;size;type;UNIX.group;UNIX.mode;UNIX.owner;
---> USER xxxxx
<--- 331 Password required for xxxxx
---> PASS xxxxxx
<--- 230 User xxxxx logged in
---> PBSZ 0
<--- 200 PBSZ 0 successful
---> PROT P
<--- 200 Protection set to Private
---> PORT 172,16,133,11,146,168
<--- 500 Illegal PORT command
---> LIST
---> ABOR
---- Closing aborted data socket
---- Chiusura del socket di controllo
It seems that LFTP renounces to connect to data socket because the remote server does not support the PORT command. Is there a way to convince LFTP can still connect to port 20? By FTP manual obviously no problem.
The issue, I think, is not that the FTP server doesn't support the PORT command (it does), but rather, it doesn't like the IP address/port that your FTP client is sending in the PORT command.
PORT 172,16,133,11,146,168
...tells the server to connect to address 172.16.133.11, port 37544*. The interesting part here is the IP address: it's an RFC 1918 address (i.e. it's a private network address). That, in turn, suggests that your FTP client is in a LAN somewhere, and is connecting to an FTP server using a public IP address.
That remote FTP server cannot connect to a private network address; by definition, RFC 1918 address are not publicly routable.
Thus it very well could be that the FTP server is trying to make a connection to the address/port given in your PORT command, fails, thus that is why the FTP server fails the command, saying:
500 Illegal PORT command
To make a PORT command work with that FTP server, you would need to discover the public IP address that that server can connect to, to reach your client machine. Let's say that this address is 1.2.3.4. Then you would need to tell lftp to use that address in its PORT command, using the ftp:port-ipv4 option.
Chances are, though, that public IP address is the address of a NAT/router/firewall, and that that NAT/router/firewall will not allow connections, from the outside world to a high numbered port (e.g. 37544), to be routed to a machine within the LAN. This is one of the issues with active FTP data transfers, i.e. FTP data transfers which use the PORT (or EPRT) commands: they are not considered "firewall-friendly".
Hope this helps!
* - why 146,168 translates to port 37544?
According to FTP's RFC959 those parameters are:
(...) 16-bit TCP port address. This address information is broken into
8-bit fields and the value of each field is transmitted as a decimal
number (in character string representation).
146 dec = 10010010 bin = A
168 dec = 10101000 bin = B
A B
10010010 10101000 bin = 37544 dec
For testing purposes I want send mail to my localhost user account rather than my webserver. I am unsure how to do this using mail.app. Any help would be appreciated.
#Tautologistics
OSX does have a built-in MTA (SMTP server), to turn it on you can type:
sudo launchctl start org.postfix.master
then you can send mail to localhost like you desire
sample showing an SMTP server running from my machine running 10.6.1
>telnet 127.0.0.1 25
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
>sudo launchctl start org.postfix.master
>telnet 127.0.0.1 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 machinename.example.com ESMTP Postfix
If you don't specifically want to use Mail.app, you can send mail using the mail command. Open Terminal and:
mail -s "Testing" `whoami`#`hostname`
<type something>
Ctrl-D to finish and send
Those are backticks, not single quotes. whoami returns the current user's username and hostname returns the local machine's hostname. It could also be explicit:
mail -s "Testing" john#mymac.local
EDIT: Just read your clarification. Mail.app stores it's data in ~/Mail, mostly in an SQLite database (the 'Envenlope Index' file). The tables of interest would be mailboxes and messages. The text of the email is stored in individual files in the respective mailbox/folder directories. This would probably be the way to go, if you want to access email that has been fetched by Mail.app (in realtime).
Yet another option would be to export your mail from the Mail.app using the mbox format and access it using the technique described by dbr. Depending on whether or not realtime access is desired, you might be able to script something up that automates the export.
I'm looking to login into my (local) mail server, access a mailbox, and do some parsing. So, I assume there's a mail server running locally but not sure how to access it
The local mail isn't stored in a POP3/IMAP server, but rather using a UNIX'y mbox. A file stored in /var/mail/ (the file-name is the users login)
For example..
$ mail dbr
Subject: hi
test
^d # ctrl+d (EOF)
$ cat /var/mail/dbr
From dbr#parabola.local Tue Dec 30 13:43:57 2008
Return-Path: <dbr#parabola.local>
X-Original-To: dbr
Delivered-To: dbr#parabola.local
Received: by parabola.local (Postfix, from userid 501)
id 4FEA1158E36; Tue, 30 Dec 2008 13:43:57 +1030 (CST)
To: dbr#parabola.local
Subject: hi
Message-Id: <20081230031357.4FEA1158E36#parabola.local>
Date: Tue, 30 Dec 2008 13:43:57 +1030 (CST)
From: dbr#parabola.local (dbr)
test
Not sure about Ruby (I had a search around, but couldn't find anything, although there is undoubtably a module for this), but I know Python has a maildir.mbox module, which would use in the following way:
>>> msgs = mailbox.mbox("/var/mail/dbr")
>>> for msg in msgs:
... print "Subject:", msg['subject']
...
Subject: hi
Unless you are running OSX Server, then there's no SMTP/IMAP/POP3 server running locally. You can get one up and running very easily using Post Fix Enabler or, if you don't mind the command line, use MacPorts to install postfix:
sudo port install postfix
Send mail from localhost LocalhostMail is a simple and fast solution for Mac OS X that lets you send email messages from your PHP-application (or any other, located on localhost) by Mail.app included with Mac OS X. If you use PHP, just add to MySQL database new messages, and our application will send them through a Mail application. LocalhostMail uses your mail account in Mail.app to send these messages, so you do not need a separate SMTP-server for your localhost.localhostmail.com
codelogic,
thanks, I did know about sending mail from the terminal. I think my question was not well thought out. I'm looking to login into my (local) mail server, access a mailbox, and do some parsing. So, I assume there's a mail server running locally but not sure how to access it.
I'm using ruby:
pop = Net::POP3.new 'macbook.local'
pop.start 'me', 'mypass'
but get a Timeout::Error: execution expired