Make FTPS connection in Ruby with double-bag-ftps gem - ruby

I am trying to get a native FTP connection work to an odd FTP server in ruby. It requires TLS and implicit SSL. I have a FileZilla client configured and working. Here's my code:
require 'double_bag_ftps'
DoubleBagFTPS.open(ftp_host, ftp_user, passwd, nil, DoubleBagFTPS::IMPLICIT, :verify_mode => OpenSSL::SSL::VERIFY_NONE) do |ftp|
...
files = ftp.list(file_path)
STDOUT.write files
end
I get the following runtime error when I run the above:
bunches of traceback lines
<path_to_gems>/double-bag-ftps-0.1.4/lib/double_bag_ftps.rb:160:in `initialize': wrong argument type nil (expected OpenSSL/SSL/CTX) (TypeError)
I can't seem to get anything out of the server with Ruby and the traditional net/ftp gem (various errors related to TLS/SSL problems). DoubleBagFTPS seems to be the most promising gem, but I still get an error. It may be the case that I am not calling the open function correctly. The only nil is the fourth parameter, but that's clearly spelled out in the DooubleBagFTPS example.
Can someone help?
Update
Per the suggestion, here's my new code
class MyFTP < Net::FTP
FTP_PORT = 990
def connect(host, port = FTP_PORT)
synchronize do
#host = host
#bare_sock = open_socket(host, port)
begin
ssl_sock = start_tls_session(Socket.tcp(host, port))
#sock = BufferedSSLSocket.new(ssl_sock, read_timeout: #read_timeout)
voidresp
if #private_data_connection
voidcmd("PBSZ 0")
voidcmd("PROT P")
end
rescue OpenSSL::SSL::SSLError, Net::OpenTimeout
#sock.close
raise
end
end
end
end
def ftp_options
{
username: 'user',
password: 'password',
ssl: true,
passive: true
}
end
MyFTP.open(ftp_host, ftp_options) do |ftp|
ftp.login
files = ftp.chdir(file_path)
files = ftp.list
STDOUT.write files
end
I'm still getting an error as follows:
---stack-trace---
<path_to_gem>/ruby/2.5.0/net/protocol.rb:52:in `connect': SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol (OpenSSL::SSL::SSLError)

So I got it working with regular old Net::FTP as follows:
def ftp_options
{
username: '<username>',
password: '<password>',
ssl: {
verify_mode: OpenSSL::SSL::VERIFY_NONE
}
}
end
Net::FTP.open(ftp_host, ftp_options) do |ftp|
ftp.login(ftp_options[:username], ftp_options[:password])
files = ftp.list
STDOUT.write files
puts "\n"
end
The one thing I don't understand is why I am forced to pass the username and password to the ftp.login method, since it's already defined in ftp_options, which was passed to Net::FTP.open(). As far as I can tell everything is set up correctly in ftp_options. For the particular server I'm connecting to, TLS/SSL is required, and that's working, so that parameter variable is being picked up... why not user/password?
Anyway, case closed for me at least. I can confirm that regular Net::FTP seems to work with at least one of these non-vanilla FTP servers requiring TLS and implicit SSL.

Related

Connection won't raise error with bad host

The application I'm working on allows users to add additional connections to other databases through a UI. I'm simply trying to create a validation that ensures that a connection can be made.
I created a separate class to test the DB connections:
class StoredProcConnection < ActiveRecord::Base
def self.abstract_class?
true # So it gets its own connection
end
end
I then create the connection:
def connect
adapter = sql_server? ? 'mssql' : 'mysql2'
default_port = sql_server? ? '1443' : '3306'
#connection_pool = StoredProcConnection.establish_connection(
adapter: adapter,
username: username,
password: password,
host: host,
database: database_name,
port: port || default_port,
timeout: 300)
end
def connection_pool
connect unless #connection_pool
#connection_pool
end
Then I validate it with this method:
def connection_test
if connection_pool.connection
#remove the connection from the StoredProcConnection pool
connection_pool.remove(connection_pool.connection)
return true
else
return false
end
rescue Exception => error
logger.info "unable to create connection with connection.id = #{id} - #{error}"
return false
end
Unfortunately, when it gets to this line with a bad host address like 127.0.0.abcdefg or 666.666.666.666
if connection_pool.connect
The app gets stuck, no errors raised or anything. It just freezes and I have to shut down the server manually.
I have a workaround but it feels quite sloppy. I am just inserting my own timeout in there, but I feel like Active Record should be throwing some kind of error.
def connection_test
Timeout::timeout(3) {
if connection_pool.connection
#remove the connection from the StoredProcConnection pool
connection_pool.remove(connection_pool.connection)
return true
else
return false
end
}
rescue Exception => error
logger.info "unable to create connection with connection.id = #{id} - #{error}"
return false
end
Does anyone see anything that might be causing the freeze? It seems pretty straight forward to me. I'm not sure why the connection pool is even created in the first place with a bad host passed in.
It just freezes...
Odds are good it's not frozen, it's just waiting to see if that connection can be made. TCP/IP has a long timeout value, which fools people into thinking things are frozen, when actually it's being patient.
Few people really understand how the internet works, and how it's really a house built of straw that we try to keep running no matter what. Long IP timeouts are one of the ways that we try to make it self-healing. Software doesn't care about how long something takes, only people care.
Since it appears you're concerned about malformed IP addresses, why aren't you pre-testing them to make sure they're at least in a valid format?
Use Ruby's built-in IPAddr class and try to parse them:
require 'ipaddr'
%w[
127.0.0.abcdefg
666.666.666.666
127.0.0.1
192.168.0.1
255.255.255.255
].each do |ip|
begin
IPAddr.new ip
puts "Good: #{ ip }"
rescue IPAddr::InvalidAddressError => e
puts "#{ e }: #{ ip }"
end
end
# >> invalid address: 127.0.0.abcdefg
# >> invalid address: 666.666.666.666
# >> Good: 127.0.0.1
# >> Good: 192.168.0.1
# >> Good: 255.255.255.255
For "fun reading" there's "TCP Timeout and Retransmission" and "TCP Socket no connection timeout".

Multi-Threading in Ruby

I have a TCPserver that I made in ruby, the server seems to work, I can see that two or more clients can connect and be served by the server, but, they sometime get stuck (as in need to wait for the other client to disconnect or just get unresponsive), usually after the "pass_ok" bit, When connecting only with one client I don't see this issue.
Here is my code:
def self.main_server
begin
server = TCPServer.open(#port)
rescue Exception => e
CoreLogging.syslog_error("Cant start server: #{e}")
end
#main_pid = Process.pid
# Main Loop
Thread.abort_on_exception = true
while true
Thread.fork(server.accept) do |client|
#client = client
sock_domain, remote_port, remote_hostname, remote_ip = #client.peeraddr # Get some info on the incoming connection
CoreLogging.syslog_error("Got new connection from #{#client.peeraddr[3]} Handeled by Thread: #{Thread.current}") # Log incoming connection
#client.puts "Please enter password: " # Password testing (later will be from a config file or DB)
action = #client.gets(4096).chomp # get client password response 'chomp' is super important
if action == #password
# what to do when password is right
pass_ok
Thread.exit
else
# what to do when password is wrong
pass_fail
Thread.exit
end
end
begin
CoreLogging.syslog_error("Thread Ended (SOFT)")
rescue Exception => e
CoreLogging.syslog_error("Thread was killed (HARD)")
end
end
end
I'll leave it here for future reference and hope someone in a close situation will find it useful.
The issue was the global #client variable, which got overwritten every new thread and then inherited to the subclasses inside the thread.
using a local client variable (without the '#') got it to work as supposed.

Connect to a FTPS server with mismatched server certificate using Net::FTPTLS

I am trying to connect via Net::FTPTLS to a Microsoft-based file server (IIS) which is configured to use FTP on port 22 and requires SSL.
I connect via:
require 'net/ftptls'
ftp = Net::FTPTLS.new()
ftp.connect('host.com', port_number)
ftp.login('Username', 'Password')
ftp.puttextfile('somefile.txt', 'where/to/save/somefile.txt')
ftp.close
Problem is, I get the following error:
hostname does not match the server certificate
It seems that I have to disable the openssl peer verification: OpenSSL::SSL::VERIFY_PEER should become OpenSSL::SSL::VERIFY_NONE.
Any ideas on how to monkey-patch the Net::FTPTLS class? Has anyone done this successfully?
Instead using Net::FTPTLS, use Ruby 2.4+ with the following code:
require 'net/ftp'
ftp = Net::FTP.new(nil, ssl: {:verify_mode => OpenSSL::SSL::VERIFY_NONE})
ftp.connect('host.com', port_number)
ftp.login('Username', 'Password')
ftp.puttextfile('somefile.txt', 'where/to/save/somefile.txt')
ftp.close
What I did, rather than monkeypatching ruby itself, was bring a copy of this into /lib of my project.
module Net
class FTPTLS < FTP
def connect(host, port=FTP_PORT)
#hostname = host
super
end
def login(user = "anonymous", params = {:password => nil, :acct => nil, :ignore_cert => false})
store = OpenSSL::X509::Store.new
store.set_default_paths
ctx = OpenSSL::SSL::SSLContext.new('SSLv23')
ctx.cert_store = store
ctx.verify_mode = params[:ignore_cert] ? OpenSSL::SSL::VERIFY_NONE : OpenSSL::SSL::VERIFY_PEER
ctx.key = nil
ctx.cert = nil
voidcmd("AUTH TLS")
#sock = OpenSSL::SSL::SSLSocket.new(#sock, ctx)
#sock.connect
#sock.post_connection_check(#hostname) unless params[:ignore_cert]
super(user, params[:password], params[:acct])
voidcmd("PBSZ 0")
end
end
end
I also cleaned up the param passing a bit. You would use this like so:
require 'ftptls' # Use my local version, not net/ftptls
#ftp_connection = Net::FTPTLS.new()
#ftp_connection.passive = true
#ftp_connection.connect(host, 21)
#ftp_connection.login('user', :password => 'pass', :ignore_cert => true)
HTH
This works fine for me. #ROR
ftp = Net::FTP.new("ftps.host.com", ftp_options)
open("where/is/your/file/somefile.txt") do |file_data|
ftp.putbinaryfile(file_data, 'where/to/save/somefile.txt')
end
ftp.puttextfile('somefile.txt', 'where/to/save/somefile.txt')
def ftp_options
{
port: FTP_PORT,
username: 'ftp_user',
password: 'password',
passive: true,
ssl: { verify_mode: 0 }
}
end
Remember that you have to provide ftps.hostname.com.

How can I parse a FTP URL with the username or password having special characters?

I am trying to parse a FTP URL that has some special characters like # in the username and password:
username:p#sswrd#ftp.myhost.com/mypath
When I try:
URI.parse(url)
I get:
URI::InvalidURIError: the scheme ftp does not accept registry part: username:p#sswrd#ftp.myhost.com (or bad hostname?)
Then, I tried to encode the url:
url = URI.encode(url, '#')
But also got another error:
URI::InvalidURIError: the scheme ftp does not accept registry part: username:p%40sswrd%40ftp.myhost.com (or bad hostname?)
Finally, I tried another solution:
URI::FTP.build(:userinfo => 'username:p#sswrd', :host=>'ftp.myhost.com', :path => '/mypath')
But I also got an error:
URI::InvalidComponentError: bad component(expected user component): p#ssword
I am using ruby 1.8.7.
require 'net/ftp'
ftp=Net::FTP.new
ftp.connect("ftp.myhost.com",21)
ftp.login("username","p#sswd")
ftp.getbinaryfile("/mypath"){|data| puts data}
ftp.close
If your ftp server supports unicode:
URI::FTP.build(:userinfo => 'username:p%00%40sswrd', :host=>'ftp.myhost.com', :path => '/mypath')
should work. As indicated by this discussion.
But just realized you tried encoding and it failed. Sorry.
I needed to set passive mode as I was receiving the following error:
425 Could not open data connection to port XXXXX: Connection timed out
I did this like so:
require 'net/ftp'
ftp = Net::FTP.new
ftp.passive = true
ftp.connect("ftp.myhost.com",21)
ftp.login("username","p#sswd")
ftp.getbinaryfile("/mypath"){|data| puts data}
ftp.close
I was able to parse an ftp url and open it like this:
ftp_regex = /\/\/(.*)?\:(.*)?#([^\/]*)?\/(.*)/
user, password, host, path = ftp_file_url.match(ftp_regex).captures
ftp = Net::FTP.new
ftp.connect(host, 21)
ftp.login(user, password)
ftp.passive = true
file = ftp.getbinaryfile(path, nil)

Ruby SOAP SSL Woes

I have a SOAP client in Ruby that I'm trying to get working with a Ruby SOAP server, to no avail. The client works fine over SSL with a Python SOAP server, but not with the Ruby version. Here's what the server looks like:
require 'soap/rpc/standaloneServer'
require 'soap/rpc/driver'
require 'rubygems'
require 'httpclient'
def cert(filename)
OpenSSL::X509::Certificate.new(File.open("path to cert.cert") { |f|
f.read
})
end
def key(filename)
OpenSSL::PKey::RSA.new(File.open("path to rsaprivate.key") { |f|
f.read
})
end
class Server < SOAP::RPC::HTTPServer
~code snipped for readability~
end
server = Server.new(:BindAddress => HelperFunctions.local_ip, :Port => 1234, :SSLCertificate => cert("path to cert"), :SSLPrivateKey => key("path to rsa private key"))
new_thread = Thread.new { server.start }
I've trimmed some of the code out for readability's sake (e.g., I have some methods in there I expose) and it works fine with SSL off. But when the client tries to connect, it sees this:
warning: peer certificate won't be verified in this SSL session
/usr/lib/ruby/1.8/net/http.rb:567: warning: using default DH parameters.
/usr/lib/ruby/1.8/net/http.rb:586:in `connect': unknown protocol (OpenSSL::SSL::SSLError)
I tried taking some advice from this post and now I see this message:
/usr/lib/ruby/1.8/soap/httpconfigloader.rb:64:in `set_ssl_config': SSL not supported (NotImplementedError)
Any ideas on how to fix this would be greatly appreciated.
Arg. I was trying to follow along this link and it turns out I was missing a simple include statement:
require 'webrick/https'
That, combined with the help from the link in the original question solves the problem. Hopefully this saves someone else down the line an hour of grief :)
"SSL not supported" can be caused by not having httpclient installed.
Me too.. and don't forget to put the :SSLEnable => true spend couple of hours figuring that out...
server = Server.new(:BindAddress => HelperFunctions.local_ip, :Port => 1234, :SSLEnable => true, :SSLCertificate => cert("path to cert"), :SSLPrivateKey => key("path to rsa private key"))

Resources