Connect to uncertified host via FTP over TLS/SSL - ruby

A vendor I grab a file from is changing from FTP to FTP over SSL.
I am trying to update my code from net/ftp to net/ftptls
The new host I need to connect to is not certified and my script reports back this error.
hostname was not match with the server certificate
The vendor will not fix this.
Looking at /usr/lib/ruby/1.8/net/ftptls.rb I thought it wouldn't be too hard to monkey-patch FTPTLS to ignore the untrusted host.
I tried changing verify_mode to OpenSSL::SSL::VERIFY_NONE and commenting out the post_connection_check` line.
neither worked.
Any thoughts on how to do this?
require 'socket'
require 'openssl'
require 'net/ftp'
module Net
class FTPTLS < FTP
def connect(host, port=FTP_PORT)
#hostname = host
super
end
def login(user = "anonymous", passwd = nil, acct = nil)
store = OpenSSL::X509::Store.new
store.set_default_paths
ctx = OpenSSL::SSL::SSLContext.new('SSLv23')
ctx.cert_store = store
ctx.verify_mode = OpenSSL::SSL::VERIFY_PEER
ctx.key = nil
ctx.cert = nil
voidcmd("AUTH TLS")
#sock = OpenSSL::SSL::SSLSocket.new(#sock, ctx)
#sock.connect
#sock.post_connection_check(#hostname)
super(user, passwd, acct)
voidcmd("PBSZ 0")
end
end
end

This may be the world's slowest answer, but I ran across your question and it helped my fix it myself, so I wanted to post for posterity.
You were very close, you just need to also comment out #post_connection_check.
What I did, rather than monkeypatching ruby itself, was bring a copy of this into /lib of my project.
module Net
class FTPTLS < FTP
def connect(host, port=FTP_PORT)
#hostname = host
super
end
def login(user = "anonymous", params = {:password => nil, :acct => nil, :ignore_cert => false})
store = OpenSSL::X509::Store.new
store.set_default_paths
ctx = OpenSSL::SSL::SSLContext.new('SSLv23')
ctx.cert_store = store
ctx.verify_mode = params[:ignore_cert] ? OpenSSL::SSL::VERIFY_NONE : OpenSSL::SSL::VERIFY_PEER
ctx.key = nil
ctx.cert = nil
voidcmd("AUTH TLS")
#sock = OpenSSL::SSL::SSLSocket.new(#sock, ctx)
#sock.connect
#sock.post_connection_check(#hostname) unless params[:ignore_cert]
super(user, params[:password], params[:acct])
voidcmd("PBSZ 0")
end
end
end
I also cleaned up the param passing a bit. You would use this like so:
require 'ftptls' # Use my local version, not net/ftptls
#ftp_connection = Net::FTPTLS.new()
#ftp_connection.passive = true
#ftp_connection.connect(host, 21)
#ftp_connection.login('user', :password => 'pass', :ignore_cert => true)
HTH

I know this is probably too late for Poul but I found the double-bag-ftps gem to be sufficient and easy to use when I had to do something similar.

Related

Ruby & IMAP - Accessing Office 365 with Oauth 2.0

So MS disabled IMAP for basic auth as we all know.
I am trying to figure out how to get the OAUTH 2.0 working using ruby (not ruby on rails).
I have Azure APP and everything needed (I think), but I can not find any code related to ruby and getting the access token.
First step is completed, but next step is to get the access token.
https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
I need to read different Outlook mailboxes.
Could someone please explain how to do this?
SOLUTION for me!
Steps I took.
Made an Azure app ('Device Flow' was the easiest way to go for me) Check the Steps in the link. You also need to change some settings in your APP if you want to use IMAP. See the youtube link here between 2:50 - 4:30
Get the postman requests from this link (scroll down a little) (click here)
From postman you can use "Device Flow" requests.
Start with Device Authorization Request (you need a scope and client_id for this) I used https://outlook.office.com/IMAP.AccessAsUser.All scope.
go to the link that you got back from the request and enter the required code.
now go to Device Access Token Request and use the "device_code" from the last request and put that under code, under body.
You should get an access_token
Connect using ruby
require 'gmail_xoauth' # MUST HAVE! otherwise XOAUTH2 auth wont work
require 'net/imap'
imap = Net::IMAP.new(HOST, PORT, true)
access_token = "XXXXX"
user_name = "email#outlook.com"
p imap.authenticate('XOAUTH2',"#{user_name}", "#{access_token}")
# example
imap.list('','*').each do |folders|
p folders
end
XOAUTH2 Returns
#<struct Net::IMAP::TaggedResponse tag="RUBY0001", name="OK", data=#<struct Net::IMAP::ResponseText code=nil, text="AUTHENTICATE completed.">, raw_data="RUBY0001 OK AUTHENTICATE completed.\r\n
Just to specify
HOST = 'outlook.office365.com'
PORT = 993
UPDATE 25.01.2023
class Oauth2
require 'selenium-webdriver'
require 'webdrivers'
require 'net/http'
# Use: Oauth2.new.get_access_code
# Grants access to Office 365 emails.
def get_access_code
p "### Access Request Started #{Time.now} ###"
begin
codes = device_auth_request
authorize_device_code(codes[:user_code])
access_code = device_access_token(codes[:device_code])
access_code
rescue => e
p e
p "Something went wrong with authorizing"
end
end
def device_auth_request # Returns user_code and device_code
url = URI('https://login.microsoftonline.com/organizations/oauth2/v2.0/devicecode')
https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true
request = Net::HTTP::Post.new(url)
request.body = "client_id=YOUR_CLIENT_ID&scope=%09https%3A%2F%2Foutlook.office.com%2FIMAP.AccessAsUser.All"
response = https.request(request)
{
user_code: JSON.parse(response.read_body)["user_code"],
device_code: JSON.parse(response.read_body)["device_code"]
}
end
def device_access_token(device_code)
url = URI('https://login.microsoftonline.com/organizations/oauth2/v2.0/token')
https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true
request = Net::HTTP::Post.new(url)
request.body = "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Adevice_code&code=#{device_code}&client_id=YOUR_CLIENT_ID"
response = https.request(request)
JSON.parse(response.read_body)["access_token"]
end
def authorize_device_code(device_code)
# SELENIUM SETUP
driver = setup_selenium
driver.get "https://microsoft.com/devicelogin"
sleep(4)
# ------------------------------------------
# Give Access
element = driver.find_element(:class, "form-control")
element.send_keys(device_code)
sleep(2)
element = driver.find_element(:id, "idSIButton9")
element.submit
sleep(2)
element = driver.find_element(:id, "i0116")
element.send_keys("YOUR OUTLOOK ACCOUNT EMAIL")
sleep(2)
element = driver.find_element(:class, "button_primary")
element.click
sleep(2)
element = driver.find_element(:id, "i0118")
element.send_keys("YOUR OUTLOOK PASSWORD")
element = driver.find_element(:class, "button_primary")
element.click
sleep(2)
element = driver.find_element(:class, "button_primary")
element.click
sleep(2)
# ------------------------------------------
driver.quit
end
def setup_selenium
require 'selenium-webdriver'
# set up Selenium
options = Selenium::WebDriver::Chrome::Options.new(
prefs: {
download: {
prompt_for_download: false
},
plugins: {
'always_open_pdf_externally' => true
}
}
)
options.add_argument('--headless')
options.add_argument('--no-sandbox')
# options.add_argument('-incognito')
options.add_argument('disable-popup-blocking')
Selenium::WebDriver.for :chrome, options: options
end
end

Connect to a FTPS server with mismatched server certificate using Net::FTPTLS

I am trying to connect via Net::FTPTLS to a Microsoft-based file server (IIS) which is configured to use FTP on port 22 and requires SSL.
I connect via:
require 'net/ftptls'
ftp = Net::FTPTLS.new()
ftp.connect('host.com', port_number)
ftp.login('Username', 'Password')
ftp.puttextfile('somefile.txt', 'where/to/save/somefile.txt')
ftp.close
Problem is, I get the following error:
hostname does not match the server certificate
It seems that I have to disable the openssl peer verification: OpenSSL::SSL::VERIFY_PEER should become OpenSSL::SSL::VERIFY_NONE.
Any ideas on how to monkey-patch the Net::FTPTLS class? Has anyone done this successfully?
Instead using Net::FTPTLS, use Ruby 2.4+ with the following code:
require 'net/ftp'
ftp = Net::FTP.new(nil, ssl: {:verify_mode => OpenSSL::SSL::VERIFY_NONE})
ftp.connect('host.com', port_number)
ftp.login('Username', 'Password')
ftp.puttextfile('somefile.txt', 'where/to/save/somefile.txt')
ftp.close
What I did, rather than monkeypatching ruby itself, was bring a copy of this into /lib of my project.
module Net
class FTPTLS < FTP
def connect(host, port=FTP_PORT)
#hostname = host
super
end
def login(user = "anonymous", params = {:password => nil, :acct => nil, :ignore_cert => false})
store = OpenSSL::X509::Store.new
store.set_default_paths
ctx = OpenSSL::SSL::SSLContext.new('SSLv23')
ctx.cert_store = store
ctx.verify_mode = params[:ignore_cert] ? OpenSSL::SSL::VERIFY_NONE : OpenSSL::SSL::VERIFY_PEER
ctx.key = nil
ctx.cert = nil
voidcmd("AUTH TLS")
#sock = OpenSSL::SSL::SSLSocket.new(#sock, ctx)
#sock.connect
#sock.post_connection_check(#hostname) unless params[:ignore_cert]
super(user, params[:password], params[:acct])
voidcmd("PBSZ 0")
end
end
end
I also cleaned up the param passing a bit. You would use this like so:
require 'ftptls' # Use my local version, not net/ftptls
#ftp_connection = Net::FTPTLS.new()
#ftp_connection.passive = true
#ftp_connection.connect(host, 21)
#ftp_connection.login('user', :password => 'pass', :ignore_cert => true)
HTH
This works fine for me. #ROR
ftp = Net::FTP.new("ftps.host.com", ftp_options)
open("where/is/your/file/somefile.txt") do |file_data|
ftp.putbinaryfile(file_data, 'where/to/save/somefile.txt')
end
ftp.puttextfile('somefile.txt', 'where/to/save/somefile.txt')
def ftp_options
{
port: FTP_PORT,
username: 'ftp_user',
password: 'password',
passive: true,
ssl: { verify_mode: 0 }
}
end
Remember that you have to provide ftps.hostname.com.

How to properly implement Net::SSH port forwards

I have been trying to get port forwarding to work correctly with Net::SSH. From what I understand I need to fork out the Net::SSH session if I want to be able to use it from the same Ruby program so that the event handling loop can actually process packets being sent through the connection. However, this results in the ugliness you can see in the following:
#!/usr/bin/env ruby -w
require 'net/ssh'
require 'httparty'
require 'socket'
include Process
log = Logger.new(STDOUT)
log.level = Logger::DEBUG
local_port = 2006
child_socket, parent_socket = Socket.pair(:UNIX, :DGRAM, 0)
maxlen = 1000
hostname = "www.example.com"
pid = fork do
parent_socket.close
Net::SSH.start("hostname", "username") do |session|
session.logger = log
session.logger.sev_threshold=Logger::Severity::DEBUG
session.forward.local(local_port, hostname, 80)
child_socket.send("ready", 0)
pidi = fork do
msg = child_socket.recv(maxlen)
puts "Message from parent was: #{msg}"
exit
end
session.loop do
status = waitpid(pidi, Process::WNOHANG)
puts "Status: #{status.inspect}"
status.nil?
end
end
end
child_socket.close
puts "Message from child: #{parent_socket.recv(maxlen)}"
resp = HTTParty.post("http://localhost:#{local_port}/", :headers => { "Host" => hostname } )
# the write cannot be the last statement, otherwise the child pid could end up
# not receiving it
parent_socket.write("done")
puts resp.inspect
Can anybody show me a more elegant/better working solution to this?
I spend a lot of time trying to figure out how to correctly implement port forwarding, then I took inspiration from net/ssh/gateway library. I needed a robust solution that works after various possible connection errors. This is what I'm using now, hope it helps:
require 'net/ssh'
ssh_options = ['host', 'login', :password => 'password']
tunnel_port = 2222
begin
run_tunnel_thread = true
tunnel_mutex = Mutex.new
ssh = Net::SSH.start *ssh_options
tunnel_thread = Thread.new do
begin
while run_tunnel_thread do
tunnel_mutex.synchronize { ssh.process 0.01 }
Thread.pass
end
rescue => exc
puts "tunnel thread error: #{exc.message}"
end
end
tunnel_mutex.synchronize do
ssh.forward.local tunnel_port, 'tunnel_host', 22
end
begin
ssh_tunnel = Net::SSH.start 'localhost', 'tunnel_login', :password => 'tunnel_password', :port => tunnel_port
puts ssh_tunnel.exec! 'date'
rescue => exc
puts "tunnel connection error: #{exc.message}"
ensure
ssh_tunnel.close if ssh_tunnel
end
tunnel_mutex.synchronize do
ssh.forward.cancel_local tunnel_port
end
rescue => exc
puts "tunnel error: #{exc.message}"
ensure
run_tunnel_thread = false
tunnel_thread.join if tunnel_thread
ssh.close if ssh
end
That's just how SSH in general is. If you're offended by how ugly it looks, you should probably wrap up that functionality into a port forwarding class of some sort so that the exposed part is a lot more succinct. An interface like this, perhaps:
forwarder = PortForwarder.new(8080, 'remote.host', 80)
So I have found a slightly better implementation. It only requires a single fork but still uses a socket for the communication. It uses IO#read_nonblock for checking if a message is ready. If there isn't one, the method throws an exception, in which case the block continues to return true and the SSH session keeps serving requests. Once the parent is done with the connection it sends a message, which causes child_socket.read_nonblock(maxlen).nil? to return false, making the loop exit and therefore shutting down the SSH connection.
I feel a little better about this, so between that and #tadman's suggestion to wrap it in a port forwarding class I think it's about as good as it'll get. However, any further suggestions for improving this are most welcome.
#!/usr/bin/env ruby -w
require 'net/ssh'
require 'httparty'
require 'socket'
log = Logger.new(STDOUT)
log.level = Logger::DEBUG
local_port = 2006
child_socket, parent_socket = Socket.pair(:UNIX, :DGRAM, 0)
maxlen = 1000
hostname = "www.example.com"
pid = fork do
parent_socket.close
Net::SSH.start("ssh-tunnel-hostname", "username") do |session|
session.logger = log
session.logger.sev_threshold=Logger::Severity::DEBUG
session.forward.local(local_port, hostname, 80)
child_socket.send("ready", 0)
session.loop { child_socket.read_nonblock(maxlen).nil? rescue true }
end
end
child_socket.close
puts "Message from child: #{parent_socket.recv(maxlen)}"
resp = HTTParty.post("http://localhost:#{local_port}/", :headers => { "Host" => hostname } )
# the write cannot be the last statement, otherwise the child pid could end up
# not receiving it
parent_socket.write("done")
puts resp.inspect

Ruby - remote file download ...timeout?

Excuse the tabs. I'm trying to download a file from remote to local and I keep getting one back that is exactly 310 bytes (regardless of what file I choose to download). I tried setting the timeout to 0, but this isn't working. What am I doing wrong? Thanks!
#downloadUrl = 'https://username:password#api.net'
Net::HTTP.start(#downloadUrl) do |http|
response = http.get('/file.ext')
open('/Users/me/file.ext', "wb", :read_timeout=>0) do |file|
file.write(response.body)
end
end
EDIT: I don't want to use httpclient, I want to use standard net/http. I am almost there, but I keep getting initialize': getaddrinfo: nodename nor servname provided, or not known (SocketError) thrown at Net::HTTP.start(url.path). But when I remote "https", I get ECONNREFUSED. ...Getting closer?
url = URI.parse('https://api.net/file.ext')
#request = Net::HTTP.start(url.path)
#request.basic_auth 'username', 'password'
sock = Net::HTTP.new(url.host, 443)
sock.use_ssl = true
sock.ssl_version='SSLv3'
sock.start do |http|
response = http.get(#request)
open('/Users/me/file.ext', "wb", :read_timeout=>0) do |file|
file.write(response.body)
end
end
Using httpclient is much simpler when accessing via SSL.
gem install httpclient
I haven't tried this out, but this should work for you. Here is the rdoc.
require 'httpclient'
domain = "https://api.net/file.ext"
username = "username"
password = "password"
clnt = HTTPClient.new
clnt.set_auth(domain, username, password)
res = clnt.get_content(https_url)
You can refer to the "writing a binary file in ruby" question for saving your response to a file.
Updated Using net/http
You should be doing Net::HTTP.start(url.host, url.port) instead of Net:HTTP.start(url.path). See RDoc for Net::HTTP.start.
url = URI.parse("https://api.net/file.ext")
download_file = opne("/Users/me/file.ext", "wb")
request = Net::HTTP.start(url.host, url.port)
# .. set basic auth, verify peer etc
begin
request.request_get(url.path) do |resp|
resp.read_body { |segment| download_file.write(segment) }
end
ensure
download_file.close
end

Using passphrase callback in ruby gpgme

I am using ruby gpgme gem (1.0.8). My passphrase callback isn't called:
def passfunc(*args)
fd = args.last
io = IO.for_fd(fd, 'w')
io.puts "mypassphrase"
io.flush
end
opts = {
:passphrase_callback => method(:passfunc)
}
GPGME.decrypt(input,output, opts)
Does someone have working example of passphrase callback?
Sample of callback you can find in the following working example. It signs a file in detached mode, i.e., the signature file is separated from the original file. It uses the default keyring at ~/.gnupg or something like that. To use a different directory for your keyring, set the environment variable ENV["GNUPGHOME"]="" before call GPGME::sign().
#!/usr/bin/ruby
require 'rubygems'
require 'gpgme'
puts "Signing #{ARGV[0]}"
input = File.open(ARGV[0],'r')
PASSWD = "abc"
def passfunc(hook, uid_hint, passphrase_info, prev_was_bad, fd)
puts("Passphrase for #{uid_hint}: ")
io = IO.for_fd(fd, 'w')
io.write(PASSWD+"\n")
io.flush
end
output = File.open(ARGV[0]+'.asc','w')
sign = GPGME::sign(input, {
:passphrase_callback => method(:passfunc),
:mode => GPGME::SIG_MODE_DETACH
})
output.write(sign)
output.close
input.close
Here's another working example for you that doesn't use a detached signature. To test this, simply change 'user#host.name' to the identifier of your key and do this: GPG.decrypt(GPG.encrypt('some text', :armor => true))
require 'gpgme'
require 'highline/import'
module GPG
ENCRYPT_KEY = 'user#host.com'
#gpg = GPGME::Crypto.new
class << self
def decrypt(encrypted_data, options = {})
options = { :passphrase_callback => self.method(:passfunc) }.merge(options)
#gpg.decrypt(encrypted_data, options).read
end
def encrypt(data_to_encrypt, options = {})
options = { :passphrase_callback => self.method(:passfunc), :armor => true }.merge(options)
#gpg.encrypt(data_to_encrypt, options).read
end
private
def get_passphrase
ask("Enter passphrase for #{ENCRYPT_KEY}: ") { |q| q.echo = '*' }
end
def passfunc(hook, uid_hint, passphrase_info, prev_was_bad, fd)
begin
system('stty -echo')
io = IO.for_fd(fd, 'w')
io.puts(get_passphrase)
io.flush
ensure
(0 ... $_.length).each do |i| $_[i] = ?0 end if $_
system('stty echo')
end
$stderr.puts
end
end
end
Cheers!,
--
Carl
It is important to note that as of GnuPG 2.0 (and in 1.4 when the use-agent option is used) pinentry is used for passphrase collection. This means that the gpgme passphrase callback will not be invoked. This is described here and an example of usage can be found in the gpgme-tool example.

Resources