Log into a site and download a file - ruby - ruby

Question

I am making a script in the ruby language. This script should log into a site and remove the cookie so that it can be used later to download the file.
I have already made an identical script in bash:
curl --cookie-jar cookie.txt 'https://web.spaggiari.eu/auth-p7/app/default/AuthApi4.php?a=aLoginPwd' -H 'Connection: keep-alive' -H 'Accept: */*' -H 'X-Requested-With: XMLHttpRequest' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4295.0 Safari/537.36 Edg/88.0.680.1' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Origin: https://web.spaggiari.eu' -H 'Sec-Fetch-Site: same-origin' -H 'Sec-Fetch-Mode: cors' -H 'Sec-Fetch-Dest: empty' -H 'Referer: https://web.spaggiari.eu/home/app/default/login.php?target=atv&mode=' -H 'Accept-Language: it,it-IT;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6' -H 'Cookie: _ga=GA1.2.1118066416.1604149840; webrole=gen; webidentity=K6583250C; __auc=l7176b841757ec5618585876044; weblogin=mail.example#example.it; PHPSESSID=bhsktc6oih4k77ufg2v20edmmkl168em; __asc=4f2356e8175928173ed8b4d9596' --data-raw 'cid=&uid=mail.example%40example.it&pwd=Password&pin=&target=' --compressed
curl --cookie cookie.txt "https://web.spaggiari.eu/fml/app/default/xml_export.php?stampa=%3Astampa%3A&report_name=&tipo=agenda&data=03+11+20&autore_id=6583250&tipo_export=EVENTI_AGENDA_STUDENTI&quad=%3Aquad%3A&materia_id=&classe_id=%3Aclasse_id%3A&gruppo_id=%3Agruppo_id%3A&ope=RPT&dal=$data&al=$data&formato=xls" > /home/tommaso/Documenti/bot/allLinks.txt
I am trying to convert this script to Ruby language. I decided to use net / http and made this command for the first command. However, the part to store the cookie is missing (--cookie-jar cookie.txt):
require 'net/http'
require 'uri'
uri = URI.parse("https://web.spaggiari.eu/auth-p7/app/default/AuthApi4.php?a=aLoginPwd")
request = Net::HTTP::Get.new(uri)
request.content_type = "application/x-www-form-urlencoded; charset=UTF-8"
request["Connection"] = "keep-alive"
request["Accept"] = "*/*"
request["X-Requested-With"] = "XMLHttpRequest"
request["User-Agent"] = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4295.0 Safari/537.36 Edg/88.0.680.1"
request["Origin"] = "https://web.spaggiari.eu"
request["Sec-Fetch-Site"] = "same-origin"
request["Sec-Fetch-Mode"] = "cors"
request["Sec-Fetch-Dest"] = "empty"
request["Referer"] = "https://web.spaggiari.eu/home/app/default/login.php?target=atv&mode="
request["Accept-Language"] = "it,it-IT;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"
request["Cookie"] = "_ga=GA1.2.1118066416.1604149840; webrole=gen; webidentity=S6583250C; __auc=b7176b841757ec4912585876044; weblogin=mail.example#example.it; PHPSESSID=cjgqkc6loh4k77ufg2v20edmmkl168em; __asc=4f7916e8175928173ed8b4d9596"
req_options = {
use_ssl: uri.scheme == "https",
}
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
# response.code
# response.body
For the second code I did this command instead. However, the part to send the cookie to the site is missing (--cookie cookie.txt):
require 'net/http'
require 'uri'
uri = URI.parse("https://web.spaggiari.eu/fml/app/default/xml_export.php?stampa=%3Astampa%3A&report_name=&tipo=agenda&data=03+11+20&autore_id=6583250&tipo_export=EVENTI_AGENDA_STUDENTI&quad=%3Aquad%3A&materia_id=&classe_id=%3Aclasse_id%3A&gruppo_id=%3Agruppo_id%3A&ope=RPT&dal=&al=&formato=xls")
response = Net::HTTP.get_response(uri)
# response.code
# response.body
Can anyone tell me how to insert the missing pieces of code? Thanks so much everyone for the help.

Answer 1

You will need to store/read the cookie from a file.
Once you get the initial cookie from the website server:
response = Net::HTTP.get_response(uri)
File.write('./path/to/cookie.txt', response['set-cookie'])
and then to use it again:
# change this line to
request["Cookie"] = "_ga=.."
request["Cookie"] = File.read('./path/to/cookie.txt')
Edit:
Might have to use something other than get_response for your 2nd request so you can specify some headers:
uri = URI.parse("https://web.spaggiari.eu...")
request = Net::HTTP::Get.new(uri)
request["Cookie"] = File.read('./path/to/cookie.txt')
# you can confirm the presence of the header with this:
request.each_header.entries
# Then send it:
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
Edit #2
If this is a quick hack, you can get your curl command responses within you ruby program.. just execute them within backticks:
login_response = `curl --cookie-jar cookie.txt 'https://web.spaggiari.<...>`
all_links = `curl --cookie cookie.txt "https://web.spaggiari.eu/fml<...>?&formato=xls"`
puts all_links