http://ruby-doc.org/stdlib-1.8.7/libdoc/net/http/rdoc/Net/HTTP.html
After reading the doc very carefully, I'm writing the following code snippet for the autologin feature of my program:
url = URI.parse('http://localhost/login.aspx')
req = Net::HTTP::Post.new(url.path)
req.basic_auth 'username'
The target page asks only the correct user name, no password is needed in order to login, the basic_auth method requires two parameters, user name and password, if I leave one out, I'll get the error, I tried to write it like this "req.basic_auth 'username', ''", but I still cannot login.
Could anyone kindly give me a hint?
more info:
I also tried req.basic_auth 'username', '', it didn't seem to be working, I know this because there's another line of line follow right after this one, which is basically doing auto form submission.
x = Net::HTTP.post_form(URI.parse("http://localhost/NewTask.aspx"), params)
puts x.body
And the puts result came back with the redirect to login page body.
You can consider using ruby mechanize gem. a login example will be much simpler(from official site), for this one, you will not need to do the agent cert and private key thing:
require 'rubygems'
require 'mechanize'
# create Mechanize instance
agent = Mechanize.new
# set the path of the certificate file
agent.cert = 'example.cer'
# set the path of the private key file
agent.key = 'example.key'
# get the login form & fill it out with the username/password
login_form = agent.get("http://example.com/login_page").form('Login')
login_form.Userid = 'TestUser'
login_form.Password = 'TestPassword'
# submit login form
agent.submit(login_form, login_form.buttons.first)
Related
I want to retrieve my driving license number, issue_date, and expiry_date from this website("https://sarathi.nic.in:8443/nrportal/sarathi/HomePage.jsp"). When I try to fetch it, I get the error Mechanize::ResponseCodeError: 500 => Net::HTTPInternalServerError for https://sarathi.nic.in:8443/nrportal/sarathi/DlDetRequest.jsp -- unhandled response.
This is the code that I wrote to scrape:
require 'mechanize'
require 'logger'
require 'nokogiri'
require 'open-uri'
require 'openssl'
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
agent = Mechanize.new
agent.log = Logger.new "mech.log"
agent.user_agent_alias = 'Mac Safari 4'
Mechanize.new.get("https://sarathi.nic.in:8443/nrportal/sarathi/HomePage.jsp")
page=agent.get('https://sarathi.nic.in:8443/nrportal/sarathi/HomePage.jsp') # opening home page.
page = agent.page.links.find { |l| l.text == 'Status of Licence' }.click # click the link.
page.forms_with(:name=>"dlform").first.field_with(:name=>"dlform:DLNumber").value="TN38 20120001119" #user input to text field.
page.form_with(:name=>"dlform").field_with(:name=>"javax.faces.ViewState").value="SUBMIT" #submit button value assigning.
page.form(:name=>"dlform",:action=>"/nrportal/sarathi/DlDetRequest.jsp") #to specify the form i need.
agent.cookie_jar.clear!
gg=agent.submit page.forms.last #submitting my form
It isn't working since you are clearing off the cookies before submitting the form, hence removing all the input data you provided. I could get it working by removing it simply as:
...
page.forms_with(:name=>"dlform").first.field_with(:name=>"dlform:DLNumber").value="TN38 20120001119" #user input to text field
form = page.form(:name=>"dlform",:action=>"/nrportal/sarathi/DlDetRequest.jsp")
gg = agent.submit form, form.buttons.first
Note that you do not need to set the value for #submit button, rather pass the submit button while form submission itself.
I am using following code :-
require 'rubygems'
require 'mechanize'
agent = Mechanize.new
agent.get("https://www.rottentomatoes.com/user/account/login/") do |login_page|
inside_page = login_page.form_with(:action => 'https://www.rottentomatoes.com/user/account/login/') do |f|
f.login_username = "random#mailinator.com"
f.login_password = "123456"
end.click_button
end
There isn't any issue with your code, the issue is how Rotten Tomatoes handles a login, they redirect back to the homepage via JavaScript in the HTML body. I added a single line to your code (and added my credentials):
puts agent.page.body
The Result:
<script>
window.top.location='http://www.rottentomatoes.com/';
</script>
So, you can either use their API or if you want to proceed and execute the JavaScript to follow the redirect you can use WATIR or Selenium.
I want to collect manufacturers and their medicine details from http://www.mims.com/India/Browse/Alphabet/All?cat=Company&tab=company.
Mechanize gem is used to extract content from html page with help of ryan Tutorial
I can login successfully but couldn't reach desination page http://www.mims.com/India/Browse/Alphabet/All?cat=Company&tab=company.
I have tried so far
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'mechanize'
agent = Mechanize.new
agent.user_agent = 'Individueller User-Agent'
agent.user_agent_alias = 'Linux Mozilla'
agent.get("https://sso.mims.com/Account/SignIn") do |page|
#login_page = a.click(page.link_with(:text => /Login/))
# Submit the login form
login_page = page.form_with(:action => '/') do |f|
f.SignInEmailAddress = 'username of mims'
f.SignInPassword = 'secret'
end.click_button
url = 'http://www.mims.com/India/Browse/Alphabet/A?cat=drug'
page = agent.get url # here checking authentication if success then redirecting to destination
p page
end
Note: I have shared dummy login credential for your testing
After clicks on 'CompaniesBrowse Company Directory' link, page redirecting with flash message "you are redirecting...", Mechanize gem caches this page.
Question:
1) How to get the original page(after redirection).
I found problem cases that MIMS site auto submit form with page onload callback for checking authentication. It is not working with machanize gem.
Solution
Manually submitting the form two times solves this issue. Example
url = 'http://www.mims.com/India/Browse/Alphabet/A?cat=drug'
page = agent.get url # here checking authentication if success then redirecting to destination
p page
page.form.submit
agent.page.form.submit
I want login to vk.com or m.vk.com without Ruby. But my code dosen't work.
require 'net/http'
email = "qweqweqwe#gmail.com"
pass = "qeqqweqwe"
userUri = URI('m.vk.com/index.html')
Net::HTTP.get(userUri)
res = Net::HTTP.post_form(userUri, 'email' => email, 'pass' => pass)
puts res.body
First of all, you need to change userUri to the following:
userUri = URI('https://login.vk.com/?act=login')
Which is where the vk site expects your login parameters.
I'm not very faimilar with vk, but you probably need a way to handle the session cookie. Both receiving it, and providing it for future requests. Can you elaborate on what you're doing after login?
Here is the net/http info for cookie handling:
# Headers
res['Set-Cookie'] # => String
res.get_fields('set-cookie') # => Array
res.to_hash['set-cookie'] # => Array
puts "Headers: #{res.to_hash.inspect}"
This kind of task is exactly what Mechanize is for. Mechanize handles redirects and cookies automatically. You can do something like this:
require 'mechanize'
agent = Mechanize.new
url = "http://m.vk.com/login/"
page = agent.get(url)
form = page.forms[0]
form['email'] = "qweqweqwe#gmail.com"
form['pass'] = "qeqqweqwe"
form.submit
puts agent.page.body
I ran into a problem when using Mechanize to submit a login form. For example, if I need to log into bitbucket:
a = Mechanize.new
a.get('https://bitbucket.org/') do |page|
login_page = a.click(page.link_with(text: 'Log In'))
my_page = login_page.form_with(action: '/account/signin/') do |f|
# The "username" and "password" below are the values of the "name" attribute of the two login form fields
f.username = 'MY_ACCOUNT_NAME'
f.password = 'MY_PASSWORD'
end.click_button
end
That's pretty straight forward, however, not all login forms have the same "name" value on those two fields. WordPress' login form, for example, uses "log" and "pwd" . This would invalidate the above code.
I want to pass some parameters into this method so that it can be used on different login forms.
I attempted to follow "How to convert from a string to object attribute name?" but was unsuccessful:
# auth_info is a hash
def website_login(auth_info)
a = Mechanize.new
a.get(auth_info[:login_page_url]) do |page|
land_page = page.form_with(action: auth_info[:login_form_action]) do |f|
# Now I am stuck with these two lines
????????
????????
# I tried to do it like this. Not working.
f.instance_variable_set('#'+auth_info[:username_field_name], auth_info[:username])
f.instance_variable_set('#'+auth_info[:password_field_name], auth_info[:password])
end.click_button
end
end
Really appreciate if someone can help out.
Resolved. It's like this:
f.send(auth_info[:username_field_name] + '=', auth_info[:username])
f.send(auth_info[:password_field_name] + '=', auth_info[:password])