I have a form similiar to THIS and want to be submit data to it from a CSV file using ruby. Here is what I have been trying to do:
require 'uri'
require 'net/http'
params = {
'field15157482-first' => 'bip',
'field15157482-last' => 'bop',
'field15157485' => 'bip#bob.com',
'field15157487' => 'option1'
'fsSubmitButton1196962' => 'Submit'
}
x = Net::HTTP.post_form(URI.parse('http://www.formstack.com/forms/?1196833-GxMTxR20GK'), params)
I keep getting A valid form ID was not supplied. I have a hunch I am using the wrong URL but I don't know what to replace it with.
I would use the the API but I don't have access to the token hence my stone age approach. Any suggestions would be much appreciated.
The form uses hidden variables and cookies to attempt to maintain a "unique session". Fortunately, Mechanize makes handling 'sneaky' forms quite easy.
require "mechanize"
form_uri = "http://www.formstack.com/forms/?1196962-617Z6Foyif"
#agent = Mechanize.new
page = #agent.get form_uri
form = page.forms[0]
form.fields_with(:class => /fsField/).each do |field|
field.value = case field.name
when /first/ then "First Name"
when /last/ then "Last Name"
else "email#address.com"
end
end
page = form.submit form.buttons.first
puts
puts "=== Response Header"
puts
puts page.header
puts
puts "=== Response Body"
puts
puts page.body
Looking at the source on http://www.formstack.com/forms/?1196833-GxMTxR20GK and the example in your link, it appears that formstack forms post to index.php, and require a form id to be passed in to identify which form is being submitted.. Looking at the forms in both examples, you'll see a field similar to this:
<input type="hidden" name="form" value="1196833" />
Try adding the following to your params hash:
'form' => '1196883' # or other appropriate form value
You may also need to include the other hidden fields for a valid submit.
Related
I'm trying to get to google play developer console using ruby. But first I have to login. I'm trying like this:
def try_post(url, body = {}, headers = {})
unless #agent #This just creates a new mechanize instance
setup
end
puts 'Logging in'
# Hardcoded for testing purposes
#agent.get 'https://accounts.google.com/ServiceLogin?service=androiddeveloper&passive=1209600&continue=https://play.google.com/apps/publish/%23&followup=https://play.google.com/apps/publish/#identifier'
form = #agent.page.forms.find {|f| f.form_node['id'] == "gaia_loginform"}
unless form
raise 'No login form'
end
form.field_with(:id => "Email").value = #config.email
form.click_button
form = #agent.page.forms.find {|f| f.form_node['id'] == "gaia_loginform"}
unless form
raise 'No login form'
end
form.field_with(:name => "Passwd").value = #config.password
form.click_button
if #agent.page.uri.host != "play.google.com"
STDERR.puts "login failed? : uri = " + #agent.page.uri.to_s
raise 'Google login failed'
end
# #agent.post(url, body)
end
However this fails spectacularly. I tried a few other ways (trying to populate Passwd-hidden, finding field by id and so on) but no luck. I think that the password does not get entered since when I try to puts #agent.page.body after the final click_button I see enter password text somewhere in HTML.
What am I doing wrong and how can I fix it?
I've been digging around a bit more and found out that it's not that simple and I could not login with mechanize in any way.
So I ended up with using watir which was fairly simple and straightforward. Here's an example:
browser.goto LOGIN_URL
browser.text_field(:id, 'Email').set #config.email
browser.button(:id, 'next').click
browser.text_field(:id, 'Passwd').wait_until_present
browser.text_field(:id, 'Passwd').set #config.password
browser.button(:id, 'signIn').click
# Here I wait until an element on my target page is visible and then continue
browser.link(:href, '#SOMETHING').wait_until_present
Hope it helps.
Need to post to existing web page (no login required) and post parameters for submit where multiple submit forms tags exist and contains identical tags with the same NAME and VALUE tags; for example, on the same page this INPUT Submit is repeated 3 times under different FORM tags:
< INPUT TYPE='Submit' NAME='submit_button' VALUE='Submit Query' >
My Ruby code runs ok for identifying the fields on the form tags, but fails on the page.forms[x].action post with 405 HTTPMethodNotAllowed for https://pdb.nipr.com/html/PacNpnSearch -- unhandled response.
Ruby code:
class PostNIPR2
def post(url)
button_count = 0
agent = Mechanize.new
page = agent.get(url)
page.forms.each do |form|
form.buttons.each do |button|
if(button.value == 'Submit Query')
button_count = button_count + 1
if (button_count == 3)
btn_submit_license = button.name
puts button
puts btn_submit_license
puts button.value
end
end
end
end
begin
uform = page.forms[1]
uform.license = "0H20649"
uform.state = "CA"
uform.action = 'https://pdb.nipr.com/html/PacNpnSearch'
rescue Exception => e
error_page = e.page
end
page = agent.submit(uform)
end
url = "https://pdb.nipr.com/html/PacNpnSearch.html"
p = PostNIPR2.new
p.post(url)
end
Is your question how to select that button? If so:
form.button_with(:name => 'submit_button')
or submit the form like this:
next_page = form.submit form.button_with(:name => 'submit_button')
Also you are changing the form's action for some reason and that will explain the 405s
You are correct, sorry about the comment code - the question was to have the form.license and form.state updated with the input params then have the form.submit post the form.button_with(:name => 'Submit Query' - I did this and received the 405 HTTPMethodNotAllowed, while for https://pdb.nipr.com/html/PacNpnSearch -- unhandled response. But now I have changed the code to agent.page.form_with(:name => 'license_form') which now correctly finds the form I need to post to; then I get the form.button_with(:value => 'Submit Query') and then utilize the agent.submit(form, button). Now I get the correct result.
I ran into a problem when using Mechanize to submit a login form. For example, if I need to log into bitbucket:
a = Mechanize.new
a.get('https://bitbucket.org/') do |page|
login_page = a.click(page.link_with(text: 'Log In'))
my_page = login_page.form_with(action: '/account/signin/') do |f|
# The "username" and "password" below are the values of the "name" attribute of the two login form fields
f.username = 'MY_ACCOUNT_NAME'
f.password = 'MY_PASSWORD'
end.click_button
end
That's pretty straight forward, however, not all login forms have the same "name" value on those two fields. WordPress' login form, for example, uses "log" and "pwd" . This would invalidate the above code.
I want to pass some parameters into this method so that it can be used on different login forms.
I attempted to follow "How to convert from a string to object attribute name?" but was unsuccessful:
# auth_info is a hash
def website_login(auth_info)
a = Mechanize.new
a.get(auth_info[:login_page_url]) do |page|
land_page = page.form_with(action: auth_info[:login_form_action]) do |f|
# Now I am stuck with these two lines
????????
????????
# I tried to do it like this. Not working.
f.instance_variable_set('#'+auth_info[:username_field_name], auth_info[:username])
f.instance_variable_set('#'+auth_info[:password_field_name], auth_info[:password])
end.click_button
end
end
Really appreciate if someone can help out.
Resolved. It's like this:
f.send(auth_info[:username_field_name] + '=', auth_info[:username])
f.send(auth_info[:password_field_name] + '=', auth_info[:password])
I am trying to figure out if mechanize sends correct post query.
I want to log in to a forum (please see html source, mechanize log in my other question) but I get only the login page again. When looking into it I can see that firefox sends out post with parameters like
auth_username=myusername&auth_password=mypassword&auth_login=Login but my script sends
auth_username=radek&auth_password=mypassword is that ok or the &auth_login=Login part must be present?
When I tried to add it using login_form['auth_login'] = 'Login' I got an error gems/mechanize-0.9.3/lib/www/mechanize/page.rb:13 inmeta': undefined method search' for nil:NilClass (NoMethodError)
It seems to me that auth_login is a form button not a field (I don't know if it matters)
[#<WWW::Mechanize::Form
{name nil}
{method "POST"}
{action
"http://www.somedomain.com/login?auth_successurl=http://www.somedomain.com/forum/yota?baz_r=1"}
{fields
#<WWW::Mechanize::Form::Field:0x36946c0 #name="auth_username", #value="">
#<WWW::Mechanize::Form::Field:0x369451c #name="auth_password", #value="">}
{radiobuttons}
{checkboxes}
{file_uploads}
{buttons
#<WWW::Mechanize::Form::Button:0x36943b4
#name="auth_login",
#value="Login">}>
]
My script is as follow
require 'rubygems'
require 'mechanize'
require 'logger'
agent = WWW::Mechanize.new {|a| a.log = Logger.new("loginYOTA.log") }
agent.follow_meta_refresh = true #Mechanize does not follow meta refreshes by default, we need to set that option.
page = agent.get("http://www.somedomain.com/login?auth_successurl=http://www.somedomain.com/forum/yota?baz_r=1")
login_form = page.form_with(:method => 'POST') #works
puts login_form.buttons.inspect
puts page.forms.inspect
STDIN.gets
login_form.fields.each { |f| puts "#{f.name} : #{f.value}" }
#STDIN.gets
login_form['auth_username'] = 'myusername'
login_form['auth_password'] = 'mypassword'
login_form['auth_login'] = 'Login'
STDIN.gets
page = agent.submit login_form
#Display message if logged in
puts page.parser.xpath("/html/body/div/div/div/table/tr/td[2]/div/strong").xpath('text()').to_s.strip
puts
puts page.parser.xpath("/html/body/div/div/div/table/tr/td[2]/div").xpath('text()').to_s.strip
output = File.open("login.html", "w") {|f| f.write(page.parser.to_html) }
You can find more code, html, log in my other related question log in with browser and then ruby/mechanize takes it over?
the absence of one parameter compare to firefox in POST caused mechanize not to log in. Adding new parameter solved this problem. So it seems to me that the web server requires &auth_login=Login parameter to be in POST.
You can read how to add new field to mechanize form in another question.
there is a public class method to add field to mechanize form
I tried ..
#login_form.field.new('auth_login','Login')
#login_form.field.new('auth_login','Login')
and both gives me an error undefined method "new" for #<WWW::Mechanize::Form::Field:0x3683cbc> (NoMethodError)
I tried login_form.field.new('auth_login','Login') which gives me an error
mechanize-0.9.3/lib/www/mechanize/page.rb:13 n `meta': undefined method `search' for nil:NilClass (NoMethodError)
but at the time I submit the form. The field does not exist in html source. I want to add it so POST query sent by my script will contain auth_username=myusername&auth_password=mypassword&auth_login=Login So far it sends only auth_username=radek&auth_password=mypassword which might be why I cannot get logged in. Just my thought.
The script looks like
require 'rubygems'
require 'mechanize'
require 'logger'
agent = WWW::Mechanize.new {|a| a.log = Logger.new("loginYOTA.log") }
agent.follow_meta_refresh = true #Mechanize does not follow meta refreshes by default, we need to set that option.
page = agent.get("http://www.somedomain.com/login?auth_successurl=http://www.somedomain.com/forum/yota?baz_r=1")
login_form = page.form_with(:method => 'POST')
puts login_form.buttons.inspect
puts page.forms.inspect
#STDIN.gets
login_form.fields.each { |f| puts "#{f.name} : #{f.value}" }
login_form['auth_username'] = 'radeks'
login_form['auth_password'] = 'TestPass01'
#login_form['auth_login'] = 'Login'
#login_form.field.new('auth_login','Login')
#login_form.field.new('auth_login','Login')
#login_form.fields.each { |f| puts "#{f.name} : #{f.value}" }
#STDIN.gets
page = agent.submit login_form
#Display welcome message if logged in
puts page.parser.xpath("/html/body/div/div/div/table/tr/td[2]/div/strong").xpath('text()').to_s.strip
puts
puts page.parser.xpath("/html/body/div/div/div/table/tr/td[2]/div").xpath('text()').to_s.strip
output = File.open("login.html", "w") {|f| f.write(page.parser.to_html) }
The .inspect of the form looks like
[#<WWW::Mechanize::Form
{name nil}
{method "POST"}
{action
"http://www.somedomain.com/login?auth_successurl=http://www.somedomain.com/forum/yota?baz_r=1"}
{fields
#<WWW::Mechanize::Form::Field:0x36946c0 #name="auth_username", #value="">
#<WWW::Mechanize::Form::Field:0x369451c #name="auth_password", #value="">}
{radiobuttons}
{checkboxes}
{file_uploads}
{buttons
#<WWW::Mechanize::Form::Button:0x36943b4
#name="auth_login",
#value="Login">}>
]
I think what you're looking for is
login_form.add_field!(field_name, value = nil)
Here are the docs:
http://rdoc.info/projects/tenderlove/mechanize
The difference between this and the method WWW::Mechanize::Form::Field.new is not much, aside from the fact that there aren't many ways to add fields to a form. Here's how the add_field! method is implemented....you can see that it's exactly what you'd expect. It instantiates a Field object, then adds it to the form's 'fields' array. You wouldn't be able to do this in your code because the method "fields<<" is a private method inside "Form."
# File lib/www/mechanize/form.rb, line 65
def add_field!(field_name, value = nil)
fields << Field.new(field_name, value)
end
On a side note, according to the docs you should be able to do the first variation you proposed:
login_form['field_name']='value'
Hope this helps!
another way how to add new field is to so at the time of posting the form
page = agent.post( url, {'auth_username'=>'myusername', #existing field
'auth_password'=>'mypassword', #existing field
'auth_login'=>'Login'}) #new field