I'm using webdriver and Ruby...
So I was able to write text into a tinymce field using the script below. However on the last line, driver.execute... I would like to change the static value 'bob' to be the value of the variable being passed, parValue. I've tried few modifications to the driver.execute_script line, but I continue to get errors. In other words, I don't know javascript and I am unable to find a way to do this.
I've tried replacing the code and use sendkeys, but that does not print anything to my tinymce box. Is there a way to use the value being passed in from parValue and replace 'bob'?
def enterValues(driver,parField,parValue)
tinymce_frame = driver.find_element(:id => parField)
driver.switch_to.default_content
driver.switch_to.frame(tinymce_frame)
editor_body = driver.find_element(:tag_name => 'body')
driver.execute_script("arguments[0].innerHTML = 'bob'", editor_body)
end
this seems kind of simple so I'm worried I might be misunderstanding but you could use ruby's string interpolation to replace bob with parvalue:
def enterValues(driver,parField,parValue)
tinymce_frame = driver.find_element(:id => parField)
driver.switch_to.default_content
driver.switch_to.frame(tinymce_frame)
editor_body = driver.find_element(:tag_name => 'body')
driver.execute_script("arguments[0].innerHTML = '#{parValue}'", editor_body)
end
Related
I am trying to scrape a website and store data in XML using Mechanize and Nokogiri. I didn't set up a Rails project and I am only using Ruby and IRB.
I wrote this method:
def mechanize_club
agent = Mechanize.new
agent.get("http://www.rechercheclub.applipub-fft.fr/rechercheclub/")
form = agent.page.forms.first
form.field_with(:name => 'codeLigue').options[0].select
form.submit
page2 = agent.get('http://www.rechercheclub.applipub-fft.fr/rechercheclub/club.do?codeClub=01670001&millesime=2015')
body = page2.body
html_body = Nokogiri::HTML(body)
codeclub = html_body.search('.form').children("tr:first").children("th:first").to_i
#codeclubs << codeclub
filepath = '/davidgeismar/Documents/codeclubs.xml'
builder = Nokogiri::XML::Builder.new(encoding: 'UTF-8') do |xml|
xml.root {
xml.codeclubs {
#codeclubss.each do |c|
xml.codeclub {
xml.code_ c.code
}
end
}
}
end
puts builder.to_xml
end
My first problem is that I don't know how to test my code.
I call ruby webscraper.rb in my console, the file is treated I think, but it doesn't create an XML file in the specified path.
Then, more specifically I am quite sure this code is wrong as I didn't get a chance to test it.
Basically what I am trying to do is to submit a form several times:
agent = Mechanize.new
agent.get("http://www.rechercheclub.applipub-fft.fr/rechercheclub/")
form = agent.page.forms.first
form.field_with(:name => 'codeLigue').options[0].select
form.submit
I think this code is ok, but I dont want it to only select options[0], I want it to select an option, then scrape all the data I need, then go back to page, then select options[1]... until there are no more options (an iteration I guess).
the file is treated I think, but it doesnt create an xml file in the specified path.
There is nothing in your code that creates a file. You print some output, but don't do anything to open or write a file.
Perhaps you should read the IO and File documentation and review how you are using your filepath variable?
The second problem is that you don't call your method anywhere. Though it's defined and Ruby will see it and parse the method, it has no idea what you want to do with it unless you invoke the method:
def mechanize_club
...
end
mechanize_club()
I have a ruby script that has a hash.
Example:
animal_sound = { 'dog' => 'bark', 'cat' => 'meow' }
I want to add 'snake' => 'hiss'
Example:
myscript.rb --addsound "'snake' => 'hiss'"
Then in my script have it add it to animal_sound.
Example:
animal_sound.merge! 'snake' => 'hiss'
=> {"dog"=>"bark", "cat"=>"meow", "snake"=>"hiss"}
Is there a way to do this?
Here is the whole script:
#!/usr/bin/env ruby
require 'rubygems'
require 'micro-optparse'
options = Parser.new do |p|
p.option :addsound, "add sound"
end.process!
animal_sound = { 'dog' => 'bark', 'cat' => 'meow' }
if options[:add_sound]
newsound = options[:add_sound]
animal_sound.merge! newsound
end
puts animal_sound
When I run my script I get:
$ bin/myscript.rb --addsound "'snake' => 'hiss'"
bin/myscript.rb:14:in `merge!': can't convert true into Hash (TypeError)
from bin/myscript.rb:14:in `<main>'
SOLVED:
Using PSkocik's solution I got the script to work using animal, sound = options[:addsound].split(' => '); animal_sound[animal] = sound
I also used Simone Carletti's idea to simplify the CLI command. FYI it also works if I want to pass in hash format, like myscript.rb --addsound "'snake' => 'hiss'". Of course the split has to be changed back to split(' => '). I like the simpler CLI using the :.
Example:
myscript.rb --addsound snake:hiss
Final Code:
#!/usr/bin/env ruby
require 'rubygems'
require 'micro-optparse'
options = Parser.new do |p|
p.option :addsound, "add sound", default: ""
end.process!
animal_sound = { 'dog' => 'bark', 'cat' => 'meow' }
if options[:addsound]
animal, sound = options[:addsound].split(':')
animal_sound[animal] = sound
end
puts animal_sound
Command line:
$ bin/myscript.rb --addsound snake:hiss
{"dog"=>"bark", "cat"=>"meow", "snake"=>"hiss"}
I never could get the merge to work.
Each post was helpful. Thanks.
It's a good idea to keep the CLI interface detached from the underlying implementation. In fact, you may decide to switch the script in the future from Ruby to another language, and you don't really want to change the way the code is invoked.
My suggestion is to pass a serialized value, for example
myscript.rb --addsound snake:hiss
In the code, simply decompose the content and merge it.
if options[:add_sound]
animal, sound = options[:add_sound].split(":")
animal_sound.merge!(animal => sound)
end
p.option :addsound, "add sound"
^ this makes it a flag (true or false)
What you want is make it into a switch whose value is the next argument:
p.option :addsound, "add sound", default: ""
^ this makes it a switch, the string value will be assigned to options[:addsound]
newsound = options[:addsound]
^ Here you need to drop the underscore and parse the string into a hash.
Eval is evil.
For example, you could split it on ' => ' and forget about quoting:
newsound = [ options[:addsound].split(' => ') ].to_h #and then merge it
(Passing the argument like so --addsound snake:hiss and then splitting on ':' instead of ' => ' is another good option.)
^splitting on ' => ' should yield a two-member array. Here I put it into another array (arrays of two-member arrays are convertible to hashes) to make it convertible into a hash.
Or you do completely without merging and constructing another hash:
animal, sound = options[:addsound].split(' => ')
animal_sound[animal] = sound
In regards to your error
Notice the line if options[:add_sound]. That basically evaluates to if true. You are getting your error because you are setting newsound to true, and trying to merge a Boolean into a hash. To my knowledge, the .merge only works like so: hash1.merge(hash2).
Passing command line argument
Rather than passing the argument "'snake' => 'hiss'", I suggest making this a comma-delineated list, like so: "snake,hiss". From there, in your if options[:add_sound] block, you can split the string into an array, using a comma as a splitter. Finally, rather than using .merge, you can add your key:value as you normally would for any hash in Ruby. animal_sound[arr[0]] = arr[1].
Mind you, this method will work best with a single key:value pair. I am sure you can submit multiple pairs, but you would need to (by this method) split into more arrays by an additional character(like / maybe).
So this is ruby right, and while I do have a solution already, which I'll show below, its not tight. Feels like I'm using ahem "C++ iterators", if you will. Too many lines of code. Not like ruby.
Anyway, I'm wondering if there is classier way to do this:
b = Watir::Browser.new
b.goto "javascriptinjectedtablevalues.com" #not real website url:)
# desired urls in list are immediately located within <span> tags with a "class" of
#"name" plus a custom html attribute attribute of "data-bind" = "name: $data". that's it
# unless I wanted to use child-selectors which I'm not very good at
allrows = b.spans(:class => "name").each_with_index.map do |x, i|
[0, x.attribute_value("data-bind")]
end
real_row_ids = allrows.select{|i, databind| databind == "name: $data" }.map(&:first) #now I have all correct span ids
spans = real_row_ids.map {|id| b.spans(:class => "name")[id] }
Now that's a little messy in my opinion. But it leaves artifacts so I can debug and go back and stuff.
I could use this command to just grab a just the spans
spans = b.spans(:class => "name").map do |span|
[span, span.attribute_value("data-bind")]
end.select {|span, databind| databind == "name: $data"}.map(&:first)
but that still feels messy having no artifacts to show for it to use for later when trying to isolate other html tags nearby the span.
I'm hoping there is something like this pseudo code for watir:
b.spans(:class => "name").with_custom_attributes(:key => "data-bind", :value => "name: $data")
that's what I'd really like to do. superman-patching this custom method onto Watir within a rails initializer would be the optimal solution second to it already existing within Watir!
Watir already supports using data attributes for locators. You simply need to replace the dashes with underscores.
For example:
b.spans(:class => 'name', :data_bind => "name: $data")
Would match elements like:
<span class="name" data-bind="name: $data">
Similarly, you can use a regex when matching the data attribute:
b.spans(:class => 'name', :data_bind => /name/)
I know is a very simple question but I've been stuck for an hour and I just can't understand how this works.
I need to scrape some stuff from my school's library so I need to insert 'CE' to a text field and then click on a link with text 'Clasificación'. The output is what I am going to use to work. So here is my code.
require 'rubygems'
require 'open-uri'
require 'nokogiri'
require 'mechanize'
url = 'http://biblio02.eld.edu.mx/janium-bin/busqueda_rapida.pl?Id=20110720161008#'
searchStr = 'CE'
agent = Mechanize.new
page = agent.get(url)
searchForm = page.form_with(:method => 'post')
searchForm['buscar'] = searchStr
clasificacionLink = page.link_with(:href => "javascript:onClick=set_index_and_submit(\'51\');").click
page = agent.submit(searchForm,clasificacionLink)
When I run it, it gives me this error
janium.rb:31: undefined method `[]=' for nil:NilClass (NoMethodError)
Thanks!
I think your problem is actually on line 13, not 31, and I'll even tell why I think that. Not only does your script not have 31 lines but, from the fine manual:
form_with(criteria)
Find a single form matching criteria.
There are several forms on that page that have method="post". Apparently Mechanize returns nil when it can't exactly match the form_with criteria including the single part mentioned in the documentation; so, if your criteria matches more than one thing, form_with returns nil instead of choosing one of the options and you end up trying to do this:
nil['buscar'] = searchStr
But nil doesn't have a []= method so you get your NoMethodError.
If you use this:
searchForm = page.form_with(:name => 'forma')
you'll get past the first part as there is exactly one form with name="forma" on that page. Then you'll have trouble with this:
clasificacionLink = page.link_with(:href => "javascript:onClick=set_index_and_submit(\'51\');").click
page = agent.submit(searchForm, clasificacionLink)
as Mechanize doesn't know what to do with JavaScript (at least mine doesn't). But if you use just this:
page = agent.submit(searchForm)
you'll get a page and then you can continue building and debugging your script.
mu's answer sounds reasonable. I am not sure if this is strictly necessary, but you might also try to put braces around searchStr.
searchForm['buscar'] = [searchStr]
This is a newbie question as I am attempting to learn Ruby by myself, so apologies if it sounds like a silly question!
I am reading through the examples of why's (poignant) guide to ruby and am in chapter 4. I typed the code_words Hash into a file called wordlist.rb
I opened another file and typed the first line as require 'wordlist.rb' and the rest of the code as below
#Get evil idea and swap in code
print "Enter your ideas "
idea = gets
code_words.each do |real, code|
idea.gsub!(real, code)
end
#Save the gibberish to a new file
print "File encoded, please enter a name to save the file"
ideas_name = gets.strip
File::open( 'idea-' + ideas_name + '.txt', 'w' ) do |f|
f << idea
end
When I execute this code, it fails with the following error message:
C:/MyCode/MyRubyCode/filecoder.rb:5: undefined local variable or method `code_words' for main:Object (NameError)
I use Windows XP and Ruby version ruby 1.8.6
I know I should be setting something like a ClassPath, but not sure where/how to do so!
Many thanks in advance!
While the top-level of all files are executed in the same context, each file has its own script context for local variables. In other words, each file has its own set of local variables that can be accessed throughout that file, but not in other files.
On the other hand, constants (CodeWords), globals ($code_words) and methods (def code_words) would be accessible across files.
Some solutions:
CodeWords = {:real => "code"}
$code_words = {:real => "code"}
def code_words
{:real => "code"}
end
An OO solution that is definitely too complex for this case:
# first file
class CodeWords
DEFAULT = {:real => "code"}
attr_reader :words
def initialize(words = nil)
#words = words || DEFAULT
end
end
# second file
print "Enter your ideas "
idea = gets
code_words = CodeWords.new
code_words.words.each do |real, code|
idea.gsub!(real, code)
end
#Save the gibberish to a new file
print "File encoded, please enter a name to save the file"
ideas_name = gets.strip
File::open( 'idea-' + ideas_name + '.txt', 'w' ) do |f|
f << idea
end
I think the problem might be that the require executes the code in another context, so the runtime variable is no longer available after the require.
What you could try is making it a constant:
CodeWords = { :real => 'code' }
That will be available everywhere.
Here is some background on variable scopes etc.
I was just looking at the same example and was having the same problem.
What I did was change the variable name in both files from code_words to $code_words .
This would make it a global variable and thus accesible by both files right?
My question is: wouldn't this be a simpler solution than making it a constant and having to write CodeWords = { :real => 'code' } or is there a reason not to do it ?
A simpler way would be to use the Marshal.dump feature to save the code words.
# Save to File
code_words = {
'starmonkeys' => 'Phil and Pete, those prickly chancellors of the New Reich',
'catapult' => 'chucky go-go', 'firebomb' => 'Heat-Assisted Living',
'Nigeria' => "Ny and Jerry's Dry Cleaning (with Donuts)",
'Put the kabosh on' => 'Put the cable box on'
}
# Serialize
f = File.open('codewords','w')
Marshal.dump(code_words, f)
f.close
Now at the beginning of your file you would put this:
# Load the Serialized Data
code_words = Marshal.load(File.open('codewords','r'))
Here's the easy way to make sure you can always include a file that's in the same directory as your app, put this before the require statement
$:.unshift File.dirname(__FILE__)
$: is the global variable representing the "CLASSPATH"