Ruby: get web elements inside the Shadow-Root DOM - ruby

I'm new to Ruby automation test using cucumber and selenium-webdriver.
I received the source code automation from another guy. The way he find element on page some thing looks like:
element(:error_message) { browser.elements(class: 'input-invalidate') }
Now I need to access the element inside a ShadowRoot, did some researches but could not get answer for the Ruby code.
Below picture is an example, I'd like to get the div tag with id="maincontainer" inside that shadowroot, anyone can help please?
Thanks you

This is now supported in Selenium 4.0, here's a working example:
driver.get('http://watir.com/examples/shadow_dom.html')
shadow_host = driver.find_element(id: 'shadow_host')
shadow_root = shadow_host.shadow_root
shadow_content = shadow_root.find_element(id: 'shadow_content')

Also verify that you have selenium devtools gem installed, and at least chrome v96 with newest chromedriver
This will work like Titus pointed out when locating element with Selenium
browser = Watir::Browser.new
browser.goto "http://watir.com/examples/shadow_dom.html"
shadow_host = browser.driver.find_element(id: 'shadow_host')
shadow_root = shadow_host.shadow_root
shadow_content = shadow_root.find_element(id: 'shadow_content')
You can also locate element with Watir and then call shadow_root on underlying Selenium element
shadow_host = browser.div(id: 'shadow_host') #Watir::Div
shadow_root = shadow_host.wd.shadow_root
edit: this should also work in theory - converting ShadowRoot to Watir element, but it breaks afterwards.
browser.goto "http://watir.com/examples/shadow_dom.html"
shadow_host = browser.div(id: 'shadow_host') #Watir::Div
shadow_root = shadow_host.wd.shadow_root #Selenium::WebDriver::ShadowRoot
watir_shadow = browser.div(element: shadow_root) #Watir::Div
watir_shadow.divs.count #undefined method `keys' for nil:NilClass
I might be doing something wrong :) best to ask #titusfortner
/watirs/watir-7.1.0/lib/watir/locators/element/selector_builder.rb:73:in `merge_scope?'
/watirs/watir-7.1.0/lib/watir/locators/element/selector_builder.rb:50:in `normalize_selector'
/watirs/watir-7.1.0/lib/watir/locators/element/selector_builder.rb:28:in `build'
/watirs/watir-7.1.0/lib/watir/element_collection.rb:47:in `build'
/watirs/watir-7.1.0/lib/watir/element_collection.rb:18:in `initialize'
/watirs/watir-7.1.0/lib/watir/container.rb:28:in `new'
/watirs/watir-7.1.0/lib/watir/container.rb:28:in `elements'

Related

Scroll automation appium

I'm doing an automation in Ruby and Appium on mobile, and I need to access a card that is out of range, and I'm using all possible methods and it just happens error ... Does anyone have a solution? I need to down the Recycler view all the way down
Edit:
code
class AuditoriaController
def initialize
#util = UtilMethods.new
#objects = PageAuditoria.new
main = MenuPrincipal.new
#menus = main.menus
#tela_principal = main.tela_principal
#objects_auditoria = #objects.tela_auditoria
end
def acessar_auditoria
data_sync = find_element(:xpath, #tela_principal[:msg_sincronizacao]).text
data_sync.slice!("Última atualização: ")
t = Time.now
while(t.strftime("%d/%m/%Y %H:%M:%S") != data_sync)
btn_sync = find_element(:xpath, #tela_principal[:view_sync])
btn_sync.click
break;
end
list = find_element(:xpath, #tela_principal[:lista])
list.scrollIntoView()
#menu_auditoria = find_element(:xpath, #menus[:menu_auditoria])
#if(menu_auditoria)
# #util.logger("ACHEI AUDITORIA")
#end
end
First get the element(that is card)
Then add the below code
browser.execute_script('arguments[0].scrollIntoView();', card);
Hope it will help you. If you need any more information just comment here
EDIT:
JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.execute_script('arguments[0].scrollIntoView();', card)
Here driver is the driver you are using for automation that is appium driver. Here card is the element that you want to get and scroll upto it.
In ruby:
def scroll_to_element(element)
#driverAppium.execute_script('mobile: scroll', name: element.name)
end
This will scroll to the element which you are looking for even if it is offscreen. You should be able to modify for use with Java.
scroll_to_exact(elementname) should do the job for you! This will scroll to through the elements until the specified is visible. This is in Appium_lib for ruby

How to select an inline element using watir in ruby

I want the text "HEATMAPS" to be clicked after the webpage is opened. I have tried number of click methods, including recognizing as hyperlink, as text, as using xpath etc. None of them worked. I feel, I am either misunderstanding the links, as to be a hyperlink or choosing a wrong xpath.
Link of the web page
PFB the code below
require 'watir-webdriver'
require 'watir-ng'
WatirNg.patch!
WatirNg.register(:ng_scope).patch!
browser = Watir::Browser.new
browser.goto 'http://app.vwo.com/#/campaign/108/summary? token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0='
lin = browser.link :text=> 'HEATMAPS'
lin.exist?
lin.click
Can someone please guide me on this, as to how I can make that link with the text "HEATMAPS" in the page get clicked.
The error i get:
`This code has slept for the duration of the default timeout waiting for an Element to exist. If the test is still passing, consider using Element#exists? instead of rescuing UnknownObjectException
C:/Ruby22/lib/ruby/gems/2.2.0/gems/watir-6.1.0/lib/watir/elements/element.rb:507:in `rescue in wait_for_exists': timed out after 30 seconds, waiting for {:text=>"HEATMAPS", :tag_name=>"a"} to be located (Watir::Exception::UnknownObjectException)
from C:/Ruby22/lib/ruby/gems/2.2.0/gems/watir-6.1.0/lib/watir/elements/element.rb:497:in `wait_for_exists'
from C:/Ruby22/lib/ruby/gems/2.2.0/gems/watir-6.1.0/lib/watir/elements/element.rb:515:in `wait_for_present'
from C:/Ruby22/lib/ruby/gems/2.2.0/gems/watir-6.1.0/lib/watir/elements/element.rb:533:in `wait_for_enabled'
from C:/Ruby22/lib/ruby/gems/2.2.0/gems/watir-6.1.0/lib/watir/elements/element.rb:656:in `element_call'
from C:/Ruby22/lib/ruby/gems/2.2.0/gems/watir-6.1.0/lib/watir/elements/element.rb:114:in `click'
from C:/Users/Mrityunjeyan/Documents/GitHub/Simpleprograms/webautomation.rb:10:in `<main>'`
This would display me the inner_html text but still wouldnt click
lin = browser.span(:class => 'ng-scope').inner_html
puts lin
The problem is that the link's text is not "HEATMAPS". It is actually "Heatmaps". The text locator is case-sensitive, which means you need:
lin = browser.link :text=> 'Heatmaps'
You can see this if you inspect the HTML:
<a ui-sref="campaign.heatmap-clickmap" href="#/analyze/analysis/108/heatmaps?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D">
<!-- boIf: isAnalyticsCampaign -->
<span bo-if="isAnalyticsCampaign" class="ng-scope">Heatmaps</span>
<!-- boIf: !isAnalyticsCampaign -->
</a>
It only looks like "HEATMAPS" due to the styling. One of the styles includes a text-transform: uppercase; which visually capitalizes the text. Watir does not interpret the styles, so only knows that the text node is "Heatmaps".
Once you have identified the link, clicking still has a problem:
lin.click
#=> Selenium::WebDriver::Error::UnknownError:
#=> unknown error: Element <a ui-sref="analyze.heatmaps" ng-class="{selected: (locationContains('analyze', 'heatmap') && !locationContains('analyze', '/analysis') && state.current.name !== 'campaign.heatmap-clickmap') || (isAnalyzeHeatmapEnabled && (isAnalyzeDeprecatedHeatmapView || locationContains('analyze', '/analysis', 'heatmaps')) )}" data-qa="nav-main-analyze-heatmap" href="#/analyze/heatmap">...</a>
#=> is not clickable at point (90, 121).
#=> Other element would receive the click: <div ng-show="!isCROSetupView" class="">...</div>
The link being located is actually the one in the left menu, which is disabled, rather than the top menu. You need to scope the link locator to just the top menu. Using the parent ul element appears to be sufficient:
lin = browser.ul(class: 'page-nav').link(text: 'Heatmaps')
lin.click
To see the click complete, you might want to tell Chrome not to close at the end of the script. This is done by opening the browser using the following option:
caps = Selenium::WebDriver::Remote::Capabilities.chrome("chromeOptions" => {'detach' => true})
browser = Watir::Browser.new :chrome, desired_capabilities: caps
Your target link doesn't have any text. I confirmed with nokogiri that the text of the <a> tag includes the text inside the child <span> tag, and it turns out Watir works the same way.
If you use a Chrome browser, you can select:
View > Developer > Developer tools
Then you can select an element on the page, and the corresponding section in the html will be highlighted. Here is what the html looks like:
<a ui-sref="campaign.heatmap-clickmap"
href="#/analyze/analysis/108/heatmaps?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D">
<!-- boIf: isAnalyticsCampaign -->
<span bo-if="isAnalyticsCampaign" class="ng-scope">Heatmaps</span>
<!-- boIf: !isAnalyticsCampaign --> </a>
The basic structure is:
<a><span>Heatmaps</span></a>
Chrome even has a feature where you can right click on an element and get its xpath, which you can use with Watir.
However, when I execute my program to go to that page, I see Chrome launch, and then I'm presented with a login page rather than the page that was presented to me at your link.
Arghhh...what a colossal waste of time. I thought Watir must be broken. With the proper url, I can get the link using several different techniques:
1)
require 'watir'
require 'watir-ng' #<=NOTE THIS
WatirNg.register(:'bo_if').patch! #<= NOTE THIS
br = Watir::Browser.new :chrome
br.goto 'http://app.vwo.com/#/analyze/analysis/108/summary?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D'
target_link = br.span(
class: 'ng-scope',
'bo_if': 'isAnalyticsCampaign',
).parent #<= NOTE THIS
puts target_link.text #HEATMAPS
puts target_link.href #http://app.vwo.com/#/analyze/analysis/1.....
2)
require 'watir'
require 'watir-ng' #<=NOTE THIS
WatirNg.register(:ui_sref).patch! #<=NOTE THIS
br = Watir::Browser.new :chrome
br.goto 'http://app.vwo.com/#/analyze/analysis/108/summary?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D'
target_link = br.link(
ui_sref: 'campaign.heatmap-clickmap',
href: '#/analyze/analysis/108/heatmaps?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D'
)
puts target_link.text #HEATMAPS
puts target_link.href #http://app.vwo.com/#/analyze/analysis/108...
3) Getting the xpath by right clicking on the link in Chrome:
require 'watir'
br = Watir::Browser.new :chrome
br.goto 'http://app.vwo.com/#/analyze/analysis/108/summary?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D'
target_link = br.link(
xpath: '//*[#id="main-container"]/ul/li[3]/a'
)
puts target_link.text #HEATMAPS
puts target_link.href #http://app.vwo.com/#/analyze/analysis/108....
xpath can handle any tag or attribute name, unlike Watir, so it seems like a good tool in that regard.
4) A less brittle xpath:
require 'watir'
br = Watir::Browser.new :chrome
br.goto 'http://app.vwo.com/#/analyze/analysis/108/summary?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D'
a_href = "#/analyze/analysis/108/heatmaps?token=eyJhY2NvdW50X2lkIjoxNTA3MzQsImV4cGVyaW1lbnRfaWQiOjEwOCwiY3JlYXRlZF9vbiI6MTQ0NDgxMjQ4MSwidHlwZSI6ImNhbXBhaWduIiwidmVyc2lvbiI6MSwiaGFzaCI6IjJmZjk3OTVjZTgwNmFmZjJiOTI5NDczMTc5YTBlODQxIn0%3D"
target_link = br.link(
xpath: %Q{ //a[#href="#{a_href}"] } +
'[#ui-sref="campaign.heatmap-clickmap"]' +
'[child::span[#class="ng-scope"][#bo-if="isAnalyticsCampaign"][text()="Heatmaps"]]'
)
The xpath looks for an <a> tag with two attributes:
//a[#href="blah"][#ui-sref="bleh"]
which has a child <span> (i.e. a direct child) with three attributes:
[child::span[#class="blih"][#bo-if="bloh"][text()="Heatmaps"]]
After I programmatically click the link:
target_link.click
Watir goes to the next page.

Selenium webdriver ruby: Unable to read text value some times

Scenario:
There is a text in my webpage
I am using xpath to locate it
myxpath=//table[#id='table44']/tbody/tr[1]/td[1]/span[2]
I am trying to get it value using
value=driver.find_element(:xpath, myxpath).text
But problem is :sometimes it gets value & sometime it doesn't
& i am not able to understand the cause of this problem
Any alternative that i can try ?
You can write using explicit wait.
my_xpath = "//table[#id='table44']/tbody/tr[1]/td[1]/span[2]"
wait = Selenium::WebDriver::Wait.new(:timeout => 10)
element = wait.until { driver.find_element(:xpath, my_xpath) }
puts element.text

Crawl data using ruby mechanize

I am crawling data from http://www.mca.gov.in/DCAPortalWeb/dca/MyMCALogin.do?method=setDefaultProperty&mode=53
Below is the code I have tried :
uri = "http://www.mca.gov.in/DCAPortalWeb/dca/MyMCALogin.do?method=setDefaultProperty&mode=53"
#html, html_content = #mobj.get_data(uri)
agent = Mechanize.new
html_page = agent.get uri
html_form = html_page.form
html_form.radiobuttons_with(:name => 'search',:value => '2')[0].check
html_form.submit
puts html_page.content
Error :
var/lib/gems/1.9.1/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:308:in `fetch': 500 => Net::HTTPInternalServerError for http://www.mca.gov.in/DCAPortalWeb/dca/ProsecutionDetailsSRAction.do -- unhandled response (Mechanize::ResponseCodeError)
from /var/lib/gems/1.9.1/gems/mechanize-2.7.3/lib/mechanize.rb:1281:in `post_form'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.3/lib/mechanize.rb:548:in `submit'
from /var/lib/gems/1.9.1/gems/mechanize-2.7.3/lib/mechanize/form.rb:223:in `submit'
from ministry_corp_aff.rb:32:in `start'
from ministry_corp_aff.rb:52:in `<main>'
If I manually click on the 3rd radio button and then submit it, I get a .zip file. I was trying to fetch data from the .xls file from that zip..
The radio button has an onclick even handler that triggers the execution of some javascript. In addition, clicking on the Submit <a> tag also causes some javascript to execute. That javascript probably sets some values that are returned with the form, which the server examines.
Mechanize cannot execute the javascript. You need selenium webdriver for that.

I am trying to use Curl::Easy.http_put but have some issues with the data argument

I'm struggling with a ruby script to upload some pictures to moodstocks using their http interface
here is the code that I have so far
curb = Curl::Easy.new
curb.http_auth_types = :digest
curb.username = MS_API
curb.password = MS_SECRET
curb.multipart_form_post = true
Dir.foreach(images_directory) do |image|
if image.include? '.jpg'
path = images_directory + image
filename = File.basename(path, File.extname(path))
puts "Upload #{path} with id #{filename}"
raw_url = 'http://api.moodstocks.com/v2/ref/' + filename
encoded_url = URI.parse URI.encode raw_url
curb.url = encoded_url
curb.http_put(Curl::PostField.file('image_file', path))
end
end
and this is the error that I get
/Library/Ruby/Gems/2.0.0/gems/curb-0.8.5/lib/curl/easy.rb:57:in `add': no implicit conversion of nil into String (TypeError)
from /Library/Ruby/Gems/2.0.0/gems/curb-0.8.5/lib/curl/easy.rb:57:in `perform'
from upload_moodstocks.rb:37:in `http_put'
from upload_moodstocks.rb:37:in `block in <main>'
from upload_moodstocks.rb:22:in `foreach'
from upload_moodstocks.rb:22:in `<main>'
I think the problem is in how I give the argument to the http_put method, but I have tried to look for some examples of Curl::Easy.http_put and have found nothing so far.
Could anyone point me to some documentation regarding it or help me out on this.
Thank you in advance
There are several problems here:
1. URI::HTTP instead of String
First, the TypeError you encounter comes from the fact that you pass a URI::HTTP instance (encoded_url) as curb.url instead of a plain Ruby string.
You may want to use encoded_url.to_s, but the question is why do you do this parse/encode here?
2. PUT w/ multipart/form-data
The second problem is related to curb. At the time of writing (v0.8.5) curb does NOT support the ability to perform a HTTP PUT request with multipart/form-data encoding.
If you refer to the source code you can see that:
the multipart_form_post setting is only used for POST requests,
the put_data setter does not support Curl::PostField-s
To solve your problem you need an HTTP client library that can combine Digest Authentication, multipart/form-data and HTTP PUT.
In Ruby you can use rufus-verbs, but you will need to use rest-client to build the multipart body.
There is also HTTParty but it has issues with Digest Auth.
That is why I greatly recommend to go ahead with Python and use Requests:
import requests
from requests.auth import HTTPDigestAuth
import os
MS_API_KEY = "kEy"
MS_API_SECRET = "s3cr3t"
filename = "sample.jpg"
with open(filename, "r") as f:
base = os.path.basename(filename)
uid = os.path.splitext(base)[0]
r = requests.put(
"http://api.moodstocks.com/v2/ref/%s" % uid,
auth = HTTPDigestAuth(MS_API_KEY, MS_API_SECRET),
files = {"image_file": (base, f.read())}
)
print(r.status_code)

Resources