Very new with xpath and trying to learn how to use it.
I have a xpath in chrome browser that changes except for the last 2 or 3 characters.
I was told about the ends-with() function. However, I seem to be doing something wrong. Because it is not finding the id element.
I am using Auto-It with web-driver if that maters. This is what I have tried.
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, '//*[ends-with(#id,"_4")]') ;;enters dob
Tried these as well.
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//*substring(#id, string-length(#id) - string-length('_4') +1) = '_4'")
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//*[substring(#id, string-length(#id) - string-length('_4') + 1 ) = '_4']")
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//*[#id[substring(., string-length(.) - string-length('_4') + 1 ) = '_4']]")
I know the first part of my code is okay because these lines do work.
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, '//*[#id="sysmenu-searchbarinput"]')
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, '//*[#id="sysmenu-searchbarbutton"]')
Here is what my normal xpath would be if it was not dynamic.
//*[#id="in972_4"]
<input id="in8nM_4" type="text" spellcheck="false" style="box-sizing:border-box;width:100%;vertical-align:top;color:inherit;" name="in8nM_14" class="navitem s_92" data-evt="Fi Fo" data-evh-fi="Pweb.input.inputTextFocusInWrapper" data-evh-fo="Pweb.input.inputTextFocusOutWrapper" data-evt-props="["altStyle"]" data-befieldchanged-url="./0000000000016jz9v3.mthd" aria-labelledby="y8nM_1u">
As you are trying to do, this XPath could work:
//*[substring(#id, string-length(#id) - string-length('_4') + 1 ) = '_4']
Even better to put the predicate on the #id because you are using it multiple times. Like this:
//*[#id[substring(., string-length(.) - string-length('_4') + 1 ) = '_4']]
Related
I can't find how to write this query in rom-sql. Is it possible to add plain sql to where?
It looks for announcements which do not intersect with requests.
announcements.when and requests.when are tstzrange columns in postgres.
SELECT "announcements".* FROM "announcements" WHERE (
(SELECT COUNT(requests.id)
FROM requests
WHERE requests.id IN (1,2,3) AND
(TSTZRANGE(lower(requests.when) - INTERVAL '1 HOUR', upper(requests.when) + INTERVAL '1 HOUR', '()') &&
TSTZRANGE(lower(announcements.when)), upper(announcements.when), '()') AND
requests.user_id = 42
) = 0
)
Check out the 'Advance Postgres support' section of the docs, http://api.rom-rb.org/rom-sql/ROM/SQL/Postgres/Types looks like there is support TsTzRange range('tstzrange', range_read_type(:tstzrange))
I found a solution to my problem.
Not sure if it's the right way in rom-sql.
I'll accept better answer if any appear.
Until that, here's my solution - Sequel.lit
In my case I defined a method in Announcements relation:
def available(user_id)
request_ids = requests.by_user_id(user_id).confirmed.pluck(:id)
# +- 1 hour around request is not available
# last argument '()' means "do not include boundaries"
request_when = "TSTZRANGE(
lower(requests.when) - INTERVAL '1 HOUR',
upper(requests.when) + INTERVAL '1 HOUR',
'()'
)"
announcement_when = "TSTZRANGE(lower(announcements.when), upper(announcements.when), '()')"
where(Sequel.lit(
"(SELECT COUNT(requests.id)
FROM requests
WHERE requests.id IN (:request_ids) AND
#{request_when} && #{announcement_when} AND
requests.user_id = :user_id) = 0",
user_id: user_id, request_ids: request_ids
))
end
I try to use scrapy xpath to scrape a page, but it seems it cannot capture the tag with predicates when I use a for loop,
# This package will contain the spiders of your Scrapy project
from cunyfirst.items import CunyfirstSectionItem
import scrapy
import json
class CunyfristsectionSpider(scrapy.Spider):
name = "cunyfirst-section-spider"
start_urls = ["file:///Users/haowang/Desktop/section.htm"]
def parse(self, response):
url = response.url
yield scrapy.Request(url, self.parse_page)
def parse_page(self, response):
n = -1
for section in response.xpath("//a[contains(#name,'MTG_CLASS_NBR')]"):
print(response.xpath("//a[#name ='MTG_CLASSNAME$10']/text()"))
n += 1
class_num = section.xpath('text()').extract_first()
# print(class_num)
classname = "MTG_CLASSNAME$" + str(n)
date = "MTG_DAYTIME$" + str(n)
instr = "MTG_INSTR$" + str(n)
print(classname)
class_name = response.xpath("//a[#name = classname]/text()")
I am looking for a tags with name as "MTG_CLASSNAME$" + str(n), with n being 0,1,2..., and I am getting empty output from my xpath query. Not sure why...
PS.
I am basically trying to scrape course and their info from https://hrsa.cunyfirst.cuny.edu/psc/cnyhcprd/GUEST/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL?FolderPath=PORTAL_ROOT_OBJECT.HC_CLASS_SEARCH_GBL&IsFolder=false&IgnoreParamTempl=FolderPath%252cIsFolder&PortalActualURL=https%3a%2f%2fhrsa.cunyfirst.cuny.edu%2fpsc%2fcnyhcprd%2fGUEST%2fHRMS%2fc%2fCOMMUNITY_ACCESS.CLASS_SEARCH.GBL&PortalContentURL=https%3a%2f%2fhrsa.cunyfirst.cuny.edu%2fpsc%2fcnyhcprd%2fGUEST%2fHRMS%2fc%2fCOMMUNITY_ACCESS.CLASS_SEARCH.GBL&PortalContentProvider=HRMS&PortalCRefLabel=Class%20Search&PortalRegistryName=GUEST&PortalServletURI=https%3a%2f%2fhome.cunyfirst.cuny.edu%2fpsp%2fcnyepprd%2f&PortalURI=https%3a%2f%2fhome.cunyfirst.cuny.edu%2fpsc%2fcnyepprd%2f&PortalHostNode=ENTP&NoCrumbs=yes
with filter applied: Kingsborough CC, fall 18, BIO
Thanks!
Well... I've visited the website you put in the question description, I used element inspection and searched for "MTG_CLASSNAME" and I got 0 matches...
So I will give you some tools:
In your settings.py set that:
LOG_FILE = "log.txt"
LOG_STDOUT=True
then print the response body ( response.body ) where you should ( in the top of parse_page function in this case ) and search it in log.txt
Check there if there is what you are looking for.
If there is, use this https://www.freeformatter.com/xpath-tester.html (
or similar ) to check your xpath statement.
In addition, change for section in response.xpath("//a[contains(#name,'MTG_CLASS_NBR')]"):
by for section in response.xpath("//a[contains(#name,'MTG_CLASS_NBR')]").extract():, this will raise an error when you get the data that you are looking for.
So I have 13 binary values, which I call b_1... b_13, and based off these values I'd like to either set something I call indic_j to a previously defined string called inf_j, or nothing at all. Is it possible to do this without using 13 "If..." statements? What I have tried is below:
inf_1 = "aaaaa"
inf_2 = "bbbbb"
... and so on defining 13 infs, where aaaaa, bbbbb etc are names of columns in a table that I want to select.
FOR j = 1 to 13
IF b_j = 1 THEN "indic_"+j = inf_j + ",";
ELSE "indic_"+j = ""
ENDIF
ENDFOR
Also, before this I haven't introduced anything called indic_1, indic_2, etc. Is this needed?
My end goal is to transfer selected columns over to Excel. I've no problems doing this with predetermined columns, but I'm not sure how to allow for selected columns only.
I've tried using 13 IF statements, but I'm getting operator/operand type mismatch errors. My code currently is
IIF(b_1 = 1, indic_1 = inf_1 + ",",indic_1 = "")
IIF(b_2 = 1, indic_1 = inf_2 + ",",indic_1 = "")
IIF(b_3 = 1, indic_1 = inf_3 + ",",indic_1 = "")
and so on for 13 times, and then
SELECTIONRANG = indic_1 + indic_2 + indic_3 + indic_4 + indic_5 + indic_6 +indic_7 + indic_8 + indic_9 + indic_10 + indic_11 + indic_12 + indic_13
SELECTIONRANGE = LEFT(SELECTIONRANG,LEN(Selectionrang)-1)
You could create te variable name as a string and use it with &
As:
ind = 13
Var = "inf_" + ind
&Var ** inf_13
I am trying to create random Lat/Lngs inside a bounding box. The below code creates Lat/Lngs and they plot on the map... and while they are in the right country, they are not in the right "box"
my bounding box looks like this:
top_left = 51.514881121099904, -0.1293877362387679
bottom_right = 51.492151803709525, -0.09693947143827275
minLat = 51.514881121099904
maxLat = 51.492151803709525
minLng = -0.09693947143827275
maxLng = -0.1293877362387679
My code looks like this:
markers = []
10.times do
x = minLat + rand(maxLat - minLat)
y = minLng + rand(maxLng - minLng)
latLng = [x,y]
markers << latLng
end
markers.each do |x|
puts "#{x[0]} , #{x[1]}"
end
But its not outputting the right lat/Lngs. Is there something I am doing wrong? Am I looking at this the wrong way?
For example one run outputs:
51.51778835694538 , 0.8479098023328967
51.56834519895925 , 0.70003581382595
52.47933844269927 , 0.6167209565236026
51.52116824209034 , 0.8997051592766233
52.26830135905681 , 0.7514635122980192
52.143607260212896 , 0.21838042686322268
52.04649651836466 , 0.6683244476203801
52.057410769358576 , -0.08734906181439461
52.053408170846915 , 0.1600069328435182
51.8991829618331 , 0.3993135670912452
But clearly 52.26830135905681 is not between the min and max lat... so I am a little confused
Does rand() do some sort or rounding up. Can it work on such large floats?
How is rand() defined? Can't remember off the top of my head...
You can use ruby's Random class (http://www.ruby-doc.org/core-1.9.3/Random.html) and do something like this:
rng = Random.new(seed)
lat = minLat + rng.rand * (maxLat - minLat)
lng = minLng + rng.rand * (maxLng - minLng)
rand uses to_i.abs on it's argument. Better use rand(minLat..maxLat).
Also, in your code it seems the min and max values are mixed up.
maxLat = 51.514881121099904
minLat = 51.492151803709525
maxLng = -0.09693947143827275
minLng = -0.1293877362387679
markers = []
10.times do
markers << [rand(minLat..maxLat), rand(minLng..maxLng)]
end
markers.each do |lat, lng|
puts "#{lat} , #{lng}"
end
I have one date, let's say '2010-12-20' of a flight departure, and two times, for instance, '23:30' and '02:15'.
The problem: I need to get datetimes (yyyy-MM-dd HH:mm:ss, for example, 2010-12-17 14:38:32) of both of these dates, but I don't know the day of the second time (it can be the same day as departure, or the next one).
I am looking for the best solution in Ruby on Rails. In PHP would just use string splitting multiple times, but I believe, that Rails as usually, has a much more elegant way.
So, here is my pseudo code, which I want to turn into Ruby:
depart_time = '23:30'
arrive_time = '02:15'
depart_date = '2010-12-20'
arrive_date = (arrive.hour < depart.hour and arrive.hour < 5) ? depart_date + 1 : depart_date
# Final results
depart = depart_date + ' ' + depart_time
arrive = arrive_date + ' ' + arrive_time
I want to find the best way to implement this in Ruby on Rails, instead of just playing with strings.
This is just pure Ruby, nothing to do with Rails:
require 'date'
depart_time = DateTime.strptime '23:30', '%H:%M'
arrive_time = DateTime.strptime '02:15', '%H:%M'
arrive_date = depart_date = Date.parse( '2010-12-20' )
arrive_date += 1 if arrive_time.hour < depart_time.hour and arrive_time.hour < 5
puts "#{depart_date} #{depart_time.strftime '%H:%M'}",
"#{arrive_date} #{arrive_time.strftime '%H:%M'}"
#=> 2010-12-20 23:30
#=> 2010-12-21 02:15