Ruby script to select and submit options from selection lists - ruby
it's my first time posting a question here! I am trying to do a certain ruby script that travels through a website to get information, but I found that this website has a certain select list with submit buttons and I'd like to know how I can select a certain item from the list and issue a click on one of the submit buttons and then save the returned page to a variable.
The select list is as follows:
<table width="700" border="0" cellspacing="0" cellpadding="2" class="caixa">
<tr>
<td align="center">
<br>
<select name="CodEstab" size="9" class="inputtext">
<option value='0110' selected>0110 - Universidade dos Açores - Angra do Heroísmo<option value='0130'>0130 - Universidade dos Açores - Ponta Delgada<option value='0200'>0200 - Universidade do Algarve<option value='0201'>0201 - Universidade do Algarve - Faculdade de Ciências Humanas e Sociais<option value='0203'>0203 - Universidade do Algarve - Faculdade de Ciências e Tecnologia<option value='0204'>0204 - Universidade do Algarve - Faculdade de Economia<option value='0300'>0300 - Universidade de Aveiro<option value='0400'>0400 - Universidade da Beira Interior<option value='0501'>0501 - Universidade de Coimbra - Faculdade de Ciências e Tecnologia<option value='0502'>0502 - Universidade de Coimbra - Faculdade de Direito<option value='0503'>0503 - Universidade de Coimbra - Faculdade de Economia<option value='0504'>0504 - Universidade de Coimbra - Faculdade de Farmácia<option value='0505'>0505 - Universidade de Coimbra - Faculdade de Letras<option value='0506'>0506 - Universidade de Coimbra - Faculdade de Medicina<option value='0507'>0507 - Universidade de Coimbra - Faculdade de Psicologia e de Ciências da Educação<option value='0508'>0508 - Universidade de Coimbra - Faculdade de Ciências do Desporto e Educação Física<option value='0602'>0602 - Universidade de Évora - Escola de Ciências e Tecnologia<option value='0603'>0603 - Universidade de Évora - Escola de Artes<option value='0604'>0604 - Universidade de Évora - Escola de Ciências Sociais<option value='0700'>0700 - Universidade de Lisboa<option value='0701'>0701 - Universidade de Lisboa - Faculdade de Ciências<option value='0702'>0702 - Universidade de Lisboa - Faculdade de Direito<option value='0703'>0703 - Universidade de Lisboa - Faculdade de Farmácia<option value='0704'>0704 - Universidade de Lisboa - Faculdade de Letras<option value='0705'>0705 - Universidade de Lisboa - Faculdade de Medicina<option value='0708'>0708 - Universidade de Lisboa - Instituto de Geografia e Ordenamento do Território<option value='0709'>0709 - Universidade de Lisboa - Instituto de Educação<option value='0710'>0710 - Universidade de Lisboa - Faculdade de Psicologia<option value='0801'>0801 - Universidade Técnica de Lisboa - Faculdade de Medicina Veterinária<option value='0802'>0802 - Universidade Técnica de Lisboa - Faculdade de Arquitectura<option value='0803'>0803 - Universidade Técnica de Lisboa - Instituto Superior de Agronomia<option value='0804'>0804 - Universidade Técnica de Lisboa - Instituto Superior de Ciências Sociais e Políticas<option value='0805'>0805 - Universidade Técnica de Lisboa - Instituto Superior de Economia e Gestão<option value='0806'>0806 - Universidade Técnica de Lisboa - Faculdade de Motricidade Humana<option value='0807'>0807 - Universidade Técnica de Lisboa - Instituto Superior Técnico<option value='0808'>0808 - Universidade Técnica de Lisboa - Instituto Superior Técnico (instalações no Tagus Park)<option value='0901'>0901 - Universidade Nova de Lisboa - Faculdade de Ciências Médicas<option value='0902'>0902 - Universidade Nova de Lisboa - Faculdade de Ciências Sociais e Humanas<option value='0903'>0903 - Universidade Nova de Lisboa - Faculdade de Ciências e Tecnologia<option value='0904'>0904 - Universidade Nova de Lisboa - Faculdade de Economia<option value='0906'>0906 - Universidade Nova de Lisboa - Instituto Superior de Estatística e Gestão de Informação<option value='0911'>0911 - Universidade Nova de Lisboa - Faculdade de Direito<option value='1000'>1000 - Universidade do Minho<option value='1101'>1101 - Universidade do Porto - Faculdade de Ciências da Nutrição e Alimentação<option value='1102'>1102 - Universidade do Porto - Faculdade de Arquitectura<option value='1103'>1103 - Universidade do Porto - Faculdade de Ciências<option value='1104'>1104 - Universidade do Porto - Faculdade de Economia<option value='1105'>1105 - Universidade do Porto - Faculdade de Engenharia<option value='1106'>1106 - Universidade do Porto - Faculdade de Farmácia<option value='1107'>1107 - Universidade do Porto - Faculdade de Letras<option value='1108'>1108 - Universidade do Porto - Faculdade de Medicina<option value='1109'>1109 - Universidade do Porto - Faculdade de Psicologia e de Ciências da Educação<option value='1110'>1110 - Universidade do Porto - Instituto de Ciências Biomédicas Abel Salazar<option value='1111'>1111 - Universidade do Porto - Faculdade de Desporto<option value='1113'>1113 - Universidade do Porto - Faculdade de Medicina Dentária<option value='1114'>1114 - Universidade do Porto - Faculdade de Direito<option value='1201'>1201 - Universidade de Trás-os-Montes e Alto Douro - Escola de Ciências Agrárias e Veterinárias<option value='1202'>1202 - Universidade de Trás-os-Montes e Alto Douro - Escola de Ciências Humanas e Sociais<option value='1203'>1203 - Universidade de Trás-os-Montes e Alto Douro - Escola de Ciências e Tecnologia<option value='1204'>1204 - Universidade de Trás-os-Montes e Alto Douro - Escola de Ciências da Vida e do Ambiente<option value='1205'>1205 - Universidade de Trás-os-Montes e Alto Douro - Escola de Ciências Humanas e Sociais (Chaves)<option value='1300'>1300 - Universidade da Madeira<option value='5302'>5302 - Universidade de Lisboa - Faculdade de Belas-Artes<option value='5402'>5402 - Universidade do Porto - Faculdade de Belas-Artes<option value='6600'>6600 - Universidade de Lisboa - Faculdade de Medicina Dentária<option value='6800'>6800 - ISCTE - Instituto Universitário de Lisboa
</select>
<br><br>
</td>
</tr>
<tr>
<td align="right" class="tablebuttons">
<input type="hidden" name="CodR" value="11">
<input type="submit" name="listagem" value="Últimos Colocados" style="font-size: 10px;">
<input type="submit" name="listagem" value="Últimos Colocados por Etapa" style="font-size: 10px;">
<input type="submit" name="listagem" value="Lista de Colocados" style="font-size: 10px;">
<input type="submit" name="listagem" value="Lista Ordenada de Candidatos" style="font-size: 10px;">
</td>
</tr>
As you can see, there's a preselected option, but I want to select another option and submit with the value "Lista Ordenada de Candidatos". Is this possible?
Also, in the page that comes after this one, there's another selection list, but in that page I want to select each one of the options and submit, any ideas?
Use Mechanize to fetch the page.
Find the form you want on the page.
Find the <select> somewhere in the form.
Change the value of the <select>.
Submit the form and get the new page.
Repeat as you wish.
# encoding: UTF-8
require 'mechanize'
Mechanize.new.get('http://foo.com/bar') do |page|
# Find the form you want here, since you didn't show it to us
form = page.form_with(:action => "blah/blah")
form.field_with(:name=>"CodEstab").value = "Lista Ordenada de Candidatos"
next_page = form.submit
# Find the form on next_page, manipulate it, submit it
end
For more information, see the Mechanize Examples documentation, and the documentation for a Page, Form, SelectList, and MultiSelectList.
As shown in that last link, if you want to select many options in a multi-select, simply set the value of it to an array of strings. If you don't know the values and want to select them all:
# Assuming you are using Ruby 1.8.7 or later
myselect.value = myselect.options.map(&:value)
Related
How to fix this problem Ce site est inaccessibleharry.magento.com in magento 2?
enter image description here Ce site est inaccessibleharry.magento.com n'autorise pas la connexion. Voici quelques conseils : Vérifier la connexion Vérifier le proxy et le pare-feu ERR_CONNECTION_REFUSED
how get only specific fields from elsticsearch
i have elasticsearch indice i have filed name "titile":Rolex i want to write DSL to save
ratemax,ratemin fields as variable
can this be done here is my sample json
"vente": [
"62402c696a271a7d0ceeef2f"
],
"status": [
0
]
},
"ignored_field_values": {
"description.keyword": [
"ROLEX \nPRINCE BRANCARD \"EATON QUARTER CENTURY CLUB\" \nVERS 1930 \nMontre bracelet en or jaune 14K sur cuir. \nBOITIER : rectangulaire curvex cintré en or jaune 14K (belle oxydation de l'or). Fond gravé d'une dédicace: \"Presented to Fred Peartree to mark a quarter century of continuous service with the T. Eaton, 1916-1941.\" \nCADRAN : duodial, argenté deux tons, personnalisé Eaton, inscrit \"1/4 Century Club\" à la place des index. Aiguilles en acier bleui. \nMOUVEMENT: mécanique, certifié chronometre, Ultra-Prima. \nBRACELET : cuir avec boucle ardillon en or Rolex. \nBoîtier et mouvement signés Rolex. \nNumérotée 53207. \nDIM. 42 x 24 mm. \n \nCette montre fait partie d'une série personnalisée par un industriel canadien nommé Eaton. Cet entrepreneur avait l'habitude d'en faire cadeau à ses employés pour les remercier de leur fidélité. \nA 14K gold manual winding wristwatch by Rolex from the 30's."
]
}
}
Authors and affiliations in the YAML of RMarkdown
I know this question was already asked in the past in this forum (1, 2, 3). Before you mark this as duplicated, I tried all the answers with no success. Most of the questions were asked long ago, and some updates related with pandoc might affect nowadays results. The issue is that I am writing a scientific paper using RMarkdown, and would like to export the results either in HTML, PDF or Word files. More importantly is that there are 12 authors signing the papers. Some of the authors present more than one affiliation, and some authors presenting the same affiliation. My question is very clear: How can I edit the YAML to include all the authors with all the affiliations in the YAML in order to export to different formats (HTML, PDF, DOC)? I tried this YAML: --- title: "My title" author: - name: Mario Modesto-Mata^1,2^ email: paleomariomm#gmail.com - name: Christopher^1^ - name: Seaghán Mhartain^2^ - name: Rita Yuri Ynoue^1^ address: - code: 1 address: Instituto de Astronomía, Geofísica e Ciências Atmosféricas, Universidade de São Paulo - code: 2 address: Faculdade de Medicina, Universidade de São Paulo date: "1 October 2018" output: pdf_document: number_sections: yes toc: yes toc_depth: 4 word_document: toc: yes html_document: css: Scripts accesorios/estiloboton.css number_sections: yes theme: sandstone toc: yes toc_depth: 4 bibliography: references.bib csl: science.csl --- PDF output This is what I see when I export to PDF the .Rmd file: Instead of the authors, I see true and no affiliations. HTML output I see the authors and not all the affiliation numbers. The affiliations themselves remain hidden. DOCX output Neither names nor affiliations appear in the final Word file. My system I am using the latest RStudio version (Version 1.1.453), running on Linux Mint 19 Cinnamon. UPDATE: original example --- title: "My title" author: - Mario Modesto-Mata: email: paleomariomm#gmail.com institute: [cenieh, ucl1, ppex] correspondence: true - M. Christopher Dean: institute: [ucl2, nhm] - Yuliet Quintino: institute: ubu - Rebeca García-González: institute: ubu - Rodrigo S. Lacruz: institute: nyu - Timothy G. Bromage: institute: nyu - Cecilia García-Campos: institute: [cenieh, ucl1] - Marina Martínez de Pinillos: institute: cenieh - Laura Martín-Francés: institute: [bor, cenieh] - María Martinón-Torres: institute: [cenieh, ucl1] - Eudald Carbonell: institute: [iphes, urv] - Juan Luis Arsuaga: institute: [isciii, ucm] - José María Bermúdez de Castro: institute: [cenieh, ucl1] institute: - cenieh: Centro Nacional de Investigación sobre la Evolución Humana (CENIEH), Paseo Sierra de Atapuerca 3, 09002, Burgos, Spain - ucl1: Department of Anthropology, University College London, London, WC1H 0BW, UK - ucl2: Department of Cell and Developmental Biology, University College London, Gower Street, London, WC1E 6BT, UK - ubu: Laboratorio de Evolución Humana, Unierisdad de Burgos, Edificio I+D+i, Burgos, Spain - ppex: Equipo Primeros Pobladores de Extremadura, Casa de Cultura Rodríguez Moñino, Cáceres, Spain - nhm: Centre for Human Evolution Research (CHER), Department of Earth Sciences, Natural History Museum, London, SW7 5BD, UK - nyu: New York University - bor: De la Préhistoire à l'Actuel - Culture, Environnement et Anthropologie, University of Bordeaux, CNRS, MCC, PACE, UMR 5199 F_33615, Pessac Cedex, France - iphes: Institut Català de Paleoecologia Humana i Evolució Social (IPHES), Zona Educacional 4, Campus Sescelades, Edifici W3, Universitat Rovira i Virgili, Tarragona, Spain - urv: Àrea de Prehistòria, Universitat Rovira i Virgili, Avinguda de Catalunya 35, 43002, Tarragona, Spain - isciii: Centro mixto UCM-ISCIII de Evolución y Comportamiento humanos, Madrid, Spain - ucm: Departamento de Geodinámica, Estratigrafía y Paleontología, Facultad de Ciencias Geológicas, Universidad Complutense de Madrid, Spain date: "1 October 2018" output: pdf_document: number_sections: yes toc: yes toc_depth: 4 pandoc_args: - '--lua-filter=scholarly-metadata.lua' - '--lua-filter=author-info-blocks.lua' html_document: css: Scripts accesorios/estiloboton.css number_sections: yes theme: sandstone toc: yes toc_depth: 4 word_document: toc: yes pandoc_args: - '--lua-filter=scholarly-metadata.lua' - '--lua-filter=author-info-blocks.lua' bibliography: references.bib csl: science.csl ---
There is, to the best of my knowledge, no one-size-fits-it-all solution as of now. If the target was only PDF, I'd suggest rticles by RStudio. It's great. A solution which also works with docx is more difficult. One possibility is to use pandoc Lua filters. The repository collecting useful filters contains two filters which will help you: scholarly-metadata and author-info-blocks. (Disclosure: I wrote these.) Place the .lua files in your directory, change the YAML structure a bit, and instruct pandoc to run the filters: --- title: "My title" author: - Mario Modesto-Mata: email: paleomariomm#gmail.com institute: [astro, med] correspondence: true - name: Christopher institute: astro - name: Seaghán Mhartain institute: med - name: Rita Yuri Ynoue institute: astro institute: - astro: Instituto de Astronomía, Geofísica e Ciências Atmosféricas, Universidade de São Paulo - med: Faculdade de Medicina, Universidade de São Paulo date: "1 October 2018" output: word_document: toc: yes pandoc_args: - '--lua-filter=scholarly-metadata.lua' - '--lua-filter=author-info-blocks.lua' pdf_document: number_sections: yes toc: yes toc_depth: 4 pandoc_args: - '--lua-filter=scholarly-metadata.lua' - '--lua-filter=author-info-blocks.lua' --- This will be the PDF output: while this is what it looks like in Word: The affiliation and contact information is added to the body text, which is why the toc is displayed above it.
Postal (Zip) Code database / Address validation
I have CountryCode and PostalCode and I need to get the list of cities which match options (in real-time). I already checked out some databases and APIs as http://www.zippopotam.us or http://www.geonames.org/ but they are unusable because of lack of countries (I need countries worldwide with up-to-date information). Another option is GoogleAPI (too expensive in our case), GeoCodeNames and Mapanet (both expensive, I'm looking to spend 400$/year maximum). Npm packages did not help either, all I was able to found were US/Canada oriented or incomplete (I need worldwide, including middle-east and Africa).
Geocode.xyz provides this information. for example: https://geocode.xyz/27777?region=DE Will return: 20 Bökenbusch, Ganderlesee, Germany 27777 » Confidence Score: 0.5 Ganderkesee DE 27777 Germany Bremen DE 27777 Germany Schierbrok DE 27777 Germany Wildeshausen DE 27777 Germany Delmenhorst DE 27777 Germany Gandereksee DE 27777 Germany Ganderkesee - Urneburg DE 27777 Germany Garderkesee / Bookholzberg DE 27777 Germany Immer DE 27777 Germany Garderkesee DE 27777 Germany Ganderkese-Heide DE 27777 Germany Ganderkesee-Bookholzberg DE 27777 Germany Landkreis Oldenburg DE 27777 Germany Hohenböken DE 27777 Germany Ganderlesee DE 27777 Germany
Quick Union Functional Style [closed]
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am trying to mix study with pleasure by implementing the exercises from Sedgewick and Wayne's "Algorithm's 4th Edition" in scala (and, to the best of my know-how, in a functional style).
At the exercises 1.5.7 I blocked and hope that someone could help me out. So the task is to write the QuickUnion implementation. You may find Java implementation here.
I've written my scala version and it seems to make sense, but the console output is outright discouraging. I am creating an array of 625 initial components but ... the number of components at the end is .. below zero
Without further ado this is my implementation:
package ca.vgorcinschi.algorithms1_5_7.quickUnionUF
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
class QuickUnionUF(size: Int) extends AbstractUF(size) {
override val array = (0 to size).toArray
override def find(p: Int): Int = {
validate(p)
Stream.iterate(p)(p => array(p)).dropWhile { _ != array(p) }.head
}
override def connected(p: Int, q: Int): Boolean = find(p) == find(q)
override def union(p: Int, q: Int) = {
/*
* need to declare Futures outisde
* for-comprehension loop to run in parallel
*/
val fRootP = Future { find(p) }
val fRootQ = Future { find(q) }
for {
rootP <- fRootP
rootQ <- fRootQ
if (rootP != rootQ)
} {
array(rootP) = rootQ
count -= 1
}
}
}
The parent abstract class and the underlying trait are both in my github repository. For ease of reading I am copying below the Main class:
import scala.io.Source
import ca.vgorcinschi.algorithms1_5_7.quickUnionUF.QuickUnionUF
import scala.util.matching.Regex
object Main extends App {
val allLines = Source.fromFile("src/main/resources/mediumUF.txt").getLines().toList
//get & print the initial number of components
val size = allLines.head.trim.toInt
println(s"Le nombre initial de composants est $size")
val uf = new QuickUnionUF(size)
val Integer = new Regex("(-)?(\\d+)")
for (line <- allLines.tail) {
val p = Integer findFirstIn line
assert(p.isDefined)
val intP = p.get toInt
val q = Integer findFirstIn line.substring(p.get.length).replace("\\s+", "")
assert(q.isDefined)
val intQ = q.get.toInt
if (!uf.connected(intP, intQ)) {
uf.union(intP, intQ)
println(s"L`union fait sur $intP et $intQ")
println(s"Le nombre des composants est maintenant - ${uf.counter()}")
}
}
}
In addition these are the first and the last lines of the run output:
> runMain Main
[info] Compiling 1 Scala source to /home/vgorcinschi/workspace-scala/Algorithms/Chapter 1 Fundamentals/algorithms1_5_7/target/scala-2.12/classes...
[info] Running Main
Le nombre initial de composants est 625
L`union fait sur 528 et 503
Le nombre des composants est maintenant - 624
L`union fait sur 548 et 523
Le nombre des composants est maintenant - 623
L`union fait sur 389 et 414
Le nombre des composants est maintenant - 622
L`union fait sur 446 et 421
Le nombre des composants est maintenant - 621
L`union fait sur 552 et 553
Le nombre des composants est maintenant - 620
L`union fait sur 154 et 155
...
Le nombre des composants est maintenant - -67
L`union fait sur 463 et 464
Le nombre des composants est maintenant - -67
L`union fait sur 10 et 35
Le nombre des composants est maintenant - -67
L`union fait sur 619 et 594
Le nombre des composants est maintenant - -67
L`union fait sur 403 et 402
Le nombre des composants est maintenant - -67
I do fear that the shoe pinches somewhere around my futures or the stream based find, but I can't seem to figure it out...
I think I got it. The solution was odd because it makes me wonder how does the original solution even worked. Basically I just rewrote the counter() in the parent class to return a array.distinct.length , i.e. the distinct integers or the roots:
/*
* takes the initial number of components
* as its argument
*/
abstract class AbstractUF(var count: Int) extends UF{
override def counter() = array.distinct.length
val array: Array[Int]
...
}
Output:
Le nombre initial de composants est 625
L`union fait sur 528 et 503
Le nombre des composants est maintenant - 625
L`union fait sur 548 et 523
Le nombre des composants est maintenant - 624
L`union fait sur 389 et 414
Le nombre des composants est maintenant - 623
L`union fait sur 446 et 421
Le nombre des composants est maintenant - 622
L`union fait sur 552 et 553
Le nombre des composants est maintenant - 621
L`union fait sur 154 et 155
Le nombre des composants est maintenant - 620
...
Le nombre des composants est maintenant - 266
L`union fait sur 295 et 320
Le nombre des composants est maintenant - 266
L`union fait sur 160 et 159
Le nombre des composants est maintenant - 266
L`union fait sur 463 et 464
Le nombre des composants est maintenant - 266
L`union fait sur 10 et 35
Le nombre des composants est maintenant - 266
L`union fait sur 619 et 594
Le nombre des composants est maintenant - 265
L`union fait sur 403 et 402
Le nombre des composants est maintenant - 265