Click() on HtmlArea and nothing changes, why? - htmlunit

I'm trying my hand at HtmlUnity and have ran into trouble when I try to click an area with javaScript.
Here is the code:
import java.io.IOException;
import java.net.MalformedURLException;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlArea;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlMap;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class ToPost {
/**
* #param args
* #throws IOException
* #throws MalformedURLException
* #throws FailingHttpStatusCodeException
*/
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
HtmlPage page;
final WebClient webClient = new WebClient();
page = webClient.getPage("http://www.hidrografico.pt/previsao-mares.php");
System.out.println(page.getTitleText());
HtmlPage pagePortoLeixoes = setPort(page, "362,64,440,90");
System.out.println("Are they the same? "+page.asXml().equals(pagePortoLeixoes.asXml()));
}
private static HtmlPage setPort(HtmlPage page, String coordinatesPort) throws IOException {
HtmlMap map = page.getHtmlElementById("FPMap1");
Iterable<HtmlElement> childAreas = map.getChildElements();
HtmlArea tempArea;
for (HtmlElement htmlElement : childAreas) {
tempArea = (HtmlArea) htmlElement;
if(tempArea.getCoordsAttribute().equals(coordinatesPort)){
System.out.println("Found Leixoes! --> "+ tempArea.asXml());
return tempArea.click();
}
}
return null;
}
}
I don't show it here but I double-check in my full code that I'm really not in the page I want.
What is happening? Why doesn't the click work?

HtmlUnit .click() often works poorly when "complex" javascript is involved.
http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/html/HtmlElement.html#click()
Simulates clicking on this element, returning the page in the window that has the focus after the element has been clicked. Note that the returned page may or may not be the same as the original page, depending on the type of element being clicked, the presence of JavaScript action listeners, etc
In this case, you'll have to find another way to catch the data.
What i did see is that using .rss links, it gives you direct links to cities ...
eg : http://www.hidrografico.pt/previsao-mares-aveiro.php
Another way would have been to forge a POST request (check for exemple with Httpfox which requests are done when you're stuck getting a page)

Related

Extracting information from XML-Files by URL-Source takes a lot of time

I would like to extract specific information from 108 Xml files. The general source is also a XML-File with further URLs as resources.
XML-Source
The static method getURL() extracts the URLs in order to set them as URL-paths within a for loop in the main method. The programm works, but it takes approx. 5 minutes to get the data from all files. Any ideas how to increase the performance?
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import org.jdom2.Document;
import org.jdom2.Element;
import org.jdom2.Namespace;
import org.jdom2.filter.Filters;
import org.jdom2.input.SAXBuilder;
import org.jdom2.xpath.XPathExpression;
import org.jdom2.xpath.XPathFactory;
public class XmlReader2 {
public static void main(String[] args) throws IOException {
for (int i = 0; i < getURL().size(); i++) {
URL url = new URL(getURL().get(i));
try {
Document doc = new SAXBuilder().build(url);
final String getDeath = String
.format("//ns:teiHeader/ns:profileDesc/ns:particDesc/ns:listPerson/ns:person/ns:death");
XPathExpression<Element> xpath = XPathFactory.instance().compile(getDeath, Filters.element(), null,
Namespace.getNamespace("ns", "http://www.tei-c.org/ns/1.0"));
String test;
for (Element elem : xpath.evaluate(doc)) {
test = elem.getValue();
if (elem.getAttributes().size() != 0) {
test = elem.getAttributes().get(0).getValue();
}
System.out.println(elem.getName() + ": " + test);
}
} catch (org.jdom2.JDOMException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public static List<String> getURL() throws IOException {
List<String> urlList = new ArrayList<>();
URL urlSource = new URL("http://www.steinheim-institut.de:80/cgi-bin/epidat?info=resources-mz1");
try {
Document doc = new SAXBuilder().build(urlSource);
final String getURL = String.format("/collection");
XPathExpression<Element> xpath = XPathFactory.instance().compile(getURL, Filters.element());
int i = 0;
for (Element elem : xpath.evaluate(doc)) {
while (i != elem.getChildren().size()) {
String url = elem.getChildren().get(i).getAttributes().get(1).getValue();
// System.out.println(url);
urlList.add(url);
i++;
}
}
} catch (org.jdom2.JDOMException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return urlList;
}
}
A delay of this order may be caused by fetching files from the web. Find a tool for monitoring HTTP requests issued from your machine to see what is going on. Look in particular for requests for common W3C files such as the XHTML DTD: because these files are requested so often, W3C deliberately injects a delay into the process to encourage people to use local copies of the files. If it turns out that this is the problem, there are various techniques you can use to access cached local copies.
Having said that, I'm puzzled by the logic of your code. The method getURL() appears to fetch and parse the document at http://www.steinheim-institut.de:80/cgi-bin/epidat?info=resources-mz1 every time it is called, and yet you are calling it within a loop, even using getURL().size() as your terminating condition.

second scenario in my cucumber feature file is not executing

I'm trying to learn Cucumber with selenium java . have written two scenario's , when i run my feature file which contains two scenarios , only scenario #1 is executing , for scenario #2 its throwing Java null pointer exception
Feature: POC of my framework works
Scenario: Login test
Given I navigate to the Bugzilla website
When I click on login
And I enter the values
Then I check to see if i was successfully loged in or not
Scenario: File a bug test
Given I navigate to the File a bug page
When I click on widgets
And I enter the bug details
Then Bug should be submited succefully
My step definition file :
package cucumber.features;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.Select;
import cucumber.api.java.en.And;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
public class StepDefinitions1 {
protected WebDriver driver;
protected String baseUrl;
// Scenario 1
#Given("^I navigate to the Bugzilla website$")
public void I_navigate_to_the_Bugzilla_website() throws Throwable {
driver = new FirefoxDriver();
baseUrl ="https://landfill.bugzilla.org/bugzilla-4.4-branch/index.cgi";
driver.get(baseUrl);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}
#When("^I click on login$")
public void I_click_on_login() throws Throwable {
driver.findElement(By.id("login_link_top")).click();
}
#And("^I enter the values$")
public void I_enter_the_values() throws Throwable {
driver.findElement(By.id("Bugzilla_login_top")).clear();
driver.findElement(By.id("Bugzilla_login_top")).sendKeys("jeevan.anekal#gmail.com");
driver.findElement(By.id("Bugzilla_password_top")).clear();
driver.findElement(By.id("Bugzilla_password_top")).sendKeys("testuser#123");
driver.findElement(By.id("log_in_top")).click();
}
#Then("^I check to see if i was successfully loged in or not$")
public void I_check_to_see_if_i_was_successfully_loged_in_or_not() throws Throwable {
System.out.println("Login Successfull");
}
// Scenario 2
#Given("^I navigate to the File a bug page$")
public void I_navigate_to_the_File_a_bug_page() throws Throwable {
driver.findElement(By.id("enter_bug")).click();
}
#When("^I click on widgets$")
public void I_click_on_widgets() throws Throwable {
driver.findElement(By.linkText("Widgets")).click();
}
#And("^I enter the bug details$")
public void I_enter_the_bug_details() throws Throwable {
new Select(driver.findElement(By.id("bug_severity"))).selectByVisibleText("trivial");
new Select(driver.findElement(By.id("cf_drop_down"))).selectByVisibleText("---");
new Select(driver.findElement(By.id("rep_platform"))).selectByVisibleText("Macintosh");
new Select(driver.findElement(By.id("op_sys"))).selectByVisibleText("Mac OS X 10.0");
driver.findElement(By.id("short_desc")).clear();
driver.findElement(By.id("short_desc")).sendKeys("OS crashed");
driver.findElement(By.id("comment")).clear();
driver.findElement(By.id("comment")).sendKeys("Os debugging issue");
}
#Then("^Bug should be submited succefully$")
public void Bug_should_be_submited_succefully() throws Throwable {
System.out.println("Bug submitted successfully");
}
}
Looks like you are not calling the browser and navigating to the page in scenario 2
You need do do something similar to this
#Given("^I navigate to the File a bug page$")
public void I_navigate_to_the_File_a_bug_page() throws Throwable {
driver = new FirefoxDriver();
baseUrl ="https://landfill.bugzilla.org/bugzilla-4.4-nch/index.cgi";
driver.get(baseUrl);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.findElement(By.id("enter_bug")).click();
enter code here
}
or use #Before hooks to initialize the browser
I think I kind get your problem here. I had encountered similar problem before.
So when running the feature file, either by clicking on the run button from the bar or right click in the feature file then run..., take a look at what it says when actually clicking it. Because if you hover over a certain scenario and click run, it will only run that scenario, not the whole feature file where there is more than 1 scenarios.

javafx java.lang.NullPointerException as executing event handler function

I am trying to learn some JavaFx these days. I set up a simple MVC and it works well until I click the button to invoke click envet. It throws java.lang.NullPointerException. I think the problem is that the instance variable "controller" is not initialized after GUI launched. But I do initialize it in the main method. Below is view class and what I did in the main method.
package javafxdemogui;
import javafx.application.Application;
import javafx.event.ActionEvent;
import javafx.event.EventHandler;
import javafx.geometry.Pos;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.TextArea;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.HBox;
import javafx.stage.Stage;
/**
*
* #author Jason
*/
public class DemoView extends Application {
private TextArea inputText;
private TextArea outputText;
private DemoController controller;
#Override
public void start(Stage primaryStage) {
BorderPane borderPane = new BorderPane();
this.inputText = new TextArea();
this.outputText = new TextArea();
this.inputText.setWrapText(true);
this.outputText.setWrapText(true);
this.outputText.setEditable(false);
borderPane.setTop(inputText);
HBox hbox = new HBox();
hbox.setSpacing(10);
Button resetBtn = new Button("reset");
Button copyInputBtn = new Button("copyInput");
hbox.getChildren().addAll(resetBtn, copyInputBtn);
hbox.setAlignment(Pos.CENTER);
borderPane.setCenter(hbox);
borderPane.setBottom(outputText);
resetBtn.setOnAction(new EventHandler<ActionEvent>() {
#Override
public void handle(ActionEvent event) {
controller.processResetEvent();
}
});
copyInputBtn.setOnAction(new EventHandler<ActionEvent>() {
#Override
public void handle(ActionEvent event) {
controller.processCopyEvent(inputText.getText());
}
});
Scene scene = new Scene(borderPane, 600, 400);
primaryStage.setTitle("JavaFXDemoGUI");
primaryStage.setScene(scene);
primaryStage.show();
}
public void registerObserver(DemoController controller) {
this.controller = controller;
}
/**
* Updates input display based on String provided as argument.
*
* #param input new value of input display
*/
public void updateInputDisplay(String input) {
this.inputText.setText(input);
}
/**
* Updates output display based on String provided as argument.
*
* #param output new value of output display
*/
public void updateOutputDisplay(String output) {
this.outputText.setText(output);
}
/**
* The main() method is ignored in correctly deployed JavaFX application.
* main() serves only as fallback in case the application can not be
* launched through deployment artifacts, e.g., in IDEs with limited FX
* support. NetBeans ignores main().
*
* #param args the command line arguments
*/
public void viewLaunch(String[] args) {
launch(args);
}
}
And What I did in the main method....
public static void main(String[] args) {
/*
* Create instances of the model, view, and controller objects, and
* initialize them; view needs to know about controller, and controller
* needs to know about model and view
*/
DemoModel model = new DemoModel();
DemoView view = new DemoView();
DemoController controller = new DemoController(model, view);
view.registerObserver(controller);
view.viewLaunch(args);
}
I advise placing the main() method in your application class and not doing anything in main but launching the application.
I haven't tried it, but I'd be willing to bet that when Application.launch is invoked, that it generates a new instance of your application class, so effectively all of the code you have written in main before the launch is ignored.
I know that for a while, for Java 8 the Oracle team were considering not invoking main on startup for launching a JavaFX application (not sure what the eventual outcome of that was though, perhaps they still invoke the main method).
What you really should do instead is handle all of your initialization in the init or start methods of your application. Also note (in JavaFX 2.2) that if you do stuff in init there are some restrictions on JavaFX objects which can be instantiated (as you are not yet on the JavaFX application thread), for example you can't create Tooltips or WebViews off the JavaFX application thread. For this reason most of the JavaFX applications you see end up creating their UI on the JavaFX application thread at the front of the start method.
Also, a good approach is to shelve any long running tasks which can be done off of the JavaFX application thread (such as reading a database into something like your DemoModel) off to a JavaFX concurrent task, that way you can get progress feedback and messages from that long running task back to your UI to update an initialization status (if your framework requires that level of sophistication).

how to get modified ajax content with htmlunit

I wanted to share with you how to retrieve the content of a html page which is changed by ajax.
The following code returns the old page.
public class Test {
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException, InterruptedException {
String url = "valid html page";
WebClient client = new WebClient(BrowserVersion.FIREFOX_17);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setRedirectEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(true);
client.getOptions().setCssEnabled(true);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.setAjaxController(new NicelyResynchronizingAjaxController());
HtmlPage page = client.getPage(url);
System.out.println(page.getWebResponse().getContentAsString());
}
}
What is happening here?
The answer is that page.getWebResponse() confers to the initial page.
In order to get to the updated the content we have to use the page variable itself
package utils;
import java.io.IOException;
import java.net.MalformedURLException;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class Test {
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException, InterruptedException {
String url = "valid html page";
WebClient client = new WebClient(BrowserVersion.FIREFOX_17);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setRedirectEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(true);
client.getOptions().setCssEnabled(true);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.setAjaxController(new NicelyResynchronizingAjaxController());
HtmlPage page = client.getPage(url);
System.out.println(page.asXml());
System.out.println(page.getWebResponse().getContentAsString());
}
}
I found the hint in the following link
http://htmlunit.10904.n7.nabble.com/Not-expected-result-code-from-htmlunit-td28275.html
Ahmed Ashour yahoo.com> writes:
Hi,You shouldn't use WebResponse, which is meant to get the actual content from
the server.You should use htmlPage.asText() or .asXml()Yours,Ahmed

Is there a working example of HTMLUnit log in and few clicks

Possibly showing Javascript Test Support
package htmlunitpoc;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlForm;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlSubmitInput;
/**
*
* #author
*/
public class HtmlPoc {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws Exception {
WebClient wc = new WebClient();
HtmlPage page = (HtmlPage) wc.getPage("http://www.google.com");
HtmlForm form = page.getFormByName("f");
HtmlSubmitInput button = (HtmlSubmitInput) form.getInputByName("btnG");
HtmlPage page2 = (HtmlPage) button.click();
}
}
but i get:
Nov 17, 2010 3:41:14 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'text/javascript'.
BUILD SUCCESSFUL (total time: 4 seconds)
Which does not help as it does not run as a Unit test, and shows Pass/Fail etc.
I am using netbeans 6.9.1
That's because you haven't written it as a unit test. HtmlUnit is somewhat mis-named, as it's not a test runner itself, instead it's a "headless browser" which allows you to interact with a website from Java as if you were a browser.
Try this instead:
import junit.framework.TestCase;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlForm;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlSubmitInput;
public class HtmlPoc
extends TestCase
{
public void test()
throws Exception
{
WebClient wc = new WebClient();
HtmlPage page = (HtmlPage) wc.getPage("http://www.google.com");
HtmlForm form = page.getFormByName("f");
HtmlSubmitInput button = (HtmlSubmitInput) form.getInputByName("btnG");
HtmlPage page2 = (HtmlPage) button.click();
assertNotNull( page2 ) ;
}
}

Resources