I need to load a page from an HTML string, not from a server.
I use the method found in this answer :
String html = "<html><head><script>" +
"alert('hey 1'); " +
"function OnLoadEvent() { alert('hey 2'); }" +
"</script></head>" +
"<body onload='OnLoadEvent()'></body></html>"
URL url = new URL("http://www.example.com");
StringWebResponse response = new StringWebResponse(html, url);
WebClient client = new WebClient()
client.getOptions().setJavaScriptEnabled(true);
HtmlPage page = HTMLParser.parseHtml(response, client.getCurrentWindow());
I have also set an AlertHandler as follows:
client.setAlertHandler(new AlertHandler() {
#Override
public void handleAlert(Page arg0, String arg1) {
System.out.println("ALERT: " + arg1);
}
});
The example HTML above alerts "hey 1" only -- I would expect it to alert "hey 1" and "hey 2" -- this implies the body onload event is not firing.
Is there something extra I need to do to make the onload event fire?
I was missing a call to HtmlPage.initialize().
Related
I am building a simple Quarkus project.
I have a greeting resource class
public class GreetingResource {
String htmlString = "<!DOCTYPE html>\n" +
"<html lang=\"en\">\n" +
"<head>\n" +
" <meta charset=\"UTF-8\">\n" +
" <title>Title</title>\n" +
"</head>\n" +
"<body>\n" +
"Hello !\n" +
"</body>\n" +
"</html>";
#GET
#Produces(MediaType.TEXT_HTML)
public Response get() {
return Response.status(Response.Status.OK).entity(htmlString).build();
}
}
This code renders fine on a browser under localhost:8080/hello.
However, when I change the response status code to BAD_REQUEST, it just displays the raw htmlString.
is there a way so that it renders properly with a 400 request? Eventually, I want to render the error myself using quarkus qute
I'm new to HtmlUnit (using version 2.30). Working in Eclipse on a Mac. I'm trying to create a stock data scraper by logging onto my Ameritrade account and manipulating the watch lists I've created there. First login form leads to the two-step security page where the challenge question is asked. I don't know why/how the site knows that it wants to challenge my userid/password in the first place. Because it looks like a new browser?
But anyway I fill out the form on the second page with the answer to the challenge question and submit. Instead of taking me to my account's home page, it once again takes me to the two-step security page with the same challenge question. Here is the relevant code:
final int sleepMinSeconds = 1;
final int sleepRandomSeconds = 4;
final long javascriptTimeout = 10000;
System.out.println("HtmlUnitTest");
String applicationName = "Mozilla";
String applicationVersion = "5.0 (Windows NT 6.3; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0";
final String userAgent = applicationName + "/" + applicationVersion;
BrowserVersion browserVersion = new BrowserVersion.BrowserVersionBuilder(BrowserVersion.FIREFOX_52)
.setApplicationName(applicationName)
.setApplicationVersion(applicationVersion)
.setUserAgent(userAgent)
.build();
WebClient webClient = new WebClient(browserVersion);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(java.util.logging.Level.ALL);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(java.util.logging.Level.ALL);
webClient.setIncorrectnessListener(new com.gargoylesoftware.htmlunit.IncorrectnessListener() {
#Override public void notify(String arg0, Object arg1) {}
});
webClient.setJavaScriptErrorListener(new com.gargoylesoftware.htmlunit.javascript.JavaScriptErrorListener() {
#Override public void timeoutError(HtmlPage arg0, long arg1, long arg2) {}
#Override public void scriptException(final HtmlPage arg0, final com.gargoylesoftware.htmlunit.ScriptException arg1) {}
#Override public void malformedScriptURL(HtmlPage arg0, String arg1, java.net.MalformedURLException arg2) {}
#Override public void loadScriptError(HtmlPage arg0, java.net.URL arg1, Exception arg2) {}
});
webClient.setCssErrorHandler(new com.gargoylesoftware.htmlunit.SilentCssErrorHandler());
webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setDoNotTrackEnabled(true);
webClient.getOptions().setActiveXNative(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getOptions().setPrintContentOnFailingStatusCode(true);
webClient.getCookieManager().setCookiesEnabled(true);
webClient.getOptions().setDownloadImages(true);
String loginURL = "https://www.tdameritrade.com/home.page";
System.out.println("Connecting to " + loginURL + " (" + webClient.getBrowserVersion() + ")");
System.out.print(" Waiting to avoid being detected as a robot...");
Thread.sleep((long)(Math.random()*sleepRandomSeconds) * 1000);
System.out.print(" Done waiting.\n");
HtmlPage page = webClient.getPage(loginURL);
System.out.println("title text: " + page.getTitleText());
System.out.print(" \nWaiting for Javascript to complete...");
webClient.waitForBackgroundJavaScript(javascriptTimeout);
System.out.println("\nOK");
System.out.print(" Waiting to avoid being detected as a robot...");
Thread.sleep((long)(sleepMinSeconds + Math.random()*sleepRandomSeconds) * 1000);
System.out.print(" Done waiting.\n");
System.out.println("Logging in...");
HtmlForm form = page.getFormByName("form-login");
HtmlTextInput useridField = form.getInputByName("tbUsername");
HtmlPasswordInput passwordField = form.getInputByName("tbPassword");
useridField.type("<userid>");
passwordField.type("<password>");
HtmlButton button = form.getButtonByName("");
System.out.println("button value: " + button.getValueAttribute());
// Did this to make sure I had right button, which was unnamed.
// Value is "Log in", so I proceed.
HtmlPage page2 = button.click();
System.out.print(" \nWaiting for Javascript to complete...");
webClient.waitForBackgroundJavaScript(javascriptTimeout);
System.out.println("\nOK");
System.out.print(" Waiting to avoid being detected as a robot...");
Thread.sleep((long)(sleepMinSeconds + Math.random()*sleepRandomSeconds) * 1000);
System.out.print(" Done waiting.\n");
HtmlElement element = page2.getHtmlElementById("loginBlock");
HtmlForm form2 = element.getEnclosingForm();
HtmlPasswordInput challengeField = form2.getInputByName("challengeAnswer");
if(page2.asXml().contains("boss")) {
System.out.println("boss question...");
challengeField.type("<answer to boss question>");
}
else if(page2.asXml().contains("street")) {
System.out.println("street question...");
challengeField.type("<answer to street question>");
}
else {
System.out.println("What?");
}
HtmlCheckBoxInput checkBox = form2.getInputByName("rememberDevice");
checkBox.setChecked(true);
HtmlInput button2 = form2.getInputByName("mAction");
System.out.println("button2 value: " + button2.getValueAttribute());
// value here is "submit" - so I proceed
HtmlPage page3 = button2.click();
System.out.print(" \nWaiting for Javascript to complete...");
webClient.waitForBackgroundJavaScript(javascriptTimeout);
System.out.println("\nOK");
webClient.close();
In other words, page2 and page3 are the same, i.e., the two-step security page. I expected page3 to be my account's home page. (I confirmed this by writing them both out as XML to separate files.) I would appreciate any help I can get on this! Thanks!
OK, lets start with some comments to your code.
Not sure what you like to archive by using a different browser setup instead of using the build in default ones. There is nothing wrong with this but be aware that changing the browser setup will NOT have any effect on the browser behavior (e.g. the supported js functionality).
Second: if you are hunting for bugs/problems i fear it is a bad idea to disable all the listeners. It might be worth to have this listener output...
Regarding all the options: Why not starting with the default setup, it is really close to real browsers.
Now some words about the login process:
It will be a great help to try to understand the login process of the real application. All this 'modern' web applications are doing a lot of strange things (async/javascript) to simulate a rich ui on the not that rich platform.Tools like Charles WebProxy are of real help to get an idea of the communication done behind the scenes.
One common problem with this HtmlPage page3 = button2.click(); API is that the click method returns the sync result of the click. If the button is one of this fancy Ajax buttons, this is usually the page of the button itself. You are already waiting for the Ajax stuff to finish but the page will not change if there is a ajax redirect to a new page. In this case you have to do something like this after the wait call.
// there is an ajax redirect that loads a new page into this window
page3 = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();
Hope that helps a bit...
I have a very simple Web API app. Just 2 methods that put (edit or insert data) - I'd like to just hard code basic authentication into the 2 methods without hugely complicating it.
What is the minimum code I need in order to require authentication the two methods - and then how do I use this code (the APIs are called from some C# code in an MVC app).
All of the examples I find online are hugely complicated - with filters or overly elaborate authentication.
My web API methods look something like this:
public IHttpActionResult Put([FromBody]ImportObject impobj)
{
// Import data here
}
And I call it like this:
public async static Task<string> Caller(ImportObject impobj)
{
string responseString = "NA";
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
// Post new object, get response
var response = await client.PutAsJsonAsync("myservice/api/meth", impobj);
// Get response content string
var resContent = response.Content.ReadAsStringAsync();
// Build user response string (just for testing)
responseString =
"<br />result: " + resContent.Result.ToString() +
"<br />status: " + resContent.Status.ToString() +
"<br />StatusCode: " + response.StatusCode + "<br />";
}
// Output response string
return responseString;
}
I just want to add basic auth to both ends - as it's so simple happy to just hard code it into both methods (this will also let me see easily how it works too).
thx.
Scenario
Lotus Domino form with a button that made an Ajax call to an Xpage, that do some stuff (read a properties file).
Framework: prototype.js
Button code:
var now = new Date()
var n = $H({
........
now: now.getTime()
});
var url = "/" + $F("path") + "/myxpages.xsp";
var myAjax = new Ajax.Request(
url,
{
method: 'post',
parameters: n.toQueryString(),
onComplete: function(response) {
ajaxResult = response.responseText;
}
});
Xpage
myxpages.xsp has this SSJS code on afterPageLoad event
var request = facesContext.getExternalContext().getRequest();
var response = facesContext.getExternalContext().getResponse();
response.setHeader("Expires", -1);
response.setHeader("Cache-Control", "no-cache");
com.org.MyGetProperties.readProperties(request,response);
MyGetProperties class
This class is deployed in WebContent/WEB-INF/classes
public class MyGetProperties {
static PrintWriter out = null;
public static synchronized void readProperties(HttpServletRequest request, HttpServletResponse response) throws Exception {
try {
*(DO SOME STUFF HERE)*
out = new PrintWriter(response.getWriter());
// return result
out.println("OK");
} catch (Exception e) {..}
}
}
Sometimes ajaxResult variable in ajax response call is empty, sometimes is "OK", as expected (seems to be something related to cache, but i think i've managed it correctly).
The behavior is different on different production server, i don't know if depends on server configurations.
Could be a PrintWriter problem?
Short answer: don't. Long answer: use the Ajax control. You put that on your page and your URL changes to myxpages.xsp/nameyougavetheajaxcontrolproperty
This way you can be sure not to run foul of any cached result or pending operation. It also has a property where you can specify a Java class directly. That class extends (need to Google that - answered it on SO before) which gives you direct access to request/response
Update:
You need to tell that you are done:
facesContext.responseComplete();
See my original post on XAgents, the revision and some thought on testing.
I'm trying to write this simple UI which should show a message with two parameters (name, email2send). But, when I run it, I get only this:
The content of the variable msg is not shown in the pannel, only the pannel title. What is the right way to do this?
// Show confirmation
function showMSG(name, email2Send) { // for ease of use I give the urls as parameters but you could define these urls in the function as well
var app = UiApp.createApplication().setHeight(60).setWidth(200);
app.setTitle("Msg send with sucess!");
var msg = "You request to " + email2Send + " a response from " + name;
app.add(app.createVerticalPanel().setTag(msg));
var doc = SpreadsheetApp.getActive();
doc.show(app);
}
THANKS in advance for any help!
The TAG of a widget is not visible, its purpose is actually to store string data in a way that is not visible.
To show a text use a label or an html widget.
example in your code :
function showMSG(name, email2Send) { // for ease of use I give the urls as parameters but you could define these urls in the function as well
var app = UiApp.createApplication().setHeight(60).setWidth(200);
app.setTitle("Msg send with sucess!");
var msg = "You request to " + email2Send + " a response from " + name;
app.add(app.createVerticalPanel().add(app.createLabel(msg)));
var doc = SpreadsheetApp.getActive();
doc.show(app);
}