Can anyone point me to a current example of how to use HTMLUnit to interact with a webpage (submit a form, click, etc) along with the current jars that work with the example? I donwloaded all the jars on the htmlunit page and all its dependent jars, and I keep getting NoSuchMethod errors, so I'm guessing I have mismatched jars.
Hi if you're having issues to set up HtmlUnit this answer might help How to setup HtmlUnit in an Eclipse project?
Regarding form submission and clicking, this is the example taken from the HtmlUnit documenation itself:
public void submittingForm() throws Exception {
final WebClient webClient = new WebClient();
// Get the first page
final HtmlPage page1 = webClient.getPage("http://some_url");
// Get the form that we are dealing with and within that form,
// find the submit button and the field that we want to change.
HtmlForm form = page1.getFormByName("myform");
HtmlSubmitInput button = form.getInputByName("submitbutton");
HtmlTextInput textField = form.getInputByName("userid");
// Change the value of the text field
textField.setValueAttribute("root");
// Now submit the form by clicking the button and get back the second page.
final HtmlPage page2 = button.click();
webClient.closeAllWindows();
}
HTMLUnit is integrated in Selenium WebDriver. http://www.seleniumhq.org/projects/webdriver/
the best advice that I can give you is, since you are working with Java, make it a Maven Project and add the dependencies for selenium.webdriver. Building the project as Maven avoids having common mistakes like saving the jars in a wrong directory or forgetting 1 jar
Related
I'm struggling to make reCAPTCHA work in my Orbeon form.
I'm using ORBEON 2018.2.3.201905172253 PE within SAP Commerce (Hybris).
After submitting the form, I get the error "Please check your answer to the CAPTCHA challenge" even though the captcha is correct.
Created a common Form and added reCAPTCHA as per official Orbeon documentation: https://doc.orbeon.com/form-runner/component/captcha
Added my private and public keys to the properties file
Added the reCAPTCHA property to my form:
<property
as="xs:string"
name="oxf.fr.detail.captcha.yforms.myForm"
value="reCAPTCHA"/>
This is the process executed when clicking the SUBMIT button:
<property as="xs:string" name="oxf.fr.detail.process.save-final.yforms.myForm">
validate-all
then save
then send(
uri = "https://myurl",
method = "PUT"
)
then navigate(uri = "https://myurl")
</property>
I see the reCAPTCHA at the bottom of my form as expected
When I click SUBMIT after filling up the form and completing the captcha successfully, I see a popup saying "There are validation errors. Please retry once all fields have been properly filled-out."
And the form displays this error for the captcha field:
"Please check your answer to the CAPTCHA challenge."
So I'm clearly missing something. I know there are some configurations needed for the fr-verify-done and fr-verify-error to be added, but the Orbeon documentation does not specify how to configure those through PROPERTIES. They only say how to add those event listeners while adding the captcha component by hand (editing the source code of the form through Form Builder).
Things that I tried and didn't work:
Dispatching an fr-verify event when submitting:
xf:dispatch(name = "fr-verify", targetid = "captcha")
then validate-all
then save
then send(
uri = "https://myurl",
method = "PUT"
)
then navigate(uri = "https://myurl")
Adding the reCAPTCHA component to the form source code as per Orbeon documentation:
<fr:recaptcha id="my-captcha">
<xf:send ev:event="fr-verify-done" submission="save-submission"/>
<xf:action ev:event="fr-verify-error">
<xf:toggle case="failure-case"/>
<xf:dispatch target="my-captcha" name="fr-reload"/>
</xf:action>
</fr:recaptcha>
When I click SAVE after adding the excerpt above, the Form Builder crashes. Same happens if I try to add the same excerpt to a form through the online demo form builder provided by Orbeon : https://demo.orbeon.com/demo/fr/orbeon/builder/new
That is super weird since I'm adding code provided by them using the online last version of the form builder, and it still crashes when trying to save it. So that makes me think the problem I have is not related to my platform, but with Orbeon itself.
This is the error I get in the online demo for Form Builder after adding the recaptcha excerpt to the source code of the form: "Cannot read property 'call' of undefined'
There're no relevant LOGS that I can provide, I activated debug level logs for Orbeon and I'm not getting any useful info.
I would really appreciate if someone can guide me through with either:
- Configuring the fr-verify events without changing the source code of the form using Form Builder, OR
- Adding the reCAPTCHA component through Form Builder successfully without making it crash, OR
- Anything you can come up with to make my form with captcha work
Thanks in advance,
David
SAP Hybris 6.6 includes Orbeon Forms 2017.1.1, which uses Google's reCAPTCHA 1, which Google doesn't support anymore. Your options are either:
To upgrade to a version of SAP Hybris that includes Orbeon Forms 2018.1 or 2017.2.2 or newer (those version use Google's reCAPTCHA 2).
To use the SimpleCaptcha instead of reCAPTCHA.
Have tried using Selenium after reading some tutorial for web scraping ..
The aim is to web/screen scrape a page that loads the required data after an Ajax call when (this ajax call made after Initial page load)..
The second aim is to run Selenium code in the background (not opening any browser) to allow loading the page (including the Ajax call) , retrieve the Final HTML and perform required processing locally ..
the code till now is as follows (code tutorial from http://www.geekonweb.com/net/web-page-scraping-using-selenium-and-net/)
public ActionResult Index()
{
//--
//Below path should contain IEDriverServer.exe
var chrome = new ChromeDriver(#"file path");
chrome.Url = #"<url>";
chrome.Navigate();
//extract the html
//var retval = ie.ExecuteScript("return document.body.outerHTML");
string result = chrome.PageSource;
return View();
}
currently have not been able to find a way to run Selenium Silently (without GUI). kindly assist if that can be done.
Secondly kindly tell that how can Selenium be told to wait for the Ajax call to finish and then retrieve the data.
regards,
Here is a question on how to wait until an element is present. This is done to wait for the AJAX.
Here is a question on weather it's possible to run selenium headless.
In the Java EE environment, I want to load an XML and XSL file, render the output to a browser window, and save the rendered page as PDF.
I would like to do it all programmatically. I was looking at HtmlUnit to use as a headless browser. That part seems to work, but does HtmlUnit have any API to invoke a "print" function or similar function to persisted the rendered output? I was thinking of some way to link it in with iText.
I would recommend trying out flying-saucer which in this case basically is a CSS/XHTML enabled iText wrapper.
I wrote a simple example below. Don't forget the necessary dependencies for HtmlUnit and flying-saucer.
//Set up a new WebClient using your favourite settings
WebClient webClient = new WebClient();
//Fetch page
HtmlPage page = webClient.getPage("url-to-target.resource");
//Set PDF target output file
String outputFile = "firstdoc.pdf";
OutputStream os = new FileOutputStream(outputFile);
//Set up flying-saucer IText based renderer
ITextRenderer renderer = new ITextRenderer();
//Create PDF
renderer.setDocumentFromString(page.asXml();
renderer.layout();
renderer.createPDF(os);
os.close();
If you're doing reporting in Java i highly recommend you to use Jasper Reports, it's relatively easy to use, you can design the report graphically and it can take XML as input and give you a report in many formats (including PDF).
I have an MVC3 app for which I want to implement claims support. My goal is as follows:
provide a SignIn link, which when clicked displays a popup window with username/password and Facebook/WindowsLive/Google etc. links
automatically redirect to my SignIn page when a protected controller is accessed e.g. /Order/Delete
I've set up the application and providers in AppFabricLabs.com and included the STS in my project. I've also created an implementation of IAuthorizationFilter so I can mark my controllers as [WifAuth] and successfully get the OnAuthorization method called. I've implemented the use-case where the visitor has not been authenticated like this:
private static void AuthenticateUser(AuthorizationContext context)
{
var fam = FederatedAuthentication.WSFederationAuthenticationModule;
var signIn = new SignInRequestMessage(new Uri(fam.Issuer), fam.Realm);
context.Result = new RedirectResult(signIn.WriteQueryString());
}
and successfully get AppFabricLabs page with my Identity Provider choices (haven't figured out how to customise that page). When I log in my returnUrl gets called so I land in a controller method /Home/FederationResult, however the form posted to me contains only wa and wresult fields but I need wctx to know where to send the user... I haven't been able to figure out why.
the wresult is an XML document that contains (amongst a bzillion other things) the name and e-mail address of the user logging in but sadly does not contain the url to which the user was headed.
have I failed to configure something or am I just off base? thoughts anyone?
e
Just specify a Context for the SignInRequestMessage:
signIn.Context = HttpContext.Current.Request.RawUrl;
The wctx parameter is included in every request/response and also part of the form posted finally to your site.
HI, is possible to add request headers in WP7 WebBrowser control?
There is no way to do this. If you need to change headers you'll need to use HttpWebRequest.
You could intercept the requests from the WebBrowser control and make them yourself via HWR but this could get complicated very quickly.
No - I don't think there's any API hook available for this.
It's a similar problem to the "change the user agent" request discussed in Bring back mobile version of website in WebBrowser control for wp7?
Sorry to necro but the answers here are wrong. Headers can be added to a WebBrowser through the Navigate method.
WebBrowser.Navigate(YourURI, null, YourCustomHeaderString)
See this page: http://msdn.microsoft.com/en-us/library/windowsphone/develop/ff626636(v=vs.105).aspx
.
These headers will only apply to the first page navigated to through your code. If you want the headers to stay the same even when users click a link inside the web browser control, add this for the WebBrowser's navigating event:
private void browser_Navigating(object sender, NavigatingEventArgs e)
{
string url = e.Uri.ToString();
if(!url.Contains("YESHEADERS"))
{
e.Cancel = true;
string newUrl;
if(url.Contains("?"))
{
newUrl = url + "&YESHEADERS";
}
else
{
newUrl = url + "?YESHEADERS";
}
browser.Navigate(newUrl, null, "fore:" + Variables.GetForeground() + "")
}
}
Here's what that does:
We create an indicator, YESHEADERS, that tells us whether or not we have added custom headers.
When the WebBrowser tries to Navigate, we check whether or not the URL it is navigating to, e.Uri, contains YESHEADERS.
If it does, we've already added our headers. Take no action
If it does not, cancel the current navigation. Create a new URL equal to the old URL plus our indicator. We add YESHEADERS on to the new URL in it's query string. If you are not familiar with query strings that is fine, just know that they are extra strings on the URL that have no effect in our case. About Query Strings
Then, we navigate to the new URL, and add our custom headers.
In short, if we have our indicator YESHEADERS the web browser knows that we've added our custom headers, if we don't have YESHEADERS, than the web browser needs to add the headers.