I'm trying to login to Facebook page using HtmlUnit and view its HTML content. I'm trying to fill up the login credentials through HtmlUnit but I don't see the session being carried when the submit button is clicked.
Couldnt find much content on htmlunit session management classes. I have also attached the code that I'm currently using to attempt this problem. Any help appreciated!
WebClient webClient = new WebClient();
HtmlPage page1 = webClient.getPage("https://www.facebook.com");
List<HtmlForm> listF = page1.getForms();
HtmlForm form = null;
for(int i=0; i<listF.size(); i++)
{
if(listF.get(i).getId().equals("login_form"))
{
form = listF.get(i);
}
}
HtmlTextInput uName = form.getInputByName("email");
HtmlPasswordInput passWord = form.getInputByName("pass");
HtmlSubmitInput button = form.getInputByValue("Log In");
uName.setValueAttribute(FACEBOOK_UNAME);
passWord.setValueAttribute(FACEBOOK_PASS);
HtmlPage page2 = button.click();
Found the answer. Just enabled cookies before starting to get webpages. It works.
Added the below piece of code
WebClient webClient = new WebClient();
CookieManager cookieMan = new CookieManager();
cookieMan = webClient.getCookieManager();
cookieMan.setCookiesEnabled(true);
Another advice is if you are trying to restore HTTP session in HtmlUnit, instead of using webClient.getCookieManager().addCookie(cookie);, use this instead :
webClient.addCookie("cookieName=cookieValue", URL, null);
Related
I’m using OIDC client in my Xamarin App(Code below) So when I make the call oidcClient.LogoutAsync browser opens, and while I’m awaiting the response back from the browser to the app if I need to minimise the app for some reason. The call then gets interrupted and the app stays on the browser, Never gives a response back, unless user manually closes the browser. Now I have no control over the browser from my app, Is there a way to avoid this. Maybe not open the browser altogether and logout silently. Please suggest a workaround.
private OidcClient InitializeAuthClient()
{
var options = new OidcClientOptions
{
Authority = Config.AuthorityURL,
ClientId = Config.ClientId,
Scope = Config.Scope,
RedirectUri = Config.RedirectUri,
ClientSecret = Config.ClientSecret,
PostLogoutRedirectUri = Config.RedirectUri,
Browser = new WebAuthenticatorBrowser(),
};
return new OidcClient(options);
}
OidcClient oidcClient = InitializeAuthClient();
LogoutResult logoutResult = await oidcClient.LogoutAsync(new LogoutRequest
{ BrowserDisplayMode = IdentityModel.OidcClient.Browser.DisplayMode.Hidden, IdTokenHint = authData.idToken });
I tried to login in some sites with HtmlUnit with this code:
HtmlPage page1 = webClient.getPage("https://somesites.com/");
HtmlForm form = (HtmlForm) page1.getElementById("FormLogin");
HtmlInput name = form.getInputByName("user_name");
name.setValueAttribute("my_password");
HtmlInput password = form.getInputByName("password");
password.setValueAttribute("my_password");
HtmlButton button = (HtmlButton) page1.getElementById("btnX");
HtmlPage page2 = button.click();
And this sites using ajax to show its data. But this ajax call can not be accomplish if login not succeeded. In chrome browser I can do this by typing in address bar like https://somesites.com/success/site.json so If I delete some cookies variable this never return any result.
My question was, how to make ajax call that maintain browser session after login success?
First of all, you want to set a AjaxHandler for your Webclient
webClient.setAjaxController(new AjaxController(){
#Override
public boolean processSynchron(HtmlPage page, WebRequest request, boolean async)
{
return true;
}
});
then, also make sure that the cookies are being stored (note that this should only be necessary if you need to initialize a new WebClient with old Cookies):
CookieManager cookieManager = new CookieManager();
webClient.setCookieManager(cookieManager);
Now you have a WebClient which stores your Cookies. Do your working login, and then simply execute the ajax call via Javascript:
page.executeJavaScript("[ajax call]").getJavaScriptResult();
If you only want to access the json data, you can simply navigate to the url and get the text. IF you are successfully logged in, it will work:
HtmlPage jsonPage = webClient.getPage("https://somesites.com/success/site.json ");
System.out.println(jsonPage.asText());
From there on you can parse it as JSON or do whatever you need to do with it.
Good day!
Do send a message to the forum.
When sending my data break
added to the header charset, it did not help
What could be the problem?
WebClient webClient = new WebClient(BrowserVersion.CHROME);
page = webClient.getPage(urlPage);
HtmlForm formEditPost = page.getFormByName("postform");
HtmlTextInput subjectInput = formEditPost.getInputByName("subject");
subjectInput.setText("заголовок");
HtmlTextArea messageElement = formEditPost.getTextAreaByName("message");
messageElement.setText("сообщение");
HtmlSubmitInput sendMessage = formEditPost.getInputByName("post");
page = sendMessage.click();
I am testing forum PhpBB with help of HTML unit. I want to add message in one of topic of PhpBB forum. I enter the message in textarea and click the submit button, but new page doesn't have my message. What seems to be the problem?
final ArrayList<HtmlForm> formList=(ArrayList<HtmlForm>) page.getByXPath("//form[#id='postform']");
final HtmlTextArea myMessage=formList.get(0).getTextAreaByName("message");
myMessage.type("text");
final HtmlSubmitInput submit_post = formList.get(0).getInputByName("post");
page=submit_post.click();
if (page.asText().contains("text")) {
System.out.println("right");
}
Thank you for help!
One way around this is to construct a custom http post request.
Use FIddler to capture the traffic and then replicate it with HTMLUnit. It should look something like below
public Page postMessage() throws Exception
{
URL url = new URL("YOURURL");
WebRequest requestSettings = new WebRequest(url, HttpMethod.POST);
requestSettings.setAdditionalHeader("Accept", "*/*");
requestSettings.setAdditionalHeader("Content-Type", "application/x-www-form-urlencoded");
requestSettings.setAdditionalHeader("Referer", "REFERER");
requestSettings.setAdditionalHeader("Accept-Language", "en-US,en;q=0.8");
requestSettings.setAdditionalHeader("Accept-Encoding", "gzip,deflate,sdch");
requestSettings
.setRequestBody("YOURPOSTREQUEST");
Page page = webClient.getPage(requestSettings);
return page;
}
In my WinPhone app I'm accessing a REST service.
At the beginnings I was using this code:
WebClient wc = new WebClient();
wc.Credentials = credentials;
wc.Headers["App-Key"] = appKey;
wc.DownloadStringCompleted +=
(o, args) => MessageBox.Show(args.Error == null ? "OK" : "Error");
wc.DownloadStringAsync(uri);
but it suddenly stopped working returning me a "The remote server returned an error: NotFound" error. After a google session and some clicks in the control panel, I didn't get it to work.
I decided to try this other way:
HttpWebRequest request = HttpWebRequest.CreateHttp(uri);
request.Credentials = credentials;
request.Headers["App-Key"] = appKey;
request.BeginGetResponse(asResult =>
{
var response = request.EndGetResponse(asResult) as HttpWebResponse;
StreamReader reader = new StreamReader(response.GetResponseStream());
string responseString = reader.ReadToEnd();
Dispatcher.BeginInvoke(
() => MessageBox.Show(response.StatusCode.ToString()));
}, null);
and it works.
I also tried to run the first snipped pointing the URI to google's home page and it works (I had to remove the credentials, of course).
Can anyone explain what's going on?
UPDATE
I managed to get it working by replacing the
wc.Credentials = new NetworkCredentials(username, password);
with
wc.Headers["Authorization"] = "Basic someBase64encodedString";
but i still wonder what happened and which are the differences between the first and the second line.
PS: the test URI is: https://api.pingdom.com/api/2.0/checks but you will need an app-key from them.
When using the Credentials property, the HttpWebRequest implementation will wait the challenge response from server before to send the 'Authorization' header value.
But this can be an issue in some cases, so you have to force Basic authentication by providing directly the Authorization header.
Example when using a REST Client library like Spring.Rest :
RestTemplate template = new RestTemplate("http://example.com");
template.RequestInterceptors.Add(new BasicSigningRequestInterceptor("login", "password"));
string result = template.GetForObject<string>(uri);