HtmlUnit does not return all headers - htmlunit

final WebClient webClient = new WebClient();
final HtmlPage page = webClient.getPage(webPageURL);
final String pageAsXml = page.asXml();
final String pageAsText = page.asText();
List <NameValuePair> response = page.getWebResponse().getResponseHeaders();
for (NameValuePair header : response) {
log.info(header.toString() + " = " + header.getValue());
The web page returns more than one header. But the log shows only the first header. How do I get rest of headers? The header I am looking for is of Content Type: application/javascript;charset=ISO-8859-1
The web page is an internal web page.

The code you provided is working for me. Actually, I replaced the header.toString() for header.getName():
final WebClient webClient = new WebClient();
final HtmlPage page = webClient.getPage("http://www.debian.org");
List<NameValuePair> response = page.getWebResponse().getResponseHeaders();
for (NameValuePair header : response) {
System.out.println(header.getName() + " = " + header.getValue());
}
The output is:
Date = Thu, 22 Aug 2013 19:48:54 GMT
Server = Apache
Content-Location = index.en.html
Vary = negotiate,accept-language,Accept-Encoding
TCN = choice
Last-Modified = Thu, 22 Aug 2013 15:31:17 GMT
ETag = "3887-4e48afb257b40"
Accept-Ranges = bytes
Cache-Control = max-age=86400
Expires = Fri, 23 Aug 2013 19:48:54 GMT
Content-Encoding = gzip
Content-Length = 4605
Keep-Alive = timeout=15, max=100
Connection = Keep-Alive
Content-Type = text/html
Content-Language = en
As you can see, the Content-Type header is there. Can you confirm the server is actually sending that piece of data (it is a common header so it should).

Related

HttpsUrlConnection: Response different from Postman API

I am posting an https request to client's API ,testing from Postman API , I am getting proper response, response is in xml structure , however using the conventional way of httpsurlconnection api in java , I am getting response which is trimmed .
Response in Postman :
<data>
<status>02231013757|1518968|Failed</status>
<status>041580082657|1518969|Failed</status>
</data>
Response header in Java :
Server: Apache-Coyote/1.1Set-Cookie:JSESSIONID=68735C46DABA9D068FE1DB1D0FE83454; Path=/awp; Secure Expires: Thu, 01 Jan 1970 00:00:00 GMT Pragma: no-cache Content-Length: 14 Date: Wed, 19 Apr 2017 13:25:26 GMT Content-Type: text/html X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1
Response fetched as String : <data></data>
As you can find the content-length is 14 in response header, so this issue is at client's service end or am I missing something ?
below snippet I am using to read the content :
read = new InputStreamReader(rc.getInputStream() );
StringBuilder sb = new StringBuilder();
int ch = read.read();
while( ch != -1 )
{
sb.append((char)ch);
ch = read.read();
}
resp = sb.toString();

Unable to POST data using TLS 1.2

I am trying to POST data to a server who is just using TLS1.2
When i am running the code i am getting the following response from server.
HttpResponseProxy{HTTP/1.1 415 Unsupported Media Type [Date: Wed, 07 Sep 2016 17:42:57 CST, Server: , Strict-Transport-Security: max-age=63072000; includeSubdomains; preload, Pragma: no-cache, Content-Length: 0, charset: ISO-8859-1, X-ORACLE-DMS-RID: 0, X-ORACLE-DMS-ECID: 343fa6c0-ed24-4003-ad58-342caf000404-00000383, Set-Cookie: JSESSIONID=1ZMIvt1NqrtWpHgHs4mMmYyTPUGTOQgrA9biCE3Dok5v0gDCPXu6!681252631; path=/; secure; HttpOnly;HttpOnly;Secure, Cache-Control: no-store, P3P: policyref="/w3c/p3p.xml", CP="WBP DSP NOR AMT ADM DOT URT POT NOT", Keep-Alive: timeout=5, max=250, Connection: Keep-Alive, Content-Type: text/xml, Content-Language: en] [Content-Type: text/xml,Content-Length: 0,Chunked: false]}
I am using the below code to post the data to server . I am using apache httpcomponents-client-4.5.2.
private static Registry<ConnectionSocketFactory> getRegistry() throws KeyManagementException, NoSuchAlgorithmException {
SSLContext sslContext = SSLContexts.custom().build();
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext,
new String[]{"TLSv1.2"}, null, SSLConnectionSocketFactory.getDefaultHostnameVerifier());
return RegistryBuilder.<ConnectionSocketFactory>create()
.register("https", sslConnectionSocketFactory)
.register("https", sslConnectionSocketFactory)
.build();
}
public static void main(String[] args) throws Exception
{
PoolingHttpClientConnectionManager clientConnectionManager = new PoolingHttpClientConnectionManager(getRegistry());
clientConnectionManager.setMaxTotal(100);
clientConnectionManager.setDefaultMaxPerRoute(20);
HttpClient client = HttpClients.custom().setConnectionManager(clientConnectionManager).build();
HttpPost request = new HttpPost("https://someserver.com/dataupload");
File file = new File("C://Nible//code//_client//Request.xml");
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.setContentType(ContentType.TEXT_XML);
FileBody fileBody = new FileBody(file);
builder.addPart("my_file", fileBody);
HttpEntity reqEntity = builder.build();
request.setEntity(reqEntity);
HttpResponse response = client.execute(request);
System.out.println(response);
}
Can you please tell me what wrong am i doing?
I tried using below code instead of MultipartEntityBuilder. I am still getting the same error.
EntityBuilder builder = EntityBuilder.create();
builder.setFile(file);
builder.setContentType(ContentType.TEXT_XML);
If i am sending the BLANK REQUEST to server then also i am getting the same error. Blank error means i am not putting any thing in request just
HttpPost request = new HttpPost("https://someserver.com/dataupload");
HttpResponse response = client.execute(request);
I suspect of these lines in your code:
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.setContentType(ContentType.TEXT_XML);
Multipart entities cannot be of content-type xml. They must be one of these types:
multipart/mixed
multipart/alternative
multipart/digest
multipart/parallel
(See RFC 1341 7.2)
I guess you should use one of these content-types for the multipart entity, and set text/xml as the content type of the single part:
FileBody fileBody = new FileBody(file, ContentType.TEXT_XML);
(Another issue is that I don't see necessary to send a multipart for just one file: You could leave out the MultipartEntityBuilder object and build directly a FileEntity.)

Sendind files to web servicce : HTTP Error 415 Unsupported media type

I try from my rest client to send files to my web service but i keep getting 415 when i'm sure of the media type i'm sending.
I'm using spring MVC :
Here code of the rest client :
public Response uploadFile(FormDataMultiPart multipart, ...) {
WebTarget target = client.target(uri);
Response response = target.
path(*path*).
request().
header("Content-Type", "multipart/form-data").
post(Entity.entity(multipart, MediaType.MULTIPART_FORM_DATA), Response.class);
return response;
}
And here how i use my function :
#RequestMapping(value = "*map path*", method = RequestMethod.POST)
public ModelAndView saveAttachedFiles(#RequestParam("file") MultipartFile myFile, Model model, HttpServletRequest request) {
try {
File file = new File( myFile.getOriginalFilename());
myFile.transferTo(file);
FileDataBodyPart filePart = new FileDataBodyPart("file", file);
FormDataContentDisposition contentDisposition = FormDataContentDisposition.name("file").fileName(file.getName()).build();
filePart.setContentDisposition(contentDisposition);
FormDataMultiPart formDataMultiPart = new FormDataMultiPart();
formDataMultiPart.bodyPart(filePart);
Response reponse = clientRest.uploadFile(formDataMultiPart, ...);
}
Here request header of my client (from logs)
31 > POST ***
31 > Content-Type: multipart/form-data
31 > Cookie: $Version=1;JSESSIONID=***
31 > Referer: ***
--Boundary_1_523348906_1460533877988
Content-Type: image/jpeg
Content-Disposition: form-data; filename="Penguins.jpg"; name="file"
Here code of the web services
#POST
#Path("/file")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition contentDispositionHeader,
#QueryParam("uuid") String uuid) {
AttachedFileDto attachedFileDto;
try {
** Processing the file **
}
}
And the web service response header
31 < 415
31 < Access-Control-Allow-Credentials: true
31 < Access-Control-Allow-Headers: Content-Type, Accept, X-Requested-With,Cache-Control,Pragma
31 < Access-Control-Allow-Methods: GET,POST,HEAD,OPTIONS,PUT,DELETE
31 < Access-Control-Allow-Origin: http://localhost:3333
31 < Connection: Keep-Alive
31 < Content-Length: 0
31 < Content-Type: application/json
31 < Date: Wed, 13 Apr 2016 07:51:16 GMT
31 < Keep-Alive: timeout=5, max=100
31 < Server: Apache
Do you have any clue ?
Thanks =)
Found the solution to my problem, i was confused between PathParam and QueryParam.
Service was like this :
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition contentDispositionHeader,
#QueryParam("uuid") String uuid)
And i was putting my uuid parameter in path function :
public Response uploadFile(FormDataMultiPart multipart, ...) {
WebTarget target = client.target(uri);
Response response = target.
path(*path*?uuid=12315465).
request().
header("Content-Type", "multipart/form-data").
post(Entity.entity(multipart, MediaType.MULTIPART_FORM_DATA), Response.class);
return response;
}
Added the queryParam function :
public Response uploadFile(FormDataMultiPart multipart, ...) {
WebTarget target = client.target(uri);
Response response = target.
path(*path*).
queryParam("uuid", uuid).
request().
post(Entity.entity(multipart, multipart.getMediaType()));
return response;
}
And now it works =)

NSHTTPCookie cookiesWithResponseHeaderFields

When making a call to an API, I receive this as response:
<NSHTTPURLResponse: 0x7f96f0510f80> { URL: https://insula.magister.net/api/sessie } { status code: 201, headers {
"Cache-Control" = "max-age=120, private";
"Content-Length" = 0;
Date = "Thu, 30 Jul 2015 17:27:11 GMT";
"Set-Cookie" = "SESSION_ID=c8714acb-22f8-4295-90ee-82e9469890e1; domain=insula.magister.net; path=/api; secure; httponly";
"Strict-Transport-Security" = "max-age=31536000";
Vary = "Accept-Encoding,Cookie";
"X-Frame-Options" = DENY;
} }
print(HTTPResponse.allHeaderFiles) prints the following output:
[Set-Cookie: SESSION_ID=65a59001-de75-41ff-ad22-70549baa96f5; domain=insula.magister.net; path=/api; secure; httponly, Strict-Transport-Security: max-age=31536000, Date: Thu, 30 Jul 2015 17:33:13 GMT, X-Frame-Options: DENY, Content-Length: 0, Cache-Control: max-age=120, private, Vary: Accept-Encoding,Cookie]
But I can't get cookiesWithResponseHeaderFields to work, I should pass a [String:String] array as first parameter, but the response doesn't seem to meet.
let cookies = NSHTTPCookie.cookiesWithResponseHeaderFields(HTTPResponse.allHeaderFields, forURL: url)
But as HTTPResponse.allHeaderFiles is an array of type [NSObject:AnyObject] it can't meet the [String:String] parameter type.
Who knows what to do?
There's a similar question with a great answer.
You should cast the allHeaderFields property to the type you need, that is [String:String], like this:
let foundCookies: [NSHTTPCookie]?
if let responseHeaders = HTTPResponse.allHeaderFields as? [String:String] {
foundCookies = NSHTTPCookie.cookiesWithResponseHeaderFields(responseHeaders, forURL:url)
}

CouchDB: Get server-time with an http request in CouchApp

I need the server-time for "user-is-online" stats in my CouchApp. I work with jquery.couch.js and would prefer to have a url, e.g. /db/_design/app/time - which gets me a timestamp.
How do I realize this?
A show function could do that:
function(doc, req) {
// _design/myapp/_show/now
// First version possibly incompatible with some spidermonkey versions.
//var now = new Date();
var now = new Date().getTime();
var output = JSON.parse(JSON.stringify(now)) + "\n";
return { code: 200
, headers: { "Content-Type": "text/plain"
}
, body:output
};
}
The server also includes a Date header that you might want to use.
$ curl -D- http://localhost:5984
HTTP/1.1 200 OK
Server: CouchDB/1.1.0 (Erlang OTP/R14B)
Date: Fri, 27 May 2011 00:28:31 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 40
Cache-Control: must-revalidate
{"couchdb":"Welcome","version":"1.1.0"}

Resources