Unable to POST data using TLS 1.2 - apache-commons-httpclient

I am trying to POST data to a server who is just using TLS1.2
When i am running the code i am getting the following response from server.
HttpResponseProxy{HTTP/1.1 415 Unsupported Media Type [Date: Wed, 07 Sep 2016 17:42:57 CST, Server: , Strict-Transport-Security: max-age=63072000; includeSubdomains; preload, Pragma: no-cache, Content-Length: 0, charset: ISO-8859-1, X-ORACLE-DMS-RID: 0, X-ORACLE-DMS-ECID: 343fa6c0-ed24-4003-ad58-342caf000404-00000383, Set-Cookie: JSESSIONID=1ZMIvt1NqrtWpHgHs4mMmYyTPUGTOQgrA9biCE3Dok5v0gDCPXu6!681252631; path=/; secure; HttpOnly;HttpOnly;Secure, Cache-Control: no-store, P3P: policyref="/w3c/p3p.xml", CP="WBP DSP NOR AMT ADM DOT URT POT NOT", Keep-Alive: timeout=5, max=250, Connection: Keep-Alive, Content-Type: text/xml, Content-Language: en] [Content-Type: text/xml,Content-Length: 0,Chunked: false]}
I am using the below code to post the data to server . I am using apache httpcomponents-client-4.5.2.
private static Registry<ConnectionSocketFactory> getRegistry() throws KeyManagementException, NoSuchAlgorithmException {
SSLContext sslContext = SSLContexts.custom().build();
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext,
new String[]{"TLSv1.2"}, null, SSLConnectionSocketFactory.getDefaultHostnameVerifier());
return RegistryBuilder.<ConnectionSocketFactory>create()
.register("https", sslConnectionSocketFactory)
.register("https", sslConnectionSocketFactory)
.build();
}
public static void main(String[] args) throws Exception
{
PoolingHttpClientConnectionManager clientConnectionManager = new PoolingHttpClientConnectionManager(getRegistry());
clientConnectionManager.setMaxTotal(100);
clientConnectionManager.setDefaultMaxPerRoute(20);
HttpClient client = HttpClients.custom().setConnectionManager(clientConnectionManager).build();
HttpPost request = new HttpPost("https://someserver.com/dataupload");
File file = new File("C://Nible//code//_client//Request.xml");
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.setContentType(ContentType.TEXT_XML);
FileBody fileBody = new FileBody(file);
builder.addPart("my_file", fileBody);
HttpEntity reqEntity = builder.build();
request.setEntity(reqEntity);
HttpResponse response = client.execute(request);
System.out.println(response);
}
Can you please tell me what wrong am i doing?
I tried using below code instead of MultipartEntityBuilder. I am still getting the same error.
EntityBuilder builder = EntityBuilder.create();
builder.setFile(file);
builder.setContentType(ContentType.TEXT_XML);
If i am sending the BLANK REQUEST to server then also i am getting the same error. Blank error means i am not putting any thing in request just
HttpPost request = new HttpPost("https://someserver.com/dataupload");
HttpResponse response = client.execute(request);

I suspect of these lines in your code:
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
builder.setContentType(ContentType.TEXT_XML);
Multipart entities cannot be of content-type xml. They must be one of these types:
multipart/mixed
multipart/alternative
multipart/digest
multipart/parallel
(See RFC 1341 7.2)
I guess you should use one of these content-types for the multipart entity, and set text/xml as the content type of the single part:
FileBody fileBody = new FileBody(file, ContentType.TEXT_XML);
(Another issue is that I don't see necessary to send a multipart for just one file: You could leave out the MultipartEntityBuilder object and build directly a FileEntity.)

Related

How to set up HTTPS communication using Apache HttpClient

I am trying to gather and store URLs and associated metadata from the Cochrane Library review collection (https://www.cochranelibrary.com/cdsr/reviews/topics) for a project. But when I do a regular HTTP Get requests I get a timed out 419 error.
HttpResponseProxy{HTTP/1.1 419 status code 419 [Date: Thu, 08 Dec 2022 11:36:36 GMT, Content-Length: 0, Connection: keep-alive, Cf-Railgun: direct (starting new WAN connection), CF-Cache-Status: DYNAMIC, Server: cloudflare, CF-RAY: 7765344b8cb53338-EWR] [Content-Length: 0,Chunked: false]}
My only requirements for this project are java 1.7 and Apache HttpClient 4.5
This is my code to reproduce the error:
SSLContext sslContext = SSLContexts.createSystemDefault();
SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(sslContext);
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLSocketFactory(sslConnectionFactory)
.build();
HttpGet request = new HttpGet("https://www.cochranelibrary.com/");
CloseableHttpResponse response = httpClient.execute(request);
HttpEntity entity = response.getEntity();
System.out.println(entity);
httpClient.close();
response.close();

how to handle "Transfer-Encoding=chunked" in SI HttpRequestExecutingMessageHandler

I am calling an external server using HttpRequestExecutingMessageHandler. I am using a JSON to object transformer to convert the JSON data. But I am getting the following exception.
Caused by: com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
at [Source: (String)"�
when I checked the headers I found the following.
Transfer-Encoding=chunked
Is this the reason for the exception log?
The outbound gateway and JsonToObjectTransformer are given below:
#ServiceActivator(inputChannel = "channelOutboundRequest")
#Bean
public HttpRequestExecutingMessageHandler outboundGateway() {
final HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler(
endpoint);
handler.setExpectedResponseType(String.class);
handler.setHttpMethod(HttpMethod.POST);
handler.setOutputChannelName("channelResponse");
handler.setAdviceChain(Collections.singletonList(advice()));
return handler;
}
#Bean
#Transformer(inputChannel = "channelResponse", outputChannel = "channelReply")
public JsonToObjectTransformer transformer(ObjectMapper objectMapper) {
final JsonObjectMapper<?, ?> mapper = new Jackson2JsonObjectMapper(objectMapper);
return new JsonToObjectTransformer(DetailsDTO.class, mapper);
}
If the header is causing the issue, how can I handle the response?
Note: If I hit the external server directly using postman, I am getting the response in JSON structure.
I have no idea what's wrong here.. If I use the simple restemplate call like below, it works properly.
JSONObject jsonObject = new JSONObject("{\"code\":\"F001\",\"transactionId\":\"1008566223232\"}");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setBearerAuth("token");
HttpEntity<String> request =
new HttpEntity<String>(jsonObject.toString(), headers);
String respns = restTemplt.postForObject("http://endpoint", request, String.class);
System.out.println(respns);
JSONObject response = new JSONObject(respns);
One difference I could find was in the response headers. those are given below:
Outbound gateway response headers :-
{Transfer-Encoding=chunked, http_requestMethod=GET, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#d8b195e, Server=nginx, Accept=/, Connection=keep-alive, User-Agent=PostmanRuntime/7.28.0, Host=localhost:8901, Accept-Encoding=gzip, deflate, br, http_statusCode=200 OK, Date=1622533072000, Authorization=Bearer token, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#d8b195e, Cache-Control=no-cache, ETag=W/"1009-5SzdL+uWyY6ZcMWht5dMtm2Sxlc", Content-Encoding=gzip, http_requestUrl=http://inboundurl, id=be07fc8d-d478-5fa9-33e4-61a2b5f92468, Content-Length=207, contentType=application/json;charset=utf-8, Content-Type=application/json, requestFrom=CUSTOM_HEADER, timestamp=1622533092827}
Normal restTemplate call response header
[Server:"nginx", Date:"Tue, 01 Jun 2021 07:34:54 GMT", Content-Type:"application/json; charset=utf-8", Content-Length:"4105", Connection:"keep-alive", Access-Control-Allow-Origin:"*", Content-Security-Policy:"default-src 'self';base-uri 'self';block-all-mixed-content;font-src 'self' https: data:;frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests", X-DNS-Prefetch-Control:"off", Expect-CT:"max-age=0", X-Frame-Options:"SAMEORIGIN", Strict-Transport-Security:"max-age=15552000; includeSubDomains", X-Download-Options:"noopen", X-Content-Type-Options:"nosniff", X-Permitted-Cross-Domain-Policies:"none", Referrer-Policy:"no-referrer", X-XSS-Protection:"0", ETag:"W/"1009-llD9DqxYkEsjyikWajYk+16cb1k""]
Can anyone please help?
After many trials and errors, I found the reason for this. Accept-Encoding=gzip, deflate, br this header in the outbound gateway request is the root cause. the response I am getting is a long string and it is getting compressed because of this header. i added a header filter before the outbound gateway to remove this header.
#Bean
#Transformer(inputChannel = "channelHeaderFilterReq", outputChannel = "channelHeaderFilterRes")
public HeaderFilter filter() {
return new HeaderFilter("Accept-Encoding");
}
Now everything works fine..!!

Construct MulipartFormDataInput request for REST API?

I have a REST API with the following signature:
#POST
#Path("/bulkControlsMapping")
#Consumes({ MediaType.MULTIPART_FORM_DATA })
Response saveControlsMapping(#NonNull MultipartFormDataInput input);
and in my API I'm extracting my data from the input as:
Map<String, List<InputPart>> uploadForm = input.getFormDataMap();
// For text
InputPart inputPart = uploadForm.get(TEXT_KEY).get(0);
// For file
InputPart inputPart = uploadForm.get(CONTROLS_FILE_KEY).get(0);
I'm calling this REST API from Spring MVC web service, where I'm constructing my request as:
Client client = ClientBuilder.newClient().register(MultipartWriter.class);
String endPoint = "https://localhost:8080/";
WebTarget webTarget = client.target(endPoint).path("bulkControlsMapping");
MultipartFormDataOutput multipartFormDataOutput = new MultipartFormDataOutput();
multipartFormDataOutput.addFormData(CONTROLS_FILE_KEY,
file, MediaType.APPLICATION_OCTET_STREAM_TYPE);
multipartFormDataOutput.addFormData(TEXT_KEY, countryCode, MediaType.TEXT_PLAIN_TYPE);
Response response = webTarget.request()
.post(Entity.entity(multipartFormDataOutput, MediaType.MULTIPART_FORM_DATA));
But I'm getting 400 bad Request Exception in this due to no Content-Disposition header found within the part. What is the correct way to build this request?
[java] 03 Sep 2017 21:16:56,300 [WARN] (http-bio-0.0.0.0-8080-exec-12) org.jboss.resteasy.core.ExceptionHandler: Failed executing POST /bulkControlsMapping
[java] org.jboss.resteasy.spi.ReaderException: java.lang.RuntimeException: Could find no Content-Disposition header within part
[java] at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:183) ~[resteasy-jaxrs-3.0.10.Final.jar:?]
[java] at org.jboss.resteasy.core.MethodInjectorImpl.injectArguments(MethodInjectorImpl.java:89) ~[resteasy-jaxrs-3.0.10.Final.jar:?]
[java] at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:112) ~[resteasy-jaxrs-3.0.10.Final.jar:?]
[java] at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296) ~[resteasy-jaxrs-3.0.10.Final.jar:?]
[java] at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250) ~[resteasy-jaxrs-3.0.10.Final.jar:?]
When I test my API with Postman Client, I could see that correct header structure should be like this, but I'm not able to build it manually:
POST /bulkControlsMapping HTTP/1.1
Host: localhost:8080
Cache-Control: no-cache
Postman-Token: 0af37ee0-9623-43b6-f5f3-bd2c01bdd84c
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="bulkMappingFile"; filename="Beta-Test-2.csv"
Content-Type: text/csv
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="countryCode"
MX
------WebKitFormBoundary7MA4YWxkTrZu0gW--
UPDATE:
I got it to work by using MultiPartFeature,
Client client = ClientBuilder.newClient().register(MultiPartFeature.class, JacksonFeature.class);
and different methods for generating a request using FormDataMultiPart and FormDataContentDisposition.
FormDataMultiPart formDataMultiPart = new FormDataMultiPart();
FormDataContentDisposition.FormDataContentDispositionBuilder builder = FormDataContentDisposition
.name("controlsBulkMappingFile");
log.error("FILENAME: " + file.getName());
builder.fileName(file.getName());
FormDataContentDisposition formDataContentDisposition = builder.build();
FileInputStream fileInputStream = null;
try {
fileInputStream = new FileInputStream(file);
formDataMultiPart.bodyPart(
new FormDataBodyPart((file.getName()), fileInputStream,
MediaType.APPLICATION_OCTET_STREAM_TYPE)
.contentDisposition(formDataContentDisposition));
} catch (FileNotFoundException ex) {
System.out.println("&&&&&&: " + ex.getLocalizedMessage());
}
// COUNTRY CODE
FormDataContentDisposition.FormDataContentDispositionBuilder builder2 = FormDataContentDisposition
.name("countryCode");
FormDataContentDisposition formDataContentDisposition2 = builder2.build();
formDataMultiPart.bodyPart(new FormDataBodyPart("countryCode", countryCode,
MediaType.TEXT_PLAIN_TYPE)).contentDisposition(formDataContentDisposition2);
log.error("MULTIPART: " + formDataMultiPart.getBodyParts().toString());
Entity<FormDataMultiPart> entity = Entity
.entity(formDataMultiPart, MediaType.MULTIPART_FORM_DATA_TYPE);
Response response = webTarget.request().accept(MediaType.APPLICATION_JSON)
.post(entity);
log.error("BULK MAPPING: Response 1: {}", response);
BulkControlsMappingResponse mappingResponse = response
.readEntity(BulkControlsMappingResponse.class);
My API consumes MediaType.MULTIPART_FORM_DATA and produces MediaType.APPLICATION_JSON. Now when I use the MultiPartFeature alone, I'm able to send the request but not able to parse the response. I'm getting
[tomcat:launchProperties] org.glassfish.jersey.message.internal.MessageBodyProviderNotFoundException: MessageBodyWriter not found for media type=multipart/form-data, type=class org.glassfish.jersey.media.multipart.FormDataMultiPart, genericType=class org.glassfish.jersey.media.multipart.FormDataMultiPart.
I got this to work by using different features while sending the request, and different for response as follows:
Client client = ClientBuilder.newClient().register(MultiPartFeature.class);
WebTarget webTarget = client.target(endPoint).path("bulkControlsMapping")
.register(JacksonFeature.class);

Sendind files to web servicce : HTTP Error 415 Unsupported media type

I try from my rest client to send files to my web service but i keep getting 415 when i'm sure of the media type i'm sending.
I'm using spring MVC :
Here code of the rest client :
public Response uploadFile(FormDataMultiPart multipart, ...) {
WebTarget target = client.target(uri);
Response response = target.
path(*path*).
request().
header("Content-Type", "multipart/form-data").
post(Entity.entity(multipart, MediaType.MULTIPART_FORM_DATA), Response.class);
return response;
}
And here how i use my function :
#RequestMapping(value = "*map path*", method = RequestMethod.POST)
public ModelAndView saveAttachedFiles(#RequestParam("file") MultipartFile myFile, Model model, HttpServletRequest request) {
try {
File file = new File( myFile.getOriginalFilename());
myFile.transferTo(file);
FileDataBodyPart filePart = new FileDataBodyPart("file", file);
FormDataContentDisposition contentDisposition = FormDataContentDisposition.name("file").fileName(file.getName()).build();
filePart.setContentDisposition(contentDisposition);
FormDataMultiPart formDataMultiPart = new FormDataMultiPart();
formDataMultiPart.bodyPart(filePart);
Response reponse = clientRest.uploadFile(formDataMultiPart, ...);
}
Here request header of my client (from logs)
31 > POST ***
31 > Content-Type: multipart/form-data
31 > Cookie: $Version=1;JSESSIONID=***
31 > Referer: ***
--Boundary_1_523348906_1460533877988
Content-Type: image/jpeg
Content-Disposition: form-data; filename="Penguins.jpg"; name="file"
Here code of the web services
#POST
#Path("/file")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition contentDispositionHeader,
#QueryParam("uuid") String uuid) {
AttachedFileDto attachedFileDto;
try {
** Processing the file **
}
}
And the web service response header
31 < 415
31 < Access-Control-Allow-Credentials: true
31 < Access-Control-Allow-Headers: Content-Type, Accept, X-Requested-With,Cache-Control,Pragma
31 < Access-Control-Allow-Methods: GET,POST,HEAD,OPTIONS,PUT,DELETE
31 < Access-Control-Allow-Origin: http://localhost:3333
31 < Connection: Keep-Alive
31 < Content-Length: 0
31 < Content-Type: application/json
31 < Date: Wed, 13 Apr 2016 07:51:16 GMT
31 < Keep-Alive: timeout=5, max=100
31 < Server: Apache
Do you have any clue ?
Thanks =)
Found the solution to my problem, i was confused between PathParam and QueryParam.
Service was like this :
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition contentDispositionHeader,
#QueryParam("uuid") String uuid)
And i was putting my uuid parameter in path function :
public Response uploadFile(FormDataMultiPart multipart, ...) {
WebTarget target = client.target(uri);
Response response = target.
path(*path*?uuid=12315465).
request().
header("Content-Type", "multipart/form-data").
post(Entity.entity(multipart, MediaType.MULTIPART_FORM_DATA), Response.class);
return response;
}
Added the queryParam function :
public Response uploadFile(FormDataMultiPart multipart, ...) {
WebTarget target = client.target(uri);
Response response = target.
path(*path*).
queryParam("uuid", uuid).
request().
post(Entity.entity(multipart, multipart.getMediaType()));
return response;
}
And now it works =)

Httpcomponents-client proxy problems

I have problems when trying to connect to the internet via a proxy using the new httocomponent-client module
If I use directly the Proxy object and HttpURLConnection everything goes fine:
URL u = new URL("http://www.google.com");
Proxy proxy = new Proxy(Type.HTTP, new InetSocketAddress("somehost", 8080));
HttpURLConnection con = (HttpURLConnection) u.openConnection(proxy);
con.setRequestMethod("GET");
System.out.println(con.getResponseCode());
Now I try to do the same with the new api:
HttpHost proxy = new HttpHost("somehost", 8080, "http");
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
HttpHost targetHost = new HttpHost("http://www.google.com");
HttpGet httpGet = new HttpGet("/");
try {
HttpResponse httpResponse = httpClient.execute(targetHost, httpGet);
System.out.println(httpResponse.toString());
} catch (Exception e) {
e.printStackTrace();
}
but I get this:
HTTP/1.1 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ) [Via: 1.1 xxx, Proxy-Authenticate: Negotiate, Proxy-Authenticate: Kerberos, Proxy-Authenticate: NTLM, Connection: Keep-Alive, Proxy-Connection: Keep-Alive, Pragma: no-cache, Cache-Control: no-cache, Content-Type: text/html, Content-Length: 7079 ]
I also tried
ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
httpClient.getConnectionManager().getSchemeRegistry(),new MyProxySelector());
httpClient.setRoutePlanner(routePlanner);
Where MyProxySelector return the Proxy that I nned but no result.
Why using the new API makes the proxy authentication required inside the code?
I don't know why the solution with ProxySelectorRoutePlanner doesn't work, are you sure you start your JVM with the proxy settings ?
It looks like you need to add this line :
httpClient.getCredentialsProvider().setCredentials(new AuthScope("yourProxyHost", Port),
new UsernamePasswordCredentials("yourUsername", "yourPass"));

Resources