Spring RestTemplate receives "401 Unauthorized" - spring

I am using the following to retrieve JSON via RestTemplate in Spring 4:
protected DocInfoResponse retrieveData(String urlWithAuth) {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.add("Authorization", "Basic " + auth.getSig());
HttpEntity<String> request = new HttpEntity<String>(headers);
ResponseEntity<DocInfoResponse> response = restTemplate.exchange(urlWithAuth, HttpMethod.GET, request, DocInfoResponse.class);
return response.getBody();
}
I used the same code (with different response class) to successfully get a JSON doc from the same site (with different parameters to get a different doc).
When I execute the above code I receive the following stack trace (in part):
Caused by: org.springframework.web.client.HttpClientErrorException: 401 Unauthorized
at
org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
Can anyone point me to why this might be receiving the exception?

I found that my issue originally posted above was due to double encryption happening on the auth params. I resolved it by using UriComponentsBuilder and explicitly calling encode() on the the exchange().
SyncResponse retrieveData(UriComponentsBuilder builder) {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.set("Accept", MediaType.APPLICATION_JSON_VALUE);
HttpEntity<String> request = new HttpEntity<String>(headers);
ResponseEntity<SyncResponse> response = restTemplate.exchange(builder.build().encode().toUri(), HttpMethod.GET, request, SyncResponse.class);
return response.getBody();
}
My UriComponentsBuilder was built using:
UriComponentsBuilder buildUrl(String urlString) {
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(urlString);
return auth.appendAuth(builder);
}
(The auth.appendAuth() adds additional .queryParams() needed by the target service in urlString.)
The call to execute this was retrieveData(buildUrl(urlString));.

After investigating on my own problem, I realized that FireFox RESTClient was successful because I was connected to the target URL. The Basic Auth I thought I was using, was not so basic after all.
Eventually, I read the doc of the app i was trying to connect to and realized they propose a connection token mechanism. Now it works.
After reading your code, I say it looks quite OK, although I'm not sure what is your object auth on which you call getSig.
First things first: try to access your service from any client, like a web browser, a PostMan or RESTClient. Make sure you successfully retrieve your infos WITHOUT being connected to your app!!!
Depending on the result, I say you should, either try to encrypt manually your Authorization token (you'll easilly find posts on this site to show you how to) or try another connection mechanism.

The process of creating the Authorization header is relatively straightforward for Basic Authentication, so it can pretty much be done manually with a few lines of code:
HttpHeaders createHeaders(String username, String password){
return new HttpHeaders() {{
String auth = username + ":" + password;
byte[] encodedAuth = Base64.encodeBase64(
auth.getBytes(Charset.forName("US-ASCII")) );
String authHeader = "Basic " + new String( encodedAuth );
set( "Authorization", authHeader );
}};
}
Then, sending a request becomes just as simple:
RestTemplate restTemplate = new RestTemplate();
restTemplate.exchange
(uri, HttpMethod.POST, new HttpEntity<T>(createHeaders(username, password)), clazz);
https://www.baeldung.com/how-to-use-resttemplate-with-basic-authentication-in-spring#manual_auth

Related

Java S3 upload using Spring RestTemplate

I want to make this call using SpringBoot RestTemplate to upload a file to a S3 bucket: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
PUT /my-image.jpg HTTP/1.1
Host: myBucket.s3.<Region>.amazonaws.com
Date: Wed, 12 Oct 2009 17:50:00 GMT
Authorization: authorization string
Content-Type: text/plain
Content-Length: 11434
x-amz-meta-author: Janet
Expect: 100-continue
[11434 bytes of object data]
and
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder.rootUri("")
.additionalInterceptors((request, body, execution) -> {
request.getHeaders().add("Authorization",
"Bearer a0d78d7922f333ee22d75bea53d01hhkjk83f5ac03f11ccd87787");
return execution.execute(request, body);
}).build();
}
I've tried
Resource resource = new ClassPathResource("logback.xml");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.TEXT_PLAIN);
HttpEntity<byte[]> requestEntity
= new HttpEntity<>(StreamUtils.copyToByteArray(resource.getInputStream()), headers);
Map<String, Object> parameters = new HashMap<>(4);
parameters.put("cors_enabled", true);
parameters.put("acl", "private");
parameters.put("key", "my-key");
parameters.put("Bucket", "parameters.put("Bucket", "https://cloud.linode.com/object-storage/buckets/eu-central-1/my-bucket-2020");");
restTemplate.put("https://api.linode.com/v4/object-storage/buckets", requestEntity, parameters);
but I got
org.springframework.web.client.HttpClientErrorException$MethodNotAllowed: 405 METHOD NOT ALLOWED: [{"errors": [{"reason": "Method Not Allowed"}]}]
also when Getting I have a problem:
MultiValueMap<String, Object> body
= new LinkedMultiValueMap<>();
UriComponentsBuilder builder =
UriComponentsBuilder.fromHttpUrl("https://api.linode.com/v4/object-storage/buckets/eu-central-1/my-bucket-2020/object-url");
builder.queryParam("method", "GET");
builder.queryParam("name", "43f959d9-a11a-4f2cec88fd7e.JPG");
body.add("method", "GET");
body.add("name", "43f959d9-a11a-4f2cec88fd7e.JPG");
HttpHeaders headers = new HttpHeaders();
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);
restTemplate.postForEntity(builder.build().encode().toUri(),
requestEntity, LinodeResponse.class);
and the response:
org.springframework.web.client.HttpClientErrorException$BadRequest: 400 BAD REQUEST: [{"errors": [{"reason": "name is required", "field": "name"}, {"reason": "method is required", "field": "method"}]}]
ans when accessing with AWS-SDK I have this error:
com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records.
Linode seems to offer an API to generate presigned urls for interact with objects in S3.
To use the API, first, you can create two POJO that represent the request and response we will send and receive from the API so we can use to serialize an deserialize JSON information.
For the request object:
public class LinodeGeneratePresignedUrlRequest {
private String method;
private String name;
#JsonProperty("content_type")
private String contentType;
#JsonProperty("expires_in")
private int expiresIn;
// Getters and setters
}
And for the response:
pubic class LinodeGeneratePresignedUrlResponse {
private String url;
// Getters and setters
}
These objects match the information required by the endpoint.
If you want to create an object in your bucket with the Linode API, you first need to request a presigned URL. Once obtained, you will use this URL to perform the actual operation over the bucket object. The operation is defined by the method parameter passed to the API. Consider the following example:
// Obtain a reference to the RestTemplate instance.
// It should support the interchange of JSON information
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
// Set content type to the one required by the Linode API application/json
headers.setContentType(MediaType.APPLICATION_JSON);
// Set the appropriate credentials for the Linode API
String token = "your token";
headers.set(HttpHeaders.AUTHORIZATION, "Bearer" + token);
// Create the presigned url request
LinodeGeneratePresignedUrlRequest linodeGeneratePresignedUrlRequest =
new LinodeGeneratePresignedUrlRequest();
// Operation to perform when you interact with AWS later
// In this case, PUT because you need to create a new object
linodeGeneratePresignedUrlRequest.setMethod("PUT");
// The object name: can match or not the actual file you want to upload
linodeGeneratePresignedUrlRequest.setName("my-object-name.pdf");
// As you are performing an upload (PUT, POST), indicate the content type of
// the information you are uploading to AWS. It should match the provided later
// when you interact with AWS. For instance, consider that you are uploading a PDF file
linodeGeneratePresignedUrlRequest.setContentType("application/pdf");
// Optionally, you can set the expiration time of the generated presigned url
// By default, an hour (3600 seconds)
// Perform the actual Linode API invocation
HttpEntity<LinodeGeneratePresignedUrlRequest> requestEntity =
new HttpEntity<LinodeGeneratePresignedUrlRequest>(linodeGeneratePresignedUrlRequest, headers);
// The Linode API URL for your cluster and bucket
String linodeApiUrl = "https://api.linode.com/v4/object-storage/buckets/eu-central-1/my-bucket-2020/object-url";
HttpEntity<LinodeGeneratePresignedUrlResponse> responseEntity = restTemplate.exchange(linodeApiUrl, HttpMethod.POST, requestEntity, LinodeGeneratePresignedUrlResponse.class);
// Linde wil provide a response with a property named 'url' corresponding
// to the presigned url that we can use to interact with AWS S3
LinodeGeneratePresignedUrlResponse linodeGeneratePresignedUrlResponse = responseEntity.getBody();
String signedUrl = linodeGeneratePresignedUrlResponse.getUrl();
// Now, send the actual file.
// I am following the example provided in the AWS documentation:
// https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObjectJavaSDK.html adapt for RestTemplate
HttpHeaders headersForS3 = new HttpHeaders();
// You should provide the same content type you indicated previously
headersForS3.set("Content-Type", "application/pdf");
Resource resource = new FileSystemResource("my-object-name.pdf");
HttpEntity<byte[]> requestEntityForS3 =
new HttpEntity<>(
StreamUtils.copyToByteArray(resource.getInputStream()), headersForS3);
// You should use the same HTTP verb as indicated in
// the 'method' parameter before
restTemplate.exchange(signedUrl, HttpMethod.PUT, requestEntityForS3, Void.class);
The process for retrieving the object created is very similar:
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
String token = "your token";
headers.set(HttpHeaders.AUTHORIZATION, "Bearer" + token);
LinodeGeneratePresignedUrlRequest linodeGeneratePresignedUrlRequest =
new LinodeGeneratePresignedUrlRequest();
// Instead of PUT, indicate that you want to retrieve the object
linodeGeneratePresignedUrlRequest.setMethod("GET");
// your object name
linodeGeneratePresignedUrlRequest.setName("my-object-name.pdf");
HttpEntity<LinodeGeneratePresignedUrlRequest> requestEntity =
new HttpEntity<LinodeGeneratePresignedUrlRequest>(linodeGeneratePresignedUrlRequest, headers);
String linodeApiUrl = "https://api.linode.com/v4/object-storage/buckets/eu-central-1/my-bucket-2020/object-url";
HttpEntity<LinodeGeneratePresignedUrlResponse> responseEntity = restTemplate.exchange(linodeApiUrl, HttpMethod.POST, requestEntity, LinodeGeneratePresignedUrlResponse.class);
LinodeGeneratePresignedUrlResponse linodeGeneratePresignedUrlResponse = responseEntity.getBody();
String signedUrl = linodeGeneratePresignedUrlResponse.getUrl();
// Read the object from your bucket
byte[] objectBytes = restTemplate.getForObject(signedUrl, byte[].class);
// And use the information as you need
Files.write(Paths.get("my-object-name.pdf"), objectBytes);
Of course, if Linode provides you the appropriate credentials, you can also use the AWS SDK to interact with S3 directly.
Spring equivalent of the cURL command you've provided can be:
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
String token = "";
headers.set(HttpHeaders.AUTHORIZATION, token);
JSONObject data = new JSONObject();
data.put("cors_enabled", true);
data.put("acl", "private");
HttpEntity<String> requestEntity = new HttpEntity<String>(data.toString(), headers);
String url = "https://api.linode.com/v4/object-storage/buckets/eu-central-1/bonansa15122020/access";
HttpEntity<String> responseEntity = restTemplate.exchange(url, HttpMethod.PUT, requestEntity, String.class);
In your first example, you've haven't provided Authorization header, so you're getting a 401 response. You're aren't using the RestTemplate you'd created using RestTemplateBuilder here.
In your second example, it seems the request body isn't a JSON (you're reading the logback file, so highly unlikely there's a JSON inside it). It seems the Linode API expects a JSON body.
Update:
I believe you can use PUT request as part of POST request to endpoint https://api.linode.com/v4/object-storage/buckets/{clusterId}/{bucket}/object-url
More details here - https://developers-linode.netlify.app/api/v4/object-storage-buckets-cluster-id-bucket-object-url#post
I won't be able to test because I don't have account with linode.
I think another viable solution would be to use aws sdk for s3 to upload files to the linode endpoint.
Here is the simple example -
https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/java/example_code/s3/src/main/java/aws/example/s3/PutObject.java
Original:
Based on the linode api documentation the method should be post.
https://www.linode.com/docs/api/object-storage/#object-storage-bucket-create
restTemplate.post("https://api.linode.com/v4/object-storage/buckets", requestEntity, parameters)
Also please review the request body to conform to documentation.

Kerberos client - kerberoRestTemplate not working

Im trying to consume an api which is authenticating with Kerberos. I have referred the below spring documentation related to KerberosRestTemplate.reference link, im passing the correct keytab file and the userPrincipal values as mentioned in the reference doc. But still im receiving 401 from the server.
But when I execute the kinit command in the terminal it receives a ticket from KDC and with that, im able to execute the curl command and get a working response.
KerberosRestTemplate kerberosRestTemplate = new KerberosRestTemplate("svc_dfsd.keytab", "svc_dfsd#sswe.AD");
String url="https://wexample.com:20550/aggr_subscriber_summary_hbase/03434809824";
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.TEXT_XML));
HttpEntity<String> entity = new HttpEntity<String>("parameters", headers);
ResponseEntity<String> response = kerberosRestTemplate.exchange(url, HttpMethod.GET, entity, String.class);
Can you suggest any other better approach to do this or fix this. All your comments are highly appreciated!!!
Troubleshooting Kerberos might be tricky since the errors are often misleading and Java implementation does a lot if implicit actions (canonicalization of URLs, etc.).
I suggest trying Kerb4J library which allows you to generate the kerberos token explicitly:
SpnegoClient spnegoClient = SpnegoClient.loginWithKeyTab("svc_dfsd#sswe.AD", "svc_dfsd.keytab");
SpnegoContext context = spnegoClient.createContext("https://wexample.com"); // Will result in HTTP/wexample.com SPN
RestTemplate restTemplate = new RestTemplate();
String url="https://wexample.com:20550/aggr_subscriber_summary_hbase/03434809824";
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.TEXT_XML));
headers.add("Authorization", context.createTokenAsAuthroizationHeader());
HttpEntity<String> entity = new HttpEntity<String>("parameters", headers);
ResponseEntity<String> response = restTemplate .exchange(url, HttpMethod.GET, entity, String.class);
If default SPN resolution works for you, you can also use SpnegoRestTemplate from this Kerb4J:
SpnegoClient spnegoClient = SpnegoClient.loginWithKeyTab("svc_dfsd#sswe.AD", "svc_dfsd.keytab");
SpnegoRestTemplate spnegoRestTemplate = new SpnegoRestTemplate(spnegoClient);
String url="https://wexample.com:20550/aggr_subscriber_summary_hbase/03434809824";
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.TEXT_XML));
HttpEntity<String> entity = new HttpEntity<String>("parameters", headers);
ResponseEntity<String> response = spnegoRestTemplate.exchange(url, HttpMethod.GET, entity, String.class);
Disclaimer: I'm the author of Kerb4J

RestTemplate call returns 401 Unauthorized

Background
I am trying to consume a REST endpoint hosted on IBM Cloud API from my SpringBoot application using RestTemplate. I am using the following snippet to make the call:
RestTemplate send = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setCacheControl(CacheControl.noCache());
headers.set("x-ibm-client-id", clientId);
headers.set("x-ibm-client-secret", clientSecret);
HttpEntity<BodyEntity> httpEntity = new HttpEntity<>(bodyEntity, headers);
send.exchange(ENDPOINT_URL, HttpMethod.POST, httpEntity, Object.class);
I used the following snippet to configure RestTemplate
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder.build();
}
Problem
Using this snippet, when the call is made I receive 401 Unauthorized. When I made the same call using Postman, I received correct response from server without any problem.
Since I received 401 response code I set to further investigate the request by logging headers and body and other parts of request.
I implemented ClientHttpRequestInterceptor to log outgoing requests to further debug the issue and added this interceptor to my RestTemplate config as follows:
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
// new code
builder.interceptors(new LoggingClientHttpRequestInterceptor());
return builder.build();
}
After making the request again, I could see in the log that the outgoing call contained all details as it should e.g. Headers and Body were correct.
After this, I changed the whole thing to use Apache HTTP Client as follows:
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost(URL);
String reqString = "BODY";
httpPost.setEntity(new StringEntity(reqString, ContentType.APPLICATION_JSON));
httpPost.setHeader("accept", "application/json");
httpPost.setHeader("content-type", "application/json");
httpPost.setHeader("cache-control", "no-cache");
httpPost.setHeader("x-ibm-client-id", clientId);
httpPost.setHeader("x-ibm-client-secret", clientSecret);
CloseableHttpResponse response = httpClient.execute(httpPost);
try {
System.out.println("Response status: " + response.getStatusLine());
HttpEntity entity1 = response.getEntity();
System.out.println("Response :" + entity1.toString());
} finally {
response.close();
}
Using the snippet above, I executed the request and received correct response.
Question
Why RestTemplate call returns and error whereas HttpClient returns correct response?
Do I need to further configure RestTemplate?
What have I missed?

400 Bad Request for Spring RestTemplate and GET on Salesforce.com Connected Apps

Having an issue to create a basic API to the Salesforce.com object queries using Connected Apps and a Spring RestTemplate client. The inital authentication step using a POST operation is working fine and returns the Salesforce instance URL as well as the required access-token. The following GET operation fails with an Error 400. The URL and access-token were both validated by a Chrome POSTMAN plugin, that in combination returns a valid JSON response.
It is also worth to mention that the traditional implementation has not worked well and that "setAccept" was a later attempt to try to solve the parsing of the mediatype.
Below the code in error:
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.parseMediaType("application/json")));
headers.add("User-Agent", USER_AGENT);
headers.add("Authorization", "Bearer " + accessToken);
HttpEntity<String> request = new HttpEntity<String>(headers);
ResponseEntity<String> responseBodyForGet = restRetrieveSalesforceData
.exchange(url, HttpMethod.GET, request, String.class);
The solution found to fix the 400 bug at the Spring RestTemplate GET operation is the following:
URI targetUrl = UriComponentsBuilder.fromUriString(url).build().toUri();
HttpHeaders headers = new HttpHeaders();
headers.add("Authorization", "Bearer " + accessToken);
HttpEntity<String> request = new HttpEntity<String>(headers);
logger.info("Requesting GET from: " + url);
ResponseEntity<String> responseBodyForGet = restRetrieveSalesforceData.exchange(targetUrl,
HttpMethod.GET, request, String.class);
I've added the solution to the report posted above. Observe the use of the first statement UriComponentsBuilder, which solved the 400 issue at the GET.

How to access Spring REST API in JHipster with Spring RestTemplate

I have set up JHipster like described on its homepage with some entities. Frontend with AngularJS works great and also the API page, lets me test my services as expected.
Now I am trying to write a REST-Client using Spring's RestTemplate like this:
public List<SomeEntity> getAllEntities(){
URI uri = URI.create("http://localhost:8080/api/entities");
HttpHeaders httpHeaders = this.createHeaders("admin", "admin")
ResponseEntity<SomeEntity[]> responseEntity = restTemplate.exchange(uri, HttpMethod.GET, new HttpEntity<SomeEntity>(httpHeaders), SomeEntity[].class);
return Arrays.asList(responseEntity.getBody());
}
private HttpHeaders createHeaders(final String username, final String password ){
HttpHeaders headers = new HttpHeaders(){
{
String auth = username + ":" + password;
byte[] encodedAuth = Base64.encode(
auth.getBytes(Charset.forName("US-ASCII")) );
String authHeader = "Basic " + new String( encodedAuth );
set( "Authorization", authHeader );
}
};
headers.add("Content-Type", "application/json");
headers.add("Accept", "application/json");
return headers;
}
But this results in the following error:
[WARN] org.springframework.web.client.RestTemplate - GET request for "http://localhost:8080/api/entities" resulted in 401 (Unauthorized); invoking error handler
Now I am not sure, if and how I need to adapt my HttpHeaders or if my simple basic-auth handling approach at all is wrong.
The way you authenticate is wrong, it seems you chose session authentication when generating your app, so this requires form-based auth not http basic auth and it requires being able to store session cookie and CSRF cookie so most likely using commons http client.
Maybe choosing xauth token authentication when generating your app would be simpler.
Once you get this working you will have CORS issues as soon as your client won't run on same host as your JHipster app.

Resources