I recently encountered some unexpected behaviour and was wondering if this was in fact intended functionality when using WebClient. With a client config as shown below, the uri variables in the template are being overridden with the fields from a POJO used as an arg to bodyValue.
class ExampleRequest {
private String namespace = "jar";
private String service = "zoo";
...
}
...
this.client = WebClient.builder()
.baseUrl("http://{service}.{namespace}.svc.cluster.local")
.defaultUriVariables(Map.of("service", "foo", "namespace", "bar"))
.build();
this.client.post()
.uri("/baz")
.bodyValue(new ExampleRequest())
.retrieve()
...
The above ends up calling out to http://zoo.jar.svc.cluster.local/baz, not http://foo.bar.svc.cluster.local/baz. I wasn't expecting the body of the message in this instance to be picked up as uri variables. :confused:
This seems strange to me, but if this is expected could point me to the source/docs of where this expansion is taking place..?
Could you please use standard HashMap implementation and recheck? I tested locally for me it is working as expected, I mean foo and bar chosen:
Map<String, String > params = new HashMap<>();
params.put("service", "foo");
params.put("namespace", "bar");
WebClient client = WebClient.builder()
.baseUrl("http://{service}.{namespace}.svc.cluster.local")
.defaultUriVariables(params)
.build();
Related
I am trying to fetch live data from NSE options trading. Below code is not working and the request made is stuck without any response.
Any workaround on this?
public void getLiveBankNiftyData() {
String RESOURCE_PATH = "https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY";
ResponseEntity<Object[]> responseEntity = restTemplate.getForEntity(RESOURCE_PATH, Object[].class);
Object[] objects = responseEntity.getBody();
}
i tried this
// request url
String url = "https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY";
// create an instance of RestTemplate
RestTemplate restTemplate = new RestTemplate();
// make an HTTP GET request
String json = restTemplate.getForObject(url, String.class);
// print json
System.out.println(json);
I found a way out. Instead of using RestTemplate I used WebClient and this solved the issue.
i'm trying to send a POST request with body data as described here: https://scrapyrt.readthedocs.io/en/stable/api.html#post.
Here's what i've tried to do but it gives me HTTP code 500
String uri = "http://localhost:3000";
WebClient webClient = WebClient.builder()
.baseUrl(uri)
.build();
LinkedMultiValueMap map = new LinkedMultiValueMap();
String q = "\"url\": \"https://blog.trendmicro.com/trendlabs-security-intelligence\",\"meta\":{\"latestDate\" : \"18-05-2020\"}}";
map.add("request", q);
map.add("spider_name", "blog");
BodyInserter<MultiValueMap<String, Object>, ClientHttpRequest> inserter2
= BodyInserters.fromMultipartData(map);
Mono<ItemsList> result = webClient.post()
.uri(uriBuilder -> uriBuilder
.path("/crawl.json")
.build())
.body(inserter2)
.retrieve()
.bodyToMono(ItemsList.class);
ItemsList tempItems = result.block();
Here's what i've tried to do but it gives me HTTP code 500
Most likely because you're sending the wrong data in a mixture of wrong formats with the wrong type:
You're using multipart form data, not JSON
You're then setting the request parameter as a JSON string (q)
The JSON string you're using in q isn't even valid (it's at least missing an opening curly brace) - and handwriting JSON is almost universally a bad idea, leverage a framework to do it for you instead.
Instead, the normal thing to do would be to create a POJO structure that maps to your request, so:
public class CrawlRequest {
private CrawlInnerRequest request;
#JsonProperty("spider_name")
private String spiderName;
//....add the getters / setters
}
public class CrawlInnerRequest {
private String url;
private String callback;
#JsonProperty("dont_filter")
private String dontFilter;
//....add the getters / setters
}
...then simply create a CrawlRequest, set the values as you wish, then in your post call use:
.body(BodyInserters.fromValue(crawlRequest))
This is a rather fundamental, basic part of using a WebClient. I'd suggest reading around more widely to give yourself a better understanding of the fundamentals, it will help tremendously in the long run.
For me following code worked:
public String wcPost(){
Map<String, String> bodyMap = new HashMap();
bodyMap.put("key1","value1");
WebClient client = WebClient.builder()
.baseUrl("domainURL")
.build();
String responseSpec = client.post()
.uri("URI")
.headers(h -> h.setBearerAuth("token if any"))
.body(BodyInserters.fromValue(bodyMap))
.exchange()
.flatMap(clientResponse -> {
if (clientResponse.statusCode().is5xxServerError()) {
clientResponse.body((clientHttpResponse, context) -> {
return clientHttpResponse.getBody();
});
return clientResponse.bodyToMono(String.class);
}
else
return clientResponse.bodyToMono(String.class);
})
.block();
return responseSpec;
}
I've got a spring boot application that is defining a RestTemplate bean as follows:
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder.build();
}
Also, pulling in spring-boot-starter-actuator and io.micrometer:micrometer-registry-prometheus.
When I use the injected RestTemplate as follows:
#Autowired
private RestTemplate restTemplate;
private String uriTemplate = "http://my.domain.com/bookstore-api/books";
public List<Book> getBooksByAuthor(String author) {
// create URI for "http://my.domain.com/bookstore-api/books?author={authorId}"
UriComponentsBuilder uriComponentsBuilder = UriComponentsBuilder
.fromUriString(uriTemplate)
.queryParam("author", author);
// make the GET
ResponseEntity<Book[]> responseEntity = restTemplate.getForEntity(uriComponentsBuilder.toUriString(),Book[].class);
// rest ommitted for brevity
}
When getBooksByAuthor("Tolkien") is called, we can then hit /metrics/prometheus and see the following:
http_client_requests_seconds_count{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author=Tolkien",} 2.0
http_client_requests_seconds_sum{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author=Tolkien",} 0.253227898
This would be fine, except that there are lots of authors out there, and eventually I will get the "too many tags" exception.
I would prefer to have the following (similar to how path variables get templated):
http_client_requests_seconds_count{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author={author}",} 2.0
http_client_requests_seconds_sum{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author={author}",} 0.253227898
Is this possible to achieve by modifying the way I use UriComponentsBuilder? The closest thing I've found is to define my own RestTemplateExchangeTagsProvider, and override the default to do some crazy regex replacement.
Just fixed same issue in SpringBoot 2.4.5 using:
responseEntity = restTemplate.exchange(
config.getDataUrl(),
HttpMethod.GET,
httpEntity,
new ParameterizedTypeReference<String>() {},
rowId);
where getDataUrl resolves to:
https://data-service-dev.apps.cloud.net/api/hbase/getData?rowId={rowId}
metrics:
http_client_requests_seconds_count{clientName="data-service-dev.apps.cloud.net",method="GET",outcome="SUCCESS",status="200",uri="/api/hbase/getData?rowId={rowId}",} 1.0
...
I have had a same problem. Maybe this information will help you.
In my case restTemplate.setUriTemplateHandler(handler) had overwritten annonymous
MetricsClientHttpRequestInterceptor#createUriTemplateHandler.
And the original url templates had not been stored into memory for prometheus.
DefaultUriBuilderFactory builderFactory = new DefaultUriBuilderFactory();
builderFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
customizer.customize(restTemplate);
restTemplate.setUriTemplateHandler(handler);
So,
I changed order of the commands:
restTemplate.setUriTemplateHandler(handler);
customizer.customize(restTemplate);
Please check that there are no setting commands for restTemplate after MetricsClientHttpRequestInterceptor.customize(restTemplate).
I'm trying to POST a multipart/form-data using Spring RestTemplate with a byte array as the file to upload and it keeps failing (Server rejects with different kinds of errors).
I'm using a MultiValueMap with ByteArrayResource. Is there something I'm missing?
Yes there is something missing.
I have found this article:
https://medium.com/#voziv/posting-a-byte-array-instead-of-a-file-using-spring-s-resttemplate-56268b45140b
The author mentions that in order to POST a byte array using Spring RestTemplate one needs to override getFileName() of the ByteArrayResource.
Here is the code example from the article:
private static void uploadWordDocument(byte[] fileContents, final String filename) {
RestTemplate restTemplate = new RestTemplate();
String fooResourceUrl = "http://localhost:8080/spring-rest/foos"; // Dummy URL.
MultiValueMap<String, Object> map = new LinkedMultiValueMap<String, Object>();
map.add("name", filename);
map.add("filename", filename);
// Here we
ByteArrayResource contentsAsResource = new ByteArrayResource(fileContents) {
#Override
public String getFilename() {
return filename; // Filename has to be returned in order to be able to post.
}
};
map.add("file", contentsAsResource);
// Now you can send your file along.
String result = restTemplate.postForObject(fooResourceUrl, map, String.class);
// Proceed as normal with your results.
}
I tried it and it works!
I added an issue to send a request from java client to Python service in FastApi and sending a ByteArrayResource instaead of simple byte[] fixed the issue.
FastAPI server returned: "Expected UploadFile, received: <class 'str'>","type":"value_error""
I am using spring webclient to make a Facebook graph api request with url containing {comment_count}
But, getting this exception
java.lang.IllegalArgumentException: Not enough variable values available to expand reactive spring
Code Snippet :
import org.springframework.stereotype.Component;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;
#Component
public class Stackoverflow {
WebClient client = WebClient.create();
public Mono<Post> fetchPost(String url) {
// Url contains "comments{comment_count}"
return client.get().uri(url).retrieve()
.bodyToMono(Post.class);
}
}
I know the solution with resttemplate, But i need to use spring webclient.
You can create your URL using UriComponentsBuilder as follows
webClient.get().uri(getFacebookGraphURI(3)).retrieve().bodyToMono(Object.class);
private URI getFacebookGraphURI(int comments){
return UriComponentsBuilder.fromHttpUrl("https://graph.facebook.com")
.pathSegment("v3.2", "PAGE_ID", "posts").queryParam("fields", "comments{comment_count}")
.queryParam("access_token", "acacaaac").build(comments);
}
OR
int commentsCount = 3; webClient.get().uri(UriComponentsBuilder.fromHttpUrl("https://graph.facebook.com")
.pathSegment("v3.2", "PAGE_ID", "posts").queryParam("fields", "comments{comment_count}")
.queryParam("access_token", "acacaaac").build().toString(),commentsCount).retrieve().bodyToMono(Object.class);
The solution I use is to disable the encoding in DefaultUriBuilderFactory
val urlBuilderFactory = DefaultUriBuilderFactory("https://foo.bar.com").apply {
encodingMode = EncodingMode.NONE
}
val wc = wcb
.clone()
.uriBuilderFactory(urlBuilderFactory)
.build()
It's in Kotlin, in Java you just have to use DefaultUriBuilderFactory#setEncodingMode(EncodingMode) with NONE as parameter.
Due to this change of behavior, you have to encode your query params yourself. To do so, I use the
val query = URLEncoder.encode(query_as_string, StandardCharsets.UTF_8.toString())
And I can perform call like this:
wc.get()
.uri { it
.path(graphqlEndpoints)
.queryParam("variables", query)
.build()
}
.retrieve()
.bodyToFlux<String>()
// ...
For Java (as per comment by Anurag below) it will be something like this:
var uriBuilderFactory = new DefaultUriBuilderFactory(baseUrl);
uriBuilderFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
var wc = WebClient.builder()
.baseUrl(baseUrl)
.uriBuilderFactory(uriBuilderFactory)
// ... other options
.build();
wc.get()
.uri(uriBuilder -> uriBuilder
.path(path)
// value still needs to be URL-encoded as needed (e.g., if value is a JSON
.queryParam("param", value))
.build())
.retrieve()
.bodyToMono(...);
As noted above, you will still need to URL-encode the parameter value. With this approach, you are merely avoiding "double URL encoding" by replacing the urlBuilderFactory.