#CacheEvict is not working in SpringBoot - spring

#Cacheable(value = "apis", key = "#request")
public Object queryCenterAPI(QCRequest request,HttpHeaders headers) throws JSONException, ParseException {
RestTemplate restTemplate = new RestTemplate();
restTemplate.setErrorHandler(new ToolsResponseHandler());
Response res=new Response();
HashMap<String,String> map=new HashMap<String,String>();
logger.info("No Caching^^^^^^^^^^");
Gson gson = new Gson();
String requestJson = gson.toJson(request);
HttpEntity<String> requestEntity = new HttpEntity<String>(requestJson, headers);
System.out.println("Request Body "+requestEntity);
Object response = null;
try {
response = restTemplate.postForObject(QCUtils.queryURL, requestEntity, Object.class);
logger.info("1st response>"+response);
response = response.toString().replaceAll("\\\\", "");
System.out.println("Final response "+response);
}catch (HttpClientErrorException httpEx) {
logger.info("Error:"+httpEx);
}
return response;
}
#CacheEvict(value = "apis", key = "#request")
public void resetOnRequest(QCRequest request) {
// Intentionally blank
System.out.println("Evict in Progrsss......");
}
Caching is working fine but I am unable to use #CacheEvict annotation.I guess cacheEvict method is called immediately after Cacheable method.
The resetOnRequest() method is not called after Cachable method(queryCenterAPI).

Cache evict method should be called from a different class otherwise it will not work, same for method with #Cacheable annotation.

Another different way instead the #CacheEvict annotation, would be using the CacheManager > https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/cache/CacheManager.html
For single value;
public void evictSingleCacheValue(String cacheName, String cacheKey) {
cacheManager.getCache(cacheName).evict(cacheKey);
}
For all values;
public void evictAllCacheValues(String cacheName) {
cacheManager.getCache(cacheName).clear();
}

Using ehcache works for me andAdding the below Xml file and Config file is enough to do caching and cache evict.
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="ehcache.xsd"
updateCheck="true"
monitoring="autodetect"
dynamicConfig="true">
<diskStore path="java.io.tmpdir" />
<cache name="apis"
eternal="false"
maxEntriesLocalHeap="10000"
maxEntriesLocalDisk="1000"
diskSpoolBufferSizeMB="20"
timeToIdleSeconds="200" timeToLiveSeconds="900"
memoryStoreEvictionPolicy="LFU"
transactionalMode="off">
</cache>
public class AppConfig {
#Bean
public CacheManager cacheManager() {
return new EhCacheCacheManager(ehCacheCacheManager().getObject());
}
#Bean
public EhCacheManagerFactoryBean ehCacheCacheManager() {
EhCacheManagerFactoryBean cmfb = new EhCacheManagerFactoryBean();
cmfb.setConfigLocation(new ClassPathResource("ehcache.xml"));
cmfb.setShared(true);
return cmfb;
}
}
We only need to configure in xml file and give the time after which cache eviction take place.

I think you will need to switch to the "aspectj" mode in order for the #CacheEvict to work fine.
From Spring documentation:
The default advice mode for processing caching annotations is "proxy"
which allows for interception of calls through the proxy only; local
calls within the same class cannot get intercepted that way. For a
more advanced mode of interception, consider switching to "aspectj"
mode in combination with compile-time or load-time weaving.
Second option would be to try to move the #CacheEvict method into another class.

Related

Spring cloud open Feign Not ignoring null Values while Encoding

I am working on a Spring boot application. We are using Spring cloud open Feign for making rest calls. We are using the default GsonEncoder(), but for some reason gson is not excluding the null properties while encoding the payload.
Config:
return Feign.builder()
.options(ApiOptions())
.encoder(new GsonEncoder())
.decoder(new GsonDecoder())
.target(ApiClient.class, "URL");
Client:
#FunctionalInterface
#FeignClient(
value = "apiTest",
url = "urlHere"
)
public interface ApiClient {
#PostMapping("path/to/service")
AiResponse getDetails(ApiRequest apiRequest);
}
ApiRequest.java:
public class ApiRequest {
private String userName;
private String userId;
private String password;
//#setters and getters
}
But while making the request, the Request Body is :
{
"userName" : "test,
"userId" : null,
"password": "password"
}
My Understanding is that Gson should automatically remove the null while serializing the Request Body. But i can see null properties exist in request.
I even tried with Custom Encoders (Jackson from below):
https://github.com/OpenFeign/feign/blob/master/jackson/src/main/java/feign/jackson/JacksonEncoder.java
As per below, it should not include null while serialzing the requestBody, but still i can see null values being passed in request.
https://github.com/OpenFeign/feign/blob/master/jackson/src/main/java/feign/jackson/JacksonEncoder.java#L39
Below are the dependencies:
Spring clou version : 2020.0.2
org.springframework.cloud:spring-cloud-starter-openfeign
io.github.openfeign:feign-gson:9.5.1
Any suggestions would be really helpful. Thanks in Advance.!
You need to provide OpenFeign with custom Encoder to let it skip nulls.
In my case this beans helps me.
#Configuration
public class ExternalApiConfiguration {
#Bean
public Feign.Builder feignBuilder() {
return Feign.builder()
.contract(new SpringMvcContract());
}
#Bean("feignObjectMapper")
public ObjectMapper feignObjectMapper() {
final ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
return objectMapper;
}
#Bean
public Encoder feignEncoder(#Qualifier("feignObjectMapper") ObjectMapper objectMapper) {
HttpMessageConverter jacksonConverter = new MappingJackson2HttpMessageConverter(objectMapper);
ObjectFactory<HttpMessageConverters> objectFactory = () -> new HttpMessageConverters(jacksonConverter);
return new SpringEncoder(objectFactory);
}
And use this configuration if FeignClient
#FeignClient(configuration = ExternalApiConfiguration.class)

Streaming upload via #Bean-provided RestTemplateBuilder buffers full file

I'm building a reverse-proxy for uploading large files (multiple gigabytes), and therefore want to use a streaming model that does not buffer entire files. Large buffers would introduce latency and, more importantly, they could result in out-of-memory errors.
My client class contains
#Autowired private RestTemplate restTemplate;
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
int REST_TEMPLATE_MODE = 1; // 1=streams, 2=streams, 3=buffers
return
REST_TEMPLATE_MODE == 1 ? new RestTemplate() :
REST_TEMPLATE_MODE == 2 ? (new RestTemplateBuilder()).build() :
REST_TEMPLATE_MODE == 3 ? restTemplateBuilder.build() : null;
}
and
public void upload_via_streaming(InputStream inputStream, String originalname) {
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
InputStreamResource inputStreamResource = new InputStreamResource(inputStream) {
#Override public String getFilename() { return originalname; }
#Override public long contentLength() { return -1; }
};
MultiValueMap<String, Object> body = new LinkedMultiValueMap<String, Object>();
body.add("myfile", inputStreamResource);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body,headers);
String response = restTemplate.postForObject(UPLOAD_URL, requestEntity, String.class);
System.out.println("response: "+response);
}
This is working, but notice my REST_TEMPLATE_MODE value controls whether or not it meets my streaming requirement.
Question: Why does REST_TEMPLATE_MODE == 3 result in full-file buffering?
References:
How to forward large files with RestTemplate?
How to send Multipart form data with restTemplate Spring-mvc
Spring - How to stream large multipart file uploads to database without storing on local file system -- establishing the InputStream
How to autowire RestTemplate using annotations
Design notes and usage caveats, also: restTemplate does not support streaming downloads
In short, the instance of RestTemplateBuilder provided as an #Bean by Spring Boot includes an interceptor (filter) associated with actuator/metrics -- and the interceptor interface requires buffering of the request body into a simple byte[].
If you instantiate your own RestTemplateBuilder or RestTemplate from scratch, it won't include this by default.
I seem to be the only person visiting this post, but just in case it helps someone before I get around to posting a complete solution, I've found a big clue:
restTemplate.getInterceptors().forEach(item->System.out.println(item));
displays...
org.SF.boot.actuate.metrics.web.client.MetricsClientHttpRequestInterceptor
If I clear the interceptor list via setInterceptors, it solves the problem. Furthermore, I found that any interceptor, even if it only performs a NOP, will introduce full-file buffering.
public class SimpleClientHttpRequestFactory { ...
I have explicitly set bufferRequestBody = false, but apparently this code is bypassed if interceptors are used. This would have been nice to know earlier...
#Override
public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException {
HttpURLConnection connection = openConnection(uri.toURL(), this.proxy);
prepareConnection(connection, httpMethod.name());
if (this.bufferRequestBody) {
return new SimpleBufferingClientHttpRequest(connection, this.outputStreaming);
}
else {
return new SimpleStreamingClientHttpRequest(connection, this.chunkSize, this.outputStreaming);
}
}
public abstract class InterceptingHttpAccessor extends HttpAccessor { ...
This shows that the InterceptingClientHttpRequestFactory is used if the list of interceptors is not empty.
/**
* Overridden to expose an {#link InterceptingClientHttpRequestFactory}
* if necessary.
* #see #getInterceptors()
*/
#Override
public ClientHttpRequestFactory getRequestFactory() {
List<ClientHttpRequestInterceptor> interceptors = getInterceptors();
if (!CollectionUtils.isEmpty(interceptors)) {
ClientHttpRequestFactory factory = this.interceptingRequestFactory;
if (factory == null) {
factory = new InterceptingClientHttpRequestFactory(super.getRequestFactory(), interceptors);
this.interceptingRequestFactory = factory;
}
return factory;
}
else {
return super.getRequestFactory();
}
}
class InterceptingClientHttpRequest extends AbstractBufferingClientHttpRequest { ...
The interfaces make it clear that using InterceptingClientHttpRequest requires buffering body to a byte[]. There is not an option to use a streaming interface.
#Override
public ClientHttpResponse execute(HttpRequest request, byte[] body) throws IOException {

Combining #SqsListener and #RequestMapping

We're currently in the middle of migrating our current architecture into Spring-AWS-based microservices. One of my tasks is to research on how our microservices communicate with one another. I'm aiming to set-up a hybrid system of RESTful HTTP endpoints and SQS producers and consumers.
As an example, I have the below code:
#SqsListener("request_queue")
#SendTo("response_queue")
#PostMapping("/send")
public Object send(#RequestBody Request request, #Header("SenderId") String senderId) {
if (senderId != null && !senderId.trim().isEmpty()) {
logger.info("SQS Message Received!");
logger.info("Sender ID: ".concat(senderId));
request = new Gson().fromJson(payload, Request.class);
}
Response response = processRequest(request); // Process request
return response;
}
Theoretically, this method should be able to handle the following:
Receive a Request object via HTTP
Continually poll the request_queue for a message containing the Request object
As an HTTP endpoint, the method returns no error. However, as an SQS listener, it runs into the following exception:
org.springframework.messaging.converter.MessageConversionException:
Cannot convert from [java.lang.String] to [com.oriente.salt.Request] for
GenericMessage [payload={"source":"QueueTester","message":"This is a wonderful
message send by queue from Habanero to Salt. Spicy.","msisdn":"+639772108550"},
headers={LogicalResourceId=salt_queue, ApproximateReceiveCount=1,
SentTimestamp=1523444620218, ....
I've tried to annotate the Request param with #Payload, but to no avail. Currently I've also set-up the AWS config via Java, as seen below:
ConsuerAWSSQSConfig.java
#Configuration
public class ConsumerAWSSQSConfig {
#Bean
public SimpleMessageListenerContainer simpleMessageListenerContainer() {
SimpleMessageListenerContainer msgListenerContainer = simpleMessageListenerContainerFactory()
.createSimpleMessageListenerContainer();
msgListenerContainer.setMessageHandler(queueMessageHandler());
return msgListenerContainer;
}
#Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory() {
SimpleMessageListenerContainerFactory msgListenerContainerFactory = new SimpleMessageListenerContainerFactory();
msgListenerContainerFactory.setAmazonSqs(amazonSQSClient());
return msgListenerContainerFactory;
}
#Bean
public QueueMessageHandler queueMessageHandler() {
QueueMessageHandlerFactory queueMsgHandlerFactory = new QueueMessageHandlerFactory();
queueMsgHandlerFactory.setAmazonSqs(amazonSQSClient());
QueueMessageHandler queueMessageHandler = queueMsgHandlerFactory.createQueueMessageHandler();
List<HandlerMethodArgumentResolver> list = new ArrayList<>();
HandlerMethodArgumentResolver resolver = new PayloadArgumentResolver(new MappingJackson2MessageConverter());
list.add(resolver);
queueMessageHandler.setArgumentResolvers(list);
return queueMessageHandler;
}
#Lazy
#Bean(name = "amazonSQS", destroyMethod = "shutdown")
public AmazonSQSAsync amazonSQSClient() {
AmazonSQSAsync awsSQSAsync = AmazonSQSAsyncClientBuilder.standard().withRegion(Regions.AP_SOUTHEAST_1).build();
return awsSQSAsync;
}
}
What do you guys think?

hecache is always returning the same result

I'm developing a web service system. I'm using Spring framework and hecache.
the problem is that when i do my first request to the URL I get the correct result but when I do a second request with another parameter it still returning the same result as the first request. When the cache expires it returns the correct result the first time.
This is my Spring Service method
#Cacheable(value="getUserInformation", key="#progid")
public Object getUserInformation(String login, String progid, HttpServletRequest request){
System.out.println( login.concat(progid) );
try {
SimpleJdbcCall simpleJdbcCall = new SimpleJdbcCall(jdbcTemplate).
withCatalogName("siv_pck_general_functions").
withFunctionName("fn_get_user_information");
SqlParameterSource out = new MapSqlParameterSource().addValue("p_loginname", login).addValue("p_programid", progid);
map.put("Result", simpleJdbcCall.executeFunction(List.class, out) );
logger.info( LogUtils.getTypeMessage(request.getRemoteAddr(), request.getRequestURI(), LogUtils.INFO ));
} catch (Exception e) {
map.put("Result","Error");
}
return map;
}
This is my ecache configuration
<cache name="getUserInformation"
maxEntriesLocalHeap="10000"
maxEntriesLocalDisk="1000"
eternal="false"
diskSpoolBufferSizeMB="20"
timeToIdleSeconds="300"
timeToLiveSeconds="600"
memoryStoreEvictionPolicy="LFU"
transactionalMode="off">
<persistence strategy="localTempSwap" />
</cache>
This is my Web Service mapping
#RequestMapping( value="/getUserInformation/{loginName}/{programId}/{token}", method = RequestMethod.GET, produces="application/json;charset=UTF-8" )
public Object getUserInformation( #PathVariable String loginName, #PathVariable String programId, #PathVariable String token,HttpServletRequest request){
httpServletRequest = request;
String result = validateToken(token);
if ( "OK".equals(result) ){
map.put("Result", service.getUserInformation(loginName, programId, request) );
}else{
map.put("Result",result);
}
return map;
}

Hystrix-javanica #fallbackMethod Last Cached Response

I'm looking to do something like the following:
#HystrixCommand(fallbackMethod = "returnLastGoodCachedCopy")
void performRequest(String someArg) {
// Make HTTP request using RestTemplate configured with EHCache
}
void returnLastGoodCachedCopy(String someArg) {
// Return the last successful response
}
For a little more background I am using Spring Boot, and setting up the RestTemplate like so to use EHCache:
#Bean
public RestTemplate restTemplate() {
return new RestTemplate(clientHttpRequestFactory());
}
#Bean
public CloseableHttpClient httpClient() {
CacheConfig cacheConfig = CacheConfig.custom()
.setMaxCacheEntries(this.httpClientProperties.getMaxCacheEntries())
.setMaxObjectSize(this.httpClientProperties.getMaxObjectSize())
.setHeuristicCachingEnabled(this.httpClientProperties.getHeuristicLifetimeEnabled())
.setHeuristicDefaultLifetime(this.httpClientProperties.getHeuristicLifetimeSeconds()).build();
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.httpClientProperties.getConnectTimeout())
.setSocketTimeout(this.httpClientProperties.getSocketTimeout()).build();
Ehcache httpEhcache = (Ehcache) this.ehCacheManager.getCache(httpClientProperties.getCacheName())
.getNativeCache();
EhcacheHttpCacheStorage ehcacheHttpCacheStorage = new EhcacheHttpCacheStorage(httpEhcache);
CloseableHttpClient cachingClient = CachingHttpClients.custom().setCacheConfig(cacheConfig)
.setHttpCacheStorage(ehcacheHttpCacheStorage).setDefaultRequestConfig(requestConfig).build();
return cachingClient;
}
private ClientHttpRequestFactory clientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory(httpClient());
return factory;
}
So my one thought was to use the same request EHCache, but I'm not sure that's an appropriate solution considering that cache is based on cache-control headers, which I'd like to be separated from so I can return a valid response regardless if it is expired or not.
My other thought was to configure a separate EHCache cache, and store the responses myself, then I can access those more easily considering I set the format of the key and the value.
Before I go down this path I want to see if there is already anything built into Hystrix that handles this situation, or if there are any other recommended approaches.

Resources