In our Spring Boot application (2.0.4.RELEASE), we use Zipkin to integrate distributed tracing.
When creating the integration manually with a 10% sampling rate, meaning with a #Configuration like this:
#Configuration
public class ZipkinConfiguration {
#Value("${grpc.zipkin.endpoint:}")
private String zipkinEndpoint;
#Bean
public SpanCustomizer currentSpanCustomizer(Tracing tracing) {
return CurrentSpanCustomizer.create(tracing);
}
#Bean
public Tracing tracing(#Value("${spring.application.name}") String serviceName) {
return Tracing.newBuilder().localServiceName(serviceName).spanReporter(spanReporter()).build();
}
private Reporter<Span> spanReporter() {
return AsyncReporter.create(sender());
}
private Sender sender() {
return OkHttpSender.create(zipkinEndpoint);
}
}
our application has a 50 percentile performance of about 19ms and a 99.9 percentile of about 90ms at around 10 requests per second.
When integrating Sleuth 2.0.2.RELEASE instead like this in gradle:
compile "org.springframework.cloud:spring-cloud-starter-sleuth:2.0.2.RELEASE"
compile "org.springframework.cloud:spring-cloud-sleuth-zipkin:2.0.2.RELEASE"
the performance drops massively to a p50 of 49ms and a p999 of 120ms.
I tried disabling the different parts of the Sleuth integration (spring.sleuth.async.enabled, spring.sleuth.reactor.enabled, etc.).
Disabling all these integrations brings the performance to p50: 25ms, p999: 103 ms. Just having Sleuth adds about 15-25% of overhead.
It turns out that the one thing with the significant impact is setting spring.sleuth.log.slf4j.enabled to false. If all other integrations are enabled, but this is disabled, the performance stays within the Sleuth overhead mentioned above, although nothing is logged.
So my question is:
Is there a way to avoid the overhead by Sleuth (compared to "manual" tracing) and especially the one done by the SLF4J integration?
The option is to disable Slf4j integration as you mentioned. When a new span / scope is created, we go through Slf4j to put data in MDC and it takes time unfortunately. Disabling that will save it.
Related
I currently have a Spring Boot based application where there is no active cache. Our application is heavily dependent on key-value configurations which we maintain in an Oracle DB. Currently, without cache, each time I want to get any value from that table, it is a database call. This is, expectedly causing a lot of overhead due to high number of transactions to the DB. Hence, the need for cache arrived.
On searching for caching solutions for SpringBoot, I mostly found links where we are caching object while any CRUD operation is performed via the application code itself, using annotations like #Cacheable, #CachePut, #CacheEvict, etc. but this is not applicable for me. I have a master data of key-value pairs in the DB, any change needs approvals and hence the access is not directly provided to the user, it is made once approved directly in the DB.
I want to have these said key-values to be loaded at startup time and kept in the memory, so I tried to implement the same using #PostConstruct and ConcurrentHashMap class, something like this:
public ConcurrentHashMap<String, String> cacheMap = new ConcurrentHashMap<>();
#PostConstruct
public void initialiseCacheMap() {
List<MyEntity> list = myRepository.findAll();
for(int i = 0; i < list.size(); i++) {
cacheMap.put(list.get(i).getKey(), list.get(i).getValue());
}
}
In my service class, whenever I want to get something, I am first checking if the data is available in the map, if not I am checking the DB.
My purpose is getting fulfilled and I am able to drastically improve the performance of the application. A certain set of transactions were earlier taking 6.28 seconds to complete, which are now completed in mere 562 milliseconds! however, there is just one problem which I am not able to figure out:
#PostConstruct is called once by Spring, on startup, post dependency injection. Which means, I have no means to re-trigger the cache build without restart or application downtime, this is not acceptable unfortunately. Further, as of now, I do not have the liberty to use any existing caching frameworks or libraries like ehcache or Redis.
How can I achieve periodic refreshing of this cache (let's say every 30 minutes?) with only plain old Java/Spring classes/libraries?
Thanks in advance for any ideas!
You can do this several ways, but how you can also achieve this is by doing something in the direction of:
private const val everyThrityMinute = "0 0/30 * * * ?"
#Component
class TheAmazingPreloader {
#Scheduled(cron = everyThrityMinute)
#EventListener(ApplicationReadyEvent::class)
fun refreshCachedEntries() {
// the preloading happens here
}
}
Then you have the preloading bits when the application has started, and also the refreshing mechanism in place that triggers, say, every 30 minutes.
You will require to add the annotation on some #Configuration-class or the #SpringBootApplication-class:
#EnableScheduling
We are using OptaPlanner(8.2.0) library in Spring Boot to solve knapsack problem using construction heuristic algorithm.
While running the application we observed that threads created by SolverManager are not getting released even after solving the problem. Because of that, performance of the application starts degrading after some time. Also, solver manager starts responding slowly of the increased thread count.
We also tried with latest version(8.17.0) but issue still persist.
Termination conditions:
<termination>
<millisecondsSpentLimit>200</millisecondsSpentLimit>
</termination>
optaplanner:
solver:
termination:
best-score-limit: 0hard/*soft
Code:
#Component
#Slf4j
public class SolutionManager {
private final SolverManager<Solution, String> solutionManager;
public SolutionManager(SolverManager<Solution, String> solutionManager) {
this.solutionManager = solutionManager;
}
public Solution getSolutionResponse(String solutionId, Solution unsolvedProblem)
throws InterruptedException, ExecutionException {
SolverJob<Solution, String> solve = solutionManager.solve(solutionId, unsolvedProblem);
Solution finalBestSolution = solve.getFinalBestSolution();
return finalBestSolution;
}
}
Thread metrics:
I wasn't able to reproduce the problem; after a load represented by solving several datasets in parallel, the number of threads drops back to the same value as before the load started.
The chart you shared doesn't clearly suggest there is a thread leak either; if you take a look at ~12:40 PM and compare it with ~2:00 PM, the number of threads actually did decrease.
Let me also add that the getFinalBestSolution() method actually blocks the calling thread until the solver finishes. If you instead use solve(ProblemId_ problemId, Solution_ problem, Consumer<? super Solution_> finalBestSolutionConsumer), this method returns immediately and the Consumer you provide is called when the solver finishes.
It looks like you might not be using OptaPlanner Spring Boot Starter.
If that's the case, upgrade to a recent version of OptaPlanner and add a dependency to optaplanner-spring-boot-starter. See the docs spring quickstart and the optaplanner-quickstarts repository (in the directory technology) for an example how to use it.
What I want?
To use caching and metrics
Why?
Faster response
Some metric data to evaluate things like: total hits, average duration, minimum duration, max duration... etc
I tried:
#CacheResult
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-cache</artifactId>
</dependency>
and
#SimplyTimed
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-metrics</artifactId>
</dependency>
Applied both of them like:
#GET
#CacheResult(cacheName = "someData")
#SimplyTimed
#Produces(MediaType.APPLICATION_JSON)
public List<String> getSome() {
return ... some data;
}
Both work as expected (on the first call)... sweet!
The thing is, because caching only runs the method on the first call, subsequent calls are handled straight through the cache and the metrics are no longer recorded.
I know that quarkus-cache extension is still on preview. As far as I know, microprofile have no business with caching.
And yes...
"Micrometer is the recommended approach to metrics for Quarkus. Use the SmallRye Metrics extension when it’s required to retain MicroProfile specification compatibility."
At this point I didn't find any objective/elegant solution through micrometer. From what I saw so far, I would have to abandon quarkus-cache and quarkus-smallrye-metrics and work manually with Caffeine and micrometer metrics.
Which brings me to the question: is there any possibility for quarkus-cache to keep metrics recording work out of the box, whether is annotation quarkus-smallrye-metrics or any other annotation metrics framework alike?
As explained before, the constraints about quarkus-cache and quarkus-smallrye-metrics "looks like" mutually exclusive. I understand that's a tough call. Please, go easy on me, ok?
Since with these libraries code generation happens at build time, sometimes it matters in which sequence you use the annotations or even in rare cases, like with lombok, the sequence of dependencies.
So as a wild guess it might be worth to try adding the metrics first:
#GET
#SimplyTimed //<-------------------------------/ like this
#CacheResult(cacheName = "someData") //<------/
#Produces(MediaType.APPLICATION_JSON)
public List<String> getSome() {
return ... some data;
}
I have a JAX-RS service backend written in Spring boot 2 and using micrometer for metrics.
By default the metrics contain very nice http metrics as http_server_requests_seconds_count.
These work great for normal GET and PUT requests. One method in my service uses JAX-RS AsyncRequest though.
#GET
#Produces(APPLICATION_JSON)
#Path("next/{position}")
public void getNext(
#PathParam("position") Long position,
#Suspended final AsyncResponse response) throws InterruptedException {
...
// Somewhere in the code in another thread I invoke
response.resume(entity);
...
}
These calls do not seem to be counted at all. Is there a way to enable counting them?
If not should I simply feed the same counters manually from my code?
I have a spring MVC rest service that returns data in XML. I would like to cache this xml response. How can I achieve this? Is it possible to do this using mvc:interceptors?
You could make this work, but I think there are better solutions.
First, if you want to use Spring MVC interceptors, you'll use the postHandle method to store something in your cache and the preHandle to check the cache and possible circumvent processing. The question is, what do you store in the cache. You would need to store the complete response. This means that you would have to easily get the full response from your ModelAndView in postHandle. This may or may not be easy, depending on how you're doing things.
You're most likely better off using a different caching mechanism all together. I recommend caching at the web server level. This is especially true if you're looking to cache in the interceptor level as that is right "next" to the web server and I don't see any benefit in re-inventing the wheel there. Apache has a cache module. So does nginx. Varnish is pretty awesome too.
I should also mention that you should not cache until you've determined that you need to (don't prematurely optimize). This is a waste of your time and effort. Secondly, when you've determined that you do have performance issues that need to be fixed (and caching is the correct solution), you should cache the right data in the right place.
Now, say you've determined that you do have a performance problem and some sort of caching is a good solution. The next thing to determine is what can be cached. If, for every URL, you return the same data, then caching at the web server (Apache, nginx, Varnish, etc.) level will be your best bet.
Often, you will have cases where two clients will hit the same URL and get different data. This is most easily seen on a site like Facebook. I see different data when I'm logged in than my friend sees. In this case, you will not be able to cache at the web server level. You will need to cache inside your application. Usually this means caching at the database level.
I couldn't disagree with the optimization part of the solution more.
Web requests are inherently slow as you're loading data from a remote location, possibly a few thousand miles away. Each call must suffer a full TCP round-trip time for at least the packets themselves, possibly the connect and fin for each request, which for connect is a three packet synchronous exchange before you start to transfer data.
US coast-to-coast latency is about 50ms on a good day, so every connection suffers a 150ms penalty, which for most implementations is incurred for every request.
Caching the response on the client-side removes this latency entirely, and if the service has correct headers on their response, is trivial. If they don't, you'll have to define a caching policy, which for the most part isn't particularly difficult. Most API calls are either real-time or not.
In my opinion, caching REST responses isn't premature optimization, it's common sense.
Don't use spring cache it is not what you need. You need to reduce load to your Server, not speed up your inner spring application execution.
Try use som HTTP-related caching strategies.
You can add one of HTTP-headers to your requests
#cache expires in 3600 seconds
cache-control: private, max-age=3600
#hash of your content
ETag: "e6811cdbcedf972c5e8105a89f637d39-gzip"
# redirect caching to any HTTP header
vary: User-Agent
Detailed description of caching techniques
Spring example
#RequestMapping (value = "/resource/1.pdf", produces = "application/octet-stream")
public ResponseEntity<InputStreamResource> getAttachement (#RequestParam (value = "id") Long fileId)
{
InputStreamResource isr = new InputStreamResource(javaInputStream);
HttpHeaders headers = new HttpHeaders();
//other headers
headers.setCacheControl("private, max-age=3600");
return new ResponseEntity<>(irs, headers, HttpStatus.OK);
}
I use this and it works with awesome speed.
Really easy to use spring + ehcache:
1)Controller:
#Cacheable("my.json")
#RequestMapping("/rest/list.json")
public ResponseEntity list(#RequestParam(value = "page", defaultValue = "0", required = false)
int pageNum,
#RequestParam(value = "search", required = false)
String search) throws IOException {
...
}
2) At ehcache.xml some like this:
<cache name="my.json" maxElementsInMemory="10000" eternal="true" overflowToDisk="false"/>
3) Configure spring. I'm using spring javaconf style:
#Configuration
#EnableCaching
public class ApplicationConfiguration {
#Bean
public EhCacheManagerFactoryBean ehCacheManagerFactoryBean() throws MalformedURLException {
EhCacheManagerFactoryBean ehCacheManagerFactoryBean = new EhCacheManagerFactoryBean();
ehCacheManagerFactoryBean.setConfigLocation(new ClassPathResource("ehcache.xml"));
return ehCacheManagerFactoryBean;
}
#Bean
#Autowired
public EhCacheCacheManager cacheManager(EhCacheManagerFactoryBean ehcache) {
EhCacheCacheManager ehCacheCacheManager = new EhCacheCacheManager();
ehCacheCacheManager.setCacheManager(ehcache.getObject());
return ehCacheCacheManager;
}
}
At the application level, I would go with a plain Java cache as EHCache. EHCache is pretty easy to integrate with methods on Spring beans. You could annotate your service methods as #Cacheable and it's done. Check it out at EHCache Spring Annotations.
At the HTTP level, Spring MVC provides a useful ETag filter. But I think it would be better if you could configure this kind of caching at the server level more than at app level.
As of Spring 3.1, you can use the #Cachable annotation. There is also support for conditional caching, and some sibling annotations like #CachePut, #CacheEvict and #Caching for more fine grained control.
Spring currently supports two different cache managers, one that is backed by a ConcurrentHashMap and one that is backed by Ehcache.
Lastly, don't forget to read the details about how to enable the annotations.