After some researching, I have discovered that there is a spring actuator project which is used to monitor and log information about the application. Here is the link that suggests how to put log in to my own custom repository for post processing. [http://www.sedooe.com/2017/08/tracing-requests-and-responses-with-spring-boot-actuator/ ]
However, the total in/out byte counts are not included. I know that I can try to wrap HttpServeletRequest and HttpServeletResponse (in the filter, pass them to the filter chain so that it can count in/out throughput whenever they are consumed or written out) which will return custom SerlvetInputStream and ServletOutputStream that can count the number of bytes.
This seems to be a common problem in business application and there should already be a solution so I wonder if there is any easier way to achieve this.
From my understanding the Spring-Boot 1.5 actuator implementation doesn't provide such a functionality. (Assuming you are talking about Spring-Boot 1.5.)
What you could do is having a look at the Micrometer project. It's the new actuator metrics "backend" for Spring-Boot 2.0. In case you are still using Spring-Boot 1.5 you can use the micrometer-spring-legacy library in your app.
Key thing to understand here is that for Spring-Boot 1.5 this is a metric backend which is living next to the "old" actuator.
What the project provides is instrumentation for embedded Jetty and Tomcat server. Allthough only the latter does provide in and out traffic metrics.
Assuming Tomcat, you'll get tomcat_global_received_bytes_total and tomcat_global_sent_bytes_total metrics over which you can reason about in your favorite monitoring system.
Related
Let me preface this with the fact that I am still very new to Apache Camel. I'm still trying to understand how it all works, and what needs to be done (and HOW to do it) to achieve a particular effect.
I am trying to develop a Spring Boot application that will use Apache Camel to handle the transmission (and possibly also receipt) of data to/from a number of possible sources and destinations. The purpose of the application is to provide a means to produce/generate network traffic, at the network application level, that will be fed into another Spring Boot application - let's call this the target. We are trying to observe and measure the effects various network loads have on the target.
We would like to be able to transmit data via a number of protocols, including: ftp, http/s, file systems (nfs), various mail protocols (smtp, pop) and data streaming protocols for voice and video. There may be other protocols added at a later time. The data itself is irrelevant, we just need to be able to transmit data via various protocols with various loads.
These applications/services will be running in a containerized environment (Docker) that will be run within our local development and test environment, as well as possibly in a cloud environment, such as AWS. We have used Docker, Ansible, Terraform and are currently working towards using Kubernetes and Istio to manage the configuration, deployment, and operation of these applications.
We need to be able to provide specific configurations of Camel routes for particular deployments.
It would appear that the preferred method to configure Camel routes is via Java DSL, rather than XML DSL. The Camel documentation and nearly every other source of information I've found have a strong bias towards using Java DSL. Examples of XML DSL route configuration are far and few.
My initial impression is that going the Java DSL route (excuse the pun), would not work well with our need to be able to deploy a Camel application with a specific route configuration. It seems like you are required to have Java DSL defined route configurations hardwired into the code.
We think that it will be easier to provide a specific route configuration via an XML file that can be included in a deployment, hence why I've been trying to investigate and experiment with XML DSL. Perhaps we are mistaken in this regard.
My question to the community is: Considering what I've described above, can the Java DSL approach be used to meet the requirements as I've described them? Can we use Java DSL in a way that allows for dynamic route configuration? Keep in mind we would not be attempting to change configuration during operation, just in the course of performing a deployment.
If Java DSL could be used for this purpose, it would be very much appreciated if pointers to documentation, examples, etc. could be provided.
For your use cases you could use XML DSL also. Anyhow below book covers most aspects Camel development with examples. In this book authors describes XML DSL use for most of java DSL examples.
https://www.manning.com/books/camel-in-action-second-edition
In below github repository you can find the source code for all the examples listed in above book.
https://github.com/camelinaction/camelinaction2
Simple tutorial and github repository for Apache Camel using Spring boot.
https://www.baeldung.com/apache-camel-spring-boot
https://github.com/eugenp/tutorials/tree/master/spring-boot-modules/spring-boot-camel
Maven Plugin for build and deployment of spring boot container application into Kubernetes cluster
https://maven.fabric8.io/
In case if your company can afford some funding for your effort look at below link which provides commercial offerings around Camel.
https://camel.apache.org/manual/latest/commercial-camel-offerings.html
Thanks
Madhu Gupta
Our team has a few projects which use the Java DSL for building routes. In order to make them dynamic, there are control structures for iterating and setting endpoints based off configurations. That works for us because the routes are basically all the same, just with different sources and sinks.
If you could dynamically add/change the XML DSL files in a way that doesn't involve redeploying your application, that might be a viable route to follow. One might, for example, change the camel.springboot.xml-routes property to point to a folder which changes as needed.
I'm new to metrics in general and especially Micrometer, so this might be a dumb question:
Micrometer describes itself on the home page as a "facade" "without vendor lock-in", "think SLF4J, but for metrics". With "built-in support for [...] Netflix Atlas". The docs say it's included in Spring Boot 2.
So what I'd expect is the ability to configure the monitoring system on start-up - just as I would with SLF4J. So this doc describes a setting management.metrics.export.atlas.enabled (among others) for Spring Boot. But even with this setting auto-wiring a MeterRegistry registry fails as follows:
Parameter 4 of constructor in [snip] required a bean of type 'io.micrometer.core.instrument.MeterRegistry' that could not be found.
Action:
Consider defining a bean of type 'io.micrometer.core.instrument.MeterRegistry' in your configuration.
Google led me to Baeldung where I read about some micrometer-registry-atlas dependency plus providing a MeterRegistrybean of type AtlasMeterRegistry. This works, but it's not what I call a "facade without vendor lock-in", but I guess I'm just doing it wrong?
How can I provide the monitoring system during runtime, switching between Atlas and any other without re-compiling?
#crusy you are actually right, but the feature is part of the Actuator module. This is not well documented and I was lucky to find the answer in the Spring Boot Gitter channnel https://gitter.im/spring-projects/spring-boot/archives/2019/01/24?at=5c4980b00721b912a5cdc02f.
You will notice that the Metrics section in the Spring Boot documentation is under Actuator: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#production-ready-metrics.
This means that in order for Micrometer to work out of the box, you need to include Actuator in your build. E.g for Gradle:
implementation ('org.springframework.boot:spring-boot-starter-actuator')
The MeterRegistry bean will be there now.
One of the definitions I found on Wikipedia says that vendor lock-in:
makes a customer dependent on a vendor for products and services,
unable to use another vendor without substantial switching costs.
Micrometer helps to unify interfaces to collect metrics (timers, gauges, counters, etc.), but there is no standard on how these metrics are shipped to backends (Atlas, Prometheus, etc.). That is the main reason why you need to define a specific library (micrometer-registry-atlas), properties, and sometimes also additional configuration.
Micrometer doesn't bring costs needed to switch to another backend to zero, but at least they are kept to minimum.
So I'm developing an application on top of Dropwizard and in one component I have to call an external rest service. I want to do it using JerseyClient.
Now there are 2 implementations available, the one from dropwizard and one from jersey. Using the Builder I have to choose between
io.dropwizard.client.JerseyClientBuilder.JerseyClientBuilder
and
org.glassfish.jersey.client.JerseyClientBuilder
Now the former requires Environment as well as JerseyClientConfiguration being passed to it.
Is there a good reason for using the dropwizard implementation over the vanilla one? What's the difference (except the timeout)?
Thank you
After some more digging and asking around, got an answer (below). It seems that it is better to use the JerseyClientBuilder that comes with Dropwizard as it is better integrated:
One can use JerseyClientConfiguration in order to configure the JerseyClientBuilder via the application configuration (service.yml file).
Passing the Environment information enables use of the managed thread pool that is integrated within Dropwizard's lifecycle, so when Dropwizard gets shut down, so does the client.
Dropwizard's Metrics get integrated into each client so you can see the latency and rate of calls for each one, as well as metrics around the thread pool sizes.
Is there a Spring Integration endpoint which connects to Gigaspaces?
As a general point, I am also interested to know what is the best documentation for using Spring together with Gigaspaces. Am surprised that there does not appear to be a lot of material written on this. Is Gigaspaces still the preferred option for scaling Spring applications, or are there better solutions?
GigaSpaces XAP using Spring as its native configuration. XAP container running Spring container internally.
Every XAP component (data grid node , data grid proxy , event handler ....) exposed via Spring. IDE integration and Unit tests done via Spring application context.
You can deploy spring app as is into XAP. XAP will scale it and make it HA.
See more:
http://docs.gigaspaces.com/xap102tut/spring-integration.html
http://docs.gigaspaces.com/sbp/spring-data.html
Is the question is "How my application based on spring-integration can easily push objects into GigaSpaces XAP?" or "How can I use the spring-integration framework from my code deployed in GigaSpaces XAP (so collocated with the data)?"
For the first question, I am unaware of any off-the-shelf end-points. But it is very easy to program: you will have to decorate your pojo (by annotation for example, to say where are indexes etc.)
You can use JMS integration (use GS XAP as a JMS broker), but I don't think it is the best way here...
For the second question, a GigaSpaces XAP application is mainly a Spring context. By default, there is no use of Spring integration, but it is very very easy to integrate as we have already in a Spring stack.
I want to deploy an application with a web interface. I want to use Spring YARN for this because that eases all the basic setup, and I can start the application with java -jar.
What steps do I have to do to:
have my application expose a web interface
have the tracking URI I get when submitting it proxy to that web interface
Unfortunately, I cannot find anything about this on the net, there is npthing on that particular issue in the Spring documentation and Google searches do not get me the correct results either.
Easiest way to do this is simply use Spring YARN Boot application model and framework is then trying to do the heavy lifting on your behalf. I actually showed a demo of this during my session at SpringOne 2GX 2014. You can find my session recording from youtube https://www.youtube.com/watch?v=qlvX7_r9aUA.
Interesting stuff for this particular feature is at the end (starting from 1:16:22) and you can see how web server address is registered into YARN resource manager and how I query it using a Spring YARN Boot CLI (around 1:32:13). Spring YARN will actually see that there is an embedded servlet context and registers it automatically. In this demo property "server.port=0" makes tomcat to choose random port which is then registered.
Code for this particular UI demo can be found from github https://github.com/SpringOne2GX-2014/JanneValkealahti-SpringYarn/tree/master/gs-yarn-rabbit. Demo was around RabbitMQ just to have some real UI functionality and not just a dummy hello world page.
There's also more up-to-date sample in https://github.com/spring-projects/spring-hadoop-samples/tree/master/boot/yarn-store-groups which doesn't have a real UI(just Boot management endpoints). Thought it's relatively easy to add Spring MVC magic there just by following normal Boot functionality(i.e. following https://spring.io/guides/gs/rest-service).
Lemmy know if this helps!