Spring Boot Actuator endpoint configuration doesn't seem to be working as expected - spring

I have a very simple spring boot application. It is just a zuul reverse proxy. there is no security or anything other than basic settings to discover our services via eureka and path mapping on a per service basis. I'm trying to prevent our actuator endpoints from being publicly exposed but still want the health check endpoint to be used for our ELB but want do not want it to report on the health of all the services it is aware of (i want it to be sensitive). While trying to figure out what properties i need to set to get the expected behavior, i am experiencing very unexpected behavior.
For example, when i set the property endpoints.sensitive=true, this DOES NOT change the default value of the health check endpoint to be sensitive. This seems to go against what the documentation says.
http://docs.spring.io/spring-boot/docs/1.4.2.RELEASE/reference/htmlsingle/#production-ready-customizing-endpoints
Likewise, you can also choose to globally set the “sensitive” flag of
all endpoints. By default, the sensitive flag depends on the type of
endpoint (see the table above). For example, to mark all endpoints as
sensitive except info:
endpoints.sensitive=true
endpoints.info.sensitive=false
In fact, when running in debug, i never see the org.springframework.boot.actuate.endpoint.EndpointProperties#isSensitive get called.
To get health endpoint to be sensitive, i need to explicitly set the property endpoints.health.sensitive=true. Oddly, when this setting is provided, now org.springframework.boot.actuate.endpoint.EndpointProperties#isSensitive gets called.
So this is great, my health check endpoint is now just reporting UP or DOWN and nothing else. But now I want the health check endpoint to be the ONLY endpoint enabled. So i set endpoints.enabled=false and endpoints.health.enabled=true which should disable all the endpoints except health. However, this does not seem to be the case. In my instance, I am able to hit /routes, /resume, /pause, /hystrix.stream, and others. I was only able to determine this when i disabled all endpoints with endpoints.enabled=false and then enabled the actuator endpoint with endpoints.actuator.enabled=true and that allowed me to hit the actuator endpoint which then reported that these endpoints were enabled.
{
"links": [
{
"rel": "self",
"href": "http://localhost:9200/actuator"
},
{
"rel": "resume",
"href": "http://localhost:9200/resume"
},
{
"rel": "pause",
"href": "http://localhost:9200/pause"
},
{
"rel": "hystrix.stream",
"href": "http://localhost:9200/hystrix.stream"
},
{
"rel": "env",
"href": "http://localhost:9200/env"
},
{
"rel": "routes",
"href": "http://localhost:9200/routes"
},
{
"rel": "health",
"href": "http://localhost:9200/health"
},
{
"rel": "refresh",
"href": "http://localhost:9200/refresh"
},
{
"rel": "restart",
"href": "http://localhost:9200/restart"
}
]
}
I would have expected to ONLY see the two endpoints I explicitly enabled.
{
"links": [
{
"rel": "self",
"href": "http://localhost:9200/actuator"
},
{
"rel": "health",
"href": "http://localhost:9200/health"
}
]
}
disabling each endpoint individually does not seem remove them from the actuator endpoint but now when attempted to access, i get a "This endpoint is disabled" message which is an improvement. I however don't seem to be able to disable the routes or `hystrix.stream* endpoints as there seems to be no configuration that exposes this ability.
All this said, I am wondering if this is the expected behavior or is this a bug?

I ran into the same problem as you described here. Please check your spring boot version first! There was a bug that global 'endpoints.sensitive' settings did NOT take effect at some specified spring boot version. (Not sure about the version number exactly. It seems a refactor regression in spring boot.)
Here are some references.
Allow global sensitive override #4419
Spring Boot Actuator: setting all endpoints as sensitive makes all accessible #4368
After I updated my spring boot to version 1.3.0 RELEASE, the setting 'endpoints.sensitive = true' works for me correctly. Hopes it will also work for you. Good luck, man.

Related

IllegalThreadStateException when starting an Apache Camel Route with resequencer

I have an Spring Boot application which uses Apache Camel for routing along with ActiveMQ as a message broker. I also use Hawtio for routing monitorization and to stop/start the routes.
Everything is working great except for one single route, the one that has a resequencer used to sort a batch of messages before processing them. I can stop this route from Hawtio, but when I try to start the route again, Hawtio gives me this error:
{
"request": {
"mbean": "org.apache.camel:context=camel,name=\"MAIL_READER\",type=routes",
"attribute": "LastError",
"type": "read"
},
"value": {
"exception": {
"routeId": "MAIL_READER",
"localizedMessage": "Failed to start route MAIL_READER because of Route(MAIL_READER)[From[activemq:queue:MAIL_READER...",
"cause": {
"localizedMessage": "java.lang.IllegalThreadStateException",
"cause": {
"localizedMessage": null,
"cause": null,
"suppressed": [],
"message": null
},
"suppressed": [],
"message": "java.lang.IllegalThreadStateException"
},
"suppressed": [],
"message": "Failed to start route MAIL_READER because of Route(MAIL_READER)[From[activemq:queue:MAIL_READER..."
},
"phase": "START"
},
"timestamp": 1608036365,
"status": 200
}
If I remove the resequencer from the route, the stop/start works fine. I can pause/resume the route with the resequencer though, without any kind of problem. This is the Camel route definition in the Spring Boot application:
from("activemq:queue:mail_reader?transacted=true")
.resequence(simple("${header.priority}")).batch().timeout(30000)
.process(mailProcessor)
.to("activemq:queue:processed_mails");
Seems like the resequencer is trying to use the original threads instead creating new ones when I send the route start order. Is there some kind of known problem with the Camel resequencer and threads? I found this, and it seems to be solved...
Thanks in advance.

How to import actuator httptrace in actuator prometheus? (actuator, spring boot, grafana)

Imagine this is my http://localhost:8080/actuator ouotput:
{
"_links": {
"self": {
"href": "http://localhost:8080/actuator",
"templated": false
},
"health": {
"href": "http://localhost:8080/actuator/health",
"templated": false
},
"prometheus": {
"href": "http://localhost:8080/actuator/prometheus",
"templated": false
},
"httptrace": {
"href": "http://localhost:8080/actuator/httptrace",
"templated": false
}
}
}
Now I've hooked up my prometheus environment to /actuator/prometheus and that works fine. I als want prometheus to read my httptrace so I also added /actuator/httptrace to my prometheus config. However this does not work. The formatting on the httptrace endpoint is in json and the formatting in prometheus is in yaml, I think I need the httptrace in the prometheus yaml. Prometheus eats the yaml just fine, the json not so much.
How can I pass my httptrace to actuator/prometheus from my spring boot project? In the end my goal is to get the timeTaken value for every request in grafana.
Spring's HttpTraceRepository exposes the recent traces on an endpoint but not in prometheus as percentiles. So I can suggest two paths:
You implement your own HttpTraceRepository that wraps the one your using (default is InMemory....) and then override the method to fire off some custom Timer from [io.micrometer.core] with the timing (https://micrometer.io/docs/concepts#_timers) which will get aggregated as percentiles if you also enable via https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics.customizing.per-meter-properties
In the end my goal is to get the timeTaken value for every request in grafana.
Just use http_server_requests_seconds_* that are captured per endpoint (not request)
http_server_requests_seconds_count
is the total number of requests your application received at this endpoint
http_server_requests_seconds_sum
is the sum of the the duration of every request your application received at this endpoint
and this great post explains how to use it in prometheus, https://tomgregory.com/spring-boot-default-metrics/

How to update postgres uri value in cf vcaps env

I have a bound Postgres service to my spring application in CF (Cloud foundry)
The VCAPS env available are as following:
"postgresql": [
{
"binding_name": null,
"credentials": {
"dbname": "JDusZ6EpE1ixbTKS",
"end_points": [
{
"host": "10.11.241.2",
"network_id": "SF",
"port": "46371"
}
],
"hostname": "10.11.241.2",
"password": "SuVzOf2m5L5oNYSG",
"port": "46371",
"ports": {
"5432/tcp": "46371"
},
"uri": "postgres://eyv6avf27X9Z55Gx:SuVzOf2m5L5oNYSG#10.11.241.2:46371/JDusZ6EpE1ixbTKS",
"username": "eyv6avf27X9Z55Gx"
},
"instance_name": "mypostgres",
"label": "postgresql",
"name": "mypostgres",
"plan": "v9.6-dev",
"provider": null,
"syslog_drain_url": null,
"tags": [
"postgresql",
"relational"
],
"volume_mounts": []
}
],
I need to modefy the value of the uri to include also the current schema, I guess it needs to be as:
"uri": "postgres://eyv6avf27X9Z55Gx:SuVzOf2m5L5oNYSG#10.11.241.2:46371/JDusZ6EpE1ixbTKS?currentSchema=mycurrentschema"
Is this something possible to do? and If not what is the best practice to assign current schema for a spring app?
Thanks in advance
You have a few options.
You can talk to your service provider, the operator of the service broker from which you are obtaining your service. The service broker is the one that sets the credentials, so you could ask them to include the schema by default.
You can create a service key with cf create-service-key. The service key is like a service binding, but free floating so it's not attached to your app. It just exists as long as the service key exists. You can then create a user provided service, with cf cups and manually set whatever credentials or uri you require for your app. The downside of this approach is that you have to do a little more work to manage the service information.
You can read the current uri into your application and modify it before creating your DataSource. This is not particularly easy if you are using Spring Cloud Connectors because it handles creating the DataSource for you. I would not recommend using SCC.
Instead you can do this with the Spring Boot CloudFoundryVcapEnvironmentPostProcessor and property place holders. See the referenced Javadoc for how that works.
The other option is to use java-cvenv. That provides you with an easy way to obtain credentials information, like the URL and use that to create your own DataSource, which allows you to make slight modifications to things like the URL, if necessary.
Hope that helps!

How to remove hypermedia elements from representations produced by Spring Data REST?

When using Spring Data for my REST API, the responses returned currently include the _links field:
{
"_embedded": {
"users": [
{
"imageUrl": "some_image_url",
"name": "some name",
"id": "57420b2a0d31bb6cef4ee8e9",
"_links": {
"self": {
"href": "http://localhost:8080/users/57420b2a0d31bb6cef4ee8e9"
},
"user": {
"href": "http://localhost:8080/users/57420b2a0d31bb6cef4ee8e9{?projection}",
"templated": true
}
}
},
...
Is there a way to produce output, such that the _links field is hidden? e.g.:
{
"_embedded": {
"users": [
{
"imageUrl": "some_image_url",
"name": "some name",
"id": "57420b2a0d31bb6cef4ee8e9",
},
...
I find that because I am exposing the id field, _links are not really necessary, and mostly just clutter up my responses.
There isn't. Hypermedia is a fundamental trait of REST APIs and Spring Data REST heavily uses it to allow you to build clients that can use the links present in the responses to navigate to related resources.
Of course you can dumb down your clients to not make use of that information but that will lead to a much tighter coupling (as you can't change the URIs on the server side anymore, your clients expects to talk to a dedicated server whereas with hypermedia you can just point it to a different server etc.).
In contrast to a lot of other self-proclaimed REST frameworks out there, one of the key aspects of the framework's design is to respect the fundamental principles in REST and explicitly leverage them. Or at least, don't create incentives to easily break them. This is clearly expressed in the reference documentation and on the project website. Find out more about key design decisions in this presentation on Spring Data REST, and this one on Domain-Driven Design & REST.

HATEOAS paths are invalid when using an API Gateway in a Spring Boot app

I have two spring boot applications where one of them is acting as an API Gateway (as discussed here Spring Example). The other which is wired into the first one is exposing a profile service using spring-data-rest (spring-data-neo4j-rest).
The first application is starting on port 8080 and is using zuul to route requests to the second as follows:
zuul:
routes:
profiles:
path: /profiles/**
url: http://localhost:8083/profiles/
This all works fine and requests to http://localhost:8080/profiles are being served from the second app. The problem though is that the HATEOAS links in the response are incorrect. The response from calling that second service are correct:
{
"_links": {
"self": {
"href": "http://localhost:8083/profiles{?page,size,sort}",
"templated": true
},
"search": {
"href": "http://localhost:8083/profiles/search"
}
},
"_embedded": {
"profiles": [
{
"name": "Andrew Rutter",
"_links": {
"self": {
"href": "http://localhost:8083/profiles/0"
}
}
},
{
"name": "Andrew Rutter",
"_links": {
"self": {
"href": "http://localhost:8083/profiles/1"
}
}
}
]
},
"page": {
"size": 20,
"totalElements": 2,
"totalPages": 1,
"number": 0
}
}
But when this comes back to my API Gateway, the links are being rewritten to
{
"name": "Andrew Rutter",
"_links": {
"self": {
"href": "http://localhost:8080/profiles/profiles/0"
}
}
}
Which is the gateway path alias plus the actual service base Uri. Am I missing a zuul option to disable that behavior and just leave the hateoas uri in place with a host adjustment. Or is there a way for my service behind the gateway to be wired to / rather then the default resource endpoint of /profiles (in this case) which would avoid the undesirable path being added in.
Thanks!
Zuul or Spring-Cloud adds the "X-Forwarded-Host" header to all the forwarded requests, which Spring-hateoas respects and modifies the links appropriately. To quote from Spring-Cloud docs:
The X-Forwarded-Host header added to the forwarded requests by
default. To turn it off set zuul.addProxyHeaders = false. The prefix
path is stripped by default, and the request to the backend picks up a
header "X-Forwarded-Prefix" ("/myusers" in the examples above).
You can try the recommended fix, which is to set the zuul.addProxyHeaders=false
I had exactly the same problem. Change your config as follows:
zuul:
routes:
profiles:
path: /profiles/**
url: http://localhost:8083
stripPrefix: false
This routes all requests going to the gateway matching "/profiles/**" to your back end server "http://localhost:8083" and leaves the prefix (in your case "/profiles" since that's what matched the route).
Zuul forwards to the /profiles contextPath.
Try setting this as configuration:
zuul:
routes:
profiles:
path: /profiles/**
url: http://localhost:8083/
After struggling some time with the same problem, finally I've tried zuul.addProxyHeaders = true and it works!
Links are not broken anymore.
In the demo app I used for SpringOne in my talk about Spring Data REST, I have the following configuration to both handle URI rewrites as well as adjust prefix headers set properly.
zuul:
routes:
api:
path: /api/**
serviceId: spring-a-gram-backend
stripPrefix: false
files:
path: /files/**
serviceId: spring-a-gram-mongodb-fileservice
stripPrefix: false
See it in full at https://github.com/gregturn/spring-a-gram/blob/master/spring-a-gram-frontend/src/main/resources/application.yml#L23-L32

Resources