Spring Cloud Configuration Server leaks GIT environment via actuator endpoints - spring

I am setting up a Spring Cloud Configuration Server. Just few dependencies and an annotation. The source of properties comes from git. Server has the actuator enabled with default basic settings. I am surprised that the actuator unexpectedly reacts to any (even nonexisting endpoints) and reveals full environment (git property source) which is also used to store secrets.
pom dependencies:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.3</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>cz.leveland</groupId>
<artifactId>actutest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>actutest</name>
<description>Actuator test</description>
<properties>
<java.version>11</java.version>
<spring-cloud.version>2021.0.3</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
application.properties:
server:
port: 8080
spring:
application:
name: CONFIG-SERVER
cloud:
config:
server:
git:
uri: https://bitbucket.org/repo-name/actuator-test
clone-on-start: true
username: repouser
password: xxxxxxxxxx
default-label: master
encrypt:
keyStore:
location: classpath:/server2.jks
password: letmein
alias: mytestkey
secret: letmein
management:
endpoints:
web:
exposure:
include: "health"
Spring application:
#EnableConfigServer
#SpringBootApplication
public class ActutestApplication {
public static void main(String[] args) {
SpringApplication.run(ActutestApplication.class, args);
}
}
git application.properties contains encoded password:
spring.datasource.username=admin
spring.datasource.password={cipher}AQA50Mh4...
NOW THE PROBLEM
The server responds to ANY actuator endpoint like .../actuator/foo-bar and always returns the full git property source (example bellow).
When I remove #EnableConfigServer annotation the actuator starts working as expected. So this "feature" must be activated with spring cloud config server.
Server response to .../actuator/foo-bar:
{
"name": "actuator",
"profiles": [
"foo-bar"
],
"label": null,
"version": "da200e047354e889e6503b10cbb9cbbc7e3dbb28",
"state": null,
"propertySources": [
{
"name": "https://bitbucket.org/repo-name/actuator-test/application.properties",
"source": {
"spring.datasource.username": "admin",
"spring.datasource.password": "secret-password"
}
}
]
}
I must be doing something terribly wrong or is this a security bug?
Thank you for helping me.

Test project https://github.com/Klapsa2503/actuator-test
Actuator metrics not working
Change
management:
endpoints:
web:
exposure:
include: "health"
to
management:
endpoints:
web:
exposure:
include: "health,metrics"
so metrics are exposed and http://localhost:8080/actuator/metrics working
Endpoint leaking properties
By default spring config is exposing default properties from application.properties from your config repository. Spring config server has a strict naming convention that you should follow to prevent that. See https://www.baeldung.com/spring-cloud-configuration
Just change application.yml to something different and those properties will not be exposed.
Tried to find the code responsible for fetching those configs and the logic behind it but simply don't have time for this ConfigDataEnvironment::processAndApply

Related

Spring Boot Actuator: Failed to bind properties under

I try to enable Spring Boot Actuator (together with Swagger) and have in the pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>2.4.5</version>
</dependency>
with the following config in the yml-file:
springdoc:
api-docs:
enabled: true
show:
actuator: true
management:
endpoints:
web:
exposure:
include= "*"
expose: "*"
When I launch it I get the following exception:
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under 'management.endpoints.web.exposure' to org.springframework.boot.actuate.autoconfigure.endpoint.web.WebEndpointProperties$Exposure:
Reason: No converter found capable of converting from type [java.lang.String] to type [org.springframework.boot.actuate.autoconfigure.endpoint.web.WebEndpointProperties$Exposure]
Action:
Update your application's configuration
How to fix this Exception?
Your YAML has a = instead of a :
And remove expose:
Wrong:
management:
endpoints:
web:
exposure:
include= "*"
expose: "*"
Correct:
management:
endpoints:
web:
exposure:
include: "*"
My issue was fixed by adding following Maven dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-actuator-autoconfigure</artifactId>
</dependency>

How to activate /turbine.stream endpoint in a standalone Turbine application

I am trying to create a standalone application to collect Hystrix stream from other applications. But it does not expose the /turbine.stream endpoint by default. I am sure what is missing in my project.
Spring Boot: 2.0.4.RELEASE, Spring Cloud: Finchley.SR1
The application class:
#SpringBootApplication
#EnableDiscoveryClient
#EnableTurbine
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
The content of applicaiton.yml:
server:
port: 8383
spring:
application:
name: hystrix-turbine
management:
endpoints:
web.exposure.include: '*'
applications: hystrix
turbine:
aggregator:
clusterConfig: ${applications}
appConfig: ${applications}
# instanceUrlSuffix.default: actuator/hystrix.stream
And the maven dependencies:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-turbine</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
I created a sample project for this.
I'd suggest you check the configuration steps below:
1) Your stream URL in the Hystrix Dashboard should be:
http://localhost:{turbine app port}/turbine.stream?cluster={configured cluster in properties file}
The url should be pointing to the port of the application that has #EnableTurbine annotation in your main class.
2) Check if you are getting a response for:
http://localhost:{client app port}/actuator/hystrix.stream
(use your browser for this) (this should be coming from the application you have enabled hystrix on using #EnableCircuitBreaker)
If you're getting pings, then atleast your hystrix stream is accessible. If not, Check if you have: org.springframework.boot:spring-boot-starter-actuator in your client side dependencies and
make sure you have the below property set in application.properties file of the application that has #EnableCircuitBreaker in the main class:
management.endpoints.web.exposure.include= hystrix.stream, info, health
Check the URL again.
3) Once your get a reponse from hystrix.stream, you can now configure your cluster on the turbine app properties file:
turbine:
appConfig: {serviceId in lower case}
aggregator:
clusterConfig: {above serviceId in upper case}
after running the app, check if you've configured the cluster correctly:
http://localhost:{turbine app port}/clusters
you should'nt be getting a "[]" on your browser if all's well.
Once you see a response on the clusters endpoint, you will now be able to see the details on the dashboard when you point it to the turbine app

Eureka client exception com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server

I am new to microservices. I'm trying to create one small application for learning purpose. Here is my code:
EurekaServer - application.yml
spring:
application:
name: EurekaServer
server:
port: 8080
servlet:
context-path: /EurekaServer
eureka:
client:
fetch-registry: false
register-with-eureka: false
Eureka Server is working fine and I am able to see the dashboard at http://localhost:8080/EurekaServer
EmployeeClient: application.yml is below:
spring:
application:
name: EmployeeClient
server:
port: 8586
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8080/EurekaServer
In last line I need to write serviceUrl explicitly as on pressing ctrl+space in sts it doesnot show option serviceUrl but it shows service-url, hyphen sign. And same with defaultZone. Am I missing some jar or specific version?
My EmployeeClientApplication.java
#EnableEurekaClient
#SpringBootApplication
public class EmployeeClientApplication {
public static void main(String[] args) {
SpringApplication.run(EmployeeClientApplication.class, args);
}
}
when I try to run EmployeeClientApplication.java it gives me below exception:
com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
I also tried using #EnableDiscoveryClient in place of #EnableEurekaClient, but with no luck.
A part of EmployeeClient pom.xml is below:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.3.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<spring-cloud.version>Finchley.RELEASE</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
<dependency>
Where I'm making a mistake?
I think, you will have to change default Zone in client property from /EurekaServer to /eureka
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8080/eureka
"/eureka" is the rest endpoint to register services to eureka registry.
Make the changes and try, it should work.
In addition to this, if you are willing to change the UI dashboard url you should use eureka.dashboard.path property.
I had to use below line in EmployeeClient application.yml
defaultZone: http://localhost:8080/EurekaServer/eureka
Adding eureka.instance.hostname=localhost can solve your issue, if you have added DiscoveryServer as the application name in the server's application properties file.

Unable to connect to Command Metric Stream for Hystrix Dashboard with Spring Cloud

I have microservices project with Spring Cloud, the snippet from parent:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.7.RELEASE</version>
</parent>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Dalston.SR3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
All services are running under Eureka server:
All services are running fine. I can call make appropriate calls with Postman and everything works fine.
I have separate service which handles Hystrix dashboard, a snippet from pom:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
</dependency>
Configuration main class:
#SpringBootApplication
#EnableHystrixDashboard
public class DashboardApp {
public static void main(String[] args) {
SpringApplication.run(DashboardApp.class, args);
}
}
and config yaml file:
spring:
application:
name: Dashboard
server:
port: 8000
eureka:
client:
fetchRegistry: true
registerWithEureka: false
serviceUrl:
defaultZone: http://localhost:8761/eureka
I have next dashboard looking:
Full stack trace from the console is here. Following is some snippet:
2018-04-12 11:28:25.089 ERROR 15762 --- [qtp295055909-16] ashboardConfiguration$ProxyStreamServlet : Error proxying request: http://localhost:8082/hystrix.stream
java.lang.RuntimeException: org.eclipse.jetty.io.EofException
at org.springframework.cloud.netflix.hystrix.dashboard.HystrixDashboardConfiguration$ProxyStreamServlet.doGet(HystrixDashboardConfiguration.java:208)
....
Caused by: org.eclipse.jetty.io.EofException: null
...
Caused by: java.io.IOException: Broken pipe
...
Service itself is accessible with spring actuator:
snippet from it's pom:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Config class looks:
#EnableHystrix
#EnableEurekaClient
#SpringBootApplication
public class TableApp {
public static void main(String[] args) {
SpringApplication.run(TableApp.class, args);
}
}
How to solve this issue?
For those who are using spring boot 2, the hystrix.stream endpoint has been moved to /actuator/hystrix.stream.
For me this url worked:
http://localhost:8082/actuator/hystrix.stream
And yes, have this actuator endpoint enabled via following property:
management.endpoints.web.exposure.include=hystrix.stream
Of course you must have the actutator dependency included in your project.
Hystrix dashboard itself can't be used for monitoring several instances at once. The thing that you need is turbine+dashboard. In a couple of words turbine is an aggregator for several hystrix metrics streams.
Configuration of instance:
management:
endpoints:
web:
exposure:
include: hystrix.stream, info, health
spring:
application:
name: WRITING
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8761/eureka
The important thing here is to expose hystix.stream actuator. This endpoint will be used by the turbine to read metrics. Also, do not forget to add actuators starter.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
If you did everything correctly http://localhost:8080/actuator/hystrix.stream endpoint should become available.
Turbine config will look like:
server:
port: 8888
spring:
application:
name: TURBINE
eureka:
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://localhost:8761/eureka/
turbine:
appConfig: WRITING,READING
clusterNameExpression: new String('default')
In appConfig, you should specify service names for monitoring.
After starting turbine localhost:8888/turbine.stream will be available.
You can pass this URL to the dashboard and monitor all data aggregated for hystrix commands of discovered instances.
Github project example.
p.s.
Dependencies that you have used are deprecated. Please check maven repo
I was able to solve this issue for the spring-boot-starter-parent version 2.0.7.RELEASE and spring-cloud-dependencies version Finchley.SR2 by adding below two properties in the application.properties.
management.endpoints.web.exposure.include=*
management.endpoints.web.base-path=/
Finally, I found the solution.
Problem was that Controller API has to be market by HystrixCommand annotation.
Snippet from a documentation:
Turbine AMQP by Spring Cloud offers a different model where each
application instance pushes the metrics from Hystrix commands to
Turbine through a central AMQP broker.
I added it without any parameters to all Controller's methods, like following:
#RestController
#AllArgsConstructor
public class GuestController {
private DinnerService dinnerService;
#HystrixCommand
#PostMapping("/dinner")
public Integer startDinner(#RequestBody List<Integer> menuItems) {
return dinnerService.startDinner(menuItems);
}
#HystrixCommand
#DeleteMapping("/dinner/{tableId}")
public void finishDinner(#PathVariable Integer tableId) {
dinnerService.finishDinner(tableId);
}
}
And now all works like charming:
Now I understand that I was so close to it.
Make sure you have added this in your application.properties.
hystrix.dashboard.proxy-stream-allow-list=localhost
I had the same issues with the latest version of Spring-boot(2.3.3-XXX) and spring-cloud (Hoxton.SR7) but when I did the downgrade the version in pom.xml file then it starts working fine for me.
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.16.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
and
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Greenwich.SR6</spring-cloud.version>
</properties>
Hope, this would be helpful :)
Correct your hystrix.stream url to http://localhost:8082/actuator/hystrix.stream
Exposure 'hystrix.steam' web endpoints in your hystrix application's config file:
management:
endpoints:
web:
exposure:
include: 'hystrix.stream'
Make sure you have added your host to hystrix.dashboard.proxyStreamAllowList, in your config file of hystrix-dashboard application, it looks like:
hystrix:
dashboard:
proxy-stream-allow-list:
- 'localhost'
check details on
github: https://github.com/HuiyingWang0108/hystrix-dashboard
related microservice:
https://github.com/HuiyingWang0108/cloud-gateway
https://github.com/HuiyingWang0108/registry-service
In my case:
app like this:
Step1:
(1) change pom.xml:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
<version>2.2.10.RELEASE</version>
</dependency>
to
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
<version>2.2.10.RELEASE</version>
</dependency>
(2) added below in the application.yml
hystrix:
dashboard:
proxy-stream-allow-list: "*"
(3) and then start app, open:
http://localhost:9295/hystrix
works:
Step2:
(1) added below in the pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
(2)added below in the application.yml
management:
endpoints:
web:
exposure:
include: hystrix.stream, info, health
(3) Open:
http://localhost:9191/actuator/hystrix.steam
shows:

Configuring Spring Boot with Spring-Cloud-Connectors to use the PWS Config-Server

I am having a hard time to configure my Spring Boot application to connect to the PWS (Pivotal Web Services) provided Config-Server via Spring-Cloud-Connectors.
In the manifest.yml the config server is bound to the application, which is correctly reflected by the corresponding, VCAP_SERVICES entry:
applications:
- name: edge-service-webapp-myapp
services:
- infrastructure-config-server
memory: 512M
env:
TRUST_CERTS: api.run.pivotal.io
SPRING_PROFILES_DEFAULT: cloud
instances: 1
host: edge-service-webapp-myapp
domain: cfapps.io
buildpack: java_buildpack
{
"VCAP_SERVICES": {
"p-config-server": [
{
"credentials": {
"access_token_uri": "https://p-spring-cloud-services.uaa.run.pivotal.io/oauth/token",
"client_id": "p-config-server-84d66ea6-ebc6-xxx",
"client_secret": "***",
"uri": "https://config-b4320676-xxx.cfapps.io"
}, ...
}
The application is build with spring-boot-starter-parent 1.5.2.RELEASE, spring-cloud-dependencies Camden.SR5 and spring-cloud-services-dependencies 1.4.1.RELEASE. Also I am using spring-cloud-starter-config and spring-boot-starter-cloud-connectors as explicit dependencies.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.pivotal.spring.cloud</groupId>
<artifactId>spring-cloud-services-dependencies</artifactId>
<version>1.4.1.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Camden.SR5</version>
<type>pom</type>
<scope>import</scope>
</dependency>
....
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cloud-connectors</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
...
When I curl the config server I can see the application version available for the application-name (my-app) and the active 'cloud' profile.
spring:
application:
name: my-app
cloud:
config:
enabled: true
curl -H "Authorization: Bearer XXX" https://config-b4320676-xxx.cfapps.io/tradefoundry/cloud
{"name":"my-app","profiles":["cloud"],"label":"master","version":"389e4f909ff1303332167b2159b4d75201109d69","state":null,"propertySources":[{"name":"https://gitlab.com/myapp/configuration.git/myapp-cloud.properties","source":{"spring.thymeleaf.cache":"true","message":"Hello Cloud!"}},{"name":"https://gitlab.com/myapp/configuration.git/myapp.properties","source":{"server.compression.enabled":"true","spring.thymeleaf.cache":"true","application.version":"0.0.1"}},{"name":"https://gitlab.com/myapp/configuration.git/application.properties","source":{"server.compression.enabled":"true","spring.thymeleaf.cache":"true","application.cache.busting.enabled":"false","application.version":"0.0.1-20170202195700","server.compression.mime-types":"application/json,application/xml,text/html,text/xml,text/plain,text/css,application/javascript"}}]}
But still the application fails at startup, complaining about the missing property application.version.
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'myWebappApplication': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'application.version' in value "${application.version}"
What am I missing here? I thought the cloud connectors were all plug-and-play through autoconfiguration!?
Any help is welcome!
To use the Spring Cloud Services config server provided on PWS, you need to use a different set of client libraries as shown in the PWS docs.
Replace the dependencies on spring-boot-starter-cloud-connectors and spring-cloud-starter-config with just this one dependency:
<dependency>
<groupId>io.pivotal.spring.cloud</groupId>
<artifactId>spring-cloud-services-starter-config-client</artifactId>
</dependency>
The Spring Cloud Services config server adds additional OAuth2-based security on top of the open-source Spring Cloud Config server. This client-side library does the OAuth negotiation automatically.

Resources