Error with Eureka Server Cluster (peer awareness) - spring-boot

My question is about Spring Cloud Eureka: I'm trying to run Eureka Cluster (2 nodes, localhost:8761, localhost:8762),
but some errors appear:
RedirectingEurekaHttpClient : Request execution error: endpoint=DefaultEndpoint{ serviceUrl='http://localhost:8761/eureka/}
RetryableEurekaHttpClient :
Request execution failed with message: java.net.ConnectException: Connection refused
DiscoveryClient : DiscoveryClient_UNKNOWN/192.168.1.4:8761 - was unable to refresh its cache! status = Cannot execute request on any known server
Here are application.yml files:
Server 1:
spring:
profiles: eureka-peer1
server:
port: 8761
eureka:
instance:
hostname: eureka-peer1
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://eureka-peer2:8762/eureka/
Server 2:
spring:
profiles: eureka-peer2
server:
port: 8762
eureka:
instance:
hostname: eureka-peer2
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://eureka-peer1:8761/eureka/
File /etc/hosts:
127.0.0.1 localhost
127.0.0.1 eureka-peer1
127.0.0.1 eureka-peer2
ErekaService1Application.java , ErekaService2Application.java:
package com.example.eurekaservice1;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer;
#SpringBootApplication
#EnableEurekaServer
#EnableEurekaClient
public class EurekaService1Application {
public static void main(String[] args) {
SpringApplication.run(EurekaService1Application.class, args);
}
}
Removing #EnableEurekaClient and setting "registerWithEureka" and "fetchRegistry" to "false" gives the same results.
Moreover, standalone mode with these properties leads to the same error:
spring:
profiles: eureka-peer1
server:
port: 8761
eureka:
client:
registerWithEureka: false
fetchRegistry: false
Here is pom.xml file, automatically generated using start.spring.io:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>eureka-service-1</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>eureka-service-1</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR3</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>

Remove #EnableEurekaClient & dependency as well
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

Related

Table not found in integration tests when using TestContainers with Flyway

I am trying to connect to Postgres database instance of TestContainers but I get the error org.postgresql.util.PSQLException: ERROR: relation "proposal" does not exist in Spring data repository class "ProposalRepository" when running the "HelloRestControllerTest"
The flyway is enabled in the config.
I created a demo project with just the basic stuff.
HelloRestController:
import com.example.demo.entity.Proposal
import com.example.demo.repo.ProposalRepository
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.data.repository.findByIdOrNull
import org.springframework.http.ResponseEntity
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RestController
#RestController
class HelloRestController #Autowired constructor(val proposalRepository: ProposalRepository) {
#GetMapping("/hello")
fun hello(): ResponseEntity<Proposal>{
val result = proposalRepository.findByIdOrNull("123") //error is thrown here.
return ResponseEntity.ok(result)
}
}
ProposalRepository:
import com.example.demo.entity.Proposal
import org.springframework.data.repository.CrudRepository
interface ProposalRepository : CrudRepository<Proposal, String>
Entity:
import java.math.BigDecimal
import javax.persistence.Column
import javax.persistence.Entity
import javax.persistence.Table
import javax.persistence.Id
#Entity
#Table(name = "proposal")
data class Proposal(
#Id
#Column(name = "proposal_id")
val proposalId: String,
#Column(name = "amount", nullable = true)
val amount: BigDecimal?,
#Column(name = "user_id")
val userId: String? = null
)
Flyway SQL script:
create table proposal (
proposal_id varchar(50),
amount decimal not null ,
user_id varchar(35)
);
application-it-test.yml in src/test/resources folder:
spring:
datasource:
#driver-class-name: org.postgresql.Driver
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:tc:postgresql:11.6:///databasename
hikari:
max-lifetime: 500000
connection-timeout: 300000
idle-timeout: 600000
maximum-pool-size: 5
minimum-idle: 1
flyway:
enabled: true
url: ${spring.datasource.url}
locations: 'classpath:db/migration/postgresql'
table: FLY_VERSION
jpa:
show-sql: true
database-platform: org.hibernate.dialect.PostgreSQLDialect
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.3</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>11</java.version>
<kotlin.version>1.6.21</kotlin.version>
<spring-cloud.version>2021.0.3</spring-cloud.version>
<testcontainers.version>1.17.3</testcontainers.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib-jdk8</artifactId>
</dependency>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>json-path</artifactId>
<scope>test</scope>
<exclusions>
<!-- excluded because REST assured 4.3.0+ expects Groovy 3+ (https://github.com/rest-assured/rest-assured/issues/1283) -->
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-json</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<scope>test</scope>
<exclusions>
<!-- excluded because REST assured 4.3.0+ expects Groovy 3+ (https://github.com/rest-assured/rest-assured/issues/1283) -->
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy</artifactId>
</exclusion>
<!-- excluded because REST assured 4.3.0+ expects Groovy 3+ (https://github.com/rest-assured/rest-assured/issues/1283) -->
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-xml</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-contract-stub-runner</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>postgresql</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-bom</artifactId>
<version>${testcontainers.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
<testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<configuration>
<args>
<arg>-Xjsr305=strict</arg>
</args>
<compilerPlugins>
<plugin>spring</plugin>
<plugin>jpa</plugin>
</compilerPlugins>
</configuration>
<dependencies>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-allopen</artifactId>
<version>${kotlin.version}</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-noarg</artifactId>
<version>${kotlin.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</project>
HelloRestControllerTest:
import io.restassured.RestAssured.given
import io.restassured.http.ContentType
import org.junit.jupiter.api.Test
import org.junit.runner.RunWith
import org.springframework.boot.test.context.SpringBootTest
import org.springframework.boot.test.web.server.LocalServerPort
import org.springframework.cloud.contract.wiremock.AutoConfigureWireMock
import org.springframework.test.context.ActiveProfiles
import org.springframework.test.context.DynamicPropertyRegistry
import org.springframework.test.context.DynamicPropertySource
import org.springframework.test.context.junit4.SpringRunner
import org.testcontainers.containers.PostgreSQLContainer
import org.testcontainers.junit.jupiter.Container
import org.testcontainers.junit.jupiter.Testcontainers
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#ActiveProfiles("it-test")
#AutoConfigureWireMock(port = 1234)
#RunWith(SpringRunner::class)
#Testcontainers
class HelloRestControllerTest {
#LocalServerPort
final val portNumber = 0
val baseUrl = "http://localhost:$portNumber"
companion object {
#Container
var postgreSQL: PostgreSQLContainer<*> = PostgreSQLContainer("postgres:11.6")
#DynamicPropertySource
fun postgreSQLProperties(registry: DynamicPropertyRegistry) {
registry.add("spring.datasource.username") { postgreSQL.username }
registry.add("spring.datasource.password") { postgreSQL.password }
}
}
#Test
fun test() {
val helloResponse = given()
.contentType(ContentType.JSON)
.get("$baseUrl$portNumber/hello")
.andReturn()
println(helloResponse.body)
}
}
The code and configuration looks fine so I am not able to figure out what is the problem. What could be the issue here?
EDIT:
If I add #Sql annotation in HelloRestControllerTest as shown below, I do not see any error. It looks like Flyway is not able to run the scripts to the right database or something. Not sure why?
#Sql(scripts = ["classpath:/db/migration/postgresql/V01__PROPOSAL.sql"])
change the application-it-test.yml as shown below:
spring:
datasource:
#driver-class-name: org.postgresql.Driver
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:tc:postgresql:11.6:///databasename?stringtype=unspecified
hikari:
max-lifetime: 500000
connection-timeout: 300000
idle-timeout: 600000
maximum-pool-size: 5
minimum-idle: 1
flyway:
enabled: true
locations: 'classpath:db/migration/postgresql'
jpa:
show-sql: true
database-platform: org.hibernate.dialect.PostgreSQLDialect
under flyway variable in yml file you only need to mention the enable and locations values. Everything else like url, schemas, table, etc should be removed. Otherwise Flyway does not work together with TestContainers in integration test.
The change needs to be done in both the test yml file and real application's yml file as well.

How to secure spring cloud eureka service with basic auth?

I set up multiple instances of eureka servers on the same host. They are using host names eureka-primary, secondary and tertiary which are defined as the localhost aliases in the hosts file and everything is working just fine - they are all visible and available to each other as different instances.
The problem starts when I try to secure eureka instances with basic auth and this. The idea is add spring security dependency, to specify the security user and password on eureka servers, and to put these credentials in the defaultZone urls (configs are below), but this does not seem to work.
Eureka instances can't even register to each other and when I try to access eureka web portal I am prompted with login form and then redirected to dashboard. All dashboards are working fine and need credentials to be accessed.
I am using spring cloud Finchley.RC1 with spring boot 2.0.1.RELEASE and the same version of spring-boot-starter-security and spring-cloud-starter-netflix-eureka-server.
Eureka server pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>rs.microservices</groupId>
<artifactId>eurekaServer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>eurekaServer</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<spring-cloud.version>Finchley.RC1</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
</project>
Eureka server application.yml
---
spring:
security:
user:
name: admin
password: admin
profiles: primary
application:
name: eureka-server-clustered
server:
port: 8011
eureka:
instance:
hostname: eureka-primary
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://admin:admin#eureka-secondary:8012/eureka/,http://admin:admin#eureka-tertiary:8013/eureka/
---
spring:
security:
user:
name: admin
password: admin
profiles: secondary
application:
name: eureka-server-clustered
server:
port: 8012
eureka:
instance:
hostname: eureka-secondary
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://admin:admin#eureka-primary:8013/eureka/,http://admin:admin#eureka-tertiary:8011/eureka/
---
spring:
security:
user:
name: admin
password: admin
profiles: tertiary
application:
name: eureka-server-clustered
server:
port: 8013
eureka:
instance:
hostname: eureka-tertiary
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://admin:admin#eureka-primary:8011/eureka/,http://admin:admin#eureka-secondary:8012/eureka/
Microservice bootstrap.yml
spring:
application:
name: someService
server:
port: 0
eureka:
client:
registerWithEureka: true
fetchRegistry: true
service-url:
defaultZone: http://admin:admin#localhost:8011/eureka/,http://admin:admin#localhost:8012/eureka/,http://admin:admin#localhost:8013/eureka/
What am I doing wrong?
*EDIT
I already found multiple solutions like this one Securing Eureka in Spring cloud, but none of them really fixed my problem - as you can see our configurations are identical.
Solved!
TL;DR
The problem was the CSRF and for some reason spring couldn't authenticate user configured in application.yml
So I had to override configure methods from WebSecurityConfigurerAdapter to disable csrf and create inMemory user. Also removed spring.security.user attributes from application.yml.
Eureka server application.yml now looks like:
---
spring:
profiles: primary
application:
name: eureka-server-clustered
server:
port: 8011
eureka:
instance:
hostname: eureka-primary
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://admin:admin#eureka-secondary:8012/eureka,http://admin:admin#eureka-tertiary:8013/eureka
---
spring:
profiles: secondary
application:
name: eureka-server-clustered
server:
port: 8012
eureka:
instance:
hostname: eureka-secondary
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://admin:admin#eureka-primary:8013/eureka,http://admin:admin#eureka-tertiary:8011/eureka
---
spring:
profiles: tertiary
application:
name: eureka-server-clustered
server:
port: 8013
eureka:
instance:
hostname: eureka-tertiary
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: http://admin:admin#eureka-primary:8011/eureka,http://admin:admin#eureka-secondary:8012/eureka
Newly created WebSecurityConfig class:
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.crypto.password.NoOpPasswordEncoder;
#Configuration
public class WebSecurityConfig extends WebSecurityConfigurerAdapter{
#Override
public void configure(AuthenticationManagerBuilder auth) throws Exception {
auth.inMemoryAuthentication()
.passwordEncoder(NoOpPasswordEncoder.getInstance())
.withUser("admin").password("admin")
.authorities("ADMIN");
}
#Override
protected void configure(HttpSecurity http) throws Exception {
http
.csrf()
.disable()
.authorizeRequests()
.anyRequest().authenticated()
.and()
.httpBasic();
}
}
I had similar issue with Greenwich.SR1.
Enabled security with spring.security.user.name allowed me to login with username/password but my services could't register with Eureka service.
I made it work, and as mentioned above, the cause was indeed CSRF
This fixes it:
#Configuration
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
#Override
protected void configure(HttpSecurity http) throws Exception {
http.csrf().ignoringAntMatchers("/eureka/**");
super.configure(http);
}
}
Import spring-security in your pom.xml :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
And put this in your application.yaml :
security:
user:
name: admin
password: password
And it should work !
EDIT : I already made this, here the simpliest code that allow you to achieve this, I seggest you start from here :
pom.xml :
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>vg.step</groupId>
<artifactId>eureka.server</artifactId>
<version>0.0.1</version>
<name>vgstepeurekaserver</name>
<properties>
<org.springframework.cloud-version>1.4.4.RELEASE</org.springframework.cloud-version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-netflix</artifactId>
<version>${org.springframework.cloud-version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- Spring Boot -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-undertow</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<!-- Spring Cloud -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
</dependencies>
<build>
<finalName>app</finalName>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/main/resources</directory>
<includes>
<include>application*.yaml</include>
</includes>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
src/main/resources/application.yaml :
spring:
application:
name: #project.name#
eureka:
instance:
hostname: eureka-server
client:
registerWithEureka: false
fetchRegistry: false
serviceUrl:
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka
server:
port: 8002
security:
user:
name: admin
password: password
src/main/java/vg/step/eureka/server/Application.java :
package vg.step.eureka.server;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer;
#SpringBootApplication
#EnableEurekaServer
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}

Not able to read configuration from Consul in spring-boot application

I am creating a Spring Boot application, which will read configuration like DB properties from Consul. But I am not able to read the key value from Consul using my application. Following is, what I am trying to do.
**pom.xml**
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.tuturself</groupId>
<artifactId>spring-boot-consul</artifactId>
<version>1.0-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
<relativePath/>
</parent>
<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<spring.retry.version>1.2.1.RELEASE</spring.retry.version>
<consul.version>1.1.2.RELEASE</consul.version>
<consul.discovery.version>1.1.2.RELEASE</consul.discovery.version>
<jackson.version>2.8.1</jackson.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-all</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-consul-discovery</artifactId>
<version>${consul.discovery.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>${spring.retry.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jackson.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-consul-dependencies</artifactId>
<version>1.2.1.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>
And Following is my Main class:
#EnableRetry
#RefreshScope
#EnableDiscoveryClient
#SpringBootApplication
#ComponentScan("com.test.*")
public class SpringBootConsulApplication {
private static ConsulConfiguration consulConfiguration;
public static void main(String[] args) {
try {
String consulHost = System.getProperty("spring.cloud.consul.host");
System.out.println("consulHost ::" + consulHost);
String consulPort = System.getProperty("spring.cloud.consul.port");
System.out.println("consulPort ::" + consulPort);
String consulPrefix = System.getProperty("spring.cloud.consul.config.prefix");
System.out.println("consulPrefix ::" + consulPrefix);
new SpringApplicationBuilder(SpringBootConsulApplication.class).web(true).run(args);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
And I am reading the consul properties using the #Value annotation:
#Configuration
#EnableConfigurationProperties(PropertySourceBootstrapProperties.class)
public class ConsulConfiguration {
#Value("${cassandra.host}")
private String cassandraHost;
#Value("${cassandra.user}")
private String userName;
#Value("${cassandra.password}")
private String password;
}
I have my bootstrap.yml in resources folder:
spring:
cloud:
consul:
host: localhost
port: 8500
enabled: true
config:
enabled: true
prefix: config/application
defaultContext: apps
profileSeparator: '::'
application:
name: spring-boot-consul
Consul is up and running in my local system on localhost:8500 where I have the file config/application/spring-boot-consul.yml file;
spring:
application:
name: spring-boot-consul
cassandra:
host: 127.0.0.1:9042,127.0.0.2:9042
user: my_user
password: my_pass
pooling:
maxThread: 10
timeout: 50
keyspace:
name: test_keyspace
readConsistency: ONE
writeConsistency: ONE
When I am strating the application, it is showing not able to bind cassandra.host in my ConsulConfiguration class. Thus stopping the application. Any hints , What I am doing wrong here?
You can find a working example here.
Consul Configuration KV Store
You need to store your properties in the Consul KV store either from Consul UI or from the command line. The Consul Agent will not load your properties from the file system. To load the properties from the command line, you can use the following command once the Consul Agent is up and running. The YAML data can be read from a file by prefixing the file name with the # symbol.
./consul kv put config/application/data #spring-boot-consul.yml
where config/application/data is the key name.
If the data is successfully written in the KV, you should get the following response,
Success! Data written to: config/application/data
You can also fetch the properties from the KV by using the following command,
$ ./consul kv get config/application/data
cassandra:
host: 127.0.0.1:9042,127.0.0.2:9042
user: my_user
password: my_pass
You can also view the properties from the Consul Web UI,
Changes to bootstrap.yml
You need to modify your bootstrap.yml slightly. Here are the changes:
prefix value to config
defaultContext value to application
Added format to yaml
Added data-key by the name of data to fetch the YAML blob.
spring:
profiles: default
cloud:
consul:
host: localhost
port: 8500
config:
enabled: true
prefix: config
defaultContext: application
data-key: data
profileSeparator: '::'
format: yaml
application:
name: spring-boot-consul
Changes to ConsulConfiguration
#Configuration
#RefreshScope
public class ConsulConfiguration {
#Value("${cassandra.host}")
private String cassandraHost;
#Value("${cassandra.user}")
private String userName;
#Value("${cassandra.password}")
private String password;
#PostConstruct
public void postConstruct() {
// to validate if properties are loaded
System.out.println("** cassandra.host: " + cassandraHost);
System.out.println("** cassandra.user: " + userName);
System.out.println("** cassandra.password: " + password);
}
}
Changes to Application class,
#EnableRetry
#RefreshScope
#EnableDiscoveryClient
#EnableAutoConfiguration
#EnableConfigurationProperties
#SpringBootApplication
#ComponentScan("com.test.*")
public class SpringBootConsulApplication {
public static void main(String[] args) {
...
}
}

throw ConverterNotFoundException using spring-cloud-consul for int property

I got the following error message when starting a spring boot/cloud application which gets some properties from consul:
***************************
APPLICATION FAILED TO START
***************************
Description:
Binding to target demo.config.Config#5b4954b2 failed:
Property: config.number
Value: ${config.number:5}
Reason: Failed to convert property value of type 'java.lang.String' to required type 'int' for property 'number'; nested exception is org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.String] to type [int]
Action:
Update your application's configuration
Here's my main class:
#SpringBootApplication
#WebAppConfiguration
#EnableDiscoveryClient
#EnableConfigurationProperties
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
And the configuration property class:
#Component
#ConfigurationProperties(prefix="config")
public class Config {
private int number;
public int getNumber() {
return number;
}
public void setNumber(int number) {
this.number = number;
}
}
The application.yml file is:
server:
port: 8100
spring:
application:
name: demo
And the bootstrap.yml file:
spring:
cloud:
consul:
host: localhost
port: 8500
config:
enabled: true
prefix: config
config:
number: ${config.number:5}
Finally the related part of maven pom.xml file:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.3.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<spring-cloud.version>Dalston.SR1</spring-cloud.version>
<cucumber.version>1.2.5</cucumber.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
I found the problem is because PropertySourceBootstrapConfiguration cannot locate property source from consul in some case. If PropertySourceBootstrapConfiguration is properly initiated during the booting up, there's no such exception and the app works as expected.

Spring Cloud Config client not loading the value from config server

I am facing below issue while I try to run my Spring Cloud Config Client:
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'DATABASE_NAME' in string value "${DATABASE_NAME}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:174)
at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:204
My dependencies in POM.xml are as below:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config</artifactId>
<version>1.1.1.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Brixton.SR3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
The bootstrap.yml
spring:
application:
name: my-config-client
cloud:
services:
registrationMethod: route
config:
enabled: true
uri: http://localhost:${config.port:8888}
The application.yml is as below:
# HTTP Server
server:
port: 2222
# Spring properties
spring:
profiles:
active: dev
#Disable HTTP Basic Authentication
security:
basic:
enabled: false
The class where I am trying to access the property is as below:
#RefreshScope
#Component
public class MyProperty {
#Value("${DATABASE_NAME}")
private String databaseName;
public String getDatabaseName() {
return databaseName;
}
}
My config server is running fine. When I use this url on browser http://localhost:8888/configserver/dev, It gives the below result:
{
"name":"configserver",
"profiles":[
"dev"
],
"label":"master",
"version":"c991526a93fb776e37e18e138c7485d894d6ea4f",
"propertySources":[
{
"name":"https://onestash.abc.com/scm/kapmol/microservice-config-repo.git/configserver.properties",
"source":{
"DATABASE_NAME":"ABC",
"CONVERT_USERS":"Y",
"LRDS_JNDI_NAME":"jdbc/tds_new"
}
}
]
}
I tried with all the posts who were facing this issue. But, it is not working for me. May be, I am missing some points. If anybody can provide help, it would be great.
Thanks
There are some breaking changes with the new spring cloud module read more: here.
Bootstrap, provided by spring-cloud-commons, is no longer enabled by
default. If your project requires it, it can be re-enabled by
properties or by a new starter.
To re-enable by properties set spring.cloud.bootstrap.enabled=true or spring.config.use-legacy-processing=true. These need to be set as
an environment variable, java system property or a command line
argument.
The other option is to include the new spring-cloud-starter-bootstrap
It worked for me by adding these dependencies:
<parent>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-parent</artifactId>
<version>2020.0.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
I referred this post answered by 'spencergibb' and able to resolve the issue. I added "spring.config.name" in my bootstrap.yml file of client application and resolved the issue. Now, my bootstrap.yml look like as below:
spring:
application:
name: my-config-client
cloud:
services:
registrationMethod: route
config:
name: configserver
enabled: true
uri: http://localhost:${config.port:8888}

Resources