I'm trying to build sample microservice app using this tutorial.
Jhipster version is 4.0.6
So i've created gateway, service and started consul using this command:
docker-compose -f src/main/docker/consul.yml up
from my gateway directory.
But the error occur on Spring Boot startup, here is the log:
2017-02-22 11:52:25.679 ERROR 3168 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-02-22 11:52:32.491 WARN 3168 --- [ restartedMain] o.s.boot.SpringApplication : Error handling failed (ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#2a5b2096: startup date [Thu Jan 01 03:00:00 AST 1970]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext#5108df79)
com.ecwid.consul.transport.TransportException: java.net.ConnectException: Connection refused: connect
Could you please assist with this issue?
UPDATE:
I've found that app tries to make GET request to URL on staptup:
http://localhost:8500/v1/kv/config/armory,dev/?recurse&token=
But the only data stored in Consul K/V storage is:
KEY: config/application/data
VALUE:
configserver:
name: Docker Consul Service
status: Connected to Consul Server running in Docker
jhipster:
security:
authentication:
jwt:
secret: my-secret-token-to-change-in-production
You must copy your application yaml configurations into your consul instance as explained in the doc into central-server-config directory if consul is running in dev profile or in its git repo if consul is running in prod profile.
So assuming your app is named "armory" you should copy your src/main/resources/config/application.yml to armory.yml and for each profiles (e.g. application-dev.yml to armory-dev.yml)
Related
So every morning we're starting around ~30 spring-boot microservices.
Sometimes one of them is failing to start up.
Running Spring boot 2.1.5.
Today it was with the following error:
2021-11-15 02:18:32.269 INFO 12258023 --- [main] dk.yx.oiltanks.OilTanksApplication : The following profiles are active: prod
2021-11-15 02:19:49.406 ERROR 12258023 --- [main] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under 'spring.mvc.servlet' to org.springframework.boot.autoconfigure.web.servlet.WebMvcProperties$Servlet:
Reason: Failed to bind properties under 'spring.mvc.servlet' to org.springframework.boot.autoconfigure.web.servlet.WebMvcProperties$Servlet
Action:
Update your application's configuration
Is there any configuration parameters to increase "bind properties" time ?
All we did to get this application running again was starting it with the normal command.
Configuration file:
endpoints.cors.allowed-methods=GET
endpoints.cors.allowed-origins=*
eureka.client.serviceUrl.defaultZone=**HOSTNAME**:13499/eureka/
eureka.instance.healthCheckUrl=https\://${eureka.hostname}\:${server.port}/actuator/health
eureka.instance.homePageUrl=https\://${eureka.hostname}\:${server.port}/
eureka.instance.hostname=**HOSTNAME**
eureka.instance.nonSecurePortEnabled=false
eureka.instance.securePort=${server.port}
eureka.instance.securePortEnabled=true
eureka.instance.statusPageUrl=https\://${eureka.hostname}\:${server.port}/actuator/info
feign.client.config.default.connectTimeout=1600000
feign.client.config.default.readTimeout=1600000
logging.file.max-history=50
logging.file.max-size=100MB
logging.file=application.log
logging.level.dk.yx.yxdb=INFO
logging.level.dk.yx=INFO
logging.level.org.hibernate.SQL=INFO
logging.level.org.hibernate.type.descriptor.sql=INFO
logging.level.org.springframework.web.filter.CommonsRequestLoggingFilter=INFO
logging.level.org.springframework.web=INFO
management.endpoints.web.cors.allowed-methods=GET
management.endpoints.web.cors.allowed-origins=*
multipart.maxFileSize=-1
multipart.maxRequestSize=-1
pring.mvc.async.request-timeout=300000
server.connection-timeout=999999
server.http.port=3517
server.port=13517
server.ssl.key-alias=yx_dk
server.ssl.key-store-password=**PASSWORD**
server.ssl.key-store-type=PKCS12
server.ssl.key-store=/QOpenSys/etc/ssl/wildcard_yx_dk.p12
spring.application.name=OIL_TANK_SERVICE
spring.datasource.driver-class-name=com.ibm.as400.access.AS400JDBCDriver
spring.datasource.hibernate.default_schema=NHODATA
spring.datasource.hikari.connectionTimeout=90000
spring.datasource.hikari.idleTimeout=120000
spring.datasource.hikari.minimumIdle=3
spring.datasource.password=unreadable
spring.datasource.url=jdbc\:as400\:bmw/;naming\=system;libraries\=NHODATA,QTEMP,YXDTA24,ICEBREAK,NHOPRG,SYSAFD,QGPL,QIDU,OTCONLINE,SMS,YXDB,YXAPP,SMSDB;date format\=eur;time format\=eur;
spring.datasource.username=sqlread
spring.http.multipart.enabled=true
spring.http.multipart.maxFileSize=-1
spring.http.multipart.maxRequestSize=-1
spring.jpa.database-platform=org.hibernate.dialect.DB2Dialect
spring.jpa.hibernate.ddl-auto=none
spring.jpa.show-sql=false
spring.mvc.async.request-timeout=999999
spring.output.ansi.enabled=DETECT
spring.profiles.active=prod
spring.servlet.multipart.max-file-size=-1
spring.servlet.multipart.max-request-size=-1
yx.nps.ip=**IP**
yx.nps.username=**password**
yx.project.description=Service for handling all oiltank related stuff.
There are two components in my k8s namespace.
The first component is a simple Spring Boot Web Service with actuator/refresh endpoint exposed. I have mannualy make post request to the endpoint and it will trigger a configuration context refresh succesfully.
The second component is Spring Cloud Kubernetes Configuration Watcher which I pull the image per official document guidance and make it run in my k8s env. Per official spring document, it should detect the changes on configmap with label spring.cloud.kubernetes.config="true" and make post request to the actuator/refresh endpoint of the application whose spring.application.name is equal to the configmap name.
The second component is not working as expectted, and I don't know how to trouble shoot the root cause.
My Springboot application is called spring-boot-demo, the configmap is also named as spring-boot-demo. But I never find any mentions of "spring-boot-demo" in Spring Cloud Kubernetes Configuration Watcher's log and nor can I assure if it has sent post request to related endpoint.
I can only see the logs showed up repeatedly below:
2021-11-22 02:58:53.332 INFO 1 --- [192.168.0.1/...] .w.HttpBasedConfigMapWatchChangeDetector : Added new Kubernetes watch: config-maps-watch-event
2021-11-22 02:58:53.332 INFO 1 --- [192.168.0.1/...] .w.HttpBasedConfigMapWatchChangeDetector : Kubernetes event-based configMap change detector activated
2021-11-22 03:34:06.555 WARN 1 --- [192.168.0.1/...] .f.c.r.EventBasedConfigMapChangeDetector : ConfigMaps watch closed
io.fabric8.kubernetes.client.WatcherException: too old resource version: 5491278743 (5554041906)
at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onStatus(AbstractWatchManager.java:263) [kubernetes-client-5.5.0.jar:na]
at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:247) [kubernetes-client-5.5.0.jar:na]
at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:93) [kubernetes-client-5.5.0.jar:na]
at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:322) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:273) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.RealWebSocket$1.onResponse(RealWebSocket.java:209) [okhttp-3.14.9.jar:na]
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:174) [okhttp-3.14.9.jar:na]
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.14.9.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_312]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_312]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_312]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 5491278743 (5554041906)
... 13 common frames omitted
2021-11-22 03:34:06.605 INFO 1 --- [192.168.0.1/...] .f.c.r.EventBasedConfigMapChangeDetector : Added new Kubernetes watch: config-maps-watch-event
2021-11-22 03:34:06.605 INFO 1 --- [192.168.0.1/...] .f.c.r.EventBasedConfigMapChangeDetector : Kubernetes event-based configMap change detector activated
2021-11-22 03:34:06.607 INFO 1 --- [192.168.0.1/...] s.c.k.f.c.Fabric8ConfigMapPropertySource : Loading ConfigMap with name 'spring-cloud-kubernetes-configuration-watcher' in namespace 'my-namespace'
2021-11-22 03:34:06.621 WARN 1 --- [192.168.0.1/...] o.s.c.k.f.config.Fabric8ConfigUtils : config-map with name : 'spring-cloud-kubernetes-configuration-watcher' not present in namespace : 'my-namespace'
2021-11-22 03:34:06.625 WARN 1 --- [192.168.0.1/...] o.s.c.k.f.config.Fabric8ConfigUtils : config-map with name : 'spring-cloud-kubernetes-configuration-watcher-kubernetes' not present in namespace : 'my-namespace'
The docker image I use is springcloud/spring-cloud-kubernetes-configuration-watcher:2.1.0-RC1
Any hint to debug this issue is appreciated.
I'm building microservices using Spring Boot & Spring Cloud Config. From the log file, I can see one of my application send a request to config service for configuration every five minutes while other applications do not play the same behaviour.
06:04:28.586 [serviceId:xxx-service/traceId:545d584f10da3123/spanId:545d584f10da3123/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment - Fetching config from server at : http://config-service
06:04:30.135 [serviceId:xxx-service/traceId:545d584f10da3123/spanId:545d584f10da3123/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.log - Located environment: name=xxxservice, profiles=[dev], label=master, version=00ed4aebcd466c88b4a7df61296652944642cdc8, state=null
06:09:30.367 [serviceId:xxx-service/traceId:519fdde3a2d8ce5a/spanId:519fdde3a2d8ce5a/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment - Fetching config from server at : http://config-service
06:09:31.928 [serviceId:xxx-service/traceId:519fdde3a2d8ce5a/spanId:519fdde3a2d8ce5a/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.log - Located environment: name=xxxservice, profiles=[dev], label=master, version=00ed4aebcd466c88b4a7df61296652944642cdc8, state=null
06:14:32.035 [serviceId:xxx-service/traceId:e32c45c3a5c3fcca/spanId:e32c45c3a5c3fcca/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment - Fetching config from server at : http://config-service
06:14:33.611 [serviceId:xxx-service/traceId:e32c45c3a5c3fcca/spanId:e32c45c3a5c3fcca/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.log - Located environment: name=xxxservice, profiles=[dev], label=master, version=00ed4aebcd466c88b4a7df61296652944642cdc8, state=null
06:19:32.033 [serviceId:xxx-service/traceId:16fc982b0638d4a6/spanId:16fc982b0638d4a6/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment - Fetching config from server at : http://config-service
06:19:33.495 [serviceId:xxx-service/traceId:16fc982b0638d4a6/spanId:16fc982b0638d4a6/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.log - Located environment: name=xxxservice, profiles=[dev], label=master, version=00ed4aebcd466c88b4a7df61296652944642cdc8, state=null
06:24:32.036 [serviceId:xxx-service/traceId:f4eae364e2624d58/spanId:f4eae364e2624d58/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.getRemoteEnvironment - Fetching config from server at : http://config-service
06:24:33.640 [serviceId:xxx-service/traceId:f4eae364e2624d58/spanId:f4eae364e2624d58/parentId:] INFO o.s.c.c.c.ConfigServicePropertySourceLocator.log - Located environment: name=xxxservice, profiles=[dev], label=master, version=00ed4aebcd466c88b4a7df61296652944642cdc8, state=null
I try to understand this behaviour.
1 Is the bootstrap.yml of this service distinct to other services, leads the unique behaviour?
No, all services bootstrap.yml appear to be the same structure without different config keys:
spring:
profiles:
active: dev
cloud:
config:
name: xxxservice
label: master
uri: http://config-service
2 Is this servie trying to refresh its configuration dynamicly and periodly?
No, in order to dynamically refresh the configuration of a spring boot application, we need to:
put the #RefreshScope annotation on the class reading configuration
call with something like curl -X POST http://xx:8080/refresh to trigger refresh manually
So there should not be auto refresh behaviour periodly.
The question remaining unresolved, hope someone explain this situation directly or point any error in my statement.
I don't know why you are not seeing this with other services, but I believe that this behavior is due to the client wanting to check the health of the server periodically. From the Spring Cloud Config Client docs:
7.8.1 Health Indicator
The Config Client supplies a Spring Boot Health Indicator that attempts to load configuration from the Config Server. The health
indicator can be disabled by setting health.config.enabled=false.
The response is also cached for performance reasons. The default cache
time to live is 5 minutes. To change that value, set the
health.config.time-to-live property (in milliseconds).
So just set health.config.enabled=false, and this behavior should stop.
I am starting a Spring Boot Application with Consul.
I am getting the following error
2019-08-30 12:34:22.650 ERROR 23428 --- [ main] o.s.boot.SpringApplication : Application run failed
com.ecwid.consul.transport.TransportException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8090 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect`
I changed the default port in bootstrap.properties file.
I also used another non-Consul Spring Boot Application and it worked fine for that use-case with same port.
8090 is not the default port for Consul. You didn't say if your other successful app was on the same host or not, but make sure Consul is actually listening on that port with netstat or ss.
By default, Consul listens for API requests on port 8500.
I have setup and run consul using docker on my system using following command:
sudo docker run -p 8500:8500 consul:0.9.2
Consul is running fine as I can check from the consul UI (Image attached below):
Now, I am trying to run my spring-boot service to use this consul instance for service discovery and registration. But, whenever i start it gives me following exception:
2017-09-02 18:58:17.091 ERROR 5578 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:18.183 ERROR 5578 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:19.375 ERROR 5578 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:20.691 ERROR 5578 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:22.114 ERROR 5578 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:23.671 ERROR 5578 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:23.691 ERROR 5578 --- [ restartedMain] o.s.boot.SpringApplication : Application startup failed
com.ecwid.consul.v1.OperationException: OperationException(statusCode=500, statusMessage='Internal Server Error', statusContent='No cluster leader')
at com.ecwid.consul.v1.kv.KeyValueConsulClient.getKVValues(KeyValueConsulClient.java:159)
at com.ecwid.consul.v1.ConsulClient.getKVValues(ConsulClient.java:487)
at org.springframework.cloud.consul.config.ConsulPropertySource.init(ConsulPropertySource.java:66)
at org.springframework.cloud.consul.config.ConsulPropertySourceLocator.create(ConsulPropertySourceLocator.java:157)
at org.springframework.cloud.consul.config.ConsulPropertySourceLocator.locate(ConsulPropertySourceLocator.java:131)
at org.springframework.cloud.consul.config.ConsulPropertySourceLocator$$FastClassBySpringCGLIB$$b35ebf8.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:163)
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)
at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:152)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at org.springframework.cloud.consul.config.ConsulPropertySourceLocator$$EnhancerBySpringCGLIB$$66375879.locate(<generated>)
at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:93)
at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:567)
at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:338)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:301)
at com.pyg.auth.AuthServiceApp.main(AuthServiceApp.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
SpringBoot Main Class is annotated fine (I guess), because my spring-boot service was working fine with another consul instance earlier.
I was able to solve my issue by moving consul server and port configuration from application.yml to bootstrap.yml.
I don't know much about how it was solved and why it was unable to read from application.yml. If anyone has some details about it, pl. let me know.
Use sudo docker run -p 8500:8500 consul:0.9.2 -bootstrap to resolve the problem. I have
added -bootstrap one argument to bootstrap the server/servers and get elected as a leader. Manual bootstrapping with -bootstrap is not recommended in newer versions of Consul (0.5 and newer) as it is more error-prone.
Explanation:-
As said in docs Before a Consul cluster can begin to service requests, a server node must be elected leader. And this is reason of your exception on start of spring-boot service the leader has not been elected yet!!
Why the leader has not been elected? The list of servers involved in the cluster should be bootstrapped. And the servers can be bootstrapped using the
-bootstrap-expect configuration option. Recommended
Note:- Just for testing/learning purpose you can go ahead and create a single server because A single server deployment is highly discouraged as data loss is inevitable in a failure scenario.
Check your pom.xml.
Don't use spring-cloud-starter-consul-all. Use spring-cloud-starter-consul-discovery instead.
That worked for me.
Please check pom.xml to avoid using spring-cloud-starter-consul-all
If you use spring-cloud-starter-consul-all, please write the configuration to bootstrap.yml, the bootstrap.yml configuration file is the first to load.