I am trying to run Terracotta server 10.11 from https://www.terracotta.org/downloads/ to connect it with Ehcache in my Spring boot application. But the problem is that when i run Terracotta server instance using the server\bin\start-tc-server.bat it does not give any error but put the server in DIAGNOSTIC state as shown in logs below.
2022-11-24 14:22:01,660 INFO - Terracotta 5.8.5, as of 2021-12-16 at 22:21:34 UTC (Revision 3695ab2f870d94491c564e87c266555a7d1c096b from UNKNOWN)
2022-11-24 14:22:01,660 INFO - Extensions:
2022-11-24 14:22:01,679 INFO - PID is 23344
2022-11-24 14:22:01,965 INFO - Did not find configuration directory at: C:\Users\user\terracotta\config
2022-11-24 14:22:01,965 INFO - Starting node from config file: C:\Users\user\Downloads\ehcache-clustered-3.10.0-kit\ehcache-clustered-3.10.0-kit\server\conf\cluster.cfg
2022-11-24 14:22:02,266 INFO - Found only one node information in config file: C:\Users\user\Downloads\ehcache-clustered-3.10.0-kit\ehcache-clustered-3.10.0-kit\server\conf\cluster.cfg
2022-11-24 14:22:02,267 INFO - Starting unconfigured node: default-node
2022-11-24 14:22:02,271 INFO - Bootstrapped nomad system with root: C:\Users\user\terracotta\config
2022-11-24 14:22:02,277 INFO - Startup configuration of the node:
client-lease-duration=150s
client-reconnect-window=120s
cluster-name=clustered
failover-priority=availability
offheap-resources=main\:512MB
stripe.1.node.1.bind-address=0.0.0.0
stripe.1.node.1.group-bind-address=0.0.0.0
stripe.1.node.1.group-port=9430
stripe.1.node.1.hostname=localhost
stripe.1.node.1.log-dir=%H/terracotta/logs
stripe.1.node.1.name=default-node
stripe.1.node.1.port=9410
stripe.1.stripe-name=default-stripe
2022-11-24 14:22:02,280 INFO - Logging directory is not set. Logging only to the console
2022-11-24 14:22:02,293 INFO - Available Max Runtime Memory: 1820MB
2022-11-24 14:22:02,314 INFO - Creating server nodeID: NodeID[localhost:9410]
2022-11-24 14:22:02,546 INFO - Initializing LeaseServiceProvider with default lease length of 150000 ms
2022-11-24 14:22:02,548 INFO - Initializing org.terracotta.lease.service.LeaseServiceProvider#4cf92ef3
2022-11-24 14:22:02,549 INFO - Initializing org.terracotta.client.message.tracker.OOOMessageHandlerProvider#40f5b3f9
2022-11-24 14:22:02,556 INFO - Registered MBean with name: DiagnosticRequestHandler
2022-11-24 14:22:02,557 INFO - Registered Diagnostic Service: org.terracotta.nomad.server.NomadServer
2022-11-24 14:22:02,557 INFO - Registered Diagnostic Service: org.terracotta.dynamic_config.api.service.DynamicConfigService
2022-11-24 14:22:02,558 INFO - Registered Diagnostic Service: org.terracotta.dynamic_config.api.service.TopologyService
2022-11-24 14:22:02,558 INFO - Initializing org.terracotta.diagnostic.server.DiagnosticServiceProvider#1bf35727
2022-11-24 14:22:02,561 INFO - Initializing org.terracotta.diagnostic.server.extensions.DiagnosticExtensionsServiceProvider#410ee45a
2022-11-24 14:22:02,804 INFO - Initializing org.terracotta.management.service.monitoring.MonitoringServiceProvider#65c7455b
2022-11-24 14:22:02,804 INFO - Initializing org.terracotta.platform.ServerInfoProvider#240d561b
2022-11-24 14:22:02,806 INFO - Registered dynamic configuration change handler for setting client-reconnect-window: org.terracotta.dynamic_config.server.service.handler.ClientReconnectWindowConfigChangeHandler#74d20602
2022-11-24 14:22:02,809 INFO - Registered dynamic configuration change handler for setting log-dir: org.terracotta.dynamic_config.server.service.handler.NodeLogDirChangeHandler#67c6fc00
2022-11-24 14:22:02,810 INFO - Registered dynamic configuration change handler for setting failover-priority: ConfigChangeHandler#accept()
2022-11-24 14:22:02,810 INFO - Registered dynamic configuration change handler for setting public-hostname: ConfigChangeHandler#accept()
2022-11-24 14:22:02,811 INFO - Registered dynamic configuration change handler for setting public-port: ConfigChangeHandler#accept()
2022-11-24 14:22:02,811 INFO - Registered dynamic configuration change handler for setting cluster-name: ConfigChangeHandler#accept()
2022-11-24 14:22:02,812 INFO - Registered dynamic configuration change handler for setting lock-context: ConfigChangeHandler#accept()
2022-11-24 14:22:02,812 INFO - Registered dynamic configuration change handler for setting logger-overrides: org.terracotta.dynamic_config.server.service.handler.LoggerOverrideConfigChangeHandler#3ba87843
2022-11-24 14:22:02,813 INFO - Registered dynamic configuration change handler for setting tc-properties: org.terracotta.dynamic_config.server.api.SelectingConfigChangeHandler#16df9889
2022-11-24 14:22:02,815 INFO - Initializing org.terracotta.dynamic_config.server.service.DynamicConfigServiceProvider#29ca0612
2022-11-24 14:22:02,815 INFO - Registering implementation-provided service com.tc.services.PlatformServiceProvider#16b645b2
2022-11-24 14:22:02,816 INFO - Registering implementation-provided service com.tc.services.EntityMessengerProvider#3c352805
2022-11-24 14:22:02,816 INFO - Initializing com.tc.objectserver.persistence.NullPlatformStorageServiceProvider#149f57c4
2022-11-24 14:22:02,818 INFO - Registering implementation-provided service com.tc.services.LocalMonitoringProducer#5baa3715
2022-11-24 14:22:02,830 INFO - Creating 4 worker comm threads for default-node - L2_L1
2022-11-24 14:22:02,910 INFO - Registering implementation-provided service com.tc.services.CommunicatorService#7d51aa32
2022-11-24 14:22:02,920 INFO - HealthChecker Started
2022-11-24 14:22:02,952 INFO - Started the server in diagnostic mode
2022-11-24 14:22:02,967 INFO - Server started as default-node
2022-11-24 14:22:02,959 INFO - Terracotta Server instance has started diagnostic listening on all interfaces (address:/0.0.0.0 port:9410)
2022-11-24 14:22:03,177 INFO - Moved to State[ DIAGNOSTIC ]
According to the documentation it should be in ACTIVE state to be running properly. Still i tried to make connection with the server from my Spring boot application but it was also unable to reach it and gave TimeoutException.
I am using the following command to run the server instance:
./start-tc-server.bat -f C:\Users\user\Downloads\ehcache-clustered-3.10.0-kit\ehcache-clustered-3.10.0-kit\server\conf\cluster.cfg
Does anyone have any clue why its not getting to ACTIVE state ? maybe try to run it on your end and see if the server gets to ACTIVE state. Or is there anything i am missing ?
Thanks in Advance.
P.S I tried running older version of Terracotta server from the same downloads page and it easily goes to active state but i cannot use old version since it is not compaitable with Ehcache 3.x
You need to activate the server.
In the kit that you downloaded navigate to /tools/bin and you need to run config-tool.bat activate -f ../../server/conf/cluster.cfg
This will create a folder C:\Users\{user}\terracotta that will contain the configs and logs for the terracotta server, so the next time you start it will use the configs in the folder and automatically go to activate state.
If you need to change configs delete the folder, restart terracotta and activate it again using the config tool.
I have problem with properly configuring spring cloud config. I have dependencies
<spring.cloud.version>2021.0.3</spring.cloud.version>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
Boostrap.yml
spring:
application.name: myapp
profiles:
active: dev
config:
import: optional:configserver:http://ip:8888
cloud:
config:
enabled: true
username: admin
password: secret
Now when my application starts it loads config twice. The first load is by class ConfigServicePropertySourceLocator and is without expected profile. The second is by ConfigServerConfigDataLoader and this time it is with proper profile. It seems that the configs loaded during first load takes precedence and my application does not start.
12:32:54 [ConfigServicePropertySourceLocator:241] - Fetching config from server at : http://ip:8888
12:32:54 [ConfigServicePropertySourceLocator:165] - Located environment: name=myapp, profiles=[default], label=null, version=7d2bc5d68acd8fcca65f34f2074b1860f36e19c6, state=null
12:32:54 [MyApplication:646] - The following 1 profile is active: "dev"
12:32:54 [ConfigServerConfigDataLoader:255] - Fetching config from server at : http://ip:8888
12:32:54 [ConfigServerConfigDataLoader:255] - Located environment: name=myapp, profiles=[dev], label=null, version=7d2bc5d68acd8fcca65f34f2074b1860f36e19c6, state=null
Providing profile using param -Dspring.profiles.active=dev does not help. How to configure profile that can be read by boostrap?
Add file resources -> bootstrap.properties
spring.application.name=myapp
spring.profiles.active=dev
# ip and port of the config server
spring.cloud.config.uri=http://localhost:8888
There are two components in my k8s namespace.
The first component is a simple Spring Boot Web Service with actuator/refresh endpoint exposed. I have mannualy make post request to the endpoint and it will trigger a configuration context refresh succesfully.
The second component is Spring Cloud Kubernetes Configuration Watcher which I pull the image per official document guidance and make it run in my k8s env. Per official spring document, it should detect the changes on configmap with label spring.cloud.kubernetes.config="true" and make post request to the actuator/refresh endpoint of the application whose spring.application.name is equal to the configmap name.
The second component is not working as expectted, and I don't know how to trouble shoot the root cause.
My Springboot application is called spring-boot-demo, the configmap is also named as spring-boot-demo. But I never find any mentions of "spring-boot-demo" in Spring Cloud Kubernetes Configuration Watcher's log and nor can I assure if it has sent post request to related endpoint.
I can only see the logs showed up repeatedly below:
2021-11-22 02:58:53.332 INFO 1 --- [192.168.0.1/...] .w.HttpBasedConfigMapWatchChangeDetector : Added new Kubernetes watch: config-maps-watch-event
2021-11-22 02:58:53.332 INFO 1 --- [192.168.0.1/...] .w.HttpBasedConfigMapWatchChangeDetector : Kubernetes event-based configMap change detector activated
2021-11-22 03:34:06.555 WARN 1 --- [192.168.0.1/...] .f.c.r.EventBasedConfigMapChangeDetector : ConfigMaps watch closed
io.fabric8.kubernetes.client.WatcherException: too old resource version: 5491278743 (5554041906)
at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onStatus(AbstractWatchManager.java:263) [kubernetes-client-5.5.0.jar:na]
at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:247) [kubernetes-client-5.5.0.jar:na]
at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:93) [kubernetes-client-5.5.0.jar:na]
at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:322) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:273) [okhttp-3.14.9.jar:na]
at okhttp3.internal.ws.RealWebSocket$1.onResponse(RealWebSocket.java:209) [okhttp-3.14.9.jar:na]
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:174) [okhttp-3.14.9.jar:na]
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.14.9.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_312]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_312]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_312]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 5491278743 (5554041906)
... 13 common frames omitted
2021-11-22 03:34:06.605 INFO 1 --- [192.168.0.1/...] .f.c.r.EventBasedConfigMapChangeDetector : Added new Kubernetes watch: config-maps-watch-event
2021-11-22 03:34:06.605 INFO 1 --- [192.168.0.1/...] .f.c.r.EventBasedConfigMapChangeDetector : Kubernetes event-based configMap change detector activated
2021-11-22 03:34:06.607 INFO 1 --- [192.168.0.1/...] s.c.k.f.c.Fabric8ConfigMapPropertySource : Loading ConfigMap with name 'spring-cloud-kubernetes-configuration-watcher' in namespace 'my-namespace'
2021-11-22 03:34:06.621 WARN 1 --- [192.168.0.1/...] o.s.c.k.f.config.Fabric8ConfigUtils : config-map with name : 'spring-cloud-kubernetes-configuration-watcher' not present in namespace : 'my-namespace'
2021-11-22 03:34:06.625 WARN 1 --- [192.168.0.1/...] o.s.c.k.f.config.Fabric8ConfigUtils : config-map with name : 'spring-cloud-kubernetes-configuration-watcher-kubernetes' not present in namespace : 'my-namespace'
The docker image I use is springcloud/spring-cloud-kubernetes-configuration-watcher:2.1.0-RC1
Any hint to debug this issue is appreciated.
I'm trying to build sample microservice app using this tutorial.
Jhipster version is 4.0.6
So i've created gateway, service and started consul using this command:
docker-compose -f src/main/docker/consul.yml up
from my gateway directory.
But the error occur on Spring Boot startup, here is the log:
2017-02-22 11:52:25.679 ERROR 3168 --- [ restartedMain] o.s.c.c.c.ConsulPropertySourceLocator : Fail fast is set and there was an error reading configuration from consul.
2017-02-22 11:52:32.491 WARN 3168 --- [ restartedMain] o.s.boot.SpringApplication : Error handling failed (ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#2a5b2096: startup date [Thu Jan 01 03:00:00 AST 1970]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext#5108df79)
com.ecwid.consul.transport.TransportException: java.net.ConnectException: Connection refused: connect
Could you please assist with this issue?
UPDATE:
I've found that app tries to make GET request to URL on staptup:
http://localhost:8500/v1/kv/config/armory,dev/?recurse&token=
But the only data stored in Consul K/V storage is:
KEY: config/application/data
VALUE:
configserver:
name: Docker Consul Service
status: Connected to Consul Server running in Docker
jhipster:
security:
authentication:
jwt:
secret: my-secret-token-to-change-in-production
You must copy your application yaml configurations into your consul instance as explained in the doc into central-server-config directory if consul is running in dev profile or in its git repo if consul is running in prod profile.
So assuming your app is named "armory" you should copy your src/main/resources/config/application.yml to armory.yml and for each profiles (e.g. application-dev.yml to armory-dev.yml)
I'm running Spring Cloud Brixton SR3 - Spring Netflix Eureka, Spring Cloud Config and a Discovery Client app. When the client app is configured with
eureka.client.healthcheck.enabled=true
and
spring.cloud.config.discovery.enabled=true
the app starts up registered with Eureka with a status of "UNKNOWN" and never changes to "UP". If either of those flags is set to false then the app starts up registered as "UP".
I see this series of logging when both flags are enabled/true:
2016-07-13 17:41:38.144 INFO 13180 --- [ main] com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1468456898144, current=UP, previous=STARTING]
2016-07-13 17:41:38.144 DEBUG 13180 --- [nfoReplicator-0] c.n.discovery.InstanceInfoReplicator : Executing on-demand update of local InstanceInfo
2016-07-13 17:41:38.144 DEBUG 13180 --- [nfoReplicator-0] c.n.discovery.InstanceInfoReplicator : Canceling the latest scheduled update, it will be rescheduled at the end of on demand update
2016-07-13 17:41:38.145 INFO 13180 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1468456898145, current=UNKNOWN, previous=UP]
How do I get the app to register as UP with both of these flags enabled/true?
bootstrap.yml
spring:
application:
name: hello-brixton
# Spring Cloud Config
cloud:
config:
name: hello-brixton
discovery:
enabled: true
serviceId: config-service
eureka:
client:
registerWithEureka: true
fetchRegistry: true
serviceUrl:
defaultZone: ${SERVICE_REGISTRY_URL:http://localhost:8080/eureka/}
healthcheck:
enabled: true
I know this was asked a long time ago, but I just ran into the same problem and this was the only questions that directly matched my issue.
I am using Camden.SR2, and if you find the section Eureka’s Health Checks in the reference documentation you will see the following warning:
eureka.client.healthcheck.enabled=true should only be set in
application.yml. Setting the value in bootstrap.yml will cause
undesirable side effects like registering in eureka with an UNKNOWN
status.
That's exactly the problem. If you shift the setting of the property to the application.yml file it all works as expected.