How to view autoconfigure log output during spring boot server start
I have created a spring boot application. It uses a shared library (Spring boot jar via maven dependency). Shared library class is loaded via
META-INF/spring.factories
I have mentioned the classes from the library in spring.factories. The job of shared library is to read Vault role id and Vault
secret id value from application.properties and call a REST API and fetch secrets from Vault. After fetching the secret it sets the value again in system property.
for (Map.Entry<String, String> entry : allSecrets.entrySet())
{
System.setProperty(entry.getKey(), entry.getValue());
}
Everything is working as expected. But I am not able to see logs from shared library in my logs.
shared library's package structure is com.myorg.abc. My spring boot package structure is com.myorg.xyz
I tried the following in application properties.
logging.level.root= DEBUG
logging.level.com.myorg.xyz: DEBUG
logging.level.com.myorg.abc: DEBUG
logging.level.org.springframework.boot.autoconfigure.logging=DEBUG
I am able to get logs only from my application but not from shared library. But when I change the shared library Logger.error to System.out, then I am getting the message in my application. How to view shared library's log in my application.
Spring boot initializes logging at least 3 times. The first happens when SpringApplication is loaded. It creates an SLF4J Logger before anything in Spring is accessed. This causes whatever logging implementation you have chosen to initialize. By default, it will use the logging configuration in the Spring jar. With Log4j 2 you can override this by setting log4j.configurationFile to the location of your desired configuration either as a system property or in a log4j.component.properties file.
Everything Spring does will be logged using this configuration until it initializes the logging configuration again, which is controlled by bootstrap.yml. Finally, your application's logging configuration is initialized which is configured either from application.yml or again from bootstrap.yml.
I replaced org.springframework.boot.env.EnvironmentPostProcessor with org.springframework.context.ApplicationListener in Spring.factories and it fixed the issue. I was able to get logs from shared library in invoking application.
Spring.factories
org.springframework.context.ApplicationListener=com.mypackage.MyClassName
MyClassName.java
public class MyClassName implements ApplicationListener<ApplicationPreparedEvent>
{
private static final Logger LOGGER = LoggerFactory.getLogger(MyClassName.class);
#Override
public void onApplicationEvent(ApplicationPreparedEvent applicationPreparedEvent)
{
ConfigurableEnvironment configurableEnvironment = applicationPreparedEvent.getApplicationContext()
.getEnvironment();
String roleId = configurableEnvironment.getProperty(Constants.VAULT_ROLE_ID_LITERAL);
String secretId = configurableEnvironment.getProperty(Constants.VAULT_SECRET_ID_LITERAL);
...
Optional<String> errorMessage = ServiceUtil.validateSystemProperty(roleId, secretId);
if (!errorMessage.isPresent())
{
Map<String, String> secret = ServiceUtil.getSecret(roleId, secretId);
for (Map.Entry<String, String> entry : secret.entrySet())
{
System.setProperty(entry.getKey(), entry.getValue());
}
LOGGER.info("Successfully populated secrets from Vault in system property");
}
else
{
LOGGER.error("Failed to populate secrets from Vault in system property. Error:{}", errorMessage.get());
}
}
}
application.properties
logging.level.com.myorg.abc: DEBUG
Related
Is there any way to refresh springboot configuration as soon as we change .properties file?
I came across spring-cloud-config and many articles/blogs suggested to use this for a distributed environment. I have many deployments of my springboot application but they are not related or dependent on one another. I also looked at few solutions where they suggested providing rest endpoints to refresh configs manually without restarting application. But I want to refresh configuration dynamically whenever I change .properties file without manual intervention.
Any guide/suggestion is much appreciated.
Can you just use the Spring Cloud Config "Server" and have it signal to your Spring Cloud client that the properties file changed. See this example:
https://spring.io/guides/gs/centralized-configuration/
Under the covers, it is doing a poll of the underlying resource and then broadcasts it to your client:
#Scheduled(fixedRateString = "${spring.cloud.config.server.monitor.fixedDelay:5000}")
public void poll() {
for (File file : filesFromEvents()) {
this.endpoint.notifyByPath(new HttpHeaders(), Collections
.<String, Object>singletonMap("path", file.getAbsolutePath()));
}
}
If you don't want to use the config server, in your own code, you could use a similar scheduled annotation and monitor your properties file:
#Component
public class MyRefresher {
#Autowired
private ContextRefresher contextRefresher;
#Scheduled(fixedDelay=5000)
public void myRefresher() {
// Code here could potentially look at the properties file
// to see if it changed, and conditionally call the next line...
contextRefresher.refresh();
}
}
I am building a new spring boot application deployable to bluemix (cloud foundry) which needs to do the following:
use spring-cloud-cloudfoundry-connector to discover user-provided "properties service": read the service URL and credentials from VCAP_APPLICATION env variable.
This step is completed.
connect to properties service via HTTP call, receive JSON response, parse individual property values and expose them as application properties (in Environment object?)
What would be the correct solution for this in spring-boot app?
In older non-boot Spring app, the property service call would be initiated early in Spring lifecycle by a class that extended PropertySourcesPlaceholderConfigurer and the property collection from the service would be handled inside postProcessBeanFactory() method call of the same class.
public class CustomPopertiesFactory
extends PropertySourcesPlaceholderConfigurer
implements EnvironmentAware {
private Properties properties;
getServiceCredentials() {
// parse VCAP_APPLICATION json
final String localVcapServices = System.getProperty("VCAP_SERVICES");
// extract url, username, pwd to connect to the service
}
connectToService () {
// via HTTP request using RestTemplate
// parse JSON response and add properties to this.properties
... this.properties.put("prop1", valueFromJson);
}
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
getServiceCredentials();
connectToService();
// load values from properties service into app properties
setProperties(properties);
// continue with lifecycle and load properties from other sources
super.postProcessBeanFactory(beanFactory);
}
}
That was painful to maintain and switch between cloud and local spring profiles IMO and I am wondering if spring boot has a better way of handling external properties.
I ended up replacing "properties service" with spring-cloud-config-server
and using spring-cloud-config-client
in my spring boot application to consume properties from spring-cloud-config-server.
I tried to put <jmxConfigurator/> in logback configuration file. I am able to connect jconsole to local JVM running unit tests and interact with the logback mbean. However, when I deploy my web application to a remote Websphere application server and connect jconsole to that remote JVM, I can't see the logback mbean in the MBeans panel.
As a comparison, the web application is built with spring boot, which also register some MBeans by default. I can see MBeans of spring boot in both scenarios.
I investigated a bit further and found out logback always get MBeanServer instance from ManagementFactory.getPlatformMBeanServer(), while spring uses different approaches in Websphere/Weblogic environment.
It appears that in Websphere environment, the MBeanServer instance exposed for remote connection is NOT the default PlatformMBeanServer.
So the question is, how can I register the logback mbean to the WebSphere custom MBeanServer, rather than the default PlatformMBeanServer?
WebSphere custom MBeanServer is favoured because it is better integrated with security and clustering capabilities.
This is my workaround by extending JMXConfigurator.
Just for the record, there is no document endorsing such extension, and I didn't test it with multiple web-applications.
This class inherited most behaviors from JMXConfigurator, but it will register to / unregister from the MBeanServer that is injected by Spring.
#ManagedResource(objectName = AnnotatedJMXConfigurator.NAME, description = "Logback Configuration Management Bean")
#Component
public class AnnotatedJMXConfigurator extends JMXConfigurator {
public static final String NAME = "xxx.ch.qos.logback.classic:Name=default,Type=ch.qos.logback.classic.jmx.JMXConfigurator";
private static final LoggerContext CONTEXT = (LoggerContext) LoggerFactory.getILoggerFactory();
private static final ObjectName OBJECT_NAME;
static {
try {
OBJECT_NAME = new ObjectName(NAME);
} catch (MalformedObjectNameException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
#Autowired
public AnnotatedJMXConfigurator(MBeanServer mbs) {
super(CONTEXT, mbs, OBJECT_NAME);
}
}
I am using spring boot, and I have two external properties files, so that I can easily change its value.
But I hope spring app will reload the changed value when it is updated, just like reading from files. Since property file is easy enough to meet my need, I hope I don' nessarily need a db or file.
I use two different ways to load property value, code sample will like:
#RestController
public class Prop1Controller{
#Value("${prop1}")
private String prop1;
#RequestMapping(value="/prop1",method = RequestMethod.GET)
public String getProp() {
return prop1;
}
}
#RestController
public class Prop2Controller{
#Autowired
private Environment env;
#RequestMapping(value="/prop2/{sysId}",method = RequestMethod.GET)
public String prop2(#PathVariable String sysId) {
return env.getProperty("prop2."+sysId);
}
}
I will boot my application with
-Dspring.config.location=conf/my.properties
I'm afraid you will need to restart Spring context.
I think the only way to achieve your need is to enable spring-cloud. There is a refresh endpoint /refresh which refreshes the context and beans.
I'm not quite sure if you need a spring-cloud-config-server (its a microservice and very easy to build) where your config is stored(Git or svn). Or if its also useable just by the application.properties file in the application.
Here you can find the doc to the refresh scope and spring cloud.
You should be able to use Spring Cloud for that
Add this as a dependency
compile group: 'org.springframework.cloud', name: 'spring-cloud-starter', version: '1.1.2.RELEASE'
And then use #RefreshScope annotation
A Spring #Bean that is marked as #RefreshScope will get special treatment when there is a configuration change. This addresses the problem of stateful beans that only get their configuration injected when they are initialized. For instance if a DataSource has open connections when the database URL is changed via the Environment, we probably want the holders of those connections to be able to complete what they are doing. Then the next time someone borrows a connection from the pool he gets one with the new URL.
Also relevant if you have Spring Actuator
For a Spring Boot Actuator application there are some additional management endpoints:
POST to
/env to update the Environment and rebind #ConfigurationProperties and log levels
/refresh for re-loading the boot strap context and refreshing the #RefreshScope beans
Spring Cloud Doc
(1) Spring Cloud's RestartEndPoint
You may use the RestartEndPoint: Programatically restart Spring Boot application / Refresh Spring Context
RestartEndPoint is an Actuator EndPoint, bundled with spring-cloud-context.
However, RestartEndPoint will not monitor for file changes, you'll have to handle that yourself.
(2) devtools
I don't know if this is for a production application or not. You may hack devtools a little to do what you want.
Take a look at this other answer I wrote for another question: Force enable spring-boot DevTools when running Jar
Devtools monitors for file changes:
Applications that use spring-boot-devtools will automatically restart
whenever files on the classpath change.
Technically, devtools is built to only work within an IDE. With the hack, it also works when launched from a jar. However, I may not do that for a real production application, you decide if it fits your needs.
I know this is a old thread, but it will help someone in future.
You can use a scheduler to periodically refresh properties.
//MyApplication.java
#EnableScheduling
//application.properties
management.endpoint.refresh.enabled = true
//ContextRefreshConfig.java
#Autowired
private RefreshEndpoint refreshEndpoint;
#Scheduled(fixedDelay = 60000, initialDelay = 10000)
public Collection<String> refreshContext() {
final Collection<String> properties = refreshEndpoint.refresh();
LOGGER.log(Level.INFO, "Refreshed Properties {0}", properties);
return properties;
}
//add spring-cloud-starter to the pom file.
Attribues annotated with #Value is refreshed if the bean is annotated with #RefreshScope.
Configurations annotated with #ConfigurationProperties is refreshed without #RefreshScope.
Hope this will help.
You can follow the ContextRefresher.refresh() code implements.
public synchronized Set<String> refresh() {
Map<String, Object> before = extract(
this.context.getEnvironment().getPropertySources());
addConfigFilesToEnvironment();
Set<String> keys = changes(before,
extract(this.context.getEnvironment().getPropertySources())).keySet();
this.context.publishEvent(new EnvironmentChangeEvent(context, keys));
this.scope.refreshAll();
return keys;
}
I'm in the process of building a library which provides an opinionated configuration for applications which use our Spring Cloud Config/Eureka setup. The idea is to deliver this configuration as a custom starter with little or no spring cloud-related boilerplate in individual microservice apps.
At this point, the majority of the shared configuration that I want to put in this library consists of stuff in bootstrap.yml. I'd like to provide bootstrap.yml in my custom starter, but applications using the library still need to be able to provide their own bootstrap.yml, even if only so they can set their spring.application.name properly.
Due to the way bootstrap.yml is loaded from the classpath, Spring seems to ignore the one in the shared lib if the application has its own bootstrap.yml. I can't even use an ApplicationContextInitializer to customize the Environment because of the special way the bootstrap context treats ApplicationContextInitializers.
Does anyone have any recommendations for an approach that would work here? I want to provide a drop-in lib that makes our opinionated bootstrap config work without having to duplicate a boilerplate bootstrap.yml in all of our projects.
You can add a PropertySource in a shared library to the bootstrap properties by using the org.springframework.cloud.bootstrap.BootstrapConfiguration key in the META-INF/spring.factories file.
For example, you can create a library containing the following:
src/main/java/com/example/mylib/MyLibConfig.java
package com.example.mylib;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
#Configuration
#PropertySource("classpath:mylib-config.properties")
public class MyLibConfig {
}
src/main/resources/mylib-config.properties
eureka.instance.public=true
# or whatever...
src/main/resources/META-INF/spring.factories
org.springframework.cloud.bootstrap.BootstrapConfiguration=com.example.mylib.MyLibConfig
More details: http://projects.spring.io/spring-cloud/spring-cloud.html#_customizing_the_bootstrap_configuration
I was able to find a solution to this. The goals of this solution are:
Load the values from a yaml file in a shared library.
Allow applications using the library to introduce their own bootstrap.yml that is also loaded into the Environment.
Values in the bootstrap.yml should override values in the shared yaml.
The main challenge is to inject some code at the appropriate point in the application lifecycle. Specifically, we need to do it after the bootstrap.yml PropertySource is added to the environment (so that we can inject our custom PropertySource in the correct order relative to it), but also before the application starts configuring beans (as our config values control behavior).
The solution I found was to use a custom EnvironmentPostProcessor
public class CloudyConfigEnvironmentPostProcessor implements EnvironmentPostProcessor, Ordered {
private YamlPropertySourceLoader loader;
public CloudyConfigEnvironmentPostProcessor() {
loader = new YamlPropertySourceLoader();
}
#Override
public void postProcessEnvironment(ConfigurableEnvironment env, SpringApplication application) {
//ensure that the bootstrap file is only loaded in the bootstrap context
if (env.getPropertySources().contains("bootstrap")) {
//Each document in the multi-document yaml must be loaded separately.
//Start by loading the no-profile configs...
loadProfile("cloudy-bootstrap", env, null);
//Then loop through the active profiles and load them.
for (String profile: env.getActiveProfiles()) {
loadProfile("cloudy-bootstrap", env, profile);
}
}
}
private void loadProfile(String prefix, ConfigurableEnvironment env, String profile) {
try {
PropertySource<?> propertySource = loader.load(prefix + (profile != null ? "-" + profile: ""), new ClassPathResource(prefix + ".yml"), profile);
//propertySource will be null if the profile isn't represented in the yml, so skip it if this is the case.
if (propertySource != null) {
//add PropertySource after the "applicationConfigurationProperties" source to allow the default yml to override these.
env.getPropertySources().addAfter("applicationConfigurationProperties", propertySource);
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
#Override
public int getOrder() {
//must go after ConfigFileApplicationListener
return Ordered.HIGHEST_PRECEDENCE + 11;
}
}
This custom EnvironmentPostProcessor can be injected via META-INF/spring.factories:
#Environment PostProcessors
org.springframework.boot.env.EnvironmentPostProcessor=\
com.mycompany.cloudy.bootstrap.autoconfig.CloudyConfigEnvironmentPostProcessor
A couple things to note:
The YamlPropertySourceLoader loads yaml properties by profile, so if you are using a multi-document yaml file you need to actually load each profile from it separately, including the no-profile configs.
ConfigFileApplicationListener is the EnvironmentPostProcessor responsible for loading bootstrap.yml (or application.yml for the regular context) into the Environment, so in order to position the custom yaml properties correctly relative to the bootstrap.yml properties precedence-wise, you need to order your custom EnvironmentPostProcessor after ConfigFileApplicationListener.
Edit: My initial answer did not work. I'm replacing it with this one, which does.