There is already a question asking for logging the active configuration, there is a correct answer but the problem is that the configuration is logged only if all beans are correctly instantiated. I would like to log all properties even (mainly) if the application crash at startup. My question is more specific:
How to log all active properties of a spring boot application before the beans instantiation?
For doing this you need to register an ApplicationListener. The event to catch is the ApplicationPreparedEvent, according to the documentation:
ApplicationPreparedEvent is an event published when a SpringApplication is starting up and the
ApplicationContext is fully prepared but not refreshed. The bean
definitions will be loaded and the Environment is ready for use at
this stage.
The main method would look like this:
public static void main(String[] args) {
SpringApplication springApplication = new SpringApplication(MyApplication.class);
springApplication.addListeners(new PropertiesLogger());
springApplication.run(args);
}
I've reused the code of the answer cited in the current question but I've modified it because the context you get is not already refreshed and the structure of the environment is not exactly the same as after the startup of the application. I've also printed the properties by property sources: one for the the system environment, one for the system properties, one for the application configuration properties, etc... Note also that the ApplicationPreparedEvent can be triggered multiple times, and that properties are printed only the first time. See Spring Boot issue #8899 for details.
package com.toto.myapp.util;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.context.event.ApplicationPreparedEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.EnumerablePropertySource;
import org.springframework.core.env.PropertySource;
import java.util.LinkedList;
import java.util.List;
public class PropertiesLogger implements ApplicationListener<ApplicationPreparedEvent> {
private static final Logger log = LoggerFactory.getLogger(PropertiesLogger.class);
private ConfigurableEnvironment environment;
private boolean isFirstRun = true;
#Override
public void onApplicationEvent(ApplicationPreparedEvent event) {
if (isFirstRun) {
environment = event.getApplicationContext().getEnvironment();
printProperties();
}
isFirstRun = false;
}
public void printProperties() {
for (EnumerablePropertySource propertySource : findPropertiesPropertySources()) {
log.info("******* " + propertySource.getName() + " *******");
String[] propertyNames = propertySource.getPropertyNames();
Arrays.sort(propertyNames);
for (String propertyName : propertyNames) {
String resolvedProperty = environment.getProperty(propertyName);
String sourceProperty = propertySource.getProperty(propertyName).toString();
if(resolvedProperty.equals(sourceProperty)) {
log.info("{}={}", propertyName, resolvedProperty);
}else {
log.info("{}={} OVERRIDDEN to {}", propertyName, sourceProperty, resolvedProperty);
}
}
}
}
private List<EnumerablePropertySource> findPropertiesPropertySources() {
List<EnumerablePropertySource> propertiesPropertySources = new LinkedList<>();
for (PropertySource<?> propertySource : environment.getPropertySources()) {
if (propertySource instanceof EnumerablePropertySource) {
propertiesPropertySources.add((EnumerablePropertySource) propertySource);
}
}
return propertiesPropertySources;
}
}
📝 Show the Properties BEFORE application is ready
In my case, I needed to show the properties before the context is loaded. While debugging the app, I would like to log all the properties so that I know what's going on...
☕ Kotlin Implementation
As described at https://www.baeldung.com/spring-boot-environmentpostprocessor, the properties can be collected before the context is loaded through the use of EnvironmentPostProcessor, which is instantiated as part of Spring Factories from the call ConfigFileApplicationListener.loadPostProcessors(). At this point, you can collect all the properties and show in any specific way.
NOTE: While loading properties during this event, the context isn't ready. So, are the loggers. For this reason, the properties can be loaded before the App Banner (if any)
Also, the entry for the spring factory must be present, so create it first
org.springframework.boot.env.EnvironmentPostProcessor=\
cash.app.PropertiesLoggerEnvironmentPostProcessor
Then, create the logger
package cash.app
import org.springframework.boot.SpringApplication
import org.springframework.boot.env.EnvironmentPostProcessor
import org.springframework.core.Ordered
import org.springframework.core.annotation.Order
import org.springframework.core.env.ConfigurableEnvironment
import org.springframework.core.env.EnumerablePropertySource
import java.util.*
/**
* This is to log the properties (config and system) before the app loads. This way, we know what will be loaded
* on the app.
* Note that we can't use the logger because the context hasn't built yet at the time it loads the properties twice.
*
* As an event consumer, the method ConfigFileApplicationListener.onApplicationEnvironmentPreparedEvent is called
* while the context is building. The process is described at https://www.baeldung.com/spring-boot-environmentpostprocessor
* and one important aspect is that this class is an EnvironmentPostProcessor, only loaded before the App is loaded
* with the assistance of the "src/main/resources/META-INF/spring.factories". It is loaded by the
* ConfigFileApplicationListener.loadPostProcessors(), which looks for the list of classses in the factories.
*
* https://www.baeldung.com/spring-boot-environmentpostprocessor explains how to create AutoConfiguration classes for
* shared libraries. For the case of config, the reload of properties is detailed and explained on the docs at
* https://www.baeldung.com/spring-reloading-properties
*
* TODO: We need to hide the secrets, if they are defined here.
*
* #author Marcello.DeSales#gmail.com
*/
#Order(Ordered.LOWEST_PRECEDENCE)
class PropertiesLoggerEnvironmentPostProcessor : EnvironmentPostProcessor {
companion object {
/**
* Sharing is started immediately and never stops.
*/
private var numberOfPasses: Int = 0
private var systemProperties: MutableMap<String, String> = mutableMapOf()
}
override fun postProcessEnvironment(environment: ConfigurableEnvironment, application: SpringApplication) {
for (propertySource in findPropertiesPropertySources(environment)) {
// Avoid printing the systemProperties twice
if (propertySource.name.equals("systemProperties")) {
numberOfPasses = numberOfPasses?.inc()
} else {
System.out.println("******* \" + ${propertySource.getName()} + \" *******" )
}
// Adaptation of https://stackoverflow.com/questions/48212761/how-to-log-all-active-properties-of-a-spring-boot-application-before-the-beans-i/48212783#48212783
val propertyNames = propertySource.propertyNames
Arrays.sort(propertyNames)
for (propertyName in propertyNames) {
val resolvedProperty = environment!!.getProperty(propertyName!!)
val sourceProperty = propertySource.getProperty(propertyName).toString()
if (resolvedProperty == sourceProperty) {
if (propertySource.name.equals("systemProperties")) {
systemProperties.put(propertyName, resolvedProperty)
} else {
System.out.println( "${propertyName}=${resolvedProperty}" )
}
} else {
if (propertySource.name.equals("systemProperties")) {
systemProperties.put(propertyName, resolvedProperty ?: "")
} else {
System.out.println( "${propertyName}=${sourceProperty} ----- OVERRIDDEN =>>>>>> ${propertyName}=${resolvedProperty}" )
}
}
}
}
// The system properties show up twice in the process... The class is called twice and we only print it in the end.
if (numberOfPasses == 2) {
System.out.println("******* \" System Properties \" *******")
val sysPropertyNames = systemProperties.keys.sorted()
for (sysPropertyName in sysPropertyNames) {
val sysPropertyValue = systemProperties!!.get(sysPropertyName!!)
System.out.println( "${sysPropertyName}=${sysPropertyValue}" )
}
}
}
private fun findPropertiesPropertySources(environment: ConfigurableEnvironment): List<EnumerablePropertySource<*>> {
val propertiesPropertySources: MutableList<EnumerablePropertySource<*>> = LinkedList()
for (propertySource in environment!!.propertySources) {
if (propertySource is EnumerablePropertySource<*>) {
if (propertySource.name.equals("systemProperties") || propertySource.name.contains("applicationConfig:")) {
propertiesPropertySources.add(propertySource)
}
}
}
return propertiesPropertySources.asReversed()
}
}
🔊 Example Logs
Here's the loggers during the bootstrap of one of my services
/Users/marcellodesales/.gradle/jdks/jdk-14.0.2+12/Contents/Home/bin/java -XX:TieredStopAtLevel=1 -noverify -Dspring.output.ansi.enabled=always
....
....
2022-02-22T21:24:39 INFO [app=springAppName_IS_UNDEFINED,prof=observed,db,ppd_dev][tid=,sid=,sxp=][uid=] 74720 --- [ restartedMain] o.s.b.devtools.restart.ChangeableUrls : The Class-Path manifest attribute in /Users/marcellodesales/.gradle/caches/modules-2/files-2.1/com.sun.xml.bind/jaxb-core/2.2.7s-codec-1.11.jar
2022-02-22T21:24:39 INFO [app=springAppName_IS_UNDEFINED,prof=observed,db,ppd_dev][tid=,sid=,sxp=][uid=] 74720 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
******* " + applicationConfig: [classpath:/application.yaml] + " *******
management.endpoint.health.show-details=always
management.endpoints.web.base-path=/actuator ==========>>>>>> OVERRIDDEN =========>>>>>> management.endpoints.web.base-path=/orchestrator/actuator
management.endpoints.web.exposure.include=*
management.metrics.web.server.request.autotime.enabled=true
spring.application.name=orchestrator-service
spring.boot.admin.client.enabled=false ==========>>>>>> OVERRIDDEN =========>>>>>> spring.boot.admin.client.enabled=true
spring.cloud.discovery.client.composite-indicator.enabled=false
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
******* " + applicationConfig: [classpath:/application-ppd_dev.yaml] + " *******
spring.datasource.url=jdbc:postgresql://localhost:6433/supercash?createDatabaseIfNotExist=true ==========>>>>>> OVERRIDDEN
=========>>>>>> spring.datasource.url=jdbc:postgresql://localhost:6433/supercash?createDatabaseIfNotExist\=true
spring.devtools.livereload.enabled=true
spring.mail.properties.mail.smtp.starttls.required=true
spring.mail.test-connection=true
******* " System Properties " *******
LOG_LEVEL_PATTERN=%5p [,%X{X-B3-TraceId:-},%X{X-B3-SpanId:-},%X{X-Span-Export:-}]
PID=74720
com.sun.management.jmxremote=
file.encoding=UTF-8
ftion
java.vm.specification.version=14
java.vm.vendor=AdoptOpenJDK
java.vm.version=14.0.2+12
jboss.modules.system.pkgs=com.intellij.rt
jdk.debug=release
line.separator=
os.arch=x86_64
os.name=Mac OS X
os.version=10.16
user.name=marcellodesales
user.timezone=America/Los_Angeles
2022-02-22T21:25:16 DEBUG [app=orchestrator-service,prof=observed,db,ppd_dev][tid=,sid=,sxp=][uid=] 74720 --- [ restartedMain] o.s.b.c.c.ConfigFileApplicationListener : Activated activeProfiles observed,db,ppd_dev
_____ _____ _
/ ____| / ____| | |
| (___ _ _ _ __ ___ _ __| | __ _ ___| |__
2022-02-22T20:41:08 INFO [app=orchestrator-service,prof=observed,db,ppd_dev][tid=,sid=,sxp=][uid=] 74181 --- [ restartedMain]
Related
I have simple spring boot application with Kafka Consumers that looks like
#KafkaListener(topics="topic", groupId="SOME_CONSTANT") {
....
}
What I am required to do Is to add optional spring boot property (from env variables but that is not important) lets say:
myapp.env: TEST
And when that variable is present I should automatically update consumer group to be
SOME_CONSTANT-TEST
I am playing with SPEL
#KafkaListener(topics="topic", groupId="#{ '${myApp.env}' == null ? 'SOME_CONSTANT' : 'SOME_CONSTANT' + '-' + '${myApp.env}}'") {
....
}
But that does not seem to work :/ Any Ideas?
You can use the T operator to read the constant's value, and use the colon ':' for the case when there's no env variable:
#KafkaListener(topics="topic", groupId="#{ '${my.app.env:}' == '' ? T(com.mypackage.MyListener).SOME_CONSTANT : T(com.mypackage.MyListener).SOME_CONSTANT + '-' + '${my.app.env:}'}")
Here's a sample application with this solution:
package org.spring.kafka.playground;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaOperations;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.handler.annotation.Header;
import org.springframework.stereotype.Component;
#SpringBootApplication
public class SO71291726 {
public static void main(String[] args) {
ConfigurableApplicationContext context = SpringApplication.run(SO71291726.class, args);
try {
Thread.sleep(10000);
}
catch (InterruptedException e) {
Thread.interrupted();
throw new RuntimeException("Interrupted");
}
KafkaOperations kafkaTemplate = context.getBean("kafkaTemplate", KafkaOperations.class);
kafkaTemplate.send("topic", "My message");
}
Logger log = LoggerFactory.getLogger(this.getClass());
public static final String SOME_CONSTANT = "my-group-id-constant";
#Component
class MyListener {
#KafkaListener(topics="topic", groupId="#{ '${71291726.my.app.env:}' == '' ? T(org.spring.kafka.playground.SO71291726).SOME_CONSTANT : T(org.spring.kafka.playground.SO71291726).SOME_CONSTANT + '-' + '${71291726.my.app.env:}'}")
void listen(String message, #Header(KafkaHeaders.GROUP_ID) String groupId) {
log.info("Received message {} from group id {} ", message, groupId);
}
}
}
Output:
2022-02-28 14:26:14.733 INFO 18841 --- [ntainer#0-0-C-1] 1291726$$EnhancerBySpringCGLIB$$cf264156 : Received message My message from group id my-group-id-constant
If I add 71291726.my.app.env = TEST to the application.properties file:
2022-02-28 14:34:03.900 INFO 18870 --- [ntainer#0-0-C-1] 1291726$$EnhancerBySpringCGLIB$$e1a5933e : Received message My message from group id my-group-id-constant-TEST
I have this specific problem with my yml configuration file.
I have a multi-module maven project as follows:
app
|-- core
|-- web
|-- app
I have this configuration file in core project
#Configuration
#PropertySource("core-properties.yml")
public class CoreConfig {
}
And this mapping:
#Component
#ConfigurationProperties(prefix = "some.key.providers.by")
#Getter
#Setter
public class ProvidersByMarket {
private Map<String, List<String>> market;
}
Here are my core-properties.yml
some.key.providers:
p1: 'NAME1'
p2: 'NAME2'
some.key.providers.by.market:
de:
- ${some.key.providers.p1}
- ${some.key.providers.p2}
gb:
- ${some.key.providers.p1}
When I load the file via profile activation, for example, rename the file to application-core-properties.yml and then -Dspring.profiles.active=core-propertiesit does work however if when I try to load the file via #PropertySource("core-properties.yml") it does not and I get the following error:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-03-27 10:07:36.397 -ERROR 13474|| --- [ restartedMain] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under 'some.key.providers.by.market' to java.util.Map<java.lang.String, java.util.List<java.lang.String>>:
Reason: No converter found capable of converting from type [java.lang.String] to type [java.util.Map<java.lang.String, java.util.List<java.lang.String>>]
Action:
Update your application's configuration
Process finished with exit code 1
Bacouse you don't have equivalent properties stracture,
example
spring:
profiles: test
name: test-YAML
environment: test
servers:
- www.abc.test.com
- www.xyz.test.com
---
spring:
profiles: prod
name: prod-YAML
environment: production
servers:
- www.abc.com
- www.xyz.com
And config class should be
#Configuration
#EnableConfigurationProperties
#ConfigurationProperties
public class YAMLConfig {
private String name;
private String environment;
private List<String> servers = new ArrayList<>();
// standard getters and setters
I have resolved the issue implementing the following PropertySourceFactory detailed described in here
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Properties;
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.core.env.PropertySource;
import org.springframework.core.io.support.EncodedResource;
import org.springframework.core.io.support.PropertySourceFactory;
import org.springframework.lang.Nullable;
public class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(#Nullable String name, EncodedResource resource) throws IOException {
Properties propertiesFromYaml = loadYamlIntoProperties(resource);
String sourceName = name != null ? name : resource.getResource().getFilename();
return new PropertiesPropertySource(sourceName, propertiesFromYaml);
}
private Properties loadYamlIntoProperties(EncodedResource resource) throws FileNotFoundException {
try {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
factory.afterPropertiesSet();
return factory.getObject();
} catch (IllegalStateException e) {
// for ignoreResourceNotFound
Throwable cause = e.getCause();
if (cause instanceof FileNotFoundException)
throw (FileNotFoundException) e.getCause();
throw e;
}
}
}
I had a similar problem and found a workaround like this:
diacritic:
isEnabled: true
chars: -> I wanted this to be parsed to map but it didn't work
ą: a
ł: l
ę: e
And my solution so far:
diacritic:
isEnabled: true
chars[ą]: a -> these ones could be parsed to Map<String, String>
chars[ł]: l
chars[ę]: e
I've create a simple vaadin portlet in a Liferay 7/DXP or osgi 6 context and I noticed that my References do not get garbage collected if I use osgi declarative services with a prototype scope, but they do if I use serviceObjects. Why?
Note: I've updated this question and put an even more simple example at the end.
My main component is a prototype component which has a prototype reference to an object. If I use the osgi declarative services to declare my dependency (the HelloPresenter in the following listing), then my dependency won't be released and stays in the heap forever:
import com.vaadin.server.VaadinRequest;
import com.vaadin.ui.UI;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.component.annotations.ReferenceScope;
import org.osgi.service.component.annotations.ServiceScope;
/**
* Created by marcel
*/
#Component(
property = {
"com.liferay.portlet.display-category=VaadinHelloMvp",
"javax.portlet.display-name=VaadinHelloMvp",
"javax.portlet.name=VaadinHelloMvp",
"com.vaadin.osgi.liferay.portlet-ui=true"
},
service = UI.class,
scope = ServiceScope.PROTOTYPE
)
public class VaadinHelloMvpPortlet extends UI {
#Reference(scope = ReferenceScope.PROTOTYPE_REQUIRED)
private HelloPresenter helloPresenter;
#Override
protected void init(VaadinRequest request) {
this.setContent(helloPresenter.getViewComponent());
}
}
So I've tried get my service instance for my HelloPresenter programmatically, which this works fine:
import com.vaadin.server.VaadinRequest;
import com.vaadin.ui.UI;
import org.osgi.framework.Bundle;
import org.osgi.framework.FrameworkUtil;
import org.osgi.framework.ServiceObjects;
import org.osgi.framework.ServiceReference;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.ServiceScope;
/**
* Created by marcel
*/
#Component(
property = {
"com.liferay.portlet.display-category=VaadinHelloMvp",
"javax.portlet.display-name=VaadinHelloMvp",
"javax.portlet.name=VaadinHelloMvp",
"com.vaadin.osgi.liferay.portlet-ui=true"
},
service = UI.class,
scope = ServiceScope.PROTOTYPE
)
public class VaadinHelloMvpPortlet extends UI {
private HelloPresenter helloPresenter;
#Override
protected void init(VaadinRequest request) {
Bundle bundle = FrameworkUtil.getBundle(HelloPresenter.class);
ServiceReference<HelloPresenter> serviceReference = bundle.getBundleContext().getServiceReference(HelloPresenter.class);
ServiceObjects<HelloPresenter> serviceObjects = bundle.getBundleContext().getServiceObjects(serviceReference);
helloPresenter = serviceObjects.getService();
this.addDetachListener(event -> serviceObjects.ungetService(helloPresenter));
helloPresenter.init();
this.setContent(helloPresenter.getViewComponent());
}
}
So I wonder why my HelloPresenter won't be released by the osgi framework in the first scenario, but it does in the second?
My portlet (UI) object is also created with
serviceObjects.getService();
and released with
serviceObjects.ungetService(uiObject);
and I tried other scenarios where I set another prototype reference in my HelloPresenter, which will also produce a reference which won't be released and garbage collected. So my experience was that, whenever you create a service object which contains a prototype reference, the reference won't get released and stucks in the jvm heap, after releasing the service object
So I got the idea that either I am doing something wrong or missed a param which makes my prototype reference never getting released OR there is something wrong with mixing osgi declarative service and serviceObjects ...
Do you know how I can make my first example work? I want to use the annotations and also be sure that they become garbage collected after closing my portlet ui.
UPDATE
I've created an even more example with a singleton component to execute a gogo shell command and a prototype object which also contains a prototype reference:
#Component(
service = GogoShellService.class,
scope = ServiceScope.SINGLETON,
immediate = true,
property =
{
"osgi.command.scope=test",
"osgi.command.function=atest",
}
)
public class GogoShellService {
public String atest() {
Bundle bundle = FrameworkUtil.getBundle(APrototypeComponent.class);
ServiceReference<APrototypeComponent> serviceReference = bundle.getBundleContext().getServiceReference(APrototypeComponent.class);
ServiceObjects<APrototypeComponent> serviceObjects = bundle.getBundleContext().getServiceObjects(serviceReference);
APrototypeComponent service = serviceObjects.getService();
String s = "Hello From: " + service.sayHello();
serviceObjects.ungetService(service);
return s;
}
}
#Component(scope = ServiceScope.PROTOTYPE, service = APrototypeComponent.class, servicefactory = true)
public class APrototypeComponent {
#Reference(scope = ReferenceScope.PROTOTYPE_REQUIRED)
AProInAProComp aProInAProComp;
public String sayHello() {
String hello = "Hello From " + this.getClass().getSimpleName() + "(" + this.toString() + ") ";
if (aProInAProComp != null) {
hello += aProInAProComp.sayHello();
}
return hello;
}
}
#Component(scope = ServiceScope.PROTOTYPE, service = AProInAProComp.class)
public class AProInAProComp {
public String sayHello() {
return "Hello From " + this.getClass().getSimpleName() + "(" + this.toString() + ")";
}
}
Every time I execute the command (GogoShellService#atest) a new prototype instance is created and should also be destroyed afterwards, but I still can see this object in my heap and running the garbage collection doesn't clean this up...
osgi debug output is the following:
[org_apache_felix_scr:94] getService {de.foo.bar.bax.gogo.GogoShellService}={osgi.command.function=atest, component.name=de.foo.bar.bax.gogo.GogoShellService, component.id=2944, osgi.command.scope=test, service.id=7827, service.bundleid=51, service.scope=bundle}: stack of references: []
APrototypeComponent(2942)] ServiceFactory.getService()
AProInAProComp(2941)] ServiceFactory.getService()
AProInAProComp(2941)] This thread collected dependencies
AProInAProComp(2941)] getService (ServiceFactory) dependencies collected.
AProInAProComp(2941)] Querying state active
AProInAProComp(2941)] Changed state from active to active
APrototypeComponent(2942)] This thread collected dependencies
APrototypeComponent(2942)] getService (ServiceFactory) dependencies collected.
APrototypeComponent(2942)] Querying state satisfied
APrototypeComponent(2942)] For dependency aProInAProComp, optional: false; to bind: [[MultiplePrototypeRefPair: ref: [{de.foo.bar.bax.checkosgi.AProInAProComp}={component.name=de.foo.bar.bax.checkosgi.AProInAProComp, component.id=2941, service.id=7823, service.bundleid=51, service.scope=prototype}] has service: [true]]]
APrototypeComponent(2942)] Changed state from satisfied to active
APrototypeComponent(2942)] ServiceFactory.ungetService()
APrototypeComponent(2942)] DependencyManager: aProInAProComp close component unbinding from org.apache.felix.scr.impl.manager.ComponentContextImpl#3927bc1d at tracking count 1 refpairs: [[MultiplePrototypeRefPair: ref: [{de.foo.bar.bax.checkosgi.AProInAProComp}={component.name=de.foo.bar.bax.checkosgi.AProInAProComp, component.id=2941, service.id=7823, service.bundleid=51, service.scope=prototype}] has service: [true]]]
APrototypeComponent(2942)] Querying state active
APrototypeComponent(2942)] Changed state from active to satisfied
I don't see why my prototype instances cannot get garbage collected...
Update for new readers
As of Apache Felix SCR 2.1.14 this problem should be fixed and the perfectly valid code from the original question will no longer result in erroneous behaviour.
Original Answer
Firstly, thank you for working to create a simple example to demonstrate your question!
You are absolutely right that SCR should release all of your component's references after it is deactivated. For a ReferenceScope.PROTOTYPE_REQUIRED scoped reference this should result in the service instance being released and tidied up.
Sadly it seems as though this feature of SCR hasn't been working for a while. I raised https://issues.apache.org/jira/browse/FELIX-5974 on your behalf, so it should be fixed soon, but for now the "easy" workaround is to take control of the lifecycle yourself.
#Component(
service = GogoShellService.class,
scope = ServiceScope.SINGLETON,
immediate = true,
property =
{
"osgi.command.scope=test",
"osgi.command.function=atest",
}
)
public class GogoShellService {
public String atest() {
Bundle bundle = FrameworkUtil.getBundle(APrototypeComponent.class);
ServiceReference<APrototypeComponent> serviceReference = bundle.getBundleContext().getServiceReference(APrototypeComponent.class);
ServiceObjects<APrototypeComponent> serviceObjects = bundle.getBundleContext().getServiceObjects(serviceReference);
APrototypeComponent service = serviceObjects.getService();
String s = "Hello From: " + service.sayHello();
serviceObjects.ungetService(service);
return s;
}
}
#Component(scope = ServiceScope.PROTOTYPE, service = APrototypeComponent.class, servicefactory = true)
public class APrototypeComponent {
#Reference(scope = ReferenceScope.PROTOTYPE_REQUIRED)
ComponentServiceObjects<AProInAProComp> cso;
AProInAProComp aProInAProComp
#Activate
void start() {
aProInAProComp = cso.getService();
}
#Deactivate
void stop() {
cso.ungetService(aProInAProComp);
aProInAProComp = null;
}
public String sayHello() {
String hello = "Hello From " + this.getClass().getSimpleName() + "(" + this.toString() + ") ";
if (aProInAProComp != null) {
hello += aProInAProComp.sayHello();
}
return hello;
}
}
#Component(scope = ServiceScope.PROTOTYPE, service = AProInAProComp.class)
public class AProInAProComp {
public String sayHello() {
return "Hello From " + this.getClass().getSimpleName() + "(" + this.toString() + ")";
}
}
I have written a scheduler using OSGi R6 annotations but it doesn't seem to run :
package com.aem.sites.interfaces;
import org.osgi.service.metatype.annotations.AttributeDefinition;
import org.osgi.service.metatype.annotations.AttributeType;
import org.osgi.service.metatype.annotations.ObjectClassDefinition;
#ObjectClassDefinition(name = "Scheduler Configuration for Weather", description = "Configuration file for Scheduler")
public #interface SchedulerConfiguration {
#AttributeDefinition(
name = "sample parameter",
description="Sample String parameter",
type = AttributeType.STRING
)
public String parameter() default "scheduler";
#AttributeDefinition(
name = "Concurrent",
description = "Schedule task concurrently",
type = AttributeType.BOOLEAN
)
boolean scheduler_concurrent() default true;
#AttributeDefinition(
name = "Expression",
description = "Cron-job expression. Default: run every minute.",
type = AttributeType.STRING
)
String scheduler_expression() default "0 * * * * ?";
}
and
package com.aem.sites.schedulers;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.metatype.annotations.Designate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.aem.sites.interfaces.SchedulerConfiguration;
#Component(immediate = true,
configurationPid = "com.aem.sites.schedulers.WeatherServiceScheduler")
#Designate(ocd=SchedulerConfiguration.class)
public class WeatherServiceScheduler implements Runnable {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
private String myParameter;
#Override
public void run() {
logger.info("*******************************************Sample OSGi Scheduler is now running", myParameter);
}
#Activate
public void activate(SchedulerConfiguration config) {
logger.info("*******************************************weather service scheduler"+ myParameter);
myParameter = config.parameter();
}
}
I am following this https://github.com/nateyolles/aem-osgi-annotation-demo/blob/master/core/src/main/java/com/nateyolles/aem/osgiannotationdemo/core/schedulers/SampleOsgiScheduledTask.java but looks like I am doing something wrong here. Not sure what though.
Thanks in advance
In your WeatherSchedulerService class, you are not registering it as a service. Instead of configurationPid, you can do like this service = Runnable.class.
The correct way to create a SlingScheduler using OSGi R6 annotations is as follows -
Create your OSGi configuration class
import org.osgi.service.metatype.annotations.AttributeDefinition;
import org.osgi.service.metatype.annotations.AttributeType;
import org.osgi.service.metatype.annotations.ObjectClassDefinition;
#ObjectClassDefinition(name = "Sling Scheduler Configuration", description = "This configuration is used to demonstrates a sling scheduler in action")
public #interface SchedulerConfiguration {
#AttributeDefinition(
name = "Scheduler name",
description = "Name of the scheduler",
type = AttributeType.STRING)
public String name() default "Custom Sling Scheduler";
#AttributeDefinition(
name = "Enabled",
description = "Flag to enable/disable a scheduler",
type = AttributeType.STRING)
public boolean enabled() default false;
#AttributeDefinition(
name = "Cron expression",
description = "Cron expression used by the scheduler",
type = AttributeType.STRING)
public String cronExpression() default "0 * * * * ?";
#AttributeDefinition(
name = "Custom parameter",
description = "Custom parameter to showcase the usage of a sling scheduler",
type = AttributeType.STRING)
public String customParameter();
}
Create your Scheduler class as a service. For creating an OSGi service using R6 annotations we use #Component(service=<your-interface>.class,...).
Thus, create a service as follows
import org.apache.sling.commons.scheduler.ScheduleOptions;
import org.apache.sling.commons.scheduler.Scheduler;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Deactivate;
import org.osgi.service.component.annotations.Modified;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.metatype.annotations.Designate;
import org.redquark.aem.learning.core.configurations.SchedulerConfiguration;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
#Component(immediate = true, service = Runnable.class)
#Designate(ocd = SchedulerConfiguration.class)
public class CustomScheduler implements Runnable {
// Logger
private final Logger log = LoggerFactory.getLogger(this.getClass());
// Custom parameter that is to be read from the configuration
private String customParameter;
// Id of the scheduler based on its name
private int schedulerId;
// Scheduler instance injected
#Reference
private Scheduler scheduler;
/**
* Activate method to initialize stuff
*
* #param schedulerConfiguration
*/
#Activate
protected void activate(SchedulerConfiguration schedulerConfiguration) {
schedulerId = schedulerConfiguration.name().hashCode();
customParameter = schedulerConfiguration.customParameter();
}
/**
* Modifies the scheduler id on modification
*
* #param schedulerConfiguration
*/
#Modified
protected void modified(SchedulerConfiguration schedulerConfiguration) {
// Removing scheduler
removeScheduler();
// Updating the scheduler id
schedulerId = schedulerConfiguration.name().hashCode();
// Again adding the scheduler
addScheduler(schedulerConfiguration);
}
/**
* This method deactivates the scheduler and removes it
*
* #param schedulerConfiguration
*/
#Deactivate
protected void deactivate(SchedulerConfiguration schedulerConfiguration) {
// Removing the scheduler
removeScheduler();
}
/**
* This method removes the scheduler
*/
private void removeScheduler() {
log.info("Removing scheduler: {}", schedulerId);
// Unscheduling/removing the scheduler
scheduler.unschedule(String.valueOf(schedulerId));
}
/**
* This method adds the scheduler
*
* #param schedulerConfiguration
*/
private void addScheduler(SchedulerConfiguration schedulerConfiguration) {
// Check if the scheduler is enabled
if (schedulerConfiguration.enabled()) {
// Scheduler option takes the cron expression as a parameter and run accordingly
ScheduleOptions scheduleOptions = scheduler.EXPR(schedulerConfiguration.cronExpression());
// Adding some parameters
scheduleOptions.name(schedulerConfiguration.name());
scheduleOptions.canRunConcurrently(false);
// Scheduling the job
scheduler.schedule(this, scheduleOptions);
log.info("Scheduler added");
} else {
log.info("Scheduler is disabled");
}
}
/**
* Overridden run method to execute Job
*/
#Override
public void run() {
log.info("Custom Scheduler is now running using the passed custom paratmeter, customParameter {}",
customParameter);
}
In the activate() method, we are reading the required values. Then we are getting the schedulerId from the scheduler name.
The modified() method recalculates the schedulerId in case the OSGi configuration is modified.
In the addScheduler() method, we are registering the scheduler using the Scheduler API.
For more information and step by step execution, you can see my blog post as well - Day 13: Schedulers in AEM
I hope this helps. Happy coding!
There is no need for configurationPid in the class annotation, and also you are missing service=Runnable.class which should follow immediate=true, i.e. the class declaration should look like:
#Component(immediate = true, service=Runnable.class)
#Designate(ocd=SchedulerConfiguration.class)
public class WeatherServiceScheduler implements Runnable {
I was looking at the spring-boot documentation located here
Specifically the section regarding the order in which the properties are considered:
More specifically:
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Let me first mention that I am not having any issues loading profile specific configurations using this approach(provided that the files are located in classpath:/ or classpath:/config.
However, what I am hoping to do is implement a convention like the following:
classpath:/default/application.properties
classpath:/{profile}/application.properties
Furthermore I'd like to achieve this configuration without making use of the spring.config.location property. I'm pretty new to Spring Boot so I'm looking for some hints as how to how I would implement this convention. Based on my research It seems that this might be achievable by adding a custom ConfigFileApplicationListener. Please let me know if that is a sensible starting point or any other ideas that might be better.
Update:
It seems that if I could programmatically build out the spring.config.location list of properties I could pass in locations such as classpath:/default, classpath:{profile}. based on the spring.profiles.active environment variable. The following ConfigFileApplicationListener seems like its the one I want to call:
public void setSearchLocations(String locations)
However, I'm not sure where in the lifecycle I would make such a call.
So here is what I managed to come up with, not sure if I'll even go with this solution but I figured I'll offer it up in case there is any helpful feedback.
So I resorted to trying to set the call the setSearchLocations(String locations) method on the ConfigFileApplicationListener after it has been added to the SpringApplication but before its triggered. I did this by adding a new listener that also implements Ordered and made sure it ran before ConfigFileApplicationListener. This seems to do what I want but I'm still thinking there is a more elegant approach. I especially dont like having to iterate over the Listeners.
public class LocationsSettingConfigFileApplicationListener implements
ApplicationListener<ApplicationEnvironmentPreparedEvent>, Ordered {
/**
* this should run before ConfigFileApplicationListener so it can set its
* state accordingly
*/
#Override
public int getOrder() {
return ConfigFileApplicationListener.DEFAULT_ORDER - 1;
}
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
SpringApplication app = event.getSpringApplication();
ConfigurableEnvironment env = event.getEnvironment();
for (ApplicationListener<?> listener : app.getListeners()) {
if (listener instanceof ConfigFileApplicationListener) {
ConfigFileApplicationListener cfal = (ConfigFileApplicationListener) listener;
//getSearchLocations omitted
cfal.setSearchLocations(getSearchLocations(env));
}
}
}
A solution that doesn't require writing a new class:
public static void main(String[] args) {
SpringApplication app = new SpringApplication();
app.getListeners().stream()
.filter(listener -> listener instanceof ConfigFileApplicationListener)
.forEach(configListener -> {
((ConfigFileApplicationListener) configListener).setSearchLocations(mySearchLocations);
((ConfigFileApplicationListener) configListener).setSearchNames(mySearchNames);
});
app.setSources(singleton(MyClassName.class));
app.run(args);
}
We did something similar with an EnvironmentPostProcessor to achieve the following naming convention:
System properties
environment variables
"random" (not used, but we kept that default PropertySource)
file:${foo.home}/foo-<profile>.properties
classpath*:<appName-profile>.properties
classpath*:application-profile.properties
classpath*:<appName>.properties
classpath*:application.properties
classpath*:meta.properties
Some applications do not have their own <appName>; those that do call setApplicationName in the main class's static initializer to use those two additional files.
The hacky part here is that we do not exclude the default ConfigFileApplicationListener, but undo it by removing PropertySource ConfigFileApplicationListener.APPLICATION_CONFIGURATION_PROPERTY_SOURCE_NAME.
File FooPropertiesEnvPostProcessor.java
package example.foo.utils.spring;
import static org.springframework.core.env.AbstractEnvironment.DEFAULT_PROFILES_PROPERTY_NAME;
import java.io.IOException;
import java.util.List;
import java.util.Spliterators;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
import org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.context.config.ConfigFileApplicationListener;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.boot.env.PropertySourceLoader;
import org.springframework.boot.env.PropertySourcesLoader;
import org.springframework.boot.logging.LoggingApplicationListener;
import org.springframework.core.Ordered;
import org.springframework.core.env.AbstractEnvironment;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.core.env.PropertyResolver;
import org.springframework.core.env.PropertySource;
import org.springframework.core.env.StandardEnvironment;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternUtils;
import org.springframework.core.io.support.SpringFactoriesLoader;
/**
* Configures environment properties according to the FOO conventions.
*/
public class FooPropertiesEnvPostProcessor implements EnvironmentPostProcessor, Ordered {
/**
* Order before LoggingApplicationListener and before
* AutowiredAnnotationBeanPostProcessor. The position relative to
* ConfigFileApplicationListener (which we want to override) should not
* matter: If it runs before this, we remove its PropertySource; otherwise,
* its PropertySource remains but should do no harm as it is added at the
* end.
*/
public static final int ORDER
= Math.min(LoggingApplicationListener.DEFAULT_ORDER, new AutowiredAnnotationBeanPostProcessor().getOrder()) - 1;
static {
System.setProperty(AbstractEnvironment.DEFAULT_PROFILES_PROPERTY_NAME,
System.getProperty(AbstractEnvironment.DEFAULT_PROFILES_PROPERTY_NAME, "production"));
}
public FooPropertiesEnvPostProcessor() {
}
/**
* Property key used as the application (sub-project) specific part in
* properties file names.
* <p>
* <strong>Note:</strong> Direct access to this property key is meant for
* tests which set the property in an annotation (e.g.
* {#link IntegrationTest}). However, SpringBootApplications which need to
* set this system property before Spring initialization should call
* {#link #setApplicationName(String) setApplicationName} instead.
* </p>
*/
public static final String APP_KEY = "foo.config.name";
/**
* Sets the application name used to find property files (using
* {#link FooPropertiesEnvPostProcessor}).
*
* #param appName
* the application name
*/
public static void setApplicationName(String appName) {
System.setProperty(APP_KEY, appName);
}
/**
* Replacement for logging, which is not yet initialized during
* postProcessEnvironment.
*/
static void log(String format, Object... args) {
System.out.println(String.format(format, args));
}
static void debug(PropertyResolver env, String format, Object... args) {
String level = env.getProperty("logging.level." + FooPropertiesEnvPostProcessor.class.getName());
if ("trace".equalsIgnoreCase(level) || "debug".equalsIgnoreCase(level)) {
log(format, args);
}
}
static void trace(PropertyResolver env, String format, Object... args) {
String level = env.getProperty("logging.level." + FooPropertiesEnvPostProcessor.class.getName());
if ("trace".equalsIgnoreCase(level)) {
log(format, args);
}
}
#Override
public int getOrder() {
return ORDER;
}
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
addProperties(environment.getPropertySources(), application.getResourceLoader(), environment);
}
public static void addProperties(MutablePropertySources propSources, ResourceLoader resLoader, ConfigurableEnvironment propRes) {
trace(propRes, "FooPropertiesEnvPostProcessor.addProperties(..)");
List<PropertySourceLoader> psls = SpringFactoriesLoader.loadFactories(PropertySourceLoader.class,
PropertySourcesLoader.class.getClassLoader());
// ResourcePatternUtils does not accept null yet
// (https://jira.spring.io/browse/SPR-14500)
ResourcePatternResolver rpr = resLoader != null ? ResourcePatternUtils.getResourcePatternResolver(resLoader)
: new PathMatchingResourcePatternResolver();
final String suffix = ".properties"; // SonarQube made me declare this
String[] profiles = propRes.getActiveProfiles();
if (profiles.length == 0) {
profiles = new String[] { System.getProperty(DEFAULT_PROFILES_PROPERTY_NAME) };
}
// ConfigFileApplicationListener adds PropertySource "applicationConfigurationProperties" consisting of
// - "applicationConfig: [classpath:/${spring.config.name}-<profile>.properties]"
// - "applicationConfig: [classpath:/${spring.config.name}.properties]"
// Since we want the profile to have higher priority than the app name, we cannot just set
// "spring.config.name" to the app name, use ConfigFileApplicationListener, and add
// "application-<profile>.properties" and "application.properties".
// Instead, remove ConfigFileApplicationListener:
PropertySource<?> removedPropSource = propSources.remove(ConfigFileApplicationListener.APPLICATION_CONFIGURATION_PROPERTY_SOURCE_NAME);
trace(propRes, "removed %s from %s", removedPropSource, propSources);
// add meta.properties at last position, then others before the previously added. => resulting order:
// - { systemProperties
// - systemEnvironment
// - random } - already added automatically elsewhere
// - file:${foo.home}/foo-<profile>.properties
// - classpath:<appName>-<profile>.properties
// - classpath:application-<profile>.properties
// - classpath:<appName>.properties
// - classpath:application.properties
// - classpath:meta.properties
// By adding ${foo.home}/... (chronlogically) last, the property can be set in the previously added resources.
boolean defaultAppName = "application".equals(propRes.resolveRequiredPlaceholders("${" + APP_KEY + ":application}"));
String psn = null;
psn = addProperties(propSources, propRes, rpr, psls, true, psn, propRes.resolveRequiredPlaceholders("classpath*:meta" + suffix));
psn = addProperties(propSources, propRes, rpr, psls, true, psn, propRes.resolveRequiredPlaceholders("classpath*:application" + suffix));
if (!defaultAppName) {
psn = addProperties(propSources, propRes, rpr, psls, false,
psn, propRes.resolveRequiredPlaceholders("classpath*:${" + APP_KEY + ":application}" + suffix));
}
for (String profile : profiles) {
psn = addProperties(propSources, propRes, rpr, psls, false, psn,
propRes.resolveRequiredPlaceholders("classpath*:application-" + profile + suffix));
}
if (!defaultAppName) {
for (String profile : profiles) {
psn = addProperties(propSources, propRes, rpr, psls, false,
psn, propRes.resolveRequiredPlaceholders("classpath*:${" + APP_KEY + ":application}-" + profile + suffix));
}
}
for (String profile : profiles) {
psn = addProperties(propSources, propRes, rpr, psls, false,
psn, propRes.resolveRequiredPlaceholders("file:${foo.home:.}/foo-" + profile + suffix));
}
Stream<PropertySource<?>> propSourcesStream = StreamSupport.stream(Spliterators.spliteratorUnknownSize(propSources.iterator(), 0), false);
debug(propRes, "Property sources: %s%n", propSourcesStream.map(PropertySource::getName).collect(Collectors.joining(", ")));
}
/**
* Adds a resource given by location string to the given PropertySources, if
* it exists.
*
* #param propSources
* the property sources to modify
* #param successorName
* the name of the (already added) successor resource, i.e. the
* resource before which the new one should be added; if null,
* add as last resource
* #param location
* the location of the resource to add
* #return the name of the newly added resource, or {#code successorName} if
* not added
*/
private static String addProperties(MutablePropertySources propSources, PropertyResolver propRes, ResourcePatternResolver resLoader,
List<PropertySourceLoader> propLoaders, boolean required, String successorName, String location) {
Resource[] resources;
try {
resources = resLoader.getResources(location);
} catch (IOException e) {
throw new IllegalStateException("failed to load property source " + location + ": " + e, e);
}
if (resources.length == 0) {
debug(propRes, "%s property resource not found: %s", required ? "required" : "optional", location);
if (required) {
throw new IllegalStateException("required property source " + location + " not found");
} else {
return successorName;
}
}
String newSuccessorName = successorName;
for (Resource resource : resources) {
boolean exists = resource.exists();
debug(propRes, "%s property resource %sfound: %s%s", required ? "required" : "optional", exists ? "" : "not ", location,
uriDescription(resource, propRes));
if (!required && !exists) {
continue;
}
boolean loaded = false;
for (PropertySourceLoader propLoader : propLoaders) {
if (canLoadFileExtension(propLoader, resource)) {
newSuccessorName = addResource(propSources, propRes, resource, propLoader, newSuccessorName);
loaded = true;
break;
}
}
if (!loaded && required) {
throw new IllegalStateException("No PropertySourceLoader found to load " + resource);
}
}
return newSuccessorName;
}
private static String addResource(MutablePropertySources propSources, PropertyResolver propRes, Resource resource,
PropertySourceLoader propLoader, String successorName) {
try {
PropertySource<?> propSource = propLoader.load(resource.getDescription(), resource, null);
if (propSource == null) {
// e.g. a properties file with everything commented;
// org.springframework.boot.env.PropertiesPropertySourceLoader
// converts empty to null
return successorName;
}
if (successorName == null) {
propSources.addLast(propSource);
} else if (successorName.equals(propSource.getName())) {
// happens if APP_KEY is not set, so that
// "${APP_KEY:application}" == "application"
trace(propRes, "skipping duplicate resource %s", successorName);
} else {
propSources.addBefore(successorName, propSource);
}
return propSource.getName();
} catch (IOException e) {
throw new IllegalStateException("Unable to load configuration file " + resource + ": " + e, e);
}
}
/**
* Stolen from {#link PropertySourcesLoader}
*/
private static boolean canLoadFileExtension(PropertySourceLoader loader, Resource resource) {
String filename = resource.getFilename().toLowerCase();
for (String extension : loader.getFileExtensions()) {
if (filename.endsWith("." + extension.toLowerCase())) {
return true;
}
}
return false;
}
private static String uriDescription(Resource resource, PropertyResolver propRes) {
try {
return resource.exists() ? (" in " + resource.getURI()) : "";
} catch (IOException e) {
trace(propRes, "getURI: %s", e);
return "";
}
}
}
File META-INF/spring.factories
org.springframework.boot.env.EnvironmentPostProcessor = example.foo.utils.spring.FooPropertiesEnvPostProcessor
To get the same properties in tests, they have #ContextConfiguration(..., initializers = TestAppContextInitializer.class).
TestAppContextInitializer implements ApplicationContextInitializer<GenericApplicationContext> and calls
FooPropertiesEnvPostProcessor.addProperties in its initialize method.
Unfortunately, the EnvironmentPostProcessor seems to be missing Spring Shell by default, too. In our case (since only a tiny
part of the application uses Spring Shell), it was sufficient to restrict the <context:component-scan base-package=.../>
scope in META-INF/spring/spring-shell-plugin.xml to contain only stuff which does not need any properties set by the EnvironmentPostProcessor.
# override from outer config , eg. java -jar --spring.profiles.active=your config
spring.profiles.active=dev
spring.config.location=classpath:/${spring.profiles.active}/application.properties