I've faced an issue that on rare occasions (it might take dozens of restarts) Spring doesn't initialize all properties correctly.
I define the bean of CbKafkaConsumerConfig (my custom bean) type and check its state in the thread that was created by a method that is marked as #EventListener(ApplicationReadyEvent.class), so I expect it to be completely initialized by this point. However, this is what I see:
Values that I expected to be filled are left with placeholders.
Here's how they are defined in application.properties file. (And I've checked the spelling - it's correct, otherwise it would fail every time, not occasionally)
config-bean-prefix.msg-topics=${cb.kafka.tc-topic}
config-bean-prefix.unexpected-error-topic=${cb.kafka.unexpected-errors-topic}
These properties are defined in Vault and I expected them to be fetched and set with the power of Spring Cloud Vault. Here you can see that Vault is present as a property source AND that these properties are populated there.
At the same time, in the context there are other beans of the same type CbKafkaConsumerConfig that are referring to these properties and yet it resolved fine for them.
Here's how the bean is defined
#Bean({"myBean"})
#ConfigurationProperties(
prefix = "config-bean-prefix"
)
public CbKafkaConsumerConfig myBeanConsumer() {
return new CbKafkaConsumerConfig();
}
And the bean itself:
#Data
public class CbKafkaConsumerConfig extends CbKafkaBaseConfig {
#NotNull
#Size(
min = 1
)
private Collection<String> msgTopics;
#NotNull
private String unExpectedErrorTopic;
}
We're using Spring Boot 2.2.x however this issue is also present for Spring Boot 2.1.x.
It's not specific for this type of beans, other might fail as well while being correctly set in Vault. What could be the reason of such unpredictable behavior and what I should look into?
Turns out by default spring cloud vault is not simply fetching properties on start, every so often it's updating them. While updating, there's a short time window when properties were already deleted from property source in the context, but not filled with the new ones and it might actually happen during context initialization (super questionable behavior in my opinion) causing some beans being corrupted.
If you don't want properties to be updated in runtime just set spring.cloud.vault.config.lifecycle.enabled to false
Related
We are migrating from Gemfire 8.2.7 to 9.2.1
As part of Gemfire startup, we leverage SpringContextBootstrappingInitializer to initialize the spring-beans which #Autowire the Cache.
The same code when migrated to Gemfire 9.2.1 (along with the other stack) is failing on server startup with below error.
Gemfire 8.2.7 --> Gemfire 9.2.1
Spring-data-Gemfire 1.8.4 --> 2.0.2
Spring-Boot 1.4.7 --> 2.0.0.M7
Spring --> 5.0.2
Caused by:
org.springframework.beans.factory.NoSuchBeanDefinitionException: No
qualifying bean of type 'org.apache.geode.cache.Cache' available:
expected at least 1 bean which qualifies as autowire candidate.
Dependency annotations:
{#org.springframework.beans.factory.annotation.Autowired(required=true)}
Any pointers / changes required for GemfireConfig? Below is our JavaConfig.
#Bean
public CacheFactoryBean gemfireCache() {
return new CacheFactoryBean();
}
Looks like the ComponentScan is kicking in prior to Configuration processor. Any idea on controlling this behavior? This was lasted tested to work in Spring-Boot 1.4.6 (Spring- 4.3.8) and gets resolved with a #Depends option - but just wanted to understand if there are any fundamental changes with the ordering of bean initialization with newer Spring version.
#Configuration
#EnableAutoConfiguration(exclude = { HibernateJpaAutoConfiguration.class, BatchAutoConfiguration.class })
#Import(value = { GemfireServerConfig.class, JpaConfiguration.class, JpaConfigurableProperties.class })
#ComponentScan(basePackages = "com.test.gemfire", excludeFilters = #ComponentScan.Filter(type = FilterType.ANNOTATION, classes = Configuration.class) )
To begin, let me give you some tips since there are 3 issues with your problem statement above...
1) First, you have not made it clear why or how you are using the o.s.d.g.support.SpringContextBootstrappingInitializer Docs here.
I can only assume it is because you are launching your GemFire servers with Gfsh
using the following command...
gfsh> start server --name=MyServer --cache-xml-file=/path/to/cache.xml ...
Where your cache.xml is defined similar to this. After all, this was the original intent for using the SpringContextBootstrappingInitializer.
If this is the case, why not use the Gfsh, start server command, --spring-xml-location option instead. For example:
gfsh> start server --name=MyServer --spring-xml-location=/by/default/a/classpath/to/applicationContext.xml --classpath=/path/to/spring-data-gemfire-2.0.2.RELEASE.jar:...
By doing so, you no longer need to provide cache.xml just to declare the SpringContextBootstrappingInitializer in order to bootstrap a Spring container inside the GemFire JVM process. You can simply use the --spring-xml-location option and put SDG on the server's classpath when starting the server.
2) Second, it is not apparent what type of application component/bean you are injecting a GemFire Cache reference into (e.g. a Region or another application component class, like a DAO, etc). Providing a snippet of code showing how you injected the Cache reference, i.e. the injection point using the #Autowired annotation would have been helpful. For example:
#Service
class MyService {
#Autowired
private Cache gemfireCache;
...
}
3) #2 would have been more apparent if you included the full stack trace rather than just the NoSuchBeanDefinitionException message.
Despite the issues with your problem statement, I can infer the following:
Clearly, you are using "classpath component scanning" (with the #ComponentScan annotation) and are auto-wiring "by type"; which maybe key actually; I will come back to this later below.
You are using Spring's #Autowired annotation on a bean class field (field injection) or property (setter injection), maybe even a constructor.
The type of this field/property (or constructor parameter) is definitely org.apache.geode.cache.Cache.
Moving on...
In general, Spring will follow dependency order first and foremost. That is, if A depends on B, then B must be created before and destroyed after A. Typically, Spring will and can honor this without incident.
Beyond "dependency order" bean creation and satisfying dependencies between beans (including with the #DependsOn annotation), the order of bean creation is pretty loosely defined.
There are several factors that can influence it, such as "registration order" (i.e. the order in which bean definitions are declared, which is particularly true for beans defined in XML), "import order" (when using the #Import annotation on #Configuration classes), Java reflection (includes #Bean definitions declared in #Configuration classes), etc. Configuration organization is definitely important and should not be taken lightly.
This is 1 reason why I am not a big proponent of "classpath component scanning. While it may be convenient, it is always better, IMO, to be more "explicit" in your configuration, and the organization of your configuration, for reasons outlined here in addition to other non-apparent limitations. At worst, you should definitely be limiting the scope of the scan.
Ironically, you excluded/filtered the 1 thing that could actually help your organizational concerns... components of type #Configuration:
... excludeFilters = #ComponentScan.Filter(type = FilterType.ANNOTATION, classes = Configuration.class)
NOTE: given the exclusion, are you certain you did not exclude the the 1 #Configuration class containing your CacheFactoryBean definition? I suppose not since you say this worked after including the #DependsOn annotation.
Clearly there is a dependency defined between some application component of yours (??) and a bean of type o.a.g.cache.Cache (using #Autowired), yet Spring is failing to resolve it.
My thinking is, Spring cannot resolve the Cache dependency because 1) the GemFire cache bean either has not been created yet and 2) Spring cannot find an appropriate bean definition of the desired type (i.e. o.a.g.cache.Cache) in your configuration that would resolve the dependency and force the GemFire Cache to be created first, or 3) the GemFire Cache bean has been created first but Spring is unable to resolve the type as o.a.g.cache.Cache.
I have encountered both scenarios before and it is not exactly clear to me when each scenario happens because I simply have not traced this through yet. I have simply corrected it and moved on. I have noticed that it is version related though.
There are several ways to solve this problem.
If the problem is the later, 3), then simply declaring your dependency as type o.a.g.cache.GemFireCache should resolve the problem. So, for example:
#Repository
class MyDataAccessObject {
#Autowired
private GemFireCache gemfireCache;
...
}
The reason for this is because the o.s.d.g.CacheFactoryBean class's getObjectType() method returns a Class type generically extending o.a.g.cache.GemFireCache. This was by design since o.s.d.g.client.ClientCacheFactoryBean extends o.s.d.g.CacheFactoryBean, though I probably would not have done it that way if I had created these classes. However, it is consistent with the fact that the actual cache type in GemFire is o.a.g.internal.cache.GemFireCacheImpl which indirectly implements both the o.a.g.cache.Cache interface as well as the o.a.g.cache.client.ClientCache interface.
If your problem is the former (1) + 2), which is a bit trickier), then I would suggest you employ a smarter organization of your configuration, separated by concern. For example, you can encapsulate your GemFire configuration with:
#Configuration
class GemFireConfiguration {
// define GemFire components (e.g. CacheFactoryBean) here
}
Then, your application components, where some are dependent on GemFire components, can be defined with:
#Configuration
#Import(GemFireConfiguration.class)
class ApplicationConfiguration {
// define application beans, including beans dependent on GemFire components
}
By importing the GemFireConfiguration you are ensuring the GemFire components/beans are created (instantiated, configured and initialized) first.
You can even employ more targeted, limited "classpath component scanning" at the ApplicationConfiguration class-level in cases where you have a large number of application components (services, DAO, etc).
Then, you can have your main, Spring Boot application class drive all this:
#Configuration
#Import(ApplicationConfiguration.class)
class MySpringBootApplication {
public static void main(String[] args) {
SpringApplication.run(MySpringBootApplication.class, args);
}
}
The point is, you can be as granular as you choose. I like to encapsulate configuration by concern and clearly organize the configuration (using imports) to reflect the order in which I want my components created (constructed, configured and initialized).
Honestly, I basically organize my configuration in the order of dependencies. If my application ultimately depends on a data store and cannot function without that data store, then it makes since to ensure that is initialized first, otherwise, what is the point of starting the application.
Finally, you can always rely on the #DependsOn annotation, as you have appropriately done, to ensure that Spring will create the component before the component that expects it.
Based on the fact that the #DependsOn annotation solved your problem, then I would say this is an organizational problem and falls under the 1) / 2) category I outlined above.
I am going to dig into this a bit deeper and respond to my answer in comments with what I find.
Hope this helps!
-John
I have a need of conditionally creating one of three possible implementations of a service depending upon the environment detected by a Spring application at runtime. If Service A is available, then I want to create a concrete implementation class that uses Service A as a dependency. If Service A is not available, then I want to create an implementation using Service B as a dependency. And so-on.
Classes which depend on the implementation will Autowire the Interface and not care what the underlying Service was that got selected for the particular environment.
My first stab at this was to implement multiple #Bean methods which either return a bean or null, depending on whether the Service is available, and to then have a separate #Configuration class which #Autowire(required=false) the two possible services, conditionally creating the implementation depending on which of the #Autowired fields was not-null.
The problem here is that when required=false, Spring doesn't appear to care whether it waits around for candidates to be constructed; that is to say, the class which tries to pick the implementation might be constructed before one or both of the required=false Beans gets constructed, thus ensuring that one or both might always be null, regardless of whether it may manage to initialize correctly.
It kind of feels like I'm going against the grain at this point, so I'm looking for advice on the "right" way to do this sort of thing, where a whole set of beans might get switched out based on the availability of some outside service or environment.
Profiles don't look like the right answer, because I won't know until after my Service beans try to initialize which implementation I want to choose; I certainly won't know it at the time I create the context.
#Order doesn't achieve the goal either. Nor does #Conditional and testing on the existence of the bean (because it still might not be constructed yet). Same problem with FactoryBean- it does no good to check for the existence of beans that might not have been constructed at the time the FactoryBean is asked to create an instance.
What I really need to do is create a Bean based on the availability of other beans, but only AFTER those beans have at least had a chance to try to initialize.
Spring Profiles is your friend. You can set the current profile by way of environmental variable, command-line argument, and other methods. You can annotate a Spring-managed component so that it's created for a certain profile.
Spring Profiles from the Spring Documentation
Well in this case it turned out to be a tangential mistake that influenced the whole wrong behavior.
To give some background, my first, naive (but workable) approach looked like this:
#Autowired(required=false)
#Qualifier(RedisConfig.HISTORY)
private RLocalCachedMap<String, History> redisHistoryMap;
#Autowired(required=false)
#Qualifier(HazelcastConfig.HISTORY)
private IMap<String, History> hazelcastHistoryMap;
// RequestHistory is an interface
#Bean
public RequestHistory requestHistory() {
if (redisHistoryMap != null) {
return new RedisClusteredHistory(redisHistoryMap);
} else if (hazelcastHistoryMap != null) {
return new HazelcastClusteredHistory(hazelcastHistoryMap);
} else {
return new LocalRequestHistory(); // dumb hashmap
}
}
In other #Configuration classes, if the beans that get #Autowired here aren't available (due to missing configuration, exceptions during initialization, etc), the #Bean methods that create them return null.
The observed behavior was that this #Bean method was getting called after the RLocalCachedMap<> #Bean method got called, but before Spring attempted to create the IMap<> by calling its #Bean method. I had incorrectly thought that this had something to do with required=false but in fact that had nothing to do with it.
What actually happened was I accidentally used the same constant for both #Bean names (and consequently #Qualifiers), so presumably Spring couldn't tell the difference when it was calculating its dependency graph for this #Configuration class... because the two #Autowired beans appeared to be the same thing (because they had the same name).
(There's a secondary reason for using #Qualifier in this case, which I won't go into here, but suffice it to say it's possible to have many maps of the same type.)
Once I qualified the names better, the code did exactly what I wanted it to, albeit in a way that's somewhat inelegant/ugly.
At some point I'll go back and see if it looks more elegant / less ugly and works just as well to use #Conditional and #Primary instead of the if/else foulness.
The lesson here is that if you explicitly name beans, make absolutely sure your names are unique across your application, even if you plan to swap things around like this.
UPDATE: I just published this question also here, I might have done a better work phrasing it there.
How can I explicitly define an order in which Spring's out-of-the-box process of reading properties out of an available-in-classpath application.yml will take place BEFORE my #Configuration annotated class which reads configuration data from zookeeper and places them as system properties which are later easily read and injected into members using #Value?
I have a #Configuration class, which defines a creation of a #Bean, in a which configuration data from zookeeper is read and placed as system properties, in a way that they can easily be read and injected into members using #Value.
#Profile("prod")
#Configuration
public class ZookeeperConfigurationReader {
#Value("${zookeeper.url}")
static String zkUrl;
#Bean
public static PropertySourcesPlaceholderConfigurer zkPropertySourcesPlaceHolderConfigurer() {
PropertySourcesConfigurerAdapter propertiesAdapter = new PropertySourcesConfigurerAdapter();
new ConfigurationBuilder().populateAdapterWithDataFromZk(propertiesAdapter);
return propertiesAdapter.getConfigurer();
}
public void populateAdapterWithDataFromZk(ConfigurerAdapter ca) {
...
}
}
Right now I pass the zookeeper.url into the executed program using a -Dzookeeper.url which is added to the execution line. Right now I read it by calling directly System.getProperty("zookeeper.url").
Since I'm using Spring-Boot application, I also have a application.yml configuration file.
I would like to be able to set the zookeeper.url in the application.yml, and keep my execution line clean as possible from explicit properties.
The mission turns out to be harder than I thought.
As you can see in the above code sniplet of ZookeeperConfigurationReader, I'm trying to inject that value using #Value("${zookeeper.url}") into a member in the class which performs the actual read of data from zookeeper, but at the time the code that needs that value accesses it, it is still null. The reason for that is that in spring life cycle wise, I'm still in the phase of "configuration" as I'm a #Configuration annotated class myself, and the spring's code which reads the application.yml data and places them as system properties, hasn't been executed yet.
So bottom line, what I'm looking for is a way to control the order and tell spring to first read application.yml into system properties, and then load ZookeeperConfigurationReader class.
You can try to use Spring Cloud Zookeeper. I posted a brief example of use here
We have a specific format for our config files, where instead of having multiple files - ie, dev.properties, uat.properties, prod.properties - we instead have all the values in one file, but separated by prefixes for each environment. For example:
SERVICE_PORT=9800
DEV_SERVICE_PORT=7800
UAT_SERVICE_PORT=6600
We have an existing class (subclass of PropertyPlaceholderConfigurer) that looks up these values, and decides what prefix to add inside resolvePlaceHolder() based on the IP address where it is executing, ie for a certain IP range, use DEV_ prefix, for another, use UAT_ prefix. These values then get passed on to other beans, in some cases using the context xml, and in some using the #Value${} annotation on some bean constructors. The use of the prefixes is transparent, so all other configs will use SERVICE_PORT (in the example)
We want to change this so that instead of using IPs, we will just detect the active Spring Profile. We have a custom ApplicationContextIniitalizer in our web.xml that detects a java System property that indicates our type of environment.
The problem I have is that at the time resolvePlaceHolder() gets called, there doesn't seem to be any Active Profiles yet! What I"m doing to detect the active profile is:
create an instance of StandardEnvironment
call getActiveProfiles()
(2) always seems to return an empty array. This implies that propertyplaceholder resolution occurs before any Spring profiles are activated. Is this correct??
When does the active profile get set, in relation to other events during Spring context loading, like creating the beans, loading property files, etc?
Is it possible to detect the active profile at the time that resolvePlaceHolder() is called? Should I be extending another class instead?
Beans managed within an ApplicationContext may register to be
EnvironmentAware or #Inject the Environment in order to query profile
state or resolve properties directly.
[Source: Environment javadocs ]
Don't create an instance of StandardEnvironment, inject it into your bean!
If I am using spring frame work in my application does creating an object like this Test test = new Test() a bad way for creating an instance? Should I always use the bean config to get the objects/bean that I need? If yes, does that means I should have all the object/bean definition in spring applicationContext xml file?
If you want your object to be managed by Spring (this means that dependencies are injected, among other things) you have to use the ApplicationContext.
Calling Test test = new Test() isn't illegal, or even bad practice. It just means that Spring will have no awareness of this object, and it won't bother autowiring it's dependencies, or doing anything else that you'd expect Spring to do.
You don't necessarily need to use the applicationContext.xml file for ALL of your bean declarations. Many people favor annotations, which allow you to declare beans outside of the applicationContext.xml file.
It's worth nothing that Spring-managed beans are by default singletons (think of Servlets). If you want stateful beans that are Spring aware, you could use an ObjectFactoryCreatingFactoryBean to do something like this:
#Autowired
private ObjectFactory myWidgetFactory;
public void doStuff() {
Widget w = myWidgetFactory.getObject();
}
You can read more about this behaviour here:
http://static.springsource.org/spring/docs/3.0.x/api/org/springframework/beans/factory/config/ObjectFactoryCreatingFactoryBean.html
For me there's a big difference between objects that represent components of my application -- services, controllers, DAOs, utilities, etc. -- and objects that represent entities within my application -- Person, Order, Invoice, Account, etc. The former type of objects should absolutely be managed by Spring and injected. The latter type are typically created on the fly by the application, and that frequently will involve calling new. This is not a problem.
Test test = new Test() a bad way for
creating an instance?
Yes it is bad practice.
Should I always use the bean config
to get the objects/bean that I need?
Yes, if you are using Spring for dependency injection.
If yes, does that means I should have
all the object/bean definition in
spring applicationContext xml file?
Always! You could use Annotations too.