Trying to connect with aws-s3 using spring boot application. when i am using
import org.springframework.cloud.aws.context.support.io.ResourceLoaderBeanPostProcessor
getting aws key can not be null.
#Configuration
#EnableContextResourceLoader
#EnableContextCredentials
public class S3Configuration {
#Value("${cloud.aws.credentials.accessKey}")
private String ACCESS_KEY;
#Value("${cloud.aws.credentials.secretKey}")
private String SECRET_KEY;
#Value("${cloud.aws.region}")
private String region;
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean
public AmazonS3Client amazonS3Client() {
return new AmazonS3Client(new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY));
}
#Bean
public ResourceLoaderBeanPostProcessor resourceLoaderBeanPostProcessor() {
return new ResourceLoaderBeanPostProcessor(amazonS3Client());
}
}
If I dont use "ResourceLoaderBeanPostProcessor" class then AmazonS3Client object is creating successfully by reading properties form application.properties.
can some one help me what i am doing wrong?
I'm pretty sure that the BeanFactoryPostProcessor class that the ResourceLoaderBeanPostProcessor implements, is executed before values from application.properties are loaded/injected by the spring application. Therefore your values (when creating the bean) is null. (And that's why it works when you're not using that paticular class).
I would suggest using the default credentials chain, and have your credentials lying in a ~/.aws/ folder (on your local machine, and on your servers). see http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html for more detail.
Related
I am trying to implement Hibernate second level caching in a Spring boot project using Redisson.
I have followed this blog as a reference
https://pavankjadda.medium.com/implement-hibernate-2nd-level-cache-with-redis-spring-boot-and-spring-data-jpa-7cdbf5632883
Also i am trying to initialize the RedissionClient programmatically and not through declaratively /through a config file
Created a spring bean to be initialized which should create the RedissonClient instance.
#Configuration
#Lazy(value = false)
public class RedissonConfig {
#Bean
public RedissonClient redissionClient() {
Config config = new Config();
config.useSingleServer().setAddress("redis://127.0.0.1:6379");
return Redisson.create(config);
}
}
However this bean is never intialized and i get the following error while application startup.
Caused by: org.hibernate.cache.CacheException: Unable to locate Redisson configuration
at org.redisson.hibernate.RedissonRegionFactory.createRedissonClient(RedissonRegionFactory.java:107) ~[redisson-hibernate-53-3.12.1.jar:3.12.1]
at org.redisson.hibernate.RedissonRegionFactory.prepareForUse(RedissonRegionFactory.java:83) ~[redisson-hibernate-53-3.12.1.jar:3.12.1]
It seems Spring boot Hibernate still trying to load the Redisson config through a config file.
is it possible to load the Redission config in spring boot programmatically ?
Best Regards,
Saurav
I just did exactly this, here is how:
you need a custom RegionFactory that is similar to the JndiRedissonRegionFactory but gets its RedissonClient injected somehow.
an instance of this Class, fully configured, is put into the hibernate-properties map. Hibernates internal code is flexible: if the value of hibernate.cache.region.factory_class is a string it is treated as a FQDN. If it is an instance of Class<?>, it will be instantiated. If it is an Object, it will be used.
Spring offers a rather simple way to customize hibernate properties with a bean:
#AutoConfiguration(after = RedissonAutoConfiguration.class, before = JpaAutoConfiguration.class)
#ConditionalOnProperty("spring.jpa.properties.hibernate.cache.use_second_level_cache")
public class HibernateCacheAutoConfiguration {
#Bean
public HibernatePropertiesCustomizer setRegionFactory(RedissonClient redisson) {
return hibernateProperties -> hibernateProperties.put(AvailableSettings.CACHE_REGION_FACTORY, new SpringBootRedissonRegionFactory(redisson));
}
}
My RegionFactory is really simple:
#AllArgsConstructor
public class SpringBootRedissonRegionFactory extends RedissonRegionFactory {
private RedissonClient redissonClient;
#Override
protected RedissonClient createRedissonClient(Map properties) {
return redissonClient;
}
#Override
protected void releaseFromUse() {
}
}
I used the redisson-starter to get a RedissonClient, hence the reference to RedissonAutoConfiguration, but you could just create an instance by hand.
It is possible, but then you need to provide a custom implementation of RegionFactory to Hibernate, which can extends RedissonRegionFactory but uses your own client instance.
I 'm trying to access the properties file other than a place where all the application's configuration resides(Datasource,SessionFactory,TransactionManager), hence I'm trying to use #Environment twice in my application.
I'm trying to load a properties file to implement a email notification functionality to access the properties file content using env by auto wiring. But I'm failing while reading the property file using env.getProperty("email.smtp.host")); Getting null in the place of property value when executing the app.
Actually, I'm trying to use two Configuration file, is this what blocks me to autowire Environment property ? I tried to replace with #Component and adding email package to componentscan list as well. I'm thinking that, is Environement property applicable only to #Configuration file ? Can someone help me to debug the issue please or what I'm missing here to initiate the Environment ?
I tried to resolve this by implementing EnvironmentAware interface as below to get the property value using load() but that still didn't help me.
Tried to use PropertySourcesPlaceholderConfigurer as below.
#Bean
public static PropertySourcesPlaceholderConfigurer propertyPlaceHolderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
#Autowired
private Environment env;
#Configuration
#PropertySource("classpath:email.properties")
public class PropertiesUtil implements EnvironmentAware {
#Autowired
private Environment environment;
public void sendPlainTextEmail(String mailMessage) throws AddressException, MessagingException{
try {
// Configure SMTP
Properties properties = new Properties();
properties.setProperty("mail.smtp.host",this.env.getProperty("email.smtp.host"));
properties.setProperty("mail.smtp.port", "25");
}
#Override
public void setEnvironment(final Environment environment) {
this.environment = environment;
}
#Bean
public String load(String propertyName)
{
return environment.getRequiredProperty(propertyName);
}
}
Expecting to read the email property but getting only Null and no other information to debug.
My goal is to have a have integration tests that ensures that there isn't too many database queries happening during lookups. (This helps us catch n+1 queries due to incorrect JPA configuration)
I know that the database connection is correct because there is no configuration problems during the test run whenever MyDataSourceWrapperConfiguration is not included in the test. However, once it is added, the circular dependency happens. (see error below) I believe #Primary is necessary in order for the JPA/JDBC code to use the correct DataSource instance.
MyDataSourceWrapper is a custom class that tracks the number of queries that have happened for a given transaction, but it delegates the real database work to the DataSource passed in via constructor.
Error:
The dependencies of some of the beans in the application context form a cycle:
org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration
┌─────┐
| databaseQueryCounterProxyDataSource defined in me.testsupport.database.MyDataSourceWrapperConfiguration
↑ ↓
| dataSource defined in org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Tomcat
↑ ↓
| dataSourceInitializer
└─────┘
My Configuration:
#Configuration
public class MyDataSourceWrapperConfiguration {
#Primary
#Bean
DataSource databaseQueryCounterProxyDataSource(final DataSource delegate) {
return MyDataSourceWrapper(delegate);
}
}
My Test:
#ActiveProfiles({ "it" })
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration({ DatabaseConnectionConfiguration.class, DatabaseQueryCounterConfiguration.class })
#EnableAutoConfiguration
public class EngApplicationRepositoryIT {
#Rule
public MyDatabaseQueryCounter databaseQueryCounter = new MyDatabaseQueryCounter ();
#Rule
public ErrorCollector errorCollector = new ErrorCollector();
#Autowired
MyRepository repository;
#Test
public void test() {
this.repository.loadData();
this.errorCollector.checkThat(this.databaseQueryCounter.getSelectCounts(), is(lessThan(10)));
}
}
UPDATE: This original question was for springboot 1.5. The accepted answer reflects that, however, the answer from #rajadilipkolli works for springboot 2.x
In your case you will get 2 DataSource instances which is probably not what you want. Instead use BeanPostProcessor which is the component actually designed for this. See also the Spring Reference Guide.
Create and register a BeanPostProcessor which does the wrapping.
public class DataSourceWrapper implements BeanPostProcessor {
public Object postProcessBeforeInitialization(Object bean, String beanName) {
if (bean instanceof DataSource) {
return new MyDataSourceWrapper((DataSource)bean);
}
return bean;
}
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
}
Then just register that as a #Bean instead of your MyDataSourceWrapper.
Tip: Instead of rolling your own wrapping DataSource you might be interested in datasource-proxy combined with datasource-assert which has counter etc. support already (saves you maintaining your own components).
Starting from spring boot 2.0.0.M3 using BeanPostProcessor wont work.
As a work around create your own bean like below
#Bean
public DataSource customDataSource(DataSourceProperties properties) {
log.info("Inside Proxy Creation");
final HikariDataSource dataSource = (HikariDataSource) properties
.initializeDataSourceBuilder().type(HikariDataSource.class).build();
if (properties.getName() != null) {
dataSource.setPoolName(properties.getName());
}
return ProxyDataSourceBuilder.create(dataSource).countQuery().name("MyDS")
.logSlowQueryToSysOut(1, TimeUnit.MINUTES).build();
}
Another way is to use datasource-proxy version of datasource-decorator starter
Following solution works for me using Spring Boot 2.0.6.
It uses explicit binding instead of annotation #ConfigurationProperties(prefix = "spring.datasource.hikari").
#Configuration
public class DataSourceConfig {
private final Environment env;
#Autowired
public DataSourceConfig(Environment env) {
this.env = env;
}
#Primary
#Bean
public MyDataSourceWrapper primaryDataSource(DataSourceProperties properties) {
DataSource dataSource = properties.initializeDataSourceBuilder().build();
Binder binder = Binder.get(env);
binder.bind("spring.datasource.hikari", Bindable.ofInstance(dataSource).withExistingValue(dataSource));
return new MyDataSourceWrapper(dataSource);
}
}
You can actually still use BeanPostProcessor in Spring Boot 2, but it needs to return the correct type (the actual type of the declared Bean). To do this you need to create a proxy of the correct type which redirects DataSource methods to your interceptor and all the other methods to the original bean.
For example code see the Spring Boot issue and discussion at https://github.com/spring-projects/spring-boot/issues/12592.
I'm trying to disable a Zuul route to a microservice registered with Eureka at runtime (I'm using spring boot).
This is an example:
localhost/hello
localhost/world
Those two are the registered microservices. I would like to disable the route to one of them at runtime without shutting it down.
Is there a way to do this?
Thank you,
Nano
Alternatively to using Cloud Config, custom ZuulFilter can be used. Something like (partial implementation to show the concept):
public class BlackListFilter extends ZuulFilter {
#Override
public String filterType() {
return "pre";
}
...
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
String uri = ctx.getRequest().getRequestURI();
String appId = uri.split("/")[1];
if (blackList.contains(appId)) {
ctx.setSendZuulResponse(false);
LOG.info("Request '{}' from {}:{} is blocked",
uri, ctx.getRequest().getRemoteHost(), ctx.getRequest().getRemotePort());
}
return null;
}
}
where blackList contains list of application IDs (Spring Boot application name) managed for example via some RESTful API.
After a lot of efforts I came up with this solution. First, I used Netflix Archaius to watch a property file. Then I proceeded as follows:
public class ApplicationRouteLocator extends SimpleRouteLocator implements RefreshableRouteLocator {
public ApplicationRouteLocator(String servletPath, ZuulProperties properties) {
super(servletPath, properties );
}
#Override
public void refresh() {
doRefresh();
}
}
Made the doRefresh() method public by extending SimpleRouteLocator and calling its method in the overridden one of the interface RefreshableRouteLocator.
Then I redefined the bean RouteLocator with my custom implementation:
#Configuration
#EnableConfigurationProperties( { ZuulProperties.class } )
public class ZuulConfig {
public static ApplicationRouteLocator simpleRouteLocator;
#Autowired
private ZuulProperties zuulProperties;
#Autowired
private ServerProperties server;
#Bean
#Primary
public RouteLocator routeLocator() {
logger.info( "zuulProperties are: {}", zuulProperties );
simpleRouteLocator = new ApplicationRouteLocator( this.server.getServletPrefix(),
this.zuulProperties );
ConfigurationManager.getConfigInstance().addConfigurationListener( configurationListener );
return simpleRouteLocator;
}
private ConfigurationListener configurationListener =
new ConfigurationListener() {
#Override
public void configurationChanged( ConfigurationEvent ce ) {
// zuulProperties.getRoutes() do something
// zuulProperties.getIgnoredPatterns() do something
simpleRouteLocator.refresh();
}
}
}
Every time a property in the file was modified an event was triggered and the ConfigurationEvent was able to deal with it (getPropertyName() and getPropertyValue() to extract data from the event). Since I also Autowired the ZuulProperties I was able to get access to it. With the right rule I could find whether the property of Zuul
zuul.ignoredPatterns
was modified changing its value in the ZuulProperties accordingly.
Here refresh context should work (as long as you are not adding a new routing rule or removing a currently existing one), if you are adding or removing routing rules, you have to add a new bean for ZuulProperties and mark it with #RefreshScope, #Primary.
You can autowire refreshEndpoint bean for example and apply refreshEndpoint.refresh() on the listener.
Marking a custom RouteLocator as primary will cause problems as zuul already has bean of same type marked as primary.
I'm trying to build a Spring 3.1 PropertySource which reads its values from Zookeeper nodes. For connecting to Zookeeper I am using Curator from Netflix.
For that I've built a custom property source which reads the value of a property from Zookeeper and returns it. This works fine when I am resolving the property like this
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addLast(zkPropertySource);
ctx.getEnvironment().getProperty("foo"); // returns 'from zookeeper'
However, when I try to instantiate a bean which has a field with an #Value annotation then this fails:
#Component
public class MyBean {
#Value("${foo}") public String foo;
}
MyBean b = ctx.getBean(MyBean.class); // fails with BeanCreationException
This problem has most likely nothing to do with Zookeeper but with the way I'm registering the property sources and creating the beans.
Any insight is highly appreciated.
Update 1:
I'm creating the app context from an XML file like this:
public class Main {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext.xml");
ctx.registerShutdownHook();
}
}
The class which connects to Zookeeper is a #Component.
#Component
public class Server {
CuratorFramework zkClient;
public void connectToZookeeper() {
zkClient = ... (curator magic) ...
}
public void registerPropertySource() {
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addLast(zkPropertySource);
ctx.getEnvironment().getProperty("foo"); // returns 'from zookeeper'
}
#PostConstruct
public void start() {
connectToZookeeper();
registerPropertySource();
MyBean b = ctx.getBean(MyBean.class);
}
}
Update 2
This seems to work when I'm using XML-less configuration, i.e. #Configuration, #ComponentScan and #PropertySource in combination with an AnnotationConfigApplicationContext. Why doesn't it work with a ClassPathXmlApplicationContext?
#Configuration
#ComponentScan("com.goleft")
#PropertySource({"classpath:config.properties","classpath:version.properties"})
public class AppConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Answering to your Update 2: This does not work with your original configuration(registering a PropertySource using #PostConstruct) because the PropertySource is being registered very late, by this time your target bean has already been constructed and initialized.
Typically the injection of the placeholders happens via a BeanFactoryPostProcessor which is very early in the Spring lifecycle(beans have not been created at this stage) and if a PropertySource is registered at that stage, then placeholders should be resolved.
The best approach though is to use a ApplicationContextInitializer, get a handle on the applicationContext and to register the propertySource there:
public class CustomInitializer implements ApplicationContextInitializer<ConfigurableWebApplicationContext> {
public void initialize(ConfigurableWebApplicationContext ctx) {
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addFirst(zkPropertySource);
}
}