I have custom.yaml file with content:
refill-interval-millis: 1000
endpoints:
- path: /account/all
rate-limit: 10
- path: /account/create
rate-limit: 20
and my class to read that file:
#Component
#PropertySource("classpath:custom.yaml")
#ConfigurationProperties
public class Properties {
private int refillIntervalMillis;
private List<Endpoint> endpoints = new ArrayList<>();
// getters/setters
public static class Endpoint {
private String path;
private int rateLimit;
// getters/setters
}
}
So, when I run my code, refillIntervalMillis is set properly, but endpoints list is empty. Can not get why?
You cannot use #PropertySource directly for YAML-files: YAML Shortcomings
You have 2 simple options:
use application.yml and Spring Boot loads if by default (and you can use application-xxx.yml and set #ActiveProfiles(value = "xxx");
load YAML-file manually:
#Bean
public static PropertySourcesPlaceholderConfigurer properties() {
var propertySourcesPlaceholderConfigurer = new PropertySourcesPlaceholderConfigurer();
var yamlPropertiesFactoryBean = new YamlPropertiesFactoryBean();
yamlPropertiesFactoryBean.setResources(new ClassPathResource("custom.yaml"));
propertySourcesPlaceholderConfigurer.setProperties(yamlPropertiesFactoryBean.getObject());
return propertySourcesPlaceholderConfigurer;
}
Related
I worked with spring boot and redis to caching.I can cache my data that fetch from database(oracle) use #Cacheable(key = "{#input,#page,#size}",value = "on_test").
when i try to fetch data from key("on_test::0,0,10") with redisTemplate the result is 0
why??
Redis Config:
#Configuration
public class RedisConfig {
#Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration("localhost", 6379);
redisStandaloneConfiguration.setPassword(RedisPassword.of("admin#123"));
return new JedisConnectionFactory(redisStandaloneConfiguration);
}
#Bean
public RedisTemplate<String,Objects> redisTemplate() {
RedisTemplate<String,Objects> template = new RedisTemplate<>();
template.setStringSerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
//service
#Override
#Cacheable(key = "{#input,#page,#size}",value = "on_test")
public Page<?> getAllByZikaConfirmedClinicIs(Integer input,int page,int size) {
try {
Pageable newPage = PageRequest.of(page, size);
String fromCache = controlledCacheService.getFromCache();
if (fromCache == null && input!=null) {
log.info("cache is empty lets initials it!!!");
Page<DataSet> all = dataSetRepository.getAllByZikaConfirmedClinicIs(input,newPage);
List<DataSet> d = redisTemplate.opsForHash().values("on_test::0,0,10");
System.out.print(d);
return all;
}
return null;
The whole point of using #Cacheable is that you don't need to be using RedisTemplate directly. You just need to call getAllByZikaConfirmedClinicIs() (from outside of the class it is defined in) and Spring will automatically check first if a cached result is available and return that instead of calling the function.
If that's not working, have you annotated one of your Spring Boot configuration classes with #EnableCaching to enable caching?
You might also need to set spring.cache.type=REDIS in application.properties, or spring.cache.type: REDIS in application.yml to ensure Spring is using Redis and not some other cache provider.
Sample spring configuration is as below.
<int-mail:imap-idle-channel-adapter id="mailAdapter"
store-uri="imaps://${"username"}:${"password"}#imap-server:993/INBOX"
java-mail-properties="javaMailProperties"
channel="emails"
should-delete-messages="false"
should-mark-messages-as-read="true">
</int-mail:imap-idle-channel-adapter>
I wish to keep the password field encrypted in properties file and decrypt it in the code. I am not sure on how to set mailReceiver property of ImapIdleChannelAdapter to my custom version of ImapMailReceiver.
Please let me know if there is any way to do this.
All of my configurations are in XML as described above.
Above solution of adding the defifnation did not work may be I am doing something wrong. Then I tried using XML + Java configuration, as below.
#Configuration
public class EmailConfiguration {
#Bean
public ImapIdleChannelAdapter customAdapter() {
ImapIdleChannelAdapter adapter = new ImapIdleChannelAdapter(mailReceiver());
adapter.setOutputChannel(outputChannel());
adapter.setErrorChannel(errorChannel());
adapter.setAutoStartup(true);
adapter.setShouldReconnectAutomatically(true);
adapter.setTaskScheduler(taskScheduler());
return adapter;
}
#Bean
public TaskImapMailReceiver mailReceiver() {
TaskImapMailReceiver mailReceiver = new TaskImapMailReceiver("imaps://[username]:[password]#imap.googlemail.com:993/inbox");
mailReceiver.setShouldDeleteMessages(false);
mailReceiver.setShouldMarkMessagesAsRead(true);
//mailReceiver.setJavaMailProperties(javaMailProperties());
mailReceiver.setMaxFetchSize(Integer.MAX_VALUE);
return mailReceiver;
}
}
Also created empty errorChannel,outputChannel etc. I observed that Spring creates two instances one with xml config and other with java #Configuration. Where it was expected to use only java configuration. If I remove the xml config tag
then it provides sigle imap instance with my mailReceiver but runs only once does not go periodic. also does not show IMAPS logs.
Just wondering if I need to do so much to encrypt the password. Is somthing wrong with my approach.
Use Java configuration instead of XML...
#Configuration
public class MyConfigClass {
#Bean
public MyMailReceiver receiver() {
...
}
#Bean
public ImapIdleChannelAdapter adapter() {
ImapIdleChannelAdapter adapter = new ImapIdleChannelAdapter(receiver());
...
return adapter;
}
}
If you are using XML for everything else, simply add this class as a <bean/> to your XML.
EDIT
Here's an example that works fine for me...
#SpringBootApplication
public class So42298254Application {
public static void main(String[] args) {
SpringApplication.run(So42298254Application.class, args);
}
#Bean
public TestMailServer.ImapServer imapServer() {
return TestMailServer.imap(0);
}
#Bean
public ImapMailReceiver receiver() {
ImapMailReceiver imapMailReceiver = new ImapMailReceiver(imapUrl("user", "pw"));
imapMailReceiver.setHeaderMapper(new DefaultMailHeaderMapper()); // converts the MimeMessage to a String
imapMailReceiver.setUserFlag("testSIUserFlag"); // needed by the SI test server
Properties javaMailProperties = new Properties();
javaMailProperties.put("mail.debug", "true");
imapMailReceiver.setJavaMailProperties(javaMailProperties);
return imapMailReceiver;
}
private String imapUrl(String user, String pw) {
return "imap://"
+ user + ":" + pw
+ "#localhost:" + imapServer().getPort() + "/INBOX";
}
#Bean
public ImapIdleChannelAdapter adapter() {
ImapIdleChannelAdapter adapter = new ImapIdleChannelAdapter(receiver());
adapter.setOutputChannelName("handleMail");
return adapter;
}
#ServiceActivator(inputChannel = "handleMail")
public void handle(String mail, #Header(MailHeaders.FROM) Object from) {
System.out.println(mail + " from:" + from);
imapServer().resetServer(); // so we'll get the email again
}
}
My intention was to use encrypted passwords in properties files.
So I changed my approach of getting into email receiving classes. I added inherited PropertyPlaceholderConfigurer and implemented method convertPropertyValue() as below.
public class EncryptationAwarePropertyPlaceholderConfigurer extends PropertyPlaceholderConfigurer {
private static final Logger logger = LoggerFactory.getLogger(EncryptationAwarePropertyPlaceholderConfigurer.class);
#Override
protected String convertPropertyValue(String originalValue) {
if (originalValue.contains("{<ENC>}") && originalValue.contains("{</ENC>}")) {
String encryptedTaggedValue = originalValue.substring(originalValue.indexOf("{<ENC>}"), originalValue.indexOf("{</ENC>}") + 8);
String encryptedValue = originalValue.substring(originalValue.indexOf("{<ENC>}") + 7, originalValue.indexOf("{</ENC>}"));
try {
String decryptedValue = EncrypDecriptUtil.decrypt(encryptedValue);//EncrypDecriptUtil is my class for encription and decryption
originalValue = originalValue.replace(encryptedTaggedValue, decryptedValue);
} catch (GeneralSecurityException e) {
logger.error("failed to decrypt property returning original value as in properties file.", e);
}
}
return originalValue;
}
}
And changed properties file to enclose encrypted value in custuom ENC tag
as
mail.imap.task.url=imap://username:{<ENC>}encryptedPassword{</ENC>}#imap.googlemail.com:993/inbox
I found https://github.com/FasterXML/jackson-dataformat-yaml to deserialize/serialize YAML files. However, I'm having a hard time to deserialize/serialize the following:
I want to define a prefix to the actual document to be parsed as a POJO. Similar to a subtree of the document.
I want to define the POJO that represents the simple object representation instead of creating multiple objects.
The Error "Unrecognized field "spring" (class ConfigServerProperties), not marked as ignorable (one known property: "repos"])" is shown. But I don't know how to represent the prefix "spring.cloud.config.server.git" to be the root element of the POJO.
Document
spring:
cloud:
config:
server:
git:
repos:
publisher:
uri: 'https://github.company.com/toos/spring-cloud-config-publisher-config'
cloneOnStart: true
username: myuser
password: password
pullOnRequest: false
differentProperty: My Value
config_test_server_config:
uri: 'https://github.company.com/mdesales/config-test-server-config'
cloneOnStart: true
username: 226b4bb85aa131cd6393acee9c484ec426111d16
password: ""
completelyDifferentProp: this is a different one
For this document, the requirements are as follows:
* I want to define the prefix as "spring.cloud.config.server.git".
* I want to create a POJO that represents the object.
POJO
I created the following POJOs to represent this.
ConfigServerProperties: represents the top pojo containing the list of repos.
ConfigServerOnboard: represents each of the elements of the document.
Each properties are stored in a map, so that we can add as many different properties as possible.
Each class is as follows:
public class ConfigServerProperties {
private Map<String, ConfigServerOnboard> repos;
public void setRepos(Map<String, ConfigServerOnboard> repos) {
this.repos = repos;
}
public Map<String, ConfigServerOnboard> getRepos() {
return this.repos;
}
}
The second class is as follows:
public class ConfigServerOnboard {
private Map<String, String> properties;
public Map<String, String> getProperties() {
return properties;
}
public void setProperties(Map<String, String> properties) {
this.properties = properties;
}
}
Deserialize
The deserialization strategy I tried is as follows:
public static ConfigServerProperties parseProperties(File filePath)
throws JsonParseException, JsonMappingException, IOException {
ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
JsonNodeFactory jsonNodeFactory = new JsonNodeFactory(false);
jsonNodeFactory.textNode("spring.cloud.config");
// tried to use this attempting to get the prefix
mapper.setNodeFactory(jsonNodeFactory);
ConfigServerProperties user = mapper.readValue(filePath, ConfigServerProperties.class);
return user;
}
Error Returned
Exception in thread "main" com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "spring" (class com.company.platform.config.onboarding.files.config.model.ConfigServerProperties), not marked as ignorable (one known property: "repos"])
at [Source: /tmp/config-server-onboards.yml; line: 3, column: 3] (through reference chain: com.company.platform.config.onboarding.files.config.model.ConfigServerProperties["spring"])
at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62)
at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:834)
at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1094)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1470)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1448)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:282)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:140)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3798)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2740)
at com.company.platform.config.onboarding.files.config.model.ConfigServerProperties.parseProperties(ConfigServerProperties.java:37)
at com.company.platform.config.onboarding.files.config.model.ConfigServerProperties.main(ConfigServerProperties.java:42)
Edit 1: Looking for a possible SpringBoot Solution
I'm open to solutions using SpringBoot's ConfigurationProperties("spring.cloud.config.server.git"). That way, we could have the following:
#ConfigurationProperties("spring.cloud.config.server.git")
public class Configuration {
private Map<String, Map<String, String>> repos = new LinkedHashMap<String, new HashMap<String, String>>();
// getter/setter
}
Questions
How to set the root element of the document?
Deserialization must read the document and produce instances of the POJOs.
Serialization must produce the same document with updated values.
I had to come up with the following:
Create 6 classes, each of them with the property required for the prefix "spring.cloud.config.server.git"
SpringCloudConfigSpring.java
SpringCloudConfigCloud.java
SpringCloudConfigConfig.java
SpringCloudConfigServer.java
SpringCloudConfigGit.java
The holder of all of them is SpringCloudConfigFile.java.
The holder and all the classes have a reference to the next property, which has a reference to the next, etc, with their own setter/getter methods as usual.
public class SpringCloudConfigSpring {
private SpringCloudConfigCloud cloud;
public SpringCloudConfigCloud getCloud() {
return cloud;
}
public void setCloud(SpringCloudConfigCloud cloud) {
this.cloud = cloud;
}
}
Implemented the representation of the map easily.
The last one I used the reference of a TreeMap to keep the keys sorted, another map to represent any property that may be added, without changing the representation.
public class SpringCloudConfigGit {
TreeMap<String, Map<String, Object>> repos;
public TreeMap<String, Map<String, Object>> getRepos() {
return repos;
}
public void setRepos(TreeMap<String, Map<String, Object>> repos) {
this.repos = repos;
}
}
Results
Creating the verification as follows:
public static void main(String[] args) throws JsonParseException, JsonMappingException, IOException {
File config = new File("/tmp/config-server-onboards.yml");
SpringCloudConfigFile props = ConfigServerProperties.parseProperties(config);
props.getSpring().getCloud().getConfig().getServer().getGit().getRepos().forEach((appName, properties) -> {
System.out.println("################## " + appName + " #######################3");
System.out.println(properties);
if (appName.equals("github_pages_reference")) {
properties.put("name", "Marcello");
properties.put("cloneOnStart", true);
}
System.out.println("");
});
saveProperties(new File(config.getAbsoluteFile().getParentFile(), "updated-config-onboards.yml"), props);
}
The output is as follows:
################## config_onboarding #######################3
{uri=https://github.company.com/servicesplatform-tools/spring-cloud-config-onboarding-config, cloneOnStart=true, username=226b4bb85aa131cd6393acee9c484ec426111d16, password=, pullOnRequest=false}
################## config_test_server_config #######################3
{uri=https://github.company.com/rlynch2/config-test-server-config, cloneOnStart=true, username=226b4bb85aa131cd6393acee9c484ec426111d16, password=, pullOnRequest=false}
################## github_pages_reference #######################3
{uri=https://github.company.com/servicesplatform-tools/spring-cloud-config-reference-service-config, cloneOnStart=true, username=226b4bb85aa131cd6393acee9c484ec426111d16, password=, pullOnRequest=false}
There are obvious improvements required:
I'd like to have a solution with a single class;
I'd like to have a an ObjetMapper method that specifies the "subtree" of the YAML object tree that I'd like to parse.
Maybe a more sophisticated SpringBoot-like #ConfigurationProperties("spring.cloud.config.server.git") would help.
Helper methods for loading and saving the state of these instances.
Load Method
public static SpringCloudConfigFile parseProperties(File filePath)
throws JsonParseException, JsonMappingException, IOException {
ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
SpringCloudConfigFile file = mapper.readValue(filePath, SpringCloudConfigFile.class);
return file;
}
Save Properties
public static void saveProperties(File filePath, SpringCloudConfigFile file) throws JsonGenerationException, JsonMappingException, IOException {
ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
mapper.writeValue(filePath, file);
}
File Saved
It maintained the sorted keys as implemented.
i have this spring-cloud-config client class and i can access the individual properties using the #Value annotation just fine. However, i am interested to know how to read ALL the properties values from a properties file without binding each of the property's key to a #Value annotation. Basically the idea is that i would like to read all the properties value from the properties file without even knowing anything about the properties defined in the file. Any idea how i can do that?
Client Class
#EnableAutoConfiguration
#ComponentScan
#RestController
#RefreshScope
public class ConfigDemoClientApplication
{
#Value("${special}")
String special;
#RequestMapping("/restaurant")
public String hello()
{
return "Hello " + special;
}
public static void main(String[] args) {
SpringApplication.run(ConfigDemoClientApplication.class, args);
}
}
Sample Properties file
special: bargain!
amount: 200
city: New York
In this example, i would like to read all the 3 properties without defining a #Value annotation for each of them in my class. Is that possible?
Thanks for your help.
I just solved you problem creating this applicationProps bean, that is a java.util.Properties object containing all the properties of the application.
The only think needed is an autowired Environment object.
Here's the code:
#Autowired
Environment env;
//Load all the properties of the server and put them into a java Properties obj
#Bean(name = "applicationProps")
public Properties applicationProperties() {
final Properties properties = new Properties();
for(Iterator it = ((AbstractEnvironment) env).getPropertySources().iterator(); it.hasNext(); ) {
PropertySource propertySource = (PropertySource) it.next();
if (propertySource instanceof PropertiesPropertySource) {
log.info("Adding all properties contained in " + propertySource.getName());
properties.putAll(((MapPropertySource) propertySource).getSource());
}
if (propertySource instanceof CompositePropertySource){
properties.putAll(getPropertiesInCompositePropertySource((CompositePropertySource) propertySource));
}
}
return properties;
}
private Properties getPropertiesInCompositePropertySource(CompositePropertySource compositePropertySource){
final Properties properties = new Properties();
compositePropertySource.getPropertySources().forEach(propertySource -> {
if (propertySource instanceof MapPropertySource) {
log.info("Adding all properties contained in " + propertySource.getName());
properties.putAll(((MapPropertySource) propertySource).getSource());
}
if (propertySource instanceof CompositePropertySource)
properties.putAll(getPropertiesInCompositePropertySource((CompositePropertySource) propertySource));
});
return properties;
}
#Autowired
#Qualifier("applicationProps")
Properties applicationProperties;
The recursive step in getPropertiesInCompositePropertySource method is needed because the properties fetched from the config server are recursively nested in a CompositePropertySource
Hope it helps
Greetings
Try this : Its all Spring, you can maybe use it with a PostConstruct Method
Map<String,String> someMap = new HashMap<>();
Resource resource = new ClassPathResource("some.properties");
Properties props = PropertiesLoaderUtils.loadProperties(resource);
for(Object key : props.keySet()) {
someMap.put(key.toString(),props.getProperty(key.toString()));
}
I am using a properties File to store some configuration properties, that are accessed this way:
#Value("#{configuration.path_file}")
private String pathFile;
Is it possible (with Spring 3) to use the same #Value annotation, but loading the properties from a database instead of a file ?
Assuming you have a table in your database stored key/value pairs:
Define a new bean "applicationProperties" - psuedo-code follows...
public class ApplicationProperties {
#AutoWired
private DataSource datasource;
public getPropertyValue(String key) {
// transact on your datasource here to fetch value for key
// SNIPPED
}
}
Inject this bean where required in your application. If you already have a dao/service layer then you would just make use of that.
Yes, you can keep your #Value annotation, and use the database source with the help of EnvironmentPostProcessor.
As of Spring Boot 1.3, we're able to use the EnvironmentPostProcessor to customize the application's Environment before application context is refreshed.
For example, create a class which implements EnvironmentPostProcessor:
public class ReadDbPropertiesPostProcessor implements EnvironmentPostProcessor {
private static final String PROPERTY_SOURCE_NAME = "databaseProperties";
private String[] CONFIGS = {
"app.version"
// list your properties here
};
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Map<String, Object> propertySource = new HashMap<>();
try {
// the following db connections properties must be defined in application.properties
DataSource ds = DataSourceBuilder
.create()
.username(environment.getProperty("spring.datasource.username"))
.password(environment.getProperty("spring.datasource.password"))
.url(environment.getProperty("spring.datasource.url"))
.driverClassName("com.mysql.jdbc.Driver")
.build();
try (Connection connection = ds.getConnection();
// suppose you have a config table storing the properties name/value pair
PreparedStatement preparedStatement = connection.prepareStatement("SELECT value FROM config WHERE name = ?")) {
for (int i = 0; i < CONFIGS.length; i++) {
String configName = CONFIGS[i];
preparedStatement.setString(1, configName);
ResultSet rs = preparedStatement.executeQuery();
while (rs.next()) {
propertySource.put(configName, rs.getString("value"));
}
// rs.close();
preparedStatement.clearParameters();
}
}
environment.getPropertySources().addFirst(new MapPropertySource(PROPERTY_SOURCE_NAME, propertySource));
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
}
Finally, don't forget to put your spring.factories in META-INF. An example:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=
com.baeldung.environmentpostprocessor.autoconfig.PriceCalculationAutoConfig
Although not having used spring 3, I'd assume you can, if you make a bean that reads the properties from the database and exposes them with getters.