Springboot: Spring JPA - How to get datasource properties from server container - spring-boot

I use WAS Liberty Server container which provides server.xml and server.env file for configuring many things in addition to configuring DataSource properties such as url, username, password etc.
For security reasons, these properties are not known to developers for production Liberty servers. However, developers use same server.xml/server.evn files but with different DataSource properties so they can do their work.
So, I might have two server.env files like:
PRODUCTION server.env: dataSourceUrl="server-A-URL" (this URL is not known to developers)
DEVELOPMENT server.env: dataSourceUrl="server-B-URL" (this URL is known to developers)
, then the dataSourceUrl is used in server.xml files in production and development to set the url accordingly.
So, the structure of server.xml/server.env file is same for developers and production, only the DataSource url, username, password are different. This way developers can work using their own DataSource properties and once ready to deploy they app, it is handed to other team which then just changes the DataSource properties to the production values and deploys the application to production server.
With Springboot JPA, I know we can use application.properties file to set these DataSource values. But, we would like to be able to set these to the values located in server.env file. Basically to have something like this in application.properties file:
spring.datasource.url=dataSourceUrl //dataSourceUrl is set in server.env
, then be able to use this value in Java code using #Value injection like:
public class MyClass {
#Value("${spring.datasource.url}")
String dsUrl;
...
}
I have been reading about externalizing properties but I am not able to figure out how to do this

You can use Liberty's jndiEntry elements to make configured values available in JNDI. You will need the jndi-1.0 feature, after which you can configure,
<jndiEntry jndiName="spring/datasource/url" value="${dataSourceUrl}"/>
and access it in your application as:
String dsUrl = InitialContext.doLookup("spring/datasource/url");
Or, from a web or ejb component as:
#Resource(lookup = "spring/datasource/url")
String dsUrl;

Related

Environment coming as Optional.EMPTY

i am facing one issue while working on a deliverable . I am using Spring boot +Apache Kafka . In my project we are using a Password repository to store the passwords as we cannot use passwords in properties file .The passwords are stored in a key value pair .We are able to get the passwords from that repository by passing the keys. And we are able to access that repository as it is bundled as a jar.
In my use case I have also configured some passwords in that repository for different environments like DEV ,UAT AND PROD, which i need for configuration and i need those passwords before application context is created . .For that purpose i created a listener which implements ApplicationListener and i have overrided the onApplicationEvent . Something like this
public class PropertiesListener implements ApplicationListener
{
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event)
{
ConfigurableEnvironment env = env.getEnvironment();
TradingEnvironment tEvn = Optional.ofNullable(env .getProperty("environment",TradingEnvironment.class).orElse(TradingEnvironment.LOCAL);
// OTHER CODE for fetching the password based on environment
TradingEnvironment is an enum which i have created which contains environments like LOCAL,DEV,UAT AND PROD
Now issue is that when i am deploying the code in DEV or UAT environment Optional.ofNullable(env .getProperty("environment",TradingEnvironment.class) this is giving me Optional.empty and i am getting environment as LOCAL instead of DEV or UAT . Not sure why it is coming like this .
i am using external properties file and i have a propertY environment = DEV for DEV properties file environment = UAT for UAT properties file .I have placed that properties file in the server and I am calling that properties file by using #PropertySource form the main class of the Spring boot application .
Can you please suggest why i am getting Optional.EMPTY or can u suggest any other way to get the passwords by their keys before application context is loaded .
Any help or suggestion would be much appreciated

How to set properties of Tomcat JDBC Pool when using Spring Cloud Connector config?

I want to configure the properties of the Tomcat JDBC Pool with custom parameter values. The pool is bootstrapped by the spring-cloud (Spring Cloud Connector) environment (Cloud Foundry) and connected to a PostgreSQL database. In particular, I want to set the minIdle, maxIdle and initialSize properties for the given pool.
In a "spring-vanilla" environment (non-cloud) the properties can be easily set by using
application.properties / .yaml files with environment properties,
#ConfigurationProperties annotation.
However, this approach doesn't transfer to my Cloud environment, where the URL (and other parameters) are injected from the environment variable VCAP_SERVICES (via the ServiceInfo instances). I don't want to re-implement the logic which Spring Cloud already did with its connectors.
After some searching I also stumbled over some tutorials / guides, which suggest to make use of the PoolConfig object (e.g. http://cloud.spring.io/spring-cloud-connectors/spring-cloud-spring-service-connector.html#_relational_database_db2_mysql_oracle_postgresql_sql_server). However, that way one cannot set the properties I need but merely the following three:
minPoolSize,
maxPoolSize,
maxWaitTime.
Note that I don't want to set connection-related properties (such as charset), but the properties are associated with the pool itself.
In essence, I would like to do the configuration similarly to https://www.baeldung.com/spring-boot-tomcat-connection-pool (using spring.datasource.tomcat.* properties). The problem with that approach is that the properties are not considered if the datasource was created by Spring Cloud. The article https://dzone.com/articles/binding-data-services-spring, section "Using a CloudFactory to create a DataSource", claims that the following code snippet makes it so that the configuration "can be tweaked using application.properties via spring.datasource.* properties":
#Bean
#ConfigurationProperties(DataSourceProperties.PREFIX)
public DataSource dataSource() {
return cloud().getSingletonServiceConnector(DataSource.class, null);
}
However, my own local test (with spring-cloud:Greenwich.RELEASE and spring-boot-starter-parent:2.1.3.RELEASE) showed that those property values are simply ignored.
I found an ugly way to solve my problem but I think it's not appropriate:
Let spring-cloud create the DataSource, which is not the pooled DataSource directly,
check that the reference is a descendant of a DelegatingDataSource,
resolve the delegate, which is then the pool itself,
change the properties programmatically directly at the pool itself.
I do not believe that this is the right way as I am using internal knowledge (on the layering of datasources). Additionally, this approach does not work for the property initialSize, which is only considered when the pool is created.

Spring boot application properties load process change programatically to improve security

I have spring boot micro-service with database credentials define in the application properties.
spring.datasource.url=<<url>>
spring.datasource.username=<<username>>
spring.datasource.password=<<password>>
We do not use spring data source to create the connection manually. Only Spring create the database connection with JPA.(org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration)
We only provide the application properties, but spring create the connections automatically to use with the database connection pool.
Our requirement to enhance the security without using db properties in clear text. Two possible methods.
Encrypt the database credentials
Use the AWS secret manager. (then get the credential with the application load)
For the option1, jasypt can be used, since we are just providing the properties only and do not want to create the data source manually, how to do to understand by the spring framework is the problem. If better I can get some working sample or methods.
Regarding the option-2,
first we need to define secretName.
use the secertName and get the database credentials from AWS secret manager.
update the application.properties programatically to understand by spring framework. (I need to know this step)
I need to use either option1 and option2. Mentioned the issues with each option.
What you could do is use environment variables for your properties. You can use them like this:
spring.datasource.url=${SECRET_URL}
You could then retrieve these and start your Spring process using a ProcessBuilder. (Or set the variables any other way)
I have found the solution for my problem.
We need to define org.springframework.context.ApplicationListenerin spring.factories file. It should define the required application context listener like below.
org.springframework.context.ApplicationListener=com.sample.PropsLoader
PropsLoader class is like this.
public class PropsLoader implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
ConfigurableEnvironment environment = event.getEnvironment();
String appEnv = environment.getProperty("application.env");
//set new properties based on the application environment.
// calling other methods and depends on the enviornment and get the required value set
Properties props = new Properties();
props.put("new_property", "value");
environment.getPropertySources().addFirst(new PropertiesPropertySource("props", props));
}
}
spring.factories file should define under the resources package and META-INF
folder.
This will set the application context with new properties before loading any other beans.

What is advantage of using #value annotation in Spring Boot

I am new to Spring Boot and I am doing code cleanup for my old Spring Boot application.
Below code is using #Value annotation to inject filed value from properties file.
#Value("${abc.local.configs.filepath}")
private String LOCAL_ABC_CONFIGS_XML_FILEPATH;
My doubt is instead of getting value from properties file, can we not directly hardcode the value in same java class variable?
Example: private String LOCAL_ABC_CONFIGS_XML_FILEPATH="/abc/config/abc.txt"
It would be easier for me to modify the values in future as it will be in same class.
What is advantage of reading from properties file, does it make the code decoupled ?
This technique is called as externalising configurations. You are absolutely right that you can have your constants defined in the very same class files. But, sometimes, your configurations are volatile or may change with respect to the environment being deployed to.
For Example:
Scene 1:
I have a variables for DB connection details which will change with the environment. Remember, you will create a build out of your application and deploy it first to Dev, then take same build to stage and finally to the production.
Having your configurations defined externally, helps you to pre-define them at environment level and have same build being deployed everywhere.
Scene 2:
You have already generated a build and deployed and found something was incorrect with the constants. Having those configurations externalised gives you a liberty to just override it on environment level and change without rebuilding your application.
To understand more about externalising techniques read: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
Here #value is used for reading the values from properties file (it could be a any environment like dev, qa, prod) but we are writing #value on multiple fields it is not recomonded so thats instead of #value we can use #configurableProperties(prefix="somevalue>) and read the property values suppose `
#configurableProperties(prefix="somevalue")
class Foo{
string name;
string address;
}
application.properties:
somevalue.name="your name"
somevalue.address="your address"
`

Set/override Spring / Spring Boot properties at runtime

At the project with Spring Boot we use application.properties but need to configure some of these properties (like port number of logging level) based on an external configuration. We access the configuration via API so it is known only at runtime.
Is there a way to override or set some Spring properties at runtime (for example using a bean) and if yes how can this be achieved?
You could do this with Spring Cloud Config
Just for the purpose of illustration, here's a relatively quick way to see dynamic property overrides at runtime:
First, for your bean to be able to pick up changed properties, you need to annotate it with
#RefreshScope
Add the spring cloud dependency to your spring boot app, eg for gradle
compile group: 'org.springframework.cloud', name: 'spring-cloud-starter', version: '1.1.1.RELEASE'
( NB You also need the spring boot actuator dependency.)
With the app running, you can view your current config at eg
http://localhost:8080/env
eg if you have a property 'my.property' in application.properties, you'll see something like:
"applicationConfig: [classpath:/application.properties]": {
"my.property": "value1",
etc
To change the value, POST my.property=value2 to /env as application/x-www-form-urlencoded
eg
curl -X POST http://localhost:8080 -d my.property=value2
GET /env again and you'll see the new value appears under the "manager" section
To apply the changed properties, do an empty POST to /refresh. Now your bean will have the new value.
Could you use system properties to pass in the variable? If you configure the PropertyPlaceholderConfigurer you can set the precedence of system properties vs file properties.
For example, something like:
#Bean public PropertyPlaceholderConfigurer placeHolderConfigurer() {
PropertyPlaceholderConfigurer props = new PropertyPlaceholderConfigurer()
props.setSystemPropertiesMode( PropertyPlaceholderConfigurer.SYSTEM_PROPERTIES_MODE_OVERRIDE )
props.setLocations(new
PathMatchingResourcePatternResolver().getResources("classpath:/**.properties"));
props
}
The above would load your .properties file, but we set the priority to be system variables first, so if you set a system variable that will override the same variable in the config.
Alternatively, looking at the docs, Spring recommends defining a search order in your Environment:
[PropertyPlaceholderConfigurer is still appropriate for use when]
existing configuration makes use of the "systemPropertiesMode" and/or "systemPropertiesModeName" properties. Users are encouraged to
move away from using these settings, and rather configure property
source search order through the container's Environment; however,
exact preservation of functionality may be maintained by continuing to
use PropertyPlaceholderConfigurer.
Hopefully one of the above should sort out what you need?

Resources