i am facing one issue while working on a deliverable . I am using Spring boot +Apache Kafka . In my project we are using a Password repository to store the passwords as we cannot use passwords in properties file .The passwords are stored in a key value pair .We are able to get the passwords from that repository by passing the keys. And we are able to access that repository as it is bundled as a jar.
In my use case I have also configured some passwords in that repository for different environments like DEV ,UAT AND PROD, which i need for configuration and i need those passwords before application context is created . .For that purpose i created a listener which implements ApplicationListener and i have overrided the onApplicationEvent . Something like this
public class PropertiesListener implements ApplicationListener
{
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event)
{
ConfigurableEnvironment env = env.getEnvironment();
TradingEnvironment tEvn = Optional.ofNullable(env .getProperty("environment",TradingEnvironment.class).orElse(TradingEnvironment.LOCAL);
// OTHER CODE for fetching the password based on environment
TradingEnvironment is an enum which i have created which contains environments like LOCAL,DEV,UAT AND PROD
Now issue is that when i am deploying the code in DEV or UAT environment Optional.ofNullable(env .getProperty("environment",TradingEnvironment.class) this is giving me Optional.empty and i am getting environment as LOCAL instead of DEV or UAT . Not sure why it is coming like this .
i am using external properties file and i have a propertY environment = DEV for DEV properties file environment = UAT for UAT properties file .I have placed that properties file in the server and I am calling that properties file by using #PropertySource form the main class of the Spring boot application .
Can you please suggest why i am getting Optional.EMPTY or can u suggest any other way to get the passwords by their keys before application context is loaded .
Any help or suggestion would be much appreciated
Related
I use WAS Liberty Server container which provides server.xml and server.env file for configuring many things in addition to configuring DataSource properties such as url, username, password etc.
For security reasons, these properties are not known to developers for production Liberty servers. However, developers use same server.xml/server.evn files but with different DataSource properties so they can do their work.
So, I might have two server.env files like:
PRODUCTION server.env: dataSourceUrl="server-A-URL" (this URL is not known to developers)
DEVELOPMENT server.env: dataSourceUrl="server-B-URL" (this URL is known to developers)
, then the dataSourceUrl is used in server.xml files in production and development to set the url accordingly.
So, the structure of server.xml/server.env file is same for developers and production, only the DataSource url, username, password are different. This way developers can work using their own DataSource properties and once ready to deploy they app, it is handed to other team which then just changes the DataSource properties to the production values and deploys the application to production server.
With Springboot JPA, I know we can use application.properties file to set these DataSource values. But, we would like to be able to set these to the values located in server.env file. Basically to have something like this in application.properties file:
spring.datasource.url=dataSourceUrl //dataSourceUrl is set in server.env
, then be able to use this value in Java code using #Value injection like:
public class MyClass {
#Value("${spring.datasource.url}")
String dsUrl;
...
}
I have been reading about externalizing properties but I am not able to figure out how to do this
You can use Liberty's jndiEntry elements to make configured values available in JNDI. You will need the jndi-1.0 feature, after which you can configure,
<jndiEntry jndiName="spring/datasource/url" value="${dataSourceUrl}"/>
and access it in your application as:
String dsUrl = InitialContext.doLookup("spring/datasource/url");
Or, from a web or ejb component as:
#Resource(lookup = "spring/datasource/url")
String dsUrl;
I have a Maven/SpringBootApplication that takes its properties from a Spring config Server. I need to override the values of these properties using command line arguments. unfortunately, the properties keep the values provided by the config server and are not overridden by the command line arguments.
I have confirmed that the parameters are properly passed to the App as I can see being passed to SpringApplication.run.
I can see in the function ConfigurableApplicationContext of Spring Framework the environment carrying the arguments in environment.propertysources.propertySourceList.SimpleCommandLinePropertySource.source.optionArgs
If I try to set a Spring-defined value (e.g. --logging.level.org.springframework.web=TRACE) it works, meaning Spring logs traces
I read all possible threads on the subject but none seem to apply to my problem.
This is my Spring boot app (args are beeing passed to the SpringApplication)
#SpringBootApplication
#ComponentScan("com.mycompany")
public class App {
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
Here is the component and the property
#Component
public class TaskProcessor implements com.mycompnay.fwk.task.engine.TaskProcessor {
private RestTemplate restTemplate = new RestTemplate();
#Value("${mycompany.converter.converter-uri.office}")
private String converterUriOffice;
}
The parameter being passed is received by the app (extracted from debugger):
0:"--debug=true"
1:"--logging.level.org.springframework.web=TRACE"
2:"--mycompany.converter.converter-uri.office=foo"
hash:0
value:char[44]#25
I expect the property converterUriOffice to have the value foo
Instead it gets its value from the config server (http://localhost:3000/convert/office)
Links from Devilluminati did the job. Thanks a lot! To make it as clear as possible, here is what I had to do.
1- My application has a matching YML file served by the config server called application.yml
2- Inside the application.yml, I have two profiles and I only wanted the ability to override the arguments while using the local profile.
So here is what I had to add to application.yml:
spring:
profiles: local
cloud:
config:
override-system-properties: false
Once I did that (and restarted the config server to make sure it pulls the latest YML), I am able to override the value above by passing the following to the command line:
--mycompany.converter.converter-uri.office=foo
found following in the documentation https://cloud.spring.io/spring-cloud-static/Edgware.SR2/single/spring-cloud.html#overriding-bootstrap-properties
Overriding the Values of Remote Properties The property sources that
are added to you application by the bootstrap context are often
"remote" (e.g. from a Config Server), and by default they cannot be
overridden locally, except on the command line. If you want to allow
your applications to override the remote properties with their own
System properties or config files, the remote property source has to
grant it permission by setting spring.cloud.config.allowOverride=true
(it doesn’t work to set this locally). Once that flag is set there are
some finer grained settings to control the location of the remote
properties in relation to System properties and the application’s
local configuration: spring.cloud.config.overrideNone=true to override
with any local property source, and
spring.cloud.config.overrideSystemProperties=false if only System
properties and env vars should override the remote settings, but not
the local config files.
same problem here with the solution https://github.com/spring-cloud/spring-cloud-config/issues/907
The documentation is not very clear cause it says that command line
arguments always take precedence over remote configuration.
The property sources that are added to you application by the
bootstrap context are often "remote" (e.g. from a Config Server), and
by default they cannot be overridden locally, except on the command
line.
To achieve that, you have to activate the
spring.cloud.config.override-system-properties=false configuration
(the documentation only talk about System properties but it seems to
be applicable to command line arguments too).
I have spring boot micro-service with database credentials define in the application properties.
spring.datasource.url=<<url>>
spring.datasource.username=<<username>>
spring.datasource.password=<<password>>
We do not use spring data source to create the connection manually. Only Spring create the database connection with JPA.(org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration)
We only provide the application properties, but spring create the connections automatically to use with the database connection pool.
Our requirement to enhance the security without using db properties in clear text. Two possible methods.
Encrypt the database credentials
Use the AWS secret manager. (then get the credential with the application load)
For the option1, jasypt can be used, since we are just providing the properties only and do not want to create the data source manually, how to do to understand by the spring framework is the problem. If better I can get some working sample or methods.
Regarding the option-2,
first we need to define secretName.
use the secertName and get the database credentials from AWS secret manager.
update the application.properties programatically to understand by spring framework. (I need to know this step)
I need to use either option1 and option2. Mentioned the issues with each option.
What you could do is use environment variables for your properties. You can use them like this:
spring.datasource.url=${SECRET_URL}
You could then retrieve these and start your Spring process using a ProcessBuilder. (Or set the variables any other way)
I have found the solution for my problem.
We need to define org.springframework.context.ApplicationListenerin spring.factories file. It should define the required application context listener like below.
org.springframework.context.ApplicationListener=com.sample.PropsLoader
PropsLoader class is like this.
public class PropsLoader implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
ConfigurableEnvironment environment = event.getEnvironment();
String appEnv = environment.getProperty("application.env");
//set new properties based on the application environment.
// calling other methods and depends on the enviornment and get the required value set
Properties props = new Properties();
props.put("new_property", "value");
environment.getPropertySources().addFirst(new PropertiesPropertySource("props", props));
}
}
spring.factories file should define under the resources package and META-INF
folder.
This will set the application context with new properties before loading any other beans.
I am new to Spring Boot and I am doing code cleanup for my old Spring Boot application.
Below code is using #Value annotation to inject filed value from properties file.
#Value("${abc.local.configs.filepath}")
private String LOCAL_ABC_CONFIGS_XML_FILEPATH;
My doubt is instead of getting value from properties file, can we not directly hardcode the value in same java class variable?
Example: private String LOCAL_ABC_CONFIGS_XML_FILEPATH="/abc/config/abc.txt"
It would be easier for me to modify the values in future as it will be in same class.
What is advantage of reading from properties file, does it make the code decoupled ?
This technique is called as externalising configurations. You are absolutely right that you can have your constants defined in the very same class files. But, sometimes, your configurations are volatile or may change with respect to the environment being deployed to.
For Example:
Scene 1:
I have a variables for DB connection details which will change with the environment. Remember, you will create a build out of your application and deploy it first to Dev, then take same build to stage and finally to the production.
Having your configurations defined externally, helps you to pre-define them at environment level and have same build being deployed everywhere.
Scene 2:
You have already generated a build and deployed and found something was incorrect with the constants. Having those configurations externalised gives you a liberty to just override it on environment level and change without rebuilding your application.
To understand more about externalising techniques read: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
Here #value is used for reading the values from properties file (it could be a any environment like dev, qa, prod) but we are writing #value on multiple fields it is not recomonded so thats instead of #value we can use #configurableProperties(prefix="somevalue>) and read the property values suppose `
#configurableProperties(prefix="somevalue")
class Foo{
string name;
string address;
}
application.properties:
somevalue.name="your name"
somevalue.address="your address"
`
I have a springboot application which connects to database, currently the database password is in plain text inside the application properties.
What is the standard for securely protecting password in PROD environment?
How to change the database password if the application password is inside the application properties which is built in as part of the JAR and especially when the application is live?
You could use jasypt to handle the encryption and then use Jasypt's Spring integration or this Jasypt Spring Boot Starter to wire it into Spring.
This will allow you to define an encrypted database password property, for example in application.properties e.g.
db.password=ENC(.....)
The other part of your question is:
How to change the database password if the application password is inside the application properties
You can do this by overring properties defined in your properties file with system properties. For example: -Ddb.password='....'. You could also define an additional properties source which is external to your JAR and can be edited at runtime. For example:
#PropertySources({
#PropertySource(value = "classpath:/**.properties"),
#PropertySource(value = "file:/some/external/directory/override.properties", ignoreResourceNotFound = true)
})
public class Application {
// ...
}
Creating the file /some/external/directory/override.properties and populating it with db.password=... would cause your application - on next restart - to use that property value.