I am wondering what is the best way to configure an WAS Liberty installation, allowing to to switch from a DEV environment configuration to an UAT(testing) environment configuration dynamically.
To elaborate, we have a similar setup with our glassfish servers, we simply configure system properties for both in the Glassfish console. For example
hostname.uat="some uat value"
hostname.dev="some dev value"
Dropping the ".uat" or ".dev" in the system property configuration in Glassfish makes that property active. In Glassfish, this can be done dynamically and while the application is running (no need to reboot).
Is there or can someone elaborate how I could achieve a similar setup in WAS Liberty?
Thank-you kindly
You can create a server.env file in two possible places:
${wlp.install.dir}/etc/server.env (properties are applied to all servers) or
${server.config.dir}/server.env (properties applied only to one server)
and specify any environment variables in that file.
For example:
# Specify properties and values
admin.email=dev.admin#domain.com
admin.email.uat=uat.admin#domain.com
To access these properties in an application environment (such as a Servlet) do the following:
System.getenv("admin.email"); // returns "dev.admin#domain.com"
Other useful properties can be specified in the server.env file as well such as JAVA_HOME, WLP_USER_DIR, WLP_OUTPUT_DIR, and WLP_DEBUG_ADDRESS.
For IBM's full doc on this, see: Customizing the Liberty Environment.
What we do is generate the Liberty server with Ansible, where the variables can be added to an ansible inventory based on environment.
So, our deployments essentially drop and recreate the Liberty server by using ansible templates and roles to stamp it out as needed.
Lastly, we make use of Hasicorp Vault (you can also use ansible-vault) for credentials or secrets at deploy time to fetch credentials. This is then injected into Ansible as JSON and used to stamp out the server.xml and other related configuration files.
Related
I know Springboot applications can use application.properties or application.yaml files to retrieve variables like database connectivity setting, username, password etc.
However, and because of sensitive information, our development team has these only for test environment but not for production environments. These are not available to developers.
These are provided by security team and set up directly on server in server configuration files on Liberty server (these are server.xml files located in the server installation directory).
Developers have their own instance of Liberty server running where they have their own versions of server.xml files. If we could make Springboot read these files, then we could make it mimic production server environments and make transition easier instead of reading local application.properties files.
Is this possible?
If not, what would be a workaround?
First, usually developer/application doesn't need direct access to props like database connectivity setting, username, password because all that is configured in server in data source configuration, so application just needs JNDI name of the datasource to connect to it.
Second, if you use technology that cannot be configured in server, developers and security team should utilize environment variables for such props.
Liberty can read env variables, or also define them via server.env and then utilize in server.xml config for example:
<dataSource jndiName="jdbc/myDS">
...
<properties.db2.jcc serverName="${JDBC_HOST}" portNumber="${JDBC_PORT}" databaseName="${JDBC_DB}" user="${JDBC_USER}" password="${JDBC_PASSWORD}"/>
</dataSource>
Similarly you can use env vars in your springboot either configuring it in application.properties like this:
jdbc.user=${JDBC_USER}
or directly in code:
#Value("${JDBC_USER}")
private String jdbcUser;
Utilizing env variables has additional benefit that you can override them later if deploying in containers for example.
I will talk about alternative solution.
Firstly I do not understand how developer will access production server properties.
And for securing sensitive properties you could use property encryptor tool. Sensitive properties will be in encrypted format in application.properties and during server startup it will decrypt encrypted properties and use accordingly.
Here is a such library for property encrytion library
My Spring Boot app will have 4 different environments:
Local; running locally on my machine
Dev
UAT
Prod
When a new user registers for my app, the backend sends them an email to verify their address/account. In this email there will be a link; the user clicks the link which verifies them in the database and allows them to now login and use the app.
These links of course have to have the environment built into them:
Locally, the link might be http://localhost:9200/v1/data/accounts/verify?vt=12345
In dev the link might be: http://dev.myapp.example.com/v1/data/accounts/verify?vt=12345
In UAT the link might be: http://uat.myapp.example.com/v1/data/accounts/verify?vt=12345
In Prod the link might be: http://myapp.example.com/v1/data/accounts/verify?vt=12345
In all three environments, the hostname + port are different. Locally I specify localhost:9200 (both localhost + port). Non-locally I don't need to specify port because the app will be running on nodes behind a load balancer. In production I don't need an environment-specific subdomain like dev or uat.
In order for my emails to work correctly in all 4 environments, I need to set an environment variable that is then used by my email generator to generate links correctly.
I could use something homegrown, such as a custom property in application.properties like emailDomain, and set the property different in each environment's properties file.
But I feel like the smart folks that make up the Spring Boot community have probably already solved this problem for me...have they? Is there already a Spring Boot app property I can set that will take care of all this for me?
In the comments, I read that your main concern is being able to update the property without having to modify your .JAR/.WAR or changing some code.
This is possible since you can externalize your Spring boot configuration. If you check the Externalized Configuration docs, you can see that it looks for properties within:
OS environment variables.
...
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar
(application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
So, ideally, you can put an application.properties file next to your JAR, and update the properties in that file depending on the environment you run on.
I understand there's multiple ways of handling application property files and profiles in Spring Boot and I've seen multiple questions and answers on how to handle each but I'm trying to find the "best" way of handling it for a new project, if there is one.
The application is hosted in Weblogic 12c on production/pre-prod (with a jndi database connection) and ran locally in tomcat (with hardcoded database details) for development. I'd like it so that when the project is built via gradle and deployed to production it uses the jndi properties file and when ran locally it defaults to the hardcoded datasource with minimal changes required.
src/main/resources/application.properties
# DEV
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# DEV
# PROD
# spring.datasource.jndi-name=
# spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# PROD
From my understanding the recommended way is to externalize the property files and place the required one in a config directory alongside the WAR file for any differing config which is then automatically picked up and used?
You should consider creating multiple profiles. This means: Either multiple properties-Files, or multiple profiles in one file:
See https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html
I would recommend to use multiple application-ENV.properties, e.g.
application-prod.properties and application-preprod.properties.
There is always one active profile and settings from the application.properties (without any profile suffix) are used as default values if not overwritten in a specific profile-file.
Depdending on your environment (local, prod etc.) you should set an environment variable (start the java-process/application server with that environment variable), e.g.:
SPRING_PROFILES_ACTIVE=prod
On your local machine you would set:
SPRING_PROFILES_ACTIVE=dev
With this variable you can control, which profile is currently active.
Also consider integrating the active profile into you Continious Integration/Deployment settings.
Please note that putting plain text passwords hardcoded into committed files is not a good idea. Consider using jasypt or a spring cloud config server for your prod database configuraiton or use any mechanism that your cloud provider provides to you if you use any. Microsoft Azure for example provides a KeyVault for sensitive data.
https://cloud.spring.io/spring-cloud-config/multi/multi_spring-cloud-config.html
http://www.jasypt.org/
If you use gradle good solution is to set application.properties and test.properties files and put into them properties for prod and preprod respectively.
Then run application with different vm arguments: -Dspring.profiles.active=test for test.properties and without arguments for application.properties
Use gradle instruments and configure them once for test and prod. For example: gradle bootWar configure without vm arguments, gradle bootWarTest with vm arguments -Dspring.profiles.active=test. Save once you configs and you will create war for different environments only selecting between two buttons in gradle.
We are looking to move our dev+prod WebSphere full profile app to Liberty.
Currently, we build only once (using Ant scripts) and deploy the same package (i.e. EAR) to our functional, UAT and production environments.
Database and MQ connections (and related sensitive data like usernames and passwords) are directly set via the WAS admin console for each environment, so there is no such data in our EAR.
A few non sensitive settings that change per environment (mail server address etc), are kept in a file suffixed with the (e.g configuration_.properties). All these files are bundled within the EAR. Each WebSphere defines a JVM property to specify the environment they are running (prd, uat, fnc, lab etc). When the application starts, it reads the files that is associated with the environment. That works great.
Now with Liberty, the connection/MQ pools, LDAP users etc are defined in server.xml.
Questions:
how to manage the server.xml file(s) that replace the job done via the WAS console by the authorized admins?
how to define the database name/port/host/user/password needed for those access per environnment? keep one server.xml file per environnement?
is there a way to have a "base" server.xml file and "override" the database name/port/host/user/password etc at startup on runtime?
or maybe there are more clever strategies?
We don't know yet if we will run Liberty in a traditional ND/Cluster way or into a docker infrastructure (this is all very new to us..).
How do you handle this?
Thanks in advance.
You can do the same thing in Liberty, just using different methods.
1) in your server.xml files, use variables wherever needed:
${this.style} for referencing system/bootstrap properties or server.xml defined variables, or ${env.ENV_VAR} for referencing Environment variables
2) add in a per-environment server.env file, or use configDropins/overrides to define environment-specific server.xml snippets (this answers one of your questions: yes, you can have a base server.xml and use environment-specific overrides)
More information here: http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.core.doc/ae/cwlp_config.html?lang=en
And here (specifically includes and dropins):
http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.core.doc/ae/twlp_setup_basics.html?lang=en
I currently have a number of deployable applications that work in a distributed fashion to solve a business problem. We are currently using a number of property configuration files to provide changing configuration per environment based off a system environment variable. All these deployable application share common configuration for database and messaging. This is currently achieved by picking up property files from the class path and having both deployed apps share a common jar for each connection (db, jms) containing property files.
I am looking to start using Spring Config Server to externalize this configuration if possible. I have a question about how to share this common config.
Currently it looks something like this:-
Web1
- database
- jms
Messaging1
- database
- jms
In this situation both deployed apps share the same connections and these connections change per environment (lab, prf, prd, etc). How can I achieve the same with the Spring Configuration Server where I have app config for each deployable app?
Application.yml
Web1.yml
Web1-dev.yml
Messaging1.yml
Messaging1-dev.yml
If a connection property changed for an environment I would need to make the change to each deployable app configuration rather than making it just once.
Is there currently anyway to achieve this? Am I just missing a simple point?
I found working solution here https://cloud.spring.io/spring-cloud-config/single/spring-cloud-config.html, paragraph "2.1.5 Sharing Configuration With All Applications". It says:
With file-based (i.e. git, svn and native) repositories, resources
with file names in application* are shared between all client
applications (so application.properties, application.yml,
application-*.properties etc.). You can use resources with these file
names to configure global defaults and have them overridden by
application-specific files as necessary.
You should create application.properties or application.yml at the top level of configuration repository (if it is git or svn based). Don't forget to commit the changes.
This is how I have configured for my setup.
1 All Common properties across all services and environments will be in root->application.properties files
2 All Common properties across all environments specific to service will be root->service-X.properties files
3: Similarly, to have common properties across specific environment use env->application.properties file
server:
port: 8888
spring:
cloud:
config:
server:
git:
uri:[git repo]
search-paths: /,/{profile}/
Finally found a solution. It's buried in the issues at github ...
https://github.com/spring-cloud/spring-cloud-config/issues/32
It worked liked described. I only noticed, that you need to put the files in a /config folder to make it work. If you put it in the root the file ist used by the configserver itself and is not included in the config requests.
application.properties/application.yml will be shared across all applications.
application-DEV.properties/application-DEV.yml will be shared across all DEV environment applications. You can replace DEV with any spring profile.
{applicationName}.properties/{applicationName}.yml will be shared across the give application.