Is there a way to get Liquibase migrations working using Quarkus when the datasources are know only at runtime? For example: One data source per-tenant in a multi-tenant application. The Development, Testing, Staging, and Production environments will all have different datasources.
According to https://quarkus.io/guides/datasource#configuring-multiple-datasources: “Even when only one database extension is installed, named databases need to specify at least one build time property so that Quarkus knows they exist.”
But I’ve found that runtime-only datasources actually work under the dev profile, and only don’t work under the prod profile. So I’m hopeful.
If there’s not a way to do this under the prod profile using Quarkus datasources, then is there a recommended approach?
Minimal reproducible example here: https://github.com/paulmossman/quarkus-liquibase-with-runtime-datasources/blob/main/README.md
Related
Testcontainers can manage dockerized service dependencies, like a database, Kafka, Elasticsearch, and so on for integration testing.
Can I configure my Spring Boot application to manage these service dependencies during local development?
For example, my Spring Boot application needs a MySQL database.
I would like to integrate it with Testcontainers to provide a Docker container with MySQL not only during the tests execution, but at application startup during local development too.
Testcontainers provides an API to manage applications and services in Docker containers. It's incredibly useful for integration testing, where having a programmatically configured, isolated, repeatable environments is an essential requirement for trustworthy tests.
Because of that Testcontainers has integrations with the frameworks like Spring and Quarkus, and tes frameworks like JUnit, Spock, etc to automatically tie the lifecycle of your containerized dependencies to the lifecycle of the tests.
However, Testcontainers API is generic and doesn't have to run during the tests. For example, Quarkus has a feature called Dev Services which automatically creates a container for your database (or other service dependencies, for example Kafka, Redis, etc) when your application tries to access the database, but the configuration is not present.
You can think about it like this, if you have the data access repository classes initialized and wired, but no datasource.url in the config -- it'll spin up the database using testcontainers and configure the app to use it (just like it would happen during tests, but instead used for local development).
Spring Boot doesn't have an automated feature like that currently, there's an open issue to investigate these local development setups with Testcontainers.
If you're open to manually add a feature for your particular application, you can look at the prototype linked from that issue here: https://github.com/joshlong/testcontainers-auto-services-prototype
It's a bit more involved because it integrates with the Spring DevTools, but here are the essential parts that need to be taken care of:
Check that you need to use the database (in your application it can be a given).
Verify the configuration to use the database is absent (if the database is already configured you don't need to spin up a new one)
Create a container using Testcontainers API, either using an appropriate module or the GenericContainer with any Docker image.
Provide the configuration back to the application. For the database that would be the jdbcUrl, username, password, database name, r2dbcUrl and any other relevant properties.
You can take a look at the video with Josh Long where this concept was tried: https://www.youtube.com/watch?v=1PUshxvTbAc&t=2450s
It would also work in the production environments, but the usefulness of the ephemeral Databases, might be limited.
So, I have my application based on spring and hibernate. The user produces some data (in my case the data is kind of development itself) which is persisted by hibernate.
But for now this won't be accepted by large enterprises. They want to have a development enviroment, a test environment and a production. What I need to implement is a way to deploy data from one environment to another.
To be clear: I am not asking about deploying the application, but its data.
Are there best practices to implement this feature?
To maintain DDL and use same across various environments use liquibase or flyaway which also integrates with seamlessly with spring.
If you want DML to be migrated then vendor specific data migration can be used.
I think you are mostly looking at DDL only so either of above is better solution
I am trying to find the best way to configure my Spring Boot Web application to easily switching between the following data sources for both local testing and deployment.
H2 in memory db. Local testing only.
Dev oracle. Local testing and deployment.
Prod oracle. Deployment only.
By local testing, I mean to test in IDE environment (Eclipse). Dev and prod oracle databases are set up on two remote servers.
After some research, there are different ways to switch from one data source to another.
Use Spring profile. Using H2 and Oracle with Spring Boot. Set up the following files in classpath, application.properties, application-h2. properties and application-dev.properties. While connections for h2 and dev are defined in corresponding properties files, spring.profiles.active is set in application.properties. My understanding is this property can be overridden during build process by specifying spring.profiles.active. However, it seems to be a JVM variable, how do I set it running maven?
Maven profile. Create multiple profiles in pom and a filter pointing to application properties files. The profile specified by -P option during maven build will determine which application properties file to look. However, according to maven application with multi environment configuration can't deploy on tomcat, this will generate multiple wars for different deployment. So method 1 is preferred. Plus, it does not apply to switching datasources while testing locally.
Persistence units. Define different persistence units for different data sources in persistence.xml. Use EntityManager by choosing a specific unit. Variation of this method include having a variable in unit names which is determined in application.properties.
JNDI lookup. Set up a jndi name in application.properties with spring.datasource.jndi-name. The actual database information including url and credentials will be specified in context.xml in the tomcat folder where the war will be deployed.
My mind is set on local testing environment. Gonna go with method 1. Switching between H2 in memory and oracle is so easy just by changing the property in application.properties. Since the testing is usually done in IDE, war does not need to be generated, although answers are welcome for run maven install with spring.profiles.active.
As far as deployment, JNDI is definitely the way to go. However, I am concerned that the two properties in application.properties: spring.profiles.active and spring.datasource.jndi-name may be conflicting with each other. If I have spring.profiles.active=h2 and then tried to deploy the war to prod server, does it try to connect to h2 based on the spring profile or to prod db based on jdni-name? What is the best practice to accommodate all scenarios with enough flexibility?
Also is a explicit configuration class for DataSource required such as Configure Mutiple DataSource in Spring Boot with JNDI? My understanding is application.properties and spring profile should be enough to handle it, right?
Definitely use Spring profiles.
You don't want to use Maven profiles as it creates different artifacts. Ask your QA/Release engineers how they feel about having different artifacts for different environments :). They wouldn't be happy.
H2 is what you want to use in CI server integration testing as well. Such integration testing is fast and easy.
Instead of changing profile in application.properties, consider defining profile via command line parameter. So that configuration file changes are not required to run your application in different profiles.
I have a requirement to load properties for different environments like DEV, QA and I have different properties file for each environment. So I solved this by setting environment property in server and accessing this value to load respective property files. When googled I found that Spring Environment Profiles provides the solution for similar scenarios. However, even here I have to set active-profile variable in server as environment variable.
What are the benefits of using Spring Environment Profiles over my native approach?
Profile lets you override anything in the Spring Context, properties, beans etc, from environment to environment, customer to customer. It is a easy and clean way to have custom implementations at any level of your beans.
For example, Lets assume your are building a product which read data from a relational database, you can develop DAO layer with profile="default". Then if another customer of yours or you yourself want to provide NoSQL support, you can develop another DAO layer with profile="nosql". This will make sure you can same product on both support based on profile. Easy and clean.
I am working on a project which have profile="local" which will help you bring application locally with out any database dependency (kind of mock mode). You can think of million other applications like to make use of Profile concept.
We are working with a Spring 3 application that runs on several environments (test, UAT and Production) these environments are managed by a third party company so we have almost no access to the servers.
We have tried with Jboss System Properties and Maven2 Profiles. Both solutions worked fine, however we don't want to tie the application to one specific Server (Jboss in this case) and we don't want to do environment specific builds (required for Maven2 profiles).
Is there a good way we could have environment specific properties for the app that do not require different builds for each environment and require no modifications on the server side and that could also run on different servers? (some sort of PropertyPlaceHolderConfigurer that could read property files outside the app context should do the trick)
Environment-specific builds are not a bad option.
But spring 3.1 is providing what you are looking for - environment specific configuration. See this and this