Related
tldr
How to best "put" runtime application.properties values from a spring boot app in a javascript file
[Admittedly this is probably a stupid question for the experienced spring dev...so please share how to solve this problem "the spring way"]
Context
We have tiny app which a front-end developer put together and "threw over the wall"..It defines a properties file settings.js:
var SERVERROOT = 'http://solr:8983/solr/operations/select/';
referenced by html :
<script type='text/javascript' src="js/settings.js"></script>
I would like to define the solr path in application.yml at runtime as follows:
app:
solr:
path: 'http://solr:8983/solr/operations/select/'
Question
What is the best way to "populate" this settings.js value from application.properties (or equivalent, i.e. command line arguments, e.g.:
-Dapp.solr.path=...
)
Possible
I thought about putting 'settings.js' as a thymeleaf template (settings.js.html) and having a spring controller populate the model from application.properties.
I didn't know if a more "native spring" method existed.
Thanks!
You could use a ResourceTransformer:
#Component
public class InjectSolrPathResourceTransformer implements ResourceTransformer {
private final MySetting settings; // inject via constructor
#Override
public Resource transform(HttpServletRequest request, Resource resource, ResourceTransformerChain transformerChain) throws IOException {
if (request.getServletPath().equals("/js/settings.js")) {
byte[] bytes = FileCopyUtils.copyToByteArray(resource.getInputStream());
String content = new String(bytes, StandardCharsets.UTF_8);
content = content.replace("var SERVERROOT = SERVERROOT_VALUE",
"var SERVERROOT = '" + settings.getSolrPath() + "'");
return new TransformedResource(resource, content.getBytes(StandardCharsets.UTF_8));
} else {
return resource;
}
}
}
This assumes that you change settings.js to be:
var SERVERROOT = SERVERROOT_VALUE
To use the ResourceTransformer, register it in your WebMvcConfigurer:
#Configuration
public class WebConfig implements WebMvcConfigurer {
// inject this via constructor
private final InjectSolrPathResourceTransformer resourceTransformer;
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/js/**")
.addResourceLocations("classpath:/public/js/")
.addTransformer(resourceTransformer);
}
}
The answer of #Wim above is correct but with new spring boot edition use the below code to register your transformer;
#Configuration
public class WebConfig implements WebMvcConfigurer {
// inject this via constructor
private final InjectSolrPathResourceTransformer resourceTransformer;
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/js/**")
.addResourceLocations("classpath:/public/js/")
.resourceChain(false)
.addTransformer(resourceTransformer);
}
}
I'm trying to disable a Zuul route to a microservice registered with Eureka at runtime (I'm using spring boot).
This is an example:
localhost/hello
localhost/world
Those two are the registered microservices. I would like to disable the route to one of them at runtime without shutting it down.
Is there a way to do this?
Thank you,
Nano
Alternatively to using Cloud Config, custom ZuulFilter can be used. Something like (partial implementation to show the concept):
public class BlackListFilter extends ZuulFilter {
#Override
public String filterType() {
return "pre";
}
...
#Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
String uri = ctx.getRequest().getRequestURI();
String appId = uri.split("/")[1];
if (blackList.contains(appId)) {
ctx.setSendZuulResponse(false);
LOG.info("Request '{}' from {}:{} is blocked",
uri, ctx.getRequest().getRemoteHost(), ctx.getRequest().getRemotePort());
}
return null;
}
}
where blackList contains list of application IDs (Spring Boot application name) managed for example via some RESTful API.
After a lot of efforts I came up with this solution. First, I used Netflix Archaius to watch a property file. Then I proceeded as follows:
public class ApplicationRouteLocator extends SimpleRouteLocator implements RefreshableRouteLocator {
public ApplicationRouteLocator(String servletPath, ZuulProperties properties) {
super(servletPath, properties );
}
#Override
public void refresh() {
doRefresh();
}
}
Made the doRefresh() method public by extending SimpleRouteLocator and calling its method in the overridden one of the interface RefreshableRouteLocator.
Then I redefined the bean RouteLocator with my custom implementation:
#Configuration
#EnableConfigurationProperties( { ZuulProperties.class } )
public class ZuulConfig {
public static ApplicationRouteLocator simpleRouteLocator;
#Autowired
private ZuulProperties zuulProperties;
#Autowired
private ServerProperties server;
#Bean
#Primary
public RouteLocator routeLocator() {
logger.info( "zuulProperties are: {}", zuulProperties );
simpleRouteLocator = new ApplicationRouteLocator( this.server.getServletPrefix(),
this.zuulProperties );
ConfigurationManager.getConfigInstance().addConfigurationListener( configurationListener );
return simpleRouteLocator;
}
private ConfigurationListener configurationListener =
new ConfigurationListener() {
#Override
public void configurationChanged( ConfigurationEvent ce ) {
// zuulProperties.getRoutes() do something
// zuulProperties.getIgnoredPatterns() do something
simpleRouteLocator.refresh();
}
}
}
Every time a property in the file was modified an event was triggered and the ConfigurationEvent was able to deal with it (getPropertyName() and getPropertyValue() to extract data from the event). Since I also Autowired the ZuulProperties I was able to get access to it. With the right rule I could find whether the property of Zuul
zuul.ignoredPatterns
was modified changing its value in the ZuulProperties accordingly.
Here refresh context should work (as long as you are not adding a new routing rule or removing a currently existing one), if you are adding or removing routing rules, you have to add a new bean for ZuulProperties and mark it with #RefreshScope, #Primary.
You can autowire refreshEndpoint bean for example and apply refreshEndpoint.refresh() on the listener.
Marking a custom RouteLocator as primary will cause problems as zuul already has bean of same type marked as primary.
I'm wondering what the best way to load initial database data before the application starts? What I'm looking for is something that will fill my H2 database with data.
For example, I have a domain model "User" I can access users by going to /users but initially there won't be any users in the database so I have to create them. Is there anyway to fill the database with data automatically?
At the moment I have a Bean that gets instantiated by the container and creates users for me.
Example:
#Component
public class DataLoader {
private UserRepository userRepository;
#Autowired
public DataLoader(UserRepository userRepository) {
this.userRepository = userRepository;
LoadUsers();
}
private void LoadUsers() {
userRepository.save(new User("lala", "lala", "lala"));
}
}
But I very much doubt that is the best way of doing it. Or is it?
You can create a data.sql file in your src/main/resources folder and it will be automatically executed on startup. In this file you can add some insert statements, eg.:
INSERT INTO users (username, firstname, lastname) VALUES
('lala', 'lala', 'lala'),
('lolo', 'lolo', 'lolo');
Similarly, you can create a schema.sql file (or schema-h2.sql) as well to create your schema:
CREATE TABLE task (
id INTEGER PRIMARY KEY,
description VARCHAR(64) NOT NULL,
completed BIT NOT NULL);
Though normally you shouldn't have to do this since Spring boot already configures Hibernate to create your schema based on your entities for an in memory database. If you really want to use schema.sql you'll have to disable this feature by adding this to your application.properties:
spring.jpa.hibernate.ddl-auto=none
More information can be found at the documentation about Database initialization.
If you're using Spring Boot 2, database initialization only works for embedded databases (H2, HSQLDB, ...). If you want to use it for other databases as well, you need to change the initialization mode property:
spring.sql.init.mode=always # Spring Boot >=v2.5.0
spring.datasource.initialization-mode=always # Spring Boot <v2.5.0
If you're using multiple database vendors, you can name your file data-h2.sql or data-mysql.sql depending on which database platform you want to use.
To make that work, you'll have to configure the datasource platform property:
spring.sql.init.platform=h2 # Spring Boot >=v2.5.0
spring.datasource.platform=h2 # Spring Boot <v2.5.0
If I just want to insert simple test data I often implement a ApplicationRunner. Implementations of this interface are run at application startup and can use e.g. a autowired repository to insert some test data.
I think such an implementation would be slightly more explicit than yours because the interface implies that your implementation contains something you would like to do directly after your application is ready.
Your implementation would look sth. like this:
#Component
public class DataLoader implements ApplicationRunner {
private UserRepository userRepository;
#Autowired
public DataLoader(UserRepository userRepository) {
this.userRepository = userRepository;
}
public void run(ApplicationArguments args) {
userRepository.save(new User("lala", "lala", "lala"));
}
}
You can add a spring.datasource.data property to application.properties listing the sql files you want to run. Like this:
spring.datasource.data=classpath:accounts.sql, classpath:books.sql, classpath:reviews.sql
//or (depending on SB version)
spring.sql.init.data-locations=classpath:accounts.sql, classpath:books.sql, file:reviews.sql
The sql insert statements in each of these files will then be run, allowing you to keep things tidy.
If you put the files in the classpath, for example in src/main/resources they will be applied. Or replace classpath: with file: and use an absolute path to the file
If you want to run DDL type SQL then use:
spring.datasource.schema=classpath:create_account_table.sql
// depending on spring version
spring.sql.init.schema-locations=classpath:create_account_table.sql
Edit: these solutions are great to get you up and running quickly, however for a more production ready solution it would be worth looking at a framework such as flyway, or liquibase. These frameworks integrate well with spring, and provide a quick, consistent, version-controlled means of initialising schema, and standing-data.
There are multiple ways how to achieve this. I prefer to use one of following options:
Option 1: Initializing with CommandLineRunner bean:
#Bean
public CommandLineRunner loadData(CustomerRepository repository) {
return (args) -> {
// save a couple of customers
repository.save(new Customer("Jack", "Bauer"));
repository.save(new Customer("Chloe", "O'Brian"));
repository.save(new Customer("Kim", "Bauer"));
repository.save(new Customer("David", "Palmer"));
repository.save(new Customer("Michelle", "Dessler"));
// fetch all customers
log.info("Customers found with findAll():");
log.info("-------------------------------");
for (Customer customer : repository.findAll()) {
log.info(customer.toString());
}
log.info("");
// fetch an individual customer by ID
Customer customer = repository.findOne(1L);
log.info("Customer found with findOne(1L):");
log.info("--------------------------------");
log.info(customer.toString());
log.info("");
// fetch customers by last name
log.info("Customer found with findByLastNameStartsWithIgnoreCase('Bauer'):");
log.info("--------------------------------------------");
for (Customer bauer : repository
.findByLastNameStartsWithIgnoreCase("Bauer")) {
log.info(bauer.toString());
}
log.info("");
}
}
Option 2: Initializing with schema and data SQL scripts
Prerequisites:
application.properties
spring.jpa.hibernate.ddl-auto=none
Explanation:
Without ddl-auto SQL scripts will be ignored by
Hibernate and trigger default behavior - scanning project for
#Entity and/or #Table annotated classes.
Then, in your MyApplication class paste this:
#Bean(name = "dataSource")
public DriverManagerDataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("org.h2.Driver");
dataSource.setUrl("jdbc:h2:~/myDB;MV_STORE=false");
dataSource.setUsername("sa");
dataSource.setPassword("");
// schema init
Resource initSchema = new ClassPathResource("scripts/schema-h2.sql");
Resource initData = new ClassPathResource("scripts/data-h2.sql");
DatabasePopulator databasePopulator = new ResourceDatabasePopulator(initSchema, initData);
DatabasePopulatorUtils.execute(databasePopulator, dataSource);
return dataSource;
}
Where scripts folder is located under resources folder (IntelliJ Idea)
Hope it helps someone
Update 04-2021: Both options are great to combine with Spring Profiles as this will help you to avoid creating additional config files making your life as the developer easy.
You can use something like this:
#SpringBootApplication
public class Application {
#Autowired
private UserRepository userRepository;
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Bean
InitializingBean sendDatabase() {
return () -> {
userRepository.save(new User("John"));
userRepository.save(new User("Rambo"));
};
}
}
In Spring Boot 2 data.sql was not working with me as in spring boot 1.5
import.sql
In addition, a file named import.sql in the root of the classpath is executed on startup if Hibernate creates the schema from scratch (that is, if the ddl-auto property is set to create or create-drop).
Note very important if you insert Keys cannot be duplicated do not use ddl-auto property is set to update because with each restart will insert same data again
For more information you vist spring websit
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html
Spring Boot allows you to use a simple script to initialize your database, using Spring Batch.
Still, if you want to use something a bit more elaborated to manage DB versions and so on, Spring Boot integrates well with Flyway.
See also:
Spring Boot Database initialization
You can simply create a import.sql file in src/main/resources and Hibernate will execute it when the schema is created.
If you came here and nothing seems to work for you, then it might be the case that you are affected from some changes that were introduced with Spring Boot 2.5 and onwards.
Here is the total set of properties which I use for postgresql.
spring:
sql.init.mode: always <-----------------
datasource:
url: jdbc:postgresql://localhost:5432/products
username:
password:
jpa:
defer-datasource-initialization: true <------------------
hibernate:
ddl-auto: create-drop <----------------
database-platform: org.hibernate.dialect.PostgreSQLDialect
I have also marked with <--- the relevant properties for the current topic in order to achieve the following.
ORM vendor will create database schema for you from Java Entities model.
After database schema is created, initial data will be loaded to database from the file data.sql
Ps: Don't forget to add the file with initial data, data.sql under src/main/resources
Also as reference: Spring Boot 2.5 release notes
Here is the way I got that:
#Component
public class ApplicationStartup implements ApplicationListener<ApplicationReadyEvent> {
/**
* This event is executed as late as conceivably possible to indicate that
* the application is ready to service requests.
*/
#Autowired
private MovieRepositoryImpl movieRepository;
#Override
public void onApplicationEvent(final ApplicationReadyEvent event) {
seedData();
}
private void seedData() {
movieRepository.save(new Movie("Example"));
// ... add more code
}
}
Thanks to the author of this article:
http://blog.netgloo.com/2014/11/13/run-code-at-spring-boot-startup/
I solved similar problem this way:
#Component
public class DataLoader {
#Autowired
private UserRepository userRepository;
//method invoked during the startup
#PostConstruct
public void loadData() {
userRepository.save(new User("user"));
}
//method invoked during the shutdown
#PreDestroy
public void removeData() {
userRepository.deleteAll();
}
}
You're almost there!
#Component
public class DataLoader implements CommandLineRunner {
private UserRepository userRepository;
public DataLoader(UserRepository userRepository) {
this.userRepository = userRepository;
}
#Override
public void run(String... args) throws Exception {
LoadUsers()
}
private void LoadUsers() {
userRepository.save(new User("lala", "lala", "lala"));
}
}
you can register and event listener to achieve that like below:
#EventListener
public void seed(ContextRefreshedEvent event) {
userRepository.save(new User("lala", "lala", "lala"));
}
When the ContextRefreshEvent is fired, we get access to all autowired beans in the application — including models and repositories.
If someone are struggling in make this to work even following the accepted answer, for me only work adding in my src/test/resources/application.yml the H2 datasource details:
spring:
datasource:
platform: h2
url: jdbc:h2:mem:test;DB_CLOSE_DELAY=-1
driver-class-name: org.h2.Driver
username: sa
password:
If you want to insert only few rows and u have JPA Setup. You can use below
#SpringBootApplication
#Slf4j
public class HospitalManagementApplication {
public static void main(String[] args) {
SpringApplication.run(HospitalManagementApplication.class, args);
}
#Bean
ApplicationRunner init(PatientRepository repository) {
return (ApplicationArguments args) -> dataSetup(repository);
}
public void dataSetup(PatientRepository repository){
//inserts
}
You can use the below code. In the following code a database insertion occurs during the startup of the spring boot application.
#SpringBootApplication
public class Application implements CommandLineRunner {
#Autowired
private IService<Car> service;
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Override
public void run(String... args) throws Exception {
for(int i=1; i<=1000; i++) {
Car car = new Car();
car.setName("Car Name "+i);
book.setPrice(50 + i);
service.saveOrUpdate(car);
}
}
}
This will also work.
#Bean
CommandLineRunner init (StudentRepo studentRepo){
return args -> {
// Adding two students objects
List<String> names = Arrays.asList("udara", "sampath");
names.forEach(name -> studentRepo.save(new Student(name)));
};
}
The most compact (for dynamic data) put #mathias-dpunkt solution into MainApp (with Lombok #AllArgsConstructor):
#SpringBootApplication
#AllArgsConstructor
public class RestaurantVotingApplication implements ApplicationRunner {
private final VoteRepository voteRepository;
private final UserRepository userRepository;
public static void main(String[] args) {
SpringApplication.run(RestaurantVotingApplication.class, args);
}
#Override
public void run(ApplicationArguments args) {
voteRepository.save(new Vote(userRepository.getOne(1), LocalDate.now(), LocalTime.now()));
}
}
One possibility is using incorrect JDBC URL. make sure it is jdbc:h2:mem:testdb
I created a library that facilitates initial/demo data loading in a Spring Boot application. You can find it at https://github.com/piotrpolak/spring-boot-data-fixtures
Once the data fixtures starter is on the classpath, it will automatically try to load DICTIONARY data upon application startup (this behavior can be controlled by properties) - all you need to do is to register a bean implementing DataFixture.
I find loading initial data by code superior to loading it using SQL scripts:
the logic of your fixtures lives close to your application logic/domain model and it is subject to refactoring as your domain evolves
you benefit from incremental demo data updates - imagine a QA environment with some user data (that needs not to be lost after application deploy) but at the same time you want to add data for the new features you developed
Example data fixture:
/**
* You can have as many fixture classes as you want.
* #Order annotation is respected for the fixtures belonging to the same set.
* You can make your demo database to be incrementally updated with fresh data
* each time the application is redeployed - all you need to do is to write
* a good condition in `canBeLoaded()` method.
*/
#Component
public class InitialDataFixture implements DataFixture {
private final LanguageRepository languageRepository;
// ...
#Override
public DataFixtureSet getSet() {
return DataFixtureSet.DICTIONARY;
}
/**
* We want to make sure the fixture is applied once and once only.
* A more sophisticated condition can be used to create incremental demo data
* over time without the need to reset the QA database (for example).
*/
#Override
public boolean canBeLoaded() {
return languageRepository.size() == 0;
}
/**
* The actual application of the fixture.
* Assuming that data fixtures are registered as beans, this method can call
* other services and/or repositories.
*/
#Override
public void load() {
languageRepository.saveAll(Arrays.asList(
new Language("en-US"), new Language("pl-PL")));
}
}
The concept is inspired by the Symfony Doctrine Data Fixtures bundle.
For those using MysqlDriver, I tried using Init attribute of #bean annotation and it works.
After created the Schema and Data sql file in the path of resources\Scripts
Add the line in application.properties
spring.jpa.hibernate.ddl-auto=none
Edit the Application content:
package com.spring_mvaen.demo;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.jdbc.datasource.init.DatabasePopulator;
import org.springframework.jdbc.datasource.init.DatabasePopulatorUtils;
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
#SpringBootApplication
public class DemoApplication implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#Override
public void run(String... arg0) throws Exception {
System.out.println("Hello world from Command Line Runner");
}
#Bean(name = "dataSource")
public DriverManagerDataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/db_spring_rest?useUnicode=true&useLegacyDatetimeCode=fa lse&serverTimezone=UTC&createDatabaseIfNotExist=true&allowPublicKeyRetrieval=true&useSSL=false");
dataSource.setUsername("root");
dataSource.setPassword("root");
// schema init
Resource initSchema = new ClassPathResource("scripts/schema.sql");
Resource initData = new ClassPathResource("scripts/data.sql");
DatabasePopulator databasePopulator = new ResourceDatabasePopulator(initSchema, initData);
DatabasePopulatorUtils.execute(databasePopulator, dataSource);
return dataSource;
}
}
If you want to do insert quick some queries, you can do with h2 data.sql queries as well
application.properties include:
spring.jpa.show-sql=true
spring.h2.console.enabled=true
spring.datasource.url=jdbc:h2:mem:testdb
#This directs the data.sql file and help it to run
spring.sql.init.data-locations=classpath:data.sql
spring.jpa.defer-datasource-initialization=true
data.sql file include:
INSERT INTO todo (id, username, description, target_date, is_done) VALUES (10001, 'lighteducation', 'Learn dance', CURRENT_DATE ,false);
INSERT INTO todo (id, username, description, target_date, is_done) VALUES (10002, 'lighteducation', 'Learn Angular14', CURRENT_DATE, false);
INSERT INTO todo (id, username, description, target_date, is_done) VALUES (10003, 'lighteducation', 'Learn Microservices', CURRENT_DATE,false);
P.S.: data.sql file should be inside src/main/resources
Your #Entity include
#Getter
#Setter
#AllArgsConstructor
#ToString
#Entity
public class Todo {
#Id
#GeneratedValue
private Long id;
private String username;
private String description;
private Date targetDate;
private boolean isDone;
protected Todo() {
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Todo todo = (Todo) o;
return id == todo.id;
}
#Override
public int hashCode() {
return Objects.hash(id);
}
}
That is it basically. it will be in memory, it means when you restart application data will arese and will be again same as queries show.
But it is easy for quick check
also you can access the path with http://localhost:8080/h2-console/ or you can edit the path from .properties file
Spring Boot allows us to replace our application.properties files with YAML equivalents. However, I seem to hit a snag with my tests. If I annotate my TestConfiguration (a simple Java config), it is expecting a properties file.
For example this doesn't work:
#PropertySource(value = "classpath:application-test.yml")
If I have this in my YAML file:
db:
url: jdbc:oracle:thin:#pathToMyDb
username: someUser
password: fakePassword
And I'd be leveraging those values with something like this:
#Value("${db.username}") String username
However, I end up with an error like so:
Could not resolve placeholder 'db.username' in string value "${db.username}"
How can I leverage the YAML goodness in my tests as well?
Spring-boot has a helper for this, just add
#ContextConfiguration(initializers = ConfigFileApplicationContextInitializer.class)
at the top of your test classes or an abstract test superclass.
Edit: I wrote this answer five years ago. It doesn't work with recent versions of Spring Boot. This is what I do now (please translate the Kotlin to Java if necessary):
#TestPropertySource(locations=["classpath:application.yml"])
#ContextConfiguration(
initializers=[ConfigFileApplicationContextInitializer::class]
)
is added to the top, then
#Configuration
open class TestConfig {
#Bean
open fun propertiesResolver(): PropertySourcesPlaceholderConfigurer {
return PropertySourcesPlaceholderConfigurer()
}
}
to the context.
As it was mentioned #PropertySource doesn't load yaml file. As a workaround load the file on your own and add loaded properties to Environment.
Implemement ApplicationContextInitializer:
public class YamlFileApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
try {
Resource resource = applicationContext.getResource("classpath:file.yml");
YamlPropertySourceLoader sourceLoader = new YamlPropertySourceLoader();
PropertySource<?> yamlTestProperties = sourceLoader.load("yamlTestProperties", resource, null);
applicationContext.getEnvironment().getPropertySources().addFirst(yamlTestProperties);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
Add your initializer to your test:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = Application.class, initializers = YamlFileApplicationContextInitializer.class)
public class SimpleTest {
#Test
public test(){
// test your properties
}
}
#PropertySource can be configured by factory argument. So you can do something like:
#PropertySource(value = "classpath:application-test.yml", factory = YamlPropertyLoaderFactory.class)
Where YamlPropertyLoaderFactory is your custom property loader:
public class YamlPropertyLoaderFactory extends DefaultPropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
if (resource == null){
return super.createPropertySource(name, resource);
}
return new YamlPropertySourceLoader().load(resource.getResource().getFilename(), resource.getResource(), null);
}
}
Inspired by https://stackoverflow.com/a/45882447/4527110
Another option is to set the spring.config.location through #TestPropertySource:
#TestPropertySource(properties = { "spring.config.location = classpath:<path-to-your-yml-file>" }
#PropertySource only supports properties files (it's a limitation from Spring, not Boot itself). Feel free to open a feature request ticket in JIRA.
From Spring Boot 1.4, you can use the new #SpringBootTest annotation to achieve this more easily (and to simplify your integration test setup in general) by bootstrapping your integration tests using Spring Boot support.
Details on the Spring Blog.
As far as I can tell, this means you get all the benefits of Spring Boot's externalized config goodness just like in your production code, including automatically picking up YAML config from the classpath.
By default, this annotation will
... first attempt to load #Configuration from any inner-classes, and if that fails, it will search for your primary #SpringBootApplication class.
but you can specify other configuration classes if required.
For this particular case, you can combine #SpringBootTest with #ActiveProfiles( "test" ) and Spring will pick up your YAML config, provided it follows the normal Boot naming standards (i.e. application-test.yml).
#RunWith( SpringRunner.class )
#SpringBootTest
#ActiveProfiles( "test" )
public class SpringBootITest {
#Value("${db.username}")
private String username;
#Autowired
private MyBean myBean;
...
}
Note: SpringRunner.class is the new name for SpringJUnit4ClassRunner.class
The approach to loading the yaml properties, IMHO can be done in two ways:
a. You can put the configuration in a standard location - application.yml in the classpath root - typically src/main/resources and this yaml property should automatically get loaded by Spring boot with the flattened path name that you have mentioned.
b. The second approach is a little more extensive, basically define a class to hold your properties this way:
#ConfigurationProperties(path="classpath:/appprops.yml", name="db")
public class DbProperties {
private String url;
private String username;
private String password;
...
}
So essentially this is saying that load the yaml file and populate the DbProperties class based on the root element of "db".
Now to use it in any class you will have to do this:
#EnableConfigurationProperties(DbProperties.class)
public class PropertiesUsingService {
#Autowired private DbProperties dbProperties;
}
Either of these approaches should work for you cleanly using Spring-boot.
Since Spring Boot 2.4.0 you can use ConfigDataApplicationContextInitializer as follows:
#SpringJUnitConfig(
classes = { UserAccountPropertiesTest.TestConfig.class },
initializers = { ConfigDataApplicationContextInitializer.class }
)
class UserAccountPropertiesTest {
#Configuration
#EnableConfigurationProperties(UserAccountProperties.class)
static class TestConfig { }
#Autowired
UserAccountProperties userAccountProperties;
#Test
void getAccessTokenExpireIn() {
assertThat(userAccountProperties.getAccessTokenExpireIn()).isEqualTo(120);
}
#Test
void getRefreshTokenExpireIn() {
assertThat(userAccountProperties.getRefreshTokenExpireIn()).isEqualTo(604800);
}
}
See also: https://www.baeldung.com/spring-boot-testing-configurationproperties#YAML-binding
I found a workaround by using #ActiveProfiles("test") and adding an application-test.yml file to src/test/resources.
It ended up looking like this:
#SpringApplicationConfiguration(classes = Application.class, initializers = ConfigFileApplicationContextInitializer.class)
#ActiveProfiles("test")
public abstract class AbstractIntegrationTest extends AbstractTransactionalJUnit4SpringContextTests {
}
The file application-test.yml just contains the properties that I want to override from application.yml (which can be found in src/main/resources).
I have tried all of the listed questions, but all of them not work for my task: using specific yaml file for some unit test.
In my case, it works like this:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(initializers = {ConfigFileApplicationContextInitializer.class})
#TestPropertySource(properties = {"spring.config.location=file:../path/to/specific/config/application.yml"})
public class SomeTest {
#Value("${my.property.value:#{null}}")
private String value;
#Test
public void test() {
System.out.println("value = " + value);
}
}
it's because you have not configure snakeyml.
spring boot come with #EnableAutoConfiguration feature.
there is snakeyml config too when u call this annotation..
this is my way:
#Configuration
#EnableAutoConfiguration
public class AppContextTest {
}
here is my test:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(
classes = {
AppContextTest.class,
JaxbConfiguration.class,
}
)
public class JaxbTest {
//tests are ommited
}
I needed to read some properties into my code and this works with spring-boot 1.3.0.RELEASE
#Autowired
private ConfigurableListableBeanFactory beanFactory;
// access a properties.yml file like properties
#Bean
public PropertySource properties() {
PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer = new PropertySourcesPlaceholderConfigurer();
YamlPropertiesFactoryBean yaml = new YamlPropertiesFactoryBean();
yaml.setResources(new ClassPathResource("properties.yml"));
propertySourcesPlaceholderConfigurer.setProperties(yaml.getObject());
// properties need to be processed by beanfactory to be accessible after
propertySourcesPlaceholderConfigurer.postProcessBeanFactory(beanFactory);
return propertySourcesPlaceholderConfigurer.getAppliedPropertySources().get(PropertySourcesPlaceholderConfigurer.LOCAL_PROPERTIES_PROPERTY_SOURCE_NAME);
}
Loading custom yml file with multiple profile config in Spring Boot.
1) Add the property bean with SpringBootApplication start up as follows
#SpringBootApplication
#ComponentScan({"com.example.as.*"})
public class TestApplication {
public static void main(String[] args) {
SpringApplication.run(TestApplication.class, args);
}
#Bean
#Profile("dev")
public PropertySourcesPlaceholderConfigurer propertiesStage() {
return properties("dev");
}
#Bean
#Profile("stage")
public PropertySourcesPlaceholderConfigurer propertiesDev() {
return properties("stage");
}
#Bean
#Profile("default")
public PropertySourcesPlaceholderConfigurer propertiesDefault() {
return properties("default");
}
/**
* Update custom specific yml file with profile configuration.
* #param profile
* #return
*/
public static PropertySourcesPlaceholderConfigurer properties(String profile) {
PropertySourcesPlaceholderConfigurer propertyConfig = null;
YamlPropertiesFactoryBean yaml = null;
propertyConfig = new PropertySourcesPlaceholderConfigurer();
yaml = new YamlPropertiesFactoryBean();
yaml.setDocumentMatchers(new SpringProfileDocumentMatcher(profile));// load profile filter.
yaml.setResources(new ClassPathResource("env_config/test-service-config.yml"));
propertyConfig.setProperties(yaml.getObject());
return propertyConfig;
}
}
2) Config the Java pojo object as follows
#Component
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonInclude(Include.NON_NULL)
#ConfigurationProperties(prefix = "test-service")
public class TestConfig {
#JsonProperty("id")
private String id;
#JsonProperty("name")
private String name;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
3) Create the custom yml (and place it under resource path as follows,
YML File name : test-service-config.yml
Eg Config in the yml file.
test-service:
id: default_id
name: Default application config
---
spring:
profiles: dev
test-service:
id: dev_id
name: dev application config
---
spring:
profiles: stage
test-service:
id: stage_id
name: stage application config
<dependency>
<groupId>com.github.yingzhuo</groupId>
<artifactId>spring-boot-stater-env</artifactId>
<version>0.0.3</version>
</dependency>
Welcome to use my library. Now yaml, toml, hocon is supported.
Source: github.com
This is not an answer to the original question, but an alternative solution for a need to have a different configuration in a test...
Instead of #PropertySource you can use -Dspring.config.additional-location=classpath:application-tests.yml.
Be aware, that suffix tests does not mean profile...
In that one YAML file one can specify multiple profiles, that can kind of inherit from each other, read more here - Property resolving for multiple Spring profiles (yaml configuration)
Then, you can specify in your test, that active profiles (using #ActiveProfiles("profile1,profile2")) are profile1,profile2 where profile2 will simply override (some, one does not need to override all) properties from profile1.
project demo url: https://github.com/Forest10/spring-boot-family/tree/spring-boot-with-yml
I run this answer in my prod env!!! so if you against this ans. please test first!!!
There is no need to add like YamlPropertyLoaderFactory or YamlFileApplicationContextInitializer. You should convert your idea
Follow these steps:
Just add applicationContext.xml like
#ImportResource({"classpath:applicationContext.xml"})
to your ApplicationMainClass.
and your applicationContext.xml should write like this
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:context="http://www.springframework.org/schema/context"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
default-autowire="byName"
xmlns="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd">
<context:property-placeholder location="classpath*:*.yml"/>
</beans>
This can help scan your application-test.yml
db:
url: jdbc:oracle:thin:#pathToMyDb
username: someUser
password: fakePassword
Enhancing Mateusz Balbus answer.
Modified YamlFileApplicationContextInitializer class where YAML location is defined per test class. It does not work per test, unfortunately.
public abstract class YamlFileApplicationContextInitializer
implements ApplicationContextInitializer<ConfigurableApplicationContext> {
/***
* Return location of a YAML file, e.g.: classpath:file.yml
*
* #return YAML file location
*/
protected abstract String getResourceLocation();
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
try {
Resource resource = applicationContext.getResource(getResourceLocation());
YamlPropertySourceLoader sourceLoader = new YamlPropertySourceLoader();
PropertySource<?> yamlTestProperties = sourceLoader.load("yamlTestProperties", resource, null);
applicationContext.getEnvironment().getPropertySources().addFirst(yamlTestProperties);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
Usage:
Create subclass of YamlFileApplicationContextInitializer with defined getResourceLocation() method and add this subclass into #SpringApplicationConfiguration annotation.
This way it is easiest to make the test class itself.
#RunWith(SpringRunner.class)
#SpringApplicationConfiguration(classes = Application.class, initializers = SimpleTest.class)
public class SimpleTest extends YamlFileApplicationContextInitializer {
#Override
protected String getResourceLocation() {
return "classpath:test_specific.yml";
}
#Test
public test(){
// test your properties
}
}
Here's an improved version of YamlPropertyLoaderFactory which supports the new PropertySource.ignoreResourceNotFound, based on this answer:
Java:
public final class YamlPropertyLoaderFactory extends DefaultPropertySourceFactory {
private final YamlPropertySourceLoader yamlPropertySourceLoader = new YamlPropertySourceLoader();
#NotNull
public PropertySource createPropertySource(
#Nullable String name,
#NotNull EncodedResource resource
) {
try {
String parsedName;
if (name != null && !name.equals(""))
parsedName = name;
else parsedName = resource.getResource().getFilename();
return yamlPropertySourceLoader.load(parsedName, resource.getResource()).get(0);
} catch (Exception e) {
Exception possibleFileNotFoundException = ExceptionUtils.throwableOfType(e, FileNotFoundException.class);
throw possibleFileNotFoundException != null ? possibleFileNotFoundException : e;
}
}
}
// Usage
#PropertySource(
value = {"file:./my-optional-config.yml"},
factory = YamlPropertyLoaderFactory.class,
ignoreResourceNotFound = true
)
Kotlin:
class YamlPropertyLoaderFactory : DefaultPropertySourceFactory() {
private val yamlPropertySourceLoader = YamlPropertySourceLoader()
override fun createPropertySource(
name: String?,
resource: EncodedResource
): PropertySource<*> = try {
(
yamlPropertySourceLoader.load(
if (name != null && name.isNotBlank()) name else resource.resource.filename,
resource.resource
)
)[0]
} catch (e: Exception) {
throw ExceptionUtils.throwableOfType(e, FileNotFoundException::class.java) ?: e
}
}
// Usage
#PropertySource(
value = ["file:/my-optional-config.yml"],
factory = YamlPropertyLoaderFactory::class,
ignoreResourceNotFound = true
)
I have a Spring 3.1 #Configuration that needs a property foo to build a bean. The property is defined in defaults.properties but may be overridden by the property in overrides.properties if the application has an active override Spring profile.
Without the override, the code would look like this, and work...
#Configuration
#PropertySource("classpath:defaults.properties")
public class MyConfiguration {
#Autowired
private Environment environment;
#Bean
public Bean bean() {
...
// this.environment.getRequiredProperty("foo");
...
}
}
I would like a #PropertySource for classpath:overrides.properties contingent on #Profile("overrides"). Does anyone have any ideas on how this could be achieved? Some options I've considered are a duplicate #Configuration, but that would violate DRY, or programmatic manipulation of the ConfigurableEnvironment, but I'm not sure where the environment.getPropertySources.addFirst() call would go.
Placing the following in an XML configuration works if I inject the property directly with #Value, but not when I use Environment and the getRequiredProperty() method.
<context:property-placeholder ignore-unresolvable="true" location="classpath:defaults.properties"/>
<beans profile="overrides">
<context:property-placeholder ignore-unresolvable="true" order="0"
location="classpath:overrides.properties"/>
</beans>
Update
If you're trying to do this now, check out Spring Boot's YAML support, particularly the 'Using YAML instead of Properties' section. The profile support there would make this question moot, but there isn't #PropertySource support yet.
Add the overriding #PropertySource in a static inner class. Unfortunately, you must specify all property sources together which means creating a "default" profile as the alternative to "override".
#Configuration
public class MyConfiguration
{
#Configuration
#Profile("default")
#PropertySource("classpath:defaults.properties")
static class Defaults
{ }
#Configuration
#Profile("override")
#PropertySource({"classpath:defaults.properties", "classpath:overrides.properties"})
static class Overrides
{
// nothing needed here if you are only overriding property values
}
#Autowired
private Environment environment;
#Bean
public Bean bean() {
...
// this.environment.getRequiredProperty("foo");
...
}
}
I suggest, defining two files, where the second is optional with the profile as suffix:
#Configuration
#PropertySources({
#PropertySource("classpath:/myconfig.properties"),
#PropertySource(value = "classpath:/myconfig-${spring.profiles.active}.properties", ignoreResourceNotFound = true)
})
public class MyConfigurationFile {
#Value("${my.prop1}")
private String prop1;
#Value("${my.prop2}")
private String prop2;
}
You can do:
<context:property-placeholder location="classpath:${spring.profiles.active}.properties" />
Edit: if you need something more advanced, you can register your PropertySources on application startup.
web.xml
<context-param>
<param-name>contextInitializerClasses</param-name>
<param-value>com.xxx.core.spring.properties.PropertySourcesApplicationContextInitializer</param-value>
</context-param>
file you create:
public class PropertySourcesApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final Logger LOGGER = LoggerFactory.getLogger(PropertySourcesApplicationContextInitializer.class);
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
LOGGER.info("Adding some additional property sources");
String[] profiles = applicationContext.getEnvironment().getActiveProfiles()
// ... Add property sources according to selected spring profile
// (note there already are some property sources registered, system properties etc)
applicationContext.getEnvironment().getPropertySources().addLast(myPropertySource);
}
}
Once you've done it you just need to add in your context:
<context:property-placeholder/>
I can't really answer to your question about multiple profiles but I guess you activate them on such an initializer, and you could register the appropriate PropertySource items during profile activations.
I can't think of any other way than one you have suggested Emerson, which is to define this bean in a separate #Configuration file with an #Profile annotation:
#Configuration
#Profile("override")
#PropertySource("classpath:override.properties")
public class OverriddenConfig {
#Autowired
private Environment environment;
#Bean
public Bean bean() {
//if..
}
}
In case you need to support multiple profiles you could do something like this:
#Configuration
public class Config {
#Configuration
#Profile("default")
#PropertySource("classpath:application.properties")
static class DefaultProperties {
}
#Configuration
#Profile("!default")
#PropertySource({"classpath:application.properties", "classpath:application-${spring.profiles.active}.properties"})
static class NonDefaultProperties {
}
}
That way you don't need to define a static configuration class for each profile.
Thanks David Harkness for putting me into the right direction.
Note: This answer provides an alternate solution to using properties files with #PropertySource. I went this route because it was too cumbersome trying to work with multiple properties files that may each have overrides while avoiding repetitive code.
Create a POJO interface for each related set of properties to define their names and types.
public interface DataSourceProperties
{
String driverClassName();
String url();
String user();
String password();
}
Implement to return the default values.
public class DefaultDataSourceProperties implements DataSourceProperties
{
public String driverClassName() { return "com.mysql.jdbc.Driver"; }
...
}
Subclass for each profile (e.g. development, production) and override any values that differ from the default. This requires a set of mutually-exclusive profiles, but you can easily add "default" as the alternative to "overrides".
#Profile("production")
#Configuration
public class ProductionDataSourceProperties extends DefaultDataSourceProperties
{
// nothing to override as defaults are for production
}
#Profile("development")
#Configuration
public class DevelopmentDataSourceProperties extends DefaultDataSourceProperties
{
public String user() { return "dev"; }
public String password() { return "dev"; }
}
Finally, autowire the properties configurations into the other configurations that need them. The advantage here is that you don't repeat any #Bean creation code.
#Configuration
public class DataSourceConfig
{
#Autowired
private DataSourceProperties properties;
#Bean
public DataSource dataSource() {
BoneCPDataSource source = new BoneCPDataSource();
source.setJdbcUrl(properties.url());
...
return source;
}
}
I am still not convinced I'll stick with this over manually configuring properties files based on the active profiles in a servlet context initializer. My thought was that doing manual configuration would not be as amenable to unit testing, but I'm not so sure now. I really prefer reading properties files to a list of property accessors.
All mentioned here solutions are a bit awkward, work only with one profile preset, and they won't work with more/other profiles. Currently a Spring team refuses to introduce this feature. But here's the working workaround I've found:
package com.example;
public class MyPropertySourceFactory implements PropertySourceFactory, SpringApplicationRunListener {
public static final Logger logger = LoggerFactory.getLogger(MyPropertySourceFactory.class);
#NonNull private static String[] activeProfiles = new String[0];
// this constructor is used for PropertySourceFactory
public MyPropertySourceFactory() {
}
// this constructor is used for SpringApplicationRunListener
public MyPropertySourceFactory(SpringApplication app, String[] params) {
}
#Override
public void environmentPrepared(ConfigurableBootstrapContext bootstrapContext, ConfigurableEnvironment environment) {
activeProfiles = environment.getActiveProfiles();
}
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource encodedResource) throws IOException {
logger.info("Loading: {} with profiles: {}", encodedResource.toString(), activeProfiles);
// here you know all profiles and have the source Resource with main
// properties, just try to load other resoures in the same path with different
// profile names and return them as a CompositePropertySource
}
}
To make it working you have to have src/main/resources/META-INF/spring.factories with the following content:
org.springframework.boot.SpringApplicationRunListener=com.example.MyPropertySourceFactory
Now you can put your custom properties file somewhere and load it with #PropertySources:
#Configuration
#PropertySource(value = "classpath:lib.yml", factory = MyPropertySourceFactory.class)
public class PropertyLoader {
}