I have a Spring Boot 2.3 reactive application with webflux and r2dbc. Normally it runs on MS Sql database. I want unit tests to run on H2. I got to the point when correct database driver is loaded based on which application.properties file is in use (main or test). But I can't figure out how to run SQL scripts to create schema and load data.
Tried the following without success:
schema-XXX.sql, data-XXX.sql, they work fine with non-reactive JPA.
#Sql annotation referring to the *.sql files in resources directory.
ConnectionFactoryInitializer bean located in the same package as application configuration files, but under the test source tree. #Configuration and #Bean are used properly, but the bean is not instantiated. Maybe this is the problem?
Nothing in the log suggests that there was even a failed attempt to execute *.sql files.
Thank you.
Related
I'm working with a spring boot project, currently I need to create 2 files before spring boot starts, in the junit test part (when I run the project into the server, it runs the testing part before, and spring boot use these files), so my idea is about create these files from the testing part (junit test), do you have another idea?
this is not working for this case, I need to create the files before it happens
public class xclass(){
//code before spring starts
main method(Springstarts.class);
}
then, one of the problem is about the db2 connection, because this kind of connection in application.properties needs necessary (and from the beginning) the jks files to connect, example:
spring.datasource.url=jdbc:db2:host:port/bdd:sslConnection=true;sslTrustStoreLocation=folder/file1.jks;
spring.datasource.username=userasdf1234
spring.datasource.password=passwordforconnect1234example
so, we need to create jks files before all of this runs, how can I do that?
I am writing a Spring Batch job and don't care about restarts and don't want the hassle of creating, securing and managing an Oracle schema for Spring Batch database objects. H2 on the file system is more than enough for us.
The issue is that I am writing an batch job that needs to connect to an Oracle database and failing miserably just trying to get 2 data sources, transaction managers and entity managers. Right now I am trying two H2 database and haven't even tried configuring Oracle database yet.
I have used two Oracle data sources in other Spring Boot applications successfully in the past but this is my first attempt with Spring Batch that has configuration code to create the data source.
I have tried creating two DataSource, EntityManagerFactoryBean and TransactionManager with one using default spring.datasources configuration, default bean names and #Primary.
I have tried creating just a second DataSource, EntityManagerFactoryBean and TransactionManager with different bean names. This seems to have an issue that the TransactionManager orEntityManager is already assigned to the thread.
I have tried creating a dataSource for batch but run into circular bean creation errors.
I have tried creating a BatchConfigurer and that runs into circular bean creation errors.
I have tried creating a JobRepositoryFactoryBean but a default one is still created.
I have tried using a #PersistenceContext on my #Entity classes.
It shouldn't be this hard so I must be missing something. Any help would be appreciated.
Thanks, Wes.
I think I have some success. I am able to eventually run the batch job using Spring Boot 2.2.0 and Spring Batch 4.2.0 but I have to wait for a 5 minute timeout while creating the Entity Manager for H2 for the Spring Batch repository. The Oracle Entity Manager is registered really quickly and before the H2 Entity Manager despite the H2 Entity Manager being #Primary.
I have a separate configuration class for each of my two DataSources. Each one is annotated with #Configuration, #EnableTransactionManagement and #EnableJPARepository.
The one for Spring Batch is using the standard bean names, dataSource, entityManagerFactory and transactionManager. Each #Bean is annotated with #Primary.
One setting that I needed was to add a .packages("org") to the entityManagerFactory bean. This would pick up all of org.springframework including Spring Batch.
The only other real change I have made from common implementations is to set the dialect in the JPA Properties of the Entity Manager.
I needed the spring.main.allow-bean-definition-overriding: true setting.
There may be more to my solution that I should share but I have been at this for a couple of days and have gone in circles. I even remember getting what appeared as a hung process time and and killed the job thinking it was hung. I may have had some "success" early on but just was too quick to kill the execution.
I would still like to know why it is taking so long to create the H2 Entity Manager.
Thanks, Wes.
My project has this requirement where user uploads a CSV file which has to be pushed to sql server database.
I am following the below basic example to load CSV file into a sql sever database.
https://github.com/michaelcgood/Spring-Batch-CSV-Example
runninng this repo by making change of datasource, here we using sql server instead of in-memory db.
here is the addition to POM file:
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<<artifactId>sqljdbc4</artifactId>
<version>4.0</version>
<</dependency>
additions to applications.properties file
spring.datasource.url=jdbc:sqlserver://localhost;databaseName=springbootdb
spring.datasource.username=sa
spring.datasource.password=Projects#123
Below is the error generated while running the code base, we could not found exact root cause we tried the multiple approaches as mentioned in the google but no luck, while running this spring batch+spring boot 2.0.4 application we are facing the below errors:
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
o.s.boot.autoconfigure.jdbc.Datasource.initailizer disabled(not running DDL scripts)
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE
where JOB_NAME = ? order by JOB_INSTANCE_ID desc]; nested exception is java.sql.SQLServerException: Invalid object name 'BATCH_JOB_INSTANCE'.
We are assuming the root cause one of the below?
1.We are using the spring starter project for creating spring batch configuration so not aware how we could define the default table schema and all.
2.maybe we don't have access and/or you need to define the schema. http://forum.spring.io/forum/spring-projects/batch/63771-badsqlgrammarexception-running-commandlinejobrunner
3.Not sure on the why error is saying Invalid object name 'BATCH_JOB_INSTANCE' instead of object doesnot exist?
What are the list of sql queries to create metadate table for spring batch application, to create it manually before running application.
4.we are struggling from couple of days to find the root cause of issue, Please suggest approch by taking the above github code base sample.
any help here would be appreciated. Please let us know if we missed anything here. Thanks In advance
The project you are linking to uses the in-memory hsqldb instance so Spring Boot will automatically create Spring Batch meta-data tables for you.
If you want to adapt the sample to your project and use sql server (which is not embedded in your JVM), Spring Boot will not create Spring Batch tables (unless you instruct it to do so). In this case, you have two options:
Run the Spring Batch DDL script for sqlserver yourself against the db server you are using before running your app
Or instruct Spring Boot to do it for you by specifying the following properties in your application.properties file:
spring.batch.initialize-schema=always
spring.batch.schema=classpath:org/springframework/batch/core/schema-sqlserver.sql
I'm suing Spring Boot with JPA. I tried to create src/main/resources/import.sql script for provisioning users into application properties:
spring.datasource.jndi-name=java:/global/production
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MariaDBDialect
spring.jpa.show-sql = true
spring.jpa.hibernate.ddl-auto = update
spring.datasource.data=classpath:import.sql
But unfortunately when I deploy the application the script is not run. Do you know with JPA what is the proper way to create SQL provisioning script?
If you need to initialized your database (if I understand you correctly), all you need to do in your Spring Boot project is just provide file data.sql with your DML scripts in your root classpath location (i.e. in resources dir):
Spring Boot can automatically create the schema (DDL scripts) of your DataSource and initialize it (DML scripts). It loads SQL from the standard root classpath locations: schema.sql and data.sql, respectively.
Source.
Another approach you can find here.
With the #Sql annotation you can specify the path of the file you want to run before the tests. For example:
#TestPropertySource("classpath:application-test.properties")
#Sql("/data.sql")
class MyClassTest
You can read more in here.
I got an app with an existing database that has Oracle only field type (e.g. "binary-float") mapped on the entity class with #ColumnDefinition annotation.
Things runs fine when running the app normally by launching the Application class.
However I can't seem to find a way to write junit tests easily. In another Spring Boot app, I have been using different profile to define a normal datasource that points to Oracle and a junit test datasource that points to the h2 in-memory db. I stayed mostly within JPAQL and common sql standard when using direct sql. Problem is, this scheme doesn't work if the JPA mapping annotation is database specific.
Any suggestions?