Tests fail to run with H2 database on Micronaut project with Flyway migrations - h2

I've got a unit test suite in a Micronaut 2.0.3 project that's supposed to run on an H2 DB. Since I started using Flyway for my DB migrations, I'm getting an error coming from H2 being unable to run the migrations.
The error is
Bean definition [javax.sql.DataSource] could not be loaded: Migration V1__schema.sql failed
-------------------------------
SQL State : 42001
Error Code : 42001
Message : Syntax error in SQL statement "CREATE DATABASE[*] IF NOT EXISTS MY_DB"; expected "OR, FORCE, VIEW, ALIAS, SEQUENCE, USER, TRIGGER, ROLE, SCHEMA, CONSTANT, DOMAIN, TYPE, DATATYPE, AGGREGATE, LINKED, MEMORY, CACHED, LOCAL, GLOBAL, TEMP, TEMPORARY, TABLE, SYNONYM, PRIMARY, UNIQUE, HASH, SPATIAL, INDEX"; SQL statement:
CREATE DATABASE if not exists my_db [42001-200]
Location : db/migration/V1__schema.sql (/somelocation/build/resources/main/db/migration/V1__schema.sql)
Line : 1
Statement : CREATE DATABASE if not exists my_db
io.micronaut.context.exceptions.BeanInstantiationException: Bean definition [javax.sql.DataSource] could not be loaded: Migration V1__schema.sql failed
-------------------------------
My application.yml is configured to point at a MariaDB instance in Azure like so:
micronaut:
application:
name: myapp
server:
context-path: /myapp/api
cors:
enabled: true
endpoints:
all:
enabled: true
sensitive: false
datasources:
default:
url: jdbc:mariadb://${ENV_DB_HOST_PORT:myurl.mariadb.database.azure.com}:${ENV_DB_PORT:3306}/${ENV_DB_SCHEMA:my_db}?useSSL=${ENV_DB_USE_SSL:true}&useUnicode=true&serverTimezone=Europe/London&autoReconnect=true
driverClassName: org.mariadb.jdbc.Driver
username: myusername
password: mypassword
dialect: MYSQL
allow-pool-suspension: false
connection-test-query: SELECT 1
connection-timeout: 30000
max-lifetime: 60000
maximum-pool-size: 10
minimum-idle: 1
flyway:
datasources:
default:
enabled: true
But my application-test.yml points at the H2 DB for testing purposes:
datasources:
default:
url: jdbc:h2:mem:devDb;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
driverClassName: org.h2.Driver
username: sa
password: ''
jpa:
default:
properties:
hibernate:
hbm2ddl:
auto: update
And my migration file V1__schema.sql looks like the following:
CREATE DATABASE if not exists my_db;
use my_db;
DROP TABLE IF EXISTS `preset`;
CREATE TABLE `preset` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`preset_name` varchar(30) NOT NULL,
`preset_value` varchar(4000) DEFAULT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
DROP TABLE IF EXISTS `note`;
CREATE TABLE `note` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`code` varchar(30) NOT NULL,
`text` varchar(4000) DEFAULT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

The error is pretty much self explanatory
Message : Syntax error in SQL statement "CREATE DATABASE[*] IF NOT EXISTS MY_DB";
CREATE DATABASE if not exists my_db;
According to H2 commands, CREATE DATABASE is not a valid command for H2. You probably need the command CREATE SCHEMA if not exists my_db;

Related

H2: always try to create tabes with file database

I have configured the application-local.yml like this to do some tests in local:
spring:
datasource:
url: jdbc:h2:file:./data/db-test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
username: dbtest
password: dbtest
driver-class-name: org.h2.Driver
jpa:
show-sql: true
generate-ddl: false
properties:
# to prevent this issue : https://stackoverflow.com/questions/4588755/disabling-contextual-lob-creation-as-createclob-method-threw-error
hibernate.temp.use_jdbc_metadata_defaults : false
hibernate:
dialect: org.hibernate.dialect.H2Dialect
hibernate:
ddl-auto: update
h2:
console:
enabled: true
path: /public/h2-console
liquibase:
enabled: false
The first time the server is started, I see some logs from Hibernate to create tables and the file for the database is created.
The issue is that when I restart the server, I have some errors because it tries to create again the same tables, constraints... I have the ddl-auto set to update so I do not understand why. I tried to update the generate-ddl value to false but it is the same.
Here is an exemple of what I have:
Hibernate: create table myTable (id bigint not null, ..., primary key (id))
15:59:49.252 [restartedMain] WARN o.h.t.s.i.ExceptionHandlerLoggedImpl - GenerationTarget encountered exception accepting command : Error executing DDL "create table myTable (id bigint not null, ..., primary key (id))" via JDBC Statement
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "create table myTable(id bigint not null, ..., primary key (id))" via JDBC Statement
at org.hibernate.tool.schema.internal.exec.GenerationTargetToDatabase.accept(GenerationTargetToDatabase.java:67)
at org.hibernate.tool.schema.internal.AbstractSchemaMigrator.applySqlString(AbstractSchemaMigrator.java:562)
at org.hibernate.tool.schema.internal.AbstractSchemaMigrator.applySqlStrings(AbstractSchemaMigrator.java:507)
at org.hibernate.tool.schema.internal.AbstractSchemaMigrator.createTable(AbstractSchemaMigrator.java:277)
at org.hibernate.tool.schema.internal.GroupedSchemaMigratorImpl.performTablesMigration(GroupedSchemaMigratorImpl.java:71)
at org.hibernate.tool.schema.internal.AbstractSchemaMigrator.performMigration(AbstractSchemaMigrator.java:207)
at org.hibernate.tool.schema.internal.AbstractSchemaMigrator.doMigration(AbstractSchemaMigrator.java:114)
at org.hibernate.tool.schema.spi.SchemaManagementToolCoordinator.performDatabaseAction(SchemaManagementToolCoordinator.java:184)
at org.hibernate.tool.schema.spi.SchemaManagementToolCoordinator.process(SchemaManagementToolCoordinator.java:73)
at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:318)
What am I doing wrong please?
The spring.jpa.generate-ddl: true is forcing spring data JPA to recreate the tables. Just set this to false or delete the key as the default is false. This should fix the Problem. Docs

Database migration cause errors for H2 database

When i run tests the following migration file causes an
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "ALTER TABLE ACCOUNT
ADD IS_PROVIDER_ROOT_ACCOUNT VARCHAR(1) NOT NULL,[*]
ADD PROVIDER_ORGANISATION_ID VARCHAR(255) NULL"; SQL statement:
alter table account
add is_provider_root_account varchar(1) not null,
add provider_organisation_id varchar(255) null [42000-200]
error
alter table account
add is_provider_root_account varchar(1) not null,
add provider_organisation_id varchar(255) null;
The thing is, if I remove any one of the adds there are no errors. So what can I do here?
My testing configuration file:
spring.datasource.url=jdbc:h2:mem:testdb:MODE=MYSQL
spring.datasource.username=sa
spring.datasource.password=secret
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.h2.console.enabled=true
Looking at the H2 syntax (note parentheses), when adding multiple columns, one should do:
alter table account
add (
is_provider_root_account varchar(1) not null,
provider_organisation_id varchar(255) null
);

Understanding SLQ syntax error in Flyway initial migration

I am configuring Flyway for a brand new Spring Boot project. This is the initial migration provided by DBA.
The application works in SQL Server, however I am struggling to make it work on H2 for in-memory testing purposes. The DBA didn't provide (and maybe is not going to provide) a dialect-neutral version of the DDL.
I tried to use H2's SQL Server mode
application.yaml
spring:
datasource:
url: jdbc:h2:mem:almc-be;Mode=MSSQLServer
flyway:
check-location: true
enabled: true
encoding: utf-8
locations: classpath:/flyway/${spring.datasource.platform:sqlserver}
create-schemas: true
default-schema: dbo
schemas: dbo
Beginning of the script
-- Create tables section -------------------------------------------------
-- Table dbo.Users
CREATE TABLE [dbo].[Users]
(
[UserId] Int NOT NULL,
[UserName] Nvarchar(50) NOT NULL,
[UserSurname] Nvarchar(50) NOT NULL,
[UserEmail] Nvarchar(100) NOT NULL
)
go
-- Add keys for table dbo.Users
On startup, I get the following error
2021-07-22 15:48:43.189 WARN 6316 --- [ restartedMain] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException:
Migration V202107221432__DDL.sql failed
---------------------------------------
SQL State : 42000
Error Code : 42000
Message : Syntax error in SQL statement "CREATE TABLE ""dbo"".""Users""
(
""UserId"" INT NOT NULL,
""UserName"" NVARCHAR(50) NOT NULL,
""UserSurname"" NVARCHAR(50) NOT NULL,
""UserEmail"" NVARCHAR(100) NOT NULL
)
GO[*]
ALTER TABLE ""dbo"".""Users"" ADD CONSTRAINT ""PK_Users"" PRIMARY KEY (""UserId"")
GO
Yes, the escape sequence is changed in the logs from the SQL file
What is the cause of the error and how can I fix it without rewriting the script into a dialect-neutral version myself?

Load H2 source data from custom sql with Spring

Having a problem with H2 set up in Spring. I placed all my sql data into one file named data.sql, however when I change it to anything else - it cannot be identified. Any idea how to set up multiple separate files?
Let's say i have a table User and some data inserted, but aiming to have 2 separate files, e.g. user-schema and user-data, and so on - multiple schema files with same number of insert data files.
My current spring.properties looks as follows:
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.H2Dialect
spring.h2.console.enabled=true
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=create-drop
spring.h2.console.path=/h2
My current data.sql looks as follows:
DROP TABLE IF EXISTS User;
CREATE TABLE User
(
id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(20) NOT NULL,
surname VARCHAR(20) NOT NULL,
role VARCHAR(20) NOT NULL,
email VARCHAR(30) NOT NULL
);
INSERT INTO User (name, surname, role, email)
VALUES ('Thor', '', 'admin', 'thor#marvel.com'),
('Hulk', '', 'user', 'hulk#marvel.com'),
('Venom', '', 'user', 'venom#marvel.com'),
('Spider', 'Man', 'user', 'spider-man#marvel.com'),
('Super', 'Man', 'user', 'super-man#marvel.com');
If you want to separate your data input to many files you should provide information to Spring about files location:
spring.datasource.data=classpath*:sql/mock-*.sql
spring.datasource.initialization-mode=always
In my project we keep our .sql files in resource/sql folder and every file name mock-*.sql, like mock-user.sql, mock-role-.sql that why I have wildcard in path. Anyway in spring.datasource.data you have to provide path to file with sql inserts.
spring.datasource.initialization-mode=always tells Spring to always initialize DB from files. You should configure that property since you "create-drop" database each time your tests starts.
Spring documentation about data and schema initialization: https://docs.spring.io/spring-boot/docs/2.1.x/reference/html/howto-database-initialization.html

I can`t see my created tables in HSQLDB, what could be the issue?

My environment is HSQLDB, Maven and spring-boot.
I have created 2 entity POJOs. I do see the CREATE TABLE command under testedb.log file. But when I open Data Source Explorer in Eclipse, I can`t see my tables, albeit I do see all the system tables.
I have looked at this question too, but no vail: Where can I see the HSQL database and tables
Here is my partial pom.xml:
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>2.4.0</version>
<scope>runtime</scope>
</dependency>
Here is my partial application.properties:
# DataSource
#spring.datasource.driverClassName=org.hsqldb.jdbc.JDBCDriver
spring.datasource.url=jdbc:hsqldb:file:resources/db/testedb;DATABASE_TO_UPPER=false
#spring.datasource.url=jdbc:hsqldb:mem:memTestdb
spring.datasource.username=sa
spring.datasource.password=
# Hibernate
spring.jpa.show-sql=true
#spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.hibernate.ddl-auto=create
And below is my HSQLDB created on disk:
HSQLDB folder in my workspace
Here is my partial testedb.script:
SET FILES LOG SIZE 50
CREATE USER SA PASSWORD DIGEST 'd41d8cd98f00b204e9800998ecf8427e'
ALTER USER SA SET LOCAL TRUE
CREATE SCHEMA PUBLIC AUTHORIZATION DBA
SET SCHEMA PUBLIC
CREATE SEQUENCE PUBLIC.HIBERNATE_SEQUENCE AS INTEGER START WITH 1
CREATE MEMORY TABLE PUBLIC.ENQUIRY_HISTORY(ID BIGINT NOT NULL PRIMARY KEY,FROM_AMOUNT DOUBLE NOT NULL,FROM_CURRENCY VARCHAR(255) NOT NULL,QUERY_DATE TIMESTAMP NOT NULL,TO_AMOUNT DOUBLE NOT NULL,TO_CURRENCY VARCHAR(255) NOT NULL,USER_ID BIGINT NOT NULL,VERSION INTEGER NOT NULL)
CREATE MEMORY TABLE PUBLIC.USERS(ID BIGINT NOT NULL PRIMARY KEY,EMAIL VARCHAR(255) NOT NULL,LAST_LOGIN TIMESTAMP NOT NULL,PASSWORD VARCHAR(255) NOT NULL,VERSION VARCHAR(255) NOT NULL)
ALTER SEQUENCE SYSTEM_LOBS.LOB_ID RESTART WITH 1
ALTER SEQUENCE PUBLIC.HIBERNATE_SEQUENCE RESTART WITH 1
SET DATABASE DEFAULT INITIAL SCHEMA PUBLIC
GRANT USAGE ON DOMAIN INFORMATION_SCHEMA.SQL_IDENTIFIER TO PUBLIC
Please see above that the CREATE TABLE contains the word MEMORY even though I have created file DB.
And by testsdb.log:
/*C12*/SET SCHEMA PUBLIC
drop table enquiry_history if exists
drop table users if exists
drop sequence hibernate_sequence if exists
create sequence hibernate_sequence start with 1 increment by 1
create table enquiry_history (id bigint not null, from_amount float not null, from_currency varchar(255) not null, query_date timestamp not null, to_amount float not null, to_currency varchar(255) not null, user_id bigint not null, version integer not null, primary key (id))
create table users (id bigint not null, email varchar(255) not null, last_login timestamp not null, password varchar(255) not null, version varchar(255) not null, primary key (id))
/*C14*/SET SCHEMA PUBLIC
DISCONNECT
/*C17*/SET SCHEMA PUBLIC
And Finally here is my screen shot of Database
Data Source Explorer
Any pointer will be awesome, thanks for your time.
file and mem are in-process modes. For testing/debugging, if you need concurrent access to data from another process, start database in Server mode.
Check various available modes here.
I was able to see the tables using in build HSQLDB interface, now a fancy one but it still works for me.
I used the following [listed in this answer https://stackoverflow.com/a/35240141/8610216]
java -cp /path/to/hsqldb.jar org.hsqldb.util.DatabaseManager
And then specify path your database:
jdbc:hsqldb:file:mydb
There was one more thing that I was doing incorrect; it was the path of the db. The correct path is spring.datasource.url=jdbc:hsqldb:file:src/main/resources/db/userx;DATABASE_TO_UPPER=falsein application.properties.

Resources