I have a spring boot application. And I have configured liquibase to create tables. I have few issues though. The tables are not getting created inside the schema and in the location provided in application.yml. Also, if i change the username from default sa to "root", the spring boot liquibase scripts fail with the error "wrong username or password".
Here is my application.yml file:
debug: false
server:
port: 9110
servlet:
context-path: '/v1/ss-db-service'
tomcat:
remoteip:
protocol-header: x-forwarded-proto
spring:
jpa:
properties:
hibernate:
dialect: org.hibernate.dialect.MySQL5Dialect
hibernate:
ddl-auto: none
show-sql: true
h2:
console:
enabled: true
path: /h2-console
settings:
web-allow-others: true
liquibase:
enabled: true
contexts: dev
user: sa
password:
database-change-log-table: 'db_changeset_log'
database-change-log-lock-table: 'db_changeset_lock'
change-log: classpath:liquibase/changesets/MASTER_changeset.xml
# default-schema: AGENCY_SS_REQUEST
datasource:
agency-ss-request:
url: jdbc:h2:mem:AGENCY_SS_REQUEST;MODE=MYSQL;DB_CLOSE_DELAY=-1;INIT=RUNSCRIPT FROM 'classpath:schema.sql';
username: sa
password:
logging:
level:
'org.hibernate.SQL': INFO
'org.hibernate.type.descriptor.sql': INFO
'liquibase.statement': WARN
'liquibase.resource': WARN
I tried setting the scehma with default-schema. But spring boot application fails to start with "Schema not found error".
This is my schema.sql file:
CREATE SCHEMA IF NOT EXISTS AGENCY_SS_REQUEST;
SET SCHEMA AGENCY_SS_REQUEST;
Here is liquibase configuration files:
Here is my liquibase.properties file: (Though, changing anything in this file does not seem to take effect:
### Maven Liqubase Config File
url=jdbc:h2:mem:AGENCY_SS_REQUEST;MODE=MYSQL;DB_CLOSE_DELAY=-1;INIT=RUNSCRIPT FROM
'classpath:schema.sql';
username=sa
password=
driver=org.h2.Driver
Master_changeset.xml:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-4.0.xsd">
<property name="now" value="now()" dbms="h2"/>
<property name="now" value="now()" dbms="mysql"/>
<property name="floatType" value="float4" dbms="postgresql, h2"/>
<property name="floatType" value="float" dbms="mysql, oracle, mssql, mariadb"/>
<property name="clobType" value="clob" dbms="h2"/>
<property name="clobType" value="clob" dbms="mysql, oracle, mssql, mariadb, postgresql"/>
<property name="uuidType" value="varchar(36)" dbms="h2, mysql, mariadb"/>
<!-- Initial USER_PRODCER_INFO Schema -->
<include file="./REQUEST_TABLE_20230116_changeset.xml" relativeToChangelogFile="true"/>
</databaseChangeLog>
REQUEST_INFO_20210203_01.xml:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xmlns:pro="http://www.liquibase.org/xml/ns/pro"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd
http://www.liquibase.org/xml/ns/pro http://www.liquibase.org/xml/ns/pro/liquibase-pro-4.0.xsd
http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-4.0.xsd">
<changeSet id="REQUEST_INFO_20210203_01" author="snagesh">
<createTable tableName="ss_request">
<column name="ss_request_id" type="VARCHAR(36)">
<constraints nullable="false" primaryKey="true"/>
</column>
--- there are more fields
</createTable>
</changeSet>
</databaseChangeLog>
with the above configuration, if i run the application, i see in the spring boot console that liquibase has created tables in a random memory location. if i open that in h2 console i see the below things:
Here, tables are created outside the schema. also in some random memory location.
If I access the h2 console i have configured in application.yml, just the h2 tables are there. none of my tables are created in the location.
This is my entity file:
#Data
#Entity
#NoArgsConstructor
#AllArgsConstructor
#Table(name = "ss_request", schema = "AGENCY_SS_PORTAL")
public class SSRequest {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "ss_request_id", nullable = false)
private Integer ssRequestId;
--- more fields
My questions are:
why can't i see my tables in jdbc:h2:mem:AGENCY-SS-PORTAL this location, but in a random location(I get this random location from spring boot application logs)?
Why are not my tables getting created inside the schema. and why does the application fail to start if i enforce the schema creation with default-schema: AGENCY_SS_REQUEST
why does the application fail with username/password error, if i change the password from sa to root. is liquibase looking at any other configuration?
Note: I am doing this project on intelliJ. so if any extra configuration is needed, let me know
Related
I have a spring boot v2.7.1 project running with liquibase and hibernate.
When i execute the command gradle diffChangeLog liquibase is generating tables with _AUD suffix and then hibernate is blowing an error
ERROR: relation "comment_aud" does not exist.
here are the relative information from my configuration files:
build.gradle
plugins {
id 'org.springframework.boot' version "${springBootVersion}" #2.7.1
id 'io.spring.dependency-management' version '1.0.11.RELEASE'
id 'java'
id "jacoco"
id 'org.liquibase.gradle' version '2.1.1'
}
apply plugin: 'org.liquibase.gradle'
depedencies {
implementation "org.liquibase:liquibase-core"
liquibaseRuntime 'org.liquibase:liquibase-gradle-plugin:2.1.1'
liquibaseRuntime 'org.liquibase:liquibase-core:4.8.0'
liquibaseRuntime 'info.picocli:picocli:4.6.3'
liquibaseRuntime 'org.liquibase.ext:liquibase-hibernate5:4.13.0'
liquibaseRuntime "org.postgresql:postgresql"
liquibaseRuntime 'org.springframework.boot:spring-boot-starter-data-jpa'
}
liquibase {
activities {
main {
driver 'org.postgresql.Driver'
url 'jdbc:postgresql://localhost:5432/database'
username 'admin'
password 'admin'
changeLogFile "src/main/resources/liquibase/changelogs/" + new Date().format("yyyy_MM_dd-HH_mm_ss") + "-changelog.xml"
referenceUrl 'hibernate:spring:mypackagedir.entity' +
'?dialect=org.hibernate.dialect.PostgreSQLDialect' +
'&implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy' +
'&?physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy'
defaultSchemaName ""
classpath "build/classes/java/main"
logLevel "debug"
}
}
runList = 'main'
}
application.yml
spring:
datasource:
driver-class-name: org.postgresql.Driver
password: ${database.password}
url: ${database.host}
username: ${database.username}
jpa:
hibernate:
ddl-auto: none
properties:
org:
hibernate:
envers:
global_with_modified_flag: true
store_data_at_delete: true
audit_table_suffix: _aud
database-platform: org.hibernate.dialect.PostgreSQL92Dialect
show-sql: true
liquibase:
enabled: true
part of the generated changelog after i do diffChangeLog
...
<createTable tableName="comment_AUD">
<column name="REV" type="INTEGER">
<constraints nullable="false" primaryKey="true" primaryKeyName="comment_AUDPK"/>
</column>
<column name="REVTYPE" type="SMALLINT"/>
...
as you can see liquibase is generating the audit table with _AUD although i set it to _aud in hibernate properties
i tried the following:
adding and removing all or one of naming_strategies in build.gradle file
&implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy&?physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
adding removing naming strategies in application.yml properties:
jpa:
hibernate:
naming:
implicit-strategy: org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
physical-strategy: org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
changing the suffix to audit_table_suffix:_AUD
but nothing helps! i am running in circles now and can't find a solution for this. can someone tell me how to fix this please?
Thanks in advance
This is my config:
#Bean
#QuartzDataSource
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource quartzDataSource() {
return DataSourceBuilder.create().build();
}
and this is my app.yml:
datasource:
url: my-url
jdbcUrl: ${spring.datasource.url}
username: 'root'
password: 'root'
...
quartz:
job-store-type: jdbc
jdbc:
initialize-schema: always
wait-for-jobs-to-complete-on-shutdown: true
properties:
org:
quartz:
dataSource:
quartz-data-source:
provider: hikaricp
driver: com.mysql.cj.jdbc.Driver
URL: ${spring.datasource.url}
user: ${spring.datasource.username}
password: ${spring.datasource.password}
maximumPoolSize: 5
connectionTestQuery: SELECT 1
validationTimeout: 5000
idleTimeout: 1
scheduler:
instanceId: AUTO
instanceName: my-project-scheduler
jobStore:
class: org.quartz.impl.jdbcjobstore.JobStoreTX
driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
useProperties: false
misfireThreshold: 60000
clusterCheckinInterval: 30000
isClustered: true
dataSource: quartz-data-source
threadPool:
class: org.quartz.simpl.SimpleThreadPool
threadCount: 1
threadPriority: 5
threadsInheritContextClassLoaderOfInitializingThread: true
My question:
If I set initialize-schema: always then the qrtz tables are created on each application startup.
On the other side, if I set initialize-schema: never then I get an error on the first startup that the qrt tables are missing.
Is there a way to configure it to initialize the qrtz tables only if they do not exist?
You are gonna need a migration tool to handle the database creation.
Spring Boot provides two options: Flyway and LiquidBase.
Choose one, create migration scripts and you are up and running.
I personally like the Flyway approach.
You just add implementation 'org.flywaydb:flyway-core' to your build.gradle file (or the maven alternative).
Then add this to your application.yml
spring:
flyway:
enabled: true
baseline-on-migrate: true
Then create db/migration folder in resources folder and put in your migration scripts eg. V1_0_0__db_init.sql (flyway has its own naming convention).
To get the create SQL scripts I recommend that you export them from running database.
Also do not forget to change the spring.jpa.hibernate.ddl-auto to validate.
I have created all the models for my spring boot application and successfully specified the relation between them as per business logic. In my properties file, I set the hibernate auto-ddl option as create to generate all the tables as per the relationship between the entities. I am also using liquibase for database migration.
The problem is that, from the logs I can see that hibernate is altering the table before creating it and therefore throwing the run time exception saying "Table not found". Why it is altering the table before creating it? How to solve this issue?
Any help would be highly appreciable. Thanks in advance.
Some logs example
Error executing DDL "alter table application drop foreign key FKrtuepaxepo3o6x0pkn0w62ucg" via JDBC Statement.
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "APPLICATION" not found; SQL statement:
alter table application drop foreign key FKrtuepaxepo3o6x0pkn0w62ucg
.......
.......
.......
Hibernate: drop table if exists application
create table application (id bigint not null auto_increment
alter table application add constraint UK_5jl5nuoh207t0japuutb4avd4 unique (application_name)
Why it is expecting table name as "APPLICATION" before creating "application"
#Table(name = "application")
#EqualsAndHashCode(onlyExplicitlyIncluded = true)
public class Application extends AbstractAuditingEntity implements Serializable {
private static final long serialVersionUID = 1L;
}
I am using H2 database in the development and MYSQL in production. But, previously, I was using the "org.hibernate.dialect.MySQL5InnoDBDialect" dialect in "application-dev.yml". Changing it to h2 dialect(org.hibernate.dialect.H2Dialect) solved the problem.
But,what should be the proper dialect in application-prod.yml? I am using MYSQL in production. Current dialect I am using here is "org.hibernate.dialect.MySQL5InnoDBDialect". This will further create problem in production.
application-prod.yml
datasource:
# type: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://localhost:3306/appscoredb
username:
password:
hikari:
poolName: Hikari
auto-commit: false
# h2:
# console:
# enabled: true
# settings:
# web-allow-others: true
jpa:
database-platform: org.hibernate.dialect.MySQL5InnoDBDialect
database: mysql
hibernate:
ddl-auto: none
show-sql: true
properties:
hibernate.id.new_generator_mappings: true
hibernate.connection.provider_disables_autocommit: true
hibernate.cache.use_second_level_cache: false
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: false
liquibase:
# Remove 'faker' if you do not want the sample data to be loaded automatically
contexts: prod, faker
enabled: false
It is solved by using this dialect: org.hibernate.dialect.MySQL5Dialect
I hope it could help someone in near future.
I've created this application-local.json into src/main/resources:
{
"spring": {
"datasource": {
"url": "jdbc:postgresql://xxx:yyy/db",
"username": "xxx",
"password": "xxx",
"driverClassName": "org.postgresql.Driver"
},
"profiles": {
"active": "local"
}
}
}
By other hand, apliication.yml:
spring:
jpa:
generate-ddl: false
show-sql: true
properties:
hibernate:
format_sql: true
jdbc:
lob:
non_contextual_creation: true
profiles:
active: local
management:
security:
enabled: false
endpoints:
web:
exposure:
include: '*'
---
spring:
profiles: local
server:
port: 8092
Currently, I'm getting this message:
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to auto-configure a DataSource: 'spring.datasource.url' is not specified and no embedded datasource could be auto-configured.
Reason: Failed to determine a suitable driver class
Action:
Consider the following:
If you want an embedded database (H2, HSQL or Derby), please put it on the classpath.
If you have database settings to be loaded from a particular profile you may need to activate it (no profiles are currently active).
When Spring application runs, it load properties from value of application.yml->spring->profiles->active. Spring supports only yml and .properties file as a source.
So, in your case Spring will look for application-local.yml or application-local.properties to read profile specific property.
But here, you have defined property file as a application-local.json and that a reason why spring is not reading values and you are getting exception.
Solution
Create application-local.yml or application-local.properties and paste your content and try. It should work.
Here is sample DB configuration.
spring.datasource.url=jdbc:mysql://localhost:3306/_schema name_
spring.datasource.username=_username_
spring.datasource.password=_password_
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
spring.jpa.show-sql = true
logging.level.org.hibernate.SQL=debug
I'm trying to test my Spring Boot application with an embedded database h2. As for dev and prod, I will be using a MySQL database.
I would like to have different application.yml and schema.sql file for each mode.
The project structure is:
src
--main
----resources
------application.yml
------schema.sql
--test
----resources
------application-test.yml
------schema-test.sql
This is my RespositoryTest :
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest
#DataJpaTest
public class IUserRepositoryTest {
#Autowired
private TestEntityManager entityManager;
#Autowired
private IUserRepository userRepository;
#Test
public void Should_ReturnEmployee_ForExistingEmail() {
User oneUser=new User("john","doe","example#email.com");
entityManager.persist(oneUser);
List<User> userList=userRepository.findUserByEmail("example#email.com");
assertThat(userList).isNotEmpty();
assertThat(userList.get(0)).isNotNull();
assertThat(userList.get(0).getEmail()).isEqualTo("example#email.com");
}
This is my test/resources/application-test.yml:
spring:
profiles: test
datasource:
url: jdbc:h2:mem:test;INIT=create schema IF NOT EXISTS mydb;DB_CLOSE_DELAY=-1
platform: h2
username: sa
password:
driverClassName: org.h2.Driver
jpa:
hibernate:
ddl-auto: create-drop
properties:
hibernate:
default-schema: mydb
dialect: org.hibernate.dialect.H2Dialect
This is my test/resources/schema-test.sql:
CREATE SCHEMA IF NOT EXISTS MYDB
As for my main/resources/application.yml:
logging:
level:
org.springframework.web: DEBUG
org:
hibernate:
SQL: DEBUG
spring:
jpa:
hibernate:
ddl-auto: update
properties:
hibernate:
dialect: org.hibernate.dialect.MySQL5Dialect
datasource:
url: jdbc:mysql://localhost:3306/mydb
username: root
password: toor
database:
driverClassName: com.mysql.jdbc.Driver
When I run my app as a spring boot one, the main application.yml is used and all is good, but when I run my tests, I get this error:
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL via JDBC Statement
Caused by: org.h2.jdbc.JdbcSQLException: Schema "MYDB" not found; SQL statement
Which causes all my tests to fail.
When I try to use this project structure:
src
--main
----resources
------application.yml
------schema.sql
--test
----resources
------application.yml
------schema.sql
The test succed but when I run my app as a spring boot, the test/resources/application.yml is the one being used instead of the main one.
Your tests are running with the "default" profile so it will only load the "default" configurations with no suffix (i.e. -test).
Try adding #ActiveProfiles("test") to your test class to enable the test profile.