I have an existing spring boot application and i need to connect to Aurora DB using liquibase and create tables . I have added all the required steps as below but when application is getting deployed not seeing any logs specific to liquibase and the changeSet are not getting executed . Please help do debug the issue .
1.db.changelog-master.xml (created under resources/db/changelog )
<preConditions>
<not>
<tableExists tableName="table1"/>
</not>
</preConditions>
<changeSet id="1" author="name">
<createTable tableName="category">
<column name="id" type="int" autoIncrement="true">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(250)">
<constraints unique="true" nullable="false"/>
</column>
</createTable>
</changeSet>
</databaseChangeLog>
application.properties
#Liquibase configuration
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://aws-aurora-conn-url:3306/database-name
spring.datasource.username=usename
spring.datasource.password=password
spring.liquibase.change-log=classpath:db/changelog/db.changelog-master.xml
spring.liquibase.enabled=true
build.gradle added these dependencies
implementation 'org.liquibase:liquibase-core'
implementation 'mysql:mysql-connector-java'
Thanks in advance
Issue was due to precondition check failure. Added the condition check in db.changelog-master.xml and was able to connect it . this will not fail the application instance instead its logs it and proceed
<preConditions onFail="WARN">
<not>
<tableExists tableName="table1"/>
</not>
Related
I have a database with multiple schemas with same tables.
Inorder to update a particular schema "tenant1" below syntax can be used
<changeSet id="1660156908997-1" author="manu">
<addColumn schemaName="tenant1" tableName="question">
<column name="weightage" type="INT">
<constraints nullable="false"/>
</column>
</addColumn>
</changeSet>
Inorder to update two schemas viz., "tenant1" and "tenant2", below command can be used,
<changeSet id="1660156908997-1" author="manu">
<addColumn schemaName="tenant1" tableName="question">
<column name="weightage" type="INT">
<constraints nullable="false"/>
</column>
</addColumn>
<addColumn schemaName="tenant2" tableName="question">
<column name="weightage" type="INT">
<constraints nullable="false"/>
</column>
</addColumn>
</changeSet
But can we automate this and update all schemas with a single entry inside changeset?
Do you know about changelog parameters?
What I would do is to parameterise the changes and then for each of your tenant definitions in your application execute the same changelog but with different parameters.
ChangeLog could look like this:
<changeSet id="1660156908997-1-${tenant}" author="manu">
<addColumn schemaName="${tenant}" tableName="question">
<column name="weightage" type="INT">
<constraints nullable="false"/>
</column>
</addColumn>
</changeSet>
I am on a SpringBoot project that is using liquibase-core 4.2.0 and I would like to insert fake data for my local executions using the spring profile 'dev'.
I added a liquibase context to each of my profiles (example of application-dev.yaml) :
spring:
profiles:
active: dev
liquibase:
contexts: dev
And added a context in the <include /> of my changelog file :
<?xml version="1.0" encoding="utf-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<property name="now" value="now()" dbms="h2"/>
<property name="now" value="GETDATE()" dbms="mssql"/>
<property name="floatType" value="float4" dbms="h2"/>
<property name="floatType" value="float" dbms="mssql"/>
<property name="clobType" value="clob" dbms="h2, mssql"/>
<property name="uuidType" value="uuid" dbms="h2, mssql"/>
<include file="changelog/1.0.0/financial_security/schema.xml" relativeToChangelogFile="true"/>
<include file="changelog/1.0.0/financial_security/local_data_for_dev.xml" relativeToChangelogFile="true" context="dev"/>
<include file="changelog/1.0.0/financial_security/tag-1.0.xml" relativeToChangelogFile="true"/>
</databaseChangeLog>
My file local_data_for_dev.xml looks like this :
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<changeSet id="20200422091200-fakedata" author="me">
<!-- TABLE UC_250_COUNTRIES -->
<loadData
file="db/changelog/1.0.0/financial_security/fake-data/uc_250_countries.csv"
separator=";"
tableName="uc_250_countries">
<column name="id" type="${uuidType}"/>
<column name="country_code" type="string"/>
<column name="label" type="string"/>
<column name="value" type="string"/>
<column name="created_date" type="datetime"/>
<column name="updated_date" type="datetime"/>
</loadData>
[....]
</changeSet>
</databaseChangeLog>
It's working really well, but for test purpose I tried changing my changelog to have another context :
<include file="changelog/1.0.0/financial_security/local_data_for_dev.xml" relativeToChangelogFile="true" context="prod"/>
And (after cleaning up the database) when I restarted my application with 'dev' profile, it still inserted the fake data. Why ?
From last note here
Starting with Liquibase 3.5, you can specify a context attribute in <include> or <includeAll> tags. If specified, the given context is added to all changesets in the included file(s).
So try to remove context from your include and it should work.
Seems like liquibase.contexts is not a global variable, but it is bound to the changelog it associate with. I have 2 changelogs for 2 database in my application.yml:
liquibase:
change-log: classpath:some_changelog.xml
second-liquibase:
change-log: classpath:some_other_changelog.xml
The changelog that inserts fake data is from second-liquibase, so I need to set the variable second-liquibase.contexts in my application-dev.yml:
liquibase:
contexts: somecontext
second-liquibase:
contexts: dev
In first change set table tb_transformation is already created. Below is the 4th changeset where i insert extra column to table when precondition passes. But its not executing the changeset. When i remove precondition and execute its inserting successfully.
<changeSet id="2020-03-004-add-columns-to-tb_transformation" author="TAAS">
<preConditions onFail="CONTINUE">
<tableExists tableName="TB_TRANSFORMATION"/>
</preConditions>
<addColumn tableName="TB_TRANSFORMATION">
<column name="MARKET_INFRASTRUCTURE" type="varchar(255)">
<constraints nullable="false"/>
</column>
</addColumn>
Are there some environments where the table might not exist? If not, and everything is being 'controlled' by Liquibase, then you should just be able to remove the precondition.
I am trying to Creating New Liquibase Projects with Oracle 12c database.My oracle database is located on a remote server.This is my changelog for my project and it is saved as dbchangelog.xml on my machine where Liquibase is running
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<changeSet id="1" author="bob">
<createTable tableName="department">
<column name="id" type="int">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(50)">
<constraints nullable="false"/>
</column>
<column name="active" type="boolean"
defaultValueBoolean="true"/>
</createTable>
</changeSet>
</databaseChangeLog>
The liquibase.properties file is saved as follows
changeLogFile: /home/dbchangelog.xml
url : jdbc:oracle:thin:#<oracle_db_ip>:1521/ORCLDB
username : <user>
password : <password>
driver: oracle.jdbc.OracleDriver
classpath: /home/ojdbc6.jar
Do i need to keep tnsname.ora on my machine where liquibase is running? if yes where should i keep it ??
Liquibase uses jdbc API, so you don't need tnsname.ora, neither oracle client on your machine. ORA-12514 means that ORCLDB is probably wrong.
You can find the SID using this query:
select instance from v$thread
I've created schema and populated it via Maven liquibase plugin:
<plugin>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<version>2.0.5</version>
<configuration>
<propertyFile>src/main/resources/db/config/db.config.properties</propertyFile>
<changeLogFile>src/main/resources/db/changelog/db.changelog-master.xml</changeLogFile>
</configuration>
</plugin>
Properties file:
driver: org.hsqldb.jdbcDriver
#HSQLDB Embedded in file
url: jdbc:hsqldb:file:src/main/resources/db/hsqldb/dataFile
username: SA
password:
As I see in the output when invoke mvn liquibase:update:
[INFO] Executing on Database: jdbc:hsqldb:file:src/main/resources/db/hsqldb/dataFile
INFO 24.04.13 10:00:liquibase: Successfully acquired change log lock
INFO 24.04.13 10:00:liquibase: Creating database history table with name: DATABASECHANGELOG
INFO 24.04.13 10:00:liquibase: Reading from DATABASECHANGELOG
INFO 24.04.13 10:00:liquibase: Reading from DATABASECHANGELOG
INFO 24.04.13 10:00:liquibase: ChangeSet src/main/resources/db/changelog/db.changelog-1.0.xml::1::sav ran successfully in 7ms
INFO 24.04.13 10:00:liquibase: ChangeSet src/main/resources/db/changelog/db.changelog-1.0.xml::2::sav ran successfully in 3ms
INFO 24.04.13 10:00:liquibase: Successfully released change log lock
INFO 24.04.13 10:00:liquibase: Successfully released change log lock
db.changelog-master.xml contains:
<include file="src/main/resources/db/changelog/db.changelog-1.0.xml"/>
db.changelog-1.0.xml contains:
<changeSet id="1" author="sav">
<createTable tableName="testTable">
<column name="id" type="int">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(50)">
<constraints nullable="false"/>
</column>
<column name="active" type="boolean" defaultValueBoolean="true"/>
</createTable>
</changeSet>
<changeSet id="2" author="sav">
<insert tableName="testTable">
<column name="id" value="1"/>
<column name="name" value="First String"/>
</insert>
<insert tableName="testTable">
<column name="id" value="2"/>
<column name="name" value="Second String"/>
<column name="active" value="false"/>
</insert>
</changeSet>
It seems everything is OK. Now i'm going to src/main/resources/db/hsqldb/ folder and see three files:
dataFile.log
dataFile.properties
dataFile.script
But I don't see a CREATE TABLE testTable DDL statement in the dataFile.script.
Next in Intelli IDEA I configure datasource plugin (set jdbc hsql driver, url: jdbc:hsqldb:file:/src/main/resources/db/hsqldb/dataFile, user: sa ). Connect it, invoke query:
SELECT * FROM INFORMATION_SCHEMA.SYSTEM_TABLES;
I cannot see the table I try to create.
The value of hsqldb_type field for each table - is MEMORY. I expect it to be FILE of something similar.
Any ideas?:)
PS:
1. Maven repository returns HSQLDB as the first search result, and its last version is 1.8.0.10. Actually I had to use HSQLDB DATABASE with its 2.2.9 version. It solved the problem of table creation.
2. I had to use an absolute path to the file AND the ';ifexists=true' property to connect to the existing database in IDEA datasource plugin. As a result my url connection string in the property file differ from the same used in the plugin.
You need to be careful about the use of file paths:
This is a relative path:
driver: org.hsqldb.jdbcDriver
#HSQLDB Embedded in file
url: jdbc:hsqldb:file:src/main/resources/db/hsqldb/dataFile
This is an absolute path within the current drive:
Next in Intelli IDEA I configure datasource plugin (set jdbc hsql driver, url: jdbc:hsqldb:file:/src/main/resources/db/hsqldb/dataFile, user: sa )
Try using absolute paths.
Apart from that, in your attempt to check things in an existing database, connect explicitly requiring the database to exist by adding ;ifexists=true to the connection URL.
As we are not sure Liquibase shuts down the database correctly or not, you can add a property to the Liquibase connection URL to ensure data is written fully ;hsqldb.write_delay=false. We are assuming you are using HSQLDB 2.x for this property.
With HSQLDB 1.8.x, liquibase <= 3.2.0 never read tables on database.
This is because catalog should be set null when retrieve JDBC metadata.
We could not force the catalog to be null, because the method AbstractJdbcDatabase.correctSchema() should not set catalog=schema, if supportCatalogs() return false.
I will post a issue for this.