apply migration in all schema that i have using liquibase - spring-boot

I developed a website using spring boot in this application I'm using architecture multi tenant to manage my database. I want to use Liquibase as a DB migration tool. The problem is that when i do migration the new modification(modification means by add new columns to different tables and also add new tables) is only apply in schema public and doesn't apply on the others sachems , what i want , when i do migration i want the new modification apply on all sachems
ps : i'm using hibernate to create new sachems

Liquibase allows dynamic substitution of properties in changelog files. We can configure multiple properties inside a file and then use them wherever required. In your case, we can configure properties "schema1", "schema2" with some value and then use it in changelog file using ${schema1} or ${schema2} syntax as per requirement.
In liquibase.properties file, we will configure these properties as follows:
schema1=ABC
schema2=PQR
Liquibase assigns or prioritizes value for configured property in below order:
As an attribute passed to your liquibase runner.
As a JVM sytem property
As an environment variable
As a CLI attribute if you are running liquibase through command line
In liquibase.properties file
In the parameters block (property element of the DATABASECHANGELOG table)
You can do it as below example code snippet:
1. Adding a column to some table in schema ABC:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<changeSet author="authorName" id="some-unique-id" dbms="${dbType}" context="some-context">
<sql endDelimiter=";" splitStatements="true" stripComments="true">
**My SQL query/ transactional logic goes here**
ALTER TABLE "${schema1}"."TableName" ADD COLUMN COLUMNNAME DATATYPE;
</sql>
</changeSet>
</databaseChangeLog>
2. Creating a table in PQR schema:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<changeSet author="authorName" id="some_unique_id" dbms="${dbType}" context="some_context">
<createTable tableName="TableName" schemaName="${schema2}">
<column name="id" type="VARCHAR(200)" />
<column name="name" type="VARCHAR(255)"/>
</createTable>
</changeSet>
</databaseChangeLog>
Note: above example is using 2 properties (schema1 and schema2). You can use only even more than that.
If you need help with creating "liquibase.properties" file, visit this link
Cheers!

Related

Liquibase migration with H2 and Spring boot

I use Spring boot and H2 DB. I want to include Liquibase in my app. I have have multiple modules app. There are entity, dao, service, controller, etc modules. I created Liquibase files in Dao module.
This is application.property:
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_ON_EXIT=FALSE
spring.datasource.username=sa
spring.datasource.password=password
spring.liquibase.change-log=classpath:db/migration/master.xml
This is liquibase.property:
classpath=
changeLogFile=db/migration/master.xml
url=jdbc:h2:mem:testdb;DB_CLOSE_ON_EXIT=FALSE
username=sa
password=password
driver=org.h2.Driver
And here is master.xml:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
https://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">
<include file="v-1-0-0/changelog-project-v-1.0.0-cumulative.xml"/>
<include file="v-1-0-1/changelog-project-v-1.0.1-cumulative.xml"/>
</databaseChangeLog>
And I have an error:
Invocation of init method failed; nested exception is liquibase.exception.ChangeLogParseException: liquibase.exception.SetupException: The file v-1-0-0/changelog-project-v-1.0.0-cumulative.xml was not found in
- Spring resources
But, I think that I don't need to specify files form master.xml in properties.
Please, help.
enter image description here
Looks like the path to the changelog-project-v-1.0.0-cumulative.xml is incorrect. Try using relativeToChangelogFile="true"
In master.xml put the following:
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
https://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">
<include file="v-1-0-0/changelog-project-v-1.0.0-cumulative.xml" relativeToChangelogFile="true"/>
<include file="v-1-0-1/changelog-project-v-1.0.1-cumulative.xml" relativeToChangelogFile="true"/>
</databaseChangeLog>

How do I use liquibase contexts to insert fake data on only authorized profiles?

I am on a SpringBoot project that is using liquibase-core 4.2.0 and I would like to insert fake data for my local executions using the spring profile 'dev'.
I added a liquibase context to each of my profiles (example of application-dev.yaml) :
spring:
profiles:
active: dev
liquibase:
contexts: dev
And added a context in the <include /> of my changelog file :
<?xml version="1.0" encoding="utf-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<property name="now" value="now()" dbms="h2"/>
<property name="now" value="GETDATE()" dbms="mssql"/>
<property name="floatType" value="float4" dbms="h2"/>
<property name="floatType" value="float" dbms="mssql"/>
<property name="clobType" value="clob" dbms="h2, mssql"/>
<property name="uuidType" value="uuid" dbms="h2, mssql"/>
<include file="changelog/1.0.0/financial_security/schema.xml" relativeToChangelogFile="true"/>
<include file="changelog/1.0.0/financial_security/local_data_for_dev.xml" relativeToChangelogFile="true" context="dev"/>
<include file="changelog/1.0.0/financial_security/tag-1.0.xml" relativeToChangelogFile="true"/>
</databaseChangeLog>
My file local_data_for_dev.xml looks like this :
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
<changeSet id="20200422091200-fakedata" author="me">
<!-- TABLE UC_250_COUNTRIES -->
<loadData
file="db/changelog/1.0.0/financial_security/fake-data/uc_250_countries.csv"
separator=";"
tableName="uc_250_countries">
<column name="id" type="${uuidType}"/>
<column name="country_code" type="string"/>
<column name="label" type="string"/>
<column name="value" type="string"/>
<column name="created_date" type="datetime"/>
<column name="updated_date" type="datetime"/>
</loadData>
[....]
</changeSet>
</databaseChangeLog>
It's working really well, but for test purpose I tried changing my changelog to have another context :
<include file="changelog/1.0.0/financial_security/local_data_for_dev.xml" relativeToChangelogFile="true" context="prod"/>
And (after cleaning up the database) when I restarted my application with 'dev' profile, it still inserted the fake data. Why ?
From last note here
Starting with Liquibase 3.5, you can specify a context attribute in <include> or <includeAll> tags. If specified, the given context is added to all changesets in the included file(s).
So try to remove context from your include and it should work.
Seems like liquibase.contexts is not a global variable, but it is bound to the changelog it associate with. I have 2 changelogs for 2 database in my application.yml:
liquibase:
change-log: classpath:some_changelog.xml
second-liquibase:
change-log: classpath:some_other_changelog.xml
The changelog that inserts fake data is from second-liquibase, so I need to set the variable second-liquibase.contexts in my application-dev.yml:
liquibase:
contexts: somecontext
second-liquibase:
contexts: dev

Simplified and Traditional Chinese characters are stored as ? in the Sybase database using Liquibase in Spring Boot project

I am trying to save Chinese characters in Sybase database with Liquibase, but ended up getting "?" for each of the characters.
I referred this
but it didn't help.
I am using the 3.6.1 version of liquibase-core (also tried with 3.6.3)
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
<version>3.6.1</version>
</dependency>
Here are the contents of CREATE and INSERT scripts
CREATE Table:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd
http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd" >
<changeSet author="me" id="1" >
<createTable tableName="person" >
<column name="name" type="unitext" encoding="utf8" >
</column>
<column name="address" type="VARCHAR(255)" />
</createTable>
<rollback>
<dropTable tableName="person"/>
</rollback>
</changeSet>
</databaseChangeLog>
INSERT Query:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd
http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">
<changeSet author="me" id="2" runOnChange="true" >
<insert tableName="person" >
<column name="name" value="汉字" encoding="utf8"/>
<column name="address" value="DNI"/>
</insert>
</changeSet>
And when I start the Spring Boot application, the scripts get executed. And I get "??" in place of "汉字" when I query the table.
As in the code, I have already put encoding=utf8. Not sure what is missing.
Sybase version: ASE 15.
Any help is appreciated.
And the solution lies in the DB connection params.
jdbc:sybase:host:port?useUnicode=true&CHARSET=utf8
Apart from the fact that you did not specify which Sybase database type you are working with (there are 3 different Sybase database products), what you are showing is the client side of things. What is equally important is whether the server is set up to support Unicode characters (various ways of doing that). That is what you need to find out.

how set failOnError=“false” in Liquibase maven?

how can i set failOnError=“false” for all changeSet when generate changeLog with maven Liquibase:Diff
<changeSet author="aliakbarazizkhani (generated)" id="1468579261546-1" failOnError="false">
<addColumn tableName="core_feedback">
<column name="solveuser" type="bytea"/>
</addColumn>
</changeSet>
This is not an option you can set on the command line. You would need to manually edit the changelog after it was generated to add the failOnError attribute to the appropriate change sets. It would be inadvisiable to apply that to all generated change sets.

Liquibase not picking up seed data with Spring Boot

I am using Spring-Boot 1.2.1, and Liquibase to create both H2 (testing) and PostgreSQL (QA & Production) databases. I have a couple of tables that I want to seed when the db is created. However, despite trying both dataLoad and sqlFile, nothing is getting inserted. My sql file is just a bunch of insert statements such as:
INSERT INTO state (Name, Code) VALUES('Alabama','AL');
INSERT INTO state (Name, Code) VALUES('Alaska','AK');
Here is my relevant changelog-master.xml:
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.3.xsd"
objectQuotingStrategy="QUOTE_ONLY_RESERVED_WORDS">
...
<changeSet id="3" author="me">
<createTable tableName="STATE">
<column name="code" type="VARCHAR(10)">
<constraints primaryKey="true"/>
</column>
<column name="name" type="VARCHAR(100)"/>
</createTable>
<sqlFile dbms="h2, PostgreSQL"
encoding="utf8"
endDelimiter="\nGO"
path="src/main/resources/db/changelog/data/states.sql"
relativeToChangelogFile="true"
splitStatements="true"
stripComments="true"/>
</changeSet>
Here is my project structure:
When I startup my spring-boot app, I can see that the State table is created, but it has zero rows in it. I also tried taking the out of changeset 3 and using this:
<changeSet id="4" author="me">
<loadData file="data/state.csv" tablename="STATE" schemaName="edentalmanager" relativeToChangelogFile="true">
<column name="name" type="VARCHAR(100)"/>
<column name="code" type="VARCHAR(10)"/>
</loadData>
</changeSet>
The csv file is basically:
Alabama,AL
Alaska,AK
...
I dont' see any messages in the console logs that Liquibase is trying to create or insert the data into the table. Nor do I get any exceptions or error messages.
UPDATE:
If I copy off the state.sql as /resources/data.sql then spring-boot picks up the file and executes the sql just fine. Unfortunately, this means every time I startup, it will try and insert those values again, causing startup exceptions (duplicate key violations) But, rather than rely on a single file, I would prefer Liquibase to execute them as part of the changeset as data needs change.
I think there are two issues in your changeset 3:
The value of the path property directs to a not existing resource. When you define relativeToChangelogFile="true" then Liquibase will look for classpath:/db/changelog/src/main/resources/.../states.sql. The correct path should be path="../data/states.sql".
If the given path is not correct Liquibase should throw an exception. If you didn't get one means Liquibase decided to not execute that part because of other conditions. One of those conditions could be the dbms property. Your changeset should work with a H2 database. It should not work with a PostgreSQL database because of a wrong type name. Try dbms="h2, postgresql" instead.
After those changes I got a filled table based on your project structure.
This is because by default Hibernate drops the schema when initializing. So first Liquibase creates the data and when finished Hibernate drops the tables and recreates them according to the JPA entities you defined.
You can verify this by looking for the log record:
2017-01-19 11:03:48.692 INFO 15161 --- [ main] org.hibernate.tool.hbm2ddl.SchemaExport : HHH000227: Running hbm2ddl schema export
You can tell Hibernate not to do anything with the DB schema by setting this application property:
spring.jpa.hibernate.ddl-auto=none
I ran into this when upgrading to spring boot 2.4.1 from 2.2.0-RELEASE version. The 2.2.0-RELEASE version had liquibase-core:jar:3.8.0 as dependency and it was accurately taking relativeToChangelogFile="true". However the 2.4.1 version has liquibase-core:jar:3.10.3 version which breaks the loadData feature.
The csv file I was loading and the changelog.xml file were in the same class directory in my case src/main/resources/liquibase/version2/changelog.xml and src/main/resources/liquibase/version2/xref_specialty_codes.csv
Here is the snippet from my changelog.xml for loading the data which did not work with the later version of liquibase
<changeSet author="anon"
id="xref_specialty_codes_load_data"
objectQuotingStrategy="LEGACY">
<loadData catalogName="adb"
schemaName="public"
encoding="UTF-8"
file="xref_specialty_codes.csv"
relativeToChangelogFile="true"
separator=","
tableName="xref_specialty_codes"
usePreparedStatements="true">
<column name="id" type="NUMERIC"/>
...
....
</loadData>
</changeSet>
I load a hierarchy of change log files with the top one at src/main/resources/liquibase/changelog.xml with the following layout.
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">
<property name="now" value="now()" dbms="postgresql"/>
<include file="src/main/resources/liquibase/version1/changelog.xml" />
<include file="src/main/resources/liquibase/version2/changelog.xml" />
....
</databaseChangeLog>

Resources