Currently I am on an oracle schema which is maintained with liquibase. The liquibase xmls have some oracle specific usage of queries for dataload and using sequences etc (also some usage specific to oracle).
I would like to move to postgres. Would it be possible to generateChangeLog from oracle schema to execute it against a postgres db (or a changelog that is completely liquibase specific so that it will automatically convert it to target database at runtime)? I do not see an option to specify target database type on generatechangelog. Is there any way to achieve that?
Yes this is possible, although it's easier when you start the changelog with multiple DBMS in mind.
<createSequence> changes should be fine, however if you have any explicit calls to .nextval those would need to be changed.
You can run changeSets conditionally depending on the current DBMS:
<changeSet dbms="postgresql">
...
</changeSet>
or use dbms="oracle". I think (not sure though) the DBMS attribute does not change the MD5 checksum, so you can change your existing change log without it breaking against your existing installations.
How much you need to adjust your current changelog depends on how specific you were when e.g. specifying data types, e.g. you can't use varchar2 for obvious reasons, and changing that will change the MD5 checksum which will break running the changelog against your existing installations.
I would try to change the changelog you have to be as much DBMS independent as possible and then bite the bullet and run it once with clearCheckSums against existing installations.
Related
My understanding of using Liquibase is as follows:
It is a repository for SQL queries migration on to Production database.
If xml is prefered ,it has a changelog.xml file which contains changesets which carry instructions in xml tags to perform actions like create, drop alter table etc.
What I want to achieve and for which I am evaluating if liquibase can serve my purpose.
My deployment process on UAT/Production have EAR and DB scripts execution as the major aspects.
I already have the .sql files which are to be fired in a particular sequence post which the EAR is to be brought up. These .sql files contain DDL as well as DML queries and stored procedures.
My aim is to automate the execution of these .sql scripts which is presently carried out manually.
Since liquibase works on an xml to generate the .sql, can my situation where .sqls are already available be proceeded with. Any inputs will be much appreciated.
Liquibase supports multiple formats. XML is only one of them. You can use SQL format. You will have to just add some comments in your scripts with meta information for Liquibase.
You did not provide much information, but what you want to do is exactly what Liquibase is for - upgrade database structure and data when you release a new version of your Java (?) application.
We are working in a very big organization, many Databases (of many types), many schemas, many users.
Does LB has to work with some Source Control (for locking the files
when many users exist in the organization and using the same DB,
same Schema, etc)?
What is the best practice of working with LB in a very big
organization, many concurrent users?
Can SQLCL general sql format type or just xml format type?
Is there some integration with SQL Developer? I mean, suppose a user
changes an objects via sql developer, what happens then?
We get this type of question all the time, after folks get a handle of how to automate DB changes, next step is typically to add it into an existing CI/CD workflow.
Yes, Liquibase works with any source control. Most users are using
Git. But you can use Git, TFS, SVN, CVS... Once you are up and
running with Liquibase, you just need to make sure that your scripts
are in source control and you are good to go.
Besides 3rd party source control tools, Liquibase has tracking tables called "DATABASECHANGELOG" tables that keep track of the changes applied to your database when using Liquibase deployments.
Here is some more information about getting started and How Liquibase Works. https://www.liquibase.org/get_started/how-lb-works.html
Liquibase has one more table that it uses internally called "DATABASECHANGELOGLOCK" table.
This table was designed to prevent multiple Liquibase users running deployments concurrently - potentially leaving the Database in a bad state. Once the Liquibase deployment (the liquibase update command) is done, the "DATABASECHANGELOGLOCK" will allow the next Liquibase user to deploy.
You can use both SQL and XML formats (or even JSON and YAML formats).
When using SQL, you have a few options:
Best option is to use Formatted SQL changeLogs https://www.liquibase.org/documentation/sql_format.html
https://www.liquibase.org/get_started/quickstart_sql.html
You can use plain raw SQL files referenced from an XML changeLog
https://www.liquibase.org/documentation/changes/sql_file.html
When using XML, can find all the available change types (also called changeSets) available in the following page (on the left of the page)
https://www.liquibase.org/documentation/changes/
XML changeLog are more agnostic and sometimes can be used for different Database platforms when doing migrations. Also, many of the change types in XML have the ability to be rolled back automatically. The reason that this is possible with XML is because Liquibase uses it own built in functions to figure out inverse statements like "create table" to be "drop table".
For each of those changeSets you can find out if they are auto rollback eligible (at the bottom of the page). For example, create table changeSet will be Auto Rollback = yes.
https://www.liquibase.org/documentation/changes/create_table.html
I'd like to have my version only the latest table definition in my repository, (no change sets), and have liquibase figure out which changes are needed when patching my databases. Please take note that I have a very big database schema (1000+ tables) installed in hundreds of customer sites, with different versions each one, and I really don't know which objects each version has
How can I make a liquibase-based installer for my application, given my set of table definitions, and hundreds of databases with about 12 different versions of objects on each one?
To be more specific, I'd like liquibase to compare my table definitions with the production database, and emit the alter table statements required to make the database current with my latest version.
I could contribute code if necessary in order to get this done
Liquibase and tools like it (for example flyway) are primarily designed to support database migrations. A migration is where every change to the DB is tracked so that it can be replayed on target environments thereby keeping them in sync with development (although time-shifted). It's all about keeping your schema under revision control.
Your use case is a little different. If I understand correctly you're trying to retrofit Liquibase onto a series of environments that you are not 100% certain match your application's current schema?
I would only recommend migration tools like liquibase if you intend to use them going forwards. If all you want is a DB diff tool, I would suggest you look elsewhere.
To perform an initial sync then I would suggest you investigate the diffLog command, coupled with changeLogSync command to initialize liquibase on the target DB.
comparing databases and genrating sql script using liquibase
I have my liquibase schema defined initially for PostreSQL. Now, I have to modify the schema file to support Oracle. I have a change-set that has a <sql> tag. It has a query that accesses the pg_catalog table to set a value to the sequence. However, this will not work for Oracle. If I remove it, the Liquibase complains with check-sum validation fail. It complains even if I have an empty <sql> tag or some other query specified within. Within this change-set, I have many other create-table statement, so I cannot just remove oracle from dbms attribute. Is there any way I can suppress this sql from running for Oracle?
The dbms attribute on the changeset is the mechanism designed to handle this problem..... Sounds like you're trying to do too much within one changesest (but I guess you've figured that out)
The checksum validation failure is liquibase's safety mechanism designed to defend the database against someone tampering with the schema files.
How to fix it is to use the clearChecksums option when running liquibase. It tells liquibase to recompute the checksums for the changesets already in the database. This will enable your postgres database instance to accept the alterations to its changesets that you've made for Oracle.
Hey guys,
I need to figure out a way to back up and also migrate our Oracle database from our production schema to the dev schema and the other way around.
We have bunch of config tables that drive how systems on our platform run, and when setting up new systems or doing maintenance, we need to update our config tables. We want to be able to work on the dev schemas and after setting up a system/feature, we want to be able to migrate all those configs to the dev schemas.
I thought of running a procedure where we give the ID of the system (from the main table) and i would go through all the tables and select nvl(..) and if it doesn't exist, i would insert into, and if it does exist then i just run an update on that row.
This code will get very messy and complicated especially since the whole config schema is very complex and it might be hard to handle all the keys properly.
Another option i was looking at was triggers, so when setting up a new system, there would be a log of all the statements we ran while setting up/editing a system, then we would run it on our production schema.
I'm on a coop term, and have only been working with databases for 6 months, so i don't know that much and any information/advice would be greatly appericiated.
(We use pl/sql)
What about using export / import (or datapump) to bring over the config tables?
Check out data comparison tools like this
Think TOAD has one built in. I'm sure there are others out there too.
It is common to have tables in a schema that are what we call "static data", i.e. the users don't change it because it controls how the application works.
Each change to config data should not be run ad-hoc in the target environment. Instead, you design and code your DML carefully in one or more scripts, which get tested in a dev environment, checked into change control, and can be re-run in any environment when required.