I'm using flyway 8.5.13 with springboot version 2.7.1 for db migrations. Migrations run successfully, but at times it inserts the new script in flyway_schema_history table with type='Delete' and also randomly marks some of the old scripts as DELETE.
I run repair but as for DELETE also success flag is marked as 1 so repair doesn't help here. To re-run the script, i've to manually remove the entries from flyway_schema_history table which are marked as 'DELETE'. Once these rows are removed manually same script executes successfully. If there is any problem with the migration script then after just deletion of row it should not work again.
Here is the screenshot for your reference
Please suggest.
Thank you,
Jagdish
Tried to run flyway repair script, but that doesn't help much because successful flag is always set to 1. Repair only helps if script is not executed successfully and marked as 0 in the flyway_schema_history table.
Related
I'm using liquibase with oracle database, after executing update command for a specific changeSet, and the log for this changeSet is inserted at DATABASECHANGELOG table, however if I executed a rollbackCount command to rollback that changeSet, the inserted log is deleted and i can't find the history of the changes that were executed and rolledback again.
Liquibase's DATABASECHANGELOG table only records what's in the database. It's not an audit history tool.
For audit, you can look at using Liquibase Hub which records update and rollback operations so you'll have a record of when the operation was performed.
You can also run rollback-*-sql command prior to running an actual rollback. For example, rollback-count-sql or rollback-sql or rollback-to-date-sql. These commands will output rollback SQL to console but not execute rollback on the database. You'll need to follow up with the actual rollback operation.
Run
liquibase rollback-count-sql 10
Examine the generated SQL or log it to a file. If happy with generated SQL then run:
liquibase rollback-count 10
I am not clear how the executeAsync works in Camunda 7.15.0 version.
Using Java code in spring-boot application, I am trying to migrate few process instances from one process version to another using migration plan.
In java code when I use execute() method then the code is obviously executed immediately.
import org.camunda.bpm.engine.RuntimeService;
final MigrationPlan migrationPlan = runtimeService.createMigrationPlan(fromProcessDefinitionId, toProcessDefinitionId).mapEqualActivities().build();
final ProcessInstanceQuery processQuery = runtimeService.createProcessInstanceQuery().processDefinitionId(fromProcessDefinitionId);
runtimeService.newMigration(migrationPlan).processInstanceQuery(processQuery).executeAsync();
But When I use executeAsync() method then I see the batch job waiting in the batches section but does not complete. How to know when will it execute?
Issue can be recreated in https://github.com/firstpostt/camunda-demo-migration. It needs postgres database and credentials need to be given in application.yml
There is an entry in act_ru_batch table. I don't see any entry in act_ru_job table
Can I configure in bpm-platform.xml file to make sure my migration plan runs within next 15 minutes when I use executeAsync() method?
Is there any option to force-trigger the batch from the admin cockpit when needed?
Found the problem. The issue was with Postgres database but I am not sure what is the exact root cause because the issue is not easy to recreate. I dropped the database and created it again from scratch which seems to resolve the issue (I used the same sql scripts again using flyway so nothing changed w.r.t database schema. The only difference I can think of is that after some data is populated already into the tables then I created the unique constraint on camunda tables which might have caused the issue. Now I created unique constraint immediately before populating any data into the tables)
P.S:
I was using unique constraint in postgres Database https://docs.camunda.org/manual/7.5/user-guide/process-engine/database/#additional-database-schema-configuration
I figured out that the issue was with database because I tried with h2 filesystem database(camunda.bpm.database.schema-update as true and spring.datasource.url=jdbc:h2:~/camunda;DB_CLOSE_ON_EXIT=false) and batch worked fine.
Then I used postgres database (without the unique constraint script) and batch worked fine. When I created a new database schema with the unique constraint script then the batch did not work and even if I dropped the constraint the batch did not work anymore
So I dropped the database and created a new database again without unique constraint and then the batch worked fine. After that I added the unique constraint and the batch still works fine.
I am not able to recreate the issue consistently but my guess is that it has something to do with the unique constraint. If you are not using this unique constraint then I am sure that this problem will never occur at all
I'm seeing flyway seemingly skip migration "V.*" scripts and fail on a later script. Script V1 creates a table, then script V2 alters it. Except that no table gets created, the database is a fresh DB2 instance then it legitimately complains when an alter is attempted.
I have a related question still open, but it would be helpful to get Spring-Boot/Flyway to log the specific SQL that was tried. Not looking for a log of the SQL migration, but the exact SQL in the migration that was attempted.
I tried adding -Dlogging.level.flyway.core.dbsupport.SqlScript=DEBUG but didn't see any change.
I have an existing database (version x), and I can generate ChangeLog file by using below command
mvn liquibase:generateChangeLog -Dliquibase.outputChangeLogFile=d:\output.xml
After that, I try to remove one table in database directly, How can I user Liquibase to rollback my database to version x ?
Once you have started using Liquibase, you should avoid making changes directly to the database.
Let's simplify the scenario to make it easier to describe. Say at version x your database has a single table called TABLE1 and nothing else. You run the generateChangeLog command, and you get a changelog that has a single changeset that says "create table TABLE1". When Liquibase creates that changeset, it gives it an id. After creating the changelog, you will then want to record in that database that the database and changelog are 'in sync' by running the liquibase changelogSync command - this creates a second table named DATABASECHANGELOG and adds a row to that table with the id of the changeset and some other information.
If you then manually delete the table, Liquibase doesn't 'know' that you have done this, so you would need to also manually let liquibase know that you had done that delete. You would do that by removing the row from the databasechangelog table. You could then re-create the table and get back to version x by running the liquibase update command.
I have a local instance of a database that I recently created using DbContext.Database.Create(), so the __MigrationHistory table exists with an InitalCreate entry that matches the code at the moment.
Some code-based migrations exist in the Migrations folder, however. These will be run in our development and staging environments to bring those databases in line with the code. I don't need to apply them locally, however, since I created the database using the current code.
I now need to make a change to the model and create the corresponding migration. But when I run Add-Migration TestMigration, I get the following error
Unable to generate an explicit migration because the following explicit
migrations are pending:
[201203271113060_AddTableX,
201203290856574_AlterColumnY]
Apply the pending explicit migrations before attempting to generate
a new explicit migration.
What should I do in this case? I can't point the Add-Migration tool at another environment because it's not guaranteed that version matches what I have locally. I want a migration that matches only the changes I've made.
It seems I have a few options but none are ideal:
Delete the other migrations from the Migrations folder, run the Add-Migration command, upgrade the database, then restore the old migrations. This is simple but seems a bit hackish.
Revert to the version of the model in source control that the first migration was applied to, then build this and use it to create the database. Then get the latest version, apply all the migrations, then I'm ready to add my migration. This seems like a lot of effort!
Create the migration manually.
Does anyone have any suggestions about how to manage this?
We are planning to use a variant of your Option #1...
Our Standard Operating Procedure is to generate a SQL script for each migration (using the -script option of update-database), in order to have SQL scripts to be applied to end-user "production" databases by InstallShield (we plan to use EF update-database only for developer databases).
Thus, we have both the Migration .cs files and the corresponding .sql files for all migrations in our Migrations folder.
So rather than deleting the migrations from the Migrations folder (as you proposed in #1), we use SQL Mgmt Studio to manually apply just the parts of the .sql files that do the inserts into _MigrationHistory.
That brings the _MigrationHistory of the local database up-to-date with the changes that are already incorporated into that database.
But it's a kludge, and we're still looking for a better solution.
DadCat
What I've found works best is very simple: don't use DbContext.Database.Create() once you've enabled migrations. If you want to programmatically create a new database, use the migrations API instead.
var migrator = new DbMigrator(new Configuration());
migrator.Update();
Then you've got the full migration history and adding further migrations works just as expected.
You either need to run "update-database" from the package manager console to push your changes to the database OR you can delete the pending migration file ([201203271113060_AddTableX]) from your Migrations folder and then re-run "add-migration" to create a brand new migration based off of your edits.
I have encountered the same problem.
If you run
Update-database
and then run
Add-Migration YourMigrationName
This solves the problem
simply exclude the old migration file from the solution files.