I'm seeing flyway seemingly skip migration "V.*" scripts and fail on a later script. Script V1 creates a table, then script V2 alters it. Except that no table gets created, the database is a fresh DB2 instance then it legitimately complains when an alter is attempted.
I have a related question still open, but it would be helpful to get Spring-Boot/Flyway to log the specific SQL that was tried. Not looking for a log of the SQL migration, but the exact SQL in the migration that was attempted.
I tried adding -Dlogging.level.flyway.core.dbsupport.SqlScript=DEBUG but didn't see any change.
Related
I am not clear how the executeAsync works in Camunda 7.15.0 version.
Using Java code in spring-boot application, I am trying to migrate few process instances from one process version to another using migration plan.
In java code when I use execute() method then the code is obviously executed immediately.
import org.camunda.bpm.engine.RuntimeService;
final MigrationPlan migrationPlan = runtimeService.createMigrationPlan(fromProcessDefinitionId, toProcessDefinitionId).mapEqualActivities().build();
final ProcessInstanceQuery processQuery = runtimeService.createProcessInstanceQuery().processDefinitionId(fromProcessDefinitionId);
runtimeService.newMigration(migrationPlan).processInstanceQuery(processQuery).executeAsync();
But When I use executeAsync() method then I see the batch job waiting in the batches section but does not complete. How to know when will it execute?
Issue can be recreated in https://github.com/firstpostt/camunda-demo-migration. It needs postgres database and credentials need to be given in application.yml
There is an entry in act_ru_batch table. I don't see any entry in act_ru_job table
Can I configure in bpm-platform.xml file to make sure my migration plan runs within next 15 minutes when I use executeAsync() method?
Is there any option to force-trigger the batch from the admin cockpit when needed?
Found the problem. The issue was with Postgres database but I am not sure what is the exact root cause because the issue is not easy to recreate. I dropped the database and created it again from scratch which seems to resolve the issue (I used the same sql scripts again using flyway so nothing changed w.r.t database schema. The only difference I can think of is that after some data is populated already into the tables then I created the unique constraint on camunda tables which might have caused the issue. Now I created unique constraint immediately before populating any data into the tables)
P.S:
I was using unique constraint in postgres Database https://docs.camunda.org/manual/7.5/user-guide/process-engine/database/#additional-database-schema-configuration
I figured out that the issue was with database because I tried with h2 filesystem database(camunda.bpm.database.schema-update as true and spring.datasource.url=jdbc:h2:~/camunda;DB_CLOSE_ON_EXIT=false) and batch worked fine.
Then I used postgres database (without the unique constraint script) and batch worked fine. When I created a new database schema with the unique constraint script then the batch did not work and even if I dropped the constraint the batch did not work anymore
So I dropped the database and created a new database again without unique constraint and then the batch worked fine. After that I added the unique constraint and the batch still works fine.
I am not able to recreate the issue consistently but my guess is that it has something to do with the unique constraint. If you are not using this unique constraint then I am sure that this problem will never occur at all
I've always thought that if a migration applied with flywaydb is removed (both: entry from DB schema_version table and migration sql file) then application (written in spring framework with flyway in classpath) will crash at startup because of invalid checksums.
Actually (I tested it today) when both migration file and DB entry is removed application starts just fine, which I found a bit confusing.
How does flyway calculate the checksums? Is it based only on content of migration file being applied or previous migrations are also taken into account?
Also, is it safe to remove already applied migrations if both file and table entry is removed?
Many questions here. For your main question, this behavior is controlled by the ignoreMissingMigrations flag. See https://flywaydb.org/documentation/commandline/migrate
Checksums are calculated based on the content of the migrations files (ignoring line-endings due to Git CRLF transformations).
If both a migration file and its entry in the metadata table are removed Flyway doesn't know anything about it anymore. This however doesn't mean it is safe, as it may impact your ability to recreate your database schema later.
In my project, current approach is to create database if not already exists using CreateDatabaseIfNotExists and doing seeding initial data from that Intializer as well. I also added Code First Migration support after upgrade to Entity 4.4, so that in the future when we change the modle/database structure we can update client side database without drop their exist database.
However it didn't seems to working well, for example, I am now stuck on design time where forms wouldn't load and the error message is something like The model backing the 'myEntities' context has changed since the database was created. Consider using Code First Migrations to update the database (http://go.microsoft.com/fwlink/?LinkId=238269).. But the model and database is indeed the updated version, just seems Migration didn't recognize the database generated by CreateDatabaseIfNotExists, but at the same time all seems working well at run time.
Also after that I noticed that if I let CreateDatabaseIfNotExists initialize a database, Add-migration afterwards will fail and complain that pending migration and ask me to do a update-database. When I try to do a the update-database, it will fail as well because the migration path seems assume the database is in initial setup state and will trying to running all the migration scripts while none should be run as the database generated by CreateDatabaseIfNotExists is indeed sync with current model and should not be migrated at all.
I discovered that there is a MigrationHistory table in System Tables, that table will always save the database initialize history regardless of whether the initializer is CreateDatabaseIfNotExists or MigrateDatabaseToLatestVersion. The difference is that if database is initialized by CreateDatabaseIfNotExists1, everytime the database initialized, the migriationId for that initialize record will be different, butMigrateDatabaseToLatestVersion` will always save same set of migrationId for each of the Migration steps. I guess that is how Entity Framework 5.0 works.
So in the end, I give up and rewrite my DB Access code to seeding the initial database data
in other part of my code rather than the CreateDatabaseIfNotExists or MigrationsConfiguration` class because either suit my need.
I have my liquibase schema defined initially for PostreSQL. Now, I have to modify the schema file to support Oracle. I have a change-set that has a <sql> tag. It has a query that accesses the pg_catalog table to set a value to the sequence. However, this will not work for Oracle. If I remove it, the Liquibase complains with check-sum validation fail. It complains even if I have an empty <sql> tag or some other query specified within. Within this change-set, I have many other create-table statement, so I cannot just remove oracle from dbms attribute. Is there any way I can suppress this sql from running for Oracle?
The dbms attribute on the changeset is the mechanism designed to handle this problem..... Sounds like you're trying to do too much within one changesest (but I guess you've figured that out)
The checksum validation failure is liquibase's safety mechanism designed to defend the database against someone tampering with the schema files.
How to fix it is to use the clearChecksums option when running liquibase. It tells liquibase to recompute the checksums for the changesets already in the database. This will enable your postgres database instance to accept the alterations to its changesets that you've made for Oracle.
I have a local instance of a database that I recently created using DbContext.Database.Create(), so the __MigrationHistory table exists with an InitalCreate entry that matches the code at the moment.
Some code-based migrations exist in the Migrations folder, however. These will be run in our development and staging environments to bring those databases in line with the code. I don't need to apply them locally, however, since I created the database using the current code.
I now need to make a change to the model and create the corresponding migration. But when I run Add-Migration TestMigration, I get the following error
Unable to generate an explicit migration because the following explicit
migrations are pending:
[201203271113060_AddTableX,
201203290856574_AlterColumnY]
Apply the pending explicit migrations before attempting to generate
a new explicit migration.
What should I do in this case? I can't point the Add-Migration tool at another environment because it's not guaranteed that version matches what I have locally. I want a migration that matches only the changes I've made.
It seems I have a few options but none are ideal:
Delete the other migrations from the Migrations folder, run the Add-Migration command, upgrade the database, then restore the old migrations. This is simple but seems a bit hackish.
Revert to the version of the model in source control that the first migration was applied to, then build this and use it to create the database. Then get the latest version, apply all the migrations, then I'm ready to add my migration. This seems like a lot of effort!
Create the migration manually.
Does anyone have any suggestions about how to manage this?
We are planning to use a variant of your Option #1...
Our Standard Operating Procedure is to generate a SQL script for each migration (using the -script option of update-database), in order to have SQL scripts to be applied to end-user "production" databases by InstallShield (we plan to use EF update-database only for developer databases).
Thus, we have both the Migration .cs files and the corresponding .sql files for all migrations in our Migrations folder.
So rather than deleting the migrations from the Migrations folder (as you proposed in #1), we use SQL Mgmt Studio to manually apply just the parts of the .sql files that do the inserts into _MigrationHistory.
That brings the _MigrationHistory of the local database up-to-date with the changes that are already incorporated into that database.
But it's a kludge, and we're still looking for a better solution.
DadCat
What I've found works best is very simple: don't use DbContext.Database.Create() once you've enabled migrations. If you want to programmatically create a new database, use the migrations API instead.
var migrator = new DbMigrator(new Configuration());
migrator.Update();
Then you've got the full migration history and adding further migrations works just as expected.
You either need to run "update-database" from the package manager console to push your changes to the database OR you can delete the pending migration file ([201203271113060_AddTableX]) from your Migrations folder and then re-run "add-migration" to create a brand new migration based off of your edits.
I have encountered the same problem.
If you run
Update-database
and then run
Add-Migration YourMigrationName
This solves the problem
simply exclude the old migration file from the solution files.