liquibase.lockservice.StandardLockService.waitForLock - bioinformatics

Question

related: Issue on github: https://github.com/broadinstitute/cromwell/issues/7009
I'm using a bioinformatics java program named cromwell (https://github.com/broadinstitute/cromwell/) which, as far as I understand, uses liquibase and hsqldb to store data. I'm not familiar with liquibase. The hsqldb is a local file:
jdbc:hsqldb:file:cromwell-executions/cromwell-db/cromwell-db;
shutdown=false;
hsqldb.default_table_type=cached;hsqldb.tx=mvcc;
hsqldb.result_max_memory_rows=10000;
hsqldb.large_data=true;
hsqldb.applog=3;
hsqldb.lob_compressed=true;
hsqldb.script_format=3
Whenever I try to start the program I got the following stacktrace:
java -Dconfig.file=${PWD}/app.conf -jar ${CROMWELL_JAR} run test.wdl --inputs input.json
(...)
2023-02-08 16:32:11,54] [info] checkpointClose synched
[2023-02-08 16:32:11,57] [info] checkpointClose script done
[2023-02-08 16:32:11,57] [info] dataFileCache commit start
[2023-02-08 16:32:11,57] [info] dataFileCache commit end
[2023-02-08 16:32:11,69] [info] checkpointClose end
[2023-02-08 16:32:11,69] [info] Checkpoint end - txts: 5342
[2023-02-08 16:32:21,70] [info] Checkpoint start
[2023-02-08 16:32:21,70] [info] checkpointClose start
[2023-02-08 16:32:21,70] [info] checkpointClose synched
[2023-02-08 16:32:21,74] [info] checkpointClose script done
[2023-02-08 16:32:21,74] [info] dataFileCache commit start
[2023-02-08 16:32:21,76] [info] dataFileCache commit end
[2023-02-08 16:32:21,82] [info] checkpointClose end
[2023-02-08 16:32:21,82] [info] Checkpoint end - txts: 5348
[2023-02-08 16:32:21,89] [error] Failed to instantiate Cromwell System. Shutting down Cromwell.
liquibase.exception.LockException: Could not acquire change log lock. Currently locked by fdb0:cafe:d0d0:ceb4:ba59:9fff:fec3:33de%p1p1 (fdb0:cafe:d0d0:ceb4:ba59:9fff:fec3:33de%p1p1) since 2/8/23, 4:23 PM
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:270)
at liquibase.Liquibase.lambda$update$1(Liquibase.java:214)
at liquibase.Scope.lambda$child$0(Scope.java:180)
at liquibase.Scope.child(Scope.java:189)
at liquibase.Scope.child(Scope.java:179)
at liquibase.Scope.child(Scope.java:158)
at liquibase.Liquibase.runInScope(Liquibase.java:2405)
at liquibase.Liquibase.update(Liquibase.java:211)
at liquibase.Liquibase.update(Liquibase.java:197)
at cromwell.database.migration.liquibase.LiquibaseUtils$.updateSchema(LiquibaseUtils.scala:74)
at cromwell.database.migration.liquibase.LiquibaseUtils$.updateSchema(LiquibaseUtils.scala:46)
at cromwell.services.ServicesStore$EnhancedSqlDatabase$.$anonfun$initialized$1(ServicesStore.scala:11)
at cromwell.services.ServicesStore$EnhancedSqlDatabase$.$anonfun$initialized$1$adapted(ServicesStore.scala:11)
at cromwell.database.slick.SlickDatabase.$anonfun$withConnection$1(SlickDatabase.scala:156)
at slick.jdbc.SimpleJdbcAction.run(StreamingInvokerAction.scala:70)
at slick.jdbc.SimpleJdbcAction.run(StreamingInvokerAction.scala:69)
at slick.basic.BasicBackend$DatabaseDef$$anon$3.liftedTree1$1(BasicBackend.scala:276)
at slick.basic.BasicBackend$DatabaseDef$$anon$3.run(BasicBackend.scala:276)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1589)
so ,as far as I understand liquibase is not able to lock a file although I deleted all the (+hidden) files in my working directory.
I don't care about db versions, is there a genetic way to disable liquibase (e.g using java properties ?)
is there a place (my home ?) where liquibase writes the locks ?
How can I fix this ?

Answer 1

Cromwell developer here. This can happen if the server stopped unexpectedly partway through a previous migration and never released its lock.
The easiest solution is to delete the HSQL pages from disk and start over. Another option would be to use a standalone DB client program to unset the value in DATABASECHANGELOGLOCK.