grails database migration plugin problems after upgrade to grails 3 - spring

I was using a previous version grails-database-migration plugin for a while and never had any big issues with it. However, lately I upgraded the whole project to grails 3.0.9 and did some additional development, the behavior as follows:
Imported the current prod DB structure into local machine (that DB copy is without the latest changes and new entities)
Execute: grails -Dgrails.env=staging dbm-gorm-diff changlog.xml
What I expected at this point is new changlog.xml file with all changes of existing entities and new ones.
What I get:
Newly defined entities automatically got added into the DB.
The changes in changlog.xml only included the changes of already existing tables, such as:
also, If I try running grails -Dgrails.env=staging run-app
ERROR grails.boot.GrailsApp - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'springLiquibase_dataSource': Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: liquibase.integration.spring.SpringLiquibase.createDatabase(Ljava/sql/Connection;Lliquibase/resource/ResourceAccessor;)Lliquibase/database/Database;
FAILURE: Build failed with an exception.
What went wrong: Execution failed for task ':bootRun'.
Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java''
finished with non-zero exit value 1
...
...
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. | Error Failed to start server (Use --stacktrace to see the full trace)
Here is the portion of my application.yml
dataSource:
pooled: true
url: jdbc:mysql://127.0.0.1:3306/triz?useUnicode=yes&characterEncoding=UTF-8
driverClassName: "com.mysql.jdbc.Driver"
jmxExport: true
username: root
password: password
dialect: org.hibernate.dialect.MySQL5InnoDBDialect
properties:
jmxEnabled: true
initialSize: 5
maxActive: 50
minIdle: 5
maxIdle: 25
maxWait: 10000
maxAge: 600000
timeBetweenEvictionRunsMillis: 5000
minEvictableIdleTimeMillis: 60000
validationQuery: SELECT 1
validationQueryTimeout: 3
validationInterval: 15000
testOnBorrow: true
testWhileIdle: true
testOnReturn: false
jdbcInterceptors: ConnectionState
defaultTransactionIsolation: 2
environments:
development:
dataSource:
dbCreate: create
# url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
test:
dataSource:
dbCreate: update
url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
staging:
dataSource:
url: jdbc:mysql://127.0.0.1:3306/triz_staging?useUnicode=yes&characterEncoding=UTF-8
and gradle.build
buildscript {
ext {
grailsVersion = project.grailsVersion
}
repositories {
mavenCentral()
mavenLocal()
maven { url "https://repo.grails.org/grails/core" }
}
dependencies {
classpath "org.grails:grails-gradle-plugin:$grailsVersion"
classpath 'com.bertramlabs.plugins:asset-pipeline-gradle:2.5.0'
// classpath 'com.bertramlabs.plugins:less-asset-pipeline:2.6.7'
classpath "org.grails.plugins:hibernate:4.3.10.5"
classpath 'org.grails.plugins:database-migration:2.0.0.RC4'
}
}
...
...
dependencies {
...
compile 'org.liquibase:liquibase-core:3.3.2'
runtime 'org.grails.plugins:database-migration:2.0.0.RC4'
}
UPDATE
I have another way to approach this problem:
My plan was to generate a changelog based on my current prod DB and then generate a diff for the changes I made. Sounds simple and straightforward; however, it didn't work out as expected. Here is what I did:
Dumped prod DB
Removed liquibase tables
Run: grails dbm-generate-changelog changelog-init.xml --add
At this point, I expected changelog-init.xml to contain the current state of DB. But, instead it applied the changes based on my models first, and then tried generating the diff. Eventually, I ended up with a changelog including my entire existing DB with changes applied from gorm.
What am I doing wrong here?
Additional Observations
It looks like, whenever I try to run ANY migration related commands, grails applies all the changes before that, even through my config says:
staging:
dataSource:
dbCreate: ~
url: jdbc:mysql://127.0.0.1:3306/triz_staging?useUnicode=yes&characterEncoding=UTF-8
properties:
jmxEnabled: true
Also tried completely removing dbCreate. Didn't change anything...
I am done, have no idea where to move next!!!

Well, here is a deal...
I am not sure if that was the real reason, but all I did is moved datasource config from application.yml to application.groovy and everything got back to normal.
I would be happy to hear thoughts.
Thanks.

Related

Spring Boot- Profile setting Through YML file is not working

In Spring Boot program, profile setting using application.yml is not working whereas the same program is working using applicaiton.properties. the following actions has been performed but it ended up in vain
In Eclipse ide Project-Clean has been done
Maven project updated
Spaces in YML file removed and corrected
YML file is deleted and recreated.
applicaiton.yml
spring:
profiles:
active: dev
applicaiton-dev.yml
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql:///ntspbms616db
username: root
password: root
dbcp2:
max-total: 100
initial-size: 10
max-conn-lifetime-millis: 10000000
type: org.apache.commons.dbcp2.BasicDataSource
The error i got after running the program is
Enter the Employee Name
Raja
Enter the Employee Designation
Manager
Enter the Basic Salary
9898999
12:28:40.622 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.yaml.snakeyaml.scanner.ScannerException: while scanning for the next token
found character '\t(TAB)' that cannot start any token. (Do not use \t(TAB) for indentation)
in 'reader', line 4, column 2:
How to resolve the issue?
This error means literally what is says.
The "tab" is not permitted by the YAML specs:
To maintain portability, tab characters must not be used in
indentation, since different systems treat tabs differently. Note that
most modern editors may be configured so that pressing the tab key
results in the insertion of an appropriate number of spaces.
So, you should double check for the tab characters in your yaml files and if exist, replace them by double space.
Also, just to add, I can see incorrect formatting of dbcp2 section, without any intendation, so, it should look like this:
spring:
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql:///ntspbms616db
username: root
password: root
dbcp2:
max-total: 100
initial-size: 10
max-conn-lifetime-millis: 10000000
type: org.apache.commons.dbcp2.BasicDataSource

Execute sql file on every startup using liquibase configuration

I want to create a SQL initialization file to populate test data every time on Spring startup:
databaseChangeLog:
- changeSet:
id: 0001
author: test
dbms: postgres
changes:
- sqlFile:
- relativeToChangelogFile: true
- path: data.sql
Do you know how I can configure it to be executed on every Spring startup?
You can use the following property. Now the specific changeSet will run every time
- changeSet:
runAlways: true
https://docs.liquibase.com/concepts/basic/changeset.html
runAlways Executes the changeset on every run, even if it has been run before.

Unable to disable flyway via spring.flyway.enabled

I use flyway for my local profile to populate a DB for integration tests.
application.yml:
spring:
(...)
flyway.enabled: false
application-local.yml:
spring:
(...)
flyway:
enabled: true
license-key: (...)
locations: classpath:mssql/migrations
Expected:
App doesn't try to autoconfigure flyway during startup with profile dev
Actual:
The following profiles are active: dev
(...)
org.springframework.boot.web.reactive.context.AnnotationConfigReactiveWebServerApplicationContext
Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException:
(...)
Factory method 'flyway' threw exception; nested exception is org.springframework.boot.autoconfigure.flyway.FlywayMigrationScriptMissingException:
Cannot find migration scripts in: [classpath:db/migration] (please add migration scripts or check your Flyway configuration)
Versions:
flyway-core 6.3.1
spring boot 2.2.5
You specified application-local.yml and it means that you specified local profile but you run the application with dev profile. There are two potential issues:
You run the app with the wrong profile: should be local instead of
dev
Or you don't include local profile to dev https://docs.spring.io/spring-boot/docs/1.1.6.RELEASE/reference/html/boot-features-profiles.html

How to update loggingService of container.v1.cluster with deployment-manager

I want to set the loggingService field of an existing container.v1.cluster through deployment-manager.
I have the following config
resources:
- name: px-cluster-1
type: container.v1.cluster
properties:
zone: europe-west1-b
cluster:
description: "dev cluster"
initialClusterVersion: "1.13"
nodePools:
- name: cluster-pool
config:
machineType: "n1-standard-1"
oauthScopes:
- https://www.googleapis.com/auth/compute
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
- https://www.googleapis.com/auth/monitoring
management:
autoUpgrade: true
autoRepair: true
initialNodeCount: 1
autoscaling:
enabled: true
minNodeCount: 3
maxNodeCount: 10
ipAllocationPolicy:
useIpAliases: true
loggingService: "logging.googleapis.com/kubernetes"
masterAuthorizedNetworksConfig:
enabled: false
locations:
- "europe-west1-b"
- "europe-west1-c"
When I try to run gcloud deployment-manager deployments update ..., I get the following error
ERROR: (gcloud.deployment-manager.deployments.update) Error in Operation [operation-1582040492957-59edb819a5f3c-7155f798-5ba37285]: errors:
- code: NO_METHOD_TO_UPDATE_FIELD
message: No method found to update field 'cluster' on resource 'px-cluster-1' of
type 'container.v1.cluster'. The resource may need to be recreated with the new
field.
The same succeeds if I remove loggingService.
Is there a way to update loggingService using deployment-manager without deleting the cluster?
The error NO_METHOD_TO_UPDATE_FIELD is due to updating "initialClusterVersion" when you issued the update call to GKE. This field is only used on creation of the cluster, and the type definition doesn't currently allow for it to be updated later. So that should remain static at the original value and will have no effect on the deployment moving forward or try to delete/comment that line.
Even when the previous entry is true, there is also no method to update the logging service, actually Deployment Manager doesn't have many update methods, so, try using the gcloud command to update the cluster directly, keep in mind that you have to use the monitoring service together with the logging service, so, the commando would look like:
gcloud container clusters update px-cluster-1 --logging-service=logging.googleapis.com/kubernetes --monitoring-service=monitoring.googleapis.com/kubernetes --zone=europe-west1-b

How-to: Grails 3.0.2 + Oracle Database 12c?

I am making my first steps with Grails and try to create a hello world application with an Oracle Database 12c.
Unfortunately the tutorial documentation on the database specific part at www.grails.org seems not that comprehensive (in contrast for example to that of Ruby on Rails, which I used now several years) - especially when it comes to non-H2 databases.
Is there some recent tutorial on how to get a Grails 3.0.2 app up and running with an Oracle Database 12c?
Mainly I am interested on a) where to put the corresponding JDBC driver and b) how the database configuration should look like.
For a)
grails.org talks about a "lib" directory but does not explain, where it exists or should be created. Other sources say, that the "lib" directory is outdated since several Grails versions and that JDBC drivers should be loaded via dependencies from some repositories - which is obviously not possible with the closed-source Oracle JDBC driver.
For b)
Currently, I have created basic database configuration at <my-app>/grails-app/conf/DataSource.groovy for Oracle 12c:
dataSource {
url = "jdbc:oracle:thin:#my-server:my-port:my-sid"
driverClassName = "oracle.jdbc.OracleDriver"
dialect = "Oracle10gDialect"
username = "my-user"
password = "my-pass"
}
Should that be enough for a simple hello world app?
Everything I have found so far at the web is outdated either on the Grails side (covering versions 2.x oder 1.x) and/or the Oracle side (covering versions 10* oder 11*).
Thanks for any hints!
Update (just for the record and to bring together the comments below):
1)
Edit <myapp>/grails-app/conf/application.yml.
hibernate:
jdbc:
use_get_generated_keys: true
(...)
dataSource:
pooled: true
jmxExport: true
driverClassName: oracle.jdbc.OracleDriver
username: <myuser>
password: <mypassword>
(...)
environments:
development:
dataSource:
dbCreate: create
url: jdbc:oracle:thin:#<myserver>:<myport>:<mysid>
(...)
2)
Get the corresponding Oracle JDBC driver that fits your JDK and Oracle DB release. In my case (openJDK 7 + Oracle 12c) this was ojdbc7.jar release 12.1.0.2.0 from the Oracle Website: http://www.oracle.com/technetwork/database/features/jdbc/index.html
3)
Place this JDBC driver in <myapp>/lib (create this directory, if it does not yet exists).
4)
Add details of this JDBC driver to the dependencies section of <myapp>/build.gradle (modify if necessary to fit your environment/releases):
(...)
dependencies {
(...)
runtime "com.oracle.ojdbc7:12.1.0.2.0"
(...)
}
(...)
5)
Create a Maven artifact (modify if necessary to fit your environment/releases):
mvn install:install-file -Dfile=lib/ojdbc7.jar -DgroupId=com.oracle -DartifactId=ojdbc7 -Dversion=12.1.0.2.0 -Dpackaging=jar -DgeneratePom=true
6)
Add the path to the JDBC driver to your CLASSPATH: export CLASSPATH=/<somewhere>/<myapp>/lib/ojdbc7.jar
7)
Enjoy ...
yes this will be OK for meanwhile. But there are details missing.
I'll post detailed instructions as soon as i get to the office.
Actually there is a sample for mySql somewhere on the grails site, and from it it's pretty clear how to make same for an oracle DB.
I just have created a new app and compared it with myne for oracle 11g.
Here it is:
./build.gradle, add following line to "dependencies" section:
runtime "com.oracle:ojdbc14:10.2.0.3.0"
./grails-app/conf/application.yml, add following to "hibernate" section:
jdbc:
use_get_generated_keys: true
./grails-app/conf/application.yml, dataSource: section should be like:
dataSource:
pooled: true
jmxExport: true
driverClassName: oracle.jdbc.OracleDriver
username: YOURUSERNAME
password: yoursecret
./grails-app/conf/application.yml, environments: configuration, all three environments should be like:
environments:
development:
dataSource:
dbCreate: create
url: jdbc:oracle:thin:#somehost.net:1521:YOURORAINSTANCE
test:
dataSource:
dbCreate: update
url: jdbc:oracle:thin:#somehost.net:1521:YOURORAINSTANCE
production:
dataSource:
dbCreate: update
url: jdbc:oracle:thin:#somehost.net:1521:YOURORAINSTANCE
And the Hibernate dialect for Oracle you have is pretty old one, I think you'd better remove it, without specifying it explicitly all works fine.
You can get the JDBC driver through this maven command.
mvn install:install-file \
-Dfile=/lib/ojdbc7.jar \
-DgroupId=com.oracle \
-DartifactId=ojdbc7 \
-Dversion=12.1.0.1 \
-Dpackaging=jar \
-DgeneratePom=true
download ojdbc7.jar from http://www.oracle.com/technetwork/database/features/jdbc/jdbc-drivers-12c-download-1958347.html
create a <myapp>/lib folder in the main folder of your application (lib folder was removed from Grails 3)
copy ojdbc7.jar to lib folder
add this line to the <myapp>/build.gradle file
dependencies{
(...)
runtime fileTree(dir: 'lib', include: '*.jar')
(...)
}
create lines in the <myapp>/grails-app/conf/application.yml
hibernate:
(...)
jdbc:
use_get_generated_keys: true
dataSource:
pooled: true
jmxExport: true
driverClassName: oracle.jdbc.OracleDriver
username: DBUSERNAME
password: dbpassword
environments:
development:
dataSource:
dbCreate: create
url: jdbc:oracle:thin:#localhost.net:1521:dbname
test:
dataSource:
dbCreate: update
url: jdbc:oracle:thin:#localhost.net:1521:dbname
production:
dataSource:
dbCreate: update
url: jdbc:oracle:thin:#locaLhost.net:1521:dbname
Hibernate as of 2.0.1 (Grails 2.0.1) supported a property "dialect". As shown below. Presume there's a similar property still for newer versions of Hibernate.
File .../grails-app/conf/DataSource.groovy
// Global data source settings
// Note that these global properties values are composited with
// the environment-specific properties under "environments"
dataSource {
pooled = true
driverClassName = "oracle.jdbc.OracleDriver"
...
// Required for function with Oracle 12c and above.
// Not required for Oracle 11.2 and below. For the present works with both.
// Specifying Oracle 10 dialect is sufficient.
// Oracle 12c specific dialect support is not available for Hibernate 2.0.1.
// Works with either ojdbc7 or ojdbc8 jars.
dialect = org.hibernate.dialect.Oracle10gDialect
}
Applying the "dialect" property was sufficient to run older code against both Oracle 11g, 12c. It bought time for code migration to newer versions and dependencies.

Resources