I use flyway for my local profile to populate a DB for integration tests.
application.yml:
spring:
(...)
flyway.enabled: false
application-local.yml:
spring:
(...)
flyway:
enabled: true
license-key: (...)
locations: classpath:mssql/migrations
Expected:
App doesn't try to autoconfigure flyway during startup with profile dev
Actual:
The following profiles are active: dev
(...)
org.springframework.boot.web.reactive.context.AnnotationConfigReactiveWebServerApplicationContext
Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException:
(...)
Factory method 'flyway' threw exception; nested exception is org.springframework.boot.autoconfigure.flyway.FlywayMigrationScriptMissingException:
Cannot find migration scripts in: [classpath:db/migration] (please add migration scripts or check your Flyway configuration)
Versions:
flyway-core 6.3.1
spring boot 2.2.5
You specified application-local.yml and it means that you specified local profile but you run the application with dev profile. There are two potential issues:
You run the app with the wrong profile: should be local instead of
dev
Or you don't include local profile to dev https://docs.spring.io/spring-boot/docs/1.1.6.RELEASE/reference/html/boot-features-profiles.html
Related
I have Spring boot 2.7.0 application which connects config server with following configuration to decrypt secrets. But when I include below configuration in (application.yml or bootstrap.yml) getting binding error. Not sure what is the root cause of this issue as format is correct for the keys but still getting this error. Same config is working fine with Spring boot 2.2.1.
I have used same config in application.yml without bootstrap.yml as it depricated in 2.7.0 but still same issue.
encrypt:
key-store:
location: file:${key.path}
password: anthem$$docker
alias: docker
secret: anthem$$docker
org.springframework.boot.context.properties.bind.BindException: Failed to bind properties under 'encrypt.key-store.alias' to java.lang.String
at org.springframework.boot.context.properties.bind.Binder.handleBindError(Binder.java:384)
at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:344)
at org.springframework.boot.context.properties.bind.Binder.lambda$bindDataObject$4(Binder.java:469)
at org.springframework.boot.context.properties.bind.Binder$$Lambda$78/234145890.bindProperty(Unknown Source)
at org.springframework.boot.context.properties.bind.JavaBeanBinder.bind(JavaBeanBinder.java:95)
at org.springframework.boot.context.properties.bind.JavaBeanBinder.bind(JavaBeanBinder.java:83)
at org.springframework.boot.context.properties.bind.JavaBeanBinder.bind(JavaBeanBinder.java:59)
at org.springframework.boot.context.properties.bind.Binder.lambda$bindDataObject$5(Binder.java:473)
Description:
Failed to bind properties under 'encrypt.key-store.alias' to java.lang.String:
Property: encrypt.key-store.alias
Value: docker
Origin: class path resource [bootstrap.yml] - 40:12
Reason: org.springframework.boot.context.config.InactiveConfigDataAccessException: Inactive property source 'Config resource 'class path resource [bootstrap.yml]' via location 'optional:classpath:/' (document #5)' imported from location 'class path resource [bootstrap.yml]' cannot contain property 'encrypt.key-store.alias' [origin: class path resource [bootstrap.yml] - 40:12]
Action:
Update your application's configuration
I have a Spring Boot application that I am able to run locally without errors. I am trying to run it in a Docker Container, but I am getting new errors that I don't understand.
Dockerfile:
FROM maven:3.8-jdk-8 AS build
COPY . /usr/app
WORKDIR /usr/app
RUN mvn clean package
FROM openjdk:8-alpine
COPY --from=build /usr/app/target/myapp-*.jar /usr/app/myapp.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/app/myapp.jar"]
Top of the error stack:
WARN 8 --- [ main] com.amazonaws.util.EC2MetadataUtils : Unable to retrieve the requested metadata (/latest/dynamic/instance-identity/document). Failed to connect to service endpoint:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100) ~[aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:70) ~[aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:75) ~[aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66) ~[aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:402) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:371) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:367) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.util.EC2MetadataUtils.getEC2InstanceRegion(EC2MetadataUtils.java:282) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.regions.InstanceMetadataRegionProvider.tryDetectRegion(InstanceMetadataRegionProvider.java:59) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.regions.InstanceMetadataRegionProvider.getRegion(InstanceMetadataRegionProvider.java:50) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.regions.AwsRegionProviderChain.getRegion(AwsRegionProviderChain.java:46) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.client.builder.AwsClientBuilder.determineRegionFromRegionProvider(AwsClientBuilder.java:475) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:458) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424) [aws-java-sdk-core-1.11.792.jar!/:na]
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46) [aws-java-sdk-core-1.11.792.jar!/:na]
at org.springframework.cloud.aws.autoconfigure.secretsmanager.AwsSecretsManagerBootstrapConfiguration.smClient(AwsSecretsManagerBootstrapConfiguration.java:63) [spring-cloud-starter-aws-secrets-manager-config-2.2.6.RELEASE.jar!/:2.2.6.RELEASE]
I am not sure if this error matters, as I saw something similar locally (though that is resolved now) and the app still started up.
Error creating bean (I think this is the cause of the crash):
ERROR 23 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration': Unsatisfied dependency expressed through field 'propertySourceLocators'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'awsSecretsManagerPropertySourceLocator' defined in class path resource [org/springframework/cloud/aws/autoconfigure/secretsmanager/AwsSecretsManagerBootstrapConfiguration.class]: Unsatisfied dependency expressed through method 'awsSecretsManagerPropertySourceLocator' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smClient' defined in class path resource [org/springframework/cloud/aws/autoconfigure/secretsmanager/AwsSecretsManagerBootstrapConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.amazonaws.services.secretsmanager.AWSSecretsManager]: Factory method 'smClient' threw exception; nested exception is com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.
I compared the local maven version against the one in the the first image, and they are the same. My local java version is 1.8.0_275 while I have tried using images with versions 1.8.0_212 and 1.8.0_312 (I haven't found an image with the exact same version). I even tried copying over the working jar into the image instead of building it as part of the Docker build, but I got the exact same errors.
Edit: Here is my bootstrap.yml file that has the aws region
aws:
secretsmanager:
name: myapp-s3-creds
cloud:
aws:
region:
static: eu-west-1
If you have a docker-compose.yml file, you can specify the ff environment vars at the environment tag:
environment:
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
Create a .env file in the directory of your docker-compose file with the values of the aws env vars set:
AWS_ACCESS_KEY_ID = your_access_key_id
AWS_SECRET_ACCESS_KEY = your_secret_access_key
On docker-compose build or up, the values will be copied from the .env file into the placeholders.
Currently the scenario is I am trying to implement
I used Jhipster Registry app directly from git-hub to use it as Eureka registry and Spring-Cloud-Config server(be default feature though).
I need to centralize the configurations of micro-service-apps' configurations files in Jhipster registry.
All these we have to do without using Docker as current project doesn't uses Docker.
To do that I made changes to
bootstrap.yml file as shown below( to run as composite profile though native type with local file system as we don't have access to production git repository. So chose file-system only with dev /composite profile only.
Currently my boostrap.yml looks as below
spring:
application:
name: jhipster-registry
profiles:
active: dev
include: composite
cloud:
config:
server:
bootstrap: true
composite:
- type: native #git
search-locations: file:/C:/Jhipster_Eureka/jhipster-registry-master/central-config/
prefix: /config
fail-fast: true
name: jhipster-registry
profile: composite
I have also created a gateway.yml file under central-config folder( gateway is one of micro-service sample app whose configuration I am trying to centralize, be it eureka client, data-source everything which comes under by default to application-dev.yml in the Jhipster micro-service gateway app I generated.)
by stating all configuration like below
gateway.yml
server:
port: 8888
management:
health:
diskspace:
enabled: false
# ===================================================================
# JHipster Sample Spring Cloud Config.
# ===================================================================
# Property used on app startup to check the config server status
configserver:
name: JHipster Registry config server
status: Connected to the JHipster Registry config server!
# Default JWT secret token (to be changed in production!)
jhipster:
security:
authentication:
jwt:
# It is recommended to encrypt the secret key in Base64, using the `base64-secret` property.
# For compabitibily issues with applications generated with older JHipster releases,
# we use the non Base64-encoded `secret` property here.
# secret: my-secret-key-which-should-be-changed-in-production-and-be-base64-encoded
# The `base64-secret` property is recommended if you use JHipster v5.3.0+
# (you can type `echo 'secret-key'|base64` on your command line)
base64-secret: bXktc2VjcmV0LWtleS13aGljaC1zaG91bGQtYmUtY2hhbmdlZC1pbi1wcm9kdWN0aW9uLWFuZC1iZS1iYXNlNjQtZW5jb2RlZAo=
spring:
profiles:
active: dev
include:
- swagger
eureka:
instance:
prefer-ip-address: true
client:
service-url:
defaultZone: http://admin:admin#localhost:8761/eureka/
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:mysql://localhost:3306/conference?useUnicode=true&characterEncoding=utf8&useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC&createDatabaseIfNotExist=true
username: root
password: root
hikari:
poolName: Hikari
auto-commit: false
data-source-properties:
cachePrepStmts: true
prepStmtCacheSize: 250
prepStmtCacheSqlLimit: 2048
useServerPrepStmts: true
jpa:
show-sql: true
liquibase:
# Remove 'faker' if you do not want the sample data to be loaded automatically
contexts: dev
Issue#1
If I run my registry it's coming up properly without any issue. If I try to run "gateway" micro-service app without defining data-source definitions mentioned in it's own configuration file(application-dev.yml) it's failing at runtime.
Exception Stack trace during runtime
2020-08-06 21:45:58.301 WARN 28804 --- [ restartedMain] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servletEndpointRegistrar' defined in class path resource [org/springframework/boot/actuate/autoconfigure/endpoint/web/ServletEndpointManagementContextConfiguration$WebMvcServletEndpointManagementContextConfiguration.class]:
Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar]: Factory method 'servletEndpointRegistrar' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'healthEndpoint' defined in class path resource [org/springframework/boot/actuate/autoconfigure/health/HealthEndpointConfiguration.class]: Unsatisfied dependency expressed through method 'healthEndpoint' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'healthContributorRegistry' defined in class path resource [org/springframework/boot/actuate/autoconfigure/health/HealthEndpointConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.health.HealthContributorRegistry]: Factory method 'healthContributorRegistry' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'org.springframework.boot.actuate.autoconfigure.jdbc.DataSourceHealthContributorAutoConfiguration': Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception; nested exception is org.springframework.boot.autoconfigure.jdbc.DataSourceProperties$DataSourceBeanCreationException: Failed to determine a suitable driver class
I am surprised how eureka instance it picked up from that gateway.yml file but not the data source, so what went wrong here?
Issue#2
In the gateway.app yml server.port:8888 is mentioned, but would it be spring-cloud-config-server's port or the micro-service app port e.g., 8083
.
I observed if i try to alter that port from 8888 to 8083 it's giving error at runtime not able to find discovery client, what's the issue or am i still missing something configuration-wise?
Please let me know if the query is detailed enough now to respond.
Docker is absolutely NOT required for JHipster micro services architecture using jhipster-registry, your use case is well supported and the issues you encountered are only due to your configuration errors.
gateway.yml is not indented correctly so property names for datasource do not match and this is why you get this error about being unable to find JDBC driver.
The eureka block in the middle of the spring block completely broke it and anyway eureka config is shared by all apps so it should be placed in application-dev.yml and application-prod.yml.
gateway.yml as any other files that are in environment repository (central-config folder when using native) must be exactly the same format as your local application.yml files.
If you are not comfortable with YAML files you could switch to normal properties.
About file naming in native repository, Spring Cloud Config Server doc says:
If the repository is file-based, the server creates an Environment from application.yml (shared between all clients) and foo.yml (with foo.yml taking precedence). If the YAML files have documents inside them that point to Spring profiles, those are applied with higher precedence (in order of the profiles listed). If there are profile-specific YAML (or properties) files, these are also applied with higher precedence than the defaults.
So basically it means that in your central-config folder you should have these files:
application.yml: all properties common to all apps when no profile is set
application-dev.yml: all properties common to all apps when dev profile is set. This is where you will put the JWT secret for dev, the url for dev Eureka server (often localhost)
application-prod.yml: all properties common to all apps when prod profile is set. This is where you will put the JWT secret for prod, the url for prod Eureka server
gateway.yml: all properties of gateway app when no profile is set
gateway-dev.yml: all properties of gateway app when dev profile is set. This is where you will put dev datasource
gateway-prod.yml: all properties of gateway app when prod profile is set. This is where you will put prod datasource
So when your gateway bootstraps with dev profile, it will gets a combination of application.yml, application-dev.yml, gateway.yml and gateway-dev.yml
This mechanism can work only if you follow the naming convention {app name}-{profile}.yml
So, having named a file application-dev-gateway.yml just meant properties common to all apps with dev-gateway profile which is probably not what you meant.
I'm trying to run one of my microservices in debug mode.
For these I added jvmArguments to a command that starts application.
mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000" -Dspring.profiles.active="dev" -DAUTH0_CLIENT_ID="somevalue123" -DAUTH0_CLIENT_SECRET="somevalue123" -DACMS_CRYPTO_KEY="somevalue123" -DACMS_NODE_NAME="n/a" -DACMS_POD_IP="n/a" -DACMS_POD_NAMESPACE="n/a" -DACMS_POD_NAME="n/a"
As I'm using .yml file to provide client_id, client_secret and crypto_key I need to pass them also. Here is the config file:
acms:
crypto:
key: ${ACMS_CRYPTO_KEY}
node:
name: ${ACMS_NODE_NAME}
pod:
name: ${ACMS_POD_NAME}
namespace: ${ACMS_POD_NAMESPACE}
ip: ${ACMS_POD_IP}
env:
info:
enabled: false
auth0:
clientId: ${AUTH0_CLIENT_ID}
clientSecret: ${AUTH0_CLIENT_SECRET}
Without debug mode everything works fine. But when I'm adding
-Dspring-boot.run.jvmArguments="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"
I receive and error: Application run failed
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'acmsConfig': Injection of autowired
dependencies failed; nested exception is
java.lang.IllegalArgumentException: Could not resolve placeholder
'ACMS_CRYPTO_KEY' in value "${ACMS_CRYPTO_KEY}"
So how can I add debug mode command without breaking other arguments?
SivaKumar answer solved the problem. Working command looks like this:
mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Dspring.profiles.active=dev -DAUTH0_CLIENT_ID=testid -DAUTH0_CLIENT_SECRET=testsecret -DACMS_CRYPTO_KEY=testkey -DACMS_NODE_NAME=n/a -DACMS_POD_IP=n/a -DACMS_POD_NAMESPACE=n/a -DACMS_POD_NAME=n/a"
I was using a previous version grails-database-migration plugin for a while and never had any big issues with it. However, lately I upgraded the whole project to grails 3.0.9 and did some additional development, the behavior as follows:
Imported the current prod DB structure into local machine (that DB copy is without the latest changes and new entities)
Execute: grails -Dgrails.env=staging dbm-gorm-diff changlog.xml
What I expected at this point is new changlog.xml file with all changes of existing entities and new ones.
What I get:
Newly defined entities automatically got added into the DB.
The changes in changlog.xml only included the changes of already existing tables, such as:
also, If I try running grails -Dgrails.env=staging run-app
ERROR grails.boot.GrailsApp - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'springLiquibase_dataSource': Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: liquibase.integration.spring.SpringLiquibase.createDatabase(Ljava/sql/Connection;Lliquibase/resource/ResourceAccessor;)Lliquibase/database/Database;
FAILURE: Build failed with an exception.
What went wrong: Execution failed for task ':bootRun'.
Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java''
finished with non-zero exit value 1
...
...
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. | Error Failed to start server (Use --stacktrace to see the full trace)
Here is the portion of my application.yml
dataSource:
pooled: true
url: jdbc:mysql://127.0.0.1:3306/triz?useUnicode=yes&characterEncoding=UTF-8
driverClassName: "com.mysql.jdbc.Driver"
jmxExport: true
username: root
password: password
dialect: org.hibernate.dialect.MySQL5InnoDBDialect
properties:
jmxEnabled: true
initialSize: 5
maxActive: 50
minIdle: 5
maxIdle: 25
maxWait: 10000
maxAge: 600000
timeBetweenEvictionRunsMillis: 5000
minEvictableIdleTimeMillis: 60000
validationQuery: SELECT 1
validationQueryTimeout: 3
validationInterval: 15000
testOnBorrow: true
testWhileIdle: true
testOnReturn: false
jdbcInterceptors: ConnectionState
defaultTransactionIsolation: 2
environments:
development:
dataSource:
dbCreate: create
# url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
test:
dataSource:
dbCreate: update
url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
staging:
dataSource:
url: jdbc:mysql://127.0.0.1:3306/triz_staging?useUnicode=yes&characterEncoding=UTF-8
and gradle.build
buildscript {
ext {
grailsVersion = project.grailsVersion
}
repositories {
mavenCentral()
mavenLocal()
maven { url "https://repo.grails.org/grails/core" }
}
dependencies {
classpath "org.grails:grails-gradle-plugin:$grailsVersion"
classpath 'com.bertramlabs.plugins:asset-pipeline-gradle:2.5.0'
// classpath 'com.bertramlabs.plugins:less-asset-pipeline:2.6.7'
classpath "org.grails.plugins:hibernate:4.3.10.5"
classpath 'org.grails.plugins:database-migration:2.0.0.RC4'
}
}
...
...
dependencies {
...
compile 'org.liquibase:liquibase-core:3.3.2'
runtime 'org.grails.plugins:database-migration:2.0.0.RC4'
}
UPDATE
I have another way to approach this problem:
My plan was to generate a changelog based on my current prod DB and then generate a diff for the changes I made. Sounds simple and straightforward; however, it didn't work out as expected. Here is what I did:
Dumped prod DB
Removed liquibase tables
Run: grails dbm-generate-changelog changelog-init.xml --add
At this point, I expected changelog-init.xml to contain the current state of DB. But, instead it applied the changes based on my models first, and then tried generating the diff. Eventually, I ended up with a changelog including my entire existing DB with changes applied from gorm.
What am I doing wrong here?
Additional Observations
It looks like, whenever I try to run ANY migration related commands, grails applies all the changes before that, even through my config says:
staging:
dataSource:
dbCreate: ~
url: jdbc:mysql://127.0.0.1:3306/triz_staging?useUnicode=yes&characterEncoding=UTF-8
properties:
jmxEnabled: true
Also tried completely removing dbCreate. Didn't change anything...
I am done, have no idea where to move next!!!
Well, here is a deal...
I am not sure if that was the real reason, but all I did is moved datasource config from application.yml to application.groovy and everything got back to normal.
I would be happy to hear thoughts.
Thanks.