I have configured spring boot application to take properties from my environment but strangely I am facing an error while starting my application.
I have added the properties in my ~/.bash_profile and also did source ~/.bash_profile after adding them to the profile.
This is how my bootstrap.properties look like:
spring.application.name=gamification
spring.cloud.vault.enabled=${VAULT_ENABLE:true}
spring.cloud.vault.fail-fast=false
spring.cloud.vault.token=${VAULT_TOKEN}
spring.cloud.vault.scheme=${VAULT_SCHEME}
spring.cloud.vault.host=${VAULT_HOST}
spring.cloud.vault.port=${VAULT_PORT:8200}
I am getting this error:
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.vault.config.VaultReactiveBootstrapConfiguration]: Constructor threw exception; nested exception is java.lang.IllegalArgumentException: Scheme must be http or https
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:216) ~[spring-beans-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117) ~[spring-beans-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:310) ~[spring-beans-5.2.4.RELEASE.jar:5.2.4.RELEASE]
... 30 common frames omitted
Caused by: java.lang.IllegalArgumentException: Scheme must be http or https
at org.springframework.util.Assert.isTrue(Assert.java:118) ~[spring-core-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.vault.client.VaultEndpoint.setScheme(VaultEndpoint.java:167) ~[spring-vault-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.cloud.vault.config.VaultConfigurationUtil.createVaultEndpoint(VaultConfigurationUtil.java:91) ~[spring-cloud-vault-config-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.cloud.vault.config.VaultReactiveBootstrapConfiguration.<init>(VaultReactiveBootstrapConfiguration.java:110) ~[spring-cloud-vault-config-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_231]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_231]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_231]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_231]
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:203) ~[spring-beans-5.2.4.RELEASE.jar:5.2.4.RELEASE]
... 32 common frames omitted
I added a debug point in Vault Endpoint and found this:
Here as you can see, the VAULT_HOST is being taken as VAULT_HOST instead of the value of that environment variable, and same with the VAULT_SCHEME
[EDIT]
Adding bash_profile vault configuration:
export VAULT_ENABLE=true
export VAULT_SCHEME=http
export VAULT_HOST=vault-1.dev.lokal
export VAULT_PORT=8200
export VAULT_TOKEN=5F97X
[EDIT #2]
Tried out the solution suggested by #Gopinath
I am getting environment as null when trying to autowire it
The root cause of the problem can be found form this error message:
org.springframework.core.convert.ConverterNotFoundException:
No converter found capable of converting
from type [java.lang.String]
to type [org.springframework.cloud.vault.config.VaultProperties$Config]
The above message indicates that the VaultProperties object could not be initialized using the string parameter supplied.
Here is the link to documentation and instructions on configuring VaultProperties:
https://spring.io/guides/gs/vault-config/
Some more information to help understand vault:
References:
Spring Cloud Vault: https://cloud.spring.io/spring-cloud-vault/
Hashicorp Vault: https://www.vaultproject.io
What is a Vault?
A vault is a secure storage space meant for storing secret information.
Hashicorp Vault is one tool that offers vault functionality for cloud applications.
What is Spring Boot Vault?
Spring Boot applications commonly require secret information for those to work.
Some examples of secret information are:
Database password
Private key
API key
Usually, the input parameters are passed to Spring boot application through the
"application.properties" file or "bootstrap.properties" file.
The use of such properties file poses a security risk, if secret data is directly mentioned in the file.
Spring Boot Vault addresses this risk.
It pulls secret information from vault and supplies to the application at the start-up time.
The .properties file will only tell the application the names of parameters that it can expect from Vault.
The actual values of the parameters will be taken from vault.
How to setup Vault?
Step 1: Install and launch HashiCorp Vault from
https://www.vaultproject.io/downloads.html:
Step 2: After installing Vault, test whether it works, by launching
it in a console window.
> vault server --dev --dev-root-token-id="spring-boot-vault-demo"
==> Vault server configuration:
Api Address: http://127.0.0.1:8200
Cgo: disabled
Cluster Address: https://127.0.0.1:8201
Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
Log Level: info
Mlock: supported: false, enabled: false
Recovery Mode: false
Storage: inmem
Version: Vault v1.4.1
WARNING! dev mode is enabled!
.....
You may need to set the following environment variable:
PowerShell:
$env:VAULT_ADDR="http://127.0.0.1:8200"
cmd.exe:
set VAULT_ADDR=http://127.0.0.1:8200
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: +Dihvgj/oRN2zo6/97ZqpWt086/CFRZEPkuauDu4uQo=
Root Token: spring-boot-vault-demo
Step 3: Store some secret data in the vault,
by running these commands in a separate command window:
> set VAULT_ADDR=http://127.0.0.1:8200
> set VAULT_TOKEN=spring-boot-vault-demo
> vault kv put secret/spring-boot-vault-demo password=££££$$$$%%%%
Key Value
--- -----
created_time 2020-05-02T09:59:41.2233332Z
deletion_time n/a
destroyed false
version 1
I did this:
I made a shell script called setenv.sh and put this under it:
#!/bin/bash
launchctl setenv VAULT_ENABLE true
launchctl setenv VAULT_SCHEME http
launchctl setenv VAULT_HOST vault-1.dev.lokal
launchctl setenv VAULT_PORT 8200
launchctl setenv VAULT_TOKEN 5F97X
And then, before starting the application I ran the shell script with
sudo sh setenv.sh
And the application seems to work fine without any errors. Strangely if I do it with my previous approach of adding the env variables inside the .bash_profile, it doesn't work.
Related
we have defined the following in .ini file:
Assigned Values:
opendistro_security.ssl.http.enabled= True
Kesystore type = PKCS12
keystore_filepath = /relativepath.pfx
truststore_type = PKCS12
truststore-filepath =/relativepath.pfx
opendistro_security.ssl.transport.keystore_password=""
opendistro_security.ssl.transport.truststore_password=""
We are getting below error message when we execute .Net application:
Error:
Caused by: org.elasticsearch.ElasticsearchSecurityException: Error while initializing transport SSL layer: java.io.IOException: keystore password was incorrect
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
Resolutions we tried:
Changed different passwords of Pfx as well as defined in setting_override.ini (eg. “changeit”). Also added Pfx file to respective directory.
Checked PFX file by both import/ export by creating new password and also added that in MMC and checked.
Added default password for Truststore type and Keystore Type as per the this URL
Placed Pfx file inside config folder of Elastic Search
But still we are facing issue.
In this URL they have mentioned the following settings such as:
ELASTIC_PASSWORD=password
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.keystore.path=/usr/share/elasticsearch/config/elastic-certificates.p12
- xpack.security.http.ssl.truststore.path=/usr/share/elasticsearch/config/elastic-certificates.p12
- xpack.security.http.ssl.keystore.password=password
- xpack.security.http.ssl.truststore.password=password
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.keystore.path=/usr/share/elasticsearch/config/elastic-certificates.p12
- xpack.security.transport.ssl.truststore.path=/usr/share/elasticsearch/config/elastic-certificates.p12
- xpack.security.transport.ssl.keystore.password=password
- xpack.security.transport.ssl.truststore.password=password
Queries:
Do we need to mention all this settings for our application?
Is the above setting applicable for XPack (or) it will get applicable for Open Distro also?
Is elastic search will have separate password? If so where can we check password related to Elastic Search?
Do we need to assign same password of Elastic search to Truststore type and Key store type?
I am running a stack of services/applications and would like to deploy these using a single docker-compose file. I tested this and it works flawlessly. But now, when I try to make the docker-compose file more configurable by having an .env file take control of the configuration the spring boot project that has to be deployed gives me the following error:
api | Failed to bind properties under 'spring.data.mongodb.port' to java.lang.Integer:
api |
api | Property: spring.data.mongodb.port
api | Value: '27017'
api | Origin: System Environment Property "spring.data.mongodb.port"
api | Reason: failed to convert java.lang.String to java.lang.Integer (caused by java.lang.NumberFormatException: For input string: "'27017'")
The env file I am using:
SPRING_DATA_MONGODB_PORT=27017
How the back-end is receiving the configuration request in spring boot's application.properties file:
spring.data.mongodb.port=27017
The problem is clear, but how do I make sure that whenever the environment variable arrives, the '27017' in String format is casted to an Integer?
Thanks!
I am trying discovery first implementation with Spring Cloud. I am getting failure for password decryption. I have .jks in the classpath. The same works when I go for Config First approach.
Following is the bootstrap.properties file for Config first approach.
eureka.instance.hostname=claims-dev
eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/
spring.cloud.config.name=claim
spring.config.import=configserver:http://localhost:8888
spring.profiles.active=dev
spring.cloud.config.fail-fast=true
Following is the bootstrap for Discovery First approach.
eureka.instance.hostname=claims-dev
eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/
spring.cloud.config.discovery.service-id=configserver
spring.cloud.config.discovery.enabled=true
spring.cloud.config.name=claim
spring.profiles.active=dev
spring.cloud.config.fail-fast=true
For Discovery First I am getting below exception
Caused by: java.lang.UnsupportedOperationException: No decryption for FailsafeTextEncryptor. Did you configure the keystore correctly?
at org.springframework.cloud.bootstrap.encrypt.TextEncryptorUtils$FailsafeTextEncryptor.decrypt(TextEncryptorUtils.java:188) ~[spring-cloud-context-3.1.0.jar:3.1.0]
at org.springframework.cloud.bootstrap.encrypt.AbstractEnvironmentDecrypt.decrypt(AbstractEnvironmentDecrypt.java:144) ~[spring-cloud-context-3.1.0.jar:3.1.0]
... 16 common frames omitted
What I am missing here?
UPDATE: Following are the config server properties
encrypt.keyStore.location=classpath:/store name
encrypt.keyStore.password=store password
encrypt.keyStore.alias=alias
encrypt.key-store.type=jks
spring.cloud.config.server.encrypt.enabled=false
It works if I opt for server side password decryption.
I'm working on a sample application where I want to connect to the Hashicorp vault to get the DB credentials. Below is the bootstrap.yml of my application.
spring:
application:
name: phonebook
cloud:
config:
uri: http://localhost:8888/
vault:
uri: http://localhost:8200
authentication: token
token: s.5bXvCP90f4GlQMKrupuQwH7C
profiles:
active:
- local,test
The application builds properly when the vault server is unsealed. Maven fetches the database username from the vault properly. When I run the build after sealing the vault, the build is failing due to the below error.
org.springframework.vault.VaultException: Status 503 Service Unavailable [secret/application]: error performing token check: Vault is sealed; nested exception is org.springframework.web.client.HttpServerErrorException$ServiceUnavailable: 503 Service Unavailable: [{"errors":["error performing token check: Vault is sealed"]}
How can I resolve this? I want maven to get the DB username and password during the build without any issues from the vault even when though it is sealed.
It's a profit of Vault that it's not simple static storage, and on any change in the environment, you need to perform some actions to have a stable workable system.
Advice: create a script(s) for automation the process.
Example. I have a multi-services system and some of my services use Vault to get the configuration.
init.sh:
#!/bin/bash
export VAULT_ADDR="http://localhost:8200"
vault operator unseal <token1>
vault operator unseal <token2>
vault operator unseal <token3>
vault login <main token>
vault secrets enable -path=<path>/ -description="secrets for My projects" kv
vault auth enable approle
vault policy write application-policy-dev ./application-policy-DEV.hcl
application.sh:
#!/bin/bash
export VAULT_ADDR="http://localhost:8200"
vault login <main token>
vault delete <secret>/<app_path>
vault delete sys/policy/<app>-policy
vault delete auth/approle/role/<app>-role
vault kv put <secret>/<app_path> - < <(yq m ./application.yaml)
vault policy write <app>-policy ./<app>-policy.hcl
vault write auth/approle/role/<app>-role token_policies="application-policy"
role_id=$(vault read auth/approle/role/<app>-role/role-id -format="json" | jq -r '.data.role_id')
secret_id=$(vault write auth/approle/role/<app>-role/secret-id -format="json" | jq -r '.data.secret_id')
token=$(vault write auth/approle/login role_id="${role_id}" secret_id=${secret_id} -format="json" | jq -r '.auth.client_token')
echo 'Token:' ${token}
where <app> - the name of your application, application.yaml - file with configuration, <app>-policy.hcl - file with policy
Of course, all these files should not be public, only for Vault administration.
On any changes in the environment or Vault period termination just run init.sh. For getting a token for the application run application.sh. Also if you need to change a configuration parameter, change it in application.yaml, run application.sh and use result token.
Script result (for one of my services):
Key Value
--- -----
token *****
token_accessor *****
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
Success! Data deleted (if it existed) at: <secret>/<app>
Success! Data deleted (if it existed) at: sys/policy/<app>-policy
Success! Data deleted (if it existed) at: auth/approle/role/<app>-role
Success! Data written to: <secret>/<app>
Success! Uploaded policy: <app>-policy
Success! Data written to: auth/approle/role/<app>-role
Token: s.dn2o5b7tvxHLMWint1DvxPRJ
Process finished with exit code 0
I am attempting to run and test a Spring Boot application that I have packaged into a zip file and unpacked on a Linux VM. The zip contains everything the application needs (at least to my knowledge). When I attempt to execute the application, it starts but quickly fails because it cannot load a keystore needed for SSH/TLS secure communications.
I have the following in my application.yml:
server:
port: 8091
ssl:
enabled: true
protocol: TLS
trust-store-type: JKS
trust-store: classpath:keystore/server.keystore
trust-store-password: <hidden>
key-store-type: JKS
key-store: classpath:keystore/ra/server.keystore
key-store-password: <hidden>
The directory structure on the test system is as follows:
[centos#route-assessor route-assessor]$ ls -R
.:
config elastic-apm-agent-1.10.0.jar lib run-route-assessor.sh services-0.0.1-SNAPSHOT.jar
./config:
application.yml keystore log4j2.xml
./config/keystore:
mp ra rg server.keystore
./config/keystore/mp:
server.keystore
./config/keystore/ra:
server.keystore
./config/keystore/rg:
server.keystore
./lib
<dependency jars>
Here is the exception thrown:
[2019-10-23 13:21:31.419] main WARN : context.AnnotationConfigServletWebServerApplicationContext:557 - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Could not load key store 'classpath:keystore/server.keystore'
The Spring Boot "runtime" obviously sees and accesses the config directory, but doesn't seem to see the keystore directory contained within. Do I need to specify the paths differently in application.yml or do I need to put the keystore files somewhere else?
Note: I can run this application with the application.yml configured as shown from eclipse without any problem. Granted, all resources are located in src/main/resources for that situation.
UPDATE:
As per #borban's suggestion, I modified the application.yml as follows:
key-store: file:config/keystore/ra/server.keystore
trust-store: file:config/keystore/server.keystore
That seems to have solved one problem, but I'm not out of the woods yet:
[2019-10-23 15:07:17.671] main ERROR: boot.SpringApplication:821 - Application run failed
org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
...
Caused by: java.lang.IllegalStateException: no valid keystore
...
As far as I know, my keystore files are valid and correct (I've been using them on my Windows development box for months). They are copied over as part of the zip distribution. Is there something maybe I'm missing?
I'm also a little concerned with a few other messages in the log. I'm not sure if they're related, but it seems that they could be:
[2019-10-23 15:07:10.153] main WARN : resource.Resource:126 - java.lang.IllegalArgumentException: URI is not hierarchical
[2019-10-23 15:07:10.155] main WARN : resource.Resource:126 - java.lang.IllegalArgumentException: URI is not hierarchical
I don't recall seeing them before.
Are you are trying to access this keystore outside of the classpath and from the filesystem itself? From the folder structure you are giving, that seems to be the case. If you remove the "classpath" prefix and then update your path appropriately, it should work.
From this post, it looks like you have to prefix with "file"
https://maven.apache.org/plugins/maven-resources-plugin/examples/resource-directory.html
You can use maven resource plugin to add the folders you need to the plugin:
https://maven.apache.org/plugins/maven-resources-plugin/examples/resource-directory.html
then use without classpath??