I'm trying to create a Spring Boot application that regularly fetch data from AWS S3.
The AWS S3 credentials are fetched from Vault using Spring Cloud Vault when the application start.
My issue is that AWS S3 credentials have a limited lifespan due to Vault policy so I have to restart my application from time to time to obtain new credentials from Vault
Is there a way to automatically restart bean using those credentials?
TL;DR
No, there is no automatism, but you can do this yourself.
The longer read
Spring Boot and Spring Cloud aren't really intended for applying continuous updates to the configuration without interruption. Spring Cloud Config ships with Refresh Scope support that allows to annotate beans with #RefreshScope and trigger a refresh of the beans that get re-initialized. This approach requires either integration with a message bus or triggering the refresh endpoint.
The other alternative, which is limited to AWS functionality, is providing an own AWSCredentialsProvider implementation that is backed by a Vault PropertySource that applies rotation to your credential. This requires you to provide a bit of code that integrates with VaultConfigurer or even directly via SecretLeaseContainer to get secret lifecycle event callbacks. See here for an integration example.
There is a ticket asking for the same question that contains background why this pattern isn't widely applicable.
Related
I am confused about the use case where HashiCorp Vault is used to provide database secrets dynamically for Spring Boot. Lets say you have two microservices: one containing the application logic and one running a database engine. The first obviously needs to authenticate towards the database and this is where dynamic secrets come into play. Vault can provide such credentials to the first microservice so you don't have to use e.g. ENV variables in a docker-compose file managing both microservices.
The App could be a Spring Boot microservice relying on Spring Cloud Vault to handle communication with HashiCorp Vault for credentials management. The microservice asks Vault for temporary db credentials (in this case they last for one hour) when it is started. During this one hour interval, the app can connect to the database and do whatever needs to be done. After one hour, the credentials expire and no communications is allowed.
The Spring Boot Cloud Vault documentation mentions
Spring Cloud Vault does not support getting new credentials and configuring your DataSource with them when the maximum lease time has been reached. That is, if max_ttl of the Database role in Vault is set to 24h that means that 24 hours after your application has started it can no longer authenticate with the database.
In other words, after one hour, the connection is lost and there seems to be no other way to get new db credentials other then by restarting the microservice.
So if have the following questions:
What is the added value of using Vault in this particular example if you are (seemingly) forced to restart your entire application each time the TTL expires?
Does the same apply when you use static secrets instead?
Can this issue be solved without changing microservice code? (K8S, Istio, etc.?)
My guess is the intended use of Vault with Spring Boot is different compared to my understanding.
This article describes 4 possible solutions to mitigate the issue described in the question. Being valid approaches to solve the problem, a more generic (referring to the 'heavy rotation of dynamic secrets'-approach) and less aggressive (referring to the 'restarting the service when connectivity is lost'-approaches) should be in place.
Currently in our project we already implemented firebase messaging service(FCM).We already have service account created for this. Now we need to implement a pub/sub with different google and service account.
When I try to implement this its taking default credentials.
How can we configure different service account credentials for FCM and pub/sub?
Kindly let me know how can we fix this.
default credentials
Dependencies added
Error I am facing
To explicitly provide credentials for Spring Cloud GCP Pub/Sub, use the spring.cloud.gcp.pubsub.credentials.location or spring.cloud.gcp.pubsub.credentials.encoded-key property.
Documentation available here.
The error you have is unrelated to GCP authentication, though -- the issue is that two different starters are defining a Jwt parsing bean. If you don't need to extract identity from Firebase, then it can be turned off with spring.cloud.gcp.security.firebase.enabled=false. If you do need it, and com.magilhub is under your control, follow the Spring Boot suggestion and use #Qualifier to get the specific one you need.
Spring cloud vault picks up the latest secret value during application start. If the secret is updated when the application is already up and running, then it is not picked up.
I understand this is a Spring config limitation and there is a workaround with the #RefreshScope annotation and explicitly invoking the /actuator/refresh API.
Is there any other mechanism where the application can listen to secret update notifications and automate the refresh?
From the debug logs, I see that the spring cloud vault returns an updated secret with the GET call based on "min-renewal" time.
Got 'refresh' working with:
#Autowired
ContextRefresher contextRefresher;
public void refreshContext() {
contextRefresher.refresh();
}
I am still trying to understand why spring cloud vault invokes vault login and Get API every 10 seconds if the refresh is not automated.
I'm using spring cloud AWS secrets manager support to load in configuration defined by terraform which creates the application secret defaults.
Once adding a policy statement to the services accessing the secret I run into spring not starting as it's attempting to read all kinds of secrets for profiles that do not exist in secrets manager.
How can I restrict the spring cloud secrets manager support to only read secrets I have explicitly granted access without needing to create empty secrets for every profile?
This is not possible yet unfortunately. We have pull request that enables skipping loading profiles that will likely be merged in 2.3 and we are re-thinking Secrets Manager integration for 3.0.
I have been using the Spring cloud Service connectors for Pivotal cloud foundry for a long time which gets the connection details from the VCAP_SERVICES env variable. Now we have a requirement to read these credentials from Vault . I am just curious , Can I still continue to use the Service binding approach with spring cloud connector ? I would assume we don't want to expose these credentials from vault to an VCAP_SERVICES variable which defeat the purpose of the vault. Has there been any enhancements in Spring cloud connectors to read the credentials directly from Vault rather than depending the VCAP_SERVICES env variable or should I resort back to the Spring boot's default Application Properties based approach instead of the service binding approach using cloud connectors ?
The Spring Cloud Connectors project is now in maintenance mode, in favor of the newer Java CFEnv project. However, Java CFEnv is also very specific to Cloud Foundry's VCAP_SERVICES model of exposing service bindings and won't help you if the service connection info is in Vault.
I would suggest that you fall back to the Spring Boot properties-based approach using Spring Cloud Vault or Spring Cloud Config Server's Vault integration to automate fetching the properties from Vault and making them available as Spring Boot properties.