I have configs in HashiCorp vault with same names in different path.But when i try to access it, i am always end up with the config1 prop1 value is getting overridden by config2 prop1
Vault Path
path/stage/config1
prop1
path/stage/config2
prop1
Spring vault version :spring-cloud-starter-config-3.1.1
Spring boot starter version : 2.7.1
properties.yaml
spring:
application:
name: my-app
cloud:
kubernetes:
enabled: false
cloud.vault:
uri: https://vaulturi
connection-timeout: 5000
read-timeout: 15000
authentication: token
token: ${keeper.token}
namespace: name1/name2
fail-fast: true
kv:
enabled: true
backend: path/stage
default-context: config1
config:
import: vault://path/stage/config1,vault://path/stage/config2
app:
prop1:{$(prop1)}
Can i access prop1:{$(prop1)} like prop1:{$(config1.prop1)}
When i check the value in actuator/env, i get the following response
{
"name": "path/stage/config1",
"properties": {
"prop1": {
"value": "test1"
}
}
},
{
"name": "path/stage/config2",
"properties": {
"prop1": {
"value": "test2"
}
}
}
Can some one help me to fix this
Thanks
Arun
Try to import vault://path/stage and then you have config1.prop1 and config2.prop1
You can then map them to specific properties.
Related
In my application we are making two calls from my app for getting secrets from Vault, as shown below:
Login to Vault : POST call to https::/v1/auth/approle/login -- It will take role_id and secret_id as payload and response will be client_token.
Fetch secrets : GET call to https::/v1/secret/data/abc/dev/xyz.json -- It will take headers as X-Vault-Token and X-Vault-Namespace and it will give you the response as below:
{
"request_id": "......",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"data": {
"name": "ABC"
},
"metadata": {
"created_time": "...",
"deletion_time": "",
"destroyed": false,
"version": 1
}
}
Now I want to use Spring Cloud Vault Dependency to make things work through it. Please provide me the proper illustrations to make this work?
Assuming you are running spring boot and have a working Vault server configured for your app.
Add spring cloud vault maven dependency
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
Add vault configuration to bootstrap.yaml
spring:
application:
name: abc
cloud:
vault:
host: <vault-server-hostname>
port: <vault-server-port>
scheme: HTTPS
namespace: <name-of-vault-namespace>
authentication: APPROLE
app-role:
role-id: <your-application-role-id>
secret-id: <your-application-secret-id>
role: <your-application-role>
If you run your app with spring profiles, like dev, it will be picked up and added to the vault path.
Now you should be able to inject secrets stored on the path secret/data/abc/dev with #Value("${<name-of-property>}
what I'm trying
Passing binary data via Lambda integration in API gateway. Lambda returns back text.
issue
The function returns desired output when API gateway is configured from console. To implement it using serverless framework I installed serverless-apigw-binary plugin. The required binary types show up in API gateway>settings>binary media types. However on calling API I get "internal server error". The function works properly on application/json type input. After enabling-disabling lambda proxy integration and adding mappings via console, I get correct output.
serverless.yml file
org: ------
app: ---------
service: ---------
frameworkVersion: ">=1.34.0 <2.0.0"
plugins:
- serverless-python-requirements
- serverless-offline
- serverless-apigw-binary
provider:
name: aws
runtime: python3.7 #fixed with pipenv
region: us-east-1
memorySize: 128
timeout: 60
profile: ----
custom:
pythonRequirements:
usePipenv: true
useDownloadCache: true
useStaticCache: true
apigwBinary:
types: #list of mime-types
- 'application/octet-stream'
- 'application/zip'
functions:
main:
handler: handler.main
events:
- http:
path: ocr
method: post
integration: lambda
request:
passThrough: WHEN_NO_TEMPLATES
template:
application/zip: '
{
"type": "zip",
"zip": "$input.body",
"lang": "$input.params(''lang'')",
"config": "$input.params(''config'')",
"output_type": "$input.params(''output_type'')"
}'
application/json: '
{
"type": "json",
"image": $input.json(''$.image''),
"lang": "$input.params(''lang'')",
"config": "$input.params(''config'')",
"output_type": "$input.params(''output_type'')"
}'
application/octet-stream: '
{
"type": "img_file",
"image": "$input.body",
"lang": "$input.params(''lang'')",
"config": "$input.params(''config'')",
"output_type": "$input.params(''output_type'')"
}'
handler.py
def main(event, context):
# do something on event and get txt
return txt
edit
I compared swagger definitions and found this
1. API generated from console(working)
paths:
/ocr:
post:
consumes:
- "application/octet-stream"
produces:
- "application/json"
responses:
API generated from serverless framework
paths:
/ocr:
post:
consumes:
- "application/x-www-form-urlencoded"
- "application/zip"
- "application/octet-stream"
- "application/json"
responses:
produces: - "application/json" is missing. How do I add it in serverless?
How should I configure the Micronaut to get the /metrics in the Prometheus format ?
Used: micronaut 1.0.0.M3
Now:
micronaut:
...
metrics:
enabled: true
export:
prometheus:
enabled: true
and result: metrics name list
{"names":["jvm.memory.max","executor.pool.size"...]}
I need to get: metrics in the prometheus format(formats)
Micronaut Micrometer has an PrometheusEndpoint from version 1.1 that will
return in Prometheus format from /prometheus and
can be enabled in application.yml by:
endpoints:
prometheus:
sensitive: false
In combination with
micronaut:
metrics:
enabled: true
export:
prometheus:
enabled: true
step: PT1M
descriptions: true
(The documentation is missing the endpoint config but will be changed in the new release)
To piggyback on the other answers, here is an Micronaut endpoint that provides the Prometheus metrics in the format we needed:
package your.package.name
import io.micrometer.prometheus.PrometheusMeterRegistry
import io.micronaut.management.endpoint.annotation.Endpoint
import io.micronaut.management.endpoint.annotation.Read
#Endpoint(id = "prometheus", value = "/prometheus", defaultEnabled = true,
defaultSensitive = false)
class PrometheusController(val prometheusMeterRegistry: PrometheusMeterRegistry) {
#Read
fun scrape(): String {
return prometheusMeterRegistry.scrape()
}
}
At the moment, we solved the problem as follows.
Added a new endpoint. Or create a controller with a mapping on /metrics.
The new endpoint added a return of scrape().
Correlated endpoint with /prometheus (new endpoint can not be mapped on /metrics).
Disconnected endpoint metrics which by default.
Config:
micronaut:
...
metrics:
enabled: true
export:
prometheus:
enabled: true
...
endpoints:
...
metrics:
enabled: false
prometheus:
enabled: true
Haven't tested this out but based on the following:
https://github.com/micronaut-projects/micronaut-core/blob/master/configurations/micrometer-registry-prometheus/src/main/java/io/micronaut/configuration/metrics/micrometer/prometheus/PrometheusMeterRegistryFactory.java
Your yaml should look like
metrics:
prometheus:
enabled: true
don't believe the export comes into play.
This is my bootstrap.yml content file:
server.port: 8888
spring:
application:
name: configserver
profiles:
active: native, git, vault
cloud:
config:
enabled: false
server:
native:
searchLocations: classpath:config/
# searchLocations: file://${native_location}
order: 3
git:
uri: file:///home/jcabre/projects/wsec-sccs/server/repo
order: 2
vault:
host: ${vault_server_host:localhost}
port: ${vault_server_port:8200}
scheme: ${vault_server_scheme:https}
backend: ${vault_backend:configserver}
profileSeparator: /
order: 1
As you can see I've stand up three backends: native, git, vault.
So classpath:/config/application.yml content:
foo: FROM NATIVE APPLICATION
/home/jcabre/projects/wsec-sccs/server/repo/application.yml content:
foo: FROM GIT
And Vault:
$ vault kv get configserver/configclient/
=== Data ===
Key Value
--- -----
foo FROM VAULT
$vault kv get configserver/configclient/dev
=== Data ===
Key Value
--- -----
foo FROM DEV VAULT
When I try to get foo config key using curl:
$ curl -sS -X GET http://localhost:8888/configclient/default -H "X-Config-Token: ${vault_token}" | jq .
{
"name": "configclient",
"profiles": [
"default"
],
"label": null,
"version": null,
"state": null,
"propertySources": [
{
"name": "vault:configclient",
"source": {
"foo": "FROM VAULT"
}
},
{
"name": "file:///home/jcabre/projects/wsec-sccs/server/repo/application.yml",
"source": {
"foo": "FROM GIT APPLICATION"
}
}
]
}
I only get git and vault property sources, but it doesn't send me native.
How can this be happening?
Any ideas?
Not sure if you ever got an answer to this, but I had a similar problem (no native profile when Vault was enabled) so I looked through the code (latest in GitHub).
It would appear that the NativeEnvironmentRepository is only enabled if the native profile is present AND no other environment repositories are configured. So it doesn't look like you are able to do what you want in the question.
My cloud config server was returning the property files but now I am seeing the below error. Please can you let me know how this can be fixed?
This is deployed in pivotal cloud foundry environment.
{
"timestamp": 1464375520539
"status": 404
"error": "Not Found"
"exception": "org.springframework.cloud.config.server.environment.NoSuchLabelException"
"message": "No such label: master"
"path": "/couchbase-data/dev"
}
application.yml
---
spring:
cloud:
config:
server:
git:
uri: https://github.company.com/username/ordering-properties
username: username
password: "{cipher}03f0ac5cc43d913bbd45155f852d1e5c88542878491a1fc89185feea93a40084"
search-paths: couchbase-data
security:
basic:
enabled: true
user:
name: ordering_config
password: "{cipher}dc56acf65f93b5485c87de1a9965e76a2d0b642a0839027deffdbc35f922746f"
manifest.yml
---
name: orderingconfigserver
memory: 2048M
instances: 1
timeout: 180
env:
ENCRYPT_KEY: ORDERING
After I deploy the app , the first hit to the endpoint returns the below error :
{
"timestamp": 1464377154415
"status": 500
"error": "Internal Server Error"
"exception": "java.lang.IllegalStateException"
"message": "Cannot clone or checkout repository"
"path": "/couchbase-data/dev"
}
If your git repo has the main branch name as "main" instead of "master", I would recommend you to add a new property to change default-label as below:
spring.cloud.config.server.git.default-label=main
Check this link for additional info.