so I am building a SpringBootApplication and I currently have the following settings in the yml:
my:
caller:
- id: someId
url: someUrl
context: someContext
- id: someId2
url: someUrl2
context: someContext2
So basically it's a list of objects (I've created a POJO which has the 3 fields). Those objects are represented as a List in another class which is a #Component and has a #ConfigurationProperties(prefix = "my") and the List of POJOs is with #NestedConfigurationproperty.
So far everything works perfectly, Spring constructs the objects from the application.yml and fills the List just fine, however I can't simulate this configuration in CloudFoundry. I'm using a different manifest.yml for cloudfoundry, and I'm trying to place this structure in the env part of the manifest but it's not working (Cloudfoundry ignores it).
It looks something like this:
env:
my:
caller:
- id: someId
url: someUrl
context: someContext
- id: someId2
url: someUrl2
context: someContext2
I know with simple lines it works (for example ev_my_caller_id) but that way I can construct only 1 object, not a whole list and I haven't found anything in the CloudFoundry doc on how to construct multiple objects from the manifest.
If anyone has any ideas I would be really grateful!
The env block of the Cloud Foundry cli's manifest.yml file has a specific format. You cannot use an arbitrary structure and expect it to work.
The format is:
env:
var_name_1: val_1
var_name_2: val_2
See docs for more details -> https://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#env-block
If you set the env variable with the correct name, you can use that to override certain values in Spring. Thanks to Spring Boot's external configuration support. Maybe that would be an option for you here.
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
Ex: MY_PROPERTY=1234 would override my.property in application.properties.
You could use the SPRING_APPLICATION_JSON environment variable mentioned in the Spring Boot external config link that Daniel provided.
In your case, that would like something like this:
env:
SPRING_APPLICATION_JSON: '{ "my": { "caller": [{ "id": "someId", "url": "someUrl", "context": "someContext" }, { "id": "someId2", "url": "someUrl2", "context": "someContext2" }]}}'
Not sure what you are trying to do, but you can try something like this:
env:
test: '{ whatever complex object/variables }'
e.g.
env:
test: '{ variable = value, variable2 = value2 }'
Related
I am trying to read an environment variable is being set from within Gitlab configuration when the application is being built, I am doing this for achieving that purpose:
I setup a variable in the application.properties.yml of my spring boot app:
sf:
apiKey: $SF_API_KEY
In the .gitlab-ci.yml, I defined the variable to be set, as follows:
variables:
SF_API_KEY: $SF_API_KEY
all I want, is to be able to read that variable from within one of my services, as the below code depicts:
#Service
class MyService(#Value("\${sf.apiKey}") val apiKey: String)
{
fun doSomething(){
//i am seeing the variable is being set by gitlab in the build logs but
// it is not being read here properly
var result = apiKey;
logger.info { "***check apiKey: $apiKey" }
//This line lgs $SF_API_KEY as a value of my variable, but not the
// real value
}
}
Am I doing something wrong? I would appreciate any help.
Try (note the {} around SF_API_KEY):
sf:
apiKey: ${SF_API_KEY}
Take a look at the docs where this placeholder notation is detailed.
Looking at the documentation here https://serverless.com/framework/docs/providers/aws/events/apigateway/#custom-response-templates, there doesn't seem to be much detail at all about setting up these templates.
I'm currently looking to remove the default application/json content-type, which is generated during creation of the handler's integration response (pictured below), and replace it with text/html. Is there defined syntax for how to do this hidden somewhere? Or is this level of customization not possible within the current scope of the framework?
Here is my endpoint as defined in serverless.yml
events:
- http:
method: any
path: /
integration: lambda
request:
region: ${env:AwsRegion}
response:
headers:
Content-Type: "'text/html'"
template: $input.path('body')
- http:
method: any
path: /{proxy+}
Which produces the following configuration on AWS Api Gateway:
I did try modifying the specification like this, as a guess, but it threw a syntax error:
template:
text\html: $input.path('body')
Make It Work
It looks like this isn't really supported by the framework, but it can be hacked together by (ab)using statusCodes in your serverless template.
Moving the response template to under a status code, and providing a pattern for that status code, accomplishes what I think you are after. The syntax:
- http:
method: any
path: /
integration: lambda
request:
region: us-east-1
response:
headers:
Content-Type: "'text/html'"
statusCodes:
200:
pattern: ""
template:
text/html: $input.path('body')
Note: Both the pattern and the template must be present.
Is It Really That Bad?
That's up to you, ultimately. I'm calling it a hack because:
It would be much nicer to be able to supply this at the response.template level, rather than at response.statusCodes.200.template.
Specifying one or more statusCodes removes the set of default lambda error regexes that you get when you don't specify any.
It feels like working around the fact that response.template will only accept a string, whereas it could (should?) accept a string or an object (like it does under statusCodes).
Fix It?
The offending code, from /lib/plugins/aws/package/compile/events/apiGateway/lib/method/integration.js:
if (http.response.template) {
_.merge(integrationResponse.ResponseTemplates, {
'application/json': http.response.template,
});
}
if (config.template) {
const template =
typeof config.template === 'string'
? { 'application/json': config.template }
: config.template;
_.merge(integrationResponse.ResponseTemplates, template);
}
I think for this to work under response.template, the code in the first if() would need to behave more like the code in the second if().
I am following Google's tutorial on setting up Google Cloud endpoint (not AWS API Gateway) in front of my Cloud Function. I am triggering my Cloud Function to trigger an AWS lambda function, AND I am trying to pass a path parameter from my Endpoint as defined by OpenAPI spec.
Path parameters are variable parts of a URL path. They are typically used to point to a specific resource within a collection, such as a user identified by ID. A URL can have several path parameters, each denoted with curly braces { }.
paths: /users/{id}:
get:
parameters:
- in: path
name: id # Note the name is the same as in the path
required: true
schema:
type: integer
GET /users/{id}
My openapi.yaml
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: HOST
x-google-endpoints:
- name: "HOST"
allowCors: "true
schemes:
- https
produces:
- application/json
paths:
/function1/{pathParameters}:
get:
operationId: function1
parameters:
- in: path
name: pathParameters
required: true
type: string
x-google-backend:
address: https://REGION-FUNCTIONS_PROJECT_ID.cloudfunctions.net/function1
responses:
'200':
description: A successful response
schema:
type: string
The error I get when I use Endpoint URL https://REGION-FUNCTIONS_PROJECT_ID.cloudfunctions.net/function1/conversations is a TypeError from my AWS lambda function
StatusCode:200, FunctionError: "Unhandled", ExecutedVersion: "$LATEST". Payload: "errorType":"TypeError", errorMessage:"Cannot read property 'startsWith' of undefined..."
It is saying that on line
var path = event.pathParameters;
...
...
if (path.startsWith('conversations/'){...};
my path var is undefined.
I initially thought my Google Function was not correctly passing pathParameters but when I tested my Google function using triggering event {"pathParameters":"conversations"}, my Lambda returns the payload successfully.
My Google Cloud Function:
let AWS = require('aws-sdk');
AWS.config.update({
accessKeyId: 'key',
secretAccessKey: 'secret',
region: 'region'
})
let lambda = new AWS.Lambda();
exports.helloWorld = async(req,res) => {
let params = {
FunctionName:'lambdafunction',
InvocationType: "RequestRespone",
Payload: JSON.stringify(req.body)
};
res.status(200).send(await lambda.invoke(params, function(err,data){
if(err){throw err}
else{
return data.Payload
}
}).promise());
}
EDIT 1:
Seeing this Google Group post, I tried adding to my openapi.yaml file path_translation: APPEND_PATH_TO_ADDRESS, yet still I'm having no luck.
...
paths:
/{pathParameters}:
get:
...
x-google-backend:
address: https://tomy.cloudfunctions.net/function-Name
path_translation: APPEND_PATH_TO_ADDRESS
#Arunmainthan Kamalanathan mentioned in the comments that testing in AWS and Google Cloud directly with trigger event {"pathParameters":"conversations"} is not equivalent to passing req.body from my Google function to AWS lambda. I think this is where my error is occurring -- I'm not correctly passing my path parameter in the payload.
EDIT 2:
There is this Stackoverflow post concerning passing route parameters to Cloud Functions using req.path. When I console.log(req.path) I get / and console.log(req.params) I get {'0': '' }, so for some reason my path parameter is not getting passed correctly from Cloud Endpoint URL to my Google function.
I was running into the same issue when specifying multiple paths/routes within my openapi.yaml. It all depends on where you place the x-google-backend (top-level versus operation-level). This has implications on the behaviour of the path_translation. You could also overwrite path_translation yourself, as the documentation clearly describes:
path_translation: [ APPEND_PATH_TO_ADDRESS | CONSTANT_ADDRESS ]
Optional. Sets the path translation strategy used by ESP when making
target backend requests.
Note: When x-google-backend is used at the top level of the OpenAPI
specification, path_translation defaults to APPEND_PATH_TO_ADDRESS,
and when x-google-backend is used at the operation level of the
OpenAPI specification, path_translation defaults to CONSTANT_ADDRESS.
For more details on path translation, please see the Understanding
path translation section.
This means that if you want the path to be appended as a path parameter instead of a query parameter in your backend, you should adhere to the following scenario's:
Scenario 1: Do you have one cloud function in the x-google-backend.address that handles all of your paths in the openapi specification? Put x-google-backend at the top-level.
Scenario 2: Do you have multiple cloud functions corresponding to different paths? Put x-google-backend at the operation-level and set x-google-backend.path_translation to APPEND_PATH_TO_ADDRESS.
When your invocation type is RequestRespone, you can access the payload directly from the event parameter of lambda.
Look at the `Payload' parameter of the Google function:
let params = {
FunctionName:'lambdafunction',
InvocationType: "RequestRespone",
Payload: JSON.stringify({ name: 'Arun'})
};
res.status(200).send(await lambda.invoke(params)...)
Also the Lambda part:
exports.handler = function(event, context) {
context.succeed('Hello ' + event.name);
};
I hope this helps.
I wanted to fetch properties from two git repos. one is https://username#bitbucket.my.domain.com/share.git - which will have a property file contains some common key value pair and the other one is https://username#bitbucket.my.domain.com/service.git - it will have property files of all the micro services.
While I am deploying the service only one yml file (which is in https://username#bitbucket.my.domain.com/share.git repo) is read by the config server. What I am missing? How to read the property file from another repo i.e. https://username#bitbucket.my.domain.com/service.git too?
I wanted to deploy the service in PCF. So I configured the config-server in PCF with the following json.
{
"count": 1,
"git": {
"label": "feature",
"uri": "https://username#bitbucket.my.domain.com/share.git",
"username": "username",
"password": "password",
"repos": {
"configserver": {
"password": "password",
"label": "feature",
"uri": "https://username#bitbucket.my.domain.com/service.git"
"username": "username"
}
}
}
}
Name of my service is LogDemo and spring profile is active. I have created two yml files and placed in the corresponding repo. (I have given same name two both the files like LogDemo-active.yml). While I am deploying the service only one yml file (which is in https://username#bitbucket.my.domain.com/share.git repo) is read by the config server. /env is giving me the following:
{
"profiles": [
"active",
"cloud"
],
"server.ports": {
"local.server.port": 8080
},
"configService:configClient": {
"config.client.version": "234e59d4a9f80f035f00fdf07e6f9f16e5560a55"
},
"configService:https://username#bitbucket.my.domain.com/share.git/LogDemo-active.yml": {
"key1": "value1",
"key2": "value2"
},
...................
...................
What I am missing? How to read the property file from other repo i.e. https://username#bitbucket.my.domain.com/service.git too?
Below is my bootstrap.yml
spring:
application:
name: LogDemo
mvc:
view:
prefix: /
suffix: .jsp
Here is my manifest file
---
inherit: baseManifest.yml
applications:
- name: LogDemo
host: LogDemo
env:
LOG_LEVEL: INFO
spring.profiles.active: active
TZ: America/New_York
memory: 1024M
domain: my.domain.com
services:
- config-server-comp
When using multiple repos, the repos that will be applied depend on the pattern's defined for those repos. The default pattern is <repo-name>/*. Thus changing the repo name to LogDemo will activate the repo for your app, because the app name, spring.application.name, is LogDemo.
If one or more patterns match, then the repo for the matched patterns will be used. If no pattern matches then the default is used.
Full details are described in the docs here.
https://cloud.spring.io/spring-cloud-config/single/spring-cloud-config.html#_pattern_matching_and_multiple_repositories
If you don't need or want the pattern matching feature, you can use the [composite backend](
https://docs.pivotal.io/spring-cloud-services/2-0/common/config-server/composite-backends.html). The composite backend allows you to define multiple Git repositories. See the first config example here.
https://docs.pivotal.io/spring-cloud-services/2-0/common/config-server/composite-backends.html#general-configuration
trying to install the approuter currently, following this tutorial:
https://blogs.sap.com/2017/07/18/step-7-with-sap-s4hana-cloud-sdk-secure-your-application-on-sap-cloud-platform-cloudfoundry/
When pushing the approuter to CF, I get an destination error:
xs-app.json/routes/0: Format validation failed (Route references unknown destination "service-destination")
This is my manifest.yml:
---
applications:
- name: xyz
command: 'node approuter/approuter.js'
host: xyz-93deb1cd-7b72-4060-94e7-21342342
path: approuter
memory: 128M
buildpack: https://github.com/cloudfoundry/nodejs-buildpack
env:
TENANT_HOST_PATTERN: 'xyz(.*).cfapps.eu10.hana.ondemand.com'
destinations: "[{"name":"service-destination", "url": "https://gfuowbasdatq19agtuthorizations-srv.cfapps.eu10.hana.ondemand.com", "forwardAuthToken": true}]"
SAP_JWT_TRUST_ACL: '[{"clientid" : "*", "identityzone" : "*"}]'
services:
- my-xsuaa
- service-destination
This is my xs-app.json:
{
"routes": [{
"source": "/",
"target": "/",
"destination": "service-destination"
}]
}
Where is the application actually searching for this destination? I created it in my CF-account as well, pointing to my service-url.
In YAML (as well as many other markup languages, e.g. JSON) double quotes inside double quotes need to be escaped. (Cf. the YAML spec section 7.3.1 and this blog post)
So for you there are two options for your destinations variable:
1. Replacing all " inside with \" (relatively cumbersome, especially if you want to change the destinations in the future)
2. Use single quotes as boundaries. So the value of your destinations variable would look like this:
'[{"name":"service-destination", "url": "https://gfuowbasdatq19agtuthorizations-srv.cfapps.eu10.hana.ondemand.com", "forwardAuthToken": true}]'