Kong lambda plugin - Correlation id being dropped - aws-lambda

so I'm using kong as an API gateway. I have a simple lambda function on aws returning headers. I'm able to see a response from the aws lambda function, however there is no correlation id. I have set the correlation-id plugin global. Would like some clarification on why the lambda plugin is not adding the correlation-id in the headers of the original response.
exports.handler = async (event) => {
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify(event.request_headers),
};
return response;
};
My kong.yaml
_format_version: "1.1"
routes:
- name: lambda1
paths:
- /lambda1
path_handling: v0
preserve_host: false
protocols:
- http
- https
regex_priority: 0
strip_path: true
https_redirect_status_code: 426
plugins:
- name: aws-lambda
config:
aws_key: XXXXXXXXXXXXXXXXXXXXXXX
aws_region: us-east-1
aws_secret: XXXXXXXXXXXXXXXXXXXXXXXXx
awsgateway_compatible: false
forward_request_body: true
forward_request_headers: true
forward_request_method: false
forward_request_uri: false
function_name: kong-lambda-plugin
host: null
invocation_type: RequestResponse
is_proxy_integration: false
keepalive: 60000
log_type: Tail
port: 443
proxy_scheme: null
proxy_url: null
qualifier: null
skip_large_bodies: true
timeout: 60000
unhandled_status: null
enabled: true
protocols:
- grpc
- grpcs
- http
- https
plugins:
- name: correlation-id
config:
echo_downstream: false
generator: uuid#counter
header_name: correlation-id
enabled: true
protocols:
- grpc
- grpcs
- http
- https

So what worked for me was
https://github.com/Kong/priority-updater
I created the .rock file generated by the script above.
Since I'm using a dockerized version of Kong, I coppied the file over to my Kong docker host.
Installed the plugin in my Kong docker host using luarocks command.
Added the plugin to my 'kong.conf' file.
Added it to the lambda route and worked beautifully.

You are right the problem comes from the plugin priority
CorrelationIdHandler.PRIORITY = 1
AWSLambdaHandler.PRIORITY = 750
Aws Lambda plugin is breaking the chain of plugins as in the handler phase it is doing
return kong.response.exit(status, content, headers)
So the other plugins cannot been used.
You can create a custom correlationId plugin and change the PRIORITY there is a tool to handle that for you https://github.com/Kong/priority-updater
You can also create a route to add the correlation id calling another route doing the lambda call

Related

How to pass api endpoint parameter to lambda function in serverless

I created a lambda function behind an API gateway, and I am trying to implement a healthcheck for it as well, but right now I need to do this in 2 steps:
run serverless deploy so it spits out the endpoint for the api gateway.
manually insert the endpoint into the healthcheck.environment.api_endpoint param, and run serverless deploy a second time
functions:
app:
handler: wsgi_handler.handler
events:
- http:
method: ANY
path: /
- http:
method: ANY
path: '{proxy+}'
healthcheck:
environment:
api_endpoint: '<how to reference the api endpoint?>'
handler: healthcheck.handler
events:
- schedule: rate(1 minute)
custom:
wsgi:
app: app.app
pythonBin: python3
packRequirements: false
pythonRequirements:
dockerizePip: non-linux
Is there a way to get the reference to the api gateway on creation time, so it can be passed as an environment variable to the healthcheck app? the alternative I can think of, is to basically create a specific serverless.yaml just for the healthcheck purpose.
I noticed I can reconstruct the endpoint in the lambda, and grab the id like so:
healthcheck:
environment:
api_endpoint: !Ref ApiGatewayRestApi
region: ${env:region}
handler: healthcheck.handler
events:
- schedule: rate(1 minute)
and then reconstruct:
def handler(event, context):
api_id = os.environ['api_endpoint']
region = os.environ['region']
endpoint = f"https://{api_id}.execute-api.{region}.amazonaws.com/dev"
With a bit of cloudformation you can inject it directly! That way you don't need to compute it everytime in your lambda handler.
api_endpoint: {
'Fn::Join': [
'',
[
{ Ref: 'WebsocketsApi' },
'.execute-api.',
{ Ref: 'AWS::Region' },
'.',
{ Ref: 'AWS::URLSuffix' },
"/${opt:stage, self:provider.stage, 'dev'}",
],
],
},
(this is an example for a websocket API)
Is your API created through the same/another Cloudformation Stack? If so, you can reference is directly (same stack) or through a CloudFormation variable export.
https://carova.io/snippets/serverless-aws-cloudformation-output-stack-variables
https://carova.io/snippets/serverless-aws-reference-other-cloudformation-stack-variables
If you created it outside of CloudFormation, ie. in the aws console, then you'll need to add the ids into the template. Most likely by creating different environment variables based on the stage.

Envoy lua filter - http call cluster invalid. Must be configured

I am trying to create an envoy filter for all of my microservices deployed on a GKE cluster which is running istio.
In the filter I want to read 2 header values and send a request to an external service on web to get list of all groups that the user has authority of.
When I send the request I get "http call cluster invalid. Must be configured" error.
As per my understanding we need to register the host as cluster so that envoy understand where to send the request and I have added a value in "clusters" section of my filter.yaml file.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: authorization-filter
namespace: default
spec:
filters:
- listenerMatch:
listenerType: SIDECAR_INBOUND
listenerProtocol: HTTP
filterName: envoy.lua
filterType: HTTP
filterConfig:
inlineCode: |
function envoy_on_request(request_handle)
request_handle:logWarn("Inside filter")
local token = request_handle:headers():get("Authorization")
local partition = request_handle:headers():get("partition-id")
if token == nil or partition == nil then
request_handle:logErr("Token or Partition is empty")
request_handle:respond({[":status"] = "400",["Content-Type"] = "application/json"}, "{\"ErrorMessage\":\"Bad Request: Authorization or Partition-Id missing in headers\"}")
end
request_handle:logWarn("Sending request")
local headers, body = request_handle:httpCall(
"lua_cluster",
{
[":method"] = "GET",
[":authority"] = "example.com",
[":path"] = "/api/authorization/groups",
["Authorization"] = token,
["partition-id"] = partition
},
nil,
5000)
request_handle:headers():add("authorization-response-headers", headers)
end
clusters:
- name: lua_cluster
connect_timeout: 0.5s
type: strict_dns
lb_policy: round_robin
hosts:
- socket_address:
address: "example.com"
port_value: 8888
What am i missing here?

How can i use multiple path parameters from serverless framework

I'm trying to deploy my serverless app.
But have a problem like below.
An error occurred: ApiGatewayResourceServicesServiceidVar - A sibling ({id}) of this resource already has a variable path part -- only one is allowed
And below is my code.
updateApplication:
handler: handler.updateApplication
memorySize: 3008
description: Update application
timeout: 30
events:
- http:
path: services/{serviceId}/applications/{applicationId}
method: post
cors: true
authorizer: authorize
request:
parameters:
paths:
serviceId: true
applicationId: true
Any advice or suggestion would be appreciated. Thank you in advance.
The serverless framework seems to be complaining that you have defined the path parameters twice. Since you have declared it there directly below -http: you can remove the request: parameters: paths: block.
In other words, try this:
updateApplication:
handler: handler.updateApplication
memorySize: 3008
description: Update application
timeout: 30
events:
- http:
path: services/{serviceId}/applications/{applicationId}
method: post
cors: true
authorizer: authorize
Happy coding! 👍

AWS - API keys available on the Serverless Offline framework?

I use Serverless Offline to develop a Web project.
I need of API Keys to access to resource on Serverless AWS Lamda.
I have a serverless.yml with my service and my provider.
In Postman, I access to my route (http://127.0.0.1:3333/segments/UUID/test), and I haven't any error (as Forbidden message), the Lambda is executed...
test:
handler: src/Api/segment.test
events:
- http:
path: segments/{segmentUuid}/test
method: post
request:
parameters:
paths:
segmentUuid: true
private: true
The route in question is not protected by private.
https://www.npmjs.com/package/serverless-offline#token-authorizers
Serverless-offline will emulate the behaviour of APIG and create a
random token that's printed on the screen. With this token you can
access your private methods adding x-api-key: generatedToken to your
request header. All api keys will share the same token. To specify a
custom token use the --apiKey cli option.
Command will look like this:
sls offline --apiKey any-pregenerated-key
For local dev use this inside serverless.yml:
custom:
serverless-offline:
apiKey: 'your-key-here'
Or this inside serverless.ts:
custom: {
'serverless-offline': {
apiKey: 'your-key-here',
},
},
Given latest changes this configuration worked for me with serverless offline:
provider: {
name: 'aws',
region: region,
runtime: 'nodejs14.x',
stage: stage,
apiGateway:{
apiKeys: [{
name: 'test name',
value: 'sadasfasdasdasdasdafasdasasd'
}],
},
},
https://github.com/dherault/serverless-offline/issues/963

Multiple paths in http module - metricbeats

I am using http module of metricbeats to monitor jmx. I am using http module instead of the jolokia module because it lacks wildcard support at this point. The example configuration in the documents is as follows.
- module: http
metricsets: ["json"]
period: 10s
hosts: ["localhost:80"]
namespace: "json_namespace"
path: "/jolokia/"
body: '{"type" : "read", "mbean" : "kafka.consumer:type=*,client-id=*", "attribute" : "count"}'
method: "POST"
This works fine and I am able to get data to kibana. I see errors when I configure it as follows to call multiple paths.
- module: http
metricsets: ["json"]
enabled: true
period: 10s
hosts: ["localhost:80"]
namespace: "metrics"
method: POST
paths:
- path: "/jolokia/"
body: '{"type" : "read", "mbean" : "kafka.consumer:type=*,client-id=*", "attribute" : "bytes-consumed-rate"}'
- path: "/jolokia/"
body: '{"type" : "read", "mbean" : "kafka.consumer:type=*,client-id=*", "attribute" : "commit-latency-avg"}'
This does not seem to be the right config and I see that the http events have had failures.
2018/02/26 19:53:18.315740 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30000 beat.memstats.gc_next=4767600 beat.memstats.memory_alloc=4016168 beat.memstats.memory_total=47474256 libbeat.config.module.running=3 libbeat.output.read.bytes=4186 libbeat.output.write.bytes=16907 libbeat.pipeline.clients=7 libbeat.pipeline.events.active=0 libbeat.pipeline.events.published=18 libbeat.pipeline.events.total=18 libbeat.pipeline.queue.acked=18 metricbeat.http.json.events=3 metricbeat.http.json.failures=3
Documentation on how to setup http module: Example configuration
I had to query multiple URL's of my REST API and I could achieve that by having multiple modules of "http" with different host URL's following is the example:
- module: http
metricsets: ["json"]
period: 3600s
hosts: ["http://localhost/Projects/method1/"]
namespace: "testmethods"
method: "GET"
enabled: true
- module: http
metricsets: ["json"]
period: 3600s
hosts: ["http://localhost/Projects/method2/"]
namespace: "testmethods"
method: "GET"
enabled: true
This made me achieve have multiple paths for the same module
Multiple paths are not supported by the http module's json metricset.
What you found in the config example is for the http module's server metricset. This metricset does not query URLs. Instead it opens an http server on the specified port, and can receive input on multiple paths which are used to separate data into different namespaces.

Resources