I am using Spring cloud function with AWS Lambda and API Gateway.
Below is my function:
#Bean
public Function<String, User> testFunction() {
return (input) -> {
return new User(1, input);
};
Also, I have specified the handler in AWS Lambda console as below:
org.springframework.cloud.function.adapter.aws.FunctionInvoker::handleRequest
If I send a Request with string body, it is successfully able to collect the input.
But I do not understand, how do I access the request query parameters inside the function?
Something like this:
https://[apigateway].aws.com/[path]?param1=value
Can someone guide me? Thanks in advance.
Related
I am working on Fabric8 unit test, now I am trying to create a CRD against KubernetesServer.
import io.fabric8.kubernetes.api.model.apiextensions.v1.CustomResourceDefinition;
public class TestCertManagerService {
#Rule
public KubernetesServer server = new KubernetesServer();
#Test
#DisplayName("Should list all CronTab custom resources")
public void testCronTabCrd() throws IOException {
// Given
//server.expect().get().withPath("/apis/stable.example.com/v1/namespaces/default/crontabs").andReturn(HttpURLConnection.HTTP_OK, ?????).once();
KubernetesClient client = server.getClient();
CustomResourceDefinition cronTabCrd = client.apiextensions().v1().customResourceDefinitions()
.load(new BufferedInputStream(new FileInputStream("src/test/resources/crontab-crd.yml")))
.get();
client.apiextensions().v1().customResourceDefinitions().create(cronTabCrd);
}
}
When I ran it, I got the following error
TestCertManagerService > testCronTabCrd FAILED
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://localhost:60690/apis/apiextensions.k8s.io/v1/customresourcedefinitions.
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:694)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:673)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:626)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:566)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:527)
at app//io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:510)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:136)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:505)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:494)
at app//io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:87)
at app//com.ibm.si.qradar.cp4s.service.certmanager.TestCertManagerService.testCronTabCrd(TestCertManagerService.java:94)
I have a few of questions:
(1) In this case, I am using v1() interface, sometimes I saw example code is using v1beta1(), what decides this version? By the way, I am using Kubernetes-client library 5.9.0
(2) In my code , I comments out this line
server.expect().get().withPath("/apis/stable.example.com/v1/namespaces/default/crontabs").andReturn(HttpURLConnection.HTTP_OK, ?????).once();
What is this statement for? In my case, I want to load a CRD, then create a CR, what is "?????" in the statement?
Any ideas for stack trace? How to fix it?
I appreciate it in advance.
From the code which you shared, it looks like you're using Fabric8 Kubernetes Mock Server in expectations mode. Expectations mode requires the user to set the REST API expectations. So the code shown below is setting some expectations from Mock Server viewpoint.
// Given
server.expect().get()
.withPath("/apis/stable.example.com/v1/namespaces/default/crontabs")
.andReturn(HttpURLConnection.HTTP_OK, getCronTabList())
.once();
These are the expectations set:
Mock Server would be requested a GET request at this URL: /apis/stable.example.com/v1/namespaces/default/crontabs . From URL we can expect a resource under stable.example.com apigroup with v1 version, default namespace and crontabs as plural.
When this URL is being hit, you're also defining response code and response body in andReturn() method. First argument is the response code (200 in this case) and second argument is the response body (a List object of CronTab which would be serialized and sent as response by mock server).
This request is only hit .once(), if KubernetesClient created by Mock Server requests this endpoint more than once; the test would fail. If you want to hit the endpoint more than once, you can use .times(..) method instead.
But in your test I see you're loading a CustomResourceDefinition from YAML and creating it which doesn't seem to match the expectations you set earlier. If you're writing a test about creating a CustomResourceDefinition, it should look like this:
#Test
#DisplayName("Should Create CronTab CRD")
void testCronTabCrd() throws IOException {
// Given
KubernetesClient client = server.getClient();
CustomResourceDefinition cronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.load(new BufferedInputStream(new FileInputStream("src/test/resources/crontab-crd.yml")))
.get();
server.expect().post()
.withPath("/apis/apiextensions.k8s.io/v1/customresourcedefinitions")
.andReturn(HttpURLConnection.HTTP_OK, cronTabCrd)
.once();
// When
CustomResourceDefinition createdCronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.create(cronTabCrd);
// Then
assertNotNull(createdCronTabCrd);
}
Bdw, if you don't like setting REST expectations. Fabric8 Kubernetes Mock Server also has a CRUD mode which mock real Kubernetes APIServer. You can enable it like this:
#Rule
public KubernetesServer server = new KubernetesServer(true, true);
then use it in test like this:
#Test
#DisplayName("Should Create CronTab CRD")
void testCronTabCrd() throws IOException {
// Given
KubernetesClient client = server.getClient();
CustomResourceDefinition cronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.load(new BufferedInputStream(new FileInputStream("src/test/resources/crontab-crd.yml")))
.get();
// When
CustomResourceDefinition createdCronTabCrd = client.apiextensions().v1()
.customResourceDefinitions()
.create(cronTabCrd);
// Then
assertNotNull(createdCronTabCrd);
}
I added CustomResourceLoadAndCreateTest and CustomResourceLoadAndCreateCrudTest tests in my demo repository: https://github.com/r0haaaan/kubernetes-mockserver-demo
I want to be able to extract the List<Payload> from the Mono<List<Payload>> to pass it to a downstream service for processing (or maybe return from the read(RequestParams params) method, instead of it returning void):
#PostMapping("/subset")
public void read(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
....
}
where reader.read(...) is a method on an autowired Spring service utilizing a webClient to get the data from external web service API:
public Mono<List<Payload>> read(String date, String assetClasses, String firmAccounts, String id, String password) {
Flux<Payload> nodes = client
.get()
.uri(uriBuilder -> uriBuilder
.path("/api/subset")
.queryParam("payloads", true)
.queryParam("date", date)
.queryParam("assetClasses", assetClasses)
.queryParam("firmAccounts", firmAccounts)
.build())
.headers(header -> header.setBasicAuth("abc123", "XXXXXXX"))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> {
System.out.println("4xx error");
return Mono.error(new RuntimeException("4xx"));
})
.onStatus(HttpStatus::is5xxServerError, response -> {
System.out.println("5xx error");
return Mono.error(new RuntimeException("5xx"));
})
.bodyToFlux(Payload.class);
Mono<List<Payload>> records = nodes
.collectList();
return records;
}
Doing a blocking result.block() is not allowed in WebFlux and throws an exception:
new IllegalStateException("block()/blockFirst()/blockLast() are blocking, which is not supported in thread ..." ;
What is a proper way to extract the contents of a Mono in WebFlux?
Is it some kind of a subscribe()? What would be the syntax?
Thank you in advance.
There is no "proper way" and that is the entire point. To get the value you need to block, and blocking is bad in webflux for many reasons (that I won't go into right now).
What you should do is to return the publisher all the way out to the calling client.
One of the things that many usually have a hard time understanding is that webflux works with a producer (Mono or Flux) and a subscriber.
Your entire service is also a producer, and the calling client can be seen as the subscriber.
Think of it as a long chain, that starts at the datasource, and ends up in the client showing the data.
A simple rule of thumb is that whomever is the final consumer of the data is the subscriber, everyone else is a producer.
So in your case, you just return the Mono<List<T> out to the calling client.
#PostMapping("/subset")
public Mono<List<Payload>> read(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
return result;
}
While the following does return the value of the Mono observable in the logs:
#PostMapping("/subset")
#ResponseBody
public Mono<ResponseEntity<List<Payload>>> read1(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
return result
.map(e -> new ResponseEntity<List<PayloadByStandardBasis>>(e, HttpStatus.OK));
}
the understanding I was seeking was a proper way to compose a chain of calls, with WebFlux, whereby a response from one of its operators/legs (materialized as as a result from a webclient call, producing a set of records, as above) could be passed downstream to another operator/leg to facilitate a side effect of saving those records in a DB, or something to that effect.
It would probably be a good idea to model each of those steps as a separate REST endpoint, and then have another endpoint for a composition operation which internally calls each independent endpoint in the right order, or would other design choices be more preferred?
That is ultimately the understanding I was looking for, so if anyone wants to share an example code as well as opinions to better implement the set of steps described above, I'm willing to accept the most comprehensive answer.
Thank you.
In AWS API Gateway I am developing lambda function for custom authorizer using .NET Core. The API will receive api-key in query string and my custom authroizer will validate the key. Based on my understanding, after validation is done the lambda function needs to return IAM policy. The awslab blurprint does not have any example for .NET core. The only example i found so far is GrandmasRecipes that is using JWT Token.
I would like to know what IAM policy the lambda function needs to return and are there any corresponding .NET Core classes for request and response?
Update 1
So below is my code for custom lambda authorizer. However i would like know:
1> What should be PrincipalID. Currently i am just setting it to User
2>CheckAuthorization method gets all the keys from aws and only check the existence by comparing the key from the request. It should also check the Usage Plans and make sure the key from the request is configured in Usage Plans
3>The role that this Authorizer is executing under is attached to AmazonAPIGatewayAdministrator policy so that it can get API Keys, whats the minimum policy do i need for this role to validate api-key?
4>Is there any in-built method in AWSSDK to do validate api-key correctly?
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]
namespace ApikeyAuthorizer
{
public class Function
{
public async Task<APIGatewayCustomAuthorizerResponse> FunctionHandler(APIGatewayCustomAuthorizerRequest authEvent, ILambdaContext context)
{
var key = authEvent.QueryStringParameters["key"];
bool authorized = await CheckAuthorization(key);
var authPolicy = new APIGatewayCustomAuthorizerResponse();
authPolicy.PrincipalID = "user";
authPolicy.PolicyDocument = new APIGatewayCustomAuthorizerPolicy();
authPolicy.PolicyDocument.Version = "2012-10-17";
authPolicy.PolicyDocument.Statement = new List<APIGatewayCustomAuthorizerPolicy.IAMPolicyStatement>();
if (authorized)
{
var statement = new APIGatewayCustomAuthorizerPolicy.IAMPolicyStatement();
statement.Action = new HashSet<string>(new string[] { "execute-api:Invoke" });
statement.Effect = "Allow";
statement.Resource = new HashSet<string>(new string[] { "arn:aws:execute-api:us-east-1:xxxxx:*/*/GET/*" });
authPolicy.UsageIdentifierKey = key;
authPolicy.PolicyDocument.Statement.Add(statement);
}
else
{
var statement = new APIGatewayCustomAuthorizerPolicy.IAMPolicyStatement();
statement.Action = new HashSet<string>(new string[] { "execute-api:Invoke" });
statement.Effect = "Deny";
statement.Resource = new HashSet<string>(new string[] { "arn:aws:execute-api:us-east-1:xxxxx:*/*/GET/*" });
authPolicy.PolicyDocument.Statement.Add(statement);
}
return authPolicy;
}
public async Task<bool> CheckAuthorization(string key)
{
Amazon.APIGateway.AmazonAPIGatewayClient client = new Amazon.APIGateway.AmazonAPIGatewayClient();
var response = await client.GetApiKeysAsync(new Amazon.APIGateway.Model.GetApiKeysRequest()
{
IncludeValues = true
});
foreach (var apiKey in response.Items)
{
if (apiKey.Value == key)
{
return true;
}
}
return false;
}
}
}
You don't need to use a Lambda Authorizer to validate API Key neither it should be used for authorisation. You can do following to configure API key validation in API Gateway.
In you API Resources section, set API Key Required true for the methods where you want to enable it
Go to API Keys section, select Create API key from Actions dropdown and create a key
Go to Usage Plans section and create a new usage plan.
After you've created a usage plan click on it and then click API Keys tab. Here click Add API Key to Usage Plan and add the key you created in step # 2
Now click on Details tab then click Add API Stage. Select your API and stage that you want to restrict with API Key.
Your API methods are now required an x-api-key HTTP header where you've enabled it. When you request API endpoint make sure you add x-api-key header with same value you have created in step 2 above. If you don't add this header or put a wrong value you will get 403 Forbidden error.
curl -X PUT \
https://XXXX.XXXXX-api.ca-central-1.amazonaws.com/PROD/XXX-microservice \
-H 'Content-Type: application/json' \
-H 'x-api-key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' \
-d '{
"initData": "HI",
"name": "vaquar khan",
"likes": "Java"
}'
Security key validation taken care by API getaway so no lambda
authorizer required
Please dont create duplicate question just update old queston ,I have answred in your question here :
- How to validate API Key in AWS Lambda function
In actions-on-google , both the request and response object need to provide as input to this library. but in lambda function, only the request object exists.
So how can i override it ?
in aws lambda the format is
exports.handler = function (event, context, callback) { // event is the request object , the response is provided using the callback() functon
}
the actions-on-google object is created as :
const DialogflowApp = require('actions-on-google').DialogflowApp;
const app = new DialogflowApp({ request: request, response: response });
To get a Google Action to work on AWS Lambda, you need to do 2 things:
Code your app in a way that it's executable on Lambda
Create an API Gateway to your Lambda Function which you can then use for Dialogflow Fulfillment
I believe the first setp can't be done off-the-shelf with the Actions SDK. If you're using a framework like Jovo, you can create code that works for both Amazon Alexa and Google Assistant, and host it on AWS Lambda.
You can find a step by step tutorial about setting up a "Hello World" Google Action, host it on Lambda, and create an API Gateway here: https://www.jovo.tech/blog/google-action-tutorial-nodejs/
Disclaimer: I'm one of the founders of Jovo. Happy to answer any further questions.
This is only a half answer:
Ok, so I dont think I can tell you how to make the action on google sdk correct working on AWS Lambda.
Maybe its easy, I just dont know and need to read everything to know it.
My, "easy to go", but at the end you will maybe have more work solution, would be just interprete the request jsons by yourself and responde with a message as shown below
This here would be a extrem trivial javascript function to create a extrem trivial JSON response.
Parameters:
Message is the string you would like to add as answer.
Slots should be an array that can be used to bias the speech recognition.
(you can just give an empty array to this function if you dont want to bias the speech).
And State is any kind of serilizable javascript object this is for your self to maintain states or something else It will be transfered between all the intents.
This is an standard response on an speech request.
You can add other plattforms than speech for this, by adding different initial prompts please see the JSON tabs from the documentation:
https://developers.google.com/actions/assistant/responses#json
function answerWithMessage(message,slots,state){
let newmessage = message.toLowerCase();
let jsonResponse = {
conversationToken: JSON.stringify(state),
expectUserResponse: true,
expectedInputs: [
{
inputPrompt: {
initialPrompts: [
{
textToSpeech: newmessage
}
],
noInputPrompts: []
},
possibleIntents: [
{
intent: "actions.intent.TEXT"
}
],
speechBiasingHints: slots
}
]
};
return JSON.stringify(jsonResponse,null, 4);
}
I am trying to get notified when an user subscribes to a stomp user destination using #SubscribeMapping annotation. My ideia is to send some initialization data to when it joins.
Although i am not being able to get this working:
Javascript:
stompClient.subscribe('/user/monitor', function(msg) {
console.log(msg);
});
Java side:
#SubscribeMapping("/monitor-user{clientId}")
public void init(#DestinationVariable("clientId") String clientId) {
messagingTemplate.convertAndSend("/user/" + clientId + "/monitor", getOps());
}
I tried many mapping combinations such as "/user/monitor-{clientId}", "/user/monitor" (Removing clientId), no success at all.
What is the expected mapping value so this get called?
Thank you!
Since the client subscribes to the "/user/monitor" queue, the server #SubscribeMapping annotation must be to "/user/monitor" and not to "/user/monitor{something}".
The server can understand what client is depending on your authentication scheme. If you use a sockjs websocket you can use HTTP authentication, and hence leverage Spring Security, and add a "Principal" parameter to your function that will hold user information:
#SubscribeMapping("/monitor")
public void init(Principal p) {
String user = p.getName();
messagingTemplate.convertAndSendToUser(user, "/monitor", getOps());
}
If you don't use http authentication you may send the server the user info, for example adding a custom STOMP header that can be accessed from the server using a SimpMessageHeaderAccessor:
#SubscribeMapping("/monitor")
public void init(SimpMessageHeaderAccessor accessor) {
String user = accessor.getFirstNativeHeader("userid");
messagingTemplate.convertAndSendToUser(user, "/monitor", getOps());
}
Another way could be to subscribe to a different queue name which contains the user information (and this maybe was your proposal). The client must use a code similar to:
stompClient.subscribe('/topic/byUser/'+userId, function(msg) {
console.log(msg);
});
so that the server can access it this way:
#SubscribeMapping("/byUser/{userId}")
public void init(#DestinationVariable String userId) {
messagingTemplate.convertAndSendToUser(userId, "/monitor", getOps());
}
but in this case keep in mind that that queue is public and hence other clients can access their messages if they knows other client names.