I am developing apis & microservices in nestJS,
this is my controller function
#Post()
#MessagePattern({ service: TRANSACTION_SERVICE, msg: 'create' })
create( #Body() createTransactionDto: TransactionDto_create ) : Promise<Transaction>{
return this.transactionsService.create(createTransactionDto)
}
when i call post api, dto validation works fine, but when i call this using microservice validation does not work and it passes to service without rejecting with error.
here is my DTO
import { IsEmail, IsNotEmpty, IsString } from 'class-validator';
export class TransactionDto_create{
#IsNotEmpty()
action: string;
// #IsString()
readonly rec_id : string;
#IsNotEmpty()
readonly data : Object;
extras : Object;
// readonly extras2 : Object;
}
when i call api without action parameter it shows error action required but when i call this from microservice using
const pattern = { service: TRANSACTION_SERVICE, msg: 'create' };
const data = {id: '5d1de5d787db5151903c80b9', extras:{'asdf':'dsf'}};
return this.client.send<number>(pattern, data)
it does not throw error and goes to service.
I have added globalpipe validation also.
app.useGlobalPipes(new ValidationPipe({
disableErrorMessages: false, // set true to hide detailed error message
whitelist: false, // set true to strip params which are not in DTO
transform: false // set true if you want DTO to convert params to DTO class by default its false
}));
how will it work for both api & microservice, because i need all at one place and with same functionality so that as per clients it can be called.
ValidationPipe throws HTTP BadRequestException, where as the proxy client expects RpcException.
#Catch(HttpException)
export class RpcValidationFilter implements ExceptionFilter {
catch(exception: HttpException, host: ArgumentsHost) {
return new RpcException(exception.getResponse())
}
}
#UseFilters(new RpcValidationFilter())
#MessagePattern('validate')
async validate(
#Payload(new ValidationPipe({ whitelist: true })) payload: SomeDTO,
) {
// payload validates to SomeDto
. . .
}
I'm going out on a limb and assuming in you main.ts you have the line app.useGlobalPipes(new ValidationPipe());. From the documentation
In the case of hybrid apps the useGlobalPipes() method doesn't set up pipes for gateways and micro services. For "standard" (non-hybrid) microservice apps, useGlobalPipes() does mount pipes globally.
You could instead bind the pipe globally from the AppModule, or you could use the #UsePipes() decorator on each route that will be needing validation via the ValidationPipe
More info on binding pipes here
As I understood, useGlobalPipes is working fine for api but not for microservice.
Reason behind this, nest microservice is a hybrid application and it has some restrictions. Please refer below para.
By default a hybrid application will not inherit global pipes, interceptors, guards and filters configured for the main (HTTP-based) application. To inherit these configuration properties from the main application, set the inheritAppConfig property in the second argument (an optional options object) of the connectMicroservice() call.
Please refer this Nest Official Document
So, you need to add inheritAppConfig option in connectMicroservice() method.
const microservice = app.connectMicroservice(
{
transport: Transport.TCP,
},
{ inheritAppConfig: true },
);
It worked for me!
Related
I want to build a NestJS project where I will separate queries and mutations into two groups: Internal and Public. In the Internal GraphQL module, I will define path and resolvers without any restrictions, but for the Public, I want to define a GraphQL module with path and JWT Guard, which will look at the same resolvers but just specific mutations and queries.
I tried to do the following:
GraphQLModule.forRootAsync<ApolloDriverConfig>({
driver: ApolloDriver,
useClass: InternalGraphQLConfig,
}),
GraphQLModule.forRootAsync<ApolloDriverConfig>({
driver: ApolloDriver,
useClass: PublicGraphQLConfig,
}),
For protecting the public endpoint (the path is defined in the PublicGraphQLConfig by GqlOptionsFactory) I added middleware by NestMiddleware where I am checking req.originalUrl - the path from the PublicGraphQLConfig. If the URL is the public one, I am checking for the JWT otherwise is a free - internal URL.
But, I do not know how and where I can define the list of queries and mutations for the Public GraphQL model because I do not want to expose all of them.
As I can see in the documentation, this approach may be unavailable, and it is impossible to do it correctly like this. Maybe I have to use directives or something else, but I firmly believe someone has a similar/same challenge and will share an idea/solution with me.
Edit 1:
Here I will add more details about resolvers and GraphQL Module configuration.
One of my resolvers looks like the following:
...
#Resolver(DogEntity)
class DogResolver {
#Mutation(...)
async firstMutation(...) {
//
}
#Mutation(...)
async secondMutation(...) {
//
}
#Query(...)
async firstQuery(...) {
//
}
#Query(...)
async secondQuery(...) {
//
}
...
}
GraphQL Module configuration looks like this:
...
#Injectable()
export class PublicGraphQLConfig implements GqlOptionsFactory {
createGqlOptions(): Promise<ApolloDriverConfig> | ApolloDriverConfig {
return {
...
resolvers: { DogResolver, ...}
path: '/my/public/route/graphql'
};
}
}
The first: It would be amazing if I could add a "global" guardian on the GrapQL Module level with, for example guards parameter in the PublicGraphQLConfig. Because it is impossible, and adding any JWT validation in the context parameter no make sense, I have to add Middleware where I'm checking the path parameter from the GraphQL Module configuration.
The Middleware looks like this:
...
#Injectable()
export class RequestResponseLoggerMiddleware implements NestMiddleware {
use(req: Request, res: Response, next: NextFunction) {
// For public endpoint, all resolvers required JWT token with Admin flag
if (req.originalUrl === '/my/public/route/graphql') {
this.validateJWT(req); // Do "throw exception inside"
}
...
The second: It would be amazing to add specific Mutation and/or Query in the GraphQL Module configuration. With the resolvers parameter, I can add only complete resolvers, but not specific queries or mutations. With this, I will be able to access the specific queries and mutations from different Endpoints with/out authorization requests.
The field in the GrapQL Module configuration like the following will be amazing (but, as I can see, it does not exist)
...
return {
...
resolvers: {
DogResolver:firstMutation(),
DogResolver:firstQuery(),
...
},
path: '/my/public/route/graphql'
};
...
In app.module.ts I have the following:
#Module({
imports: [
...,
GraphQLModule.forRoot<ApolloGatewayDriverConfig>({
server: {
context: getContext,
},
driver: ApolloGatewayDriver,
gateway: {
buildService: ({ name, url }) => {
return new RemoteGraphQLDataSource({
url,
willSendRequest({ request, context }: any) {
...
},
});
},
supergraphSdl: new IntrospectAndCompose({
subgraphs: [
{ name: 'iam', url: API_URL_IAM },
],
})
},
}),
]
...
})
here getContext is just a regular function which is not part of nestjs context (doesn't have injection, module capability) like below:
export const getContext = async ({ req }) => {
return {}
}
Is there any way to use nestjs services instead of plain old functional approach to build the context for graphql gateway in nestjs?
Thanks in advance for any kind of help.
I believe you're looking to create a service that is #Injectable and you can use that injectable service via a provider. What a provider will do is satisfy any dependency injection necessary.
In your scenario, I would import other modules as necessary. For building context, I would create a config file to create from env variables. Then create a custom provider that reads from the env variables and provides that implementation of the class/service to the other classes as their dependency injection.
For example, if I have a graphQL module. I would import the independent module. Then, I would provide in the providers section, the handler/service classes and the dependencies as an #injectable. Once your service class is created based on your config (which your provider class would handle), you would attach that service class to your GraphQL class to maybe lets say direct the URL to your dev/prod envs.
I'm using serverless stack, now attempting to add a Lambda Custom Authenticator to validate authorization tokens with Auth0 and add custom data to my request context when the authentication passes.
Everything works mostly fine at this point, except for when I cache the Authenticator response for the same token.
I'm using a 5-second cache for development. The first request with a valid token goes through as it should. The next requests in the 5-second window fail with a mysterious 500 error without ever reaching my code.
Authorizer configuration
// MyStack.ts
const authorizer = new sst.Function(this, "AuthorizerFunction", {
handler: "src/services/Auth/handler.handler",
});
const api = new sst.Api(this, "MarketplaceApi", {
defaultAuthorizationType: sst.ApiAuthorizationType.CUSTOM,
defaultAuthorizer: new HttpLambdaAuthorizer("Authorizer", authorizer, {
authorizerName: "LambdaAuthorizer",
resultsCacheTtl: Duration.seconds(5), // <-- this is the cache config
}),
routes: {
"ANY /{proxy+}": "APIGateway.handler",
},
});
Authorizer handler
const handler = async (event: APIGatewayAuthorizerEvent): Promise<APIGatewayAuthorizerResult> => {
// Authenticates with Auth0 and serializes context data I wanna
// forward to the underlying service
const authentication = await authenticate(event);
const context = packAuthorizerContext(authentication.value);
const result: APIGatewayAuthorizerResult = {
principalId: authentication.value?.id || "unknown",
policyDocument: buildPolicy(authentication.isSuccess ? "Allow" : "Deny", event.methodArn),
context, // context has the following shape:
// {
// info: {
// id: string,
// marketplaceId: string,
// roles: string,
// permissions: string
// }
// }
};
return result;
};
CloudWatch logs
☝️ Every uncached request succeeds, with status code 200, an integration ID and everything, as it's supposed to. Every other request during the 5-second cache fails with 500 error code and no integration ID, meaning it doesn't reach my code.
Any tips?
Update
I just found this in an api-gateway.d.ts #types file (attention to the comments, please):
// Poorly documented, but API Gateway will just fail internally if
// the context type does not match this.
// Note that although non-string types will be accepted, they will be
// coerced to strings on the other side.
export interface APIGatewayAuthorizerResultContext {
[name: string]: string | number | boolean | null | undefined;
}
And I did have this problem before I could get the Authorizer to work in the first place. I had my roles and permissions properties as string arrays, and I had to transform them to plain strings. Then it worked.
Lo and behold, I just ran a test right now, removing the context information I was returning for successfully validated tokens and now the cache is working 😔 every request succeeds, but I do need my context information...
Maybe there's a max length for the context object? Please let me know of any restrictions on the context object. As the #types file states, that thing is poorly documented. This is the docs I know about.
The issue is that none of the context object values may contain "special" characters.
Your context object must be something like:
"context": {
"someString": "value",
"someNumber": 1,
"someBool": true
},
You cannot set a JSON object or array as a valid value of any key in the context map. The only valid value types are string, number and boolean.
In my case, though, I needed to send a string array.
I tried to get around the type restriction by JSON-serializing the array, which produced "[\"valueA\",\"valueB\"]" and, for some reason, AWS didn't like it.
TL;DR
What solved my problem was using myArray.join(",") instead of JSON.stringify(myArray)
The EndpointName property in a ConsumerDefinition file seems to be ignored by MassTransit. I know the ConsumerDefinition is being used because the retry logic works. How do I get different commands to go to a different queue? It seems that I can get them all to go through one central queue but I don't think this is best practice for commands.
Here is my app configuration that executes on startup when creating the MassTransit bus.
Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
cfg.Host(_config.ServiceBusUri, host => {
host.SharedAccessSignature(s =>
{
s.KeyName = _config.KeyName;
s.SharedAccessKey = _config.SharedAccessKey;
s.TokenTimeToLive = TimeSpan.FromDays(1);
s.TokenScope = TokenScope.Namespace;
});
});
cfg.ReceiveEndpoint("publish", ec =>
{
// this is done to register all consumers in the assembly and to use their definition files
ec.ConfigureConsumers(provider);
});
And my handler definition in the consumer (an azure worker service)
public class CreateAccessPointCommandHandlerDef : ConsumerDefinition<CreateAccessPointCommandHandler>
{
public CreateAccessPointCommandHandlerDef()
{
EndpointName = "specific";
ConcurrentMessageLimit = 4;
}
protected override void ConfigureConsumer(
IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<CreateAccessPointCommandHandler> consumerConfigurator
)
{
endpointConfigurator.UseMessageRetry(r =>
{
r.Immediate(2);
});
}
}
In my app that is sending the message I have to configure it to send to the "publish" queue, not "specific".
EndpointConvention.Map<CreateAccessPointsCommand>(new Uri($"queue:specific")); // does not work
EndpointConvention.Map<CreateAccessPointsCommand>(new Uri($"queue:publish")); // this does work
Because you are configuring the receive endpoint yourself, and giving it the name publish, that's the receive endpoint.
To configure the endpoints using the definitions, use:
cfg.ConfigureEndpoints(provider);
This will use the definitions that were registered in the container to configure the receive endpoints, using the consumer endpoint name defined.
This is also explained in the documentation.
I am having fun with using moleculer-runner instead of creating a ServiceBroker instance in a moleculer-web project I am working on. The Runner simplifies setting up services for moleculer-web, and all the services - including the api.service.js file - look and behave the same, using a module.exports = { blah } format.
I can cleanly define the REST endpoints in the api.service.js file, and create the connected functions in the appropriate service files. For example aliases: { 'GET sensors': 'sensors.list' } points to the list() action/function in sensors.service.js . It all works great using some dummy data in an array.
The next step is to get the service(s) to open up a socket and talk to a local program listening on an internal set address/port. The idea is to accept a REST call from the web, talk to a local program over a socket to get some data, then format and return the data back via REST to the client.
BUT When I want to use sockets with moleculer, I'm having trouble finding useful info and examples on integrating moleculer-io with a moleculer-runner-based setup. All the examples I find use the ServiceBroker model. I thought my Google-Fu was pretty good, but I'm at a loss as to where to look to next. Or, can i modify the ServiceBroker examples to work with moleculer-runner? Any insight or input is welcome.
If you want the following chain:
localhost:3000/sensor/list -> sensor.list() -> send message to local program:8071 -> get response -> send response as return message to the REST caller.
Then you need to add a socket io client to your sensor service (which has the list() action). Adding a client will allow it to communicate with "outside world" via sockets.
Check the image below. I think it has everything that you need.
As a skeleton I've used moleculer-demo project.
What I have:
API service api.service.js. That handles the HTTP requests and passes them to the sensor.service.js
The sensor.service.js will be responsible for communicating with remote socket.io server so it needs to have a socket.io client. Now, when the sensor.service.js service has started() I'm establishing a connection with a remote server located at port 8071. After this I can use this connection in my service actions to communicate with socket.io server. This is exactly what I'm doing in sensor.list action.
I've also created remote-server.service.js to mock your socket.io server. Despite being a moleculer service, the sensor.service.js communicates with it via socket.io protocol.
It doesn't matter if your services use (or not) socket.io. All the services are declared in the same way, i.e., module.exports = {}
Below is a working example with socket.io.
const { ServiceBroker } = require("moleculer");
const ApiGateway = require("moleculer-web");
const SocketIOService = require("moleculer-io");
const io = require("socket.io-client");
const IOService = {
name: "api",
// SocketIOService should be after moleculer-web
// Load the HTTP API Gateway to be able to reach "greeter" action via:
// http://localhost:3000/hello/greeter
mixins: [ApiGateway, SocketIOService]
};
const HelloService = {
name: "hello",
actions: {
greeter() {
return "Hello Via Socket";
}
}
};
const broker = new ServiceBroker();
broker.createService(IOService);
broker.createService(HelloService);
broker.start().then(async () => {
const socket = io("http://localhost:3000", {
reconnectionDelay: 300,
reconnectionDelayMax: 300
});
socket.on("connect", () => {
console.log("Connection with the Gateway established");
});
socket.emit("call", "hello.greeter", (error, res) => {
console.log(res);
});
});
To make it work with moleculer-runner just copy the service declarations into my-service.service.js. So for example, your api.service.js could look like:
// api.service.js
module.exports = {
name: "api",
// SocketIOService should be after moleculer-web
// Load the HTTP API Gateway to be able to reach "greeter" action via:
// http://localhost:3000/hello/greeter
mixins: [ApiGateway, SocketIOService]
}
and your greeter service:
// greeter.service.js
module.exports = {
name: "hello",
actions: {
greeter() {
return "Hello Via Socket";
}
}
}
And run npm run dev or moleculer-runner --repl --hot services