Caching the axios httpService in NestJS - caching

Within our application we use the Axios HttpService to do some request to a third-party api.
Because the amount of data returned bij de api is so huge, we would like to cache the responses.
In the docs is wasn't able to find some examples of how to do this.
I'm currently doing this as follows:
#Module({
imports: [
HttpModule,
CacheModule.register({
ttl: 15,
store: redisStore,
host: 'localhost',
port: 6379,
})
]
})
export class AppModule {}
I register the CacheModule globally.
Then import it in the module where i need it.
In the service where i use the third-party api, i create an interceptor where i go and cache the reponses. Very crude and just for testing.
constructor(private readonly httpService: HttpService,
private readonly cache: CacheStore) {
httpService.axiosRef.interceptors.response.use((response) => {
cache.set(response.request.url, response.data);
return response;
}, error => Promise.reject(error));
}
First of all this doesn't run, because the CACHE_MANAGER can't be imported into the CacheModule, for some reason.
Second this is a more a Node.js way of creating such interceptors and not the NestJS way.
But is this a way to move forward or is there a more effecient way and if yes, what way is that?

The CacheModule is not the right tool here, as it means to cache ingoing requests (the requests your service receive, so it doesn't proceed them again and send back a cache result).
What you're trying to do is caching the outgoing requests (the one your service makes to 3rd-party services). For mysterious reasons, I couldn't find it documented in NestJS documentation either, but here's how you can go:
As you're using Axios, you can implement caching using the axios-cache-adapter npm package.
npm install --save axios-cache-adapter
Then you need to create an adapter (preferably in the constructor of your service, see notes below):
const cache = setupCache({
maxAge: 3600 * 1000, // 60 minutes
});
and provide this adapter as part of the AxiosRequestConfig alongside your request:
const config = {
adapter: this.cache.adapter,
};
this.httpService.get( url, config );
You should be good with some caching now!
IMPORTANT NOTES:
caching should be used, and will likely work, only on GET requests
use the setupCache function only once (in the constructor of your service for instance); if you create one cache object for every request, this cache will be empty everytime, defeating the purpose of it

Related

nestjs + socket.io serving websockets from microservice at nested path instead of the root of the host always returns 404 Not Found

I'm using the socket.io implementation with nestjs. My overall in-production architecture is relying on a load balancer to route requests on paths to microservice containers.
One of them contains a websocket that interacts with user data and is nested on mydomain.com/apis/user
On my gateway I've configured it to use this path:
#WebSocketGateway({ path: "/apis/user/gateway", namespace: "/apis/user/gateway" })
and I've tried variations without the namespace there and with just the path.
I also have a global prefix: app.setGlobalPrefix("apis/user"); removing this makes no difference, it seems I need to define the path to the gateway either way prefix or not.
Then on the client I'm trying to just connect to it via either adding the path in the url, or to the options object, as such:
const endpoint = "https://example.com/apis/user/gateway";
socket = io(endpoint, {
secure: true,
path: "/apis/user/gateway",
});
This works for the path and handleConnection triggers on the gateway nested there, however configuring polling on the backend does not work, the client still throws:
https://example.com/apis/user/gateway/?EIO=4&transport=polling&t=NXVpMfB 404 Not Found
I had the same problem, all the various options are confusing. I got it sorted though, here is what worked:
For the nestjs:
#WebSocketGateway({
cors: {
origin: '*',
},
namespace: '/api/v1/ws',
path: '/api/v1/ws/socket.io',
})
For socket.io:
const newSocket = io("http://localhost:3000/api/v1/ws", {
path: "/api/v1/ws/socket.io",
});

How to do graphql subscriptions with Absinthe (Elixir) and Urql?

My idea was to build a client app using react and urql and a graphql api using elixir and absinthe but at the moment it looks as if these don't really play that well together.
Is there a way to actually use the Absinthe subscriptions with any other client than Apollo? I've tried to use urql but I fail with the ws connection getting the following error:
WebSocket connection to 'ws://localhost:4000/socket/websocket' failed:
Error during WebSocket handshake: Sent non-empty
'Sec-WebSocket-Protocol' header but no response was received
Only thing I've found so far which seems to be related to this issue is this library absinthe/socket-apollo-link (https://github.com/absinthe-graphql/absinthe-socket/tree/master/packages/socket-apollo-link) but it's obviously only for Apollo.
In my failed attempt I did the following:
import React from 'react'
import { Provider, Client, dedupExchange, fetchExchange, subscriptionExchange } from 'urql'
import { cacheExchange } from '#urql/exchange-graphcache'
import { SubscriptionClient } from 'subscriptions-transport-ws'
const DataProvider = ({ children }) => {
const cache = cacheExchange({})
const subscriptionClient = new SubscriptionClient('ws://localhost:4000/socket/websocket', {})
const client = new Client({
url: 'http://localhost:4000/api',
exchanges: [
dedupExchange,
cache,
fetchExchange,
subscriptionExchange({
forwardSubscription: operations => subscriptionClient.request(operations),
}),
],
})
return <Provider value={client}>{children}</Provider>
}
export default DataProvider
This 'subscriptions-transport-ws' I found from a tutorial and that mysterious '/websocket' at the end of the ws url was mentioned in some gh issue but it seems to work (before adding that to the end of the url there was no ws connection at all).
This doesn't directly answer the question, but in case someone ends up with the same confusion and as a response to your comment here:
I found from a tutorial and that mysterious '/websocket' at the end of the ws url was mentioned in some gh issue but it seems to work (before adding that to the end of the url there was no ws connection at all).
This is due to both longpoll and websocket being viable transports. You likely had your socket defined in your Endpoint with websocket: true which gave you that route. Running mix phx.routes | grep socket would have been a helpful command.
This is obviously a late reply, but I'll try my best anyway. It seems that Absinthe has an alternative client library that's different from subscriptions-transport-ws, notably it's here: https://github.com/absinthe-graphql/absinthe-socket/blob/master/packages/socket-apollo-link/src/createAbsintheSocketLink.js#L8
If you integrate with this library instead then presumably it'll work. So that means you likely have to integrate with PhoenixSocket instead, as shown here: https://github.com/absinthe-graphql/absinthe-socket/tree/master/packages/socket-apollo-link#examples

How to add cache headers to Strapi API endpoint

I'd like to understand how to enable caching in Strapi for a specific (or any) API endpoint. At the moment when the browser hits my endpoint in the response headers I don't see any caching related headers. Is there a way to use etags and have a long cache time to allow the JSON response to be cached?
There is one mention of etags in the docs but I'm not sure how to implement. If anyone can provide more detailed information it would be appreciated.
At least for static files this can be done within Strapi itself. The middlewares documentation suggests, that there is already a middleware called public, which sets a Cache-Control header with its maxAge when it serves the files from the public/ directory.
But if you want to get your uploaded files cached (i.e. files within the public/uploads/ directory), that’s not enough, because the middleware strapi-provider-upload-local (not yet documented) runs first.
A recently published package solves this issue:
npm i strapi-middleware-upload-plugin-cache
Simply activate it by adding or modifying the config/middleware.js file with the following content:
module.exports = ({ env }) => ({
settings: {
'upload-plugin-cache': {
enabled: true,
maxAge: 86400000
}
}
});
I suggest you manage this kind of thing outside of Strapi.
With another service. For example, if you host you app on AWS, you can use CloudFront.

JWT with Angular and SpringBoot

I want to integrate a JWT authentication in my current Angular application which is connected to a spring boot backend.
I looked through quiet a lot of tutorials, but most of them did not fit, since they were connected to Auth0. I want to handle the usermanagement by myself.
I found this example:
Angular: https://medium.com/#juliapassynkova/angular-springboot-jwt-integration-p-1-800a337a4e0
Spring Boot: https://medium.com/#nydiarra/secure-a-spring-boot-rest-api-with-json-web-token-reference-to-angular-integration-e57a25806c50
Unfortunately, this Angular Frontend is Angular 4 and I'm using Angular 6 already. They use the 'angular2-jwt' which is not compatible with Angular 6. I have to use 'auth0/angular-jwt'.
My problem is, that after I use my login form, the browser opens again a login popup to login the backend. But even with the right credentials, it doesn't work. I'm guessing, that some credentials are missing in the request. In the app.module.ts are 2 old angular2-jwt code, which i don't know how to upgrade it to auth0/angular-jwt
export function authHttpServiceFactory(http: Http) {
return new AuthHttp(new AuthConfig({
headerPrefix: 'Bearer',
tokenName: TOKEN_NAME,
globalHeaders: [{'Content-Type': 'application/json'}],
noJwtError: false,
noTokenScheme: true,
tokenGetter: (() => localStorage.getItem(TOKEN_NAME))
}), http);
}
and
providers: [
{provide: AuthHttp, useFactory: authHttpServiceFactory, deps: [Http]},
]
Can someone help me with this issue. May be I'm wrong with my guessing and it could be something else?
A side note; I just changed my frontend but trying to connect to the backend from the example. If I'm using both front- and backend from the example, its working fine.
I have passed successful migration from angular2-jwt to auth0/angular-jwt several months ago. What you have to keep in mind while doing so is:
auth0/angular-jwt uses new angular HttpClient library, so if you want to use so, you have to migrate all your application to it (I suggest doing so anyway, new library is quite amazing)
new angular-jwt uses HttpInterceptors - this allows you to attach JWT directly to HttpClient filters chain. So you don't have to migrate your code above. You have to remove it completely and replace with interceptor initialization: (copy-paste from auth0/angular-jwt documentation):
import { JwtModule } from '#auth0/angular-jwt';
import { HttpClientModule } from '#angular/common/http';
export function tokenGetter() {
return localStorage.getItem('access_token');
}
#NgModule({
bootstrap: [AppComponent],
imports: [
// ...
HttpClientModule,
JwtModule.forRoot({
config: {
tokenGetter: tokenGetter,
whitelistedDomains: ['localhost:3001'],
blacklistedRoutes: ['localhost:3001/auth/']
}
})
]
})
export class AppModule {}
Remember to apply this code in your root module - then use your HttpClient without any changes and interceptors will apply JWT automatically. In case of problems - blame whitelistedDomains/blacklistedRoutes. These fields are very poorly implemented at the moment and are causing problems with regular expressions etc.
As migration is not straightforward, and took me a lot of time, feel free to ask about it. There's high probability, that I'll know the answers.

Vue.js + Vue Resource No 'Access-Control-Allow-Origin'

Cross site ajax request with Vue.js 1.0 and Vue Resource. I get the following error: XMLHttpRequest cannot load http://dev.markitondemand.com/MODApis/Api/v2/Lookup/jsonp?input=NFLX&callback=handleResponse. No 'Access-Control-Allow-Origin' header is present on the requested resource.
I have a basic understanding of the problem but not sure how to add a callback function with the request or if that is the best solution for this example. I put in the full request URL here just to make it easier to follow.
new Vue({
el: '#stockList',
data: function() {
return {
query: '',
stocks: []
};
},
ready: function() {
this.getStocks();
},
methods: {
getStocks: function() {
this.$http.get('http://dev.markitondemand.com/MODApis/Api/v2/Lookup/jsonp?input=NFLX&callback=handleResponse',
function(data) {
this.stocks = data;
}
);
}
}
})
I have almost zero understanding of networking, but I was able to get several remote apis to work using:
this.$http.jsonp
instead of
this.$http.get
"No Access-Control-Allow-Origin" header usually is a problem with the server. It means that the server is configured to only allow a person access to the API if the request comes from the same domain as the server. You either need to run the script from the website that you are requesting data from, or you need to change the server config to allow access to all domains.
If you don't have access to the server, and you don't want to run the script in the browser, then I think what you could do is use a headless browser like PhantomJS to navigate to the page, insert a script element into the dom that contains you script, execute the function and then return the data from the API. I could write the code out for you, but to be honest, it's a bit complex. You would have to know how to use node.js, and phantom.js. I've personally only used phantom.js for the Node 'html-pdf' package, but I'm sure with a little bit of reading you could figure out how to do it.
Set your local environment to http instead of https if you have no control over dev.markitondemand.com.

Resources