I am developing a rest api that serves a game. Every three minutes a job runs on server updating an important information of this game. So after this job runs, I need to invalid the cache and create a new one with recent information.
Ok, I implemented on my application MemCached, but a senior developer said that it would be very important to have other cache. He suggested to me to use Varnish, but I really don't know if it would fit in my logic.
Do you have any suggestions of how I could achieve this?
Varnish will work just fine in your case. Of course, Memcached is used for caching transient data whereas Varnish is a full page cache, so it's great for reducing the load on your backend application (whichever language it's powered with, PHP or anything).
You will not need to make any change to your application to cache things with Varnish properly (however you could go that route as well, and adjust your app to send the proper caching headers). Simply develop the VCL (Varnish Configuration Language) file with instructions on your cache policy.
Do not use complete copy paste for VCL files you find online. Add smallest snippest as possible, understand how things work and Varnish will not dissapoint you. Important would be:
Ensure that your cache varies by the API token (if you use for API authentication). You will implement this in vcl_hash procedure.
Integrate cache clearing in your job for updating information: Varnish cache can be cleared by use of a PURGE HTTP request (again, you'd need to develop the necessary VCL code for it, inside vcl_recv procedure).
You can use Mcrouter a memcached protocol router to basically replicate your Memcache.
This config can handle your request:
{
"pools": {
"A": {
"servers": [
// First Memcache address:
"memcache_1_ip:11211",
// Second Memcache address:
"memcache_2_ip:11211"
]
}
},
"route": {
"type": "OperationSelectorRoute",
"operation_policies": {
"add": "AllSyncRoute|Pool|A",
"delete": "AllSyncRoute|Pool|A",
"get": "LatestRoute|Pool|A",
"set": "AllSyncRoute|Pool|A"
}
}
Related
I need to set up a simple replication schema with a secondary database. I figured out that using arangosh I can set it up with the following commands:
db._useDatabase("myDB");
require("#arangodb/replication").setupReplication({
endpoint: "tcp://main-server:8529",
username: "user",
password: "pass",
verbose: false,
includeSystem: false,
incremental: true,
autoResync: false,
autoStart: true,
restrictType: "include",
restrictCollections: [ "Products" ]
});
This setup, however does not seem to persist. Connection going down, or server restarts make it disappear.
So, I would like to set up some monitoring and re-establishment of the replication in my Go program.
I searched both the ArangoDB website Manual pages, and Go driver documentation but I could not find anything that would allow me to run the above setup in Go using the driver.
Additionally, I didn't find how I could interface with arangosh, and possibly run the JS code above and get the results. Is that possible somehow using the Go driver?
I accidentally found a solution to this.
The Golang driver does not provide this functionality. But Arango has a pretty simple HTTP based API which allows access to all functions and features of the database engine.
Here's the link to the documentation I used: https://www.arangodb.com/docs/3.8/http/index.html
(I'm using version 3.8 because after that the type of replication I needed was no longer part of the community edition).
Setting up a replication requires just two steps:
PUT request to the /_db/yourDBname/_api/replication/applier-config with a JSON payload:
{
"endpoint":"tcp://serverIP:8529",
"database":"yourDBname",
"username":"root",
"password":"password",
"autoResync": true,
"autostart":true
}
And another PUT request to get the replication actually started, to /_db/yourDBname/_api/replication/applier-start . This one doesn't need any payload
And to see how things are going you can do a GET request to /_db/yourDBname/_api/replication/applier-state
All these requests need a JWT token that you can get with a POST request to /_open/auth with a payload of:
{
"username": "user",
"password": "passwd"
}
The token you receive will need to be included in the HTTP header as a bearer token. Pretty simple.
I am using "caches" to cache in service worker my PWA assets and make it available offline.
When I change an asset, specifically a js file, I modify at least one byte in my service worker to trigger its native update: the service worker updates and retrieves all of its previously cached assets to refresh its caches.
Yet, server responds with a cached version of the file, and whereas I own the files served I have no control over Cache-Control http header.
How can i prevent browser caching on service worker cached resources? Versioning the files with a
"?v="+version
suffix won't work, because this version cannot be passed to the or or tags that references the cached files in html files, which are static and caches will not recognize and serve offline unversioned file names.
Since "caches.addAll" does not allow AFAIK any means to specify http request headers such as Cache-Control as fetch or XMLHttpRequest do, how can I prevent additional aggressive caching stages over my assets?
I am using plain Javascript and if possible I need it to be done without any additional library. Note also that meta http-equiv tags won't solve the problem for assets other than complete html.
You can bypass the browser's cache by explicitly constructing a Request object with a cache property set to an appropriate cache mode. 'reload' is a good choice, as it will bypass the browser's cache for the outgoing request, but it will update the browser's cache with the response (so you'll have a fresher browser cache overall). If you don't even want that update to be performed, you could use 'no-store'.
Here's some code showing how to do this concisely for an array of URLs that could be passed in to cache.addAll():
async function addAllBypassCache(cacheName, urls) {
const cache = await caches.open(cacheName);
const requests = urls.map((url) => new Request(url, {
cache: 'reload',
}));
await cache.addAll(requests);
}
I'd like to understand how to enable caching in Strapi for a specific (or any) API endpoint. At the moment when the browser hits my endpoint in the response headers I don't see any caching related headers. Is there a way to use etags and have a long cache time to allow the JSON response to be cached?
There is one mention of etags in the docs but I'm not sure how to implement. If anyone can provide more detailed information it would be appreciated.
At least for static files this can be done within Strapi itself. The middlewares documentation suggests, that there is already a middleware called public, which sets a Cache-Control header with its maxAge when it serves the files from the public/ directory.
But if you want to get your uploaded files cached (i.e. files within the public/uploads/ directory), that’s not enough, because the middleware strapi-provider-upload-local (not yet documented) runs first.
A recently published package solves this issue:
npm i strapi-middleware-upload-plugin-cache
Simply activate it by adding or modifying the config/middleware.js file with the following content:
module.exports = ({ env }) => ({
settings: {
'upload-plugin-cache': {
enabled: true,
maxAge: 86400000
}
}
});
I suggest you manage this kind of thing outside of Strapi.
With another service. For example, if you host you app on AWS, you can use CloudFront.
Here is my situation:
I have a webapp that uses a lot of images on a remote server. My webapp is behind Cloudflare, although the server that the images are hosted on are not.. and this server can be very slow. It can sometimes take about 5 seconds per image.
I would like to use Cloudflare to proxy requests to this external server, but also cache them indefinitely, or at least as long as possible. The images never change so I do not mind them having a long cache life.
Is this something I should set up in a worker? As a page rule? Or just not use CLoudflare in this way?
If you can't change origin server headers, you could try following snippet in your worker:
fetch(event.request, { cf: { cacheTtl: 300 } })
As per docs:
This option forces Cloudflare to cache the response for this request,
regardless of what headers are seen on the response. This is
equivalent to setting two page rules: “Edge Cache TTL” and “Cache
Level” (to “Cache Everything”).
I think you generally just want a very long caching header on your images. Something like:
Cache-Control: public; max-age=31536000
I'm using the jQuery .get() function to load in a template file and then display the loaded HTML into a part of the page by targeting a particular DOM element. It works well but I've recently realised for reasons that confound me that it is caching my template files and masking changes I have made.
Don't get me wrong ... I like caching as much as the next guy. I want it to cache when there's no timestamp difference between the client's cache and the file on the server. However, that's not what is happening. To make it even odder ... using the same function to load the templates ... some template files are loading the updates and others are not (preferring the cached version over recent changes).
Below is the loading function I use.
function LoadTemplateFile ( DOMlocation , templateFile , addEventsFlag ) {
$.get( templateFile , function (data) {
$( DOMlocation ).html(data);
});
}
Any help would be greatly appreciated.
NEW DETAILS:
I've been doing some debugging and now see that the "data" variable that comes back to the success function does have the newer information but for reasons that are not yet clear to me what's inserted into the DOM is an old version. How in heck this would happen has now become my question.
You can set caching to off in the jQuery.get() call. See the jQuery.ajax() doc for details with the cache: false option. The details of how caching works is browser specific so I don't think you can control how it works, just whether it's on or off for any given call.
FYI, when you turn caching off, jQuery bypasses the browser caching by appending a unique timestamp parameter to the end of the URL which makes it not match any previous URL, thus there is no cache hit.
You can probably also control cache lifetime and several other caching parameters on your server which will set various HTTP headers that instruct the browser what types of caching to allow. When developing your app, you probably want caching off entirely. For deployment, I would suggest that you want it on and then when you rev your app, the safest way to deal with caching is to change your URLs slightly so the new versions of your URLs are different. That way, you always get max caching, but also users get new versions immediately.
$get will always cache by default, especially on IE. You will need to manually append a querystring, or use the ajaxSetup method to bust the cache.
As an alternative, I found in the jQuery docs that you can just override the default caching. For me I didn't have to convert all of my $.get over to $.ajax calls. Note that this also overrides default behavior for all other types of calls, like $.getScript which has the opposite default behavior of cache: false from $.get.
https://api.jquery.com/jquery.getscript/
$.ajaxSetup({
cache: false
});