I am investigating the cache-lib for C++ service.
Data usage scenarios: use local cache to reduce remote service access times. Like: Access remote api, get data, store locally, read util it expired. Access remote api again if data expired. There is no modify operation. There are many remote services, and the data structures are different. We will access some data at the same time, so it must be thread-safe. And the lib needs to be implemented in C++.
I have investigated some C++ kv storage libraries, such as the leveldbărocksdb... I have tried leveldb LRUCache and the performance is not very well. Maybe it is not friendly for random reading frequently? Is there any other kv storage lib that I should try?
It would be great if you could give me some advice. Thank you!
Related
I'm trying to build a small site that gets its data from a database (currently I use Firebase's Cloud Firestore).
I've build it using next.js and thought to host it on vercel. It looks very nice and was working well.
However, the site needs to handle ~1000 small documents - serve, search, and rarely update. In order to reduce calls to the database on every request, which is costly both in time, and in database pricing, I thought it would be better if the server could get the full list of item when it starts (or on the first request), and then hold them in memory and make data request get the data from its memory.
It worked well in the local dev server, but when I deployed it to vercel, it didn't work. It seems it forces me to work in serverless mode, where each request is separate, and I can't use a common in-memory cache to get the data.
Am I missing something and there is a way to achieve something like that with next.js on vercel?
If not, can you recommend other free cloud services that can provide what I'm looking for?
One option can be using FaunaDB and Netlify, as described in this post, but I ended up opening a free Wix site and using Wix data to store the data. I built http-functions module to provide access to the data via REST, which also caches highly used data in memory. Currently it seems to work like a charm!
I'm on a 2.6.3 Parse Server and I need to cache the results of queries, to speed things up!
I understand that Parse Server offers a Redis adapter. What exactly do I have to do, in order to start using Redis? Are there any modules I should install? Anything I should import or configure?
Also, I found this on Parse's documentation:
Those cache adapters can be cleaned at anytime internally, you should not use them to cache data and you should let parse-server manage their data lifecycle.
What do they mean by saying you should not use them to cache data and you should let parse-server manage their data lifecycle.? Should I not use the adapter?
What the doc is saying is that parse caches with it's own in-memory structure by default, but it leaves developers the option to use reddis as a substitute. To opt for that, just (1) setup redis as you typically would, (2) initialize the parse server with a RedisCacheAdapter that's been configured with your redis URL.
The point you're asking about: "you should not use them to cache data ..." means that Parse will continue to decide when to cache, when to retrieve from cache, and when to clean, etc. but it will do so by invoking the redis that you configured with.
I think the major advantage to this more elaborate setup is redis's distributed capability. If you're not running on a cluster, you may find the redis idea to be about equivalent performance-wise and a little messier setup-wise as not doing it.
I am using a react-native app with relay modern.
Currently our app's fetchQuery implementation, just does a fetch on the network (like in https://facebook.github.io/relay/docs/en/network-layer.html),
Although there is a possibility of another local-network layer like https://github.com/relay-tools/relay-local-schema which returns data from a local-db like sqlite/realm.
Is there a way to setup offline-first response from local-network layer, followed by automatic request to real network which also populates the store with fresher data (along with writing to local-db)?
Also should/can they share the same store?
From the requirements of Network.create(), it should return a promise containing the payload, there does not seem a possibility to return multiple values.
Any ideas/help/suggestions are appreciated.
What you trying to achieve its complex, and ill go for the easy approach which is long time cache.
As you might know relay modern uses a local storage and its exact copy of the data you are fetching, you can configure this store cache as per your needs, no cache on mutations.
To understand how this is achieve the best library around to customise Relay Modern or Classic network layer you can find in https://github.com/nodkz/react-relay-network-modern
My recommendation: setup your cache and watch your request.... (you going to love it)
Thinking in Relay,
https://facebook.github.io/relay/docs/en/thinking-in-relay.html
I'm curious if any cloud storage system could be configured to provide the following workflow :
Anonymous users may upload messages/files into identifiable locations which we'll call buckets.
All users should have read access to all messages/files, but no anonymous user should have permissions to modify or delete them.
Buckets have associated public keys which a moderator uses to authenticate approvals or deletions of uploads.
Unapproved messages/files are eventually culled by the system to save space.
I suspect the answer might be "Tahoe-LAFS would love for someone to implement append-only mutable files, but nobody has done so yet."
I've surveyed a number of OSS projects in the storage space and not encountered anything that would provide this workflow purely by configuration and without writing code.
While not OSS, the lowest level of Windows Azure storage is actually implemented via an append only mechanism. A video, presentation, and whitepaper can all be found here and the details in the whitepaper would be useful to anyone looking to implement something like this for Tahoe-LAFS or any other OSS cloud storage system.
I am currently building a high traffic GIS system which uses python on the web front end. The system is 99% read only. In the interest of performance, I am considering using an externally generated cache of pre-generated read-optimised GIS information and storing in an SQLite database on each individual web server. In short it's going to be used as a distributed read-only cache which doesn't have to hop over the network. The back end OLTP store will be postgreSQL but that will handle less than 1% of the requests.
I have considered using Redis but the dataset is quite large and therefore it will push up the administrative cost and memory cost on the virtual machines this is being hosted on. Memcache is not suitable as it cannot do range queries.
Am I going to hit read-concurrency problems with SQLite doing this?
Is this a sensible approach?
Ok after much research and performance testing, SQLite is suitable for this. It has good request concurrency on static data. SQLite only becomes an issue if you are doing writes as well as heavy reads.
More information here:
http://www.sqlite.org/lockingv3.html
if usage case is just a cache why don't you use something like
http://memcached.org/.
You can find memcached bindings for python in pypi repository.
Another options is that you use materialized views in postgres, this way you will keep things simple and have everything in one place.
http://tech.jonathangardner.net/wiki/PostgreSQL/Materialized_Views