Preload of WSGI in, for example, gunicorn, permits shared read-only RAM-resident data structures in the parent process, prior to pre-forking worker processes.
How can a worker process queue up a modification to its parent's data structure, so that the parent makes the modification to the shared data structure and then re-spawns the worker processes with the new data?
A use case:
resource-intensive preload
high frequency reads
rare writes
writes involving complex memory allocation operations. i.e. the multiprocess preallocation of a block of memory in, say, an array, isn't really adequate due to non-trivial memory allocations.
Related
I've read responses to a couple similar questions on stackoverflow, and although it seems like sharing a single instance for two purposes is fine, I would like to know the potential downside.
My main concern is the cache filling up the memory and slowing down or breaking the task queue. Is this possible? I use caching heavily, so should I be worried about this scenario?
Theoretically, you can use the same Redis instance for task queue and caching.
There're some downsides
Longer query time
High memory usage
High CPU usage
Backup
Any fail safe task queue, makes a lot of redis calls to move a task from one data structure to other and for other actions. You should check your task queue, how many redis calls it would make in a seconds for 1 queueu and N queues. If the number of Redis queueries is proportional to the number of queues than you should see can your Redis server handles such requests.
Since you're using same Redis instance for task queue and cache the number of entries in your cache could be very large, see it's not going beyond it's memory limit. Losing cache data is fine but you should not loose task queue data.
Due to a large number of queries the CPU utilization would increase, hopefully it won't reach 90% or so, watch for any cpu spike.
Given you're going to use same Redis server for task queue, you should enable backup for Redis server, so that you can restore tasks from the backup. When you're doing backup likely backup would be done for whole data not only task queues.
I need to use a very big hash-table, and access it from many readers and many writers in parallel. is there data structure like map, that support many reads and writes in parallel, without locking the whole structure each access?
Since you asked for a map
without locking the whole structure each access
I direct you to the following implementation:
https://github.com/cornelk/hashmap
This project implements a pure lock free hash map data structure using atomic instructions common in many CPU architectures
The regular go sync.Map still uses an underlying Mutex which locks the corresponding map datastructure.
Package sync provides the concurrent safe map.
Map is like a Go map[interface{}]interface{} but is safe for
concurrent use by multiple goroutines without additional locking or
coordination. Loads, stores, and deletes run in amortized constant
time.
Although the spec itself point out these two specific cases when it should be used(otherwise they suggest using the normal map with locking mechanism):
when the entry for a given key is only ever written once but read many times, as in caches that only grow
when multiple goroutines read, write and overwrite entries for disjoint sets of keys
I am a software developer but wannabe architect new to the server scalability world.
In the context of multiple services working with the same data set, aiming to scale for redundancies and load balancing.
The question is: In a idealistic system, should services try to optimize their internal processing to reduce the amount of queries done to the remote server cache for better performance and less bandwidth at the cost of some local memory and code base or is it better to just go all-in and query the remote cache as the single transaction point every time any transaction need processing done on the data?
When I read about Redis and even general database usage online, the later seems to be the common option. Every nodes of the scaled application have no memory and read and write directly to the remote cache on every transactions.
But as a developer, I ask if this isn't a tremendous waste of resources? Whether you are designing at electronic chips level, at inter-thread, inter-process or inter-machine, I do believe it's the responsibility of each sub-system to do whatever it can to optimize its processing without depending on the external world if it can and hence reduce overall operation time.
I mean, if the same data is read over hundreds or time from the same service without changes (write), isn't it just more logical to keep a local cache and wait for notifications of changes (pub/sub) and only read only these changes to update the cache instead reading the bigger portion of data every time a transaction require it? On the other hand, I understand that this method implies that the same data will be duplicated at multiple place (more ram usage) and require some sort of expiration system not to keep the cache from filling up.
I know Redis is built to be fast. But however fast it is, in my opinion there's still a massive difference between reading directly from local memory versus querying an external service, transfer data over network, allocating memory, deserialize into proper objects and garbage collect it when you are finished with it. Anyone have benchmark numbers between in-process dictionaries query versus a Redis query on the localhost? Is it a negligible time in the bigger scheme of things or is it an important factor?
Now, I believe the real answer to my question until now is "it depends on your usage scenario", so let's elaborate:
Some of our services trigger actions on conditions of data change, others periodically crunch data, others periodically read new data from external network source and finally others are responsible to present data to users and let them trigger some actions and bring in new data. So it's a bit more complex than a single web pages deserving service. We already have a cache system codebase in most services, and we have a message broker system to notify data changes and trigger actions. Currently only one service of each type exist (not scaled). They transfer small volatile data over messages and bigger more persistent (changing less often) data over SQL. We are in process of moving pretty much all data to Redis to ease scalability and performances. Now some colleagues are having a heated discussion about whether we should abandon the cache system altogether and use Redis as the common global cache, or keep our notification/refresh system. We were wondering what the external world think about it. Thanks
(damn that's a lot of text)
I would favor utilizing in-process memory as much as possible. Any remote query introduces latency. You can use a hybrid approach and utilize in-process cache for speed (and it is MUCH faster) but put a significantly shorter TTL on it, and then once expired, reach further back to Redis.
I'm writing a visit counter for products on a website which uses MongoDB as its' DB-Engine.
Here it says that Mongo keeps frequently accessed stuff in memory and has an integrated in-memory caching engine.
So can I just relay on this integrated caching system and dumbly set the counters up on every visit or does one still need another caching layer on a high-traffic environment?
They're two seperate things. MongoDB uses a simple paged memory management system that, by design, keeps the most accessed parts of the memory mapped disk space in memory.
As a result, this will help you most for counters that are requested frequently but do not change often. Unfortunately for website counters these two things are mutually exclusive. Because increasing counters will generally not cause MongoDB to move the document holding the counter on disk the read caching will still be fairly effective.
The main issue is your writes, basically doing an increase per visit is not going to be very cost effective. I suggest a strategy where your counter webapp caches incoming visits and only pushes counter updates every X visits or every Y seconds, whichever comes first. Your main goal here is to reduce writes per second so you definitely do not want a db write per counter visit.
Although I have never worked on the kind of system you describe, I would do the following (assuming that I have read your question correctly and that you do indeed simply want to increment the counter for each visit).
Use the $inc operator to atomically perform the incrementation, or use upserts with modifiers to create the document structure if it is not already there
Use an appropriate Write Concern to speed up updates if that is safe to do so (ie with a Write Concern of NONE your call to update will return immediately and you'll just have to trust Mongo to persist it to disk). Of course whether this is safe or not depends on the use case. If you are counting millions of hits then 1 failed hit may not be a problem.
If the scale of data you are storing is truly enormous, look into using sharding to partition writes
I am writing an application in clojure which needs to persist its data, but if some data is lost then it is not disastrous. I am wondering whether I should use ClojureQL every time I want to access data or should I use the STM + ClojureQL and just persist data asynchronously on updates/writes. Does anyone know any advantages/disadvantages of each approach. And has anyone tried the asynchronous persistence approach?
This is a classic approach to improving write latency, often called write-behind. Batching together updates and doing them in a separate asynchronous thread has several benefits:
removes (presumably slow) writes to persistent store from the update path
more efficient if you can batch updates
might allow coalescing of updates (if you have multiple writes on the same key in the same batch, you can just kill off all but the last and never write the intermediate value at all)
Write-behind also has significant disadvantages:
writes can be lost if the data is stored only in memory
it is sometimes challenging to get the background async thread to participate properly with the rest of a transactional system
the background thread(s) can get backed up since they are handling the "slow" part of the operation; write-behind works best when it's used to speed-up occasional peaks and can recover during the troughs
subsequent reads between the memory-write and the store-write will see the old value unless you also check the in-memory state - that can be either confusing or potentially expensive on read. Because of this, write-behind is often used in conjunction with read-through caching where all reads occur through the cache.
None of this really has anything to do with ClojureQL or the STM of course. :)