We have a memcached cluster running in production as a cache layer on top of MySQL. Now we are considering to replace memcached with Couchbase to avoid the cold cache issue (in case of crash) and have this nice feature of managed cache cluster.
At the same time, we want to minimize the changes to migrate to Couchbase. One way we can try is to maintain the libmemcached API and set up a http proxy to direct all request to Couchbase. This way nothing is changed in the application code. If I understand correctly, this way Couchbase is basically a managed memcache cluster. We didn't take advantage of the persistence cache items. We can't do something like flagging a certain cached item to be persistent:
# connect to couchbase like connecting to memcached
$ telnet localhost 11211
SET foo "foo value" # How can we make this item persistent in Couchbase?
I assume this is because all items are stored in memcached bucket. So the question becomes:
Can we control which item to be stored in Couchbase bucket or
memcache bucket? To do so, do we have to change libmemcached API and all the application code related to that?
Thanks!
I think you should look into running Moxi, which is a memcached proxy to couchbase. You can configure Moxi with the destination couchbase bucket.
A couchbase cluster automatically spins up a cluster-aware moxi gateway, which you could point your web/application servers to. This is what couchbase calls "server-side moxi".
Alternatively, you can either install moxi on each your web/app servers, so they simply connect to localhost:11211. Moxi handles the persistent connection the couchbase cluster. This is what couchbase calls "client-side moxi".
Related
If there are multiple read replicas, where load balancing related settings can be specified when using spring AWS libraries.
Read replicas have their own endpoint address similar to the original RDS instance. Your application will need to take care of using all the replicas and to switch between them. You'd need to introduce this algorithm into your application so it automatically detects which RDS instance it should connect to in turn. The following links can help:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Replication.html#Overview.ReadReplica
Problem: I want to cache user information such that all my applications can read the data quickly, but I want only one specific application to be able to write to this cache.
I am on AWS, so one solution that occurred to me was a version of memcached with two ports: one port that accepts read commands only and one that accepts reads and writes. I could then use security groups to control access.
Since I'm on AWS, if there are solutions that use out-of-the box memcached or redis, that'd be great.
I suggest you use ElastiCache with one open port at 11211(Memcached)then create an EC2 instance, set your security group so only this server can access to your ElastiCache cluster. Use this server to filter your applications, so only one specific application can write to it. You control the access with security group, script or iptable. If you are not using VPC, then you can use cache security group.
I believe you can accomplish this using Redis (instead of Memcached) which is also available via ElastiCache. Once the instance has been created, you will want to create a replication group and associate it to the cache cluster you already launched.
You can then add instances to the replication group. Instances within the replication group are simply replicated from the Master Cache Cluster (single Redis instance) and so are (by default) read-only.
So, in this setup, you have a master node (single endpoint) that you can write to and as many read nodes (multiple endpoints) as you would like.
You can take security a step further and assign different routing rules to the replication group (via the VPC) so the applications reading data does not have access to the master node (the only one that can write data).
I am considering using Amazon ElastiCache Redis. However, I would like to be in control of my replication, and so I would like to know if it's possible to set up redis-server on a VPS (non-Amazon) or on an EC2 Amazon to be the slave of the ElastiCache Redis instance.
If not, then is ElastiCache Redis worth using when you want to use Redis as an in-memory data storage with reliable persistency, and not only for mere "caching" of data?
Thank you,
As of Amazon's updates for Redis 2.8.22 you can no longer use non-ElastiCache replication nodes. The SYNC and PSYNC commands will be unrecognized. This change appears to affect all Redis versions, so you can't circumvent it by using a pre-2.8.22 Redis instance.
An alternative would be to use an EC2 instance as a master node, however you would lose the management benefits ElastiCache provides, needing to set up and maintain everything by yourself.
Yes, it is possible to do so. The replication protocol works on the same redis connection. So if you can connect to elastic cache from the VPS or EC2, you will also be able to install a slave on that machine.
I am developing the django-backend of a ios app. I will use cached-session using redis. Once a user logs in, I will save his session in the redis-cache (backed up by mysql), I just want to know (for the long run), can I use redis replication to keep copy of the cached session incase I scale the redis server in a master-slave format in the future. Or I should always access cache value from one particular redis server?
It makes sense to keep copy with replication of redis in master/slave format since there isn't the possibility of sharding like in mongodb for redis yet (AFAIK). So you have to get your session from one particual redis server until if you dont want to control several redis-server manually.
I have a few questions about BigCouch that i'm interesting getting answers before start using it.
Do I need to choose my shard key carefully or can just use an auto-generated GUID? I start with a single server with 1 replication, but I want to be ready when I need to add another shard
Any GUI for managing the cluster like CouchBase have, something similar to administer the DB
How can I backup the data when hosting BigCouch on EC2 (ie. snapshots)
Thanks
Since you have no started to use BigCouch yet and it looks like you need some features that are available out of the box in Couchbase (auto-sharding, administration console ...)
Why no going on Couchbase ?