Running multiple Kafka brokers on Mesos as Marathon jobs - mesos

I want to run multiple Kafka brokers on Mesos as Marathon jobs. I'm thinking what to use as a broker ID. I know that some people are using IP based broker ID, but I read that this approach is not optimal for situations when migrating the brokers to different machines. Any insights?
Related to that, is there a way to do a rolling deploy of Kafka brokers in a way that Marathon waits for the old broker instances to replicate the data to the new ones before killing them.

I created a Docker image which can start on Marathon and also scale up and down. To get a unique broker id, I concatenate the last octet of the IP address and the port number which Marathon devises to the Docker container.
I understood that this shouldn't be a problem, because normally the clients connect via ZooKeeper, which itself has the nodes and their connection info, so you shouldn't need to take care of the details. Or I misunderstood the question...
Have a look at
http://kafka.apache.org/documentation.html#replication

Related

What exactly is a 'node' in Redis

I'm reading around Redis at the moment and trying to find a good understanding of what a 'node' is terms of how Redis works. Am I right to think of it in the same was as an endpoint?
In Redis' context, a node is a server running one or more redis-server processes.
Endpoint is a network address through which you can access one or more such processes, depending on how Redis is clustered.
When using the open source Redis cluster, an endpoint is any of the processes - meaning a node's address and the port that the process listens to. Redis client libraries use the protocol to interrogate the clustered redis-server process about other members of the cluster (again, processes listening on ports on nodes), so they can establish connections to other endpoints accordingly.
Disclaimer: it appears that you're asking about AWS ElastiCache, which may or may not be using the OSS implementation in whole or partially. I do not claim to have any knowledge on that subject.
Its a type of (temporary memory [RAM]) to which network is attached. Its the smallest unit where frequently accessed data is stored by following lazy loading or write through strategy. A collection of such nodes ,where a predefined Redis process is running on each node , is called cluster.
More on node :
https://redis.io/commands/cluster-nodes/

how to communicatie rocketmq and rocketmq directly?

I have two network envionments (such as NETWORK -A and NETWORK -B). Now, I deployed rocketmq-a in the NETWORK-A and deployed rocketmq-b in the NETWORK-B, how to communicate rocketmq-a and rocketmq-b directly?
According to your comment, you have two rocketmq clusters and one message should replicate to another rocketmq cluster.
So this is a message replication.
you have two choices:
Implement a send message hook
Use a messageStore plugIn that extend AbstractPluginMessageStore and load it using broker configuration
Both of them needs to implement replication by yourself.
However, if you make them the same broker group, it is very easy.
Just make rocketmq-b as a slave of rocketmq-a, and deploy them in different machine room.
Then rocketmq-b will only provide read operations and always replicate the data from master

which port is used to gossip in akka actor cluster by the cluster members?

I am trying to debug an issue where cluster island are getting formed.
checking if there is an network issue.
which port is used to gossip in akka actor cluster by the cluster members ?
It depends on your configuration - it's the TCP port that you set up Akka to listen on. For more details refer to Cluster configuration docs (see the port and seed-node values).
Broader hints on cluster partitions:
If you ended up having a cluster partition in an Akka cluster it most likely means you're using the auto-downing feature. It's not recommended for production use, as it's rather fleaky - it relies on a simple timeout based mechanism. For more advanced downing mechanisms you can look into using the Split Brain Resolver commercial tooling, or build a downing mechanism yourself which would hook into external monitoring infrastructure (we've seen a number of teams do this).
An interesting thought to keep in mind is that perhaps you do not need auto-downing at all, and when leaving the cluster with a node you can do so cleanly by issuing Cluster.leave(address) in the code.

MQ Cluster - how to properly disable one node in production environments

I have some messages flowing through the MQ cluster by using cluster and alias queues. Some queues are defined multiple times, though the loadbalancing mechanism is used.
What is the propper way to extract one QM from the cluster without disturbing the whole message flow? Disabling the cluster-receiver channel, cluster-sender channels, or else?
Use the
suspend qmgr
command.
This suspends the queue manager from the cluster.
command reference

How To Load-Distribution in RabbitMQ cluster?

Hi I create three RabbitMQ servers running in cluster on EC2
I want to scale out RabbitMQ cluster base on CPU utilization but when I publish message only one server utilizes CPU and other RabbitMQ-server not utilize CPU
so how can i distribute the load across the RabbitMQ cluster
RabbitMQ clusters are designed to improve scalability, but the system is not completely automatic.
When you declare a queue on a node in a cluster, the queue is only created on that one node. So, if you have one queue, regardless to which node you publish, the message will end up on the node where the queue resides.
To properly use RabbitMQ clusters, you need to make sure you do the following things:
have multiple queues distributed across the nodes, such that work is distributed somewhat evenly,
connect your clients to different nodes (otherwise, you might end up funneling all messages through one node), and
if you can, try to have publishers/consumers connect to the node which holds the queue they're using (in order to minimize message transfers within the cluster).
Alternatively, have a look at High Availability Queues. They're like normal queues, but the queue contents are mirrored across several nodes. So, in your case, you would publish to one node, RabbitMQ will mirror the publishes to the other node, and consumers will be able to connect to either node without worrying about bogging down the cluster with internal transfers.
That is not really true. Check out the documentation on that subject.
Messages published to the queue are replicated to all mirrors. Consumers are connected to the master regardless of which node they connect to, with mirrors dropping messages that have been acknowledged at the master. Queue mirroring therefore enhances availability, but does not distribute load across nodes (all participating nodes each do all the work).

Resources