I have three machins like :
[consul#cjportal]$ consul members
Node Address Status Type Build Protocol DC
portal1 192.168.11.155:8301 alive client 0.7.0 2 dc1
portal0 192.168.14.100:8301 alive client 0.7.0 2 dc1
portal2 192.168.11.182:8301 alive server 0.7.0 2 dc1
and all 3 machines has the same consul config file like:
{
"service":{
"name":"portal_confgen",
"tags":[
"portal"
],
"address": "127.0.0.1",
"port": 8823,
"check":{
"name":"ping",
"script":"ping -c1 192.168.11.155",
"interval":"10s"
}
}
}
and all 3 machines running consul , only server portal2 runs consul-template,use command:
consul-template -config=/home/consul/consul-template/config/hosts.hcl -consul=localhost:8500
my consul-template config file, hosts.hcl:
template {
source = "/home/consul/consul-template/hosts.ctmpl"
destination = "/home/consul/conf/conf.d/test.conf"
}
But when I change k/v in consul storage, only localhost portal2 writes destination file correctly, remote machines portal0 and portal1 does not work.What do I miss???
You need consul-template to be ran on all servers.
What are you expect? if you run consul template only on portal2 so only it will be updated, if you will run it on portal0 and portal1 they would be updated too
Related
I am trying to connect to a Aerospike single node I set up using Vagrant on MacOSX. My AMC is running on localhost:2200. I am unable to connect to it successfully.
import com.aerospike.client.AerospikeClient;
public class AerospikeDriver {
public static void main(String[] args) {
AerospikeClient client = new AerospikeClient("127.0.0.1", 2200);
client.close();
}
}
I am getting this error in the first line itself. I tried changing the port to 3000 as well. Same error. Sometimes, I get SocketException as well.
Exception in thread "main" com.aerospike.client.AerospikeException$Connection: Error Code -8: Failed to connect to host(s):
127.0.0.1 2200 Error Code -1: java.io.EOFException
at com.aerospike.client.cluster.Cluster.seedNodes(Cluster.java:532)
at com.aerospike.client.cluster.Cluster.tend(Cluster.java:425)
at com.aerospike.client.cluster.Cluster.waitTillStabilized(Cluster.java:380)
at com.aerospike.client.cluster.Cluster.initTendThread(Cluster.java:286)
at com.aerospike.client.cluster.Cluster.<init>(Cluster.java:243)
at com.aerospike.client.AerospikeClient.<init>(AerospikeClient.java:234)
at com.aerospike.client.AerospikeClient.<init>(AerospikeClient.java:175)
at AerospikeDriver.main(AerospikeDriver.java:5)
My maven dependency for aerospike client is this:
<dependency>
<groupId>com.aerospike</groupId>
<artifactId>aerospike-client</artifactId>
<version>4.1.11</version>
</dependency>
This is my aerospike conf:
# Aerospike database configuration file.
# This stanza must come first.
service {
user root
group root
paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
pidfile /var/run/aerospike/asd.pid
# service-threads 4
# transaction-queues 4
# transaction-threads-per-queue 4
proto-fd-max 15000
node-id-interface eth1
}
logging {
# Log file must be an absolute path.
file /var/log/aerospike/aerospike.log {
context any info
}
file /var/log/aerospike/udf.log {
context udf info
context aggr info
}
}
network {
service {
address eth1
port 3000
# access-address <Published IP>
# access-address <NAT IP>
}
heartbeat {
mode multicast
multicast-group 239.1.99.222
address eth1
port 9918
protocol v3
# To use unicast-mesh heartbeats, comment out the 3 lines above and
# use the following 4 lines instead.
# mode mesh
# port 3002
# mesh-address 10.1.1.1
# mesh-port 3002
interval 150
timeout 10
}
fabric {
port 3001
address eth1
}
info {
port 3003
}
}
#namespace test {
# replication-factor 2
# memory-size 4G
# default-ttl 30d # 30 days, use 0 to never expire/evict.
#
# storage-engine memory
#}
namespace test {
replication-factor 2
memory-size 2G
default-ttl 5d # 5 days, use 0 to never expire/evict.
# To use file storage backing, comment out the line above and use the
# following lines instead.
storage-engine device {
file /opt/aerospike/data/test.dat
filesize 5G
data-in-memory true # Store data in memory in addition to file.
}
}
What am I doing wrong? Need some help here. I am very new to aerospike. I tried searching everywhere, but couldn't find anything.
UPDATE
I am now using IP address 172.28.128.4 (got it from ifconfig command) and port 3000 to connect to aerospike. I am now getting Socket Timeout Exception.
If you setup a single node on vagrant on Mac, and you running you are application in an ide on mac - say eclipse - locoalhost on vagrant is exposed to the mac as 172.28.128.3 typically. running ifconfig in your vagrant shell will confirm that. if your application is running inside vagrant itself, then 127.0.0.1 should work, in each case, your application should specify port 3000. thats where aerospike server is listening. amc is a webserver that talks to aerospike on port 3000 and serves the dashboard on port 8081 by default. so its a monitoring and management gateway to aerospike via a web browser. also, in your aerospike config, suggest you use mesh config instead of multicast though for a single node it does not matter - you are not making a cluster. If you are new, if you download CE, you get complimentary access to Aerospike Intro course in Aerospike Academy. Take advantage of that - few hours investment. Otherwise here are some intro videos on youtube. ( 02-Intro to Aerospike and 03-handson )
I setup a Mesos cluster with vm machines, machine A (master + agent) and machine B(agent only), and I also run Marathon and Mesos-dns on machine A, both agents support docker.
I startup a web application with docker via Marathon, the docker container run with bridge network mode.
When I run one instance, the container startup normally and Mesos-dns resolves it correct with the docker service internal IP (example, 172.17.0.2), but because I only run one instance and there are two agents in mesos cluster, so only one agent gets the container, the other one there is nothing, if client accesses the mesos agent which there nothing running, there is error appeared.
That means, the container is running on machine B, does not run on machine A, my docker application named test, and listen with port 5000, once I run "curl http://test.marathon.mesos:5000/" on machine B, I get the correct response, but when I run same command on machine A, there is an error appeared "curl: (7) Failed to connect to test.marathon.mesos port 5000: No route to host", the mesos-dns resolve domain to docker internal ip 172.17.0.2, but this ip is not appeared on machine A, because there is not any container running on machine A.
I also can run many instances on same Agent node without any problems, but as I know, mesos cluster and marathon are running application on agent node randomize, so all agent nodes behind load balancer could be accessed, if client access to agent node without container via load balancer, that's a problem for client.
my mesos-dns config file like below:
{
"zk":"zk://10.11.54.103:2181,10.11.54.103:2182,10.11.54.103:2183/mesos",
"masters": ["10.11.54.103:5050"],
"refreshSeconds": 60,
"ttl": 60,
"domain": "mesos",
"port": 53,
"resolvers": ["10.11.255.1","10.11.255.2","4.2.2.2"],
"timeout": 5,
"httpon": true,
"dnson": true,
"httpport": 8123,
"externalon": true,
"listener": "10.11.54.103",
"SOAMname": "ns1.mesos",
"SOARname": "root.ns1.mesos",
"SOARefresh": 60,
"SOARetry": 600,
"SOAExpire": 86400,
"SOAMinttl": 60,
"IPSources": ["netinfo", "mesos", "host"]
}
I wish the Mesos-dns can resole domain cross the whole mesos cluster, is there any idea?
From the documentation:
Mesos-DNS with Docker
If you choose to use Mesos-DNS with Docker, with a version of Mesos after 0.25, be aware that there are some caveats. By default the Docker executor publishes the IP of the Docker container into the NetworkInfo field. Unfortunately, unless you're running some kind of SDN solution, bridged, or host networking with Docker, this can prove to make the containers unreachable.
The default configuration that Mesos-DNS ships with in config.json.sample omits netinfo from the sources. The default options if you omit this field from the configuration includes netinfo. If you have trouble with Docker, ensure you check the IPSources field to omit netinfo.
IPSources defines a fallback list of IP sources for task records,
sorted by priority. If you use Docker, and enable the netinfo IPSource, it may cause tasks to become unreachable, because after Mesos 0.25, the Docker executor publishes the container's internal IP in NetworkInfo.
I'm using dnsmasq but I'm a little confused as to what gets set where. Everything works as expected, but I wasn't sure if any of my config parameters are redundant or would cause issues down the road.
1 - Do I need to set the recusors option in Consul's config?
2 - Do I still need both nameservers entry in /etc/resolv.conf?
3 - Do I need dnsmasq on all Consul clients or just the servers?
#/etc/dnsmasq.d/dnsmasq.conf`
server=/consul/127.0.0.1#8600
My Consul config looks like this:
{
"server": false,
"client_addr": "0.0.0.0",
"bind_addr": "0.0.0.0",
"datacenter": "us-east-1",
"advertise_addr": "172.16.11.144",
"data_dir": "/var/consul",
"encrypt": "XXXXXXXXXXXXX",
"retry_join_ec2": {
"tag_key": "SOMEKEY",
"tag_value": "SOMEVALUE"
},
"log_level": "INFO",
"recursors" : [ "172.31.33.2" ],
"enable_syslog": true
}
My /etc/resolv.conf looks like this:
nameserver 127.0.0.1
nameserver 172.31.33.2
1) read the documentation: https://www.consul.io/docs/agent/options.html#recursors having a recursor setup is great if you have external services registered in Consul, otherwise it's probably moot. You likely don't want ALL of your DNS traffic to hit consul directly, just the consul specific DNS traffic.
2 & 3:
It's up to you. Some people run dnsmasq on every machine. Some people centralize dnsmasq on their internal DNS servers. Both are valid configurations. If you run it on every single machine, then you probably just need 1 nameserver entry, pointed at localhost. If you run it centralized (i.e. just on your internal DNS servers) then you just point every machine at your internal DNS servers. Both are valid options.
I am new to OpenShift, and trying to run elasticsearch in a DIY cartridge. The elasticsearch app seems to be working, but I cannot connect to the 3306 port which it needs for node to node communication.
When I run the
$ rhc port-forward -a myapp
Forwarding ports ...
To connect to a service running on OpenShift, use the local address
Service Local OpenShift
------- -------------- ------------------------
java 127.0.0.1:3306 => 127.10.49.101:3306
java 127.0.0.1:9200 => 127.10.49.101:9200
But when I run my indexing groovy/grails app that uses an elasticsearch plugin to index data, it cannot connect to the OpenShift elasticsearch node. It should be using port 3306 to connect to the OpenShift elasticsearch node. The app config has
client.node = 'node'
client.hosts = [ [host:'127.0.0.1', port:3306 ] ]
Note that my indexing app worked fine when elasticsearch was run locally, but it does not connect to elasticsearch running on OpenShift.
I have 6 machines mesos cluster (3 masters and 3 slaves), I acces to mesos User interface 172.16.8.211:5050 and it works correctly and redirect to the leader if it is not. Then If I access to marathon User interface 172.16.8.211:8080 it works correctly. Summing before configuring and executing the consul-cluster marathon works well.
My problem is when I configure and run a consul cluster with 3 servers that are the mesos masters and 3 clients that are the mesos slaves. If I execute consul members it is fine, all the members alive and working together.
But now if I try to access to marathon User interface I can't, and I access to mesos User interface and I go to 'Frameworks' and does not appear marathon Framework.
ikerlan#client3:~$ consul members
Node Address Status Type Build Protocol DC
client3 172.16.8.216:8301 alive client 0.5.2 2 nyc2
client2 172.16.8.215:8301 alive client 0.5.2 2 nyc2
server2 172.16.8.212:8301 alive server 0.5.2 2 nyc2
server3 172.16.8.213:8301 alive server 0.5.2 2 nyc2
client1 172.16.8.214:8301 alive client 0.5.2 2 nyc2
server1 172.16.8.211:8301 alive server 0.5.2 2 nyc2
In Slaves tab of mesos I could see the next:
-Mesos version: 0.27.0
-Marathon version: 0.15.1
I have the next file logs, where would appear something related with this issue?
What could be the problem?
Solution:
I have see in the marathon logs '/var/log/syslog' that the problem is a problem of DNS. So I try to add the IPs of the other hosts of the cluster to the file /etc/hosts. And it resolv the problem, now it works perfectly.
You can add all the cluster hosts to the zookeeper config file, it would work