Using dnsmaqs with Consul and the confusion around recursive - consul

Question

I'm using dnsmasq but I'm a little confused as to what gets set where. Everything works as expected, but I wasn't sure if any of my config parameters are redundant or would cause issues down the road.
1 - Do I need to set the recusors option in Consul's config?
2 - Do I still need both nameservers entry in /etc/resolv.conf?
3 - Do I need dnsmasq on all Consul clients or just the servers?
#/etc/dnsmasq.d/dnsmasq.conf`
server=/consul/127.0.0.1#8600
My Consul config looks like this:
{
"server": false,
"client_addr": "0.0.0.0",
"bind_addr": "0.0.0.0",
"datacenter": "us-east-1",
"advertise_addr": "172.16.11.144",
"data_dir": "/var/consul",
"encrypt": "XXXXXXXXXXXXX",
"retry_join_ec2": {
"tag_key": "SOMEKEY",
"tag_value": "SOMEVALUE"
},
"log_level": "INFO",
"recursors" : [ "172.31.33.2" ],
"enable_syslog": true
}
My /etc/resolv.conf looks like this:
nameserver 127.0.0.1
nameserver 172.31.33.2

Answer 1

1) read the documentation: https://www.consul.io/docs/agent/options.html#recursors having a recursor setup is great if you have external services registered in Consul, otherwise it's probably moot. You likely don't want ALL of your DNS traffic to hit consul directly, just the consul specific DNS traffic.
2 & 3:
It's up to you. Some people run dnsmasq on every machine. Some people centralize dnsmasq on their internal DNS servers. Both are valid configurations. If you run it on every single machine, then you probably just need 1 nameserver entry, pointed at localhost. If you run it centralized (i.e. just on your internal DNS servers) then you just point every machine at your internal DNS servers. Both are valid options.