How consul constructs SRV record - consul

Let say I registered a service in consul, so that I can query it by something like:
curl http://localhost:8500/v1/catalog/service/BookStore.US
and it returns
[
{
"ID": "xxxxx-xxx-...",
"ServiceName": "BookStore.US",
...
}
]
If I am using consul directly in my code, it is ok. But the problem is that when I want to use the SRV record directly, it does not work.
Normally, there is a service record created by consul with the name service_name.service.consul. In the above case, it is "BookStore.US.service.consul"
So that you can use "dig" command to get it.
dig #127.0.0.1 -p 8600 BookStore.US.service.consul SRV
But when I tried to "dig" it, it failed with 0 answer session.
My question:
How does consul construct the service/SRV name (pick up some fields in the registered consul record and concat them?)
Is there any way for me to search the SRV records with wildcards, so that at least I can search the SRV name by using the key word "BookStore"

The SRV lookup is not working because Consul is interpreting the . in the service name as domain separator in the hostname.
Per https://www.consul.io/docs/discovery/dns#standard-lookup, service lookups in Consul can use the following format.
[tag.]<service>.service[.datacenter].<domain>
The tag and datacenter components are optional. The other components must be specified. Given the name BookStore.US.service.consul, Consul interprets the components to be:
Tag: BookStore
Service: US
Sub-domain: service
TLD: consul
Since you do not have a service registered by the name US, the DNS server correctly responds with zero records.
In order to resolve this, you can do one of two things.
Register the service with a different name, such as bookstore-us.
{
"Name": "bookstore-us",
"Port": 1234
}
Specify the US location as a tag in the service registration.
{
"Name": "bookstore",
"Tags": ["us"],
"Port": 1234
}
Note that in either case, the service name should be a valid DNS label. That is, it may contain only the ASCII letters a through z (in a case-insensitive manner), the digits 0 through 9, and the hyphen-minus character ('-').
The SRV query should then successfully return a result for the service lookup.
# Period in hostname changed to a hyphen
$ dig -t SRV bookstore-us.service.consul +short
# If `US` is a tag:
# Standard lookup
$ dig -t SRV us.bookstore.service.consul +short
# RFC 2782-style lookup
$ dig -t SRV _bookstore._us.service.consul +short

Related

Get failed service name inside Consul watch handler

I'm using Consul to monitor services health status. I use Consul watch command to fire a handler when some service is failed. Currently I'm using this command:
consul watch -type=checks -state=passing /home/consul/health.sh
This works, however I'd like to know inside health.sh the name of the failed service, so I could send a proper alert message containing failed service name. How can I get failed service name there?
Your script could get all the required information by reading from stdin. Information will be sent as JSON. You can easily examine those events by simply adding cat - | jq . into your handler.
The check information outputted by consul watch -type=check contains a ServiceName field that contains the name of the service the check is associated with.
[
{
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",
"Status": "passing",
"Notes": "",
"Output": "",
"ServiceID": "redis",
"ServiceName": "redis"
}
]
(See https://www.consul.io/docs/dynamic-app-config/watches#checks for official docs.)
Checks associated with services should have values in both the ServiceID and ServiceName fields. These fields will be empty for node level checks.
The following command watches changes in health checks, and outputs the name of a service when its check transitions to a state other than "passing" (i.e., warning or critical).
$ consul watch -type=checks -state=passing "jq --raw-output '.[] | select(.ServiceName!=\"\" and .Status!=\"passing\") | .ServiceName'"

Retrieve the instance ID after deploying at Vultr with the vultr-cli?

I'm scripting a process with the vultr-cli. I need to deploy a new VPS at Vultr, perform some intermediate steps, then destroy the VPS in a bash script. How do I retrieve instance values in the script after the deployment? Is there a way to capture the information as JSON or set environment variables directly?
So far, my script looks like:
#!/bin/bash
## Create an instance. How do I retrieve the instance ID
## for use later in the script?
vultr-cli instance create --plan vc2-1c-1gb --os 387 --region ewr
## With the instance ID, retrieve the main IPv4 address.
## Note: I only want the main IP, but there may be multiple.
vultr-cli instance ipv4 list $INSTANCE_ID
## Perform some tasks here with the IPv4. Assuming I created
## the instance with my SSH key, for example:
scp root#$INSTANCE_IPv4:/var/log/logfile.log ./logfile.log
## Destroy the instance.
vultr-cli instance delete $INSTANCE_ID
The vultr-cli will output the response it gets back from the API like this
☁ ~ vultr-cli instance create --plan vc2-1c-1gb --os 387 --region ewr
INSTANCE INFO
ID 87e98eb0-a189-4519-8b4e-fc46bb0a5331
Os Ubuntu 20.04 x64
RAM 1024
DISK 0
MAIN IP 0.0.0.0
VCPU COUNT 1
REGION ewr
DATE CREATED 2021-01-23T17:39:45+00:00
STATUS pending
ALLOWED BANDWIDTH 1000
NETMASK V4
GATEWAY V4 0.0.0.0
POWER STATUS running
SERVER STATE none
PLAN vc2-1c-1gb
LABEL
INTERNAL IP
KVM URL
TAG
OsID 387
AppID 0
FIREWALL GROUP ID
V6 MAIN IP
V6 NETWORK
V6 NETWORK SIZE 0
FEATURES []
So you would want to capture the ID and it's value from the response. This is a crude example but it does work.
vultr-cli instance create --plan vc2-1c-1gb --os 387 --region ewr | grep -m1 -w "ID" | sed 's/ID//g' | tr -d " \t\n\r"
We are grepping for the 1st line that has ID (which will always be be the 1st line). Then remove the word ID followed by removing any whitespace and newlines.
You will want to do something similar to with the ipv4 list call you have.
Again, there may be a better way to write out the grep/sed/tr portion but this will work for your needs. Hopefully this helps!

How to set Consul Alias Service

I currently have 2 services running on a single node (EC2 instance) with a consul client. I would like to health check both of these by hitting a single endpoint, namely: http://localhost:8500/v1/agent/health/service/id/AliasService based on the information Consul provides from https://www.consul.io/api/agent/service.html.
The issue is that I can't seem to find any sort of documentation regarding this AliasService, just that I can use it to run health checks. I've tried putting it into my service definitions but to no avail. It just seems to ignore it altogether.
It seems that what you need is to manually define both services and then attach HTTP health check to one of them and alias health check to the other. What is being aliased here is not service but health check.
For example:
$ consul services register -name ssh -port 22
$ consul services register -name ssh-alias -port 22 -address 172.17.0.1
$ cat >ssh-check.json
{
"ID": "ssh",
"Name": "SSH TCP on port 22",
"ServiceID": "ssh",
"TCP": "localhost:22",
"Interval": "10s",
"Timeout": "1s"
}
$ curl --request PUT --data #ssh-check.json http://127.0.0.1:8500/v1/agent/check/register
$ cat >ssh-alias-check.json
{
"ID": "ssh-alias",
"Name": "SSH TCP on port 22 - alias",
"ServiceID": "ssh-alias",
"AliasService": "ssh"
}
$ curl --request PUT --data #ssh-alias-check.json http://127.0.0.1:8500/v1/agent/check/register
Here I have defined two separate services and two health checks. But only the first health check is doing actual work, the second is aliasing health status from one service to the other.

How can I monitor a router with a intern ssl certificate in Nagios?

This ist my current setup:
Host config:
define host{
use generic-host ; Inherit default values from a template
host_name A+A ; The name we're giving to this host
alias A+A Objektausstattung Router ; A longer name associated with the host
address https://87.139.203.190:444 ; IP address of the host
hostgroups Router ; Host groups this host is associated with
}
Service config:
define service{
use generic-service ; Inherit default values from a template
host_name A+A
service_description HTTP
check_command check_http
}
I´ll get this error from Nagios:
check_icmp: Failed to resolve https://87.139.203.190:444
What am I doing wrong here ?
Nagios tries to resolve to ip-address and port. Try ip-address only.
address https://87.139.203.190 ; IP address of the host
Your host definition should only specify an IP address for the 'address'. The URL is not an attribute of the host, but of the HTTP check your want to perform.
The Service definition specifies the check_command, which is in turn defined in the checkcommands.cfg file. This will specify exactly what command is to be run, possibly using additional parameters passed.
You will probably want to pass the port number as a parameter, and that you are to use HTTPS. How to do this will depend on your settings. For example, you could use this in your checkcommands.cfg:
define command{
command_name check_https
command_line $USER1$/check_http -t 12 -H $HOSTADDRESS$ -f ok --ssl=1 -u "$ARG1$" -p "$ARG2$" -w $ARG3$ -c $ARG4$
}
Then you could configure your service with a checkcommand thus:
check_command check_https!/!444!1!5
This would check for the url http://87.139.203.190:444/, giving a warning if it takes over 1s and a critical if it takes over 5s to complete. TLSv1 would be used (else you might get a false positive on web servers with Poodle protection).

Remove EC2's entry from resolv.conf

I have private DNS servers and I want to write them to resolv.conf with resolvconf on Debian on AWS/EC2.
There is a problem in the order of nameserver entries.
In my resolv.conf, EC2's default nameserver is always written at first line like so:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.16.0.23
nameserver 10.0.1.185
nameserver 10.100.0.130
search ap-northeast-1.compute.internal
172.16.0.23 is EC2's default nameserver and others are mine.
How to remove EC2 entry? Or, how to move EC2 entry to third?
Here I have an interface file:
% ls -l /etc/resolvconf/run/interface/
-rw-r--r-- 1 root root 62 Jun 7 23:35 eth0
It seems that the file eth0 is automatically generated by dhcp so can't remove it permanently.
% cat /etc/resolvconf/run/interface/eth0
search ap-northeast-1.compute.internal
nameserver 172.16.0.23
My private DNS entry is here:
% cat /etc/resolvconf/resolv.conf.d/base
nameserver 10.0.1.185
nameserver 10.100.0.130
Please help.
I think I just solved a very similar problem. I was bothered by Amazon EC2's crappy internal DNS servers so I wanted to run a local caching dnsmasq daemon and use that in /etc/resolv.conf. At first I just did echo nameserver 127.0.0.1 > /etc/resolv.conf but then I realized that my change would eventually be overwritten by the DHCP client after a reboot or DHCP lease refresh.
What I've now done instead is to edit /etc/dhcp3/dhclient.conf and uncomment the line prepend domain-name-servers 127.0.0.1;. You should be able to use the prepend directive in a very similar way.
Update: These instructions are based on Ubuntu Linux but I imagine the general concept applies on other systems as well, even other DHCP clients must have similar configuration options.
I'm approaching this problem from the other direction (wanting the internal nameservers), much of what I've learned may be of interest.
There are several options to control name resolution in the VPC management console.
VPC -> DHCP option sets -> Create dhcp option set
You can specify your own name servers there.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_DHCP_Options.html
Be sure to attach this dhcp option set to your VPC to get it to take effect.
Alternatively (I found this out by mistake) local dns servers are not set if the following settings are disabled in VPC settings:
DnsHostnames
and
DnsSupport
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html
Settings can also be overridden locally (which you'll notice if you move instances between vpcs). /etc/dhcp/dhclient.conf
The following line might be of interest:
prepend domain-name-servers
Changes, of course, take effect on dhclient start.
How do I assign a static DNS server to a private Amazon EC2 instance running Ubuntu, RHEL, or Amazon Linux?
Short Description
Default behavior for an EC2 instance associated with a virtual private cloud (VPC) is to request a DNS server address at startup using the Dynamic Host Configuration Protocol (DHCP). The VPC responds to DHCP requests with the address of an internal DNS server. The DNS server addresses returned in the DHCP response are written to the local /etc/resolv.conf file and are used for DNS name resolution requests. Any manual modifications to the resolv.conf file are overwritten when the instance is restarted.
Resolution
To configure an EC2 instance running Linux to use static DNS server entries, use a text editor such as vim to edit the file /etc/dhcp/dhclient.conf and add the following line to the end of the file:
supersede domain-name-servers xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx;
Ubuntu - dhclient.conf - DHCP client configuration file 
The supersede statement
supersede [ option declaration ] ;
If for some option the client should always use a locally-configured value or values
rather than whatever is supplied by the server, these values can be defined in the
supersede statement.
The prepend statement
prepend [ option declaration ] ;
If for some set of options the client should use a value you supply, and then use the
values supplied by the server, if any, these values can be defined in the prepend
statement. The prepend statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
The append statement
append [ option declaration ] ;
If for some set of options the client should first use the values supplied by the server,
if any, and then use values you supply, these values can be defined in the append
statement. The append statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
In here someone come with solution that basically replaces the file on boot using rc.local
https://forums.aws.amazon.com/thread.jspa?threadID=74497
Edit /etc/sysconfig/network-scripts/ifcfg-eth0 to say PEERDNS=no
Create a file called /etc/resolv.backup with what you want
Add the following 2 lines to /etc/rc.local:
rm -f /etc/resolv.conf cp /etc/resolv.backup /etc/resolv.conf
This is what we are doing for our servers in the environment.
interface "eth0"
{
prepend domain-name-servers 10.x.x.x;
supersede host-name "{Hostname}";
append domain-search "domain";
supersede domain-name "DOMAIN";
}
Hope this helps.
The following worked in a Debian stretch on AWS EC2.
Just create /etc/dhcp/dhclient-enter-hooks.d/nodnsupdate:
#!/bin/sh
make_resolv_conf(){
:
}
Then you can modify /etc/resolv.conf and it will persist your changes across restarts.
Setup in crontab as
#reboot cp -r /home/.../resolv.conf /etc/resolv.conf

Resources