No such server found while disabling HAproxy disable server from cluster - ansible

My haproxy.cfg
global
log 127.0.0.1 local0
maxconn 20000
user haproxy
group haproxy
stats socket /var/run/haproxy/haproxy.sock level admin
stats timeout 2m
listen admin
bind *:8080
stats enable
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
# timeout http-request 5s
timeout connect 5000
timeout client 60000
timeout server 60000
frontend http-in
bind *:80
default_backend monkey
backend monkey
stats enable
stats uri /haproxy?stats
stats realm Strictly\ Private
stats auth admin:hello
stats auth Another_User:hi
mode http
compression algo gzip
compression type text/html text/plain text/css application/json
balance roundrobin
option httpclose
option forwardfor
default-server inter 1s fall 1
server cd-test-1 1.2.3.4:80 check
server cd-test-2 5.6.7.8:80 check
I have been using socat to disable a node from HAproxy cluster.
below is the command
echo "disable server monkey/cd-test-1"| socat stdio /var/run/haproxy/haproxy.sock
The above disables my node from haproxy. But if I use the ip address(1.2.3.4) instead of "cd-test-1" it returns No such server.
I am using ansible to automated this. I use {{inventory_hostname}} and delegate the command to my HAproxy server. Hence the issue.
- name: Disable {{ inventory_hostname }} in haproxy and letting the services drain
shell: echo "disable server monkey/{{inventory_hostname}}"| socat stdio /var/run/haproxy/haproxy.sock
become_user: root
delegate_to: "{{ item }}"
with_items: groups.haproxy_backend
This returns "No such server." and moves along.
Can someone please help me find the issue with using the IP instead of the name of server. Might be doing something very silly. Any help is appreciated.

Disabling and enabling HAproxy using socat, the server alias name has to be mentioned.
Otherwise, we will be getting a No such server error.

Related

Haproxy 503 service unavailable

We are running Haproxy and 3 nginx processes inside a single docker on the host network to access it from localhost. We have installed HaProxy version 2.6.6. 3 nginx servers are running behind it as backend servers. We are able to access them individually What is wrong with the haproxy configuration here? we are always getting the below error on curl (curl http://localhost:80) requests. Please suggest
<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
pidfile /usr/local/etc/haproxy/haproxy.pid
defaults
timeout http-request 2000
timeout queue 1000
timeout check 2000
timeout connect 2000
timeout client 5000
timeout server 5000
log global
option dontlognull
option clitcpka
option srvtcpka
option tcpka
unique-id-format %[uuid()]
unique-id-header X-Request-ID
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %ID %{+Q}r"
frontend http-in
mode http
option forwardfor
bind *:80
use_backend webservers
backend webservers
mode http
server localhost-01 localhost:8000
server localhost-02 localhost:8001
server localhost-03 localhost:8002

Experiencing latency with haproxy load balancer

I'm experiencing high latency with haproxy load balancer when backend configuration uses private network IP addresses.
But when I replace backend server addresses with public IP address or Reverse DNS name I experience no latency.
What is causing the latency?
If one uses public IP or fqdn or
Reverse DNS name, does network traffic bypasses haproxy?
Is it
allowed to use public IP or fqdn or Reverse DNS name for backend
servers in haproxy conf?
Configuration With Private Network IP Addresses
global
log /dev/log local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
maxconn 18000
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
defaults
log global
mode http
option httplog
timeout client 30s
timeout connect 4s
timeout server 30s
frontend www
bind *:80
default_backend webservers
backend webservers
mode http
balance roundrobin
server server1 10.0.0.20:80
server server2 10.0.0.30:80
Configuration With Reverse DNS or Public IP Addresses
global
log /dev/log local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
maxconn 18000
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
defaults
log global
mode http
option httplog
timeout client 30s
timeout connect 4s
timeout server 30s
frontend www
bind *:80
default_backend webservers
backend webservers
mode http
balance roundrobin
server server1 90-80-70-40.aws.com:80
server server2 90-80-70-50.aws.com:80
It looks like it was a DNS issue on the host Windows machines. Added entries to host file and it worked as expected with private network IP settings.

haproxy + spring boot writeAddress(..) failed: Connection reset by peer

I'm running HAproxy with backend Spring Boot Rest controllers. My spring log shows constant errors that look like the following:
[reactor-http-epoll-26] ERROR o.s.w.s.a.HttpWebHandlerAdapter - [9df8bfcf] Error [io.netty.channel.unix.Errors$NativeIoException: writeAddress(..) failed: Connection reset by peer] for HTTP GET "/api/v1/status", but ServerHttpResponse already committed (200 OK)
HAproxy performs an HTTP check on the url /api/v1/status. What would be the reason that I'm getting these errors?
HAProxy Config
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
# daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
ssl-default-bind-ciphers EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EX$
ssl-default-bind-options no-sslv3 no-tlsv10
tune.ssl.default-dh-param 4096
defaults
mode http
log global
option httplog
option http-server-close
option forwardfor except 127.0.0.0
option redispatch
retries 3
timeout http-request 30s
timeout queue 1m
timeout connect 30s
timeout client 30s
timeout server 30s
frontend https-in
bind *:443 ssl crt /etc/cert.pem
default_backend api
backend api
mode http
option httpchk GET /api/v1/status HTTP/1.0
http-check expect status 200
balance roundrobin
server api1 127.0.0.1:8001 check fall 3 rise 2
server api2 127.0.0.1:8002 check fall 3 rise 2
HAproxy is doing GET request, reads http response code and closing connection.
Boot is trying to send remaining parts (http headers and some json payload) but connection is already closed.
Just replace GET with OPTIONS in the line:
option httpchk GET /api/v1/status HTTP/1.0

HA Proxy fails to recognize Tomcat:8080 running on a mac

Can someone help be understand why haproxy fails to detect my tomcat instance running on port 8080? I can reach the server just fine when browsing to 127.0.0.1:8080
On start up haproxy spits out the following error:
WARNING] 141/163609 (23336) : Server cim/cim is DOWN, reason: Layer7 wrong status, code: 404, info: "Not Found", check duration: 1ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 141/163609 (23336) : backend 'cim' has no server available!
Here's the haproxy config.
global
maxconn 128
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
tune.ssl.default-dh-param 2048
defaults
mode http
log global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
option httplog
option forwardfor
option httpclose
option redispatch
option httpchk HEAD / HTTP/1.0
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 20s
timeout check 10s
frontend in-http
bind *:80
acl restricted_urls path_beg /trm/ws/v2
use_backend cim if restricted_urls
backend cim
balance roundrobin
server cim 127.0.0.1:8080 check
Tomcat is definitely up and running, and I can telnet to 127.0.0.1 8080 and receive an http response.

Haproxy redirect configuration for plex?

Hope someone can help me :)
I try to configure HAProxy for plex redirection but didn't found the solution yet.
So basically to run plex home page you should go to => IPADRESS:PORT/web which redirect to IPADRESS:PORT/web/index.html
I made this kind of redirect:
use_backend plex if { hdr_beg(Host) -i plex. }
backend plex
server plex localhost:32400 check
This is ok, i can join plex => plex.mydomain.tld/web
But i would like to be able to join plex with this URL => plex.mydomain.tld
I tried to add this line:
reqrep ^([^\ :]*)\ /(.*) \1\ /web\2
Changing is fine, my URL switch to => plex.mydomain.tld/web/index.html
But i have a 404 ERROR...
What kind of trick i should do to acces plex from plex.mydomain.tld ?
Thanks !
Found some info that helped me figure it out:
global
log 127.0.0.1 syslog
maxconn 1000
user haproxy
group haproxy
daemon
tune.ssl.default-dh-param 4096
ssl-default-bind-options no-sslv3 no-tls-tickets
ssl-default-bind-ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
defaults
log global
mode http
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
option contstats
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout check 10s
listen stats
bind *:9090
mode http
maxconn 10
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth admin:admin
frontend ALL
bind *:80
bind *:443 ssl crt /etc/haproxy/certs/nomercy.myqnapcloud.com.pem crt /etc/haproxy/certs/nomercy.myqnapcloud.com.pem
mode http
# Define path for lets encrypt
acl is_letsencrypt path_beg -i /.well-known/acme-challenge/
use_backend letsencrypt if is_letsencrypt
# Define hosts
acl host_website hdr(host) -i nomercy.myqnapcloud.com
# Direct hosts to backend
use_backend website if host_website
# Redirect port 80 to 443
# But do not redirect letsencrypt since it checks port 80 and not 443
redirect scheme https code 301 if !{ ssl_fc } !is_letsencrypt
backend letsencrypt
server letsencrypt 127.0.0.1:8888
backend website
balance roundrobin
option httpchk GET /check
cookie SERVERID insert indirect nocache
http-check expect rstring ^UP$
default-server inter 3s fall 3 rise 2
server server1 192.168.2.151:8888 check
server server2 192.168.2.152:8888 check
server server3 192.168.2.153:8888 check
listen plex
bind *:32400 ssl crt /etc/haproxy/certs/nomercy.myqnapcloud.com.pem crt /etc/haproxy/certs/nomercy.myqnapcloud.com.pem
balance roundrobin
option httpchk GET /check
http-check expect rstring ^UP$
default-server inter 3s fall 3 rise 2
server server1 192.168.2.149:32400 check port 8888
server server2 192.168.2.148:32400 check port 8888
server server3 192.168.2.147:32400 check port 8888
You can remove the ssl credentials if you don't have it installed.
The problem here unfortunately has nothing to do with your HAProxy configuration. Instead it is Plex that is causing the issue.
Example
With your configuration, when you go to plex.mydomain.tld HAProxy is adding in the /web and as a result Plex gets the following url: plex.mydomain.tld/web. This is correct however Plex will then want to forward the browser on to plex.mydomain.tld/web/index.html. But when the browser sends a request or that url, HAProxy steps in and adds that extra /web again and the resulting url that is set to Plex is plex.mydomain.tld/web/web/index.html which doesn't exist and hence the 404 error you got.
While going to plex.mydomain.tld/index.html may work, I assume all links from that page to any other page won't work due to the say issue.
To solve this you could
Look through Plex's configuration to see if it is possible to run it with out the /web.
Taking inspiration from here, you could configure HAProxy something like this:
frontend http
mode http
bind *:80
acl plex hdr_beg(Host) -i plex.
acl root_dir path_reg ^$|^/$
acl no_plex_header req.hdr_cnt(X-Plex-Device-Name) -i 0
redirect location http://plex.mydomain.tld/web/index.html 301 if no_plex_header root_dir plex
use_backend plex if plex
backend plex
server plex localhost:32400 check
The key difference being the redirect location line which will redirect from / to /web/index.html if the header X-Plex-Device-Name isn't set. The reason you have to check for the header is that it appears that plex uses / for something else.
Note: This config is an example and I haven't tested this at all
Hope that helps.
I want to echo that I used the solution provided by JamesStewy and it worked, with the minor correction;
redirect location http://plex.mydomain.tld/web/index.html code 301 if no_plex_header root_dir plex
At least, that was necessary for me (running haproxy 1.7.2).

Resources