Kong consul SRV configuration - consul

I am using 0.10.1 version of Kong which supports SRV queries. I am using consul's dns and trying to integrate it with Kong. Looking at the documentation
https://getkong.org/docs/0.10.x/loadbalancing/
I am not sure how do I specify my upstream_url in kong
For example:
# This tries reaching my service on port 80 instead of the port returned by SRV query
upstream_url = http://ms.service.consul/ms
# As per the doc I can specify a port and it is supposed to proxy the port
upstream_url = http://ms.service.consul:8989/ms
What should my upstream_url config look like with SRV records from consul
upstream_url = http://myservice.service.consul:{{do_i_need_something_here}}/myservice
Will Kong be able to automatically detect the ports where my service is running or am I missing some important point.

Related

Use Consul as xds management server for envoy proxy without consul connect

I am working on a project where I want to configure envoy using consul as the xds server. But I don't want to use consul connect. I only want to use consul for service discovery and the xds server.
I have started the consul agent with -dev mode enabling the grpc endpoints and I have registered a service hello-service which i can see in the consul ui.
My barebone envoy config
# admin web panel
admin:
access_log_path: ./admin_logs
address:
socket_address:
address: 0.0.0.0
port_value: 9902
dynamic_resources:
cds_config:
ads: {}
ads_config:
api_type: grpc
transport_api_version: v3
grpc_services:
- google_grpc:
target_uri: http://localhost:8502
stat_prefix: grpc-xds-service
I am trying to fetch some configuration like clusters/endpoints from consul for the registered hello-service, But when i start the envoy process I am getting this warning.
[2021-07-11 19:16:05.782][11825][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 13,
What I could make of this error is that the connection to the management server is failing. But In the consul logs i can see the grpc port is up.
Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
Consul's xDS server is not designed to be used outside of Connect, and as such has not been tested or validated to be able to provide CDS/EDS data outside of that context.
You may want to take a look at consul-envoy-xds or envoy-control, both of which may provide the xDS functionality you're looking for without requiring you to deploy a full-blown service mesh.

Is it possible to access Grafana and Prometheus through reverse proxy using Nginx on same server

Please is it possible to configure reverse proxy using nginx for Grafana and Prometheus on same server. I have configured Prometheus access through https(listening on port 443 and direct output to port 9090). This works fine but configuring Grafana which is on same server to be accessed through https has been impossible. I tried it listening on port 80 and direct its output to port 3000, but it always default to http port. I also tried another port for listening but never worked.
Has anyone done this before and please can you share your valuable experience. Thanks.
Maybe this docker compose can be helpful https://github.com/vegasbrianc/prometheus/blob/master/README.md
The suggestion is to move the ssl termination to any web server (NGinx, Traefik, HAProxy) and forward the request in plain text to the underline services (prometheus and grafana). Here some examples: HAProxy exposes prometheus and Traefik

Every request in being resolved to port 80 in Consul

I am very new to consul and have registered a service with consul with following configuration.
{"service": {"name": "wrapper", "tags": ["consul-wrapper"], "port":8000,"address":"127.0.0.1",}}
I have also set up dnsmasq so that dns queries get resolved by consul server running on 127.0.0.1:8600
Now whenever I try to access my service like wrapper.service.consul it gets resolved and goes to port 80 instead of 8000.
What am i missing here?
You have to ask DNS specifically for the port number... it's an SRV request.
by default DNS queries are A(or AAA for ipv6) queries, and you just get the IP address.
try asking for the SRV record directly like so:
dig SRV wrapper.service.consul

camel elasticsearch access port 80

I have port 9200 proxied via 80 on my server running elasticsearch. I have a camel route that needs to index documents to this server.
Is it supported in the camel-elasticsearch plugin? ie access elastic search via a non 9300 port?
I understand that port 9300 uses a native elasticsearch transport protocol.
What are my options here? Can I proxy 9300 via apache? I'm not sure if that works.
Or does the camel-elasticsearch plugin support http transport? Please help. Thanks.
The port you are referring to, it is the HTTP port:
http.port A bind port range. Defaults to 9200-9300.
http.publish_port The port that HTTP clients should use when communicating with this node. Useful when a cluster node is behind a
proxy or firewall and the http.port is not directly addressable from
the outside. Defaults to the actual port assigned via http.port.
http.bind_host The host address to bind the HTTP service to. Defaults to http.host (if set) or network.bind_host.
http.publish_host The host address to publish for HTTP clients to connect to. Defaults to http.host (if set) or
network.publish_host.
http.host Used to set the http.bind_host and the http.publish_host Defaults to http.host or network.host.
So you really don't need a proxy, you can have elasticsearch listen directly on port 80.
If you already have a process running on port 80; then you can proxy the connections to 9200 (and leave elastic search as default).
Java Transport Client ---> Apache HTTP Proxy(80) ----> ES (9300) [ Can
I do this? As I understand that Java Transport Client uses a non-http
protocol? ]
The protocol has nothing to do with the port.
Simply pass 80 to InetSocketTransportAddress. See the documentation for a complete example.

Modify cloudera manager port 7180 to 80

My server offered by boss can access by port 80.
How can I configure the Web UI port 7180 to 80?
It doesn't work that I modified the server_port in /etc/cloudera-scm-agent/config.ini
I can't access the UI, so the following does not work:
Cloudera Server Ports
I need configure it in config files.
I have strong belief that you should NOT change this port. It's possible in general, however you may meet some issues like those one in your case.
I can suggest you to use reverse proxy server such as Nginx or Apache. It's much safer and maybe even faster.
So as result I'll get the following proxying chain which is fully transparent to clients:
Client (your Boss) connects to server via port 80
Nginx (or Apache) is listening port 80
Nginx sends HTTP requests to Cloudera on port 7180
Nginx returns request result to client (your Boss)

Resources