I am using go-gin as server and rendering an html using the code like the following
func dashboardHandler(c *gin.Context) {
c.HTML(200, "dashboard", gin.H{
"title": "Dashboard"
})
Along with title I want to pass the remote client's IPV4 address as well. I tried using the following code to get the IP address but for localhost it gives me ::1:56797 as output. My server is running on localhost:8080
ip, port, err := net.SplitHostPort(c.Request.RemoteAddr)
fmt.Println(ip + ":" + port)
if err != nil {
fmt.Println(err.Error())
}
I followed Correct way of getting Client's IP Addresses from http.Request (Golang) for reference. Is there any way I get the IPV4 address from the request?
you can use this function to get the ip and user agent, but it will give a bracket character if you are trying from localhost but if you try from somewhere else it will work.
func GetIPAndUserAgent(r *http.Request) (ip string, user_agent string) {
ip = r.Header.Get("X-Forwarded-For")
if ip == "" {
ip = strings.Split(r.RemoteAddr, ":")[0]
}
user_agent = r.UserAgent()
return ip, user_agent
}
Related
Trying to proxy a request, and I discovered the host header is not being present in the headers received
UPDATE: as pointed out from one of the answers, I might need the Host field, this doesn't change either, host headers are still not present.
code:
request, err := http.ReadRequest(bufio.NewReader(conn))
if err != nil {
log.Println("Error reading request", err)
return
}
target, err := net.Dial("tcp", request.URL.Host)
if err != nil {
log.Println(host)
log.Println("Error reaching target", err)
}
defer target.Close()
curl command
curl -v localhost:8080
Output:
Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.79.1
> Accept: */*
>
So host header is present in the curl.
Added a debug step to the program to print headers
fmt.Println("Headers:")
for key, values := range request.Header {
fmt.Printf("%s: %v\n", key, values)
}
Output from program:
Headers:
Accept: [*/*]
User-Agent: [curl/7.79.1]
Still no presence of Host header
Now I tried passing a host header with curl:
curl -v -H 'Host: localhost:3000' localhost:8080
Same output, header present in curl but not on the program
What you want is the Host field of the Request type. From the Godoc:
// For server requests, Host specifies the host on which the
// URL is sought. For HTTP/1 (per RFC 7230, section 5.4), this
// is either the value of the "Host" header or the host name
// given in the URL itself. For HTTP/2, it is the value of the
// ":authority" pseudo-header field.
// It may be of the form "host:port". For international domain
// names, Host may be in Punycode or Unicode form. Use
// golang.org/x/net/idna to convert it to either format if
// needed.
// To prevent DNS rebinding attacks, server Handlers should
// validate that the Host header has a value for which the
// Handler considers itself authoritative. The included
// ServeMux supports patterns registered to particular host
// names and thus protects its registered Handlers.
//
// For client requests, Host optionally overrides the Host
// header to send. If empty, the Request.Write method uses
// the value of URL.Host. Host may contain an international
// domain name.
Host string
I'm using Fiber as an HTTP client to make some requests to an http server, however I'm being rate limited. On my vm I configured 5 different IP addresses (public/private) and have confirmed that they are indeed connected to the internet.
curl --interface 10.0.0.4 ipinfo.io/json
curl --interface 10.0.0.5 ipinfo.io/json
...
curl --interface 10.0.0.8 ipinfo.io/json
each one returns a different public facing ip address.
Now I'm interested in making round-robin requests using these local addresses but I'm not so sure how to go about it.
Is there some sort of property or function I can set/call to change where the outgoing request is coming from?
I've looked around at fasthttp.HostClient which fiber.Agent extends but I didn't see anything useful.
Thanks guys.
a := fiber.AcquireAgent()
req := a.Request()
req.Header.SetMethod(fiber.MethodGet)
req.SetRequestURI(fmt.Sprintf(formatUrl, args...))
if err := a.Parse(); err != nil {
h.Logger.Error("%v", err)
return fiber.StatusInternalServerError, nil, []error{err}
}
customDialer := fasthttp.TCPDialer{
Concurrency: 1000,
LocalAddr: &net.TCPAddr{
IP: h.IPPool[atomic.AddUint32(&h.IpIdx, 1)%uint32(len(h.IPPool))],
},
}
a.HostClient.Dial = func(addr string) (net.Conn, error) {
return customDialer.Dial(addr)
}
Creating a custom dialer and dial func allows you to change the local address associated with the http request.
I am trying to implement peer discovery logic through DNS in go using SRV records in cluster. I have headless service and statefulset pods ready and I am able to list all SRV records by using
kubectl run -it srvlookup --image=tutum/dnsutils --rm --restart=Never -- dig SRV demoapp.default.svc.cluster.local
but the following code does not work in cluster:
func pingdns() (url string) {
log.Println("start ping demoapp.default.svc.cluster.local.")
_, addrs, err := net.LookupSRV("dns-tcp", "tcp", "demoapp.default.svc.cluster.local")
if err != nil {
log.Println(err.Error())
return "dns wrong"
}
fmt.Println(addrs)
return "dns done."
}
error output:
lookup _dns-tcp._tcp.demoapp.default.svc.cluster.local on 10.96.0.10:53: no such host
I found example in this k8s-in-action book but it is written in NodeJS. How to do it in Golang ?
const dns = require('dns');
const dataFile = "/var/data/kubia.txt";
const serviceName = "kubia.default.svc.cluster.local";
const port = 8080;
...
var handler = function(request, response) {
if (request.method == 'POST') {
...
} else {
response.writeHead(200);
if (request.url == '/data') {
var data = fileExists(dataFile)
? fs.readFileSync(dataFile, 'utf8')
: "No data posted yet";
response.end(data);
} else {
response.write("You've hit " + os.hostname() + "\n");
response.write("Data stored in the cluster:\n");
dns.resolveSrv(serviceName, function (err, addresses) {
The app performs a DNS lookup to obtain SRV records.
if (err) {
response.end("Could not look up DNS SRV records: " + err);
return;
}
var numResponses = 0;
if (addresses.length == 0) {
response.end("No peers discovered.");
} else {
addresses.forEach(function (item) { ...
Thanks Shubham ! I read the medium post and found using grpc to make connection to SRV should browse all IPs as a RR fashion. But I am still looking for get all IPs.
Medium article:
https://medium.com/google-cloud/loadbalancing-grpc-for-kubernetes-cluster-services-3ba9a8d8fc03
Gitrepo:
https://github.com/jtattermusch/grpc-loadbalancing-kubernetes-examples#example-1-round-robin-loadbalancing-with-grpcs-built-in-loadbalancing-policy
import (
"google.golang.org/grpc/balancer/roundrobin"
"google.golang.org/grpc/credentials"
)
conn, err := grpc.Dial("dns:///be-srv-lb.default.svc.cluster.local", grpc.WithTransportCredentials(ce), grpc.WithBalancerName(roundrobin.Name))
c := echo.NewEchoServerClient(conn)
It makes calls on a list of IPs one by one. RR
Creating channel with target greeter-server.default.svc.cluster.local:8000
Greeting: Hello you (Backend IP: 10.0.2.95)
Greeting: Hello you (Backend IP: 10.0.0.74)
Greeting: Hello you (Backend IP: 10.0.1.51)
I have found my main issue is related to this problem.
Why golang Lookup*** function can't provide a server parameter?
It looks like you are looking for headless service.
Look for an example implementation: https://medium.com/google-cloud/loadbalancing-grpc-for-kubernetes-cluster-services-3ba9a8d8fc03
Kubernetes documentation: https://kubernetes.io/docs/concepts/services-networking/service/#headless-services for your reference.
To explain this: Kubernetes has a few implementations for services basically Endpoint based service and Headless service. Endpoint based services could be either with or without selectors. LoadBalancers are generally provided by Cloud providers.
Headless service are designed for client-side load balancing implementation. It looks like you are trying to implement your own DNS driven client-side load balancer.
I'm basically trying to write a reverse proxy server so that when I curl localhost:8080/get it proxies the request to https://nghttp2.org/httpbin/get.
Note: the https://nghttp2.org/httpbin/get service listed above is http/2. But this behavior happens with http/1 as well, such as https://httpbin.org/get.
I'm using httputil.ReverseProxy for this and I'm rewriting the URL while customizing the Host header to not to leak the localhost:8080 to the actual backend.
However, the request still hits the backend with Host: localhost:8080 no matter how many times I set it on the header. Similarly, I used mitmproxy to snoop on the request and it looks like the net/http.Client sets the :authority pseudo-header to localhost:8080
Here's my source code:
package main
import (
"log"
"net/http"
"net/http/httputil"
)
func main() {
proxy := &httputil.ReverseProxy{
Transport: roundTripper(rt),
Director: func(req *http.Request) {
req.URL.Scheme = "https"
req.URL.Host = "nghttp2.org"
req.URL.Path = "/httpbin" + req.URL.Path
req.Header.Set("Host", "nghttp2.org") // <--- I set it here first
},
}
log.Fatal(http.ListenAndServe(":8080", proxy))
}
func rt(req *http.Request) (*http.Response, error) {
log.Printf("request received. url=%s", req.URL)
req.Header.Set("Host", "nghttp2.org") // <--- I set it here as well
defer log.Printf("request complete. url=%s", req.URL)
return http.DefaultTransport.RoundTrip(req)
}
// roundTripper makes func signature a http.RoundTripper
type roundTripper func(*http.Request) (*http.Response, error)
func (f roundTripper) RoundTrip(req *http.Request) (*http.Response, error) { return f(req) }
When I query curl localhost:8080/get the request gets proxied to https://nghttp2.org/httpbin/get. The echoed response shows that clearly my directives setting the Host header didn't do anything:
{
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip",
"Host": "localhost:8080",
"User-Agent": "curl/7.54.0"
},
"origin": "2601:602:9c02:16c2:fca3:aaab:3914:4a71",
"url": "https://localhost:8080/httpbin/get"
}
mitmproxy snooping also clearly shows that the request was made with :authority pseudo-header set to localhost:8080:
From http.Request docs:
// For server requests, Host specifies the host on which the URL
// is sought. Per RFC 7230, section 5.4, this is either the value
// of the "Host" header or the host name given in the URL itself.
// It may be of the form "host:port". For international domain
// names, Host may be in Punycode or Unicode form. Use
// golang.org/x/net/idna to convert it to either format if
// needed.
// To prevent DNS rebinding attacks, server Handlers should
// validate that the Host header has a value for which the
// Handler considers itself authoritative. The included
// ServeMux supports patterns registered to particular host
// names and thus protects its registered Handlers.
//
// For client requests, Host optionally overrides the Host
// header to send. If empty, the Request.Write method uses
// the value of URL.Host. Host may contain an international
// domain name.
Host string
So the value of URL.Host is only used in case request.Host is empty which is not the case. Setting request.Host should resolve the issue:
req.Host = "nghttp2.org"
Related issue discussed here.
I set up a series of gRPC requests and responses which all work fine, but I'm stuck when I try to get the client IP address and user-agent who is calling my gRPC APIs.
I read the Go gRPC documentation and other sources, but didn't find much valuable information. Few of them are talking about gRPC in Golang.
Should I set up a key-value to store the IP address in the context when setting up the gRPC APIs?
In Golang GRPC, you can use
func (UserServicesServer) Login(ctx context.Context, request *sso.LoginRequest) (*sso.LoginResponse, error) {
p, _ := peer.FromContext(ctx)
request.Frontendip = p.Addr.String()
.
.
}
But, do not forget import "google.golang.org/grpc/peer"
For grpc-gateway is used, the client IP address may be retrieved through x-forwarded-for like this:
// Get IP from GRPC context
func GetIP(ctx context.Context) string {
if headers, ok := metadata.FromIncomingContext(ctx); ok {
xForwardFor := headers.Get("x-forwarded-for")
if len(xForwardFor) > 0 && xForwardFor[0] != "" {
ips := strings.Split(xForwardFor[0], ",")
if len(ips) > 0 {
clientIp := ips[0]
return clientIp
}
}
}
return ""
}
In Golang GRPC, context has 3 values
authority
content-type
user-agent
md,ok:=metadata.FromIncomingContext(ctx)
fmt.Printf("%+v%+v",md,ok)