First, newbie to Go. Next, I am trying to run the following code below and experiencing this error
2021/06/16 18:24:15 Get "https://www.cnn.com": dial tcp: lookup www.cnn.com on 192.168.100.200:53: dial udp: address 192.168.100.65: mismatched local address type
exit status 1
My end goal is to use a predefined DNS server (alternate from the OS) and create a HTTP/S connection using a specific local IP address. I also suspect this could be accomplished with less code, so would love to understand this more from someone more familiar with Go.
package main
import (
"context"
"io/ioutil"
"log"
"net"
"net/http"
"time"
)
func main() {
q := net.ParseIP("192.168.100.65")
addr := &net.IPAddr{q,""}
var (
dnsResolverIP = "8.8.8.8:53"
dnsResolverProto = "udp"
dnsResolverTimeoutMs = 5000
)
dialer := &net.Dialer{
Resolver: &net.Resolver {
PreferGo: true,
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
d := net.Dialer {
LocalAddr: addr,
Timeout: time.Duration(dnsResolverTimeoutMs) * time.Millisecond,
}
return d.DialContext(ctx, dnsResolverProto, dnsResolverIP)
},
},
}
dialContext := func(ctx context.Context, network, addr string) (net.Conn, error) {
return dialer.DialContext(ctx, network, addr)
}
http.DefaultTransport.(*http.Transport).DialContext = dialContext
httpClient := &http.Client{}
// Testing the new HTTP client with the custom DNS resolver.
resp, err := httpClient.Get("https://www.cnn.com")
if err != nil {
log.Fatalln(err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
log.Println(string(body))
}
Related
I am designing an http service and need to get the real destination IP. The network on the host looks like this（just like k8s kubeproxy doing）：
traffic arrived host -> host ipvs -> docker bridge -> container
My service(golang) runs in container, but there is an IPVS load balancer that do dnat in front, so I need to get original dst in some way, I know I can use syscall.GetsockoptIPv6Mreq to get real dst, but it just work when service run in host,if run in container,the code failed with err：syscall.GetsockoptIPv6Mreq: no such file or directory
container use docker, run in bridge mod, this is my service code：
package main
import (
"context"
"fmt"
"net"
"net/http"
"syscall"
)
const SO_ORIGINAL_DST = 80
type contextKey struct {
key string
}
var ConnContextKey = &contextKey{"http-conn"}
func SaveConnInContext(ctx context.Context, c net.Conn) context.Context {
return context.WithValue(ctx, ConnContextKey, c)
}
func GetConn(r *http.Request) net.Conn {
return r.Context().Value(ConnContextKey).(net.Conn)
}
func getOriginalDst(conn net.Conn) (string, int, error) {
tc, ok := conn.(*net.TCPConn)
if !ok {
return "", 0, fmt.Errorf("redirect proxy only support tcp")
}
f, err := tc.File()
if err != nil {
return "", 0, fmt.Errorf("get conn file error, err: %s", err)
}
defer f.Close()
addr, err := syscall.GetsockoptIPv6Mreq(int(f.Fd()), syscall.IPPROTO_IP, SO_ORIGINAL_DST)
if err != nil {
return "", 0, fmt.Errorf("syscall.GetsockoptIPv6Mreq: %s", err)
}
p0 := int(addr.Multiaddr[2])
p1 := int(addr.Multiaddr[3])
port := p0*256 + p1
ips := addr.Multiaddr[4:8]
ip := fmt.Sprintf("%d.%d.%d.%d", ips[0], ips[1], ips[2], ips[3])
return ip, port, nil
}
func hello(w http.ResponseWriter, req *http.Request) {
conn := GetConn(req)
ip, port, err := getOriginalDst(conn)
if err != nil {
fmt.Fprintf(w, fmt.Sprintf("get original dst failed: %s", err))
return
}
msg := fmt.Sprintf("source addr: %s, server addr: %s, original dst: %s:%d \n",
req.RemoteAddr, req.Context().Value(http.LocalAddrContextKey), ip, port)
fmt.Fprintf(w, msg)
}
func version(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "v4")
}
func main() {
http.HandleFunc("/", version)
http.HandleFunc("/test", hello)
addr := "0.0.0.0:9090"
srv := http.Server{
ConnContext: SaveConnInContext,
}
//server.ListenAndServe()
ln, err := net.Listen("tcp4", addr)
if err != nil {
panic(err)
}
srv.Serve(ln)
}
if service run on the host it works ok：
sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 8080 -j REDIRECT --to-ports 9090
curl localhost:8080/test
get expected answer
source addr: 127.0.0.1:21918, server addr: 127.0.0.1:9090, original dst: 127.0.0.1:9090
If the service run in container then the request will fail
➜ ~ curl 172.17.0.2:9090/test
get original dst failed: syscall.GetsockoptIPv6Mreq: no such file or directory
I have tried many ways. At first I thought it was a permission problem. Adding --privileged=true -u=root is still useless.
I expect the code can work and get original dst in container.
My PC has multiple IP addresses(ex: 10.1.1.20, 192.168.123.30, ...).
Can I extract the connected local ip address when connecting to remote server using http.Client?
If this is not possible with http.Client, is there any other possible way?
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://www.google.com", nil)
if err != nil {
panic(err)
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
// extract the local ip address???
// getsockname(?????)
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
fmt.Printf("StatusCode=%v\n", resp.StatusCode)
fmt.Printf("%v\n", string(data))
}
You can either:
loop through all network interfaces
or retrieve the preferred outbound ip address
But in both case, the fact that you are in the middle of using an http.Client and making a GET would not matter: you could get those IP addresses independently.
You can provide your own Transport implementation that extracts the outgoing local IP address right after establishing the TCP connection, e.g. like this:
client := &http.Client{
Transport: &http.Transport{
Dial: func(network, addr string) (net.Conn, error) {
conn, err := net.Dial(network, addr)
if err == nil {
localAddr := conn.LocalAddr().(*net.TCPAddr)
fmt.Println("LOCAL IP:", localAddr.IP)
}
return conn, err
},
},
}
What is the best way to add a timeout when using tls.Dial in Go?
I see the net package has net.DialTimeout, but unfortunately, the tls package doesn't have a corresponding function.
I presume I should be using a context or Dialer to implement a timeout, but I'm not an expert in Go and I can't find any good examples.
(1) I found tls.DialWithDialer, but I'm not sure how to create a net.Dialer that is configured with a timeout.
func DialWithDialer(dialer *net.Dialer, network, addr string, config *Config) (*Conn, error)
(2) I also found tls.DialContext, but I'm not sure how to use that to implement a timeout.
func (d *Dialer) DialContext(ctx context.Context, network, addr string) (net.Conn, error)
(3) I presume it might be possible to establish an initial connection using net.DialTimeout and then upgrade the connection and continue with the TLS handshake, but I can't find any examples that show how to do that.
Any help or guidance would be appreciated.
Here is my simple program that connects to a list of servers and prints some info about the certificate chain. When a server is not responding, this program hangs for a long time. All I want to do is time out after 10 seconds.
package main
import (
"bufio"
"crypto/tls"
"fmt"
"os"
)
func main() {
port := "443"
conf := &tls.Config{
InsecureSkipVerify: true,
}
s := bufio.NewScanner(os.Stdin)
for s.Scan() {
host := s.Text()
conn, err := tls.Dial("tcp", host+":"+port, conf)
if err != nil {
fmt.Println("Host:", host, "Dial:", err)
continue
}
defer conn.Close()
certs := conn.ConnectionState().PeerCertificates
for _, cert := range certs {
fmt.Println("Host:", host, "Issuer:", cert.Issuer)
}
}
}
As you mention in your question there are a few options; using DialContext is a common technique:
package main
import (
"bufio"
"context"
"crypto/tls"
"fmt"
"os"
"time"
)
func main() {
port := "443"
conf := &tls.Config{
InsecureSkipVerify: true,
}
s := bufio.NewScanner(os.Stdin)
for s.Scan() {
host := s.Text()
ctx, cancel := context.WithTimeout(context.Background(), 10 * time.Second)
d := tls.Dialer{
Config: conf,
}
conn, err := d.DialContext(ctx,"tcp", host+":"+port)
cancel() // Ensure cancel is always called
if err != nil {
fmt.Println("Host:", host, "Dial:", err)
continue
}
// warning: using defer in a loop may not have the expected result
// the connection will remain open until the function exists
defer conn.Close()
tlsConn := conn.(*tls.Conn)
certs := tlsConn.ConnectionState().PeerCertificates
for _, cert := range certs {
fmt.Println("Host:", host, "Issuer:", cert.Issuer)
}
}
}
Using the above approach makes it relatively simple to allow users of your code to cancel the request (accept a context and use it where the above has context.Background()). If this is not important to you then using a Dialer with Timeout is simpler:
conn, err := tls.DialWithDialer(&net.Dialer{Timeout: 10 * time.Second}, "tcp", host+":"+port, conf)
if err != nil {
fmt.Println("Host:", host, "Dial:", err)
continue
}
certs := conn.ConnectionState().PeerCertificates
for _, cert := range certs {
fmt.Println("Host:", host, "Issuer:", cert.Issuer)
}
conn.Close()
We are trying to simulate a GRPC service for testing purposes but we are getting problems with creating the server. Our client it's working with our GRPC provider and it is similar to the next code.
We are having the problem with the configuration of the server that receive requests with a nil
as credentials.
package main
import (
"context"
"flag"
"fmt"
"log"
"time"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc"
ecpb "google.golang.org/grpc/examples/features/proto/echo"
)
var addr = flag.String("addr", "localhost:50051", "the address to connect to")
func callUnaryEcho(client ecpb.EchoClient, message string) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
resp, err := client.UnaryEcho(ctx, &ecpb.EchoRequest{Message: message})
if err != nil {
log.Fatalf("client.UnaryEcho(_) = _, %v: ", err)
}
fmt.Println("UnaryEcho: ", resp.Message)
}
func main() {
flag.Parse()
ctx := context.TODO()
conn, err := grpc.DialContext(
ctx,
*addr,
grpc.WithTransportCredentials(credentials.NewTLS(nil)),
)
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
rgc := ecpb.NewEchoClient(conn)
callUnaryEcho(rgc, "hello world")
}
I want to achieve resolving a given domain by the DNS Server I provide.
The code below I've grabbed from https://play.golang.org/p/s2KtkFrQs7R
The result is giving the correct results - but unfortunately even when I'm giving an IP like 123.123.123.123:53 as the resolver IP...
I guess there's a fallback - but I could not find (https://golang.org/pkg/net/) the switch to turn it off...
Thanks in advance for any hint...
Matthias
package main
import (
"context"
"fmt"
"log"
"net"
"time"
)
func GoogleDNSDialer(ctx context.Context, network, address string) (net.Conn, error) {
d := net.Dialer{}
return d.DialContext(ctx, "udp", "123.123.123.123:53")
}
func main() {
domain := "www.google.com"
const timeout = 1000 * time.Millisecond
ctx, cancel := context.WithTimeout(context.TODO(), timeout)
defer cancel()
// var r net.Resolver
r := net.Resolver{
PreferGo: true,
Dial: GoogleDNSDialer,
}
records, err := r.LookupHost(ctx, domain)
if err != nil {
log.Fatal(err)
} else {
fmt.Printf("Records %v \n", records[0])
}
}