I need to be able selecting from which IP address my HTTP request should be sent. In my code right now, for DNS caching i'm using the following according to this:
defaultTransport.DialContext = func(ctx context.Context, network string, addr string) (conn net.Conn, err error) {
separator := strings.LastIndex(addr, ":")
ips, err := r.LookupHost(ctx, addr[:separator])
if err != nil {
return nil, err
}
for _, ip := range ips {
conn, err = net.Dial(network, ip+addr[separator:])
if err == nil {
break
}
}
return
}
In order to specify the local address, the only way i found is by using the following:
defaultTransport.DialContext = (&net.Dialer{
LocalAddr: localAddr,
}).DialContext
How can i achieve combining the above?
You can create your own Dialer which will use your specific Local Address using net package of Golang like this:
dialer = &net.Dialer{
LocalAddr: &net.TCPAddr{
IP: net.ParseIP("127.0.0.1"),
Port: 0,
},
}
conn, err := dialer.Dial("tcp", "127.0.0.1:2525")
You can also specify your specific Port.But using Port 0 means it will use any dynamic port value to make connection to another Server.
You could wrap the local address dialer that you want to use, and delegate the actual calls to it?
localAddrDialier := &net.Dialer{
LocalAddr: localAddr,
}
// trimmed...
defaultTransport.DialContext = func(ctx context.Context, network string, addr string) (conn net.Conn, err error) {
separator := strings.LastIndex(addr, ":")
ips, err := r.LookupHost(ctx, addr[:separator])
if err != nil {
return nil, err
}
for _, ip := range ips {
// use the localAddrDialer here for the _actual_ request?
conn, err = localAddrDialer.DialContext(ctx, network, ip+addr[separator:])
if err == nil {
break
}
}
return
}
Or vice-versa, according to which if the net.Dialer's you want to make the actual call
Related
My PC has multiple IP addresses(ex: 10.1.1.20, 192.168.123.30, ...).
Can I extract the connected local ip address when connecting to remote server using http.Client?
If this is not possible with http.Client, is there any other possible way?
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://www.google.com", nil)
if err != nil {
panic(err)
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
// extract the local ip address???
// getsockname(?????)
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
fmt.Printf("StatusCode=%v\n", resp.StatusCode)
fmt.Printf("%v\n", string(data))
}
You can either:
loop through all network interfaces
or retrieve the preferred outbound ip address
But in both case, the fact that you are in the middle of using an http.Client and making a GET would not matter: you could get those IP addresses independently.
You can provide your own Transport implementation that extracts the outgoing local IP address right after establishing the TCP connection, e.g. like this:
client := &http.Client{
Transport: &http.Transport{
Dial: func(network, addr string) (net.Conn, error) {
conn, err := net.Dial(network, addr)
if err == nil {
localAddr := conn.LocalAddr().(*net.TCPAddr)
fmt.Println("LOCAL IP:", localAddr.IP)
}
return conn, err
},
},
}
How do I find the public IP address of the machine or server my program is running on?
Like when the program is executed it detect the public of the server and print for e.g.running at 123.45.67.89
The short answer is that there's no method guaranteed to return your "public" ip address.
The first question is, what is your public ip address? The address of your machine, as seen by the system to which you are connecting, may vary depending on how your local Internet service is configured and on the service to which you're connecting:
As I mentioned in a comment, in a typical home environment your machine doesn't have a public ip address. The public address is hosted by your router.
If you're accessing a service through a proxy or vpn, your machine's address may be entirely different from when you're directly connecting to a service.
On a system with multiple interfaces, the origin address selected may depend upon the address to which you are connecting: different addresses may have different routes.
You can try using a service like http://icanhazip.com/ to try to determine your public ip. This will be correct in many but not all situations.
public IP address is a vague notion, in practice, it might or might not be a static address. What do you know about that ? It is just a endpoint valid for a certain amount of time, which depends on many factors like which interface was used to issue the query.
We can use the mainline bittorrent dht to give us some indications.
The Go language provides the cool dht package written by anacrolix.
When querying nodes with a find_peers verb we receive a packet containing the remote ip address the peer has associated with our query. This is described in bep10.
If an UDP connection is not a good option, you might opt for a query to bittorent trackers as described in bep24
Consider that peers might be malicious, thus the more results, the better.
Below program outputs the list of external network addresses associated with the computer initiating the query from the POV of the cohort of nodes queried.
Addresses are scored by the numbers of response.
read also https://www.bittorrent.org/beps/bep_0005.html
found 9 bootstrap peers
found 6 peers
4 [2001:861:51c5:xxx:40d1:8061:1fe0:xxx]:9090
2 81.96.42.191:9090
4 peers told us that we were using [2001:861:51c5:xxx:40d1:8061:1fe0:xxx]:9090, we can infer this is ipv6.
2 of them told we were using 81.96.42.191:9090, the ipv4 interface.
package main
import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"log"
"net"
"os"
"sort"
"sync"
"time"
"github.com/anacrolix/dht"
"github.com/anacrolix/dht/krpc"
"github.com/anacrolix/torrent/bencode"
)
var maxTimeout = time.Second * 5
func main() {
b, _ := ioutil.ReadFile("db.json")
var rawAddrs []string
json.Unmarshal(b, &rawAddrs)
defer func() {
if len(rawAddrs) < 1 {
return
}
if len(rawAddrs) > 30 {
rawAddrs = rawAddrs[:30]
}
buf, err := json.Marshal(rawAddrs)
if err != nil {
panic(err)
}
err = ioutil.WriteFile("db.json", buf, os.ModePerm)
if err != nil {
panic(err)
}
fmt.Fprintf(os.Stderr, "%v peers recorded\n", len(rawAddrs))
}()
bootstrap, err := parseAddrs(rawAddrs)
if err != nil {
bootstrap, err = globalBootstrapAddrs()
if err != nil {
panic(err)
}
}
findPeers := []byte(`d1:ad2:id20:abcdefghij01234567899:info_hash20:mnopqrstuvwxyz123456e1:q9:get_peers1:t2:aa1:y1:qe`)
local, err := net.ResolveUDPAddr("udp", "0.0.0.0:9090")
if err != nil {
panic(err)
}
ln, err := net.ListenUDP("udp", local)
if err != nil {
panic(err)
}
addrscores := map[string]int{}
var drain drain
defer drain.Wait()
fmt.Fprintf(os.Stderr, "found %v bootstrap peers\n", len(bootstrap))
res, errs := readResponses(ln, len(bootstrap), sendQuery(ln, bootstrap, findPeers))
drain.Errors(errs)
peers := []net.Addr{}
for d := range res {
if isValidAddr(d.IP.UDP()) {
addrscores[d.IP.String()]++
d.R.ForAllNodes(func(arg1 krpc.NodeInfo) {
peers = append(peers, arg1.Addr.UDP())
})
}
}
if len(peers) > 0 {
fmt.Fprintf(os.Stderr, "found %v peers\n", len(peers))
res, errs = readResponses(ln, len(peers), sendQuery(ln, peers, findPeers))
drain.Errors(errs)
for d := range res {
if isValidAddr(d.IP.UDP()) {
addrscores[d.IP.String()]++
}
}
}
for _, peer := range peers {
if isValidAddr(peer) {
rawAddrs = append(rawAddrs, peer.String())
}
}
addrs := make([]string, 0, len(addrscores))
for addr := range addrscores {
addrs = append(addrs, addr)
}
sort.Slice(addrs, func(i int, j int) bool {
return addrscores[addrs[i]] > addrscores[addrs[j]]
})
for _, addr := range addrs {
fmt.Printf("%-4v %v\n", addrscores[addr], addr)
}
}
type drain struct{ sync.WaitGroup }
func (d *drain) Errors(errs <-chan error) {
d.Add(1)
go func() {
defer d.Done()
for err := range errs {
fmt.Fprintln(os.Stderr, err)
}
}()
}
func parseAddrs(rawAddrs []string) (addrs []net.Addr, err error) {
for _, s := range rawAddrs {
host, port, err := net.SplitHostPort(s)
if err != nil {
panic(err)
}
ua, err := net.ResolveUDPAddr("udp", net.JoinHostPort(host, port))
if err != nil {
log.Printf("error resolving %q: %v", host, err)
continue
}
addrs = append(addrs, ua)
}
if len(addrs) == 0 {
err = errors.New("nothing resolved")
}
return
}
func globalBootstrapAddrs() (addrs []net.Addr, err error) {
bootstrap, err := dht.GlobalBootstrapAddrs("udp")
if err != nil {
return nil, err
}
for _, b := range bootstrap {
addrs = append(addrs, b.Raw())
}
return
}
func isValidAddr(addr net.Addr) bool { // so weird guys.
return addr.String() != "<nil>" && addr.String() != ":0"
}
func sendQuery(ln *net.UDPConn, peers []net.Addr, query []byte) chan error {
errs := make(chan error)
for _, addr := range peers {
go func(addr net.Addr) {
_, err := ln.WriteTo(query, addr)
if err != nil {
errs <- addressedError{Op: "send", error: err, Addr: addr}
}
}(addr)
}
return errs
}
func readResponses(ln *net.UDPConn, count int, errs chan error) (<-chan krpc.Msg, <-chan error) {
data := make(chan krpc.Msg)
var wg sync.WaitGroup
for i := 0; i < count; i++ {
wg.Add(1)
go func() {
defer wg.Done()
buf := make([]byte, 1000)
ln.SetReadDeadline(time.Now().Add(maxTimeout))
n, remoteAddr, err := ln.ReadFromUDP(buf)
if err != nil {
errs <- addressedError{Op: "rcv", error: err, Addr: remoteAddr}
return
}
var m krpc.Msg
err = bencode.Unmarshal(buf[:n], &m)
if err != nil {
errs <- addressedError{Op: "rcv", error: err, Addr: remoteAddr}
return
}
data <- m
}()
}
go func() {
wg.Wait()
close(errs)
close(data)
}()
return data, errs
}
type addressedError struct {
error
Op string
Addr net.Addr
}
func (a addressedError) Error() string {
if !isValidAddr(a.Addr) {
return fmt.Sprintf("%-5v %v", a.Op, a.error.Error())
}
return fmt.Sprintf("%-5v %v: %v", a.Op, a.Addr.String(), a.error.Error())
}
I've accomplished implementing TCP reverse proxy in GoLang. But unfortunately couldn't come up with implementing caching to a TCP reverse proxy. Is it possible to do so, if yes, is there any resource out there? Is caching possible on a TCP (Transport Layer of Network)?
Here's the simple TCP reverse proxy in Golang.
package main
import (
"io"
"log"
"net"
)
//Proxy struct
type Proxy struct {
laddr, raddr *net.TCPAddr
lconn, rconn io.ReadWriteCloser
errorSignal chan bool
}
// New Create a new Proxy instance.
func New(lconn *net.TCPConn, laddr, raddr *net.TCPAddr) *Proxy {
return &Proxy{
lconn: lconn,
laddr: laddr,
raddr: raddr,
errorSignal: make(chan bool),
}
}
//TCPAddressResolver resolves an address and returns to a struct having ip and port.
func TCPAddressResolver(addr string) (tcpAddress *net.TCPAddr, err error) {
tcpAddress, err = net.ResolveTCPAddr("tcp", addr)
return
}
func main() {
listenerAddress, err := TCPAddressResolver(":8080")
if err != nil {
log.Fatalf("Failed to resolve local address: %v", err)
}
remoteAddress, err := TCPAddressResolver(":3000")
if err != nil {
log.Fatalf("Failed to resolve remote address: %v", err)
}
listener, err := net.ListenTCP("tcp", listenerAddress)
if err != nil {
log.Fatalf("Failed to open local port to listen: %v", err)
}
log.Printf("Simple Proxy started on: %d and forwards to port %d", listenerAddress.Port, remoteAddress.Port)
for {
conn, err := listener.AcceptTCP()
if err != nil {
log.Fatalf("Failed to accept connection: %v", err)
continue
}
var p *Proxy
// HTTP is a stateless protocol thus a proxy needs to reinitiate the new next incoming call (conn)
// each time it finishes handling the previous one.
p = New(conn, listenerAddress, remoteAddress)
p.Start()
}
}
//Start initiates transmission of data to and from the remote to client side.
func (p *Proxy) Start() {
defer p.lconn.Close()
var err error
p.rconn, err = net.DialTCP("tcp", nil, p.raddr)
if err != nil {
log.Fatalf("Remote connection failure: %v", err)
}
defer p.rconn.Close()
go p.CopySrcDst(p.lconn, p.rconn)
go p.CopySrcDst(p.rconn, p.lconn)
//Wait for everything to close -- This one blocks the routine.
<-p.errorSignal
log.Printf("Closing Start routine \n")
}
func (p *Proxy) err(err error) {
if err != io.EOF {
log.Printf("Warning: %v: Setting error signal to true", err)
}
p.errorSignal <- true
}
//CopySrcDst copies data from src to dest
func (p *Proxy) CopySrcDst(src, dst io.ReadWriteCloser) {
buff := make([]byte, 1024)
for {
n, err := src.Read(buff)
if err != nil {
// Reading error.
p.err(err)
return
}
dataFromBuffer := buff[:n]
n, err = dst.Write(dataFromBuffer)
if err != nil {
// Writing error.
p.err(err)
return
}
}
}
You are asking how to save data read from an io.Reader. That's different from caching.
The easiest approach is to tee the reader into a buffer.
While you are at it, you might as well use io.Copy instead of the similar code in the question. The code in the question does not handle the case when read returns n > 0 and a non-nil error.
Use an error group to coordinate waiting for the goroutines and collecting error status.
var g errgroup.Group
var rbuf, lbuf bytes.Buffer
g.Go(func() error {
_, err := io.Copy(lconn, io.TeeReader(p.rconn, &rbuf))
return err
})
g.Go(func() error {
_, err := io.Copy(rconn, io.TeeReader(p.lconn, &lbuf))
return err
})
if err := g.Wait(); err != nil {
// handle error
}
// rbuf and lbuf have the contents of the two streams.
The name of the programming language is "Go", not "Golang" or "GoLang".
I'm sure all the services are working properly.
I have the code below:
This snippet is used for registering two endpoints.
func RegisterEndpoints(ctx context.Context, c *utils.AppConfig, r resolver.Builder) (http.Handler, error) {
var err error
mux := runtime.NewServeMux()
dialOpts := []grpc.DialOption{grpc.WithBalancerName("round_robin"), grpc.WithInsecure()}
err = protos.RegisterUserCenterHandlerFromEndpoint(ctx, mux, r.Scheme()+"://author/user-center", dialOpts)
if err != nil {
return nil, err
}
err = protos.RegisterSsoHandlerFromEndpoint(ctx, mux, r.Scheme()+"://author/sso", dialOpts)
if err != nil {
return nil, err
}
return mux, nil
}
And in my main.go，I build a resolver to resolve name to address, then register the two endpoints and listen on port 8080.
func run() error {
c := utils.GetAppConfig()
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
r := localresolver.NewResolver(fmt.Sprintf("%s:%d", c.Registry.Host, c.Registry.Port))
resolver.Register(r)
mux := http.NewServeMux()
// Register endpoints here
gw, err := routes.RegisterEndpoints(ctx, c, r)
if err != nil {
return err
}
mux.Handle("/", gw)
fmt.Println("Listening localhost:8080...")
return http.ListenAndServe(fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), mux)
}
func main() {
defer glog.Flush()
if err := run(); err != nil {
glog.Fatal(err)
}
}
But after I ran go run main.go, I found that only the last service I registered can be accessed, that is sso service (the err = protos.RegisterSsoHandlerFromEndpoint(ctx, mux, r.Scheme()+"://author/sso", dialOpts) line).
Can anyone show me an example of the correct way to register multiple endpoints via grpc-gateway? (make all the services registered with grpc-gateway can successfully be visited)
[2020-01-31] Need more help, now my code is like below:
Other code are same as before.
Additional, this is the result which name resolver shows:
There is no need to pass the ServeMux (gw) to mux var as handler, you can just ListenAndServe to the returned gw variable.
// Register endpoints here
gw, err := routes.RegisterEndpoints(ctx, c, r)
if err != nil {
return err
}
fmt.Println("Listening localhost:8080...")
return http.ListenAndServe(fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), gw)
and in RegisterEndpoints function, the endpoint parameter should be your host:port, the api endpoint should be provided in the google api annotation in the proto file.
err = protos.RegisterUserCenterHandlerFromEndpoint(ctx, mux, fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), dialOpts)
if err != nil {
return nil, err
}
err = protos.RegisterSsoHandlerFromEndpoint(ctx, mux, fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), dialOpts)
if err != nil {
return nil, err
}
I appended grpc.WithBlock() to grpc.DialOption, then all services can be accessed via grpc-gateway now.
Like below:
dialOpts := []grpc.DialOption{grpc.WithBalancerName("round_robin"), grpc.WithInsecure(), grpc.WithBlock()}
I'm trying to connect to an IMAP server through a socks5. To do so I've exported some functions from the Imap package but I got stuck at the socks5 dialer setup(1st step:) .
I think the reason is that I'm passing nil as forward (of type Dial) argument . What is the forward argument supposed to be ? It is not documented (in godoc)
func dialSocks(socks string) (Dial proxy.Dialer, err error) {
Dial, err = proxy.SOCKS5("tcp", socks, nil, nil)
return
}
func dialTLS(addr string, config *tls.Config) (c *imap.Client, err error) {
addr = defaultPort(addr, "993")
d, err := dialSocks("101.120.113.185:1328")
if err != nil {
log.Error(err)
return
}
conn, err := d.Dial("tcp", addr)
if err == nil {
host, _, _ := net.SplitHostPort(addr)
tlsConn := tls.Client(conn, setServerName(config, host))
if c, err = imap.NewClient(tlsConn, host, 60*time.Second); err != nil {
conn.Close()
}
}
return
}
func defaultPort(addr, port string) string {
_, _, err := net.SplitHostPort(addr)
if err != nil {
addr = net.JoinHostPort(addr, port)
}
return addr
}
forward is the Dialer used to connect to the proxy.
You can use proxy.Direct here if you want (which is the default within the package, if you look at the source), but all that is doing is delegating to net.Dial(network, addr). If you want more options, insert your own Dialer.