I have been playing around with golang and redis. I just stood up a simple http server and wanted to increment requests on redis. I am blowing up the connections (I think). I found that with redigo you can use connection pooling, but not sure how to implment that in go when I am serving the requests (where do you instantiate / call the pool from).
error: can't assign requested address.
Any suggestions would be appreciated....I am sure I am incorrectly making the connections, but just not sure how to change.
EDIT: Modified per pauljz's suggestions -- Works great now
var pool redis.Pool
func qryJson(rw http.ResponseWriter, req *http.Request){
incrementRedis()
}
func incrementRedis () {
t := time.Now().Format("2006-01-02 15:04:05")
conn := pool.Get()
defer conn.Close()
if _, err := conn.Do("HINCRBY", "messages", t, 1); err != nil {
log.Fatal(err)
}
}
func main() {
runtime.GOMAXPROCS(runtime.NumCPU())
pool = redis.Pool{
MaxIdle: 50,
MaxActive: 500, // max number of connections
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
panic(err.Error())
}
return c, err
},
}
http.HandleFunc("/testqry", qryJson)
log.Fatal(http.ListenAndServe(":8082", nil))
}
The redigo docs have a good starter example for connection pooling: http://godoc.org/github.com/garyburd/redigo/redis#Pool
In your case you would have a var pool redis.Pool in your package (i.e. not inside of a function).
In main(), before your ListenAndServe call, you would call pool = redis.Pool{ ... } from the redigo example to initialize the pool.
In incrementRedis() you would then do something like:
func incrementRedis () {
conn := pool.Get()
defer conn.Close()
if _, err := conn.Do("HINCRBY", "messages", t, 1); err != nil {
log.Fatal(err)
}
}
In your code, you create a connection to redis for each http request. Use a global variable to store connected redis connection and reuse it.
Related
I have a function in go which is handling connections which are coming through tcp and handled via ssh. I am trying to set an idle timeout by creating struct in the connection function.
Use case - a customer should be able to make a connection and upload/download multiple files
Reference - IdleTimeout in tcp server
Function code:
type Conn struct {
net.Conn
idleTimeout time.Duration
}
func HandleConn(conn net.Conn) {
var err error
rAddr := conn.RemoteAddr()
session := shortuuid.New()
config := LoadSSHServerConfig(session)
blocklistItem := blocklist.GetBlockListItem(rAddr)
if blocklistItem.IsBlocked() {
conn.Close()
atomic.AddInt64(&stats.Stats.BlockedConnections, 1)
return
}
func (c *Conn) Read(b []byte) (int, error) {
err := c.Conn.SetReadDeadline(time.Now().Add(c.idleTimeout))
if err != nil {
return 0, err
}
return c.Conn.Read(b)
}
sConn, chans, reqs, err := ssh.NewServerConn(conn, config)
if err != nil {
if err == io.EOF {
log.Errorw("SSH: Handshaking was terminated", log.Fields{
"address": rAddr,
"error": err,
"session": session})
} else {
log.Errorw("SSH: Error on handshaking", log.Fields{
"address": rAddr,
"error": err,
"session": session})
}
atomic.AddInt64(&stats.Stats.AuthorizationFailed, 1)
return
}
log.Infow("connection accepted", log.Fields{
"user": sConn.User(),
})
if user, ok := users[session]; ok {
log.Infow("SSH: Connection accepted", log.Fields{
"user": user.LogFields(),
"clientVersion": string(sConn.ClientVersion())})
atomic.AddInt64(&stats.Stats.AuthorizationSucceeded, 1)
// The incoming Request channel must be serviced.
go ssh.DiscardRequests(reqs)
// Key ID: sConn.Permissions.Extensions["key-id"]
handleServerConn(user, chans)
log.Infow("connection finished", log.Fields{"user": user.LogFields()})
log.Infow("checking connections", log.Fields{
//"cc": Stats.AcceptedConnections,
"cc2": &stats.Stats.AcceptedConnections})
// Remove connection from local cache
delete(users, session)
} else {
log.Infow("user not found from memory", log.Fields{"username": sConn.User()})
}
}
This code is coming from the Listen function:
func Listen() {
listener, err := net.Listen("tcp", sshListen)
if err != nil {
panic(err)
}
if useProxyProtocol {
listener = &proxyproto.Listener{
Listener: listener,
ProxyHeaderTimeout: time.Second * 10,
}
}
for {
// Once a ServerConfig has been configured, connections can be accepted.
conn, err := listener.Accept()
if err != nil {
log.Errorw("SSH: Error accepting incoming connection", log.Fields{"error": err})
atomic.AddInt64(&stats.Stats.FailedConnections, 1)
continue
}
// Before use, a handshake must be performed on the incoming net.Conn.
// It must be handled in a separate goroutine,
// otherwise one user could easily block entire loop.
// For example, user could be asked to trust server key fingerprint and hangs.
go HandleConn(conn)
}
}
Is that even possible to set a deadline for only the connections which have been idle for 20 secinds (no upload/downloads).
EDIT 1 : Following #LiamKelly's suggestions, I have made the changes in the code. Now the code is like
type SshProxyConn struct {
net.Conn
idleTimeout time.Duration
}
func (c *SshProxyConn) Read(b []byte) (int, error) {
err := c.Conn.SetReadDeadline(time.Now().Add(c.idleTimeout))
if err != nil {
return 0, err
}
return c.Conn.Read(b)
}
func HandleConn(conn net.Conn) {
//lines of code as above
sshproxyconn := &SshProxyConn{nil, time.Second * 20}
Conn, chans, reqs, err := ssh.NewServerConn(sshproxyconn, config)
//lines of code
}
But now the issue is that SSH is not happening. I am getting the error "Connection closed" when I try to do ssh. Is it still waiting for "conn" variable in the function call?
Is that even possible to set a deadline for only the connections which have been idle for 20 [seconds]
Ok so first a general disclaimer, I am going to assume go-protoproxy implements the Conn interface as we would expected. Also as you hinted at before, I don't think you can put a a struct method inside another function (I also recommend renaming it something unique to prevent Conn vs net.Conn confusion).
type SshProxyConn struct {
net.Conn
idleTimeout time.Duration
}
func (c *SshProxyConn) Read(b []byte) (int, error) {
err := c.Conn.SetReadDeadline(time.Now().Add(c.idleTimeout))
if err != nil {
return 0, err
}
return c.Conn.Read(b)
}
func HandleConn(conn net.Conn) {
This makes is more clear what your primary issue is, which you passed the normal net.Conn to your SSH server, not your wrapper class. So
sConn, chans, reqs, err := ssh.NewServerConn(conn, config)
should be EDIT
sshproxyconn := &SshProxyConn{conn, time.Second * 20}
Conn, chans, reqs, err := ssh.NewServerConn(sshproxyconn , config)
I have an application that makes about 400 reads per seconds and 100 writes per second to redis (hosted on redislabs). The application is using github.com/garyburd/redigo package as a redis proxy.
I have two functions which are the only ones being used to read and write:
func getCachedVPAIDConfig(key string) chan *cachedVPAIDConfig {
c := make(chan *cachedVPAIDConfig)
go func() {
p := pool.Get()
defer p.Close()
switch p.Err() {
case nil:
item, err := redis.Bytes(p.Do("GET", key))
if err != nil {
c <- &cachedVPAIDConfig{nil, err}
return
}
c <- &cachedVPAIDConfig{item, nil}
default:
c <- &cachedVPAIDConfig{nil, p.Err()}
return
}
}()
return c
}
func setCachedVPAIDConfig(key string, j []byte) chan error {
c := make(chan error)
go func() {
p := pool.Get()
defer p.Close()
switch p.Err() {
case nil:
_, err := p.Do("SET", key, j)
if err != nil {
c <- err
return
}
c <- nil
default:
c <- p.Err()
return
}
}()
return c
}
As you can see, I'm using the recommended connection pooling mechanism (http://godoc.org/github.com/garyburd/redigo/redis#Pool).
I'm calling these functions on every http request an endpoint on the application is getting. The problem is: once the application starts getting requests, it immediately starts throwing the error
dial tcp 54.160.xxx.xx:yyyy: connect: cannot assign requested address
(54.160.xxx.xx:yyyy is the redis host)
I see on redis that there are only about 600 connections when this starts to happen, which doesn't sound like a lot.
I tried playing with the MaxActive setting of the pool, setting it anywhere between 1000 and 50K, but the result is the same.
Any ideas?
EDIT
Here's my pool initialization code (doing this in func init):
pool = redis.Pool{
MaxActive: 1000, // note: I tried changing this to 50K, result the same
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", redisHost)
if err != nil {
return nil, err
}
if _, err := c.Do("AUTH", redisPassword); err != nil {
c.Close()
return nil, err
}
return c, err
},
}
Edit 2:
Issue solved by applying the stuff suggested in the answer below!
New code for pool init:
pool = redis.Pool{
MaxActive: 500,
MaxIdle: 500,
IdleTimeout: 5 * time.Second,
Dial: func() (redis.Conn, error) {
c, err := redis.DialTimeout("tcp", redisHost, 100*time.Millisecond, 100*time.Millisecond, 100*time.Millisecond)
if err != nil {
return nil, err
}
if _, err := c.Do("AUTH", redisPassword); err != nil {
c.Close()
return nil, err
}
return c, err
},
}
This new init makes it so that the get and set timeouts are handled by redigo internally, so I no longer need to return a channel on the getCachedVPAIDConfig and setCachedVPAIDConfig funcs. This is how they look now:
func setCachedVPAIDConfig(key string, j []byte) error {
p := pool.Get()
switch p.Err() {
case nil:
_, err := p.Do("SET", key, j)
p.Close()
return err
default:
p.Close()
return p.Err()
}
}
func getCachedVPAIDConfig(key string) ([]byte, error) {
p := pool.Get()
switch p.Err() {
case nil:
item, err := redis.Bytes(p.Do("GET", key))
p.Close()
return item, err
default:
p.Close()
return nil, p.Err()
}
}
You're closing the connection after sending on the channels, if the channel is blocking you're not closing connections, which would result in the error you're seeing. so don't just defer, close the connection explicitly.
I don't think it's the problem but a good idea regardless - set a timeout on your connections with DialTimeout.
Make sure you have a proper TestOnBorrow function to get rid of dead connections, especially if you have timeout. I usually do a PING if the connection has been idle for more than 3 seconds (the function receives the idle time as a parameter)
Try setting MaxIdle to a larger number as well, I remember having problems with pooling that were resolved by increasing that parameter in the pool.
I have a number of routes in my routes.go file and they all call my redis database. I'm wondering how I can avoid calling the dial and AUTH calls in every route.
I've tried setting variables outside the functions like this:
var (
c, err = redis.Dial("tcp", ADDRESS)
_, err = c.Do("AUTH", "testing")
)
but then the compiler doesn't like err being used twice.
First, only use var for declaring variables. You can't run code outside of functions, so there's no use in trying to create connections inside a var statement. Use init() if you need something run at startup.
The redis connections can't be used with concurrent requests. If you want to share a redis connection across multiple routes, you need to have a safe method for concurrent use. In the case of github.com/garyburd/redigo/redis you want to use a Pool. You can do the AUTH call inside the Dial function, returning a ready connection each time.
var redisPool *redis.Pool
func init() {
redisPool = &redis.Pool{
MaxIdle: 3,
IdleTimeout: 240 * time.Second,
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", server)
if err != nil {
return nil, err
}
if _, err := c.Do("AUTH", password); err != nil {
c.Close()
return nil, err
}
return c, err
},
}
}
Then each time you need a connection, you get one from the pool, and return it when you're done.
conn := redisPool.Get()
// conn.Close() just returns the connection to the pool
defer conn.Close()
if err := conn.Err(); err != nil {
// conn.Err() will have connection or Dial related errors
return nil, err
}
What I would do is instantiate a connection pool in main.go and pass the reference to the pool to your routes. This way you are setting up your redis client once, and your routes can leverage it.
I created a decorator around redigo that makes creating a Redis Client with a Connection pool very straightforward. Plus it is type-safe.
You can check it out here: https://github.com/shomali11/xredis
using redigo, I create a pool, something like so:
&redis.Pool{
MaxIdle: 80,
MaxActive: 12000, // max number of connections
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", host+":"+port)
if err != nil {
panic(err.Error())
}
return c, err
}
the problem I have is that for each time I get a new connection, I need to set the db, as I use different db's of redis since I host a number of sites on the VPS.
So, something like this:
conn := pool.Get()
defer conn.Close()
conn.Do("SELECT", dbNumber) //this is the call I want to avoid
Having to select the db each time I work with redis seems redundant and also poses a problem since I use it for sessions i.e. having code that is not mine working with my redis connection from my pool makes it "impossible" to set the correct db for it.
What I would like to do is to set the dbno for the pool so that whenever somebody asks for a new connection from the pool, it comes with the correct db already set i.e. not setting it explicitly each time.
How did you solve this in your applications?
Thanks.
You can use redis.DialOption: redis.DialDatabase, redis.DialPassword
conn, err := redis.Dial("tcp", "127.0.0.1:6379", redis.DialDatabase(1))
if err != nil {
panic(err)
}
defer conn.Close()
Select the database in your dial function:
&redis.Pool{
MaxIdle: 80,
MaxActive: 12000, // max number of connections
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", host+":"+port)
if err != nil {
return nil, err
}
_, err := c.Do("SELECT", dbNum)
if err != nil {
c.Close()
return nil, err
}
return c, nil
}
Also, return the error from dial instead of panicking.
If these libs don't support it, then you have two options:
submit a patch to automate this (the python lib does that, but be careful when keeping the state).
Wrap your redis pool with your own custom pool that automates this, something like (untested code, but you'll get the idea):
// a pool embedding the original pool and adding adbno state
type DbnoPool struct {
redis.Pool
dbno int
}
// "overriding" the Get method
func (p *DbnoPool)Get() Connection {
conn := p.Pool.Get()
conn.Do("SELECT", p.dbno)
return conn
}
pool := &DbnoPool {
redis.Pool{
MaxIdle: 80,
MaxActive: 12000, // max number of connections
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", host+":"+port)
if err != nil {
panic(err.Error())
}
return c, err
},
3, // the db number
}
//now you call it normally
conn := pool.Get()
defer conn.Close()
Best way is to use DialOptions like DialDatabase:
redisPool = &redis.Pool{
MaxIdle: AppConfig.DefaultInt("RedisMaxPool", 10),
IdleTimeout: 240 * time.Second,
Dial: func() (redis.Conn, error) {
c, err := redis.Dial(
"tcp",
AppConfig.DefaultString("RedisPath", ":6379"),
redis.DialDatabase(AppConfig.DefaultInt("RedisDB", 1)),
)
if err != nil {
return nil, err
}
return c, err
},
TestOnBorrow: func(c redis.Conn, t time.Time) error {
_, err := c.Do("PING")
return err
},
}
Connecting to Redigo and manipulating data inside a function is easy like butter, but the problem comes when you have to re-use its connection, obviously for performance/practicality reasons.
Doing it inside a function like this works:
func main() {
client, err := redis.Dial("tcp", ":6379")
if err != nil {
panic(err)
}
defer client.Close()
client.Do("GET", "test:1")
}
But bringing it outside doesn't:
var Client = redis.Dial("tcp", ":6379")
defer Client.Close()
func main() {
Client.Do("GET", "test:1")
}
With the following error(s) returned:
./main.go:1: multiple-value redis.Dial() in single-value context
./main.go:2: non-declaration statement outside function body
I've tried putting the connection as a const(ant), putting defer inside the main function to my dismay not working too.
This is an even bigger concern as I have many other functions that have to communicate to Redis, but recreating the connection to Redis everytime seems silly.
The Redigo API just shows how to create a Dial instance but doesn't go further by explaining how to re-use it.
You may've been lost in my talk, but I wanted to put a bit of context here, so my clear and concise question is: How do you go about re-using (not recreating everytime) a Redigo connection?
The best way turned out to be using Pools, which are briefly documented here: Redigo Pools.
A global variable won't eventually reuse a connection, so I ended up with something like this (using Pools as noted before):
func newPool() *redis.Pool {
return &redis.Pool{
MaxIdle: 80,
MaxActive: 12000, // max number of connections
Dial: func() (redis.Conn, error) {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
panic(err.Error())
}
return c, err
},
}
}
var pool = newPool()
func main() {
c := pool.Get()
defer c.Close()
test,_:=c.Do("HGETALL", "test:1")
fmt.Println(test)
}
If for example you want to reuse a pool inside another function you do it like this:
func test() {
c := pool.Get()
defer c.Close()
test2,_:=c.Do("HGETALL", "test:2")
fmt.Println(test2)
}
The redis.Dial() method returns client error. To fix it, you should replace:
var Client = redis.Dial("tcp", ":6379")
with:
var Client, _ = redis.Dial("tcp", ":6379")