Golang server "write tcp ... use of closed network connection" - heroku

I am beginner at Go, I had wrote small server to testing and deploy it on heroku platform. I have /logout request, which almost works, but sometimes I see something like this:
PANIC: write tcp 172.17.110.94:36641->10.11.189.195:9951: use of closed network connection
I don't know why it happens, and why sometimes it works perfectly.
My steps:
I send 1st POST request to /token-auth with body then generate token and send as response.
At 2nd I do /logout GET request with that token, and set token to Redis store
Here is full code of my redil_cli.go
package store
import (
"github.com/garyburd/redigo/redis"
)
type RedisCli struct {
conn redis.Conn
}
var instanceRedisCli *RedisCli = nil
func Connect() (conn *RedisCli) {
if instanceRedisCli == nil {
instanceRedisCli = new(RedisCli)
var err error
//this is works!!!
instanceRedisCli.conn, err = redis.Dial("tcp", "lab.redistogo.com:9951")
if err != nil {
panic(err)
}
if _, err := instanceRedisCli.conn.Do("AUTH", "password"); err != nil {
//instanceRedisCli.conn.Close()
panic(err)
}
}
return instanceRedisCli
}
func (redisCli *RedisCli) SetValue(key, value string, expiration ...interface{}) error {
_, err := redisCli.conn.Do("SET", key, value)
if err == nil && expiration != nil {
redisCli.conn.Do("EXPIRE", key, expiration[0])
}
return err
}
func (redisCli *RedisCli) GetValue(key string) (interface{}, error) {
data, err := redisCli.conn.Do("GET", key)
if err != nil{
panic(err)
}
return data, err
}
After that my function that checks Authorization header will panic while trying to do GetValue(key string) method
func (redisCli *RedisCli) GetValue(key string) (interface{}, error) {
data, err := redisCli.conn.Do("GET", key)
if err != nil{
panic(err)
}
return data, err
}
Can anyone point me, what I doing wrong?

Related

Send Push Notification - It shows successful response but not showing any notifcation

I was trying to send push notification through fcm. In the response, I get all successful response. But I didn't get any notification. In other way, I tried to send notification directly from firebase console, and I get the notfication with the same token. So, I don't know what is the bug in this code. Token is okay. Response is also OK. but after executing code, I don't get any notification.
package main
import (
"context"
"encoding/base64"
"log"
"server/config"
db "server/database"
firebase "firebase.google.com/go/v4"
"firebase.google.com/go/v4/messaging"
"github.com/gin-gonic/gin"
"google.golang.org/api/option"
)
func init() {
config.Init()
db.ConnectRedis()
}
var ctx = context.Background()
func getDecodedFireBaseKey() ([]byte, error) {
fireBaseAuthKey := config.Env.FirebaseAuthKey
decodedKey, err := base64.StdEncoding.DecodeString(fireBaseAuthKey)
if err != nil {
return nil, err
}
return decodedKey, nil
}
func SendPushNotification(deviceTokens []string) error {
decodedKey, err := getDecodedFireBaseKey()
if err != nil {
return err
}
opts := []option.ClientOption{option.WithCredentialsJSON(decodedKey)}
app, err := firebase.NewApp(ctx, nil, opts...)
if err != nil {
log.Println("Error in initializing firebase", err)
return err
}
fcmClient, err := app.Messaging(ctx)
if err != nil {
log.Fatalf("error getting Messaging client: %v\n", err)
}
response, err := fcmClient.SendMulticast(ctx, &messaging.MulticastMessage{
Notification: &messaging.Notification{
Title: "Congratulations!!",
Body: "You have just implement push notification",
},
Tokens: deviceTokens,
})
if err != nil {
return err
}
log.Println("Response success count : ", response.SuccessCount)
log.Println("Response failure count : ", response.FailureCount)
return nil
}
func main() {
r := gin.Default()
deviceTokens, _ := db.RedisClient.SMembers(ctx,"fcmToken").Result()
SendPushNotification(deviceTokens)
r.Run()
}

emersion/go-imap - imap.FetchRFC822: invalid memory address or nil pointer dereference

I am trying to fetch all E-Mails from the Server with the following source Code (this function is called in the main module):
package internal
import (
"fmt"
"io"
"io/ioutil"
"log"
"github.com/emersion/go-imap"
"github.com/emersion/go-imap/client"
"github.com/emersion/go-message"
)
func FetchEMail(server string, username string, password string) error {
//Connect to Server
log.Println("Connecting to server...")
c, err := client.DialTLS(server, nil)
log.Println("Connected to " + server)
defer c.Logout()
//check if connection successful
if err != nil {
log.Println("In connection Error")
return err
}
//err = nil
//Login
log.Println("Logging in...")
err = c.Login(username, password)
log.Println("Logged in as " + username)
//check if login successful
if err != nil {
log.Println("In login Error")
return err
}
//Select INBOX
log.Println("Selecting INBOX...")
mbox, err := c.Select("INBOX", false)
log.Println("Selected INBOX")
//check if select successful
if err != nil {
return err
}
//Fetch all messages
log.Println("Fetching all messages...")
seqset := new(imap.SeqSet)
seqset.AddRange(1, mbox.Messages)
items := []imap.FetchItem{imap.FetchRFC822}
messages := make(chan *imap.Message, 10)
done := make(chan error, 1)
go func() {
done <- c.Fetch(seqset, items, messages)
}()
//check if fetch successful
if err := <-done; err != nil {
log.Println("In fetch Error")
return err
}
log.Println("Run Successful - Terminating...")
return nil
}
This results into the following error:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x5ee505]
goroutine 1 [running]:
I already have tried imap.FetchEvelope() which works, but for some reason imap.FetchRFC822 does not work.
My main goal is to export all E-Mail attachments (.gz, .zip, ...) from all E-Mails, that is why I need the whole E-Mail, not only the Envelope.
I think the issue was in this line items := []imap.FetchItem{imap.FetchRFC822}.
First, let's clarify what the FetchItem type is. This represents the different parts of an email that can be fetched (envelope, body, UID, flags, and so on).
Then, let's talk about the Fetch method. It expects a slice of imap.FetchItem passed in. It retrieves from the emails all of the parts specified by the slice.
So what fixes your issue is replacing this line with items := []imap.FetchItem{imap.FetchRFC822, imap.FetchEnvelope}.
I fixed and tested your program as you can see from the code snippet below:
package main
import (
"fmt"
"log"
"github.com/emersion/go-imap"
"github.com/emersion/go-imap/client"
)
func FetchEMail(server string, username string, password string) error {
// Connect to Server
log.Println("Connecting to server...")
c, err := client.Dial(server)
log.Println("Connected to " + server)
defer c.Logout()
// check if connection successful
if err != nil {
log.Println("In connection Error")
return err
}
// Login
log.Println("Logging in...")
err = c.Login(username, password)
log.Println("Logged in as " + username)
// check if login successful
if err != nil {
log.Println("In login Error")
return err
}
// Select INBOX
log.Println("Selecting INBOX...")
mbox, err := c.Select("INBOX", false)
log.Println("Selected INBOX")
// check if select successful
if err != nil {
return err
}
// Fetch all messages
log.Println("Fetching all messages...")
seqset := new(imap.SeqSet)
seqset.AddRange(1, mbox.Messages)
items := []imap.FetchItem{imap.FetchRFC822, imap.FetchEnvelope}
messages := make(chan *imap.Message, 10)
done := make(chan error, 1)
go func() {
done <- c.Fetch(seqset, items, messages)
}()
for msg := range messages {
fmt.Printf("suject: %v\n", msg.Envelope.Subject)
}
// check if fetch successful
if err := <-done; err != nil {
log.Println("In fetch Error")
return err
}
log.Println("Run Successful - Terminating...")
return nil
}
func main() {
err := FetchEMail("xxxxxxx", "xxxxx", "xxxxx")
if err != nil {
panic(err)
}
}
Near the end, I added a for to print the subject of the retrieved emails. Here, you can replace the code with your own logic. The nil pointer dereference error disappears.
Let me know if this solves your issue!

Pgxpool returns "pool closed" error on Scan

I'm trying to implement pgxpool in a new go app. I keep getting a "pool closed" error after attempting a scan into a struct.
The pgx logger into gives me this after connecting. I thought the pgxpool was meant to remain open.
{"level":"info","msg":"closed connection","pid":5499,"time":"2022-02-24T16:36:33+10:30"}
Here is my router code
func router() http.Handler {
var err error
config, err := pgxpool.ParseConfig(os.Getenv("DATABASE_URL"))
if err != nil {
log.Fatalln(err)
}
log.Println(os.Getenv("DATABASE_URL"))
logrusLogger := &logrus.Logger{
Out: os.Stderr,
Formatter: new(logrus.JSONFormatter),
Hooks: make(logrus.LevelHooks),
Level: logrus.InfoLevel,
ExitFunc: os.Exit,
ReportCaller: false,
}
config.ConnConfig.Logger = NewLogger(logrusLogger)
db, err := pgxpool.ConnectConfig(context.Background(), config)
if err != nil {
log.Fatalln(err)
}
defer db.Close()
--- minio connection
rs := newAppResource(db, mc)
Then, in a helper file I setup the resource
type appResource struct {
db *pgxpool.Pool
mc *minio.Client
}
// newAppResource function to pass global var
func newAppResource(db *pgxpool.Pool, mc *minio.Client) *appResource {
return &appResource{
db: db,
mc: mc,
}
}
There "pool closed" error occurs at the end of this code
func (rs *appResource) login(w http.ResponseWriter, r *http.Request) {
var user User
var login Login
d := json.NewDecoder(r.Body)
d.DisallowUnknownFields() // catch unwanted fields
err := d.Decode(&login)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if err != nil {
fmt.Println("can't decode JSON", err)
}
if login.Email == "" {
log.Println("empty email")
return
}
log.Println(login.Email)
log.Println(login.Password)
if login.Password == "" {
log.Println("empty password")
return
}
// optional extra check
if d.More() {
http.Error(w, "extraneous data after JSON object", http.StatusBadRequest)
return
}
sqlStatement := "SELECT user_id, password FROM users WHERE active = 'true' AND email = ?"
row := rs.db.QueryRow(context.Background(), sqlStatement, login.Email)
err = row.Scan(&user.UserId, &user.Password)
if err == sql.ErrNoRows {
log.Println("user not found")
http.Error(w, err.Error(), http.StatusUnauthorized)
return
}
if err != nil {
log.Println(err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
It appears that you are doing something like the following:
func router() http.Handler {
db, err := pgxpool.ConnectConfig(context.Background(), config)
if err != nil {
log.Fatalln(err)
}
defer db.Close()
return appResource{db: db}
}
The issue with this is that the defer db.Close() runs when the function router() ends and this is before the returned pgxPool.Pool is actually used (the http.Handler returned will be used later when http requests are processed). Attempting to use a closed pgxPool.Pool results in the error you are seeing.
The simplest solution is to simply remove the defer db.Close() however you might also consider calling db.Close() as part of a clean shutdown process (it needs to remain open as long as you are handling requests).
You are using pgxpool which does differ from the standard library; however I believe that the advice given in the standard library docs applies here:
It is rarely necessary to close a DB.

How to turn DataBase access into a Function idiomatically in Go

I have built a Backend API in Go, it works however I want refactor the code for the DB access layer into a function - idiomatically.
// Get the form data entered by client; FirstName, LastName, phone Number,
// assign the person a unique i.d
// check to see if that user isn't in the database already
// if they are send an error message with the a 'bad' response code
// if they aren't in db add to db and send a message with success
func CreateStudentAccountEndpoint(response http.ResponseWriter, request *http.Request){
client, err := mongo.NewClient("mongodb://localhost:27017")
if err != nil {
log.Fatalf("Error connecting to mongoDB client Host: Err-> %v\n ", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
err = client.Connect(ctx)
if err != nil {
log.Fatalf("Error Connecting to MongoDB at context.WtihTimeout: Err-> %v\n ", err)
}
response.Header().Set("Content-Type", "application/json")
studentCollection := client.Database(dbName).Collection("students")
_, err = studentCollection.InsertOne(context.Background(),data)
if err != nil {
response.WriteHeader(501)
response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
}
// encoding json object for returning to the client
jsonStudent, err := json.Marshal(student)
if err != nil {
http.Error(response, err.Error(), http.StatusInternalServerError)
}
response.Write(jsonStudent)
}
I understand that I can create a method which returns (*mongoClient, err) as I utilise the client local variable later on in the code.
However I am lost as to how to implement the defer cancel() part because it executes once the method CreateStudenAccountEndpoint is at the end. But I am at a loss on how to implement this defer section in a method that will recognise that I want the defer to happen at the end of the function that calls the DB access layer method e.g CreateStudentAccountEndpoint not the actual db access method itself.
As I understand it, the connection should be long-lived and set up as a part of a constructor, i.e. not part of the request flow.
This will typically look something like this:
type BackendAPI struct {
client *mongo.Client
}
func NewBackendAPI(mongoURI string) (*BackendAPI, error) {
client, err := mongo.NewClient(mongoURI)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
err = client.Connect(ctx)
if err != nil {
return nil, err
}
return &BackendAPI{client}, nil
}
func (api *BackendAPI) func CreateStudentAccountEndpoint(response http.ResponseWriter, request *http.Request) {
response.Header().Set("Content-Type", "application/json")
// note the use of the long-lived api.client, which is connected already.
studentCollection := api.client.Database(dbName).Collection("students")
_, err = studentCollection.InsertOne(context.Background() ,data)
if err != nil {
response.WriteHeader(501)
response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
return // at this point, the method should return
}
// encoding json object for returning to the client
jsonStudent, err := json.Marshal(student)
if err != nil {
http.Error(response, err.Error(), http.StatusInternalServerError)
}
response.Write(jsonStudent)
}
If you worry about losing the connection, you could implement a call to api.client.Ping in there, but in my opinion this should only be attempted if you encounter a failure you believe you can recover from by reconnecting.

Requesting multiple URLs in Go

I have the following Go program: https://play.golang.org/p/-TUtJ7DIhi
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strconv"
)
func main() {
body, err := get("https://hacker-news.firebaseio.com/v0/topstories.json")
if err != nil {
panic(err)
}
var ids [500]int
if err = json.Unmarshal(body, &ids); err != nil {
panic(err)
}
var contents []byte
for _, value := range ids[0:10] {
body, err := get("https://hacker-news.firebaseio.com/v0/item/" + strconv.Itoa(value) + ".json")
if err != nil {
fmt.Println(err)
} else {
contents = append(contents, body...)
}
}
fmt.Println(contents)
}
func get(url string) ([]byte, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
body, err := ioutil.ReadAll(res.Body)
res.Body.Close()
return body, err
}
When run it throws EOF json errors on the iterative get requests, but when I hit the URLs individually they do not appear to be malformed.
What am I missing?
It looks like there's something wrong with their server, and it's closing connections without sending a Connection: close header. The client therefore tries to reuse the connection per the HTTP/1.1 specification.
You can work around this by creating your own request, and setting Close = true, or using a custom Transport with DisableKeepAlives = true
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Close = true
res, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}

Resources