Passing Data from Datastore to http.Writer - go

I was recently introduced to the wonders of the language known as go. I set myself a task of writing a RESTful API using GoLang and Google's Datastore. I am able to retrieve data from Datastore and Print it to the console using fmt.Println however the issue comes into play when I try to use the data from Datastore and pass it to the http.Handler.
I was wondering if someone could inform me of where I am going wrong or even point me in the right direction.
Here is what I have done so far
package main
import (
"log"
"fmt"
"context"
"net/http"
// "encoding/json"
"cloud.google.com/go/datastore"
)
type Item struct {
Id string `datastore:"id"`
Name string `datastore:"title"`
View int `datastore:"views"`
Brand string `datastore:"brand"`
id int64 // interger from "Name/ID" fild in datastore entities list
}
func main() {
http.HandleFunc("/", ListTasks)
http.ListenAndServe(":8080", nil)
}
//func ListTasks(w http.ResponseWriter, r *http.Request) ([]*Item, error) {
func ListTasks(w http.ResponseWriter, r *http.Request) {
ctx := context.Background()
client, err := datastore.NewClient(ctx, "my-client")
if err != nil {
log.Fatalln(err)
}
var tasks []*Item
query := datastore.NewQuery("my-query")
keys, err := client.GetAll(ctx, query, &tasks)
if err != nil {
return nil, err
}
for i, key := range keys {
tasks[i].id = key.ID
}
return tasks, nil
}
I've also looked into http Wrappers, but I'm unaware if using a wrapper is 100% necessary or if I'm just adding more to my plate.

I've removed the return tasks, nil as it appeared to be unnecessary, modified the return, nil err to log.Fatalln(nil, err) and also encoded tasks as instructed by #ThunderCat and #tkausl. My issue has been resolved, thank you.
Here is my working code
package main
import (
"log"
"context"
"net/http"
"encoding/json"
"cloud.google.com/go/datastore"
)
type Item struct {
Id string `datastore:"id"`
Name string `datastore:"title"`
View int `datastore:"views"`
Brand string `datastore:"brand"`
id int64 // interger from "Name/ID" fild in datastore entities list
}
func main() {
http.HandleFunc("/", ListTasks)
http.ListenAndServe(":8080", nil)
}
func ListTasks(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
ctx := context.Background()
client, err := datastore.NewClient(ctx, "my-client")
if err != nil {
log.Fatalln(err)
}
var tasks []*Item
query := datastore.NewQuery("my-query")
keys, err := client.GetAll(ctx, query, &tasks)
if err != nil {
log.Fatalln(nil, err)
}
for i, key := range keys {
tasks[i].id = key.ID
}
json.NewEncoder(w).Encode(tasks)
// return tasks, nil
}
It now returns [{"Id":"24X660","Name":"Fiesta","View":129,"Brand":"Ford"}]
Also thank you to #static_cast for correcting my formatting errors.

Related

How to solve the problem of access conflict to shared resources?

There is a test service with 2 requests. Those requests use a shared resource in the form of the ActualOrders variable. Suppose that hundreds of parallel queries are running, there is a chance that a data conflict will occur in the ActualOrders variable. Especially when I'm looping through an array. To prevent this, will it be enough to use a Mutex, as I did in the example below?
main.go:
package main
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"time"
)
type Order struct {
Room string `json:"room"`
UserEmail string `json:"email"`
From time.Time `json:"from"`
To time.Time `json:"to"`
}
var ActualOrders = []Order{}
var mutex sync.Mutex
func getOrders(responseWriter http.ResponseWriter, request *http.Request) {
userEmail := request.URL.Query().Get("email")
results := []Order{}
mutex.Lock()
for _, item := range ActualOrders {
if item.UserEmail == userEmail {
results = append(results, item)
}
}
mutex.Unlock()
bytes, err := json.Marshal(results)
if err != nil {
http.Error(responseWriter, err.Error(), http.StatusInternalServerError)
return
}
responseWriter.Header().Set("Content-type", "application/json")
responseWriter.WriteHeader(http.StatusOK)
responseWriter.Write(bytes)
}
func createOrder(responseWriter http.ResponseWriter, request *http.Request) {
var newOrder Order
requestBody := request.Body
defer request.Body.Close()
err := json.NewDecoder(requestBody).Decode(&newOrder)
if err != nil {
http.Error(responseWriter, err.Error(), http.StatusBadRequest)
return
}
mutex.Lock()
for _, order := range ActualOrders {
if !(newOrder.To.Before(order.From) || newOrder.From.After(order.To)) {
http.Error(responseWriter, http.StatusText(http.StatusConflict), http.StatusConflict)
return
}
}
ActualOrders = append(ActualOrders, newOrder)
mutex.Unlock()
responseWriter.WriteHeader(http.StatusCreated)
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/orders", getOrders)
mux.HandleFunc("/order", createOrder)
err := http.ListenAndServe(":8080", mux)
if errors.Is(err, http.ErrServerClosed) {
fmt.Printf("server closed\n")
} else if err != nil {
fmt.Printf("error starting server: %s\n", err)
os.Exit(1)
}
}
Using a mutex as you did will protect from data races. Your implementation can be improved though.
You can use a RWMutex, use a read-lock for the getOrders function, and a lock for the createOrder function. This will allow exclusive access to the ActualOrders variable when you are writing to it, but shared reads will be allowed:
var mutex sync.RWMutex
func getOrders(responseWriter http.ResponseWriter, request *http.Request) {
...
mutex.RLock()
...
mutex.RUnlock()
}
func createOrder(responseWriter http.ResponseWriter, request *http.Request) {
...
mutex.Lock()
for _, order := range ActualOrders {
...
}
ActualOrders = append(ActualOrders, newOrder)
mutex.Unlock()
}

Semantic way of http.Response receiver functions in Go

I just started learning GO and wrote this piece of code that writes an http.Response.Body to os.Stdout or to a file, but I'm not happy about the semantics of this.
I want the http.Response struct to have these receiver functions, so I can use it more easily throughout the entire app.
I know that the answers might get flagged as opinionated, but I still wonder, is there a better way of writing this?
Is there some sort of best practice?
package main
import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
)
type httpResp http.Response
func main() {
res, err := http.Get("http://www.stackoverflow.com")
if err != nil {
fmt.Println("Error: ", err)
os.Exit(1)
}
defer res.Body.Close()
response := httpResp(*res)
response.toFile("stckovrflw.html")
response.toStdOut()
}
func (r httpResp) toFile(filename string) {
str, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
ioutil.WriteFile(filename, []byte(str), 0666)
}
func (r httpResp) toStdOut() {
_, err := io.Copy(os.Stdout, r.Body)
if err != nil {
panic(err)
}
}
On a side note, is there a way to make the http.Get method spit out a custom type that already has access to these receiver functions without the need for casting? So i could do something like this:
func main() {
res, err := http.Get("http://www.stackoverflow.com")
if err != nil {
fmt.Println("Error: ", err)
os.Exit(1)
}
defer res.Body.Close()
res.toFile("stckovrflw.html")
res.toStdOut()
}
Thanks!
You don't have to implement these functions. *http.Response already implements io.Writer:
Write writes r to w in the HTTP/1.x server response format, including the status line, headers, body, and optional trailer.
package main
import (
"net/http"
"os"
)
func main() {
r := &http.Response{}
r.Write(os.Stdout)
}
In the example above, the zero value prints:
HTTP/0.0 000 status code 0
Content-Length: 0
Playground: https://play.golang.org/p/2AUEAUPCA8j
In case you need additional business logic in the write methods, you can embed *http.Response in your defined type:
type RespWrapper struct {
*http.Response
}
func (w *RespWrapper) toStdOut() {
_, err := io.Copy(os.Stdout, w.Body)
if err != nil {
panic(err)
}
}
But then you must construct a variable of type RespWrapper with the *http.Response:
func main() {
// resp with a fake body
r := &http.Response{Body: io.NopCloser(strings.NewReader("foo"))}
// or r, _ := http.Get("example.com")
// construct the wrapper
wrapper := &RespWrapper{Response: r}
wrapper.toStdOut()
}
is there a way to make the http.Get method spit out a custom type
No, the return types of http.Get are (resp *http.Response, err error), that's part of the function signature, you can't change it.

How correctly to make controllers to routes in Golang?

I am new in Golang and need some help. I am tring to create REST API web service without ORM.
Right now I am successfully connected to PostgreSQL database. In database I have table which called factors. I want to create CRUD operations. The problem is with controllers logic.
main.go:
package main
import (
"github.com/gorilla/mux"
"log"
"net/http"
"rest_api/configurations"
"rest_api/controllers"
)
func main() {
db, err := configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
router := mux.NewRouter()
router.StrictSlash(true)
subrouter := router.PathPrefix("/api").Subrouter()
subrouter.HandleFunc("/factors", controllers.GetFactors(db)).Methods("GET")
log.Fatal(http.ListenAndServe(":8000", router))
}
models/factors.go:
package models
type Factor struct {
ID int `json:"id"`
Name string `json:"name"`
}
How correctly looks like the GetFactors controller? Can someone show me please. For instance I pass db object to GetFactors controller as in the example below. Unfortunately it seems like it's incorrect.
controllers/factors.go:
func GetFactors(db *sql.DB, w http.ResponseWriter, req *http.Request) {
// some code
}
configurations/PostgreSQL.go:
func PostgreSQLDatabase() (*sql.DB, error) {
// Load environment variables from ".env" file.
err := godotenv.Load(".env")
if err != nil {
log.Fatal(err)
}
// Initialize database-related variables.
dbUser := os.Getenv("PostgreSQL_USER")
dbPassword := os.Getenv("PostgreSQL_PASSWORD")
dbHost := os.Getenv("PostgreSQL_HOST")
dbName := os.Getenv("PostgreSQL_DB_NAME")
dbURL := fmt.Sprintf("user=%s password=%s host=%s dbname=%s sslmode=disable", dbUser, dbPassword, dbHost, dbName)
// Create PostgreSQL database connection pool.
db, err := sql.Open("postgres", dbURL)
if err != nil {
return nil, err
}
// Ping PostgreSQL database to make sure it's alive.
err = db.Ping()
if err != nil {
log.Fatal(err)
} else {
log.Println("Web service successfully connected to remote PostgreSQL database.")
}
return db, nil
}
A pattern I like to use is to define your own Router struct that has a mux.Router as a field as well as encapsulates things like your database connection, application config and etc.
I find doing it this way makes it easily update your routes when they require different resources and development proceeds.
First create a router object that takes in the database connection on creation and makes it available to all routes you wish to use.
router.go
package main
import (
"net/http"
"database/sql"
"github.com/gorilla/mux"
)
type Router struct {
router *mux.Router
db *sql.DB
}
func NewRouter(db *sql.DB) (*Router, error) {
router := mux.NewRouter()
router.StrictSlash(true)
subrouter := router.PathPrefix("/api").Subrouter()
r := &Router{
router: router,
db: db,
}
subrouter.HandleFunc("/factors", r.GetFactors).Methods(http.MethodGet)
return r, nil
}
func (r *Router) GetFactors(w http.ResponseWriter, req *http.Request) {
// Now you can access your database via `r.db`
}
// Needed so we can pass our custom router to ListenAndServe.
func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
r.router.ServeHTTP(w, req)
}
Then in main.go you can simply create your custom router, passing it your database connection. Then the custom router can be passed directly to ListenAndServe.
main.go
package main
import (
"log"
"net/http"
"rest_api/configurations"
"rest_api/controllers"
)
func main() {
db, err := configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
router, err := NewRouter(db)
if err != nil {
log.Fatalf("error initializing router: %v", err)
}
log.Fatal(http.ListenAndServe(":8000", router))
}
Hopefully this helps.
Your func GetFactors must looks like:
func GetFactors(w http.ResponseWriter, r *http.Request) {}
and in main file you must have:
subrouter.HandleFunc("/factors", controllers.GetFactors).Methods("GET")
and with purpose to get DB connection you must have func like GetDB in your package "rest_api/configurations".
In "rest_api/configurations" you must have something like:
var db *PostgreSQLDatabase
func init() {
var err error
db, err = configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
}
func GetDB() *PostgreSQLDatabase {
return db
}
There is no correct way, it mostly opinion-based.
The semantic of HandlerFunc function should be like func(w http.ResponseWriter, r *http.Request), in order to pass database you can use closures, here is an example.
main.go
// ... some code here
subrouter.HandleFunc("/factors", controllers.GetFactors(db)).Methods("GET")
// ... some code here
controllers/factors.go
func GetFactors(db *sql.DB) http.HandlerFunc {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
// some code
})
}
Another option:
I'm not quite sure about this, but you can adjucts it to your needs. Initialize a Controller struct and pass db to it:
main.go
// ... some code here
db, err := configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
ctrl := controllers.Controller{DB: db}
subrouter.HandleFunc("/factors", ctrl.GetFactors).Methods("GET")
// ... some code here
Denote a method on the Controller struct.
Define a struct in the controllers
controllers/factors.go
type Controller struct {
DB *PostgreSQLDatabase
}
func (c Controller) GetFactors(w http.ResponseWriter, req *http.Request) {
// some code
// c.DB.MySqlMethod()
}

GORM DB Connection on other package

I start learning Go, reading about pointers, and want to split my database connection , and handler function for API. Already tried myself, by following this solution , but when i trying to read data, i am having this error
[2018-06-26 21:59:45] sql: database is closed
this is my source code.
db.go
package db
import (
"fmt"
"github.com/jinzhu/gorm"
"github.com/joho/godotenv"
"os"
)
var Db *gorm.DB
func Open() error {
var err error
_ = godotenv.Load(".env")
dbType := os.Getenv("DB_TYPE")
dbConnString := os.Getenv("DB_CONN_STRING")
Db, err = gorm.Open(dbType, dbConnString)
if err != nil {
fmt.Println(err)
}
Db.LogMode(true)
defer Db.Close()
return err
}
func Close() error {
return Db.Close()
}
person.go
package model
import (
"github.com/jinzhu/gorm"
"github.com/gin-gonic/gin"
"fmt"
"namastra/gin/result"
"namastra/gin/db"
)
type Person struct {
gorm.Model
FirstName string `json:”firstname”`
LastName string `json:”lastname”`
}
/*var db *gorm.DB
var err error*/
func GetPeople(c *gin.Context) {
var people []result.Person
if err := db.Db.Select("ID,first_name,last_name").Find(&people).Error; err != nil {
c.AbortWithStatus(404)
fmt.Println(err)
} else {
c.JSON(200, people)
}
}
main.go
package main
import (
"log"
"namastra/gin/handler"
"namastra/gin/model"
"net/http"
"time"
"github.com/adam-hanna/jwt-auth/jwt"
"github.com/gin-gonic/gin"
_ "github.com/jinzhu/gorm/dialects/postgres"
"namastra/gin/db"
)
func main() {
if err := db.Open(); err != nil {
// handle error
panic(err)
}
defer db.Close()
router := gin.Default()
router.Use(gin.Recovery())
private := router.Group("/auth")
....(ommited)
router.GET("/", gin.WrapH(regularHandler))
router.GET("/people/", model.GetPeople)
router.Run("127.0.0.1:3000")
}
Sorry for my bad english, any kind of help is appreciated.
thank you.
edit1: case closed.
solution is by removing
defer Db.Close()
from db.go.
edi2: update some knowledge i learn by working in go project
As start learning GO, usually we put everything on single main.go file, and we think to split the code to multiple files.
That is the time Dependency Injection comes to play.
we can create something like this Env to store the handler.
type Env struct {
db *sql.DB
logger *log.Logger
templates *template.Template
}
and create something like this in models/db.go
package models
import (
"database/sql"
_ "github.com/lib/pq"
)
func NewDB(dataSourceName string) (*sql.DB, error) {
db, err := sql.Open("postgres", dataSourceName)
if err != nil {
return nil, err
}
if err = db.Ping(); err != nil {
return nil, err
}
return db, nil
}
main.go files
package main
import (
"namastra/gin/models"
"database/sql"
"fmt"
"log"
"net/http"
)
type Env struct {
db *sql.DB
}
func main() {
db, err := models.NewDB("postgres://user:pass#localhost/bookstore")
if err != nil {
log.Panic(err)
}
env := &Env{db: db}
http.HandleFunc("/peoples", env.peoplesIndex)
http.ListenAndServe(":3000", nil)
}
func (env *Env) peoplesIndex(w http.ResponseWriter, r *http.Request) {
# ...
}
and in models/people.go
package models
import "database/sql"
type Book struct {
Isbn string
Title string
Author string
Price float32
}
func AllPeoples(db *sql.DB) ([]*People, error) {
rows, err := db.Query("SELECT * FROM peoples")
if err != nil {
return nil, err
}
defer rows.Close()
# ... ommited for simplicity
}
you can read the full code & explanation in Alex Edwards post

Method declarations with receivers in Go

The following errors:
./main.go:13: c.Set undefined (type redis.Conn has no field or method Set)
./main.go:19: invalid receiver type *redis.Conn (redis.Conn is an interface type)
./main.go:20: red.Send undefined (type *redis.Conn has no field or method Send)
are produced from this code:
package main
import (
"encoding/json"
"github.com/garyburd/redigo/redis"
"github.com/gorilla/mux"
"log"
"net/http"
)
func setHandler(res http.ResponseWriter, req *http.Request) {
c := connectRedis()
c.Set("foo", "bar")
data, _ := json.Marshal("{'order':1, 'weight': 100, 'reps': 5, 'rest': 1}")
res.Header().Set("Content-Type", "application/json; charset=utf-8")
res.Write(data)
}
func (red *redis.Conn) Set(key string, value string) error {
if _, err := red.Send("set", key, value); err != nil {
return err
}
}
func connectRedis() redis.Conn {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
// handle error
}
defer c.Close()
return c
}
func main() {
r := mux.NewRouter()
r.HandleFunc("/sets.json", setHandler)
http.Handle("/", r)
err := http.ListenAndServe(":7000", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
How can I register a method when I have an interface?
You are trying to add a new method to a preexisting structure from another package; you cannot do that.
The right way to do it would be to define a structure in your own package which would alias the preexisting structure and inherit all of its methods. After doing that you will be able to add your new method to your new structure.
After that you can use your own structure everywhere instead so you can access the extra methods.
You might be tempted now to see if you can attach methods and behavior to any type, say like an int or time.Time - not possible. You will be able to add methods for a type only if the type is defined in the same package.
from GoLang Tutorials - Methods on structs

Resources