Process image given multiple URL parameters in Go [closed] - go

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I would probably prefer to use pressly/chi, but I guess it makes no difference. I imagine given an input URL like this example.com/Jd8saD.jpg?resize=420x320&fit=crop&rotate=90, then it would go to the following GET function because of r.Get("/:image", ImageGET):
function ImageGET(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Get("resize") != "" {
// do something
}
if r.URL.Query().Get("crop") != "" {
// do something
}
if r.URL.Query().Get("watermark") != "" {
// do something
}
etc
}
Now, my question is, how should I design whatever function does the image processing so that it will process everything properly and efficiently? I don't expect you to write code that will handle the resizing, but how would these functions look like? Perhaps:
function Resize(size string) (imgfile?, error) {
// the resize stuff
}
What would that returned imgfile even be? A struct containing some relevant img info?

Likely,
imgfile
will satisfy the image.Image interface and not the data saved on disk (ie. the actual jpg file)
Image is a finite rectangular grid of color.Color values taken from a color model.
Lots of 3rd party golang image libraries use image.Image to manipulate images.
I would use a standard image.Image interface retrieved (read to memory) by a filename in the imageGET function and modified according to the queries. You can see also the jpeg golang library from the standard lib.
function ImageGET(w http.ResponseWriter, r *http.Request) {
// GetJPEGImageFromFileSystem must decode os.File content and
// return a golang image.Image interface
img, _ := GetJPEGImageFromFileSystem(r.URL.Query().Get("filename"))
if r.URL.Query().Get("resize") != "" {
// If resizing, write over the image.Image in Memory,
// but not the image File on disk
img, _ = Resize(img, r.URL.Query().GET("resize"))
}
// etc...
}
function Resize(img image.Image, size string) (image.Image, error) {
// the resize stuff -- or, alternatively just call a dependency in the original handler to resize
return resizedImage, nil
}

Now, my question is, how should I design whatever function does the
image processing so that it will process everything properly and
efficiently?
Depends on the package you are using for your stuff and what you want to do with it. If you look for example to the imaging package you see that they return always: *image.NRGBA
That type implements the image.Image interface.
At the next step you can use the Encode function.
func Encode(w io.Writer, img image.Image, format Format) error
As you see that function uses a io.Writer.
function ImageGET(w http.ResponseWriter, r *http.Request) {
// ...
imaging.Encode(w,img,imaging.PNG)
// ...
So you just need to use the writer from the your handler there and ready.
So to keep your function correct just return the image.Image interface.

Related

How to receive publishTime [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I have a GC function in Go, It is triggered by pubsub push subscription to a topic. I am trying to receive the publishTime and it does not appear populated.
The specs say string (Timestamp format), however, that field in my model is empty. Every other piece of information is populated except publishTime.
type PubSubMessageModel struct {
Data []byte `json:"data"`
Attributes map[string]string `json:"attributes"`
PublishTime string `json:"publishTime"`
}
Simply trying to log the property;
log.Info().Msgf("Publish Time is - %s\n", m.PublishTime)
results in an empty string:
Publish Time is -
Any suggestions?
It's not very well documented, but it is.
The event structure is different when you have a push to the function directly compare to a push Subscription with a HTTP triggered function.
Instead, you can access publishTime and messageId via the event ID and timestamp properties of the event metadata. This metadata is accessible via the context object that is passed to your function when it is invoked.
In the background context description, in Go, there is an explanation of the package to use (functions/metadata) to get the context values
Here how to achieve this
Perform a go get cloud.google.com/go
Use this dependency in your code and get the message metadata from the context
import (
"cloud.google.com/go/functions/metadata"
"context"
"log"
)
type PubSubMessage struct {
Data []byte `json:"data"`
}
// HelloPubSub consumes a Pub/Sub message.
func HelloPubSub(ctx context.Context, m PubSubMessage) error {
meta,err := metadata.FromContext(ctx)
if err != nil {
log.Println(err)
return err
}
log.Printf("publishTime, %s", meta.Timestamp)
log.Printf("messageId, %s", meta.EventID)
log.Printf("Type, %s", meta.EventType)
log.Printf("Resource, %s", meta.Resource)
return nil
}

Insert data with Gorm with reflect

I'm creating a basic REST service. My intent is to write the logic for the resources as abstractly as possible. What I mean is if I have already created a CRUD logic for endpoint /devices for example, then when I need a new resource endpoint like /cars, I should not be repeating myself over the CRUD procedures.
In another language like Python, classes and methods are first class objects and that can be stored in a list or dictionary (map) and then instantiated as needed. In Go it doesn't seem as easy. I tried to use the reflect package.
First I create a TypeRegistry according to this.
var TypeRegistry = make(map[string]reflect.Type)
TypeRegistry["devices"] = reflect.TypeOf(models.Device{}) // models.Device{} is the Gorm SQL table model
Then I have handler creator which is intended to handle the creation of all types of resources like this (error handling redacted):
func CreateOneHandler(typeString string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
jsn, _ = ioutil.ReadAll(r.Body)
jsonBytes, _ := datamapper.CreateOne(typeString, jsn)
w.Write(jsonBytes)
}
}
I'm using Chi, so I bind the handlers like this:
func addRoute(r chi.Router, endpoint string, typeString string) {
r.Route("/"+endpoint, func(r chi.Router) {
typeString := endpoint
r.Post("/", CreateOneHandler(typeString))
})
}
The idea is to, after defining the Gorm models, simply add routes by calling it repeatedly, addRoute(r, "devices"); addRoute(r, "cars") for a consistent REST interface across multiple models.
Now within CreateOne() I want to insert something into the table:
func CreateOne(typeString string, json []byte) ([]byte, error) {
modelType := typeregistry.TypeRegistry[typeString]
value := reflect.New(modelType)
db.Create(modelPtr.Elem()) // ==> Now this doesn't work
}
How do I make it work? Gorm said "create failed no such table: value". Because a reflect value or reflect type isn't the same as if I were just to instantiate objects the regular way. How do I make it work?
(A side note: given the static nature of the type switch and type assertions, I am already compromising some of my designs which would probably be possible in a language like Python. It seems to me like it's unavoidable to litter code with type switches which tried to check whether it is a device, car or any number of new models explicitly. In a regular object-oriented language maybe this would be simple polymorphic method call. Any pointer to better design would be appreciated as well.)

Convert String to Function name in GO? [duplicate]

This question already has answers here:
Call functions with special prefix/suffix
(2 answers)
Closed 6 years ago.
I am creating a Restful API.
I am passing function name and arguments in JSON
eg. "localhost/json_server?method=foo&id=1"
Lets say, i have a simple go server running
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Println("path",r.URL.Path )
fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
})
.........
function json_server(){
....
}
The r.url.Path will give me "json_server" in string. Now i want to first check if the function exists , if exists call the function as defined else throw some exception.
Is this possible to do ?
When i am doing python and i use getattr(method,args) to call the method and arguments which are in string.
I have developed interest in Go after using Docker. Any help will be appreciated.
As far as I know it's not possible to enumerate the functions of a package using the reflection api, but see this mailing list discussion for some ideas involving parsing the source files. Enumerating methods of an object is possible, which is actually more to what you describe in python.
However, I would recommend using a simple dispatch table instead of introspection, you can populate a map[string]func(), though I suspect you might want to pass some arguments to your function, e.g. the request to be handled:
var dispatch map[string]http.HandlerFunc
func init() {
dispatch = make(map[string]http.HandlerFunc)
dispatch["json_server"] = json_server
dispatch["foo"] = func(w http.ResponseWriter, r *http.Request) {
...
}
}
func ServeHTTP (w http.ResponseWriter, r *http.Request) {
if handler, exists := dispatch[req.URL.Path]; exists {
handler(w, r)
} else {
... // fallback
}
}
Or better yet, just use an existing HTTP router, such as httprouter or gorilla/mux. There are many alternatives to choose from.

Read template in init or in handler function?

I'm writing a basic server for a website. Now I face a (for me) difficult performance question. Is it better to read the template file in the init() function?
// Initialize all pages of website
func init(){
indexPageData, err := ioutil.ReadFile("./tpl/index.tpl")
check(err)
}
Or in the http.HandlerFunc?
func index(w http.ResponseWriter, req *http.Request){
indexPageData, err := ioutil.ReadFile("./tpl/index.tpl")
check(err)
indexPageTpl := template.Must(template.New("index").Parse(string(indexPageData)))
indexPageTpl.Execute(w, "test")
}
I think in the first example, after the server is started you have no need to access the disk and increase the performance of the request.
But during development I want to refresh the browser and see the new content. That can be done with the second example.
Does someone have a state-of-the-art solution? Or what is the right from the performance point of view?
Let's analyze the performance:
We name your first solution (with slight changes, see below) a and your second solution b.
One request:
a: One disk access
b: One disk access
Ten requests:
a: One disk access
b: Ten disk accesses
10 000 000 requests:
a: One disk access
b: 10 000 000 disk accesses (this is slow)
So, performance is better with your first solution. But what about your concern regarding up-to-date data? From the documentation of func (t *Template) Execute(wr io.Writer, data interface{}) error:
Execute applies a parsed template to the specified data object, writing the output to wr. If an error occurs executing the template or writing its output, execution stops, but partial results may already have been written to the output writer. A template may be executed safely in parallel.
So, what happens is this:
You read a template from disk
You parse the file into a template
You choose the data to fill in the blanks with
You Execute the template with that data, the result is written out into an io.Writer
Your data is as up-to-date as you choose it. This has nothing to do with re-reading the template from disk, or even re-parsing it. This is the whole idea behind templates: One disk access, one parse, multiple dynamic end results.
The documentation quoted above tells us another thing:
A template may be executed safely in parallel.
This is very useful, because your http.HandlerFuncs are ran in parallel, if you have multiple requests in parallel.
So, what to do now?
Read the template file once,
Parse the template once,
Execute the template for every request.
I'm not sure if you should read and parse in the init() function, because at least the Must can panic (and don't use some relative, hard coded path in there!) - I would try to do that in a more controlled environment, e.g. provide a function (like New()) to create a new instance of your server and do that stuff in there.
EDIT: I re-read your question and I might have misunderstood you:
If the template itself is still in development then yes, you would have to read it on every request to have an up-to-date result. This is more convenient than to restart the server every time you change the template. For production, the template should be fixed and only the data should change.
Sorry if I got you wrong there.
Never read and parse template files in the request handler in production, that is as bad as it can get (you should like always avoid this). During development it is ok of course.
Read this question for more details:
It takes too much time when using "template" package to generate a dynamic web page to client in golang
You could approach this in multiple ways. Here I list 4 with example implementation.
1. With a "dev mode" setting
You could have a constant or variable telling if you're running in development mode which means templates are not to be cached.
Here's an example to that:
const dev = true
var indexTmpl *template.Template
func init() {
if !dev { // Prod mode, read and cache template
indexTmpl = template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
}
}
func getIndexTmpl() *template.Template {
if dev { // Dev mode, always read fresh template
return template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
} else { // Prod mode, return cached template
return indexTmpl
}
}
func indexHandler(w http.ResponseWriter, r *http.Request) {
getIndexTmpl().Execute(w, "test")
}
2. Specify in the request (as a param) if you want a fresh template
When you develop, you may specify an extra URL parameter indicating to read a fresh template and not use the cached one, e.g. http://localhost:8080/index?dev=true
Example implementation:
var indexTmpl *template.Template
func init() {
indexTmpl = getIndexTmpl()
}
func getIndexTmpl() *template.Template {
return template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
}
func indexHandler(w http.ResponseWriter, r *http.Request) {
t := indexTmpl
if r.FormValue("dev") != nil {
t = getIndexTmpl()
}
t.Execute(w, "test")
}
3. Decide based on host
You can also check the host name of the request URL, and if it is "localhost", you can omit the cache and use a fresh template. This requires the smallest extra code and effort. Note that you may want to accept other hosts as well e.g. "127.0.0.1" (up to you what you want to include).
Example implementation:
var indexTmpl *template.Template
func init() {
indexTmpl = getIndexTmpl()
}
func getIndexTmpl() *template.Template {
return template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
}
func indexHandler(w http.ResponseWriter, r *http.Request) {
t := indexTmpl
if r.URL.Host == "localhost" || strings.HasPrefix(r.URL.Host, "localhost:") {
t = getIndexTmpl()
}
t.Execute(w, "test")
}
4. Check template file last modified
You could also store the last modified time of the template file when it is loaded. Whenever the template is requested, you can check the last modified time of the source template file. If it has changed, you can reload it before executing it.
Example implementation:
type mytempl struct {
t *template.Template
lastmod time.Time
mutex sync.Mutex
}
var indexTmpl mytempl
func init() {
// You may want to call this in init so first request won't be slow
checkIndexTempl()
}
func checkIndexTempl() {
nm := ".tpl/index.tpl"
fi, err := os.Stat(nm)
if err != nil {
panic(err)
}
if indexTmpl.lastmod != fi.ModTime() {
// Changed, reload. Don't forget the locking!
indexTmpl.mutex.Lock()
defer indexTmpl.mutex.Unlock()
indexTmpl.t = template.Must(template.New("index").ParseFiles(nm))
indexTmpl.lastmod = fi.ModTime()
}
}
func indexHandler(w http.ResponseWriter, r *http.Request) {
checkIndexTempl()
indexTmpl.t.Execute(w, "test")
}

Defining an interface method with interface return type

TLDR Here is a playground that demonstrates the issue if you try to run it: https://play.golang.org/p/myQtUVg1iq
I am making a REST API and have many types of resources that can be retrieved via a GET request
GET http://localhost/api/users
GET http://localhost/api/groups
I have a models package which abstracts how the different resources are implemented:
func(m *UserManager) Get() []Users {
// Internal logic, assume returns correct results
}
func(m *GroupManager) Get() []Groups {
// Internal logic, assume returns correct results
}
A routes file setups all the routes and handlers:
users := models.UserManager{}
groups := models.GroupManager{}
func GetUsersHandler (w http.ResponseWriter, r *http.Request) {
users := users.Get()
// Implementation details, writing to w as JSON
}
func GetGroupsHandler (w http.ResponseWriter, r *http.Request) {
groups := groups.Get()
// Implementation details, writing to w as JSON
}
func registerRoutes(r *mux.Router) {
r.handleFunc("/api/users", GetUsersHandler).Method("GET")
r.handleFunc("/api/groups", GetGroupsHandler).Method("GET")
}
I am trying to make this more generic by creating an interface and then only needing a single GetHandler. Something like this:
type Getter interface {
Get() []interface{}
}
func GetHandler(g Getter) {
return func(w http.ResponseWriter, r *http.Request) {
results := g.Get()
// Implementation details, writing to w as JSON
}
}
func registerRoutes(r *mux.Router) {
r.handleFunc("/api/users", GetHandler(&users)).Method("GET")
r.handleFunc("/api/groups", GetHandler(&groups)).Method("GET")
}
This is really close to working, the only problem is the return type from the models is a specific object type, but the interface just uses the interface return type. Is there any way to solve this without making the models return []interface{}?
https://play.golang.org/p/myQtUVg1iq
Try not to approach the problem like you would other OOP languages. You can't have covariant containers in Go, so you either have to use an empty interface{}, or you have to structure your program differently.
If your Get methods are different and you want to group types in an interface, use another method (sometimes we even have noop methods just for interfaces), or just pass in users or groups as an interface{}. You'll need to do a type switch or assertion at some point in the call chain anyway, and once you know what type it is you can handle it accordingly.
It's hard to tell without more code, but in this case, the easiest path may just be to have each type be an http.Handler itself, and it can dispatch accordingly.
I ended up avoiding this problem entirely and instead of trying to reduce the amount of code I am using the new go generate feature in Go 1.4 to create the code that is necessary for each resource.

Resources