I have the ECHO framework which should return the file on request, and it's working well
func IniExport(c echo.Context) error{
cfg := ini.Empty()
if section, err := cfg.NewSection("test_section"); err != nil {
return c.JSON(http.StatusInternalServerError, "Problems with generation of export file.")
}
if key, err := cfg.Section("test_section").NewKey("name", "value"); err != nil {
return c.JSON(http.StatusInternalServerError, "Problems with generation of export file.")
}
cfg.SaveTo("export.ini")
defer os.Remove("export.ini")
return c.Attachment("export.ini", "export.ini")
}
But question, is it possible to not create physical file export.ini and after do not remove it? Possible to return content on the fly somehow?
Thanks
I think you need Send Blob.
Send Blob Context#Blob(code int, contentType string, b []byte) can be
used to send an arbitrary data response with provided content type and
status code.
Example
func(c echo.Context) (err error) {
data := []byte(`0306703,0035866,NO_ACTION,06/19/2006
0086003,"0005866",UPDATED,06/19/2006`)
return c.Blob(http.StatusOK, "text/csv", data)
}
You can use the WriteTo function to write the cfg content to the io.Writer first and then those contents can be used instead of data(in the previous code example. Also make sure to change the content type to text/plain)
Related
I have two golang servers running on localhost.
They are using different ports.
I want to create a post request on one that sends a JSON object to the other one.
I am using the echo framework (if this matters)
The error I am getting is when I try to marshal the object for the post object:
2-valued json.Marshal(data) (value of type ([]byte, error)) where single value is expected
server 1:
type SendEmail struct {
SenderName string `json:"senderName,omitempty" bson:"senderName,omitempty" validate:"required,min=3,max=128"`
SenderEmail string `json:"senderEmail" bson:"senderEmail" validate:"required,min=10,max=128"`
Subject string `json:"subject" bson:"subject" validate:"required,min=10,max=128"`
RecipientName string `json:"recipientName" bson:"recipientName" validate:"required,min=3,max=128"`
RecipientEmail string `json:"recipientEmail" bson:"recipientEmail" validate:"required,min=10,max=128"`
PlainTextContent string `json:"plainTextContent" bson:"plainTextContent" validate:"required,min=10,max=512"`
}
func resetPassword(c echo.Context) error {
email := c.Param("email")
if email == "" {
return c.String(http.StatusNotFound, "You have not supplied a valid email")
}
data := SendEmail{
RecipientEmail: email,
RecipientName: email,
SenderEmail: “test#test”,
SenderName: “name”,
Subject: "Reset Password",
PlainTextContent: "Here is your code to reset your password, if you did not request this email then please ignore.",
}
// error here
req, err := http.NewRequest("POST", "127.0.0.1:8081/", json.Marshal(data))
if err != nil {
fmt.Println(err)
}
defer req.Body.Close()
return c.JSON(http.StatusOK, email)
}
server 2:
e.GET("/", defaultRoute)
func defaultRoute(c echo.Context) (err error) {
u := SendEmail{}
if err = c.Bind(u); err != nil {
return
}
return c.JSON(http.StatusOK, u)
}
It's always nice to meet a Gopher. A few things you might want to know, Go supports multi-value returns in that a function can return more than one value.
byteInfo, err := json.Marshal(data) // has two values returned
// check if there was an error returned first
if err != nil{
// handle your error here
}
Now the line below in your code
// error here
req, err := http.NewRequest("POST", "127.0.0.1:8081/", json.Marshal(data))
Will become this
// error here
req, err := http.NewRequest("POST", "127.0.0.1:8081/", bytes.NewBuffer(byteInfo))
And you can continue with the rest of your code. Happy Coding!
json.Marshal returns []byte and error which means you're passing 4 values to http.NewRequest.
You should call json.Marshal first and then use the result for http.NewRequest.
body, err := json.Marshal(data)
if err != nil {
// deal with error
}
req, err := http.NewRequest("POST", "127.0.0.1:8081/", body)
I am trying to restore the context with it's data after performing validation on it's data.I need the data to keep moving as need it later on in the next function.
I am new to golang and the below code is as far I could go. any help and a better approach is much appreciated.
thanks in advance.
the validation middleware
func SignupValidator(c *gin.Context) {
// Read the Body content
// var bodyBytes []byte
// if c.Request.Body != nil {
// bodyBytes, _ = ioutil.ReadAll(c.Request.Body)
// }
var user entity.User
if err := c.ShouldBindJSON(&user); err != nil {
validate := validator.New()
if err := validate.Struct(&user); err != nil {
c.JSON(http.StatusBadRequest, gin.H{
"error": err.Error(),
})
c.Abort()
return
}
// c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(bodyBytes))
}
// Read the Body content
var bodyBytes []byte
if c.Request.Body != nil {
bodyBytes, _ = ioutil.ReadAll(c.Request.Body)
}
fmt.Println(string(bodyBytes)) // this empty
c.Next()
}
route
auth.POST("login", gin.Logger(), validations.SignupValidator, func(ctx *gin.Context) {
ctx.JSON(200, videoController.Save(ctx))
})
You can try this.
ByteBody, _ := ioutil.ReadAll(c.Request.Body)
c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(ByteBody))
You can then use ByteBody however you want without side-effects on c.Request.Body
Here is an example of reading body twice with ShouldBindBodyWith, check it:
package main
import (
"log"
"net/http"
"github.com/gin-gonic/gin"
"github.com/gin-gonic/gin/binding"
)
type ParamsOne struct {
Username string `json:"username"`
}
type ParamsTwo struct {
Username string `json:"username"`
}
func main() {
r := gin.New()
r.POST("/", func(c *gin.Context) {
var f ParamsOne
// Read ones
if err := c.ShouldBindBodyWith(&f, binding.JSON); err != nil {
log.Printf("%+v", err)
}
log.Printf("%+v", f)
var ff ParamsTwo
if err := c.ShouldBindBodyWith(&ff, binding.JSON); err != nil {
log.Printf("%+v", err)
}
log.Printf("%+v", ff)
c.IndentedJSON(http.StatusOK, f)
})
r.Run(":4000")
}
Output:
$example: ./example
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] POST / --> main.main.func1 (1 handlers)
[GIN-debug] Listening and serving HTTP on :4000
2020/07/05 10:47:03 {Username:somename}
2020/07/05 10:47:03 {Username:somename}
As #Philidor has shown ShouldBindBodyWith should do the trick, in my case I decided to go with something similar to #spehlivan, because of two reasons:
ShouldBindBodyWith requires that the following binds are also ShouldBindBodyWith, it means I need to change all my previous code, which uses c.Bind
You need to explicitly tell to ShouldBindBodyWith the binding type you are trying to do, JSON, Form, ProtoBuf, etc, other binds like c.Bind detects it automatically.
This is what it looks like:
var input models.SomeInput
bodyCopy := new(bytes.Buffer)
// Read the whole body
_, err := io.Copy(bodyCopy, c.Request.Body)
if err != nil {
log.Println(err)
c.JSON(http.StatusBadRequest, gin.H{"error": "Error reading API token"})
c.Abort()
return
}
bodyData := bodyCopy.Bytes()
// Replace the body with a reader that reads from the buffer
c.Request.Body = ioutil.NopCloser(bytes.NewReader(bodyData))
err = c.Bind(&input)
// Some code here...
// Replace the body with a reader that reads from the buffer
c.Request.Body = ioutil.NopCloser(bytes.NewReader(bodyData))
Pay attention I replaced the c.Request.Body twice, for the bind in that code snippet and then at the end, for the next bind, in another place of my code (this snippet is from a middleware, the next bind is called from the controller).
In my case I needed to do this because the API Token is sent in the request body, which I don't recommend, it should be sent in the request header.
I have a function that will be called on every single HTTP GET request. The function reads a file, does some stuff to the contents of that file, and returns a slice of bytes of those contents. That slice of bytes of then written as the response body to the HTTP response writer.
Do I need to use a mutex for any of the steps in this function to prevent locking in the event of multiple HTTP requests trying to read the same file? And if so, would a simple RWMutex locking the reading of the file suffice, since I am not actually writing to it but am creating a copy of its contents?
Here is the function:
// prepareIndex will grab index.html and add a nonce to the script tags for the CSP header compliance.
func prepareIndex(nonce string) []byte {
// Load index.html.
file, err := os.Open("./client/dist/index.html")
if err != nil {
log.Fatal(err)
}
// Convert to goquery document.
doc, err := goquery.NewDocumentFromReader(file)
if err != nil {
fmt.Println(err)
}
// Find all script tags and set nonce.
doc.Find("body > script").SetAttr("nonce", nonce)
// Grab the HTML string.
html, err := doc.Html()
if err != nil {
fmt.Println(err)
}
return []byte(html)
}
I also thought about just loading the file once when main starts, but I was having a problem where only the first request could see the data and the subsequent requests saw nothing. Probably an error in the way I was reading the file. But I actually prefer my current approach because if there are any changes to index.html, I want them to be persisted to the user immediately without having to restart the executable.
Using RWMutex won't protect you from the file being modified by another program. The best option here would be to load your file in a []byte at startup, and instantiate "bytes".Buffer whenever you use goquery.NewDocumentFromReader. In order for the changes to be propagated to the user, you can use fsnotify[1] to detect changes to your file, and update your cached file ([]byte) when necessary (you will need RWMutex for that operation).
For example:
type CachedFile struct {
sync.RWMutex
FileName string
Content []byte
watcher *fsnotify.Watcher
}
func (c *CachedFile) Buffer() *bytes.Buffer {
c.RLock()
defer c.RUnlock()
return bytes.NewBuffer(c.Content)
}
func (c *CachedFile) Load() error {
c.Lock()
content, err := ioutil.ReadAll(c.FileName)
if err != nil {
c.Unlock()
return err
}
c.Content = content
c.Unlock()
}
func (c *CachedFile) Watch() error {
var err error
c.watcher, err = fsnotify.NewWatcher()
if err != nil {
return err
}
go func() {
for ev := range c.watcher.Events {
if ev.Op != fsnotify.Write {
continue
}
err := c.Load()
if err != nil {
log.Printf("loading %q: %s", c.FileName, err)
}
}
}()
err = c.watcher.Add(c.FileName)
if err != nil {
c.watcher.Close()
return err
}
return nil
}
func (c *CachedFile) Close() error {
return c.watcher.Close()
}
[1] https://godoc.org/github.com/fsnotify/fsnotify
If you're modifying the file, you need a mutex. RWMutex should work fine. It looks like you're just reading it, and in that case you should not see any locking behavior or corruption.
The reason you didn't get any data the second time you read from the same file handle is that you're already at the end of the file when you start reading from it the second time. You need to seek back to offset 0 if you want to read the contents again.
I'm using GIN as GO framework, I'm having an issue when uploading file and directly convert image as byte so I can store it in my BLOB field inside db table, so I have my piece of code like this:
func (a *AppHandler) Upload(ctx *gin.Context) {
form := &struct {
Name string `form:"name" validate:"required"`
Token string `form:"token" validate:"required"`
AppCode string `form:"app_code" validate:"required"`
}{}
ctx.Bind(form)
if validationErrors := a.ValidationService.ValidateForm(form); validationErrors != nil {
httpValidationErrorResponse(ctx, validationErrors)
return
}
file, header, err := ctx.Request.FormFile("file")
and I'm trying to store it in db like this
app.SetFile(file)
a.AppStore.Save(app)
and it returns this kind of error:
cannot use file (type multipart.File) as type []byte
*multipart.File implements io.Reader interface so you could copy its content into a bytes.Buffer like this:
file, header, err := ctx.Request.FormFile("file")
defer file.Close()
if err != nil {
return nil, err
}
buf := bytes.NewBuffer(nil)
if _, err := io.Copy(buf, file); err != nil {
return nil, err
}
and then add to your app
app.SetFile(buf.Bytes())
I'm currently developing a download server in Go. I need to limit the download speed of users to 100KB/s.
This was my code:
func serveFile(w http.ResponseWriter, r *http.Request) {
fileID := r.URL.Query().Get("fileID")
if len(fileID) != 0 {
w.Header().Set("Content-Disposition", "attachment; filename=filename.txt")
w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
w.Header().Set("Content-Length", r.Header.Get("Content-Length"))
file, err := os.Open(fmt.Sprintf("../../bin/files/test.txt"))
defer file.Close()
if err != nil {
http.NotFound(w, r)
return
}
io.Copy(w, file)
} else {
io.WriteString(w, "Invalid request.")
}
}
Then I found a package on github and my code became the following:
func serveFile(w http.ResponseWriter, r *http.Request) {
fileID := r.URL.Query().Get("fileID")
if len(fileID) != 0 {
w.Header().Set("Content-Disposition", "attachment; filename=Wiki.png")
w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
w.Header().Set("Content-Length", r.Header.Get("Content-Length"))
file, err := os.Open(fmt.Sprintf("../../bin/files/test.txt"))
defer file.Close()
if err != nil {
http.NotFound(w, r)
return
}
bucket := ratelimit.NewBucketWithRate(100*1024, 100*1024)
reader := bufio.NewReader(file)
io.Copy(w, ratelimit.Reader(reader, bucket))
} else {
io.WriteString(w, "Invalid request.")
}
}
But I'm getting this error:
Corrupted Content Error
The page you are trying to view cannot be shown because an error in
the data transmission was detected.
Here's my code on the Go playground: http://play.golang.org/p/ulgXQl4eQO
Rather than mucking around with getting the correct the content type and length headers yourself it'd probably be much better to use http.ServeContent which will do that for you (as well as support "If-Modified-Since", range requests, etc. If you can supply an "ETag" header it can also handle "If-Range" and "If-None-Match" requests as well).
As mentioned previously, it's often preferable to limit on the write side but it's awkward to wrap an http.ResponseWriter since various http functions also check for optional interfaces such as http.Flusher and http.Hijacker. It's much easier to wrap the io.ReadSeeker that ServeContent needs.
For example, something like this perhaps:
func pathFromID(fileID string) string {
// replace with whatever logic you need
return "../../bin/files/test.txt"
}
// or more verbosely you could call this a "limitedReadSeeker"
type lrs struct {
io.ReadSeeker
// This reader must not buffer but just do something simple
// while passing through Read calls to the ReadSeeker
r io.Reader
}
func (r lrs) Read(p []byte) (int, error) {
return r.r.Read(p)
}
func newLRS(r io.ReadSeeker, bucket *ratelimit.Bucket) io.ReadSeeker {
// Here we know/expect that a ratelimit.Reader does nothing
// to the Read calls other than add delays so it won't break
// any io.Seeker calls.
return lrs{r, ratelimit.Reader(r, bucket)}
}
func serveFile(w http.ResponseWriter, req *http.Request) {
fileID := req.URL.Query().Get("fileID")
if len(fileID) == 0 {
http.Error(w, "invalid request", http.StatusBadRequest)
return
}
path := pathFromID(fileID)
file, err := os.Open(path)
if err != nil {
http.NotFound(w, req)
return
}
defer file.Close()
fi, err := file.Stat()
if err != nil {
http.Error(w, "blah", 500) // XXX fixme
return
}
const (
rate = 100 << 10
capacity = 100 << 10
)
// Normally we'd prefer to limit the writer but it's awkward to wrap
// an http.ResponseWriter since it may optionally also implement
// http.Flusher, or http.Hijacker.
bucket := ratelimit.NewBucketWithRate(rate, capacity)
lr := newLRS(file, bucket)
http.ServeContent(w, req, path, fi.ModTime(), lr)
}
I'm not seeing the error, but I did notice some issues with the code. For this:
w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
You should use the mime package's:
func TypeByExtension(ext string) string
To determine the content type. (if you end up with the empty string default to application/octet-stream)
For:
w.Header().Set("Content-Length", r.Header.Get("Content-Length"))
You need to get the content length from the file itself. By using the request content length, for a GET this basically ends up as a no-op, but for a POST you're sending back the wrong length, which might explain the error you're seeing. After you open the file, do this:
fi, err := file.Stat()
if err != nil {
http.Error(w, err.Error(), 500)
return
}
w.Header().Set("Content-Length", fmt.Sprint(fi.Size()))
One final thing, when you open the file, if there's an error, you don't need to close the file handle. Do it like this instead:
file, err := os.Open(...)
if err != nil {
http.NotFound(w, r)
return
}
defer file.Close()