How to serve file from go embed - go

I have a static directory, containing a sign.html file :
//go:embed static
var static embed.FS
It is served that way and works fine :
fSys, err := fs.Sub(static, "static")
if err != nil {
return err
}
mux.Handle("/", http.FileServer(http.FS(fSys)))
On some routes though (for instance: /sign), I want to do some checks before serving the page. This is my handler :
func (h Handler) ServeSignPage(w http.ResponseWriter, r *http.Request) error {
publicKey := r.URL.Query().Get("publicKey")
err := h.Service.AuthorizeClientSigning(r.Context(), publicKey)
if err != nil {
return err
}
// this is where I'd like to serve the embed file
// sign.html from the static directory
http.ServeFile(w, r, "sign.html")
return nil
}
Unfortunately, the ServeFile displays not found. How can I serve the file from the file server within that ServeSignPage ?

Option 1
Read the file to a slice of bytes. Write the bytes to the response.
p, err := static.ReadFile("static/sign.html")
if err != nil {
// TODO: Handle error as appropriate for the application.
}
w.Write(p)
Option 2
If the path for the ServeSignPage handler is the same as the static file in the file server, then delegate to the file server.
Store the file server in a package-level variable.
var staticServer http.Handler
func init() {
fSys, err := fs.Sub(static, "static")
if err != nil {
panic(err)
}
staticServer = http.FileServer(http.FS(fSys)))
}
Use the static server as the handler:
mux.Handle("/", staticServer)
Delegate to the static server in ServeSignPage:
func (h Handler) ServeSignPage(w http.ResponseWriter, r *http.Request) error {
publicKey := r.URL.Query().Get("publicKey")
err := h.Service.AuthorizeClientSigning(r.Context(), publicKey)
if err != nil {
return err
}
staticServer.ServeHTTP(w, r)
return nil
}

Related

How to show progress during upload asynchronously with WASM

I am currently using Go WASM to upload a file to a server. During the upload it shall emit a call to update the upload progress in the UI.
I am currently using the following struct to have an indication of the progress:
type progressReporter struct {
r io.Reader
fileSizeEncrypted int64
sent int64
file js.Value
}
func (pr *progressReporter) Read(p []byte) (int, error) {
n, err := pr.r.Read(p)
pr.sent = pr.sent + int64(n)
pr.report()
return n, err
}
func (pr *progressReporter) report() {
go js.Global().Get("dropzoneObject").Call("emit", "uploadprogress", pr.file, pr.sent*100/pr.fileSizeEncrypted, pr.sent)
}
The upload happens in a promise:
func UploadChunk(this js.Value, args []js.Value) interface{} {
[...]
handler := js.FuncOf(func(this js.Value, args []js.Value) interface{} {
resolve := args[0]
reject := args[1]
go func() {
[...]
body := new(bytes.Buffer)
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile("file", "encrypted.file")
if err != nil {
return err
}
_, err = part.Write(*data)
if err != nil {
return err
}
err = writer.Close()
if err != nil {
return err
}
pReporter := progressReporter{
r: body,
fileSizeEncrypted: fileSize,
sent: offset,
file: jsFile,
}
r, err := http.NewRequest("POST", "./uploadChunk", &pReporter)
if err != nil {
return err
}
r.Header.Set("Content-Type", writer.FormDataContentType())
client := &http.Client{}
resp, err := client.Do(r)
if err != nil {
return err
}
[...]
}
}
Although the code works fine, all emit calls to update the UI are sent after the POST request is finished. Is there any way to have this call asynchronously?
The full source code can be found here

how to use the output of a bash script in a Golang function

This might not even be possible but I've been researching for the past hour and come up blank. I have a bash script that gives me a certain output and I want to add that output to Golang in order to redirect a website based on the output of the bash script. Sorry if this makes no sense, im new to Go
Here is what I have to run the bash script and output the value
func main() {
out, err := exec.Command("/bin/sh", "script.sh").Output()
if err != nil {
log.Fatal(err)
}
fmt.Printf(string(out))
}
I then want to use the value that was output there in another function and to redirect a URL heres how I would redirect to a url and I wanted the $variable to be added. This is just an example I copied off the internet but its what I want to replicate.
func redirect(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "**$variable**", 301)
}
func main() {
http.HandleFunc("/", redirect)
err := http.ListenAndServe(":8080", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
Assuming your script must be only run once at startup, then this should do the trick:
var redirectTo string
func redirect(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, redirectTo, 301)
}
func main() {
out, err := exec.Command("/bin/sh", "script.sh").Output()
if err != nil {
log.Fatal(err)
}
redirectTo = string(out)
http.HandleFunc("/", redirect)
err = http.ListenAndServe(":8080", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
Or if you don't want to have a global variable you can generate the redirect function at runtime:
func main() {
out, err := exec.Command("/bin/sh", "script.sh").Output()
if err != nil {
log.Fatal(err)
}
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, string(out), 301)
})
err = http.ListenAndServe(":8080", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
if you want to handle when out is empty, you can do:
func redirect(w http.ResponseWriter, r *http.Request) {
if len(redirectTo) == 0 {
http.Error(w, "No URL parsed!", 500)
return
}
http.Redirect(w, r, redirectTo, 301)
}
This will return an HTTP error 500 in that case.
You can also simply exit the program instead of serving:
out, err := exec.Command("/bin/sh", "script.sh").Output()
if err != nil {
log.Fatal(err)
}
if len(out) == 0 {
log.Fatal("no output was returned from script")
}
You can also add more verifications to out here if you wish, like if it is a correct URL, using net/url package for instance.

How to redirect multipart POST request to a second server in Golang?

I am trying to do the following.
|Upload file in HTML post file form|
|
⌄
|Server A forwards the multipart request|
|
⌄
|Server B receives and stores the file from the forwarded multipart request|
|
⌄
|Server A receives response from Server B when Server B is done|
Processing the multipart request on Server A is straightforward, but when I try to process the forwarded request on Server B it fails with multipart: NextPart: EOF.
I am trying to create separate frontend/backend services. Frontend only handles UI related processing, while backend will actually do some processing on the file, hence the multipart request forwarding needed.
The forwarding code on Server A is as follows.
The solution has been taken from here.
https://stackoverflow.com/a/34725635/6569715
func forwardRequest(address string, path string, r *http.Request) (interface{}, error) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
return nil, err
}
r.Body = ioutil.NopCloser(bytes.NewReader(body))
proxyReq, err := http.NewRequest(r.Method, fmt.Sprintf("%s%s", address, path), bytes.NewReader(body))
if err != nil {
return nil, err
}
for header, values := range r.Header {
for _, value := range values {
proxyReq.Header.Add(header, value)
}
}
client := &http.Client{}
resp, err := client.Do(proxyReq)
if err != nil {
return nil, err
}
defer resp.Body.Close()
return resp, nil
}
And the code on Server B to process the forwarded request:
func testMultiPart(w http.ResponseWriter, r *http.Request) {
if err := r.ParseMultipartForm(10 << 20); err != nil {
err = errors.Wrap(errors.WithStack(err), "Backend: Failed to parse form")
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprint(w, fmt.Sprintf("{\"error\":\"%s\"}", err.Error())
return
}
}
Any help is appreciated.
I managed to make it work. I believe it was just my own mistake not filling in the URI properly. In any case I will post my snippets from my solution for future reference.
The client html file form part:
<form action="/test-main/file-test" enctype="multipart/form-data" method="post">
<label for="file-upload">Upload your file :</label>
<input type="file" id="file-upload" name="file-upload" accept="image/*">
</form>
Server A code:
import (
"net/http"
"errors"
"fmt"
"log"
"io/ioutil"
"bytes"
"github.com/gorilla/mux"
)
func fileUpload(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
return log.Fatal(err)
}
r.Body = ioutil.NopCloser(bytes.NewReader(body))
// If Server A and B are separate docker images, you may need to use their docker subnet IP, like below.
proxyReq, err := http.NewRequest(r.Method, fmt.Sprintf("http://172.18.0.2:8082%s", r.RequestURI), bytes.NewReader(body))
if err != nil {
return log.Fatal(err)
}
for header, values := range r.Header {
for _, value := range values {
proxyReq.Header.Add(header, value)
}
}
client := &http.Client{}
resp, err := client.Do(proxyReq)
if err != nil {
return log.Fatal(err)
}
defer resp.Body.Close()
respBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
return log.Fatal(err)
}
// Process Server B response
// ...
}
func createRouter() *mux.Router {
r := mux.NewRouter()
testPath := r.PathPrefix("/test-main").Subrouter()
testPath.HandleFunc("/file-test", fileUpload)
return r
}
func main() {
// Create Server and Route Handlers
srv := &http.Server{
Handler: createRouter(),
Addr: ":8081",
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
}
// Start Server
go func() {
log.Println("Starting Server")
if err := srv.ListenAndServe(); err != nil {
log.Fatal(err)
}
}()
}
And Server B code:
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"github.com/gorilla/mux"
)
func uploadFile(w http.ResponseWriter, r *http.Request) {
if err := r.ParseMultipartForm(10 << 20); err != nil {
log.Fatal(err)
}
file, handler, err := r.FormFile("file-upload")
if err == http.ErrMissingFile {
return nil
}
if err != nil {
log.Fatal(err)
}
fmt.Printf("Uploaded File: %+v\n", handler.Filename)
fmt.Printf("File Size: %+v\n", handler.Size)
fmt.Printf("MIME Header: %+v\n", handler.Header)
defer file.Close()
// Create file
dst, err := os.Create(fmt.Sprintf("/some-destination-folder/%s", handler.Filename))
if err != nil {
log.Fatal(err)
}
// Copy the uploaded file to the created file on the file system.
if _, err := io.Copy(dst, file); err != nil {
if err2 := dst.Close(); err2 != nil {
log.Fatal(err)
}
log.Fatal(err)
}
dst.Close()
return nil
}
func (c *Controller) createRouter() *mux.Router {
r := mux.NewRouter()
testPath := r.PathPrefix("/test-main").Subrouter()
testPath.HandleFunc("/file-test", uploadFile)
return r
}
func main() {
// Create Server and Route Handlers
srv := &http.Server{
Handler: createRouter(),
Addr: ":8082",
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
}
// Start Server
go func() {
log.Println("Starting Server")
if err := srv.ListenAndServe(); err != nil {
log.Fatal(err)
}
}()
}
Good luck for future readers.

How to 'disable directory listing' and 'custom 404 page' handle same time

I'm using gorilla mux for routing in my http server
https://github.com/gorilla/mux
This is my code for disable directory listing
type justFilesFilesystem struct {
fs http.FileSystem
// readDirBatchSize - configuration parameter for Readdir func
readDirBatchSize int
}
func (fs justFilesFilesystem) Open(name string) (http.File, error) {
f, err := fs.fs.Open(name)
if err != nil {
return nil, err
}
return neuteredStatFile{File: f, readDirBatchSize: fs.readDirBatchSize}, nil
}
type neuteredStatFile struct {
http.File
readDirBatchSize int
}
func (e neuteredStatFile) Stat() (os.FileInfo, error) {
s, err := e.File.Stat()
if err != nil {
return nil, err
}
if s.IsDir() {
LOOP:
for {
fl, err := e.File.Readdir(e.readDirBatchSize)
switch err {
case io.EOF:
break LOOP
case nil:
for _, f := range fl {
if f.Name() == "" {
return s, err
}
}
default:
return nil, err
}
}
return nil, os.ErrNotExist
}
return s, err
}
and this is my main func
mux := mux.NewRouter()
// This line why not work?
mux.NotFoundHandler = http.HandlerFunc(NotFound)
mux.HandleFunc("/index",HandleIndex)
fs := justFilesFilesystem{fs: http.Dir("assets"), readDirBatchSize: 0}
staticFileHandler := http.StripPrefix("", http.FileServer(fs))
mux.PathPrefix("").Handler(staticFileHandler).Methods("GET")
and finally my 404 handle func
func NotFound(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "404")
}
The question is how to handle custom 404 page error if file not found and also disable directory listing in the same time.

Requesting multiple URLs in Go

I have the following Go program: https://play.golang.org/p/-TUtJ7DIhi
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strconv"
)
func main() {
body, err := get("https://hacker-news.firebaseio.com/v0/topstories.json")
if err != nil {
panic(err)
}
var ids [500]int
if err = json.Unmarshal(body, &ids); err != nil {
panic(err)
}
var contents []byte
for _, value := range ids[0:10] {
body, err := get("https://hacker-news.firebaseio.com/v0/item/" + strconv.Itoa(value) + ".json")
if err != nil {
fmt.Println(err)
} else {
contents = append(contents, body...)
}
}
fmt.Println(contents)
}
func get(url string) ([]byte, error) {
res, err := http.Get(url)
if err != nil {
return nil, err
}
body, err := ioutil.ReadAll(res.Body)
res.Body.Close()
return body, err
}
When run it throws EOF json errors on the iterative get requests, but when I hit the URLs individually they do not appear to be malformed.
What am I missing?
It looks like there's something wrong with their server, and it's closing connections without sending a Connection: close header. The client therefore tries to reuse the connection per the HTTP/1.1 specification.
You can work around this by creating your own request, and setting Close = true, or using a custom Transport with DisableKeepAlives = true
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Close = true
res, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}

Resources