I had trouble getting my React app to run on GitHub pages, so I chose to try and serve the files on my Go backend which is on Heroku. At first I was serving the React app through the main Go backend, which did serve the React app successfully but none of my other routes would work after that in the Go app, the routes needed for my React app to operate.
So I chose to create a new Heroku Go app and separate the backend and frontend on different Heroku Go apps. The frontend Go app is running fine, but the backend will intermittently work. I understand Heroku free apps go into a sleep state with a period of inactivity, but I am literally talking about the app being online for a few minutes, and then all of sudden just switching back to the default mode of saying "no such app"
Frontend Heroku Go app:
// Route: Delete Graph from database
func RouteDefault(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
http.ServeFile(w, r, "static/index.html")
}
func main() {
port := os.Getenv("PORT")
if port == "" {
port = "9000" // Default port if not specified
}
// HTTPRouter Settings and Routes
router := httprouter.New()
router.GET("/", RouteDefault)
// if not found look for a static file
static := httprouter.New()
static.ServeFiles("/*filepath", http.Dir("static"))
router.NotFound = static
handler := cors.AllowAll().Handler(router)
fmt.Println(http.ListenAndServe(":"+port, handler))
}
Backend Heroku Go app:
func main() {
// BasicAuth username and password
user := ""
pass := ""
port := os.Getenv("PORT")
if port == "" {
port = "9000" // Default port if not specified
}
DefaultUser()
// HTTPRouter Settings and Routes
router := httprouter.New()
router.POST("/login/", BasicAuth(RouteLogin, user, pass))
router.POST("/upload/", JWTAuth(RouteUpload))
router.POST("/graph/", JWTAuth(RouteGetGraph))
router.GET("/autologin/", JWTAuth(RouteAutoLogin))
handler := cors.AllowAll().Handler(router)
fmt.Println(http.ListenAndServe(":"+port, handler))
}
Frontend: https://grafulatordemo.herokuapp.com/
Backend: https://grafulator.herokuapp.com/
run the command: heroku apps -A and share the relevant output so it can be troubleshooted
The problem was on heroku's end, contacting their support team resolved it.
Related
I am trying to make simple load balancer using fiber in Go. In my computer it's working fine using http.
import (
"crypto/tls"
"log"
"os"
store "intraGo/stores/session_store"
"github.com/gofiber/fiber/v2"
"github.com/gofiber/fiber/v2/middleware/proxy"
"github.com/gofiber/template/html"
)
...
if os.Getenv("CERT_FILE") == "" || os.Getenv("KEY_FILE") == "" {
// this part is working fine
app.Use(proxy.Balancer(proxy.Config{Servers: []string{"http://localhost:9000"}}))
log.Fatal(app.Listen(":" + os.Getenv("SERVER_PORT")))
} else {
app.Use(proxy.Balancer(proxy.Config{Servers: []string{"https://servername.com"}}))
cer, err := tls.LoadX509KeyPair(os.Getenv("CERT_FILE"), os.Getenv("KEY_FILE"))
if err != nil {
log.Fatal(err)
}
config := &tls.Config{Certificates: []tls.Certificate{cer}}
ln, err := tls.Listen("tcp", ":"+os.Getenv("SERVER_PORT"), config)
if err != nil {
panic(err)
}
log.Fatal(app.Listener(ln))
}
In the production envirment using https it will give me a http 500 result with the follwing text for every request. The request will never reach the server.
HostClient can't follow redirects to a different protocol, please use Client instead
I checked Go / fiber documentation but did not find anything that could help to solve this issue. I also checked if the https part (like certificates, https server) is working fine by adding this to my app:
app.get("/", (req, res) => { res.send("Hello World!"); });
This worked as expected, so I guess the problem is in the loadbalancer itself.
Found this link while trying to find the solution on the net, I am not sure, but this might be related to the problem:
https://github.com/valyala/fasthttp/issues/841
Im using the following code which works as expected, I use from the cli gcloud auth application-default login and enter my credentials and I was able to run the code successfully from my macbook.
Now I need to run this code in my CI and we need to use different approach , what should be the approach to get the client_secret
and client_id or service account / some ENV variable, what is the way for doing it via GO code?
import "google.golang.org/api/compute/v1"
project := "my-project"
region := "my-region"
ctx := context.Background()
c, err := google.DefaultClient(ctx, compute.CloudPlatformScope)
if err != nil {
log.Fatal(err)
}
computeService, err := compute.New(c)
if err != nil {
log.Fatal(err)
}
req := computeService.Routers.List(project, region)
if err := req.Pages(ctx, func(page *compute.RouterList) error {
for _, router := range page.Items {
// process each `router` resource:
fmt.Printf("%#v\n", router)
// NAT Gateways are found in router.nats
}
return nil
}); err != nil {
log.Fatal(err)
}
Since you're using Jenkins you probably want to start with how to create a service account. It guides you on creating a service account and exporting a key to be set as a var in another CI/CD system.
Then refer to the docs from the client library on how to create a new client with source credential.
e.g.
client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))
If you provided no source, it would attempt to read the credentials locally and act as the service account running the operation (not applicable in your use case).
Many CIs support the export of specific env vars. Or your script / conf can do it too.
But if you want to run in a CI why you need such configuration? Integration tests?
Some services can be used locally for unit/smoke testing. Like pubsub, there is a way to run a fake/local pubsub to perform some tests.
Or perhaps I did not understand your question, in this case can you provide an example?
I'd like to create Signed URLs to Google Cloud Storage resources from an app deployed using CloudRun.
I set up CloudRun with a custom Service Account with the GCS role following this guide.
My intent was to use V4 Signing to create Signed URLs from CloudRun. There is a guide for this use-case where a file service_account.json is used to generate JWT config. This works for me on localhost when I download the file from google's IAM. I'd like to avoid having this file committed in the repository use the one that I provided in CloudRun UI.
I was hoping that CloudRun injects this service account file to the app container and makes it accessible in GOOGLE_APPLICATION_CREDENTIALS variable but that's not the case.
Do you have a recommendation on how to do this? Thank you.
As you say, Golang Storage Client Libraries require a service account json file to sign urls.
There is currently a feature request open in GitHub for this but you should be able to work this around with this sample that I found here:
import (
"context"
"fmt"
"time"
"cloud.google.com/go/storage"
"cloud.google.com/go/iam/credentials/apiv1"
credentialspb "google.golang.org/genproto/googleapis/iam/credentials/v1"
)
const (
bucketName = "bucket-name"
objectName = "object"
serviceAccount = "[PROJECTNUMBER]-compute#developer.gserviceaccount.com"
)
func main() {
ctx := context.Background()
c, err := credentials.NewIamCredentialsClient(ctx)
if err != nil {
panic(err)
}
opts := &storage.SignedURLOptions{
Method: "GET",
GoogleAccessID: serviceAccount,
SignBytes: func(b []byte) ([]byte, error) {
req := &credentialspb.SignBlobRequest{
Payload: b,
Name: serviceAccount,
}
resp, err := c.SignBlob(ctx, req)
if err != nil {
panic(err)
}
return resp.SignedBlob, err
},
Expires: time.Now().Add(15*time.Minute),
}
u, err := storage.SignedURL(bucketName, objectName, opts)
if err != nil {
panic(err)
}
fmt.Printf("\"%v\"", u)
}
Cloud Run (and other compute platforms) does not inject a service account key file. Instead, they make access_tokens available on the instance metadata service. You can then exchange this access token with a JWT.
However, often times, Google’s client libraries and gcloud works out of the box on GCP’s compute platforms without explicitly needing to authenticate. So if you use the instructions on the page you linked (gcloud or code samples) it should be working out-of-the-box.
I have a Go app that use Gin gonic and a Nginx reverse proxy that send trafic to another app on domain.com and send all the *.domain.com subdomains traffic directly to my go app.
My Go app then has a middleware that will read the hostname that nginx passes to it from Context and allow my handlers to know what subdomain is being request and return the proper data and cookies for said subdomain.
It's a pretty simple setup and it seems to work fine from my test in postman as all my routes are the same across all my subdomains so this way i can only use one router for all of them instead of one router per subodmain.
Now my big problem come when i'm trying to do end to end testing.
I'm setting up my test like this :
router := initRouter()
w := httptest.NewRecorder()
req, _ := http.NewRequest("POST", "/api/login", bytes.NewBuffer(jsonLogin))
req.Header.Set("Content-Type", "application/json")
router.ServeHTTP(w, req)
assert.Equal(t, 200, w.Code)
with initRouter() returning a gin engine with all my routes and middlewares loaded and the rest as a basic test setup.
Obviously the test will fail as the gin Context won't ever receive a subdomain from context and act as if everything is coming from localhost:8000.
Is there a way to either :
"Mock" a subdomain so that the router think the call is coming from foo.localhost.com instead of localhost
Setup my test suit so that the test request are routed thought nginx.. i'd prefer solution 1 as this would be a mess to setup / maintain.
Edit :
As per the httptest doc i've tried to hard code foo.localhost as the param of the NewRequest but it doesn't really behave as i need it to behave :
NewRequest returns a new incoming server Request, suitable for passing to an http.Handler for testing.
The target is the RFC 7230 "request-target": it may be either a path or an absolute URL. If target is an absolute URL, the host name from the URL is used. Otherwise, "example.com" is used.
When hardcoding http://foo.localhost.com/api/login or foo.localhost.com/api/login as the request target it directly passes it to my router under "foo.localhost.com/api/login" while nginx would just hit the /api/login directly and parse from c.Request.Host
Edit 2:
I'm currently exploring setting the host manually using :
req.Header.Set("Host", "foo.localhost")
The request returned by http.NewRequest isn't suitable for passing directly to ServeHTTP. Use one returned by httptest.NewRequest instead.
Simply set the Host field directly:
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestHelloWorld(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.Host != "foobar" {
t.Errorf("Host is %q, want foobar", r.Host)
}
})
w := httptest.NewRecorder()
r := httptest.NewRequest("GET", "/api/login", nil)
r.Host = "foobar"
mux.ServeHTTP(w, r)
}
My Web Server is Coded in Golang and supports HTTPS. I wish to leverage HTTP/2 Server Push features in the Web Server. The following Link explains how to convert HTTP Server to Support HTTP/2 :-
https://www.ianlewis.org/en/http2-and-go
However, it is not clear how to implement the Server Push notifications in Golang.
- How should I add the Server Push functionality ?
- How do I control, or manage, the documents and files to be Pushed ?
Go 1.7 and older do not support HTTP/2 server push in the standard library. Support for server push will be added in the upcoming 1.8 release (see the release notes, expected release is February).
With Go 1.8 you can use the new http.Pusher interface, which is implemented by net/http's default ResponseWriter. Pushers Push method returns ErrNotSupported, if server push is not supported (HTTP/1) or not allowed (the client has disabled server push).
Example:
package main
import (
"io"
"log"
"net/http"
)
func main() {
http.HandleFunc("/pushed", func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "hello server push")
})
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if pusher, ok := w.(http.Pusher); ok {
if err := pusher.Push("/pushed", nil); err != nil {
log.Println("push failed")
}
}
io.WriteString(w, "hello world")
})
http.ListenAndServeTLS(":443", "server.crt", "server.key", nil)
}
If you want to use server push with Go 1.7 or older use can use the golang.org/x/net/http2 and write the frames directly.
As mentioned in other answers, you can make use of Go 1.8 feature (cast the writer to http.Pusher and then use the Push method).
That comes with a caveat: you must be serving the HTTP2 traffic right from your server.
If you're behind a proxy like NGINX, this might not work. If you want to consider that scenario, you can make use of the Link header to advertise the URLs to be pushed.
// In the case of HTTP1.1 we make use of the `Link` header
// to indicate that the client (in our case, NGINX) should
// retrieve a certain URL.
//
// See more at https://www.w3.org/TR/preload/#server-push-http-2.
func handleIndex(w http.ResponseWriter, r *http.Request) {
var err error
if *http2 {
pusher, ok := w.(http.Pusher)
if ok {
must(pusher.Push("/image.svg", nil))
}
} else {
// This ends up taking the effect of a server push
// when interacting directly with NGINX.
w.Header().Add("Link",
"</image.svg>; rel=preload; as=image")
}
w.Header().Add("Content-Type", "text/html")
_, err = w.Write(assets.Index)
must(err)
}
ps.: I wrote more about this here https://ops.tips/blog/nginx-http2-server-push/ if you're interested.