Local subdomains for a standalone application - go

I have a website, which is composed by three smaller 'independent' subsites:
mysite
index.html
favicons
images
doc
index.html
css
img
js
...
editor
index.html
images
js
src
...
Where doc is a site created with Hugo :: A fast and modern static website engine, editor is the mxgraph Graphditor example; and the remaining files make a hand-made landing page.
Besides deploying to any web server, I'd like to distribute the site as an 'standalone application'. To allow so, I wrote this really simple server in go:
package main
import (
flag "github.com/ogier/pflag"
"fmt"
"net/http"
"net/http/httputil"
"os"
"path/filepath"
)
var port = flag.IntP("port", "p", 80, "port to serve at")
var dir = flag.StringP("dir", "d", "./", "dir to serve from")
var verb = flag.BoolP("verbose", "v", false, "")
func init() {
flag.Parse();
}
type justFilesFilesystem struct {
fs http.FileSystem;
}
type neuteredReaddirFile struct {
http.File
}
func (fs justFilesFilesystem) Open(name string) (http.File, error) {
f, err := fs.fs.Open(name)
if err != nil { return nil, err; }
return neuteredReaddirFile{f}, nil
}
func (f neuteredReaddirFile) Readdir(count int) ([]os.FileInfo, error) {
return nil, nil;
}
func loggingHandler(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
requestDump, err := httputil.DumpRequest(r, true)
if err != nil { fmt.Println(err); }
fmt.Println(string(requestDump))
h.ServeHTTP(w, r)
})
}
func main() {
str, err := filepath.Abs(*dir)
if err != nil { os.Exit(1); }
fmt.Printf("Serving at port %d from dir %s\n\n",*port,str)
http.ListenAndServe(fmt.Sprintf(":%d",*port), loggingHandler(http.FileServer(justFilesFilesystem{http.Dir(*dir)})))
}
As a result, I can run simpleserver -d <path-to-mysite> and browse the sites through localhost, localhost/doc and localhost/editor.
Then, I'd like to use custom (sub)domain(s) such as mylocal.app, doc.mylocal.app and editor.mylocal.app. So, I added the following line to my the /etc/hosts file: 127.0.0.1 mylocal.app. Therefore, I can browse mylocal.app, mylocal.app/editor and mylocal.app/doc. Moreover, I was able to change it to mylocal.app, mylocal.app:<editor-port> and mylocal.app:<doc-port> with different packages.
However, when I try to use a subdomain, it is not properly resolved, so any reverse-proxy strategy won't work. Since wildcards are not supported, I can add additional entries in the /etc/hosts file, but I'd prefer to avoid it.
Although, an alternative solution is to run dnsmasq, I'd like to keep the application standalone. I found some equivalent golang packages. However, I feel that many features are supported which I don't really need.
Furthermore, since I don't really have to resolve any IP, but to provide an alias of localhost, I think that a proxy could suffice. This would also be easier to configure, since the user could configure the browser only, and no system-wide modification would be required.
Yet, all the traffic from the user would be 'filtered' by my app. Is this correct? If so, can you point me to any reference to implement it in the most clean way. I know this is quite subjective, but I mean a relatively short (say 10 lines of code) snippet so that users can easily check what is going on.
EDIT
I'd like to use something like:
func main() {
mymux := http.NewServeMux()
mymux.HandleFunc("*.mylocal.app", myHandler)
mymux.HandleFunc("*", <systemDefaultHandler>)
http.ListenAndServe(":8080", mymux)
}
or
func main() {
mymux := http.NewServeMux()
mymux.HandleFunc("editor.mylocal.app", editorHandler)
mymux.HandleFunc("doc.mylocal.app", docHandler)
mymux.HandleFunc("*.mylocal.app", rootHandler)
mymux.HandleFunc("*", <systemDefaultHandler>)
http.ListenAndServe(":8080", mymux)
}
These are only snippets. A complete example is this, which was referenced in the comments by #Steve101.
However, at now, I don't know what systemDefaultHandler is. And that is not solved there.
Apart from that, #faraz suggested using goproxy. I think that the HTTP/HTTPS transparent proxy is the default handler I am looking for. But, using a package only to do that seems excessive to me. Can I achieve the same functionality with built-in resources?

Unfortunately, there's no dead simple way to do this through Go. You'll need to intercept your system's DNS requests just like dnsmasq, and that's inevitably going to require some modification the system DNS config (/etc/resolv.conf in Linux, /etc/resolver on a Mac, or firewall rule) to route your DNS requests to your app. Going the DNS has the downside that you'd need to build a DNS server inside your app similar to pow.cx, which seems unnecessarily complicated.
Since mucking with system config is inevitable, I'd vote for making changes to the hosts file on boot or when a directory is added/removed (via fsnotify.) On shutdown, you can clear the added entries too.
If you're looking to isolate these changes to a specific browser instead of make a system-wide change, you could always run your application through a proxy server like goproxy and tell your browser to use that proxy for requests. For example, you can do this in Chrome through its preferences or by setting the --proxy-server flag:
--proxy-server=<scheme>=<uri>[:<port>][;...] | <uri>[:<port>] | "direct://"
See Chrome's network settings docs for more details.
Also, if you're willing to much with browser configs, you could just use an extension to handle the requests as needed.

there is only solution that would work without proxy is that you register an domain for this and make an offical dns entry with wildcard to 127.0.0.1 like *.myApp.suche.org. So any client that resolve the ip get 127.0.0.1 and work local. this way you do not need administrator rights to modify the /etc/hosts or equivalent.
If it should work behind proxy without modify the resolver (etc/hosts etc) you can provide an wpad.dat (Javascript for Proxy detection) that say for your domain all traffic goes you server and the rest to the real proxy. If this is served in you server the script can automaticaly contain the real proxy as default.

Related

Empty request body and missing headers with custom handler

I have an Azure Function which uses a custom handler written in Go. It was all working fine until Friday and now the requests appear to be turning up with empty request bodies and missing headers? Has anyone else experienced this?
The Go handler is really simple...
func main() {
httpInvokerPort, exists := os.LookupEnv("FUNCTIONS_HTTPWORKER_PORT")
if exists {
log.Printf("FUNCTIONS_HTTPWORKER_PORT: %s\n", httpInvokerPort)
}
mux := http.NewServeMux()
mux.HandleFunc("/sign", httpTriggerHandler)
log.Println("Go server Listening...on httpInvokerPort:", httpInvokerPort)
log.Fatal(http.ListenAndServe(":"+httpInvokerPort, mux))
}
And the handler function:
func httpTriggerHandler(w http.ResponseWriter, r *http.Request) {
spew.Dump(r)
// other app logic
}
Since late last week, the spew shows an empty request body (where there should be JSON), and a few missing headers. I've not changed anything in the code, and can't see any major changes to the Azure Functions service.
I've done some troubleshooting, including MITM'ing myself outbound to ensure that the request is leaving my machine well-formed and it all seems fine. I've also tried redeploying to other Azure regions.
This is now resolved: https://github.com/Azure/azure-functions-host/issues/6444. There was a bug in the Azure Functions Host which has been fixed.

static files not served correctly when using wildcard

Using GOA, I defined a service to serve static files using a wildcard (as described in the documentation):
var _ = Service("static", func() {
Files("/static/*filepath", "./static/")
})
But when I run the service, the endpoint always retrieves all the content it finds in the ./static/ directory, it seems it doesn't take into account the wildcard section at all.
For example, if I have ./static/uploads/file1.jpg and I request localhost/static/uploads/file1.jpg or localhost/static/anything , then the service retrieves the following:
<pre>
uploads/
</pre>
Digging into the code, I believe the problem is in the generated /gen/http/static/server/server.go file:
// Mount configures the mux to serve the static endpoints.
func Mount(mux goahttp.Muxer, h *Server) {
MountCORSHandler(mux, h.CORS)
MountStatic(mux, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, "./static/")
}))
}
// MountStatic configures the mux to serve GET request made to
// "/static/*filepath".
func MountStatic(mux goahttp.Muxer, h http.Handler) {
mux.Handle("GET", "/static/*filepath", handleStaticOrigin(h).ServeHTTP)
}
For what I see, the generated code is serving what we passed as base path no matter what, it doesn't take into account if we configured a wildcard at all (it only uses it to match the request, but not to customize the file that we'll serve).
I believe this was working ok in v2, I found this issue in the process of migrating to v3.
As I said, this seems like a bug in GOA, but maybe I'm missing something here. I created an issue in the repo to obtain more information (#2321)
As per the answer in the Github issue (#2321), it seems there was an error in the docs, and we should use curly braces in the pattern:
Thank you for the report, there is a typo in the docs, the path in the design needs to be /static/{*filepath} instead (with curly braces surrounding the wildcard).

How do I set a DNS cache for the net package?

In the "net/http" package I can cache the DNS lookups by:
client := &http.Client{
Transport: &http.Transport{
Dial: (&nett.Dialer{
Resolver: &nett.CacheResolver{TTL: 5 * time.Minute},
IPFilter: nett.DualStack,
}).Dial,
},
}
then use client to retrieve websites. How do I cache the DNS lookups for the net package? for instance, a reverse DNS request:
net.LookupAddr(ip)
Since this does not use a variable I am confused as to how to get it setup and how to even know if I am using a cached instance.
The nett package seems to just have a single "Resolve" method rather than LookupAddr, LookupIP etc etc that the official net package has. So reverse lookups don't seem to be available. Here's how to do a normal lookup of address from name
package main
import (
"github.com/abursavich/nett"
"time"
)
func main() {
r := nett.CacheResolver{TTL: 5 * time.Minute}
a, _ := r.Resolve("muppet.com")
for _, i := range a {
print(i.String())
}
}
It would seem the behavior of the net.LookupAddr is to use the host resolver. The way I do this for my services in prod is to run dnsmasq on the hosts so the DNS lookups are cached per host. The docs do mention you can customize the behavior by implementing a custom Resolver as you did: https://golang.org/pkg/net/#LookupAddr
But I think the piece you're looking for is (from the top of that doc page):
DefaultResolver is the resolver used by the package-level Lookup functions and by Dialers without a specified Resolver.

ask LookupTXT function in golang

How do I change the IP address of the DNS server?
In situation, I set Google DNS server in Windows Network Settins.
And I use LookupTXT function in Golang for getting DNS txt request.
But LookupTXT parameter is just the query string.
Any help or pointers would be highly appreciated. Thanks!
This is not straigtforward to do using golang at this point. You can however use a third party DNS package that allows configuring the resolver. First install the package:
go get github.com/bogdanovich/dns_resolver
Here is an example using it and the google resolvers 8.8.8.8 and 8.8.4.4:
package main
import (
"log"
"github.com/bogdanovich/dns_resolver"
)
func main() {
resolver := dns_resolver.New([]string{"8.8.8.8", "8.8.4.4"})
// In case of i/o timeout
resolver.RetryTimes = 5
ip, err := resolver.LookupHost("google.com")
if err != nil {
log.Fatal(err.Error())
}
log.Println(ip)
// Output [216.58.192.46]
}
Source
There is an open issue in golang here, so hopefully it becomes easier to do it with the builtin net package: https://github.com/golang/go/issues/12503. It could just be a documentation problem, as it is possible now, I just can't find an example.
EDIT: actually that package only supports lookupHost: https://github.com/bogdanovich/dns_resolver/blob/master/dns_resolver.go#L51-L79
So a PR would be required to add a TXT resolver.
2nd Edit: I made a PR with txt lookup here. That project hasn't been touched in years though so it may never get accepted.

Gorilla mux routes in separate files in subfolder?

I'm trying to build a very simple Go web application, and the golang "a folder per package" structure is making things difficult for me.
I'm using github.com/gorilla/mux as the router and github.com/unrolled/render for template rendering. This means that I need to create a new router and a new renderer when the app launches, and I need all my routes to access the renderer.
This is super easy to do in a single file:
func main() {
...
r := render.New(render.Options{
// a lot of app specific setup
})
mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
r.HTML(w, http.StatusOK, "myTemplate", nil)
})
...
}
However, this is where I don't understand Go. Because I want the routes in separate files in a subfolder (my project will grow), that forces them to be in a routes package. Of course that makes the renderer variable inaccesssible. I can't just create the renderer in the routes package, because the render.New() call relies on a me passing in a ton of app specific stuff like the template folder, and helpers for asset paths.
I went down the route of making my handler functions work on a struct with an already initialized renderer...
func (app *App) Hello2(w http.ResponseWriter, r *http.Request) {
app.Renderer.HTML(w, http.StatusOK, "myTemplate", nil)
}
But I'm still confused as to how I'm going to access this app *App in the routes package when it's initialized in main. Everything in Go seems super easy if you have a flat list of files, but as soon as you want a bit of folder structure, the package setup becomes problematic.
There's probably something I'm missing here, so any help is appreciated.
Here's general info on dealing with dependencies in Go. A key trick is, you just have to declare the Render variable in a package that your views can import. You could create a myapp/render package with a var Render that's either inited in the package itself (func init()) or set from main.
But the context thing you found sounds totally sane, though it might be more than this app needs. The neat thing about it is that because the context is set in per-request code, later you could extend it to do sneaky things like use the Host: header to provide a different Layout for people loading the app via different domains. If the Layout is baked into a global, you can't. This can be a real advantage--I've tried to retrofit per-request changes onto big codebases whose config was sprayed around various global variables, and it's a pain.

Resources