How to generate OpenAPI v3 specification from Go source code? - go

Is there a way to generate OpenAPI v3 specification from go source code? Let's say I have a go
API like the one below and I'd like to generate the OpenAPI specification (yaml file) from it. Something similar to Python's Flask RESTX. I know there are tools that generate go source code from the specs, however, I'd like to do it the other way around.
package main
import "net/http"
func main() {
http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("world\n"))
})
http.ListenAndServe(":5050", nil)
}

You can employ github.com/swaggest/rest to build a self-documenting HTTP REST API. This library establishes a convention to declare handlers in a way that can be used to reflect documentation and schema and maintain a single source of truth about it.
In my personal opinion code first approach has advantages comparing to spec first approach. It can lower the entry bar by not requiring to be an expert in spec language syntax. And it may help to come up with a spec that is well balanced with implementation details.
With code first approach it is not necessary to implement a full service to get the spec. You only need to define the structures and interfaces and may postpone actual logic implementation.
Please check a brief usage example.
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"time"
"github.com/go-chi/chi"
"github.com/go-chi/chi/middleware"
"github.com/swaggest/rest"
"github.com/swaggest/rest/chirouter"
"github.com/swaggest/rest/jsonschema"
"github.com/swaggest/rest/nethttp"
"github.com/swaggest/rest/openapi"
"github.com/swaggest/rest/request"
"github.com/swaggest/rest/response"
"github.com/swaggest/rest/response/gzip"
"github.com/swaggest/swgui/v3cdn"
"github.com/swaggest/usecase"
"github.com/swaggest/usecase/status"
)
func main() {
// Init API documentation schema.
apiSchema := &openapi.Collector{}
apiSchema.Reflector().SpecEns().Info.Title = "Basic Example"
apiSchema.Reflector().SpecEns().Info.WithDescription("This app showcases a trivial REST API.")
apiSchema.Reflector().SpecEns().Info.Version = "v1.2.3"
// Setup request decoder and validator.
validatorFactory := jsonschema.NewFactory(apiSchema, apiSchema)
decoderFactory := request.NewDecoderFactory()
decoderFactory.ApplyDefaults = true
decoderFactory.SetDecoderFunc(rest.ParamInPath, chirouter.PathToURLValues)
// Create router.
r := chirouter.NewWrapper(chi.NewRouter())
// Setup middlewares.
r.Use(
middleware.Recoverer, // Panic recovery.
nethttp.OpenAPIMiddleware(apiSchema), // Documentation collector.
request.DecoderMiddleware(decoderFactory), // Request decoder setup.
request.ValidatorMiddleware(validatorFactory), // Request validator setup.
response.EncoderMiddleware, // Response encoder setup.
gzip.Middleware, // Response compression with support for direct gzip pass through.
)
// Create use case interactor.
u := usecase.IOInteractor{}
// Describe use case interactor.
u.SetTitle("Greeter")
u.SetDescription("Greeter greets you.")
// Declare input port type.
type helloInput struct {
Locale string `query:"locale" default:"en-US" pattern:"^[a-z]{2}-[A-Z]{2}$" enum:"ru-RU,en-US"`
Name string `path:"name" minLength:"3"` // Field tags define parameter location and JSON schema constraints.
}
u.Input = new(helloInput)
// Declare output port type.
type helloOutput struct {
Now time.Time `header:"X-Now" json:"-"`
Message string `json:"message"`
}
u.Output = new(helloOutput)
u.SetExpectedErrors(status.InvalidArgument)
messages := map[string]string{
"en-US": "Hello, %s!",
"ru-RU": "Привет, %s!",
}
u.Interactor = usecase.Interact(func(ctx context.Context, input, output interface{}) error {
var (
in = input.(*helloInput)
out = output.(*helloOutput)
)
msg, available := messages[in.Locale]
if !available {
return status.Wrap(errors.New("unknown locale"), status.InvalidArgument)
}
out.Message = fmt.Sprintf(msg, in.Name)
out.Now = time.Now()
return nil
})
// Add use case handler to router.
r.Method(http.MethodGet, "/hello/{name}", nethttp.NewHandler(u))
// Swagger UI endpoint at /docs.
r.Method(http.MethodGet, "/docs/openapi.json", apiSchema)
r.Mount("/docs", v3cdn.NewHandler(apiSchema.Reflector().Spec.Info.Title,
"/docs/openapi.json", "/docs"))
// Start server.
log.Println("http://localhost:8011/docs")
if err := http.ListenAndServe(":8011", r); err != nil {
log.Fatal(err)
}
}

Related

Dependency Injection of non-concurrency safe dependencies for a net/HTTP server

I am playing around with Dependency Injection and HTTP servers in Golang, and are trying to wrap my head around how to make available a logger that contains information related to the current request that is being handled. What I want to be able to do is to call a logging function inside the current http.Handler and functions/methods that is being called inside from the http.Handler. Essentially leave a request ID in every log event, so I can trace exactly what happened.
My idea was to use Dependency Injection and define all the logic as methods on interfaces that would have the logger injected. I like this approach, since it makes the dependencies painstakingly clear.
I've seen other approaches, such as Zerolog's hlog package that puts the logger into the requests context.Context, and I could then modify my code to receive a context.Context as an argument, from where I could pull out the logger and call it. While this will work, I don't like it very much, in that it kinda hides away the dependency.
My issue is that I can't seem to come up with a way of using Dependency Injection in conjunction with a HTTP handling server, where the injected dependencies are concurrency safe.
As an example, I whipped up this code (ignore the fact that it doesn't use interfaces)
package main
import (
"crypto/rand"
"fmt"
mrand "math/rand"
"net/http"
"time"
)
func main() {
l := LoggingService{}
ss := SomethingService{
LoggingService: &l,
}
handler := Handler{
SomethingService: &ss,
LoggingService: &l,
}
http.Handle("/", &handler)
http.ListenAndServe("localhost:3333", nil)
}
type Handler struct {
SomethingService *SomethingService
LoggingService *LoggingService
}
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
reqID := generateRequestID()
h.LoggingService.AddRequestID(reqID)
h.LoggingService.Log("A request")
out := h.SomethingService.DoSomething("nothing")
w.Write([]byte(out))
}
func generateRequestID() string {
buf := make([]byte, 10)
rand.Read(buf)
return fmt.Sprintf("%x", buf)
}
type SomethingService struct {
LoggingService *LoggingService
}
func (s *SomethingService) DoSomething(input string) string {
s.LoggingService.Log(fmt.Sprintf("Let's do something with %s", input))
mrand.Seed(time.Now().UnixNano())
n := mrand.Intn(3)
s.LoggingService.Log(fmt.Sprintf("Sleeping %d seconds...", n))
time.Sleep(time.Duration(n) * time.Second)
s.LoggingService.Log("Done")
return "something"
}
type LoggingService struct {
RequestID string
}
func (l *LoggingService) AddRequestID(id string) {
l.RequestID = id
}
func (l *LoggingService) Log(msg string) {
if l.RequestID != "" {
fmt.Printf("[%s] %s\n", l.RequestID, msg)
} else {
fmt.Printf("%s\n", msg)
}
}
The way that I've understood Go's HTTP server is that the handler is invoked, for each request, in it's own goroutine. Because the LoggingService is passed as a pointer, each of these goroutines are essentially accessing the same instance of LoggingService, which can result in a race condition where one request sets l.RequestID after which another request logs its requests, but ends up with the request ID from the first request.
I am sure others have run into a need to log something in a function that is called by a HTTP request and to be able to trace that log event to that specific request? However, I haven't found any examples where people are doing this.
I was thinking of delaying the instantiation of the dependencies until inside the handler. While that works, it has the side-effect of creating many instances of the dependencies - This is fine for the LoggingService, but something like the SomethingService could be talking to a DB, where it would be a waste to create a database client for every request (since most DB implementations have concurrency safe clients).

Implementing a feedparser client for the bud framework in golang

I am trying over the Bud full-stack framework and I wanted to implement a simple RSS parser to get data at each request.
I have been following pretty closely the tutorial made by Bud's creator, Matt Mueller https://www.youtube.com/watch?v=LoypcRqn-xA&t=601s and then I modified some of the code of the client he uses in the video to get hackernews json. I just want to be able to call for the data at each request and print it to the screen. However, While it compiles, I encounter an error at runtime
conjure: generate "bud/.app/controller/controller.go" > controller: unable to load definition for field Extensions in sto
ries > parser: unable to find declaration for "Extensions" in "ext"
|
I have tried using feed instead of stories as an object name, creating a struct with a pointer to gofeed.Feed but nothing worked. I hypothize this has something to do with the way I use gofeed parser but it seems I can't manage to find the issue. Do you have any suggestion on what I am doing wrong?
This is my client.go file
// Package feedparser is a simple RSS reader client that leverages gofeed
package feedparser
import (
"context"
"fmt"
"github.com/mmcdole/gofeed"
)
const baseURL = `https://ec.europa.eu/info/news/feed_en`
// Feed Client, it uses gofeed parser to open a connection with the rss/atom/json data feed
func New() *Client {
feedParser := gofeed.NewParser()
return &Client{feedParser}
}
// Feed Client
type Client struct {
*gofeed.Parser
}
// FrontPage gets the feeds and prints them as a list
func (c *Client) FrontPage(ctx context.Context) (*gofeed.Feed, error) {
feedData := `<rss version="2.0">
<channel>
<webMaster>example#site.com (Example Name)</webMaster>
</channel>
</rss>`
fd, err := c.RSSTranslator.Translate(feedData)
if err != nil {
return fd, err
}
fmt.Println(fd)
return fd, nil
}
This is my controller.go file
package controller
import (
context "context"
"github.com/AlessioNar/eunews/feedparser"
"github.com/mmcdole/gofeed"
)
type Controller struct {
// Dependencies...
FP *feedparser.Client
}
// Index of stories
// GET /
func (c *Controller) Index(ctx context.Context) (stories *gofeed.Feed, err error) {
return c.FP.FrontPage(ctx)
}
This is the original repository holding the hackernews client
https://github.com/matthewmueller/hackernews

Record and Persist API call details in KrakenD for API monetization

We are trying out KrakenD as a primary API gateway for our backend services. The plugins available are only for response manipulation, we want to go a step ahead and start recording all the API calls and persist them in a database. I was checking the example of having a custom HTTPStatusHandler, but that mainly caters to handling the status codes and as a best case we can only return the status codes from our backends as it is. Is there any working example of this scenario that we are trying? We are already using a custom middleware to grant API access based on our role based access rules.
Other commercial solutions offer API monetization out of the box, but we want to stick with KrakenD for performance reasons.
KrakenD plugins are not only for response manipulation, but a much more powerful solution. There are 4 types of plugins in krakend, what you want to do I think that fits best as an HTTP SERVER plugin.
This type of plugin intercepts all incoming traffic and you can record it in a Redis database.
I have improvised a main.go of the idea, but you should check the plugin guide in the documentation for more details.
Hope this helps illustrating the idea
package main
import (
"context"
"errors"
"fmt"
"html"
"net/http"
)
// HandlerRegisterer is the symbol the plugin loader will try to load. It must implement the Registerer interface
var HandlerRegisterer = registerer("krakend-api-monetization")
type registerer string
var logger Logger = nil
func (registerer) RegisterLogger(v interface{}) {
l, ok := v.(Logger)
if !ok {
return
}
logger = l
logger.Debug(fmt.Sprintf("[PLUGIN: %s] Logger loaded", HandlerRegisterer))
}
func (r registerer) RegisterHandlers(f func(
name string,
handler func(context.Context, map[string]interface{}, http.Handler) (http.Handler, error),
)) {
f(string(r), r.registerHandlers)
}
func (r registerer) registerHandlers(_ context.Context, extra map[string]interface{}, h http.Handler) (http.Handler, error) {
// check the passed configuration and initialize the plugin
name, ok := extra["name"].([]interface{})
if !ok {
return nil, errors.New("wrong config")
}
if v, ok := name[0].(string); !ok || v != string(r) {
return nil, fmt.Errorf("unknown register %s", name)
}
// check the cfg. If the modifier requires some configuration,
// it should be under the name of the plugin. E.g.:
/*
"extra_config":{
"plugin/http-server":{
"name":["krakend-api-monetization"],
"krakend-api-monetization":{
"redis_host": "localhost:6379"
}
}
}
*/
// The config variable contains all the keys you hace defined in the configuration:
config, _ := extra["krakend-api-monetization"].(map[string]interface{})
// The plugin will save activity in this host:
redis_host, _ := config["redis_host"].(string)
logger.Debug(fmt.Sprintf("The plugin is now storing on %s", redis_host))
// return the actual handler wrapping or your custom logic so it can be used as a replacement for the default http handler
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
// TODO: Save in Redis the request
rdb := redis.NewClient(&redis.Options{
Addr: redis_host,
Password: "", // no password set
DB: 0, // use default DB
})
// etc...
logger.Debug("Request saved:", html.EscapeString(req.URL.Path))
}), nil
}
func main() {}
type Logger interface {
Debug(v ...interface{})
Info(v ...interface{})
Warning(v ...interface{})
Error(v ...interface{})
Critical(v ...interface{})
Fatal(v ...interface{})
}

How to mock many urls to return fixture content?

I'm writing some kind of a recursive parser. The simplest form is:
Take all links from first link's page body
Repeat the first step for each link
So now I want to test it. The problem is I can't figure out the best way to mock all these pages. I use http package and I already have some tests written using httptest package (via httptest.NewServer). But it seems to be no use for my task now. I guess the best way is to use http.Client with custom Transport struct, but it's lots of boilerplate and additional smelly code. Is there a more elegant way to do this?
I have used a custom Transport a couple of times for testing clients. Usually I would create some helper types and functions to cut down on boilerplate code.
Something like this could be a start.
package main
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (r roundTripFunc) RoundTrip(req *http.Request) (resp *http.Response, err error) {
return r(req)
}
func main() {
c := &http.Client{
Transport: roundTripFunc(func(req *http.Request) (resp *http.Response, err error) {
return &http.Response{
StatusCode: 200,
Body: ioutil.NopCloser(bytes.NewBufferString("test")),
}, nil
}),
}
r, _ := c.Get("/")
fmt.Printf("%#v\n", r)
io.Copy(os.Stdout, r.Body)
}
For example if your testing a JSON API client you could make a round trip helper function takes care of decoding, encoding, headers etc. In your case maybe you could make the round trip function map host header plus URL path into file fixtures paths?

How to print raw logs to fluentd with Go rather than using a json map

I am using a golang function to print logs to fluentd. Fluent Post() method accepts only a map and not a string. I need a function to print raw text rather than forcing me to create a map.
Is that possible?
Below is my code:
package main
import (
"fmt"
"os"
"time"
"github.com/fluent/fluent-logger-golang/fluent"
)
var Flogger *fluent.Fluent
func init() {
initFluent()
}
func initFluent() {
var err error
Flogger, err = fluent.New(fluent.Config{FluentPort: 24224, FluentHost: "127.0.0.1"})
if err != nil {
fmt.Printf("Could not connect to Fluent at %s Error : %v", os.Getenv("FLUENTHOST"), err)
}
}
// DebugLog (msg string) Logs the string to Fluent server
func DebugLog(msg string) {
//Flogger.Post("debug.access", map[string]string{"data": msg})
Flogger.EncodeAndPostData("debug.access", time.Now(), map[string]string{"data": msg}) // both methods give same result
}
func main() {
msg := `{"mykey":"myval"}`
fmt.Println(msg) // prints {"mykey":"myval"}
DebugLog(msg) // prints {"data":"{\"mykey\":\"myval\"}"} with extra \
}
The function signatures as per the Fluent GoDoc as as follows:
func (f *Fluent) EncodeAndPostData(tag string, tm time.Time, message interface{}) error
func (f *Fluent) Post(tag string, message interface{}) error
Both functions accept the empty interface as the message parameter. All Go types satisfy the interface, so you can pass in anything that you'd like.
However, Fluentd is a structured logging library that primarily relies on JSON encoded data. Usually when using a third-party library, it serves well to use it the way it was intended.
Fluentd tries to structure data as JSON as much as possible: this
allows Fluentd to unify all facets of processing log data: collecting,
filtering, buffering, and outputting logs across multiple sources and
destinations
Source: Fluentd Website

Resources