This is a sample code from the Go blackfriday package:
package main
import (
"bytes"
"fmt"
"github.com/russross/blackfriday"
)
func main() {
input := []byte(`##Title
- another paragragh
This is a being rendered in a custom way.
`)
htmlFlags := 0
renderer := &renderer{Html: blackfriday.HtmlRenderer(htmlFlags, "", "").(*blackfriday.Html)}
extensions := 0
unsanitized := blackfriday.Markdown(input, renderer, extensions)
os.Stdout.Write(unsanitized)
}
// renderer implements blackfriday.Renderer and reuses blackfriday.Html for the most part,
// except overriding Link rendering.
type renderer struct {
*blackfriday.Html
}
func (options *renderer) Header(out *bytes.Buffer, text func() bool, level int, id string) {
fmt.Fprintf(out, "<custom link %q to %q>", content, link)
}
The purpose of the code is to customize the link attributes of the output HTML.
I tried to do the same but with p tags:
func (options *renderer) Paragraph(out *bytes.Buffer, text func() bool) {
fmt.Fprintf(out, "<p class='custom'>%q</p>", text)
}
But the output was this:
<h1>Title</h1>
<ul>
<li>another paragragh</li>
</ul>
<p class='custom'>%!q(func() bool=0x80a15d0)</p>
So I have no idea how to output the actual text (This is a being rendered in a custom way.). Any ideas?
This is the source code of the function:
func (options *Html) Paragraph(out *bytes.Buffer, text func() bool) {
marker := out.Len()
doubleSpace(out)
out.WriteString("<p>")
if !text() {
out.Truncate(marker)
return
}
out.WriteString("</p>\n")
}
I think your usage of function is wrong.
Take a look at Html.Paragraph and override this.
Maybe it would look like:
func (options *renderer) Paragraph(out *bytes.Buffer, text func() bool) {
marker := out.Len()
doubleSpace(out)
out.WriteString("<p class='custom'>")
if !text() {
out.Truncate(marker)
return
}
out.WriteString("</p>\n")
}
I believe all you need to do is call text
func (options *renderer) Paragraph(out *bytes.Buffer, text func() bool) {
fmt.Fprintf(out, "<p class='custom'>%q</p>\n", text())
}
Related
In Go, it's possible to create function types (https://golang.org/ref/spec#Function_types) like this
type Printer func(s string)
How can I find all of the functions which satisfy this type? For example, if I had the file below, how could I find out that consolePrinter is a Printer?
package main
import "fmt"
func main() {
printToScreen(consolePrinter)
}
// Printer defines a function that prints a string.
type Printer func(s string)
func printToScreen(p Printer) {
p("Hello")
}
// consolePrinter is a Printer (the function signature is identical).
func consolePrinter(s string) {
fmt.Println(s)
}
I tried out guru, but the implements feature doesn't seem to support function types.
guru implements ./main.go:#134
/Users/adrian/go/gurutest/main.go:10.6-10.12: signature type Printer implements only interface{}
You are looking for functions that are assignable to Printer.
The guru tool does not support an "assignable to" query.
You can write a program to find functions that are assignable to Printer using the go/types package. Follow the tutorial to type check code and use AssignableTo to find the functions.
If you are only interested in finding functions declared at package level in gofmted code, then searching your code with a regexp might be good enough. This approach is not accurate, but is simple to do in many editors or the command line.
Here consolePrinter is not of type Printer it is required to declare consolePrinter to be of type Printer.You see console.Printer is a function with same underlying type that's why it can be pass to printToScreen which takes Printer type. Have a Look at the difference with their types
package main
import (
"fmt"
"reflect"
)
func main() {
printToScreen(consolePrinter)
fmt.Println(reflect.TypeOf(consolePrinter)) // Prints func(string) as its type
}
// Printer defines a function that prints a string.
type Printer func(s string)
func printToScreen(p Printer) {
var consoleprint Printer // declare to be of type Printer
fmt.Println(reflect.TypeOf(consoleprint)) // Prints main.Printer
p("Hello")
}
// consolePrinter is a Printer (the function signature is identical).
func consolePrinter(s string) {
fmt.Println(s)
}
Playground Example
Working example, based on Thundercat's answer. guru would need to do something like this to provide support for looking up suitable functions to pass in (e.g. implementations of http.HandleFunc).
package main
import (
"fmt"
"go/ast"
"go/importer"
"go/parser"
"go/token"
"go/types"
"log"
)
const hello = `package main
import "fmt"
const x = 1;
func main() {
fmt.Println("Hello, world")
}
// Printer defines a function that prints a string.
type Printer func(s string)
func consolePrinter(s string) {
fmt.Println(s)
}
`
func main() {
fset := token.NewFileSet()
// Parse the input string, []byte, or io.Reader,
// recording position information in fset.
// ParseFile returns an *ast.File, a syntax tree.
f, err := parser.ParseFile(fset, "hello.go", hello, 0)
if err != nil {
log.Fatal(err) // parse error
}
// A Config controls various options of the type checker.
// The defaults work fine except for one setting:
// we must specify how to deal with imports.
conf := types.Config{Importer: importer.Default()}
// Type-check the package containing only file f.
// Check returns a *types.Package.
pkg, err := conf.Check("cmd/hello", fset, []*ast.File{f}, nil)
if err != nil {
log.Fatal(err) // type error
}
names, signatures := getFunctionTypes(pkg)
for i, name := range names {
fmt.Println("Functions which implement", name)
for _, implementor := range getFunctionsWhichImplement(signatures[i], pkg) {
fmt.Println(implementor)
}
}
}
func getFunctionTypes(pkg *types.Package) (names []string, signatures []*types.Signature) {
for _, name := range pkg.Scope().Names() {
o := pkg.Scope().Lookup(name)
if _, isType := o.(*types.TypeName); !isType {
continue
}
var sig *types.Signature
var isFunc bool
if sig, isFunc = o.Type().Underlying().(*types.Signature); !isFunc {
continue
}
signatures = append(signatures, sig)
names = append(names, name)
}
return
}
func getFunctionsWhichImplement(sig *types.Signature, pkg *types.Package) (fns []types.Object) {
for _, name := range pkg.Scope().Names() {
o := pkg.Scope().Lookup(name)
if _, isType := o.(*types.TypeName); isType {
continue
}
var csig *types.Signature
var isFunc bool
if csig, isFunc = o.Type().Underlying().(*types.Signature); !isFunc {
continue
}
if types.AssignableTo(sig, csig) {
fns = append(fns, o)
}
}
return
}
The output of this code is shown below:
Functions which implement Printer
func cmd/hello.consolePrinter(s string)
I'm sure there are better ways for the specific case of printing, but in general you may be better to take a more idiomatic go approach of using interfaces.
Declare an interface and types that implement the interface:
type StringPrinter interface {
PrintString(string)
}
type Console struct {
// console specific stuff
}
func (c *Console) PrintString(s string) {
// Printing code
}
type Paper struct {
// paper specific stuff
}
func (p *Paper) PrintString(s string) {
// Printing code
}
Then to print in different ways you can access through an interface in a generic way:
func main() {
var sp StringPrinter
sp = &Console{ /* member inits */ }
sp.PrintString("Hello on console")
sp = &Paper{ /* member inits */ }
sp.PrintString("Hello on paper")
}
You will be able to use guru on this form of code to find objects that implement the StringPrinter interface.
I'm working on a golang web crawler that should parse the search results on some specific search engine. The main difficulty - parsing with concurrency, or rather, in processing pagination such as
← Previous 1 2 3 4 5 ... 34 Next →. All things work fine except recursive crawling of paginated results. Look at my code:
package main
import (
"bufio"
"errors"
"fmt"
"net"
"strings"
"github.com/antchfx/htmlquery"
"golang.org/x/net/html"
)
type Spider struct {
HandledUrls []string
}
func NewSpider(url string) *Spider {
// ...
}
func requestProvider(request string) string {
// Everything is good here
}
func connectProvider(url string) net.Conn {
// Also works
}
// getContents makes request to search engine and gets response body
func getContents(request string) *html.Node {
// ...
}
// CheckResult controls empty search results
func checkResult(node *html.Node) bool {
// ...
}
func (s *Spider) checkVisited(url string) bool {
// ...
}
// Here is the problems
func (s *Spider) Crawl(url string, channelDone chan bool, channelBody chan *html.Node) {
body := getContents(url)
defer func() {
channelDone <- true
}()
if checkResult(body) == false {
err := errors.New("Nothing found there")
ErrFatal(err)
}
channelBody <- body
s.HandledUrls = append(s.HandledUrls, url)
fmt.Println("Handled ", url)
newUrls := s.getPagination(body)
for _, u := range newUrls {
fmt.Println(u)
}
for i, newurl := range newUrls {
if s.checkVisited(newurl) == false {
fmt.Println(i)
go s.Crawl(newurl, channelDone, channelBody)
}
}
}
func (s *Spider) getPagination(node *html.Node) []string {
// ...
}
func main() {
request := requestProvider(*requestFlag)
channelBody := make(chan *html.Node, 120)
channelDone := make(chan bool)
var parsedHosts []*Host
s := NewSpider(request)
go s.Crawl(request, channelDone, channelBody)
for {
select {
case recievedNode := <-channelBody:
// ...
for _, h := range newHosts {
parsedHosts = append(parsedHosts, h)
fmt.Println("added", h.HostUrl)
}
case <-channelDone:
fmt.Println("Jobs finished")
}
break
}
}
It always returns the first page only, no pagination. Same GetPagination(...) works good. Please tell me, where is my error(s).
Hope Google Translate was correct.
The problem is probably that main exits before all goroutine finished.
First, there is a break after the select statement and it runs uncodintionally after first time a channel is read. That ensures the main func returns after the first time you send something over channelBody.
Secondly, using channelDone is not the right way here. The most idomatic approach would be using a sync.WaitGroup. Before starting each goroutine, use WG.Add(1) and replace the defer with defer WG.Done(); In main, use WG.Wait(). Please be aware that you should use a pointer to refer to the WaitGroup. You can read more here.
Is it possible get information about caller function in Golang? For example if I have
func foo() {
//Do something
}
func main() {
foo()
}
How can I get that foo has been called from main?
I'm able to this in other language (for example in C# I just need to use CallerMemberName class attribute)
You can use runtime.Caller for easily retrieving information about the caller:
func Caller(skip int) (pc uintptr, file string, line int, ok bool)
Example #1: Print caller file name and line number: https://play.golang.org/p/cdO4Z4ApHS
package main
import (
"fmt"
"runtime"
)
func foo() {
_, file, no, ok := runtime.Caller(1)
if ok {
fmt.Printf("called from %s#%d\n", file, no)
}
}
func main() {
foo()
}
Example #2: Get more information with runtime.FuncForPC: https://play.golang.org/p/y8mpQq2mAv
package main
import (
"fmt"
"runtime"
)
func foo() {
pc, _, _, ok := runtime.Caller(1)
details := runtime.FuncForPC(pc)
if ok && details != nil {
fmt.Printf("called from %s\n", details.Name())
}
}
func main() {
foo()
}
expanding on my comment, here's some code that returns the current func's caller
import(
"fmt"
"runtime"
)
func getFrame(skipFrames int) runtime.Frame {
// We need the frame at index skipFrames+2, since we never want runtime.Callers and getFrame
targetFrameIndex := skipFrames + 2
// Set size to targetFrameIndex+2 to ensure we have room for one more caller than we need
programCounters := make([]uintptr, targetFrameIndex+2)
n := runtime.Callers(0, programCounters)
frame := runtime.Frame{Function: "unknown"}
if n > 0 {
frames := runtime.CallersFrames(programCounters[:n])
for more, frameIndex := true, 0; more && frameIndex <= targetFrameIndex; frameIndex++ {
var frameCandidate runtime.Frame
frameCandidate, more = frames.Next()
if frameIndex == targetFrameIndex {
frame = frameCandidate
}
}
}
return frame
}
// MyCaller returns the caller of the function that called it :)
func MyCaller() string {
// Skip GetCallerFunctionName and the function to get the caller of
return getFrame(2).Function
}
// foo calls MyCaller
func foo() {
fmt.Println(MyCaller())
}
// bar is what we want to see in the output - it is our "caller"
func bar() {
foo()
}
func main(){
bar()
}
For more examples: https://play.golang.org/p/cv-SpkvexuM
I want to be able to unmarshal yaml files less rigidly. That is, my library has a predefined number of options the yaml file must have. Then, the user should be able to extend this to include any custom options.
Here is what I have
package main
import (
"net/http"
"yamlcms"
"github.com/julienschmidt/httprouter"
)
type Page struct {
*yamlcms.Page
Title string
Date string
}
func getBlogRoutes() {
pages := []*Page{}
yamlcms.ReadDir("html", pages)
}
// This section is a work in progress, I only include it for loose context
func main() {
router := httprouter.New()
//blogRoutes := getBlogRoutes()
//for _, blogRoute := range *blogRoutes {
// router.Handle(blogRoute.Method, blogRoute.Pattern,
// func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {})
//}
http.ListenAndServe(":8080", router)
}
Here is the yamlcms package:
package yamlcms
import (
"io/ioutil"
"os"
"strings"
"gopkg.in/yaml.v2"
)
type Page struct {
Slug string `yaml:"slug"`
File string `yaml:"file"`
}
func (page *Page) ReadFile(file string) (err error) {
fileContents, err := ioutil.ReadFile(file)
if err != nil {
return
}
err = yaml.Unmarshal(fileContents, &page)
return
}
func isYamlFile(fileInfo os.FileInfo) bool {
return !fileInfo.IsDir() && strings.HasSuffix(fileInfo.Name(), ".yaml")
}
func ReadDir(dir string, pages []*Page) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
pages[i].ReadFile(fileInfo.Name())
}
}
return
}
There is a compiler issue here:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.Page in argument to yamlcms.ReadDir
My main intent in this question is to learn the idiomatic way of doing this kind of thing in Go. Other 3rd-party solutions may exist but I am not immediately interested in them because I have problems like this frequently in Go having to do with inheritance, etc. So along the lines of what I've presented, how can I best (idiomatically) accomplish what I am going for?
EDIT:
So I've made some changes as suggested. Now I have this:
type FileReader interface {
ReadFile(file string) error
}
func ReadDir(dir string, pages []*FileReader) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
(*pages[i]).ReadFile(fileInfo.Name())
}
}
return
}
However, I still get a similar compiler error:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.FileReader in argument to yamlcms.ReadDir
Even though main.Page should be a FileReader because it embeds yamlcms.Page.
EDIT: I forgot that slices of interfaces don't work like that. You'd need to allocate a new slice, convert all pages to FileReaders, call the function, and convert them back.
Another possible solution is refactoring yamlcms.ReadDir to return the contents of the files, so that they could be unmarshaled later:
// In yamlcms.
func ReadYAMLFilesInDir(dir string) ([][]byte, error) { ... }
// In client code.
files := yamlcms.ReadYAMLFilesInDir("dir")
for i := range pages {
if err := yaml.Unmarshal(files[i], &pages[i]); err != nil { return err }
}
The original answer:
There are no such things as inheritance or casting in Go. Prefer composition and interfaces in your designs. In your case, you can redefine your yamlcms.ReadDir to accept an interface, FileReader.
type FileReader interface {
ReadFile(file string) error
}
Both yamlcms.Page and main.Page will implement this, as the latter embeds the former.
I have an interface:
type Reader interface {
// Read IV and Master header
ReadMaster(p []byte, full bool) (int, error)
// Read User header
ReadUser(p []byte, full bool) (int, error)
// Read Content data
ReadContent(p []byte) (int, error)
}
And I have three structs are compatible with the interface. All the three structs have the samve method ReadUser. So I have to do:
func (r *s1) ReadUser(buf []byte, full bool) (int, error) {
//.... code 1 ....
}
func (r *s2) ReadUser(buf []byte, full bool) (int, error) {
//.... code 2 ....
}
func (r *s3) ReadUser(buf []byte, full bool) (int, error) {
//.... code 3 ....
}
However, the "code1", "code2" and "code3" above are exactly the same. It's there a good way to reduce the duplicate codes? E.g. define the function once and assign it to three struct?
Wrap it in its own type. Remembering too that interfaces in Go should only provide contracts for small specific tasks. It is very common for an interface to contain only a single method.
type UserReader interface {
ReadUser(p []byte, full bool) (int, error)
}
type UserRepo struct {
}
Add the method to that type:
func (ur *UserRepo) ReadUser(p []byte, full bool) (int, error) {
// code to read a user
}
Then, embed it in your other types:
type s1 struct {
*UserRepo
// other stuff here..
}
type s2 struct {
*UserRepo
// other stuff here..
}
type s3 struct {
*UserRepo
// other stuff here..
}
Then you can:
u := s1{}
i, err := u.ReadUser(..., ...)
u2 := s2{}
i2, err2 := u2.ReadUser(..., ...)
// etc..
..and you can also do:
doStuff(u)
doStuff(u2)
.. where doStuff is:
func doStuff(u UserReader) {
// any of the three structs
}
Click here to see it in the Playground