I try to program a first google go program. I got this working part:
package main
import (
"fmt"
"os"
"regexp"
"github.com/PuerkitoBio/goquery"
"github.com/gocolly/colly"
)
func TrimSpaceNewlineInString(s string) string {
re := regexp.MustCompile(` +\n+ +\t+`)
return re.ReplaceAllString(s, "")
}
func main() {
args := os.Args[1:]
c := colly.NewCollector()
c.OnHTML("tr",
func(e *colly.HTMLElement) {
ch := e.DOM.Children()
spalte1 := ch.Eq(0)
spalte2 := ch.Eq(1)
spalte1.Each(
func(_ int, s *goquery.Selection) {
fmt.Print(TrimSpaceNewlineInString(s.Text()), ":", TrimSpaceNewlineInString(spalte2.Text()))
})
})
c.Visit("https://deweysearchde.pansoft.de/webdeweysearch/executeSearch.html" +
"?lastScheduleRecord=669.1-669.7&lastTableRecord=&query=" + args[0] + "&_showShortNotations=off&catalogs=DNB&_catalogs=off&catalogs=GBV&_catalogs=off&catalogs=HeBIS&_catalogs=off&catalogs=SUB&_catalogs=off&catalogs=SWB&_catalogs=off&catalogs=FUB&_catalogs=off")
}
But I only what to get the 2nd column, if this is in the range form [0-9.-] and if so than I would need the following 3rd column with the DDC Classification of this DOM HTMLElement talbe.
I would like to retrieve following
600;Technik
660;Chemische Verfahrenstechnik
669;Metallurgie
669.1-669.7;Metallurgie einzelner Metalle und deren Legierungen
669.1;Eisenmetalle
Can anyone here help me and tell me how it could be done with colly Colly doc for go, which is similar to jQuery?
PS: I have tried this way - with children. But the output looks like this. I do not know why.
Notation:Thema :
Haupttafeln
600:
Technik
660:
Chemische Verfahrenstechnik
661:
Industriechemikalien
661.2-661.6:
Säuren, Basen, Salze
661.5:
Ammoniumsalze
Notation:Thema :HilfstafelnT1--0:Hilfstafel 1. StandardschlüsselT2--0:Hilfstafel 2. Geo ...
Related
package main
import (
"fmt"
"encoding/json"
"reflect"
)
func someFunc( data interface{}, out interface{} ) {
v := reflect.ValueOf(out).Elem();
fmt.Printf("Incoming type: %s\n",reflect.ValueOf(v).Kind())
v.SetCap(reflect.ValueOf(data).Len())
}
func main() {
expected := []int{1,2,3}
jsonRaw, _ := json.Marshal(expected)
var tmpData interface{}
json.Unmarshal(jsonRaw, &tmpData)
fmt.Printf("%s\n",string(jsonRaw))
fmt.Printf("%#v\n",tmpData)
result := []int{}
var tmp interface{}
tmp = result
fmt.Printf("Outcoming type: %s\n",reflect.TypeOf(&tmp))
someFunc(tmpData,&tmp)
}
I would like to operate on v parameter inside someFunc as if it were
a slice, i.e. "Incoming type"-debug message should output slice.
However, it outputs struct, as is shown here.
The ultimate goal is that I use reflection to analyze the data-parameter's contents and recover everything into out, but for now I would like to
know how to make sure the correct type of v is detected,
so that I can use it as a slice.
EDIT: It appears to be impossible (as of 2013 at least): https://groups.google.com/forum/#!topic/golang-nuts/bldM9tIL-JM
to set the size of a slice for stuff discovered at runtime.
One of the authors says something to the effect that "you have to be able
to sort the elements, i.e. implement Less()" for the values...
EDIT: In any case, I did try to use MakeSlice in this Playgound link,
and it says reflect.MakeSlice of non-slice type
EDIT: I apologize and thank you all for your comments.
What I ended up doing is the following (after an illuminating read of the source code of MakeSlice):
package main
import (
"fmt"
"encoding/json"
"reflect"
)
func someFunc( data interface{}, out interface{} ) {
v := reflect.ValueOf(out).Elem();
fmt.Printf("Incoming type: %s\n",v.Kind())
//v.SetCap(reflect.ValueOf(data).Len()) <-- doesn't work
n := reflect.ValueOf(data).Len()
s := reflect.MakeSlice(reflect.TypeOf(data),n,n)
fmt.Printf("Len= %d\n",s.Len())
}
func main() {
expected := []int{1,2,3}
jsonRaw, _ := json.Marshal(expected)
var tmpData interface{}
json.Unmarshal(jsonRaw, &tmpData)
fmt.Printf("%s\n",string(jsonRaw))
fmt.Printf("%#v\n",tmpData)
result := []int{}
someFunc(tmpData,&result)
}
What I ended up doing is the following (after an illuminating read of the source code of MakeSlice):
package main
import (
"fmt"
"encoding/json"
"reflect"
)
func someFunc( data interface{}, out interface{} ) {
v := reflect.ValueOf(out).Elem();
fmt.Printf("Incoming type: %s\n",v.Kind())
//v.SetCap(reflect.ValueOf(data).Len()) <-- doesn't work
n := reflect.ValueOf(data).Len()
s := reflect.MakeSlice(reflect.TypeOf(data),n,n)
fmt.Printf("Len= %d\n",s.Len())
}
func main() {
expected := []int{1,2,3}
jsonRaw, _ := json.Marshal(expected)
var tmpData interface{}
json.Unmarshal(jsonRaw, &tmpData)
fmt.Printf("%s\n",string(jsonRaw))
fmt.Printf("%#v\n",tmpData)
result := []int{}
someFunc(tmpData,&result)
}
It appears that there are convenience functions such as SliceOf, too.
the bottomline is that the first argument of MakeSlice is not the type
of the arguments that the slice holds, but the slice-type, e.g. []int rather than int.
I have the following in the intepreter of Tour of Go:
package main
import (
"golang.org/x/tour/wc"
str "strings"
)
func WordCount(s string) map[string]int {
results := make(map[str]int)
words := str.Fields(s)
return map[string]int{"x": 1}
}
//func main() {
// wc.Test(WordCount)
//}
This is based on https://tour.golang.org/moretypes/23
My error is
tmp/sandbox169629521/main.go:9: use of package str without selector
Trying
results := make(map[str.string]int)
now fails with
tmp/sandbox424441423/main.go:9: cannot refer to unexported name strings.string
tmp/sandbox424441423/main.go:9: undefined: strings.string
"string" is a builtin. You don't need to do strings.string:
Docs: https://golang.org/pkg/builtin/#string
For some reason, it appears that adding new element to slice using reflection doesn't update slice itself. This is the code to demonstrate:
package main
import (
"fmt"
"reflect"
)
func appendToSlice(arrPtr interface{}) {
valuePtr := reflect.ValueOf(arrPtr)
value := valuePtr.Elem()
value = reflect.Append(value, reflect.ValueOf(55))
fmt.Println(value.Len()) // prints 1
}
func main() {
arr := []int{}
appendToSlice(&arr)
fmt.Println(len(arr)) // prints 0
}
Playground link : https://play.golang.org/p/j3532H_mUL
Is there something I'm missing here?
reflect.Append works like append in that it returns a new slice value.
You are assigning this value to the value variable in the appendToSlice function, which replaces the previous reflect.Value, but does not update the original argument.
To make it more clear what's happening, take the equivalent function to your example without reflection:
func appendToSlice(arrPtr *[]int) {
value := *arrPtr
value = append(value, 55)
fmt.Println(len(value))
}
What you need to use is the Value.Set method to update the original value:
func appendToSlice(arrPtr interface{}) {
valuePtr := reflect.ValueOf(arrPtr)
value := valuePtr.Elem()
value.Set(reflect.Append(value, reflect.ValueOf(55)))
fmt.Println(value.Len())
}
https://play.golang.org/p/Nhabg31Sju
package main
import "fmt"
import "reflect"
type Foo struct {
Name string
}
func main() {
_type := []Foo{}
fmt.Printf("_type: v(%v) T(%T)\n", _type, _type)
reflection := reflect.MakeSlice(reflect.SliceOf(reflect.TypeOf(_type).Elem()), 0, 0)
reflectionValue := reflect.New(reflection.Type())
reflectionValue.Elem().Set(reflection)
slicePtr := reflect.ValueOf(reflectionValue.Interface())
sliceValuePtr := slicePtr.Elem()
sliceValuePtr.Set(reflect.Append(sliceValuePtr, reflect.ValueOf(Foo{"a"})))
sliceValuePtr.Set(reflect.Append(sliceValuePtr, reflect.ValueOf(Foo{"b"})))
sliceValuePtr.Set(reflect.Append(sliceValuePtr, reflect.ValueOf(Foo{"c"})))
values := []Foo{Foo{"d"}, Foo{"e"}}
for _, val := range values {
sliceValuePtr.Set(reflect.Append(sliceValuePtr, reflect.ValueOf(val)))
}
result := sliceValuePtr.Interface()
fmt.Printf("result: %T = (%v)\n", result, result)
}
take a look at: https://play.golang.org/p/vXOqTVSEleO
I'm trying to parse an URL like:
http://example.com/id/123
I've read through the net/url docs but it seems like it only parses strings like
http://example.com/blah?id=123
How can I parse the ID so I end up with the value of the id in the first example?
This is not one of my own routes but a http string returned from an openid request.
In your example /id/123 is a path and you can get the "123" part by using Base from the path module.
package main
import (
"fmt"
"path"
)
func main() {
fmt.Println(path.Base("/id/123"))
}
For easy reference, here's the docs on the path module. http://golang.org/pkg/path/#example_Base
You can try using regular expression as follow:
import "regexp"
re, _ := regexp.Compile("/id/(.*)")
values := re.FindStringSubmatch(path)
if len(values) > 0 {
fmt.Println("ID : ", values[1])
}
Here is a simple solution that works for URLs with the same structure as yours (you can improve to suit those with other structures)
package main
import (
"fmt"
"net/url"
)
var path = "http://localhost:8080/id/123"
func getFirstParam(path string) (ps string) {
// ignore first '/' and when it hits the second '/'
// get whatever is after it as a parameter
for i := 1; i < len(path); i++ {
if path[i] == '/' {
ps = path[i+1:]
}
}
return
}
func main() {
u, _ := url.Parse(path)
fmt.Println(u.Path) // -> "/id/123"
fmt.Println(getFirstParam(u.Path)) // -> "123"
}
Or, as #gollipher suggested, use the path package
import "path"
func main() {
u, _ := url.Parse(path)
ps := path.Base(u.Path)
}
With this method it's faster than regex, provided you know before hand the structure of the URL you are getting.
I am new to go programming language and I'm stock on this scenario on my code.
Here's my example code:
a := genreAPI{Genre{"Pop"}, Genre{"Rock"}}
fmt.Println("Value of a :", a)
The current output is: Value of a : [{Pop} {Rock}]
How can I achieved an output like this:
Value of a : [{UG9w} {Um9jaw==}]
which is a base64 encode?
I am not sure what exactly is not clear from the documentation. Not only it has a clear name which explains states what the method is doing, it also has an example.
package main
import (
"encoding/base64"
"fmt"
)
func main() {
data := []byte("Pop")
str := base64.StdEncoding.EncodeToString(data)
fmt.Println(str) // UG9w
}
Go Playground
You can customise the output of print functions by providing a String() method for your type. Either for the whole Genre or just for the name variable.
Example:
package main
import (
"encoding/base64"
"fmt"
)
type Base64String string
func (b Base64String) String() string {
return base64.StdEncoding.EncodeToString([]byte(b))
}
type Genre struct {
Name Base64String
}
func main() {
a := []Genre{Genre{"Pop"}, Genre{"Rock"}}
fmt.Println(a) // prints [{UG9w} {Um9jaw==}]
fmt.Println(string(a[0].Name)) // prints Pop
}