How to build a URL / Query in Golang - go

Background -
I need to build a URL / query based on user input from a form that will be used to make an API call.
Problem -
When building the URL, the params are not properly escaped. For example, the query "bad santa" ends up with a space between it instead of "+".
Current Output -
e.g. https://api.example.org/3/search/movie?query=bad
santa&api_key=#######
Expected Output -
e.g. https://api.example.org/3/search/movie?query=bad+santa&api_key=#######
Code Example -
Root URL -
var SearchUrl = "https://www.example.org/3/search/movie?query="
Get params taken from user input -
var MovieSearch []string = r.Form["GetSearchKey"]
API Key -
var apiKey = "&api_key=######"
I am using the ArrayToString() to parse the form input data
func ArrayToString(array []string) string{
str := strings.Join(array, "+")
return str
}
Then building the URL -
var SearchUrl = "https://api.example.org/3/search/movie?query="
var MovieSearch []string = r.Form["GetSearchKey"]
var apiKey = "&api_key=########"
UrlBuild := []string {SearchUrl, ArrayToString(MovieSearch), apiKey}
OUTPUT_STRING := ArrayToString(UrlBuild)
Question -
How to build a URL with user input GET params that are escaped properly?

Normally, one should use url package's Values.
Here's an example, that does what I think you want, on play
Both a simple main, and in http.HandlerFunc form:
package main
import "fmt"
import "net/url"
import "net/http"
func main() {
baseURL := "https://www.example.org/3/search/movie"
v := url.Values{}
v.Set("query", "this is a value")
perform := baseURL + "?" + v.Encode()
fmt.Println("Perform:", perform)
}
func formHandler(w http.ResponseWriter, r *http.Request) {
baseURL := "https://www.example.org/3/search/movie"
v := url.Values{}
v.Set("query", r.Form.Get("GetSearchKey")) // take GetSearchKey from submitted form
v.Set("api_ley", "YOURKEY") // whatever your api key is
perform := baseURL + "?" + v.Encode() // put it all together
fmt.Println("Perform:", perform) // do something with it
}
Output:
Perform: https://www.example.org/3/search/movie?query=this+is+a+value
Notice how the values are put in to query string, properly escaped, for you.

You can escape parameters using https://golang.org/pkg/net/url/#QueryEscape, instead of doing it yourself.
Besides you should be using https://golang.org/pkg/net/url/#URL to build up your url:
params := fmt.Sprintf("?query=%s&api_key=######", url.QueryEscape("name"))
perform := url.URL{
Scheme: "https",
Host: "api.example.com",
Path: "3/search/movie",
RawQuery: params,
}
fmt.Println(perform) // <- Calls .String()
I recommend to check https://golang.org/doc/effective_go.html.
If your data comes in []string:
func ArrayToQuery(values []string) string {
return url.QueryEscape(strings.Join(values, " "))
}

If MovieSearch contains one element with the value "bad santa", what you're seeing looks correct. It's joining those three strings and putting "+" between them.

If there is a space in the word you will need to replace it.
Example
package main
import (
"fmt"
"strings"
)
func main() {
fmt.Println(strings.Replace("bad santa", " ", "+", -1))
}
So you should probably do it like this
func main() {
a := []string{"bad", "santa"}
fmt.Printf("%q\n", a)
j := ArrayToString(a)
strings.Replace(j, " ", "+",-1)
fmt.Printf("%q\n", j)
}
Here is a link to the Go Documentation - https://golang.org/pkg/strings/#Replace

Related

Generated valid label value (Kubernetes)

Label values in Kubernetes need to be valid.
See IsValidLabelValue()
For example the input I receive from a rest-API of a provider, which I want to write to a label: Dedicated Server 1U.
Is there a way to generate a valid label via Go from an arbitrary string?
you can have a function to do this, for example:
func generateLabel(input string) string {
input = strings.Replace(input, " ", "-", -1)
return "api-label=" + input
}
the function replaces the spaces in the received string to "-"
you can change the key to any string you like.
you can also add a regex check to make sure that the generated value complies with the label constraints. (this depends if any special characters are being received from the API)
To accept the string even when there are unwanted characters, check the below:
package main
import (
"regexp"
"strings"
"fmt"
)
func generateLabel(input string) string {
input = strings.Replace(input, " ", "-", -1)
re := regexp.MustCompile("[^a-zA-Z0-9-]")
input = re.ReplaceAllString(input, "")
re = regexp.MustCompile("^[^a-zA-Z0-9]([-a-zA-Z0-9]*[a-zA-Z0-9])?$")
input = re.ReplaceAllString(input, "")
return "api-label=" + input
}
func main() {
label := generateLabel("Dedicated Server 1U")
fmt.Println(label) // Output: "api-label=Dedicated-Server-1U"
label1 := generateLabel("Dedicated&test")
fmt.Println(label1) // Output: "api-label=Dedicatedtest"
label2 := generateLabel("Dedicated,test##&(*!great")
fmt.Println(label2) // Output: "api-label=Dedicatedtestgreat"
}

How to parse email addresses from a long string in Golang

How can I extract only email addresses from a long string in Golang? For example:
"a bunch of irrelevant text fjewiwofjfjvnvkdlslsosiejwoqlwpwpwo
mail=jim.halpert#gmail.com,ou=f,c=US
mail=apple.pie#gmail.com,ou=f,c=US
mail=hello.world#gmail.com,ou=f,c=US
mail=alex.alex#gmail.com,ou=f,c=US
mail=bob.jim#gmail.com,ou=people,ou=f,c=US
mail=arnold.schwarzenegger#gmail.com,ou=f,c=US"
This would return a list of all the emails:
[jim.halpert#gmail.com, apple.pie#gmail.com, etc...]
Each email address would begin with "mail=" and end with a comma ",".
For this you need to breakdown the long go string into parts that you need. You can do filtration and searching using Regular Expressions to match the email pattern you see above.
Here's a piece of code using Regular Expressions to first obtain the section with "mail=" then further format the email removing the trailing ,
import (
"fmt"
"regexp"
"strings"
)
func main() {
var re = regexp.MustCompile(`(?m)mail=[A-Za-z.#0-9]+\,`)
var str = `a bunch of irrelevant text fjewiwofjfjvnvkdlslsosiejwoqlwpwpwo
mail=jim.halpert#gmail.com,ou=f,c=US
mail=apple.pie#gmail.com,ou=f,c=US
mail=hello.world#gmail.com,ou=f,c=US
mail=alex.alex#gmail.com,ou=f,c=US
mail=bob.jim#gmail.com,ou=people,ou=f,c=US
mail=arnold.schwarzenegger#gmail.com,ou=f,c=US`
for i, match := range re.FindAllString(str, -1) {
fmt.Println(match, "found at index", i)
email := strings.Split(match, "=")[1]
email = strings.ReplaceAll(email, ",", "")
fmt.Print(email)
}
}
while i agree with the comment from user datenwolf here is another version which does not involve regular expressions.
It also handle more complex emails format including comma within the local parts. Something uneasy to implement using regexp.
see https://stackoverflow.com/a/2049510/11892070
import (
"bufio"
"fmt"
"strings"
)
var str = `a bunch of irrelevant text fjewiwofjfjvnvkdlslsosiejwoqlwpwpwo
mail=jim.halpert#gmail.com,ou=f,c=US
mail=apple.pie#gmail.com,ou=f,c=US
mail=hello.world#gmail.com,ou=f,c=US
mail=alex.alex#gmail.com,ou=f,c=US
mail=bob.jim#gmail.com,ou=people,ou=f,c=US
mail=arnold.schwarzenegger#gmail.com,ou=f,c=US
mail=(comented)arnold.schwarzenegger#gmail.com,ou=f,c=US
mail="(with comma inside)arnold,schwarzenegger#gmail.com",ou=f,c=US
mail=nocommaatall#gmail.com`
func main() {
var emails []string
sc := bufio.NewScanner(strings.NewReader(str))
for sc.Scan() {
t := sc.Text()
if !strings.HasPrefix(t, "mail=") {
continue
}
t = t[5:]
// Lookup for the next comma after the #.
at := strings.Index(t, "#")
comma := strings.Index(t[at:], ",")
if comma < 0 {
email := strings.TrimSpace(t)
emails = append(emails, email)
continue
}
comma += at
email := strings.TrimSpace(t[:comma])
emails = append(emails, email)
}
for _, e := range emails {
fmt.Println(e)
}
}
You can use this package to do that :
https://github.com/hamidteimouri/htutils/blob/main/htregex/htregex.go
// Emails finds all email strings
func Emails(text string) []string {
return match(text, EmailsRegex)
}
you can use an original package from golang is regexp.Compile or regexp.MustCompile
r, _ := regexp.Compile(regexEmail)
newVariable := `a bunch of irrelevant text fjewiwofjfjvnvkdlslsosiejwoqlwpwpwo
mail=jim.halpert#gmail.com,ou=f,c=US
mail=apple.pie#gmail.com,ou=f,c=US
mail=hello.world#gmail.com,ou=f,c=US
mail=alex.alex#gmail.com,ou=f,c=US
mail=bob.jim#gmail.com,ou=people,ou=f,c=US
mail=arnold.schwarzenegger#gmail.com,ou=f,c=US`
fmt.Printf("%#v\n", r.FindStringSubmatch(newVariable))
fmt.Printf("%#v\n", r.SubexpNames())

What is the equivalent of jQuery's n-th child method?

I try to program a first google go program. I got this working part:
package main
import (
"fmt"
"os"
"regexp"
"github.com/PuerkitoBio/goquery"
"github.com/gocolly/colly"
)
func TrimSpaceNewlineInString(s string) string {
re := regexp.MustCompile(` +\n+ +\t+`)
return re.ReplaceAllString(s, "")
}
func main() {
args := os.Args[1:]
c := colly.NewCollector()
c.OnHTML("tr",
func(e *colly.HTMLElement) {
ch := e.DOM.Children()
spalte1 := ch.Eq(0)
spalte2 := ch.Eq(1)
spalte1.Each(
func(_ int, s *goquery.Selection) {
fmt.Print(TrimSpaceNewlineInString(s.Text()), ":", TrimSpaceNewlineInString(spalte2.Text()))
})
})
c.Visit("https://deweysearchde.pansoft.de/webdeweysearch/executeSearch.html" +
"?lastScheduleRecord=669.1-669.7&lastTableRecord=&query=" + args[0] + "&_showShortNotations=off&catalogs=DNB&_catalogs=off&catalogs=GBV&_catalogs=off&catalogs=HeBIS&_catalogs=off&catalogs=SUB&_catalogs=off&catalogs=SWB&_catalogs=off&catalogs=FUB&_catalogs=off")
}
But I only what to get the 2nd column, if this is in the range form [0-9.-] and if so than I would need the following 3rd column with the DDC Classification of this DOM HTMLElement talbe.
I would like to retrieve following
600;Technik
660;Chemische Verfahrenstechnik
669;Metallurgie
669.1-669.7;Metallurgie einzelner Metalle und deren Legierungen
669.1;Eisenmetalle
Can anyone here help me and tell me how it could be done with colly Colly doc for go, which is similar to jQuery?
PS: I have tried this way - with children. But the output looks like this. I do not know why.
Notation:Thema :
Haupttafeln
600:
Technik
660:
Chemische Verfahrenstechnik
661:
Industriechemikalien
661.2-661.6:
Säuren, Basen, Salze
661.5:
Ammoniumsalze
Notation:Thema :HilfstafelnT1--0:Hilfstafel 1. StandardschlüsselT2--0:Hilfstafel 2. Geo ...

How to add variable to string variable in golang

I'm trying to add a value to a variable string in golang, without use printf because I'm using revel framework and this is for a web enviroment instead of console, this is the case:
data := 14
response := `Variable string content`
so I can't get variable data inside variable response, like this
response := `Variable string 14 content`
Any idea?
Why not use fmt.Sprintf?
data := 14
response := fmt.Sprintf("Variable string %d content", data)
I believe that the accepted answer is already the best practice one. Just like to give an alternative option based on #Ari Pratomo answer:
package main
import (
"fmt"
"strconv"
)
func main() {
data := 14
response := "Variable string " + strconv.Itoa(data) + " content"
fmt.Println(response) //Output: Variable string 14 content
}
It using strconv.Itoa() to convert an integer to string, so it can be concatenated with the rest of strings.
Demo: https://play.golang.org/p/VnJBrxKBiGm
You can use text/template:
package main
import (
"strings"
"text/template"
)
func format(s string, v interface{}) string {
t, b := new(template.Template), new(strings.Builder)
template.Must(t.Parse(s)).Execute(b, v)
return b.String()
}
func main() {
data := 14
response := format("Variable string {{.}} content", data)
println(response)
}
If you want to keep the string in variable rather than to print out, try like this:
data := 14
response := "Variable string" + data + "content"

How do I parse URLs in the format of /id/123 not ?foo=bar

I'm trying to parse an URL like:
http://example.com/id/123
I've read through the net/url docs but it seems like it only parses strings like
http://example.com/blah?id=123
How can I parse the ID so I end up with the value of the id in the first example?
This is not one of my own routes but a http string returned from an openid request.
In your example /id/123 is a path and you can get the "123" part by using Base from the path module.
package main
import (
"fmt"
"path"
)
func main() {
fmt.Println(path.Base("/id/123"))
}
For easy reference, here's the docs on the path module. http://golang.org/pkg/path/#example_Base
You can try using regular expression as follow:
import "regexp"
re, _ := regexp.Compile("/id/(.*)")
values := re.FindStringSubmatch(path)
if len(values) > 0 {
fmt.Println("ID : ", values[1])
}
Here is a simple solution that works for URLs with the same structure as yours (you can improve to suit those with other structures)
package main
import (
"fmt"
"net/url"
)
var path = "http://localhost:8080/id/123"
func getFirstParam(path string) (ps string) {
// ignore first '/' and when it hits the second '/'
// get whatever is after it as a parameter
for i := 1; i < len(path); i++ {
if path[i] == '/' {
ps = path[i+1:]
}
}
return
}
func main() {
u, _ := url.Parse(path)
fmt.Println(u.Path) // -> "/id/123"
fmt.Println(getFirstParam(u.Path)) // -> "123"
}
Or, as #gollipher suggested, use the path package
import "path"
func main() {
u, _ := url.Parse(path)
ps := path.Base(u.Path)
}
With this method it's faster than regex, provided you know before hand the structure of the URL you are getting.

Resources