How do we manage a '#' in the Rest API query parameter [closed] - go

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I am facing a challenge with extracting the complete value out of the query param.
For instance, I have a localhost:8080/myAPI?name=Shop & stop #1234
I have the issue with & as well but atleast the content after & is getting populated as a separate param. I wrote some logic to manage it.
Coming to #, the content after # is not at all getting populated. I am using net/http library.
Did anyone face the similar issue?

First, in URLs the & character has special meaning: it spearates query parameters. So if the data you encode in the URL contains & characters, it has to be escaped. url.QueryEscape() can show you how the escaped form looks like:
fmt.Println(url.QueryEscape("Shop & stop "))
It outputs:
Shop+%26+stop+
Note that & is escaped like %26, and also spaces are substituted with a + sign.
So your valid input URL should look like this:
rawURL := "http://localhost:8080/myAPI?name=Shop+%26+stop+#23452"
Use the net/url package to parse valid URLs, it also supports getting the parameter values and the fragment you're looking for (the fragment is the remaining part after the # character):
u, err := url.Parse(rawURL)
if err != nil {
panic(err)
}
fmt.Println(u.Query().Get("name"))
fmt.Println(u.Fragment)
This outputs:
Shop & stop
23452
Try the examples on the Go Playground.

The bytes &, # and space have special meaning in a URL. These bytes must be escaped when included in a query value.
Use url.Values to create the escaped query:
v := url.Values{}
v.Set("name", "Shop & Stop #23452")
q := v.Encode()
u := "http://localhost:8080/myAPI?" + q
fmt.Println(u) // prints http://localhost:8080/myAPI?name=Shop+%26+stop+%2323452
You can also construct the URL using the url.URL type instead of string concatenation.
v := url.Values{}
v.Set("name", "Shop & Stop #23452")
u := url.URL{Scheme: "http", Host: "localhost:8080", Path: "/myapi", RawQuery: v.Encode()}
s := u.String()
fmt.Println(s) // prints http://localhost:8080/myAPI?name=Shop+%26+stop+%2323452
The url package encodes the # as %23. Most http server libraries will decode the %23 back to #. Here's an example using Go's net/http package:
name := req.FormValue("name") // name is "Shop & Stop #23452"
Run a complete client and server example on the Go Playground.

Related

Problem in connection string with escaping char

Im trying to develop my first API using Go, but at beginning I run into a problem.
str := "sqlserver://DESKTOP-AAAC9AC#SQLEXPRESS?database=GolangPlay"
dsn := strings.Replace(str, "#", `\`, -1)
fmt.Println("CS be like : " + dsn)
_, err := gorm.Open(sqlserver.Open(dsn), &gorm.Config{})
My connection string must contains escaping tag '' (DESKTOP-AAAC9AC\SQLEXPRESS), so I thought that firstly I might in my str string replace escaping tag with #. So that later when calling the gorm function sqlserver.Open(string) replace the # with escaping tag, but I dont understand why this function returns error with double backslash
"failed to initialize database, got error parse "sqlserver://DESKTOP-AAAC9AC\SQLEXPRESS?database=GolangPlay": invalid character "\" in host name"
but my println returns that what exactly I want
CS be like : sqlserver://DESKTOP-AAAC9AC\SQLEXPRESS?database=GolangPlay
Please explain why this is happening and help in solving it

How to convert the string representation of a Terraform set of strings to a slice of strings

I've a terratest where I get an output from terraform like so s := "[a b]". The terraform output's value = toset([resource.name]), it's a set of strings.
Apparently fmt.Printf("%T", s) returns string. I need to iterate to perform further validation.
I tried the below approach but errors!
var v interface{}
if err := json.Unmarshal([]byte(s), &v); err != nil {
fmt.Println(err)
}
My current implementation to convert to a slice is:
s := "[a b]"
s1 := strings.Fields(strings.Trim(s, "[]"))
for _, v:= range s1 {
fmt.Println("v -> " + v)
}
Looking for suggestions to current approach or alternative ways to convert to arr/slice that I should be considering. Appreciate any inputs. Thanks.
Actually your current implementation seems just fine.
You can't use JSON unmarshaling because JSON strings must be enclosed in double quotes ".
Instead strings.Fields does just that, it splits a string on one or more characters that match unicode.IsSpace, which is \t, \n, \v. \f, \r and .
Moeover this works also if terraform sends an empty set as [], as stated in the documentation:
returning [...] an empty slice if s contains only white space.
...which includes the case of s being empty "" altogether.
In case you need additional control over this, you can use strings.FieldsFunc, which accepts a function of type func(rune) bool so you can determine yourself what constitutes a "space". But since your input string comes from terraform, I guess it's going to be well-behaved enough.
There may be third-party packages that already implement this functionality, but unless your program already imports them, I think the native solution based on the standard lib is always preferrable.
unicode.IsSpace actually includes also the higher runes 0x85 and 0xA0, in which case strings.Fields calls FieldsFunc(s, unicode.IsSpace)
package main
import (
"fmt"
"strings"
)
func main() {
src := "[a b]"
dst := strings.Split(src[1:len(src)-1], " ")
fmt.Println(dst)
}
https://play.golang.org/p/KVY4r_8RWv6

Writing to a File in Golang

I'm rather new to Golang and not sure yet, how to use certain language constructs. Currently I have following code (with test debug outputs), which does not provide expected result:
json, _ := json.Marshal(struct)
fmt.Println(json)
f,_ := os.Create(fmt.Sprintf("/tmp/%s.json", "asd"))
i,_ := f.Write(json)
fmt.Println(i)
b, err := ioutil.ReadAll(f)
fmt.Print(b)
I expect the following behaviour:
translating the struct to a byte array
creating a new file
append the byte array to the file
However, the file is always empty when I run the code in my environment (AWS Lambda), as well as using it in the Golang Playground.
The output of above code looks like this:
[123 34 ... <hug array of bytes>]
1384
[]
which leads me to believe I'm using f.Write() not correctly, although I followed the package documentation. All other outputs indicate expected behavior, so what is my mistake? I'm somewhat restricted to using the File interface, otherwise I'd have gone with ioutil.WriteFile(). My assumption is a misunderstanding of pointer/values at some point, but the compiler prevented a usage of &f.
After f.Write(), your current position in the file is at the end of it, so ioutil.ReadAll() will read from that position and return nothing.
You need to call f.Sync() to make sure that the data is persistently saved to the disk, and then f.Seek(0, 0) to rewind to the beginning of the file first.
Update: from comments, it seems that you only need to serialize the JSON and pass it forward as io.Reader, for that you don't really need a file, thanks to bytes.Buffer:
data, _ := json.Marshal(s)
buf := bytes.NewBuffer(data)
b, _ := ioutil.ReadAll(buf)
fmt.Print(string(b))

unicode being output literally instead of as unicode

I am creating an IRC bot using Go as a first project to get to grips with the language. One of the bot functions is to grab data from the TVmaze API and display in the channel.
I have imported an env package which allows the bot admin to define how the output is displayed.
For example SHOWSTRING="#showname# - #status# – #network.name#"
I am trying to add functionality to it so that the admin can use IRC formatting functionality which is accessed with \u0002 this is bold \u0002 for example.
I have a function which generates the string that is being returned and displayed in the channel.
func generateString(show Show) string {
str := os.Getenv("SHOWSTRING")
r := strings.NewReplacer(
"#ID#", string(show.ID),
"#showname#", show.Name,
"#status#", show.Status,
"#network.name#", show.Network.Name,
)
result := r.Replace(str)
return result
}
From what i have read i think that i need to use the rune datatype instead of string and then converting the runes into a string before being output.
I am using the https://github.com/thoj/go-irceven package for interacting with IRC.
Although i think that using rune is the correct way to go, i have tried a few things that have confused me.
If i add \u0002 to the SHOWSTRING from the env, it returns \u0002House\u0002 - Ended - Fox. I am doing this by con.Privmsg(roomName, tvmaze.ShowLookup('house'))
However if i try con.Privmsg(roomName, "\u0002This should be bold\u0002") it outputs bold text.
What is the best option here? If it is converting the string into runes and then back to a string, how do i go about doing that?
I needed to use strconv.Unquote() on my return in the function.
The new generateString function now outputs the correct string and looks like this
func generateString(show Show) string {
str := os.Getenv("SHOWSTRING")
r := strings.NewReplacer(
"#ID#", string(show.ID),
"#showname#", show.Name,
"#status#", show.Status,
"#network.name#", show.Network.Name,
)
result := r.Replace(str)
ret, err := strconv.Unquote(`"` + result + `"`)
if err != nil {
fmt.Println("Error unquoting the string")
}
return ret
}

Scandinavian characters not working in go-lang go-instagram API bindings

Hi I'm trying to wrap my head around what seems to be a problem with multibyte support in this open source library (https://github.com/carbocation/go-instagram/). I am using the code below to retrieve information about the tag blue in swedish. How ever I get an empty array when trying.
fmt.Println("Starting instagram download.")
client := instagram.NewClient(nil)
client.ClientID = "myid"
media, _, _ := client.Tags.RecentMedia("blå", nil)
fmt.Println(media)
I have tried using the api trough the browser and there are several pictures tagged with the tag. I have also tried using the code snippet with tags in English like blue and that returns the latest pictures as well. I would be glad if any one could explain why this might happen. Id like to update the lib so it supports multi-byte but I haven't got the go knowledge required. Is this a go problem or a problem with the library?
Thank you
The problem is in validTagName():
// Strip out things we know Instagram won't accept. For example, hyphens.
func validTagName(tagName string) (bool, error) {
//\W matches any non-word character
reg, err := regexp.Compile(`\W`)
if err != nil {
return false, err
}
if reg.MatchString(tagName) {
return false, nil
}
return true, nil
}
In Go, \W matches precisely [^0-9A-Za-z_]. This validation check is incorrect.

Resources