Error embedding "\n" into a string literal - go

I need to decode a JSON string which has "\n" in it:
[
{"Name":"Neo", "Message":"Hi\n:Hello everyone"},
{"Name":"Sam","Messsage":"Hello\nEveery\nOne"}
]
I use the Golang code below:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
Messages []string `json:"Name,omitempty"`
}
func main() {
s := "[{\"Name\":\"Neo\", \"Message\":\"Hi\n:Hello everyone\"}, {\"Name\":\"Sam\",\"Messsage\":\"Hello\nEveery\nOne\"}]"
var pro Person
err := json.Unmarshal([]byte(s), &pro)
if err == nil {
fmt.Printf("%+v\n", pro)
} else {
fmt.Println(err)
fmt.Printf("%+v\n", err)
}
}
But I get the error:
ERROR invalid character '\n' in string literal

There are a few of issues here. The first is that newline is not allowed in a JSON string. Use the two bytes \n to specify a newline, not an actual newline. If you use an interpreted string literal, then the \ must be quoted with a \. Example:
"Hello\\nWorld"
No quoting is required in a raw string literal:
`Hello\nWorld`
The next issue is that JSON value is an array of object values. To handle the array, unmarshal to a slice:
var pro []Person
err := json.Unmarshal([]byte(s), &pro)
To handle the objects, define Person as a struct:
type Person struct {
Name string
Message string
}
working example on the playground.

Use backtick, like this working sample code:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
Name string `json:"Name"`
Message string `json:"Message"`
}
func main() {
s := `
[
{"Name":"Neo", "Message":"Hi\n:Hello everyone"},
{"Name":"Sam", "Message":"Hello\nEvery\nOne"}
]
`
var pro []Person
err := json.Unmarshal([]byte(s), &pro)
if err != nil {
fmt.Println(err)
}
fmt.Printf("%q\n", pro)
fmt.Println()
fmt.Println(pro)
}
output:
[{"Neo" "Hi\n:Hello everyone"} {"Sam" "Hello\nEvery\nOne"}]
[{Neo Hi
:Hello everyone} {Sam Hello
Every
One}]

Related

Reading and Unmarshalling API results in Golang

In the below program I'm extracting some data from an API.
It outputs a rather complex data.
When I ioutil.ReadAll(resp.Body), the result is of type []uint8.
If I try to read the results, its just a random array of integers.
However, I'm able to read it if I convert it to string using string(diskinfo)
But I want to use this in a Struct and having trouble unmarshalling.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"reflect"
)
type ApiResults struct {
results []struct {
statement_id int `json.statement_id`
series []struct {
name string `json.name`
tags struct {
host string `json.host`
}
columns []string `json.columns`
values []interface{} `json.values`
}
}
}
func main() {
my_url := "my_url"
my_qry := fmt.Sprintf("my_query")
resp, err := http.Get(my_url + url.QueryEscape(my_qry))
if err != nil {
fmt.Printf("ERROR: %v\n", err)
} else {
fmt.Println(reflect.TypeOf(resp))
diskinfo, _ := ioutil.ReadAll(resp.Body)
fmt.Println(reflect.TypeOf((diskinfo)))
fmt.Println(diskinfo)
fmt.Println(string(diskinfo))
diskinfo_string := string(diskinfo)
data := ApiResults{}
json.Unmarshal([]byte(diskinfo_string), &data)
//fmt.Printf("Values = %v\n", data.results.series.values)
//fmt.Printf("Server = %v\n", data.results.series.tags.host)
}
}
If I view the data as a string, I get this (formatted):
{"results":[
{"statement_id":0,
"series":[
{"name":"disk",
"tags":{"host":"myServer1"},
"columns":["time","disk_size"],
"values":[["2021-07-07T07:53:32.291490387Z",1044]]},
{"name":"disk",
"tags":{"host":"myServer2"},
"columns":["time","disk_size"],
"values":[["2021-07-07T07:53:32.291490387Z",1046]]}
]}
]}
I think my Apireturn struct is also structured incorrectly because the API results have info for multiple hosts.
But first, I doubt if the data has to be sent in a different format to the struct. Once I do this, I can probably try to figure out how to read from the Struct next.
The ioutil.ReadAll already provides you the data in the type byte[]. Therefore you can just call json.Unmarshal passing it as a parameter.
import (
"encoding/json"
"io/ioutil"
"net/http"
)
func toStruct(res *http.Response) (*ApiResults, error) {
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
defer res.Body.Close()
data := ApiResults{}
if err := json.Unmarshal(body, &data); err != nil {
return nil, err
}
return data, nil
}
There also seems to be an issue with your struct. The correct way to use struct tags is as follows. Plus, fields need to be exported for the json tag (used by json.Umarshal) to work – starting with uppercase will do it.
type ApiResults struct {
Results []struct {
StatementId int `json:"statement_id"`
Series []struct {
Name string `json:"name"`
Tags struct {
Host string `json:"host"`
} `json:"tags"`
Columns []string `json:"columns"`
Values []interface{} `json:"values"`
} `json:"series"`
} `json:"results"`
}

Go YAML parsing: mandatory fields

Summary:
I need to parse data in YAML format into golang struct. It there a way (library, attributes) to make some of the fields mandatory, i.e. to make Unmarshal function return the error in case if some field doesn't exist?
Example what is wanted:
Unmarshal function in this code should raise an error because input data doesn't contain 'b' field.
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type TestStruct struct {
FieldA string `yaml:"a"`
FieldB string `yaml:"b"`
}
func main() {
input := []byte(`{a: 1}`)
var output TestStruct
_ = yaml.Unmarshal(input, &output)
}
You can use this library's HasZero method to check wether there are any missing values in a struct. This will return true or false depending wether the struct is completely filled or not. Please see the playground example to get an idea.
But if you specifically need to tell what field is missing, you need to check wether the value is nil like in the example below.
package main
import (
"fmt"
"errors"
"gopkg.in/yaml.v2"
)
type TestStruct struct {
FieldA string `yaml:"a"`
FieldB string `yaml:"b"`
}
func main() {
input := []byte(`{a: 1}`)
var output TestStruct
if err := output.ParseFromFile(input); err != nil {
fmt.Println(err)
}
fmt.Println(output)
}
func (output *TestStruct) ParseFromFile(data []byte) error {
if err := yaml.Unmarshal(data, output); err != nil {
return err
}
if output.FieldA == "" {
return errors.New("Blank Field A")
}
if output.FieldB == "" {
return errors.New("Blank Field B")
}
return nil
}
Playground example if you need to specifically return an error

Mixed case JSON keys don't Marshal as expected, return error

Code below generates this error json: cannot unmarshal number into Go struct field TMP.a of type string
package main
import (
"encoding/json"
"fmt"
)
var b = []byte(`{"a": "str", "A": 123}`)
type TMP struct {
// A interface{} `json:"a"`
A string `json:"a"`
// A int `json:"A"`
}
func main() {
var tmp TMP
err := json.Unmarshal(b, &tmp)
if err != nil {
fmt.Println(err)
}
}
I've read through https://golang.org/pkg/encoding/json/#Marshal and cannot find anything that states this shouldn't work. What am I missing?
It's been asked that I clarify, and rightly so. What I'm really wondering is why when I use JSON with only 2 keys that differ in uppercase/lowercase why Unmarshal is not keeping the case I've provided in the struct json tag.
First of all change the names of your fields. Since they have same name only uppercase A and lowercase a. So when go try to marshal the fields it is unable to recognize between the fields.
package main
import (
"encoding/json"
"fmt"
)
var b = []byte(`{"a": "str", "B": 123}`)
type TMP struct {
// A interface{} `json:"a"`
A string `json:"A"`
B int `json:"B"`
}
func main() {
var tmp TMP
err := json.Unmarshal(b, &tmp)
if err != nil {
fmt.Println(err)
}
fmt.Println(tmp)
}
As the error says
json: cannot unmarshal string into Go struct field TMP.A of type int
even if you try to pass both fields like below it will give same error
var b = []byte(`{"a": "str", "A": 123}`)
type TMP struct {
// A interface{} `json:"a"`
a string `json:"a"`
A int `json:"A"`
}
The real problem is that you're trying to unmarshal int to a string field. Error will occur even if you remove the "a": "str", and will work fine if you double quote the 123
What is a little funny, however, is that the case is ignored with only one field. Changing your struct to:
type TMP struct {
// A interface{}json:"a"
A stringjson:"a"
B intjson:"A"
}
also works.

Use string as struct value

I have this code. What I need is to get the transaction details from the transaction ID returned from blockchain
package main
import (
"encoding/base64"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"strings"
)
type Transaction struct {
Bid string `json:"bid"`
Fun string `json:"fun"`
ID string `json:"id"`
Timestamp string `json:"timestamp"`
TraderA string `json:"traderA"`
TraderB string `json:"traderB"`
Seller string `json:"seller"`
PointAmount string `json:"pointAmount"`
PrevTransactionID string `json:"prevTransactionId"`
}
type AllTxs struct {
TXs []Transaction `json:"tx"`
}
type Transact struct {
Cert string `json:"cert"`
ChaincodeID string `json:"chaincodeID"`
Nonce string `json:"nonce"`
Payload string `json:"payload"`
Signature string `json:"signature"`
Timestamp string `json:"nanos"`
Txid string `json:"txid"`
Type int `json:"type"`
}
func main() {
resp, err := http.Get("http://blockchain_transactions_url/trans_id")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
byteArray := []byte(body)
var t Transact
json.Unmarshal(byteArray, &t)
//I get all the values base64 encoded
st, err := base64.StdEncoding.DecodeString(t.Payload)
if err != nil {
log.Fatal(err)
}
trd := string(st)
sp := strings.Split(trd, "\n")
result := strings.Join(sp, ",")
res := strings.Replace(result, ",", `", "`, -1)
ret := strings.Replace(res, `", "`, `{"`, 1) + `"}`
byteA := []byte(ret)
var tf AllTxs
json.Unmarshal(byteA, &tf)
//the tf...
ref := Transaction{}
fmt.Println(ref.Id)
}
the t.Payload I get is
"CsYBCAESgwESgAE3ZjFhY2Y2MTgxMGRhODMyMTA5NjZiNGYzNjc2NWU5NmIxY2Q0OTliODkyNmY0MDU0YWQ5NzhlNzhkZjczMDRhOGZlMDM1ZjZhYTBhODE2YzdmNjFlNGZkZDQ1MjM4M2Q5ZmU5ZDQxNmIyZGI4YTE1YmRkMjAzZmU2N2I5OTYyZho8ChBpbml0X3RyYW5zYWN0aW9uCgYwMDAxMTcKA2dpbwoEbW9oYQoBNQoCMTIKBTk4NzczCgcyMDE3NDIy"
the tf I get is
{"??7f1acf61810da83210966b4f36765e96b1cd499b8926f4054ad978e78df7304a8fe035f6aa0a816c7f61e4fdd452383d9fe9d416b2db8a15bdd203fe67b9962f<", "init_transaction", "000117", "gio", "moha", "5", "12", "98773", "2017422"}
At the last how can I get the JSON of Transaction/AllTxs type?
Given the comment, it looks like you are trying to unmarshal the string as JSON, but as it stands, the string is not valid JSON; if it were, you would be able to unmarshal it as follows, ignoring the fact that the fields to not appear to match up with the desired structure;
package main
import (
"encoding/json"
"fmt"
)
type Transaction struct {
Bin string `json:"bin"`
Fun string `json:"fun"`
ID string `json:"id"`
Timestamp string `json:"timestamp"`
TraderA string `json:"traderA"`
TraderB string `json:"traderB"`
Seller string `json:"seller"`
PointAmount string `json:"pointAmount"`
PrevTransactionID string `json:"prevTransactionId"`
}
func main() {
snippet := `{
"bin": "7f1acf61810da83210966b4f36765e96b1cd499b8926f4054ad978e78df7304a8fe035f6aa0a816c7f61e4fdd452383d9fe9d416b2db8a15bdd203fe67b9962f",
"fun": "init_transaction",
"id": "000117",
"timestamp": "gio",
"traderA": "moha",
"traderB": "5",
"seller": "12",
"pointAmount": "98773",
"prevTransactionId": "2017422"
}`
t := Transaction{}
json.Unmarshal([]byte(snippet), &t)
fmt.Println(t)
}

golang json marshal converts tags to utf symbols

How to prevent converting <p> to \u003cp\u003e in json.Marshal ?
package main
import (
"encoding/json"
"fmt"
)
type User struct {
Name string
}
func main() {
user := User{Name: "<p>Frank</p>"}
b, err := json.Marshal(&user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(user)
fmt.Println(string(b))
}
It outputs:
{<p>Frank</p>}
{"Name":"\u003cp\u003eFrank\u003c/p\u003e"}
playground
This is already explained in this question. In short - you can create your own Marshaller implementation if you need to preserve these characters, but even encoded the json output will be a perfectly valid one and should not break anything.

Resources