Dynamic JSON Decoding - go

I have this function to parse HTTP results:
func (a *Admin) ResponseDecode(structName string, body io.Reader) (interface{}, error) {
content, err := ioutil.ReadAll(body)
if err != nil {
return nil, err
}
switch structName {
case "[]Cat":
var data []Cat
err = json.Unmarshal(content, &data)
if err != nil {
return nil, err
}
return data, err
case "[]Dog":
var data []Dog
err = json.Unmarshal(content, &data)
if err != nil {
return nil, err
}
return data, err
default:
log.Fatal("Can't decode " + structName)
}
return nil, nil
}
I do a type assertion after this method :
parsed, err := a.ResponseDecode("[]Cat", resp.Body)
if err != nil {
return nil, err
}
return parsed.([]Cat), nil
but how can I do to avoid the repetition of the code :
var data []Stuff
err = json.Unmarshal(content, &data)
if err != nil {
return nil, err
}
return data, err
Every time I add an object? Usually I would use generics, but this is Go. What's the good way to do that ?

You are passing in the name of the struct, and then expecting data of that type. Instead, you can simply pass the struct:
var parsed []Cat
err := a.ResponseDecode(&parsed, resp.Body)
where:
func (a *Admin) ResponseDecode(out interface{}, body io.Reader) error {
content, err := ioutil.ReadAll(body)
if err != nil {
return nil, err
}
return json.Unmarshal(content,out)
}
In fact, you can get rid of ResponseDecode function:
var parsed []Cat
err:=json.NewDecoder(body).Decode(&parsed)

Found what I was looking for here.
This function does the job :
func (a *Admin) ResponseDecode(body io.Reader, value interface{}) (interface{}, error) {
content, err := ioutil.ReadAll(body)
if err != nil {
return nil, err
}
err = json.Unmarshal(content, &value)
if err != nil {
return nil, err
}
return value, nil}

Related

the function x509.ParsePKCS8PrivateKey return rsa.privateKey. But can't use in the encryptPKCS1v15 function

const strPrivateKey = "30820b82020100300d06092a864886f70d010101050004820b6c30820b680201000282028100acfc585f43ca36ec2dddc518b5c7d1303b658faec58b634aff16ce4b7930b93a23517f8d9c8a260f4e2eb44b01da5b6588fefe63acb68c15677"
decoded, err := hex.DecodeString(strPrivateKey)
if err != nil {
return ""
}
privateKey, err := x509.ParsePKCS8PrivateKey(decoded)
if err != nil {
return ""
}
encypt, err := rsa.EncryptPKCS1v15(rand.Reader, &privateKey.PublicKey, data)
if err != nil {
fmt.Println(err)
return ""
}
privateKey.PublicKey undefined (type any has no field or method PublicKey)
According to the doc (https://pkg.go.dev/crypto/x509#go1.19.3#ParsePKCS8PrivateKey):
func ParsePKCS8PrivateKey(der []byte) (key any, err error)
...
It returns a *rsa.PrivateKey, a *ecdsa.PrivateKey, or a ed25519.PrivateKey. More types might be supported in the future.
You should use type assertion to check the type of the key:
switch privateKey := privateKey.(type) {
case *rsa.PrivateKey:
// ...
case *ecdsa.PrivateKey:
// ...
case ed25519.PrivateKey:
// ...
default:
panic("unknown key")
}
Since rsa.EncryptPKCS1v15 expects a *rsa.PublicKey, your code can be written like this:
if privateKey, ok := privateKey.(*rsa.PrivateKey); ok {
encypt, err := rsa.EncryptPKCS1v15(rand.Reader, &privateKey.PublicKey, data)
}
BTW, the provided strPrivateKey is invalid (encoding/hex: odd length hex string). You can get some valid private keys from https://github.com/golang/go/blob/1c05968c9a5d6432fc6f30196528f8f37287dd3d/src/crypto/x509/pkcs8_test.go#L52-L124
*correct answer. I resolved privateKey.(rsa.PrivateKey)
decodedString, err := hex.DecodeString(utility.StrPrivateKey)
if err != nil {
return err
}
pkcs8PrivateKey, err := x509.ParsePKCS8PrivateKey(decodedString)
if err != nil {
return err
}
privateKey := pkcs8PrivateKey.(*rsa.PrivateKey)

How to return the errors in golang?

In this code, I am taking the data from the environment variable and using it to send the email to the given address. It only works if I exclude the errors from the code. I returned the errors and used them in middleware because I don't want to use log.Fatal(err) every time. But it gives me two errors.
1. cannot use "" (untyped string constant) as [4]string value in return statement
2. code can not execute unreachable return statement
func LoadEnvVariable(key string) (string, error) {
viper.SetConfigFile(".env")
err := viper.ReadInConfig()
return "", err
value, ok := viper.Get(key).(string)
if !ok {
return "", err
}
return value, nil
}
func Email(value [4]string) ([4]string, error) {
mail := gomail.NewMessage()
myEmail, err := LoadEnvVariable("EMAIL")
return "", err
appPassword, err := LoadEnvVariable("APP_PASSWORD")
return "", err
mail.SetHeader("From", myEmail)
mail.SetHeader("To", myEmail)
mail.SetHeader("Reply-To", value[1])
mail.SetHeader("subject", value[2])
mail.SetBody("text/plain", value[3])
a := gomail.NewDialer("smtp.gmail.com", 587, myEmail, appPassword)
err = a.DialAndSend(mail)
return "", err
return value, nil
}
use this to check error
if err != nil {
return "", err
}
func LoadEnvVariable(key string) (string, error) {
viper.SetConfigFile(".env")
err := viper.ReadInConfig()
if err != nil {
return "", err
}
value, ok := viper.Get(key).(string)
if !ok {
return "", err
}
return value, nil
}
func Email(value [4]string) ([4]string, error) {
mail := gomail.NewMessage()
myEmail, err := LoadEnvVariable("EMAIL")
if err != nil {
return "", err
}
appPassword, err := LoadEnvVariable("APP_PASSWORD")
if err != nil {
return "", err
}
mail.SetHeader("From", myEmail)
mail.SetHeader("To", myEmail)
mail.SetHeader("Reply-To", value[1])
mail.SetHeader("subject", value[2])
mail.SetBody("text/plain", value[3])
a := gomail.NewDialer("smtp.gmail.com", 587, myEmail, appPassword)
err = a.DialAndSend(mail)
if err != nil {
return "", err
}
return value, nil
}
Firstly, to error handle, check if err is nil before returning!
So not:
myEmail, err := LoadEnvVariable("EMAIL")
return "", err
but instead:
myEmail, err := LoadEnvVariable("EMAIL")
if err != nil {
return "", err
}
Secondly, a single string type "" is compatible to a fixed array size of [4]string. It may make your life easier to use named return variables, so you don't need to pass a explicit "zero" value in the event of an error. So define your function signature like so:
func Email(in [4]string) (out [4]string, err error) {
myEmail, err := LoadEnvVariable("EMAIL")
if err != nil {
return // implicitly returns 'out' and 'err'
}
// ...
out = in
return // implicitly returns 'out' and nil 'err'
}

Reuse a method on a struct

I have a method that takes a URL to a yaml file and unmarshals it into a struct. There's nothing unique about it so I'd like to use it on another struct.
type SWConfig struct {
Name string `yaml:"name"`
Version string `yaml:"version"`
BuildType string `yaml:"buildType"`
}
func (c *SWConfig) getConfiguration(url string) {
resp, err := http.Get(url)
if err != nil {
log.Fatalf("ERROR: GET request failed: %s\n", err.Error())
}
if resp.StatusCode != http.StatusOK {
log.Fatalf("ERROR: %v: %s\n", resp.Status, url)
}
source, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("ERROR: could not read response: %s\n", err.Error())
}
err = yaml.Unmarshal(source, c)
if err != nil {
log.Fatalf("ERROR: could not read YAML: %s\n", err.Error())
}
}
var swConfig SWConfig
swConfig.getConfiguration(swURL)
fmt.Println(swConfig.Name)
The other struct would have different fields, but again that shouldn't matter for this method. Is it possible to reuse this method or do I need to convert it to a function?
Use this utility function. It works with a pointer to any type.
// fetchYAML unmarshals the YAML document at url to the value
// pointed to by v.
func fetchYAML(url string, v interface{}) error {
resp, err := http.Get(url)
if err != nil {
return err
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("bad status: %v: %s\n", resp.Status, url)
}
source, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
err = yaml.Unmarshal(source, v)
return err
}
Call it like this:
func (c *SWConfig) getConfiguration(url string) {
if err := fetchYAML(url, c); err != nil {
log.Fatalf("ERROR: get value failed: %s\n", err.Error())
}
}

Reverse proxy in golang opening too many connections towards server at higher rates

I have the following code
type transport struct {
http.RoundTripper
}
func (t *transport) RoundTrip(req *http.Request) (resp *http.Response, err error) {
resp, err = t.RoundTripper.RoundTrip(req)
if err != nil {
return nil, err
}
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
err = resp.Body.Close()
if err != nil {
return nil, err
}
// modify response
return resp, nil
}
// ...
proxy := httputil.NewSingleHostReverseProxy(target)
proxy.Transport = &transport{http.DefaultTransport}
I have tried setting MaxConnsPerHost, this parameter works with value 2 alone, any other value it doesnt work

Trouble getting content type of file in Go

I have a function in which I take in a base64 string and get the content of it (PDF or JPEG).
I read in the base64 content, convert it to bytes and decode it into the file that it is.
I then create a file where I will output the decoded file (JPEG or PDF).
Then I write the bytes to it.
Then I call my GetFileContentType on it and it returns to me an empty string.
If I run the functions separately, as in I first the first function to create the decoded file, and end it. And then call the second function to get the content type, it works and returns it as JPEG or PDF.
What am I doing wrong here?
And is there a better way to do this?
func ConvertToJPEGBase64(
src string,
dst string,
) error {
b, err := ioutil.ReadFile(src)
if err != nil {
return err
}
str := string(b)
byteArray, err := base64.StdEncoding.DecodeString(str)
if err != nil {
return err
}
f, err := os.Create(dst)
if err != nil {
return err
}
if _, err := f.Write(byteArray); err != nil {
return err
}
f.Sync()
filetype, err := client.GetFileContentType(f)
if err != nil {
return err
}
if strings.Contains(filetype, "jpeg") {
// do something
} else {
// do something else
}
return nil
}
// GetFileContentType tells us the type of file
func GetFileContentType(out *os.File) (string, error) {
// Only the first 512 bytes are used to sniff the content type.
buffer := make([]byte, 512)
_, err := out.Read(buffer)
if err != nil {
return "", err
}
contentType := http.DetectContentType(buffer)
return contentType, nil
}
The problem is that GetFileContentType reads from the end of the file. Fix this be seeking back to the beginning of the file before calling calling GetFileContentType:
if _, err := f.Seek(io.SeekStart, 0); err != nil {
return err
}
A better fix is to use the file data that's already in memory. This simplifies the code to the point where there's no need for the GetFileContentType function.
func ConvertToJPEGBase64(
src string,
dst string,
) error {
b, err := ioutil.ReadFile(src)
if err != nil {
return err
}
str := string(b)
byteArray, err := base64.StdEncoding.DecodeString(str)
if err != nil {
return err
}
f, err := os.Create(dst)
if err != nil {
return err
}
defer f.Close() // <-- Close the file on return.
if _, err := f.Write(byteArray); err != nil {
return err
}
fileType := http.DetectContentType(byteArray) // <-- use data in memory
if strings.Contains(fileType, "jpeg") {
// do something
} else {
// do something else
}
return nil
}
More code can be eliminated by using ioutil.WriteFile:
func ConvertToJPEGBase64(src, dst string) error {
b, err := ioutil.ReadFile(src)
if err != nil {
return err
}
byteArray, err := base64.StdEncoding.DecodeString(string(b))
if err != nil {
return err
}
if err := ioutil.WriteFile(dst, byteArray, 0666); err != nil {
return err
}
fileType := http.DetectContentType(byteArray)
if strings.Contains(fileType, "jpeg") {
// do something
} else {
// do something else
}
return nil
}

Resources