I'm trying to deal with some error descriptions from this library because I need them to be nested JSON objects.
The errors seem to be an array originally, like this:
["String length must be greater than or equal to 3","Does not match format 'email'"]
I needed that to also include the field name of the containing error:
["Field1: String length must be greater than or equal to 3","Email1: Does not match format 'email'"]
After that I need to split each array value by colon : so I can have the field name and error description in separate variables like slice[0] and slice[1].
With that I want to make a nested JSON object like so:
{
"errors": {
"Field1": "String length must be greater than or equal to 3",
"Email1": "Does not match format 'email'"
}
}
This is my way of trying to achieve this:
var errors []string
for _, err := range result.Errors() {
// Append the errors into an array that we can use to split later
errors = append(errors, err.Field() + ":" + err.Description())
}
// Make the JSON map we want to append values to
resultMap := map[string]interface{}{
"errors": map[string]string {
"Field1": "",
"Email1": ""
},
}
// So we actually can use the index keys when appending
resultMapErrors, _ := resultMap["errors"].(map[string]string)
for _, split := range errors {
slice := strings.Split(split, ":")
for _, appendToMap := range resultMapErrors {
appendToMap[slice[0]] = slice[1] // append it like so?
}
}
finalErrors, _ := json.Marshal(resultMapErrors)
fmt.Println(string(finalErrors))
But this throws the errors
main.go:59:28: non-integer string index slice[0]
main.go:59:39: cannot assign to appendToMap[slice[0]]
Any clue how I can achieve this?
var errors = make(map[string]string)
for _, err := range result.Errors() {
errors[err.Field()] = err.Description()
}
// Make the JSON map we want to append values to
resultMap := map[string]interface{}{
"errors": errors,
}
finalErrors, _ := json.Marshal(resultMap)
fmt.Println(string(finalErrors))
Related
I am using apache-arrow/go to read parquet data.
I can parse the data to table by using apach-arrow.
reader, err := ipc.NewReader(buf, ipc.WithAllocator(alloc))
if err != nil {
log.Println(err.Error())
return nil
}
defer reader.Release()
records := make([]array.Record, 0)
for reader.Next() {
rec := reader.Record()
rec.Retain()
defer rec.Release()
records = append(records, rec)
}
table := array.NewTableFromRecords(reader.Schema(), records)
Here, i can get the column info from table.Colunmn(index), such as:
for i, _ := range table.Schema().Fields() {
a := table.Column(i)
log.Println(a)
}
But the Column struct is defined as
type Column struct {
field arrow.Field
data *Chunked
}
and the println result is like
["WARN" "WARN" "WARN" "WARN" "WARN" "WARN" "WARN" "WARN" "WARN" "WARN"]
However, this is not a string or slice. Is there anyway that i can get the data of each column with string type or []interface{} ?
Update:
I find that i can use reflect to get the element from col.
log.Println(col.(*array.Int64).Value(0))
But i am not sure if this is the recommended way to use it.
When working with Arrow data, there's a couple concepts to understand:
Array: Metadata + contiguous buffers of data
Record Batch: A schema + a collection of Arrays that are all the same length.
Chunked Array: A group of Arrays of varying lengths but all the same data type. This allows you to treat multiple Arrays as one single column of data without having to copy them all into a contiguous buffer.
Column: Is just a Field + a Chunked Array
Table: A collection of Columns allowing you to treat multiple non-contiguous arrays as a single large table without having to copy them all into contiguous buffers.
In your case, you're reading multiple record batches (groups of contiguous Arrays) and treating them as a single large table. There's a few different ways you can work with the data:
One way is to use a TableReader:
tr := array.NewTableReader(tbl, 5)
defer tr.Release()
for tr.Next() {
rec := tr.Record()
for i, col := range rec.Columns() {
// do something with the Array
}
}
Another way would be to interact with the columns directly as you were in your example:
for i := 0; i < table.NumCols(); i++ {
col := table.Column(i)
for _, chunk := range col.Data().Chunks() {
// do something with chunk (an arrow.Array)
}
}
Either way, you eventually have an arrow.Array to deal with, which is an interface containing one of the typed Array types. At this point you are going to have to switch on something, you could type switch on the type of the Array itself:
switch arr := col.(type) {
case *array.Int64:
// do stuff with arr
case *array.Int32:
// do stuff with arr
case *array.String:
// do stuff with arr
...
}
Alternately, you could type switch on the data type:
switch col.DataType().ID() {
case arrow.INT64:
// type assertion needed col.(*array.Int64)
case arrow.INT32:
// type assertion needed col.(*array.Int32)
...
}
For getting the data out of the array, primitive types which are stored contiguously tend to have a *Values method which will return a slice of the type. For example array.Int64 has Int64Values() which returns []int64. Otherwise, all of the types have .Value(int) methods which return the value at a particular index as you showed in your example.
Hope this helps!
Make sure you use v9
(import "github.com/apache/arrow/go/v9/arrow") because it have implemented json.Marshaller (from go-json)
Use "github.com/goccy/go-json" for Marshaler (because of this)
Then you can use TableReader to Marshal it then Unmarshal with type []any
In your example maybe look like this:
import (
"github.com/apache/arrow/go/v9/arrow"
"github.com/apache/arrow/go/v9/arrow/array"
"github.com/apache/arrow/go/v9/arrow/memory"
"github.com/goccy/go-json"
)
...
tr := array.NewTableReader(tabel, 6)
defer tr.Release()
// fmt.Printf("tbl.NumRows() = %+v\n", tbl.NumRows())
// fmt.Printf("tbl.NumColumn = %+v\n", tbl.NumCols())
// keySlice is for sorting same as data source
keySlice := make([]string, 0, tabel.NumCols())
res := make(map[string][]any, 0)
var key string
for tr.Next() {
rec := tr.Record()
for i, col := range rec.Columns() {
key = rec.ColumnName(i)
if res[key] == nil {
res[key] = make([]any, 0)
keySlice = append(keySlice, key)
}
var tmp []any
b2, err := json.Marshal(col)
if err != nil {
panic(err)
}
err = json.Unmarshal(b2, &tmp)
if err != nil {
panic(err)
}
// fmt.Printf("key = %s\n", key)
// fmt.Printf("tmp = %+v\n", tmp)
res[key] = append(res[key], tmp...)
}
}
fmt.Println("res", res)
This question already has answers here:
Access struct property by name
(5 answers)
Golang dynamic access to a struct property
(2 answers)
How to access to a struct parameter value from a variable in Golang
(1 answer)
Closed 9 months ago.
Came from javascript background, and just started with Golang. I am learning all the new terms in Golang, and creating new question because I cannot find the answer I need (probably due to lack of knowledge of terms to search for)
I created a custom type, created an array of types, and I want to create a function where I can retrieve all the values of a specific key, and return an array of all the values (brands in this example)
type Car struct {
brand string
units int
}
....
var cars []Car
var singleCar Car
//So i have a loop here and inside the for-loop, i create many single cars
singleCar = Car {
brand: "Mercedes",
units: 20
}
//and i append the singleCar into cars
cars = append(cars, singleCar)
Now what I want to do is to create a function that I can retrieve all the brands, and I tried doing the following. I intend to have key as a dynamic value, so I can search by specific key, e.g. brand, model, capacity etc.
func getUniqueByKey(v []Car, key string) []string {
var combined []string
for i := range v {
combined = append(combined, v[i][key])
//this line returns error -
//invalid operation: cannot index v[i] (map index expression of type Car)compilerNonIndexableOperand
}
return combined
//This is suppose to return ["Mercedes", "Honda", "Ferrari"]
}
The above function is suppose to work if i use getUniqueByKey(cars, "brand") where in this example, brand is the key. But I do not know the syntaxes so it's returning error.
Seems like you're trying to get a property using a slice accessor, which doesn't work in Go. You'd need to write a function for each property. Here's an example with the brands:
func getUniqueBrands(v []Car) []string {
var combined []string
tempMap := make(map[string]bool)
for _, c := range v {
if _, p := tempMap[c.brand]; !p {
tempMap[c.brand] = true
combined = append(combined, c.brand)
}
}
return combined
}
Also, note the for loop being used to get the value of Car here. Go's range can be used to iterate over just indices or both indices and values. The index is discarded by assigning to _.
I would recommend re-using this code with an added switch-case block to get the result you want. If you need to return multiple types, use interface{} and type assertion.
Maybe you could marshal your struct into json data then convert it to a map. Example code:
package main
import (
"encoding/json"
"fmt"
)
type RandomStruct struct {
FieldA string
FieldB int
FieldC string
RandomFieldD bool
RandomFieldE interface{}
}
func main() {
fieldName := "FieldC"
randomStruct := RandomStruct{
FieldA: "a",
FieldB: 5,
FieldC: "c",
RandomFieldD: false,
RandomFieldE: map[string]string{"innerFieldA": "??"},
}
randomStructs := make([]RandomStruct, 0)
randomStructs = append(randomStructs, randomStruct, randomStruct, randomStruct)
res := FetchRandomFieldAndConcat(randomStructs, fieldName)
fmt.Println(res)
}
func FetchRandomFieldAndConcat(randomStructs []RandomStruct, fieldName string) []interface{} {
res := make([]interface{}, 0)
for _, randomStruct := range randomStructs {
jsonData, _ := json.Marshal(randomStruct)
jsonMap := make(map[string]interface{})
err := json.Unmarshal(jsonData, &jsonMap)
if err != nil {
fmt.Println(err)
// panic(err)
}
value, exists := jsonMap[fieldName]
if exists {
res = append(res, value)
}
}
return res
}
How do I append output from a twitter search to the field Data in the SearchTwitterOutput{} struct.
Thanks!
I am using a twitter library to search twitter base on a query input. The search returns an array of strings(I believe), I am able to fmt.println the data but I need the data as a struct.
type SearchTwitterOutput struct {
Data string
}
func (SearchTwitter) execute(input SearchTwitterInput) (*SearchTwitterOutput, error) {
credentials := Credentials{
AccessToken: input.AccessToken,
AccessTokenSecret: input.AccessTokenSecret,
ConsumerKey: input.ConsumerKey,
ConsumerSecret: input.ConsumerSecret,
}
client, err := GetUserClient(&credentials)
if err != nil {
return nil, err
}
// search through the tweet and returns a
search, _ , err := client.Search.Tweets(&twitter.SearchTweetParams{
Query: input.Text,
})
if err != nil {
println("PANIC")
panic(err.Error())
return &SearchTwitterOutput{}, err
}
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
}
return &SearchTwitterOutput{
Data: "test", //data is a string for now it can be anything
}, nil
}
//Data field is a string type for now it can be anything
//I use "test" as a placeholder, bc IDK...
Result from fmt.Printf("Tweet %d - %s\n", k, v.Text):
Tweet 0 - You know I had to do it to them! #JennaJulien #Jenna_Marbles #juliensolomita #notjulen Got my first hydroflask ever…
Tweet 1 - RT #brenna_hinshaw: I was in J2 today and watched someone fill their hydroflask with vanilla soft serve... what starts here changes the wor…
Tweet 2 - I miss my hydroflask :(
This is my second week working with go and new to development. Any help would be great.
It doesn't look like the client is just returning you a slice of strings. The range syntax you're using (for k, v := range search.Statuses) returns two values for each iteration, the index in the slice (in this case k), and the object from the slice (in this case v). I don't know the type of search.Statuses - but I know that strings don't have a .Text field or method, which is how you're printing v currently.
To your question:
Is there any particular reason to return just a single struct with a Data field rather than directly returning the output of the twitter client?
Your function signature could look like this instead:
func (SearchTwitter) execute(input SearchTwitterInput) ([]<client response struct>, error)
And then you could operate on the text in those objects in wherever this function was called.
If you're dead-set on placing the data in your own struct, you could return a slice of them ([]*SearchTwitterOutput), in which case you could build a single SearchTwitterOutput in the for loop you're currently printing the tweets in and append it to the output list. That might look like this:
var output []*SearchTwitterOutput
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
output = append(output, &SearchTwitterOutput{
Data: v.Text,
})
}
return output, nil
But if your goal really is to return all of the results concatenated together and placed inside a single struct, I would suggest building a slice of strings (containing the text you want), and then joining them with the delimiter of your choosing. Then you could place the single output string in your return object, which might look something like this:
var outputStrings []string
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
outputStrings = append(outputStrings, v.Text)
}
output = strings.Join(outputStrings, ",")
return &SearchTwitterOutput{
Data: output,
}, nil
Though I would caution, it might be tricky to find a delimiter that will never show up in a tweet..
I have problem how to print a variable for, in outside for, in Go?
I'm using library GJSON gjson
I have try many way , I just entered the global variable but just appear final index,
like:
datePriodGlobal = DatePeriod.String()
and
datePriodGlobal = DatePeriod.String()
another way I try but appear just final index too, like below:
tempPayments:= "Envelope.Body.GetCustomReportResponse.GetCustomReportResult.ContractSummary.PaymentCalendarList.PaymentCalendar."
resultMapPriodTest := gjson.Get(jsonString,tempPayments + "#.Date")
resultContractsSubmittedTest := gjson.Get(jsonString, tempPayments + "#.ContractsSubmitted")
var datePriodGlobal string
for _, DatePeriod := range resultMapPriodTest.Array()[1:13] {
datePriodGlobal = fmt.Sprintf("%s", DatePeriod.String())
}
var contractsSubmittedGlobal string
for _, ContractsSubmitted := range resultContractsSubmittedTest.Array()[1:13]{
contractsSubmittedGlobal = fmt.Sprintf("%s", ContractsSubmitted.String())
}
fmt.Printf("%s | %s \t|",datePriodGlobal, contractsSubmittedGlobal)
}
I have json like this:
I will suggest just iterate over the PaymentCalendar as a slice of JSON objects rather than querying each field using the indexes as their pseudo-ids.
Here is a simple demonstration:
func main() {
jsonString := `
{
"PaymentCalendarList": {
"PaymentCalendar": [
{"ContractSubmitted": 10,
"Date": "2018-01-01T01:01:01"},
{"ContractSubmitted": 20,
"Date": "2018-01-01T02:02:02"},
{"ContractSubmitted": 30,
"Date": "2018-01-01T03:03:03"}
{"ContractSubmitted": 40,
"Date": "2018-01-01T04:04:04"}
{"ContractSubmitted": 50,
"Date": "2018-01-01T05:05:05"}
]
}
}`
result := gjson.Get(jsonString, "PaymentCalendarList.PaymentCalendar")
for _, paymentCal := range result.Array()[0:3] {
date := paymentCal.Get("Date")
contractSubmit := paymentCal.Get("ContractSubmitted")
fmt.Printf("%s | %s\n", date, contractSubmit)
}
}
Playground
"Cannot use 'DatePeriod' (type Result) as type string in assignment"
So, the variable DatePeriod is a Result type, not a String. You're specifying you want to print a string with %s, but not giving fmt.Sprintf a string, causing that error. The Sprintf is unnecessary if the value given was already a String.
Looking at gjson.go, the Result type has a String() method, so you'd want instead DatePeriod.String().
EDIT:
From your latest edit, I think I see your second issue. Your loops replace the ...Global string variables each time, so you'll only ever get the last value in the slice you've passed to range. Since your slices are identical in length, you might be better off with something like this:
resultMapPriodTest := gjson.Get(jsonString,tempPayments + "#.Date")
resultContractsSubmittedTest := gjson.Get(jsonString, tempPayments + "#.ContractsSubmitted")
dateArray := resultMapPriodTest.Array()[1:13]
contractsArray := resultContractsSubmittedTest.Array()[1:13]
for i := 0; i<len(dateArray); i++ {
d := dateArray[i].String()
c := contractsArray[i].String()
fmt.Printf("%s | %s \t|", d, c)
}
I'm having a problem with creating an empty map and append new data to it while looping on another map.
this is the error i'm getting on my IDE.
here's my data struct to be added to the map.
type Outcome struct {
QuestionIndex string
ChoiceIndex int64
Correct bool
}
func createEntryOutcome(e *entry.Entry) map[string]interface{} {
entryPicks := e.Live.Picks
outcomes := make(map[string]interface{})
for idx, pick := range entryPicks {
mappedPick := pick.(map[string]interface{})
outcomes = append(outcomes, Outcome{
QuestionIndex: idx,
ChoiceIndex: mappedPick["index"].(int64),
Correct: mappedPick["correct"].(bool),
})
}
return outcomes
}
i basically want something like below to be saved in the database.
[
{
qIndex: "1",
cIndex: 1,
correct: false,
},
{
qIndex: "1",
cIndex: 1,
correct: false,
},
]
im new to golang and any help is appreciated. thanks
As the error clearly says:
first argument to append must be slice; have map[string]interface{}
which means you need to create a slice before appending the data to outcomes which is actually slice of outcomes, like you have mentioned in the output you want.
The append function appends the elements x to the end of the slice s,
and grows the slice if a greater capacity is needed.
Create a slice of outcomes and then append the data from entryPicks to that slice:
outcomes := make([]map[string]interface{})
for idx, pick := range entryPicks {
mappedPick := pick.(map[string]interface{})
outcomes = append(outcomes, Outcome{
QuestionIndex: idx,
ChoiceIndex: mappedPick["index"].(int64),
Correct: mappedPick["correct"].(bool),
})
}
which will let you provide the outcome you want.
type Outcome struct {
QuestionIndex string
ChoiceIndex int64
Correct bool
}
func createEntryOutcome(e *entry.Entry) map[string]interface{} {
entryPicks := e.Live.Picks
var outcomes []Outcome
for idx, pick := range entryPicks {
mappedPick := pick.(map[string]interface{})
outcomes = append(outcomes, Outcome{
QuestionIndex: idx,
ChoiceIndex: mappedPick["index"].(int64),
Correct: mappedPick["correct"].(bool),
})
}
return outcomes
}
change outcomes := make(map[string]interface{}) to var outcomes []Outcome