Handling query results in Golang - go

Consider the following method which executes a gremlin query in Go and then interprets or parses the results.
func (n NeptuneGremlinGraph) Query(assetID string, version string, entityID string) ([]hz.Component, error) {
defer utils.TimeTracker(time.Now(), fmt.Sprintf("Graph Query"))
g := gremlin.Traversal_().WithRemote(n.connection)
anonT := gremlin.T__
results, err := g.V(makeId(assetID, version, entityID)).
Repeat(anonT.As("a").InE().OutV().SimplePath()).
Emit().Until(anonT.OutE().Count().Is(0)).
Filter(anonT.HasLabel("Component")).
Project("entity", "component").
By(anonT.Select("a").ElementMap()).
By(anonT.ElementMap()).
ToList()
if err != nil {
return nil, err
}
cnt := 0
for _, r := range results {
var entityID, componentID, value string
if m, ok := r.Data.(map[any]any); ok {
if entity, ok := m["entity"]; ok {
if entity, ok := entity.(map[any]any); ok {
if id, ok := entity["id"]; ok {
if id, ok := id.(string); ok {
_, _, entityID = splitId(id)
}
}
}
}
if component, ok := m["component"]; ok {
if component, ok := component.(map[any]any); ok {
if id, ok := component["component_id"]; ok {
if id, ok := id.(string); ok {
componentID = id
}
}
if v, ok := component["value"]; ok {
if v, ok := v.(string); ok {
value = v
}
}
}
}
log.Printf("%s, %s, %s\n", entityID, componentID, value)
} else {
log.Printf("not a map\n")
}
}
log.Printf("path cnt = %d\n", cnt)
return nil, nil
}
Obviously I could add helper methods to clean up the query processing code. But either way the query processing code has to deal with multiple layers of map[any]any and any values.
Am I missing some methods in the driver Result object that make this easier?

The Go GLV does not have any built in tools to assist in traversing maps. I would suggest not using the ElementMap() step if you do not need the full map. Since it appears that the only data you are looking for is the id of both “entity” and “component” as well as the component value, you could simplify your result by using a traversal which only selects these items, instead of the full element maps. The following is an example from gremlin console doing something similar to this using a sample dataset:
gremlin> g.V().repeat(__.as("a").inE().outV().simplePath()).emit().until(__.outE().count().is(0)).filter(__.hasLab
el("person")).project("entityID", "personID", "personValue").by(__.select("a").id()).by(__.id()).by(__.values()).toList()
==>{entityID=2, personID=1, personValue=marko}
==>{entityID=3, personID=1, personValue=marko}
==>{entityID=3, personID=4, personValue=josh}
==>{entityID=3, personID=6, personValue=peter}
==>{entityID=4, personID=1, personValue=marko}
==>{entityID=4, personID=1, personValue=marko}
==>{entityID=5, personID=4, personValue=josh}
==>{entityID=4, personID=1, personValue=marko}

This cleans things up, but is obviously not safe, and could lead to panics.
for _, r := range results {
var entityID, componentID, value string
if m, ok := r.Data.(map[any]any); ok {
_, _, entityID = splitId(m["entity"].(map[any]any)["id"].(string))
componentID = m["component"].(map[any]any)["component_id"].(string)
value = m["component"].(map[any]any)["value"].(string)
components = append(components, hz.Component{
EntityID: entityID,
ComponentID: componentID,
Value: value,
})
} else {
log.Printf("not a map\n")
}
}

Related

Showing the error after executing the entire code

I have a loop in which the values are sorted, as well as the comparison of these values. If there are identical values, they are removed from the same array of values. After that, I want to do the following so that the code continues to run until the end. And after that, inform users that these values (which were deleted) were not added. But I wrote the code, so this causes the execution of further code to stop. Tell me how to make it right so that even if the same values meet, the code works to the end and only then output messages about these identical values.
In addition: I give the structure in json without duplicates. But I also want to give information that errors have occurred. Tell me how to implement this moment correctly
type Result struct {
Result int `json:"result"`
Message string `json:"message"`
Status []string
}
var result Result
var searchProject = make([]struct{
ProjectId string
}, 0)
var query string
switch Type {
case sku:
query = fmt.Sprintf(`
SELECT s.project_id
FROM sku.sku_projects s
WHERE s.deleted_at IS NULL
AND s.sku_id = %d
AND s.project_id IN (%s)
`, val.FieldByName("SkuID").Int(), ProjectId)
case user:
query = fmt.Sprintf(`
SELECT u.project_id
FROM users.user_project u
WHERE u.deleted_at IS NULL
AND u.user_id = %d
AND u.project_id IN (%s)
`, val.FieldByName("UserID").Int(), ProjectId)
case reason:
query = fmt.Sprintf(`
SELECT r.project_id
FROM reasons.reason_projects r
WHERE r.deleted_at IS NULL
AND r.reason_id = %d
AND r.project_id IN (%s)
`, val.FieldByName("ReasonID").Int(), ProjectId)
default:
http.NotFound(w, r)
return
}
_ = DB.Raw(query).Scan(&searchProject)
key := make([]int, 0)
double := make([]string, 0)
if len(searchProject) > 0{
for _, v := range searchProject{
for i, value := range Projects{
if value == v.ProjectId{
key = append(key, i)
double = append(double, value)
}
}
}
for _, v := range key{
Projects = append(Projects[:v], Projects[v+1:]...)
}
}
if len(key) > 0{
if (len(Projects) != len(searchProject)) {
http.Error(w, ("You are connecting already existing bindings"), http.StatusBadRequest)
return
}else {
result.Result = 0
result.Message = "The following project bindings already exist:"
result.Status = double
utils.WriteJSON(w, result)
}
}
utils.WriteJSON(w, &Struct)
}
}
Try to use this library from hashicorp. Basically you need to collect all the errors into an array and then return it
Using multierr
var err error
// ... your logic
// where I assume you want to error
if (len(Projects) != len(searchProject)) {
err = multierr.Append(err, errors.New("you new error message"))
}
// when you return a reply
http.Error(w, err.Error(), http.StatusBadRequest)

Golang for each filtering into a new var

Im working with for each loop and var of information and filtering it by A) regex.matchString B)Timecomparrison. The filtering works well and I have the data I need but currently I'm outputting it to screen via fmt.Println in part of the loop. My goal is to take that data and build another var with the now filtered list. I guess I need to make a new variable and add to it? But how do I return that and something I can use later?
Any assistance is appreciated.
for _, thing := range things {
if thing.element1 != nil {
matched, err := regexp.MatchString(z, element1)
if err != nil {
fmt.Println(err)
}
if matched {
if timecomparrison(element2, a) {
// this is a section that needs to be added new var and returned as a var
fmt.Println("****")
fmt.Println("element1:", element1)
fmt.Println("element2:", element2)
}
}
}
}
}
I think you need something like this.
type Thing struct {
element1 string
element2 string
}
func filter() []Thing {
things := []Thing{
{element1: "element1", element2: "element2"},
}
var result []Thing
regex := "/{}d/"
date := time.Now
for _, thing := range things {
if thing.element1 != nil {
matched, err := regexp.MatchString(regex, thing.element1)
if err != nil {
fmt.Println(err)
}
if matched {
if timeComparison(thing.element2, date) {
// this is a section that needs to be added new var and returned as a var
fmt.Println("****")
fmt.Println("element1:", thing.element1)
fmt.Println("element2:", thing.element2)
result = append(result, thing)
}
}
}
}
return result
}
I cleaned the code, added a type and some data, fixed some issues and renamed some things, but you should get the idea :)

Whats the best way to get content from a generic and somehow dynamic go map?

I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.

How do I convert nested interfaces to a map of slices of maps?

I have a map of arrays of maps map[string][]map[string]string, only, when I get the data, it's in the format map[interface{}]map[interface{}][]map[interface{}]interface{}, so I'm left to do a bunch of nested type assertions, which is really clunky, takes a long time to write, is hard to read/write, and is probably error prone, like this;
if key == "identities" {
idErrMessage := "Sorry, there was a problem with an identity"
idArray, ok := setting.(map[string]interface{})
if ok {
for idType, ids := range idArray {
idGroupArray, ok := ids.([]interface{})
if ok {
for _, idGroup := range idGroupArray {
id, ok := idGroup.(map[interface{}]interface{})
if ok {
log.Println("type:", idType)
for key, val := range id {
log.Printf("%v: %v", key, val)
}
} else {
log.Fatal(idErrMessage)
}
}
} else {
log.Fatal(idErrMessage)
}
}
} else {
log.Fatal(idErrMessage)
}
}
I've been searching for a few hours now, and I can't seem to find an easier way to do this than the code above.
Is there anyway I can just v, ok := setting.(map[string]map[string][]map[string]string), or am I just stuck with the code above?
You can define a struct with the expected structure, and try to unmarshal to it with something like this:
https://github.com/mitchellh/mapstructure
I never tried it with map[interface{}]interface{} though.
This is the final code I used:
if key == "identities" {
idTypes := convertToStringMap(setting)
ghIds := convertToMapSlice(idTypes["github"])
for _, ghId := range ghIds {
for key, value := range convertToStringMap(ghId) {
log.Println(key, value)
}
}
}
it basically converts the interfaces to map[string]interface{} one step at a time, which is the best way to do it I've found, and at least you stay out of nesting hell.
These are the function to convert to the proper types:
func convertToStringMap(i interface{}) map[string]interface{} {
v, ok := i.(map[string]interface{})
if !ok {
v, ok := i.(map[interface{}]interface{})
if ok {
m2 := map[string]interface{}{}
for k, v := range v {
m2[k.(string)] = v
}
return m2
} else {
log.Fatal("There was a problem converting to a string map")
}
}
return v
}
func convertToMapSlice(i interface{}) []interface{} {
v, ok := i.([]interface{})
if !ok {
log.Fatal("There was a problem converting to a slice")
}
return v
}
Overall though, I think I'm just going to find a work around so I don't have to do this at all.

How to retrieve nested map values

I want to scan AWS DynamoDB table and then pull only a certain value. Here is my code:
package main
import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/dynamodb"
)
func main() {
svc := dynamodb.New(session.New(), &aws.Config{Region: aws.String("us-west-2")})
params := &dynamodb.ScanInput{
TableName: aws.String("my_Dynamo_table_name"),
Limit: aws.Int64(2),
}
resp, err := svc.Scan(params)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println(resp)
}
and The output is:
{
Count: 2,
Items: [{
update_time: {
N: "1466495096"
},
create_time: {
N: "1465655549"
}
},{
update_time: {
N: "1466503947"
},
create_time: {
N: "1466503947"
}
}],
LastEvaluatedKey: {
Prim_key: {
S: "1234567890"
}
},
ScannedCount: 2
}
Now, I want to retrieve the update_time value for all elements in above output. Here are my attempts:
for _, value := range resp.Items {
fmt.Println(value["create_time"]["N"])
}
and
for _, value := range resp.Items {
fmt.Println(value.create_time.N)
}
and
for _, value := range resp.Items {
fmt.Println(*value.create_time.N)
}
All above attempts error out with /var/tmp/dynamo.go:37: invalid operation: error.
I am from perl/python background and recently started learning golang.
How to retrieve nested map/array values in this case. Also, any reading references would be of great help. My google search did not reveal anything relevant.
The value of resp above is of the type *ScanOutput, which has Items type as []map[string]*AttributeValue.
To access update_time, you can try:
updateTimes := make([]string, 0)
// Items is a slice of map of type map[string]*AttributeValue
for _, m := range resp.Items {
// m is of type map[string]*AttributeValue
timeStrPtr := *m["update_time"].N
updateTimes = append(updateTimes, *timeStrPtr)
}
updateTimes should now contains all the "update_time" values as strings.
More details here.
You should use the dynamodbattribute package. it's cheaper, safer, and more readable.
Following your example:
type Row struct {
CreateTime int `dynamodbav:"create_time"`
UpdateTime int `dynamodbav:"update_time"`
}
// ...
rows := make([]*Row, len(resp.Items))
if err := dynamodbattribute.Unmarshal(resp.Items, &rows); err != nil {
// handle the error
}
// access the data
for _, row := range rows {
fmt.Println(row.CreateTime, row.UpdateTime)
}

Resources