Getting world state by value - go

I'm new to blockchain and working with hyperledger fabric (v:0.6 for now) to create an application for learning perspective.
I'm keeping a ledger of financial transactions on blockchain, soon as a transaction takes place (A web based component notifies on transaction occurrence and calls the chaincode).
The structure of transactions looks something like this:
type Transactions struct {
ReferenceNumber string `json:"ReferenceNumber"`
BillNumber string `json:"BillNumber"`
BillingCompany string `json:"BillingCompany"`
Amount string `json:"Amount"`
Status string `json:"Status"`
}
I json marshal this and save it to state with ReferenceNumber as the key.
Now I can get the transaction from state on the basis of ReferenceNumber. But what if I want to get the transaction from state on the basis of let's say 'Status' like how many transactions on the ledger have status as 'reconciled'.
Is there any way to query state not on the basis of key but value?

Worldstate level storage works at the {key,value} level. And as obvious its only intended for a single value lookup for a specified key. I think what you are looking for calls for a next level higher level of Abstraction of WorldState - called Table constructs.
fabric/examples/chaincode/go/asset_management_interactive/asset_management.go has an example on how to create a table with the columns you want. While defining the primary keys of your data structure to hold the transaction, you include Status as one of the keys and you would be able to retrieve data on the basis of Status as well.
Some sample code to create the table is as below
func createTableTwo(stub shim.ChaincodeStubInterface) error {
var columnDefsTableTwo []*shim.ColumnDefinition
columnOneTableTwoDef := shim.ColumnDefinition{Name: "colOneTableTwo",
Type: shim.ColumnDefinition_STRING, Key: true}
columnTwoTableTwoDef := shim.ColumnDefinition{Name: "colTwoTableTwo",
Type: shim.ColumnDefinition_INT32, Key: false}
columnThreeTableTwoDef := shim.ColumnDefinition{Name: "colThreeTableThree",
Type: shim.ColumnDefinition_INT32, Key: true}
columnFourTableTwoDef := shim.ColumnDefinition{Name: "colFourTableFour",
Type: shim.ColumnDefinition_STRING, Key: true}
columnDefsTableTwo = append(columnDefsTableTwo, &columnOneTableTwoDef)
columnDefsTableTwo = append(columnDefsTableTwo, &columnTwoTableTwoDef)
columnDefsTableTwo = append(columnDefsTableTwo, &columnThreeTableTwoDef)
columnDefsTableTwo = append(columnDefsTableTwo, &columnFourTableTwoDef)
return stub.CreateTable("tableTwo", columnDefsTableTwo)
}
Now to insert data into this table, as shown
if len(args) < 4 {
return nil, errors.New("insertRowTableTwo failed. Must include 4 column values")
}
col1Val := args[0]
col2Int, err := strconv.ParseInt(args[1], 10, 32)
if err != nil {
return nil, errors.New("insertRowTableTwo failed. arg[1] must be convertable to int32")
}
col2Val := int32(col2Int)
col3Int, err := strconv.ParseInt(args[2], 10, 32)
if err != nil {
return nil, errors.New("insertRowTableTwo failed. arg[2] must be convertable to int32")
}
col3Val := int32(col3Int)
col4Val := args[3]
var columns []*shim.Column
col1 := shim.Column{Value: &shim.Column_String_{String_: col1Val}}
col2 := shim.Column{Value: &shim.Column_Int32{Int32: col2Val}}
col3 := shim.Column{Value: &shim.Column_Int32{Int32: col3Val}}
col4 := shim.Column{Value: &shim.Column_String_{String_: col4Val}}
columns = append(columns, &col1)
columns = append(columns, &col2)
columns = append(columns, &col3)
columns = append(columns, &col4)
row := shim.Row{Columns: columns}
ok, err := stub.InsertRow("tableTwo", row)
if err != nil {
return nil, fmt.Errorf("insertRowTableTwo operation failed. %s", err)
}
if !ok {
return nil, errors.New("insertRowTableTwo operation failed. Row with given key already exists")
}
Now to query this data by not specifying all the keys, do as below
if len(args) < 1 {
return nil, errors.New("getRowsTableTwo failed. Must include at least key values")
}
var columns []shim.Column
col1Val := args[0]
col1 := shim.Column{Value: &shim.Column_String_{String_: col1Val}}
columns = append(columns, col1)
if len(args) > 1 {
col2Int, err := strconv.ParseInt(args[1], 10, 32)
if err != nil {
return nil, errors.New("getRowsTableTwo failed. arg[1] must be convertable to int32")
}
col2Val := int32(col2Int)
col2 := shim.Column{Value: &shim.Column_Int32{Int32: col2Val}}
columns = append(columns, col2)
}
rowChannel, err := stub.GetRows("tableTwo", columns)
if err != nil {
return nil, fmt.Errorf("getRowsTableTwo operation failed. %s", err)
}
var rows []shim.Row
for {
select {
case row, ok := <-rowChannel:
if !ok {
rowChannel = nil
} else {
rows = append(rows, row)
}
}
if rowChannel == nil {
break
}
}
jsonRows, err := json.Marshal(rows)
if err != nil {
return nil, fmt.Errorf("getRowsTableTwo operation failed. Error marshaling JSON: %s", err)
}
return jsonRows, nil
Once you have inserted the data, the API stub.GetRows("tableTwo", columns) lets you retrieve it without specifying all the key columns.
Above code is quoted from a file which was present in the Fabric github repo earlier in the following path gerrit/src/github.com/hyperledger/fabric/bddtests/chaincode/go/table/table.go
Hope this helps.

In Hyperledger Fabric v1.0 you can model your data as JSON and utilize CouchDB as the state database. In this case you can directly query on any field of the JSON.
There is an example of such data patterns in the marbles02 chaincode.
There are various other ledger and state database options in v1.0.

Related

Golang query scan not scanning query correctly into struct

I am having trouble with scanning from a pgx query in Golang. The id field is always that of the last record. If I un-comment the var person Person declaration at the top of the function, every id is 3. There are 3 records with id's from 1 to 3 in my db. When I comment that declaration and declare the variable in the rows.Next() loop I get the correct id's. I can't figure out why the personId isn't being correctly overwritten
output from marshalled JSON with the var declared at the top of the function.
[{"person_id":3,"first_name":"Mark","last_name":"Brown"},{"person_id":3,"first_name":"Sam","last_name":"Smith"},{"person_id":3,"first_name":"Bob","last_name":"Jones"}]
output after declaring person every iteration of the scan loop
[{"person_id":1,"first_name":"Mark","last_name":"Brown"},{"person_id":2,"first_name":"Sam","last_name":"Smith"},{"person_id":3,"first_name":"Bob","last_name":"Jones"}]
I have this struct
// Person model
type Person struct {
PersonId *int64 `json:"person_id"`
FirstName *string `json:"first_name"`
LastName *string `json:"last_name"`
}
Here is my query function
func getPersons(rs *appResource, companyId int64) ([]Person, error) {
// var person Person
var persons []Person
queryString := `SELECT
user_id,
first_name,
last_name,
FROM users
WHERE company_id = $1`
rows, err := rs.db.Query(context.Background(), queryString, companyId)
if err != nil {
return persons, err
}
for rows.Next() {
var person Person
err = rows.Scan(
&person.PersonId,
&person.FirstName,
&person.LastName)
if err != nil {
return persons, err
}
log.Println(*person.PersonId) // 1, 2, 3 for both var patterns
persons = append(persons, person)
}
if rows.Err() != nil {
return persons, rows.Err()
}
return persons, err
}
I believe that you have discovered a bug (or, at least, unexpected behaviour) in github.com/jackc/pgx/v4. When running Scan it appears that if the pointer (so person.PersonId) is not nil then whatever it is pointing to is being reused. To prove this I replicated the issue and confirmed that you can also fix it with:
persons = append(persons, person)
person.PersonId = nil
I can duplicate the issue with this simplified code:
conn, err := pgx.Connect(context.Background(), "postgresql://user:password#127.0.0.1:5432/schema?sslmode=disable")
if err != nil {
panic(err)
}
defer conn.Close(context.Background())
queryString := `SELECT num::int FROM generate_series(1, 3) num`
var scanDst *int64
var slc []*int64
rows, err := conn.Query(context.Background(), queryString)
if err != nil {
panic(err)
}
for rows.Next() {
err = rows.Scan(&scanDst)
if err != nil {
panic(err)
}
slc = append(slc, scanDst)
// scanDst = nil
}
if rows.Err() != nil {
panic(err)
}
for _, i := range slc {
fmt.Printf("%v %d\n", i, *i)
}
The output from this is:
0xc00009f168 3
0xc00009f168 3
0xc00009f168 3
You will note that the pointer is the same in each case. I have done some further testing:
Uncommenting scanDst = nil in the above fixes the issue.
When using database/sql (with the "github.com/jackc/pgx/stdlib" driver) the code works as expected.
If PersonId is *string (and query uses num::text) it works as expected.
The issue appears to boil down to the following in convert.go:
if v := reflect.ValueOf(dst); v.Kind() == reflect.Ptr {
el := v.Elem()
switch el.Kind() {
// if dst is a pointer to pointer, strip the pointer and try again
case reflect.Ptr:
if el.IsNil() {
// allocate destination
el.Set(reflect.New(el.Type().Elem()))
}
return int64AssignTo(srcVal, srcStatus, el.Interface())
So this handles the case where the destination is a pointer (for some datatypes). The code checks if it is nil and, if so, creates a new instance of the relevant type as a destination. If it's not nil it just reuses the pointer. Note: I've not used reflect for a while so there may be issues with my interpretation.
As the behaviour differs from database/sql and is likely to cause confusion I believe it's probably a bug (I guess it could be an attempt to reduce allocations). I have had a quick look at the issues and could not find anything reported (will have a more detailed look later).

How to Delete Multiple Items from a DynamoDB Table in Go

I've a DynamoDB table that contains items like this:
type AuthEntry struct {
UserID string `dynamodbav:"userId"`
Token string `dynamodbav:"token"`
CreatedOn time.Time `dynamodbav:"createdOn"`
}
I need to delete all the AuthEntry items older than 5 minutes (CreatedOn < now - 5 mins) and without a token (Token is empty). It is clear to me how to remove one item at a time... but I'm wondering how to delete multiple items in one shot. Thank u very much.
I was looking for an example like the one here below... and I hope it helps other newbie like me. For instance, first I use Scan to retrieve the expired entries, and then I run BatchWriteItemInput to actually delete them.
import (
"context"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
var tableName = "USER_AUTH"
...
type AuthRepository struct {
ctx context.Context
svc *dynamodb.Client
}
...
func NewAuthRepository(ctx context.Context) (*AuthRepository, error) {
cfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
return nil, err
}
return &AuthRepository{ctx, dynamodb.NewFromConfig(cfg)}, nil
}
...
func (r *AuthRepository) Collect(maxAge int) (int32, error) {
t := time.Now().Add(time.Duration(maxAge*-1) * time.Millisecond).UTC()
params := &dynamodb.ScanInput{
TableName: aws.String(tableName),
ProjectionExpression: aws.String("userId"),
ExpressionAttributeValues: map[string]types.AttributeValue{
"t": &types.AttributeValueMemberS{*aws.String(t.String())},
},
FilterExpression: aws.String("createdOn < :t"),
}
result, err := r.svc.Scan(r.ctx, params)
if err != nil {
return 0, err
}
wr := make([]types.WriteRequest, result.Count)
for _, v := range result.Items {
authEntry := &AuthEntry{}
if err := attributevalue.UnmarshalMap(v, &authEntry); err != nil {
return 0, err
}
wr = append(wr, types.WriteRequest{
DeleteRequest: &types.DeleteRequest{
Key: map[string]types.AttributeValue{
"userId": &types.AttributeValueMemberS{*aws.String(authEntry.UserID)},
},
}})
}
input := &dynamodb.BatchWriteItemInput{
RequestItems: map[string][]types.WriteRequest{
tableName: wr,
},
}
_, err = r.svc.BatchWriteItem(r.ctx, input)
return result.Count, nil
}
When it comes to deletion, you have a few options.
deleteItem - Deletes a single item in a table by primary key.
batchWriteItem - The BatchWriteItem operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can write up to 16 MB of data, which can comprise as many as 25 put or delete requests
TimeToLive - You can utilize DynamoDBs Time To Live (TTL) feature to delete items you no longer need. Keep in mind that TTL only marks your items for deletion and actual deletion could take up to 48 hours.
I'm not sure which items in your table are part of the primary key, so it's difficult to give you an example. However, this operation is the preferred method to delete multiple items at a time.

Whats the best way to get content from a generic and somehow dynamic go map?

I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.

Retrieving multiple datastore entities of the same kind in a transaction

I need to retrieve a list of items of one Kind where the key is the name of the entity.
Based on each item in the returned list I need to retrieve a list of items from another kind where the key is the name.
I think I need to run this in a transaction and I cannot find how to get multiple entities within a transaction without prior knowledge of a list of keys(GetMulti()). The code I am currently using follows with error handling removed for brevity.
Appreciate any assistance you can offer.
q := datastore.NewQuery("Plans").Namespace(TargetNamespace).
Filter("Status =", PlanActive)
_, err := ps.DSClient.GetAll(tctx, q, &sponsorPlans)
if err != nil {
return
}
for spIndex, spItem := range sponsorPlans {
updateMembers := false
// for each sponsor plan get all the related active client plans
q := datastore.NewQuery("ClientPlans").Namespace(hasu.TheConstants.Namespace).
Filter("Name =", sponsorPlans[spIndex].Name)
memberPlanKeys, err := ps.DSClient.GetAll(tctx, q, &memberPlans)
if err != nil {
return
}
for index := range memberPlans {
memberPlans[index].CapSpend = 0
memberPlans[index].RenewalDate = sponsorPlans[spIndex].RenewalDate
memberPlans[index].MemberCap = spItem.MemberCap
memberPlans[index].Services = spItem.Services
memberPlans[index].Status = spItem.Status
memberPlans[index].TotalLiveSessions = spItem.TotalLiveSessions
}
_, err = ps.DSClient.PutMulti(tctx, memberPlanKeys, memberPlans)
if err != nil {
return
}
}

How to query table with partial key values

I am working with hyperledger (v0.6) tables and have a structure as follows;
var columnDefsTableOne []*shim.ColumnDefinition
columnOneTableOneDef := shim.ColumnDefinition{Name: "RefNum",
Type: shim.ColumnDefinition_STRING, Key: true}
columnTwoTableOneDef := shim.ColumnDefinition{Name: "Amount",
Type: shim.ColumnDefinition_STRING, Key: false}
columnThreeTableOneDef := shim.ColumnDefinition{Name: "Status",
Type: shim.ColumnDefinition_STRING, Key: true}
columnDefsTableOne = append(columnDefsTableOne, &columnOneTableOneDef)
columnDefsTableOne = append(columnDefsTableOne, &columnTwoTableOneDef)
columnDefsTableOne = append(columnDefsTableOne, &columnThreeTableOneDef)
return stub.CreateTable("Recon", columnDefsTableOne)
When I query the table with RefNum only or both RefNum & Status, the row is returned. But if I try to query on the basis of Status only, nothing is returned. Is there any way to achieve this?
Below is the query code
if len(args) < 1 {
return nil, errors.New("Function failed. Must include at least key values")
}
var columns []shim.Column
col1Val := args[0]
col1 := shim.Column{Value: &shim.Column_String_{String_: col1Val}}
columns = append(columns, col1)
if len(args) > 1 {
col2Val := args[1]
col2 := shim.Column{Value: &shim.Column_String_{String_: col2Val}}
columns = append(columns, col2)
}
rowChannel, err := stub.GetRows("Recon", columns)
if err != nil {
return nil, fmt.Errorf("Operation failed. %s", err)
}
var rows []shim.Row
for {
select {
case row, ok := <-rowChannel:
if !ok {
rowChannel = nil
} else {
rows = append(rows, row)
}
}
if rowChannel == nil {
break
}
}
jsonRows, err := json.Marshal(rows)
if err != nil {
return nil, fmt.Errorf("Operation failed. Error marshaling JSON: %s", err)
}
return jsonRows, nil
The row keys are not true keys in the sense of a relational database. Internally, range queries are utilized against the concatenated set of keys. Therefore, the keys in the query must be provided in the same order as they were modeled.
The chaincode APIs have been updated in Fabric v1.0 to make the behavior more intuitive. In v1.0 you can model a composite key and then query on a subset of the keys (again, in the order that they were modeled).
To query on different combinations of keys, you would have to model each unique combination that you'd like to query on. Think of these as simple indexes.
In v1.0 you can also model your data as JSON and utilize CouchDB as the state database. In this case you can directly query on any field of the JSON.
There is an example of both data patterns in the marbles02 chaincode here:
https://github.com/hyperledger/fabric/blob/release/examples/chaincode/go/marbles02/marbles_chaincode.go.

Resources