I am working with hyperledger (v0.6) tables and have a structure as follows;
var columnDefsTableOne []*shim.ColumnDefinition
columnOneTableOneDef := shim.ColumnDefinition{Name: "RefNum",
Type: shim.ColumnDefinition_STRING, Key: true}
columnTwoTableOneDef := shim.ColumnDefinition{Name: "Amount",
Type: shim.ColumnDefinition_STRING, Key: false}
columnThreeTableOneDef := shim.ColumnDefinition{Name: "Status",
Type: shim.ColumnDefinition_STRING, Key: true}
columnDefsTableOne = append(columnDefsTableOne, &columnOneTableOneDef)
columnDefsTableOne = append(columnDefsTableOne, &columnTwoTableOneDef)
columnDefsTableOne = append(columnDefsTableOne, &columnThreeTableOneDef)
return stub.CreateTable("Recon", columnDefsTableOne)
When I query the table with RefNum only or both RefNum & Status, the row is returned. But if I try to query on the basis of Status only, nothing is returned. Is there any way to achieve this?
Below is the query code
if len(args) < 1 {
return nil, errors.New("Function failed. Must include at least key values")
}
var columns []shim.Column
col1Val := args[0]
col1 := shim.Column{Value: &shim.Column_String_{String_: col1Val}}
columns = append(columns, col1)
if len(args) > 1 {
col2Val := args[1]
col2 := shim.Column{Value: &shim.Column_String_{String_: col2Val}}
columns = append(columns, col2)
}
rowChannel, err := stub.GetRows("Recon", columns)
if err != nil {
return nil, fmt.Errorf("Operation failed. %s", err)
}
var rows []shim.Row
for {
select {
case row, ok := <-rowChannel:
if !ok {
rowChannel = nil
} else {
rows = append(rows, row)
}
}
if rowChannel == nil {
break
}
}
jsonRows, err := json.Marshal(rows)
if err != nil {
return nil, fmt.Errorf("Operation failed. Error marshaling JSON: %s", err)
}
return jsonRows, nil
The row keys are not true keys in the sense of a relational database. Internally, range queries are utilized against the concatenated set of keys. Therefore, the keys in the query must be provided in the same order as they were modeled.
The chaincode APIs have been updated in Fabric v1.0 to make the behavior more intuitive. In v1.0 you can model a composite key and then query on a subset of the keys (again, in the order that they were modeled).
To query on different combinations of keys, you would have to model each unique combination that you'd like to query on. Think of these as simple indexes.
In v1.0 you can also model your data as JSON and utilize CouchDB as the state database. In this case you can directly query on any field of the JSON.
There is an example of both data patterns in the marbles02 chaincode here:
https://github.com/hyperledger/fabric/blob/release/examples/chaincode/go/marbles02/marbles_chaincode.go.
Related
Summary
I am trying to write data from several postgres tables into a nested Go struct for the purpose of returning a single json response to a GET request within my web app.
Questions
Is the way I'm declaring a nested struct reasonable from a Go best practices perspective, or is there a reason I should avoid this method and do it another way?
What am I doing wrong in Step 3 to prevent my code from working? (I fear the answer is 'everything')
What I've got so far
I've declared my struct of structs
type MainObject struct {
SubObjects []struct {
SpecificDetail string `json:"specific-detail"`
} `json:"sub-object"`
...(other []structs)...
}
I've retrieved rows from the tables
func getMainObjectHandler(w http.ResponseWriter, r *http.Request) {
...(database connnection)...
MainObjectID := r.URL.Query().Get("moid")
if MainObjectID != "null" {
NewMainObject := MainObject{}
SubObjectDetail_rows, err := db.Query("SELECT specific_detail from the_table WHERE moid= '" + MainObjectID + "'")
if err != nil {
log.Fatalf("could not execute query: %v", err)
}
...(other db.Query rows)...
I've tried (and failed) to build the row data into the struct.
for SubObjectDetail_rows.Next() {
SpecificDetail := NewMainObject.SubObject.SpecificDetail{}
SubObjectDetail_rows.Scan(&SpecificDetail)
SubObject = append(SubObject, SpecificDetail)
}
NewMainObject = append(MainObject, SubObject)
defer persona_rows.Close()
Finally, I've set up Marshal and write.
NMOListBytes, err := json.Marshal(NewMainObject)
if err != nil {
fmt.Println(fmt.Errorf("Error: %v", err))
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(NMOListBytes)
Firstly, please use placeholders when creating your SQL query to avoid injections:
// db.Query("SELECT specific_detail from the_table WHERE moid= '" + MainObjectID + "'") // not this
db.Query("SELECT specific_detail from the_table WHERE moid=?", MainObjectID)
Unless you're using a framework like GORM you can't Scan() into a single struct value.
From the docs:
Scan copies the columns in the current row into the values pointed at
by dest. The number of values in dest must be the same as the number
of columns in Rows.
It looks like you're pulling JSON from a DB query, as you're only querying one column, so you probably want:
var bs []byte // get raw JSON bytes
err = SubObjectDetail_rows.Scan(&bs)
if err != nil { /* always check errors */ }
and then unmarshal them to your struct:
err = json.Unmarshal(bs, &SpecificDetail)
if err != nil { /* ... */ }
I've a DynamoDB table that contains items like this:
type AuthEntry struct {
UserID string `dynamodbav:"userId"`
Token string `dynamodbav:"token"`
CreatedOn time.Time `dynamodbav:"createdOn"`
}
I need to delete all the AuthEntry items older than 5 minutes (CreatedOn < now - 5 mins) and without a token (Token is empty). It is clear to me how to remove one item at a time... but I'm wondering how to delete multiple items in one shot. Thank u very much.
I was looking for an example like the one here below... and I hope it helps other newbie like me. For instance, first I use Scan to retrieve the expired entries, and then I run BatchWriteItemInput to actually delete them.
import (
"context"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
var tableName = "USER_AUTH"
...
type AuthRepository struct {
ctx context.Context
svc *dynamodb.Client
}
...
func NewAuthRepository(ctx context.Context) (*AuthRepository, error) {
cfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
return nil, err
}
return &AuthRepository{ctx, dynamodb.NewFromConfig(cfg)}, nil
}
...
func (r *AuthRepository) Collect(maxAge int) (int32, error) {
t := time.Now().Add(time.Duration(maxAge*-1) * time.Millisecond).UTC()
params := &dynamodb.ScanInput{
TableName: aws.String(tableName),
ProjectionExpression: aws.String("userId"),
ExpressionAttributeValues: map[string]types.AttributeValue{
"t": &types.AttributeValueMemberS{*aws.String(t.String())},
},
FilterExpression: aws.String("createdOn < :t"),
}
result, err := r.svc.Scan(r.ctx, params)
if err != nil {
return 0, err
}
wr := make([]types.WriteRequest, result.Count)
for _, v := range result.Items {
authEntry := &AuthEntry{}
if err := attributevalue.UnmarshalMap(v, &authEntry); err != nil {
return 0, err
}
wr = append(wr, types.WriteRequest{
DeleteRequest: &types.DeleteRequest{
Key: map[string]types.AttributeValue{
"userId": &types.AttributeValueMemberS{*aws.String(authEntry.UserID)},
},
}})
}
input := &dynamodb.BatchWriteItemInput{
RequestItems: map[string][]types.WriteRequest{
tableName: wr,
},
}
_, err = r.svc.BatchWriteItem(r.ctx, input)
return result.Count, nil
}
When it comes to deletion, you have a few options.
deleteItem - Deletes a single item in a table by primary key.
batchWriteItem - The BatchWriteItem operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can write up to 16 MB of data, which can comprise as many as 25 put or delete requests
TimeToLive - You can utilize DynamoDBs Time To Live (TTL) feature to delete items you no longer need. Keep in mind that TTL only marks your items for deletion and actual deletion could take up to 48 hours.
I'm not sure which items in your table are part of the primary key, so it's difficult to give you an example. However, this operation is the preferred method to delete multiple items at a time.
I have database store function:
func (p *ProductsRep) FindAll(PageNumber int, PaginationSize int, Query string) []*postgresmodels.Product {
Also I have SQL query look like this:
SELECT * FROM table_name.
Then I want to concat conditional action like WHERE some_value=3 if some value (in this case Query) exists then I want to get SELECT * FROM table_name WHERE some_value=3.
I tried to use fmt.Sprintf to concat, or strings.Join, or bytes.Buffer.WriteString. But everytime I getting this error:
I replace real value for understanding:
pq: column "Some value" does not exist.
How can I do "adaptive" queries, which depends on inputed function values.
I believe you are trying to query rows in the database by using parameters.
You need to make sure you don't pass this data in as RAW values, due to the potential risk of SQL injection. You can make queries by using store procedures
You can use the function Query to pass in your query with your parameters. In the example case this is $1. If you wanted to you could add $2, $3... etc depending on how many parameters you wanted to query
Here is two examples
Postgres
using "github.com/jackc/pgx/v4" driver
ctx := context.Background()
type Bar struct {
ID int64
SomeValue string
}
rows, err := conn.Query(ctx, `SELECT * FROM main WHERE some_value=$1`, "foo")
if err != nil {
fmt.Println("ERRO")
panic(err) // handle error
}
defer rows.Close()
var items []Bar
for rows.Next() {
var someValue string
var id int64
if err := rows.Scan(&id, &someValue); err != nil {
log.Fatal(err) // handle error
}
item := Bar{
ID: id,
SomeValue: someValue,
}
items = append(items, item)
}
fmt.Println(items)
MySQL Driver
https://golang.org/pkg/database/sql/#DB.QueryRow
type Bar struct {
ID int64
SomeValue string
}
rows, err := conn.Query(`SELECT * FROM main WHERE some_value=$1`, "foo")
if err != nil {
fmt.Println("ERRO")
panic(err) // handle error
}
defer rows.Close()
var items []Bar
for rows.Next() {
var someValue string
var id int64
if err := rows.Scan(&id, &someValue); err != nil {
log.Fatal(err) // handle error
}
item := Bar{
ID: id,
SomeValue: someValue,
}
items = append(items, item)
}
fmt.Println(items)
I need to retrieve a list of items of one Kind where the key is the name of the entity.
Based on each item in the returned list I need to retrieve a list of items from another kind where the key is the name.
I think I need to run this in a transaction and I cannot find how to get multiple entities within a transaction without prior knowledge of a list of keys(GetMulti()). The code I am currently using follows with error handling removed for brevity.
Appreciate any assistance you can offer.
q := datastore.NewQuery("Plans").Namespace(TargetNamespace).
Filter("Status =", PlanActive)
_, err := ps.DSClient.GetAll(tctx, q, &sponsorPlans)
if err != nil {
return
}
for spIndex, spItem := range sponsorPlans {
updateMembers := false
// for each sponsor plan get all the related active client plans
q := datastore.NewQuery("ClientPlans").Namespace(hasu.TheConstants.Namespace).
Filter("Name =", sponsorPlans[spIndex].Name)
memberPlanKeys, err := ps.DSClient.GetAll(tctx, q, &memberPlans)
if err != nil {
return
}
for index := range memberPlans {
memberPlans[index].CapSpend = 0
memberPlans[index].RenewalDate = sponsorPlans[spIndex].RenewalDate
memberPlans[index].MemberCap = spItem.MemberCap
memberPlans[index].Services = spItem.Services
memberPlans[index].Status = spItem.Status
memberPlans[index].TotalLiveSessions = spItem.TotalLiveSessions
}
_, err = ps.DSClient.PutMulti(tctx, memberPlanKeys, memberPlans)
if err != nil {
return
}
}
I'm new to blockchain and working with hyperledger fabric (v:0.6 for now) to create an application for learning perspective.
I'm keeping a ledger of financial transactions on blockchain, soon as a transaction takes place (A web based component notifies on transaction occurrence and calls the chaincode).
The structure of transactions looks something like this:
type Transactions struct {
ReferenceNumber string `json:"ReferenceNumber"`
BillNumber string `json:"BillNumber"`
BillingCompany string `json:"BillingCompany"`
Amount string `json:"Amount"`
Status string `json:"Status"`
}
I json marshal this and save it to state with ReferenceNumber as the key.
Now I can get the transaction from state on the basis of ReferenceNumber. But what if I want to get the transaction from state on the basis of let's say 'Status' like how many transactions on the ledger have status as 'reconciled'.
Is there any way to query state not on the basis of key but value?
Worldstate level storage works at the {key,value} level. And as obvious its only intended for a single value lookup for a specified key. I think what you are looking for calls for a next level higher level of Abstraction of WorldState - called Table constructs.
fabric/examples/chaincode/go/asset_management_interactive/asset_management.go has an example on how to create a table with the columns you want. While defining the primary keys of your data structure to hold the transaction, you include Status as one of the keys and you would be able to retrieve data on the basis of Status as well.
Some sample code to create the table is as below
func createTableTwo(stub shim.ChaincodeStubInterface) error {
var columnDefsTableTwo []*shim.ColumnDefinition
columnOneTableTwoDef := shim.ColumnDefinition{Name: "colOneTableTwo",
Type: shim.ColumnDefinition_STRING, Key: true}
columnTwoTableTwoDef := shim.ColumnDefinition{Name: "colTwoTableTwo",
Type: shim.ColumnDefinition_INT32, Key: false}
columnThreeTableTwoDef := shim.ColumnDefinition{Name: "colThreeTableThree",
Type: shim.ColumnDefinition_INT32, Key: true}
columnFourTableTwoDef := shim.ColumnDefinition{Name: "colFourTableFour",
Type: shim.ColumnDefinition_STRING, Key: true}
columnDefsTableTwo = append(columnDefsTableTwo, &columnOneTableTwoDef)
columnDefsTableTwo = append(columnDefsTableTwo, &columnTwoTableTwoDef)
columnDefsTableTwo = append(columnDefsTableTwo, &columnThreeTableTwoDef)
columnDefsTableTwo = append(columnDefsTableTwo, &columnFourTableTwoDef)
return stub.CreateTable("tableTwo", columnDefsTableTwo)
}
Now to insert data into this table, as shown
if len(args) < 4 {
return nil, errors.New("insertRowTableTwo failed. Must include 4 column values")
}
col1Val := args[0]
col2Int, err := strconv.ParseInt(args[1], 10, 32)
if err != nil {
return nil, errors.New("insertRowTableTwo failed. arg[1] must be convertable to int32")
}
col2Val := int32(col2Int)
col3Int, err := strconv.ParseInt(args[2], 10, 32)
if err != nil {
return nil, errors.New("insertRowTableTwo failed. arg[2] must be convertable to int32")
}
col3Val := int32(col3Int)
col4Val := args[3]
var columns []*shim.Column
col1 := shim.Column{Value: &shim.Column_String_{String_: col1Val}}
col2 := shim.Column{Value: &shim.Column_Int32{Int32: col2Val}}
col3 := shim.Column{Value: &shim.Column_Int32{Int32: col3Val}}
col4 := shim.Column{Value: &shim.Column_String_{String_: col4Val}}
columns = append(columns, &col1)
columns = append(columns, &col2)
columns = append(columns, &col3)
columns = append(columns, &col4)
row := shim.Row{Columns: columns}
ok, err := stub.InsertRow("tableTwo", row)
if err != nil {
return nil, fmt.Errorf("insertRowTableTwo operation failed. %s", err)
}
if !ok {
return nil, errors.New("insertRowTableTwo operation failed. Row with given key already exists")
}
Now to query this data by not specifying all the keys, do as below
if len(args) < 1 {
return nil, errors.New("getRowsTableTwo failed. Must include at least key values")
}
var columns []shim.Column
col1Val := args[0]
col1 := shim.Column{Value: &shim.Column_String_{String_: col1Val}}
columns = append(columns, col1)
if len(args) > 1 {
col2Int, err := strconv.ParseInt(args[1], 10, 32)
if err != nil {
return nil, errors.New("getRowsTableTwo failed. arg[1] must be convertable to int32")
}
col2Val := int32(col2Int)
col2 := shim.Column{Value: &shim.Column_Int32{Int32: col2Val}}
columns = append(columns, col2)
}
rowChannel, err := stub.GetRows("tableTwo", columns)
if err != nil {
return nil, fmt.Errorf("getRowsTableTwo operation failed. %s", err)
}
var rows []shim.Row
for {
select {
case row, ok := <-rowChannel:
if !ok {
rowChannel = nil
} else {
rows = append(rows, row)
}
}
if rowChannel == nil {
break
}
}
jsonRows, err := json.Marshal(rows)
if err != nil {
return nil, fmt.Errorf("getRowsTableTwo operation failed. Error marshaling JSON: %s", err)
}
return jsonRows, nil
Once you have inserted the data, the API stub.GetRows("tableTwo", columns) lets you retrieve it without specifying all the key columns.
Above code is quoted from a file which was present in the Fabric github repo earlier in the following path gerrit/src/github.com/hyperledger/fabric/bddtests/chaincode/go/table/table.go
Hope this helps.
In Hyperledger Fabric v1.0 you can model your data as JSON and utilize CouchDB as the state database. In this case you can directly query on any field of the JSON.
There is an example of such data patterns in the marbles02 chaincode.
There are various other ledger and state database options in v1.0.