My goal is to get to the raw driver.Value values as deserialized by a sql driver in its implementation of driver.Rows.Next(). I want to handle the conversion from the values returned by the driver to the needed target types, instead of relying on the automatic conversions built in to Rows.Scan. Note this question does not ask your opinion on whether Rows.Scan "should" be used. I don't want to use it, and I am asking if there is any way to avoid it.
A meaningful answer does not use Rows.Scan at all. The dynamic approach illustrated in Working with Unknown Columns is awful: It invokes all the overhead of Scan and destroys the type information of the source columns, instead shredding the actual driver.Values into SqlBytes.
The following hack works, but relies on the internal implementation detail that sql.Rows.Next() populates the internal field lastcols with exactly the unconverted values which I want:
vpRows := reflect.ValueOf(rows) // rows is a *sql.Rows
vRows := reflect.Indirect(vpRows) // now we have the sql.Rows struct
mem := vRows.FieldByName("lastcols") // unexported field lastcols
unsafeLastCols := unsafe.Pointer(mem.UnsafeAddr()) // Evil
plastCols := (*[]driver.Value)(unsafeLastCols) // But effective
for rows.Next() {
rowVals := *plastCols
fmt.Println(rowVals)
}
The normal solution is to implement your own sql.Scanner. But this does use rows.Scan, so it violates your mysterious requirement not to use rows.Scan.
If you truly must avoid rows.Scan, you'll need to write your own driver implementation (possibly wrapping an existing driver) which provides access to the driver.Value values without rows.Scan.
Related
Im new in Go-EVM, and some time ago I got a source code to get transaction tracking. But some functions in the source codes have been updated and changed, here are some questions I wanna ask:
The first one is: How to get *snapshot.Tree?
stateDB, err := state.New(block.Root(), state.NewDatabase(db))
Now this statements need three parameter and the lost parameter's type is *sanpshot.Tree. It is a struct, here is the link to its source code, in line 164.
The second one is: What are AsseccList and GasTipFee?
message := types.NewMessage(from, tx.To(), 0, tx.Value(), tx.Gas(), from Address, to Address, nonce, amount, gasLimit, tx.GasPrice(), GasTopfee, GasTipFee, tx.Data(), accesslist AccessList, false)
AccessList is also a struct. You can see its struct from here. What should I input into AccessList and GasTipFee?
Really appreciate it if you can help me solve these questions.
In your case you do not need to pass a tree snapshot if you do not have one. The purpose of the tree snapshot is to, if the snapshot matches with the given root of the block's trie, run what they call a pre-fetcher routine, that is in charge of preloading nodes in memory so that when the state reaches the commit phase, it is more performant because it already has most of the required nodes in memory. So in your case, you should be perfectly fine passing nil to that constructor.
As for the AccessList and GasTipFee parameters:
AccessList is an EIP-2930 access list. It's once again something optional that transactions can provide to specify the addresses and storage keys that they need access to. Once again you can provide a nil slice.
What you have called GasTipFee is called GasTipCap on master and is basically the limit value of a gas tip, as far as I understand. You can find more information on gas fees in the official documentation.
Trying to figure out how can I build query conditionally using gorm, save the common query part and execute it.
Suppose I am querying list of courses like below.
common_query_part := app.GetDB(). // <= this returns *gorm.DB
Model(&models.Course{}).
Where("status=?", models.CourseStatusPublished) //models.CourseStatusPublished is a constant
Now I would like to get the list of published courses. And also get its count. So I tried it like so.
published_course_count := *common_query_part
and then for the course list
result := common_query_part.
Offset(offset).
Limit(limit).
find(&courses)
if result.Error !=nil {
//handle error
}
and for the count
result = published_course_count.Count(&total)
if result.Error !=nil {
//handle error
}
The first part works perfectly. But second part of the query does not work, nor even generate any error. What should I do in this case? The common query part can be huge and complex. So rewriting it again just for getting the count can be error prone. So is there a way where I will keep the common query part and execute it sometimes for published courses.. sometimes for published courses count?
I don't believe GORM intends you to reuse your query that way. It is hard to say definitively but it most likely doesn't work because there is some state that is still shared.
This: published_course_count := *common_query_part does a copy by value of the gorm.DB struct, but if that struct contain any pointers(it does) then those are copied as well resulting in two separate struct with pointer to the same objects and thus still being "linked". You need a dedicated clone function, which does exist in gorm but is not exposed.
I would advise you to put the generation of the common part inside a function and call it twice, that way you don't have to copy-paste the same query.
func common_query_part() *gorm.DB {
return app.GetDB().
Model(&models.Course{}).
Where("status=?", models.CourseStatusPublished)
}
RocksDBStore<K,V> stores keys and values as byte[] on disk. It converts to/from K and V typed objects using Serdes provided while constructing the object of RocksDBStore<K,V>.
Given this, please help me understand the purpose of the following code in RocksDbKeyValueBytesStoreSupplier:
return new RocksDBStore<>(name,
Serdes.Bytes(),
Serdes.ByteArray());
Providing Serdes.Bytes() and Serdes.ByteArray() looks redundant.
RocksDbKeyValueBytesStoreSupplier is introduced in KAFKA-5650 (Kafka Streams 1.0.0) as part of KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines.
In KIP-182, there is the following sentence :
The new Interface BytesStoreSupplier supersedes the existing StateStoreSupplier (which will remain untouched). This so we can provide a convenient way for users creating custom state stores to wrap them with caching/logging etc if they chose. In order to do this we need to force the inner most store, i.e, the custom store, to be a store of type <Bytes, byte[]>.
Please help me understand why we need to force custom stores to be of type <Bytes, byte[]>?
Another place (KAFKA-5749) where I found similar sentence:
In order to support bytes store we need to create a MeteredSessionStore and ChangeloggingSessionStore. We then need to refactor the current SessionStore implementations to use this. All inner stores should by of type < Bytes, byte[] >
Why?
Your observation is correct -- the PR implementing KIP-182 did miss to remove the Serdes from RocksDBStore that are not required anymore. This was fixed in 1.1 release already.
I am new to Go, and trying to get the estimate size of a Thrift generated object in Go, the object has multiple level, and in the case of an member variable is an collection of pointers, I want to get the total size of pointer and the data that been pointed to. The sizeOf function won't work in this case, as it will only count the space taken by the pointer, not the actual space. What's the best way to get a good estimate of the size of the object? Ideally, I would like to breakdown the size into different fields and subfields
Here is how an example object looks like in Go
type Microshard struct {
Header *Header `thrift:"header,1,required"`
ValidatorStats []*ValidatorStat `thrift:"validatorStats,2,required"`
DataMap1 map[int64][]byte `thrift:"dataMap1,3,required"`
DataMap2 map[int64][]byte `thrift:"dataMap2,4,required"`
DataMap3 map[int64][]int64 `thrift:"dataMap3,5,required"`
DebugInfoMap map[string][]byte `thrift:"debugInfoMap,6,required"`
Index *IndexData `thrift:"index,7"`
}
I'm writing a service to learn Go. My main function can be found below. It starts with reading an XML file and storing them in a slice. I have a /rss endpoint which outputs a RSS feed from the items stored in the "database". This is working fine. I also have an endpoint (/add/{base64}) which is used to add a new item to that slice. Unfortunately I don't know how to do this. For some reason I need to return the new database with the added record, so it gets available to the /rss. But how?
My concrete problem is:
I know how to add a record to database
But I don't know how to return the full (including the added) database so the /rss endpoint is able to use it. So I want to let the rest.AddArticle return the new database so the /rss endpoint knows the added item.
func main() {
defer glog.Flush()
// read database
database := model.ReadFileIntoSlice()
// initialise mux router
r := mux.NewRouter()
// http handles
r.HandleFunc("/add/{base64url}", rest.AddArticle(database))
r.HandleFunc("/rss", rest.GenerateRSS(database))
// start server
http.Handle("/", r)
glog.Infof("running on port %d", *port)
http.ListenAndServe(":"+strconv.Itoa(*port), nil)
}
Or is there some other solution which does the job? I just want database to be available through all packages.
From what I can tell the problem is that you're writing to your db but you're reading from the cached version so the response of rss just doesn't reflect the model at the time the request is made. If you look at this code;
database := model.ReadFileIntoSlice()
// initialise mux router
r := mux.NewRouter()
// http handles
r.HandleFunc("/add/{base64url}", rest.AddArticle(database))
you somehow need to modify the value of database. There are many ways you could do this. A few options would be 1) define it on some object or at a package level and then directly modify it from wherever AddArticle is defined. 2) refresh your in memory version, ie before you return the results, read from the db again so you're assured to have the latest (obv performance implications) 3) don't pass database by value, make the argument a pointer instead. AddArticle is getting a copy of database rather than the address to the version you're reading from in your rss call. You could instead pass a pointer into that method so that the original copy is modified (it also performs substantially better as your model gets larger).
Based on the simplicity of your program I'd probably do 3. Realistically 2 is a more robust solution and serious enterprise software probably would require something more along those lines (your model doesn't work if your app is load balanced or something like that).