I'm currently using go-sqlmock to mock my database.
Is there any way to get the values that have been passed to the sql driver when performing a query?. Like the arg variable that is been passing as argument here:
import (
"database/sql"
)
func retrieveInfo(){
// this function returns an initialized instance of type *sql.DB
DbDriver := initDb()
query := "my query"
arg := 3
rows, err := Db_driver.Query(query, arg)
// ...
}
Then I want to test the function, and I would like to know the value of the variable arg while testing. I think it should be possible, since it is passed to the "fake" driver that go-sqlmock creates. My test logic looks like this:
import "github.com/DATA-DOG/go-sqlmock"
// Inits mock and overwrites the original driver
db, mock, err := sqlmock.New()
Db_driver = db
func TestRetrieveInfo(t *testing.T){
// query that matchs the one in retrieveInfo()
query := "..."
queryRows := sqlmock.NewRows([]string{"column1, column2"}).FromCSVString("value1, value2")
mock.ExpectQuery(query).WillReturnRows(queryRows)
}
You can wrap the real driver with instrumentation to log the values.
Use package github.com/luna-duclos/instrumentedsql.
You can capture an arg value by making your own Match method, e.g.
type ArgCapture struct {
CapturedValue driver.Value
}
func (a *ArgCapture) Match(value driver.Value) bool {
a.CapturedValue = value
return true
}
Then in your test you'd have something like
capture := &ArgCapture{}
mock.ExpectQuery(query).WithArgs(argCapture).WillReturnRows(queryRows)
Then you can do assertions or whatever on capture.CapturedValue
Related
When a Lua script returns a table array during an Eval call, how can it be converted to a []string in go?
The redis cli returns bulk replies in the following format.
1) val1
2) val2
Will go-redis eval function return the bulk entries as
["val1", "val2"]
Redis returns Lua table arrays as RESP2 arrays. The Go client then will map that response to Go native types. The relevant documentation for go-redis is found here: Lua and Go types.
The tl;dr is that Lua tables are indeed mapped to a bulk reply, and the Go client maps that to a slice of interface: []interface{}.
Both go-redis script Run and Eval return a *Cmd. You can use methods on this type to retrieve the output as Go types. Result gives (interface{}, error), which you can type-assert to whatever you want, otherwise, StringSlice is a convenience getter to retrieve []string right away.
So it looks like:
script := redis.NewScript(`
local foo = {"val1", "val2"}
return foo
`)
cmd := script.Run(/* parameters */)
i, err := cmd.Result() // (interface, error)
// or
ss, err := cmd.StringSlice() // ([]string, error)
If the values aren't actually all strings, use Slice to get the []interface{} slice, and then inspect the elements individually.
You can use the encoding/json package to convert a JSON string to a slice of strings.
package main
import (
"encoding/json"
"fmt"
)
// jsonString is the JSON string that you want to convert to a slice of strings.
const jsonString = `["value1", "value2"]`
func main() {
var stringSlice []string
// Unmarshal the JSON string into the stringSlice variable.
err := json.Unmarshal([]byte(jsonString), &stringSlice)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(stringSlice) // ["value1", "value2"]
}
I'm using goose to manage my database migrations but I need to write SQL sentences directly in the migrations file. There is a way to generate the SQL directly from the GORM model?
Unfortunately using the gorm.Session{DryRun: true} option doesn't make the migration SQL statement/s available to the caller as it does with normal queries.
The only way I can see right now would be to capture the SQL that is run by the migration when it's being logged by reimplementing the gorm.io/gorm/logger.Interface interface. Specifically, the Trace method.
type Interface interface {
LogMode(LogLevel) Interface
Info(context.Context, string, ...interface{})
Warn(context.Context, string, ...interface{})
Error(context.Context, string, ...interface{})
Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error)
}
Inside Trace you can call that fc function argument to get the SQL and RowsAffected, which you can do whatever you want with.
For example:
import (
"time"
"context"
"gorm.io/gorm/logger"
)
type RecorderLogger struct {
logger.Interface
Statements []string
}
func (r *RecorderLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
sql, _ := fc()
r.Statements = append(r.Statements, sql)
}
Now use it as:
recorder := RecorderLogger{logger.Default.LogMode(logger.Info)}
session := db.Session(&gorm.Session{
Logger: &recorder
})
session.AutoMigrate(&Model{}, ...)
// or
session.Migrator().CreateTable(&Model{}, ...) // or any method therein
// now recorder.Statements contains the statements run during migration
This is very hacky, and you may run into problems because AutoMigrate modifies the current state of the database and migrates it up to what your model requires (up to a point) and for that to work your current database must reflect the current state of your production database (or whatever database your hope to migrate). So, you could build that tool that helps you get the migration script started if you're careful, but to properly gain the advantages of a migration system like goose you'll need to get your hands dirty with the SQL :)
you can using this lib: https://github.com/sunary/sqlize
It's allowed you create sql from models, also support migration by differ between models and existing sql.
I personally would use the migration functionality that is available inside Gorm, but for your case we can do the following.
Firstly there is a feature in Gorm called Dry Run and you can use this to see the SQL statements that get executed when performing queries. Unfortunately I can't see that it is possible when using migrations. So what I suggest is to use github.com/DATA-DOG/go-sqlmock
I would usually use this for testing purposes, but you could use it temporarily to get the SQL needed for your separate migrations.
package main
import (
"database/sql"
"time"
"github.com/DATA-DOG/go-sqlmock"
"gorm.io/driver/mysql"
"gorm.io/gorm"
)
type Model struct {
ID uint64 `gorm:"primaryKey"`
Name string `gorm:"index"`
Description string
CreatedAt time.Time
LastLogin sql.NullTime
}
func main() {
sqlDB, _, err := sqlmock.New()
if err != nil {
panic(err)
}
gormDB, err := gorm.Open(mysql.New(mysql.Config{
Conn: sqlDB,
SkipInitializeWithVersion: true,
}), &gorm.Config{})
if err != nil {
panic(err)
}
defer sqlDB.Close()
gormDB.AutoMigrate(&Model{})
}
This will give you a result like this
all expectations were already fulfilled, call to ExecQuery 'CREATE TABLE `models` (`id` bigint unsigned AUTO_INCREMENT,`name` varchar(191),`description` longtext,`created_at` datetime(3) NULL,`last_login` datetime(3) NULL,PRIMARY KEY (`id`),INDEX idx_models_name (`name`))' with args [] was not expected
[0.003ms] [rows:0] CREATE TABLE `models` (`id` bigint unsigned AUTO_INCREMENT,`name` varchar(191),`description` longtext,`created_at` datetime(3) NULL,`last_login` datetime(3) NULL,PRIMARY KEY (`id`),INDEX idx_models_name (`name`))
which contains the SQL statement required. This feels incredibly hacky but will give you the result you need
I am using Go to open a file with multiple JSON entries, parse the file into a slice with a custom type, then insert the slice data into an Oracle database. According to the godror documentation at https://godror.github.io/godror/doc/tuning.html, I should be able to feed a slice into the insert command, and have the database/sql Exec method iterate thru the struct for me. I am at a loss for how to do this. I am sure there is a simple solution to this.
To slightly complicate things, I have a database column that is not in the struct for the host name of the computer the app is running on. This column should be filled in for every row the app inserts. In other words, every row of this table needs to have a column filled in with the host name of the machine is running on. Is there a more elegant way to do this than to just add a 'hostname' field to my struct that has the running system's host name, over and over again?
What follows is a simplified version of my code.
package main
import (
"database/sql"
_ "github.com/godror/godror"
)
type MyType struct {
var1 string `json:"var1"`
var2 string `json:"var2"`
}
func main() {
hostname, err := os.Hostname()
if err != nil {
//log.Println("Error when getting host name")
log.Fatal(err)
}
mySlice := parseFile("/path/to/file", false)
db, err := sql.Open("godror", "user/pass#oraHost/oraDb")
sql := `INSERT INTO mytable (var1, var2, host) values (:1 :2 :3)`
// this is the line where everything breaks down, and i am not sure
// what should go here.
_, err = db.Exec(sql, mySlice[var1], mySlice[var2], hostname)
}
func parseFile(filePath string, deleteFile bool) []MyType {
// a few lines of code that opens a text file, parses it into a slice
// of type MyType, and returns it
}
not sure, if you already went through, does this test case, TestExecuteMany help ? mentioned in https://github.com/godror/godror/blob/master/z_test.go has example usage for array insert.
res, err := tx.ExecContext(ctx,
`INSERT INTO `+tbl+ //nolint:gas
` (f_id, f_int, f_num, f_num_6, F_num_5_2, F_vc, F_dt)
VALUES
(:1, :2, :3, :4, :5, :6, :7)`,
ids, ints, nums, int32s, floats, strs, dates)
for batch insert of structs:
https://github.com/jmoiron/sqlx
Given a function with a parameter (that is a pointer to a struct) I want to instantiate a type of this parameter.
For example, for this function:
func MyFunction(myStruct *MyStruct) {}
using reflection I want to create a variable that contains exactly same as x := &MyStruct{} would contain.
This is the code example:
package main
import (
"fmt"
"reflect"
)
type MyStruct struct {
}
func main () {
reflectedFunction := reflect.TypeOf(MyFunction)
argType := reflectedFunction.In(0)
reflectedParameter := reflect.New(argType)
actual := reflectedParameter.Interface()
fmt.Println(actual)
expected := &MyStruct{}
fmt.Println(expected)
}
func MyFunction(myStruct *MyStruct) {
}
If you execute it, you'll see that they contain different info:
0xc00000e028 // actual
&{} // expected
This question isn't about why I would like to do this, so please avoid recommending not doing it, etc.
In your code, actual is a interface{} value containing a *MyStruct. As the name and documentation indicate, reflectedParameter.Interface() returns an interface{}.
using reflection I want to create a variable that contains exactly same as x := &MyStruct{} would contain.
Then you'll have to type assert it:
actual := reflectedParameter.Elem().Interface().(*MyStruct)
reflect.New creates a pointer to a new zero value of the reflected type. In your example that type is already a *MyStruct, so the value of your actual winds up being a representation of a **MyStruct, as seen in https://play.golang.org/p/Nyuc0mYmgkZ. Taking the .Elem() of that results in the correct type again, but you end up with a nil pointer (*MyStruct)(nil).
You need to take the .Elem() if that first type if you want to create a new pointer value.
reflectedParameter := reflect.New(argType.Elem())
https://play.golang.org/p/QzwTFUH3HTs
reflectedFunction := reflect.TypeOf(MyFunction)
argType := reflectedFunction.In(0)
reflectedParameter := reflect.New(argType.Elem())
actual := reflectedParameter.Interface()
fmt.Printf("%#v\n", actual)
expected := &MyStruct{}
fmt.Printf("%#v\n", expected)
Which prints
&main.MyStruct{}
&main.MyStruct{}
Does testing package support snapshot testing?
Here is my case:
package main
import (
"bytes"
"fmt"
"html/template"
)
func main() {
query := `
INSERT INTO "ADGROUP_PERFORMANCE_REPORT" (
{{.columnPrefix}}_adgroup_id,
{{.columnPrefix}}_adgroup_nme,
{{.columnPrefix}}_adgroup_status,
{{.columnPrefix}}_campaign_id,
{{.columnPrefix}}_campaign_nme,
{{.columnPrefix}}_campaign_status,
{{.columnPrefix}}_clicks,
{{.columnPrefix}}_impressions,
{{.columnPrefix}}_ctr,
{{.columnPrefix}}_average_cpc,
{{.columnPrefix}}_cost,
{{.columnPrefix}}_conversions,
{{.columnPrefix}}_average_position,
{{.columnPrefix}}_device,
google_adwords_client_customer_id
) VALUES
`
vars := make(map[string]interface{})
vars["columnPrefix"] = "adgroup_performance_report"
result := processString(query, vars)
fmt.Printf("result=%s\n", result)
}
func process(t *template.Template, vars interface{}) string {
var tmplBytes bytes.Buffer
err := t.Execute(&tmplBytes, vars)
if err != nil {
panic(err)
}
return tmplBytes.String()
}
func processString(str string, vars interface{}) string {
tmpl, err := template.New("tmpl").Parse(str)
if err != nil {
panic(err)
}
return process(tmpl, vars)
}
Now I am going to write unit test for it, I would like use snapshot testing to test the structure of the SQL query string processed by html/template pkg.
Here is the output in the stdout:
result=
INSERT INTO "ADGROUP_PERFORMANCE_REPORT" (
adgroup_performance_report_adgroup_id,
adgroup_performance_report_adgroup_nme,
adgroup_performance_report_adgroup_status,
adgroup_performance_report_campaign_id,
adgroup_performance_report_campaign_nme,
adgroup_performance_report_campaign_status,
adgroup_performance_report_clicks,
adgroup_performance_report_impressions,
adgroup_performance_report_ctr,
adgroup_performance_report_average_cpc,
adgroup_performance_report_cost,
adgroup_performance_report_conversions,
adgroup_performance_report_average_position,
adgroup_performance_report_device,
google_adwords_client_customer_id
) VALUES
I don't want to write this expected value duplicately in unit test file and assert it. I prefer using snapshot testing and it will generate a snapshot file. Something like jestjs snapshot-testing
As far as I know, the testing Package does not support something like this out of the box. There is a pattern for Go that you can utilise called "Golden file testing". The convention is to store testdata in a testdata folder alongside your test. In this case you would store the rendered template in a so called "golden file". The test itself provides an update flag to write out the latest version (so that you don't have to manually maintain the output):
var update = flag.Bool("update", false, "update .golden files")
func TestProcessString(t *testing.T) {
vars := make(map[string]interface{})
vars["columnPrefix"] = "adgroup_performance_report"
actual := processString(query, vars)
golden := filepath.Join(“testdata”, ”performance_report.golden”)
if *update {
ioutil.WriteFile(golden, actual, 0644)
}
expected, _ := ioutil.ReadFile(golden)
if !bytes.Equal(actual, expected) {
t.Fatalf("Output did not match, expected %v, recieved %v, expected, actual)
}
}
A nice example of this pattern can be found in the gofmt source code: https://golang.org/src/cmd/gofmt/gofmt_test.go
This is probably too late as an answer, but I have been working on a snapshot testing library for Golang that is "like" jest toMatchSnapshot.
I believe it matches exactly your use case, how to test long strings that you don't want to keep that string on or unit tests.
It would be as simple as and also provides a nice diff view
func TestExample(t *testing.T) {
snaps.MatchSnapshot(t ,processString(query, vars))
}
You can have a look go-snaps.