How to use golang bleve search results? - go

I am new to Go and Bleve (sorry if I'm asking trivial thingsā€¦). This search engine seems to be really nice, but I am getting stuck when it comes to deal with my search results.
Let's say we have a struct:
type Person struct {
Name string `json:"name"`
Bio string `json:"bio"`
}
Now, we extract data from the database (using sqlx lib):
rows := []Person{}
db.Select(&rows, "SELECT * FROM person")
...and index it:
index.Index, err = bleve.Open("index.bleve")
batch := index.Index.NewBatch()
i := 0
for _, row := range rows {
rowId := fmt.Sprintf("%T_%d", row, row.ID)
batch.Index(rowId, row)
i++
if i > 100 {
index.Index.Batch(batch)
i = 0
}
}
Now we have our index created. It works great.
Using the bleve command line utility, it returns data correctly:
bleve query index.bleve doe
3 matches, showing 1 through 3, took 27.767838ms
1. Person_68402 (0.252219)
Name
Doe
Bio
My name is John Doe!
2. ...
Here we see that bleve has stored Name and Bio fields.
Now I want to do it to access it from my code!
query := bleve.NewMatchAllQuery()
searchRequest := bleve.NewSearchRequest(query)
searchResults, _ := index.Index.Search(searchRequest)
fmt.Println(searchResults[0].ID) // <- This works
But I don't only want the ID and then query the database to get the rest of the date. To avoid hitting database, I would like to be able to do something like:
fmt.Println(searchResults[0].Bio) // <- This doesn't work :(
Could you please help?

Every hit in the search result is a DocumentMatch. You can see in the documentation that DocumentMatch has Fields which is a map[string]interface{} and can be accessed as follows:
searchResults.Hits[0].Fields["Bio"].(string)
Bleve doesn't include the document's fields in the results by default. You must provide a list of the fields you'd like returned to SearchRequest.Fields (the argument to index.Search). Alternatively, you can set
searchRequest.Fields = []string{"*"}
to return all fields.

Related

Save is trying to update created_at column

We are updating our project from v1 to v2.
When we try to update a row by providing only changed fields as a struct, it tries to set created_at column and returns an error. This was working back in v1. According to documentation, during update operations, fields with default values are ignored.
err := r.writeDB.Save(&Record{
Model: Model{
ID: 1,
},
Name: "AB",
}).Error
if err != nil {
return err
}
Generates the following SQL statement
[3.171ms] [rows:0] UPDATE `records` SET `created_at`='0000-00-00 00:00:00',`updated_at`='2020-11-12 15:38:36.285',`name`='AB' WHERE `id` = 1
Returns following error
Error 1292: Incorrect datetime value: '0000-00-00' for column
'created_at' at row 1
With these entities
type Model struct {
ID int `gorm:"primary_key,type:INT;not null;AUTO_INCREMENT"`
CreatedAt time.Time `gorm:"type:TIMESTAMP(6)"`
UpdatedAt time.Time `gorm:"type:TIMESTAMP(6)"`
}
type Record struct {
Model
Name string
Details string
}
There is DB.Omit which allows ignoring a column when executing an update query. But this requires a lot of refactoring in the codebase. Does the behavior changed or is there something missing?
This might help you. Change the structure field (or add to replace default gorm.Model field) like this:
CreatedAt time.Time `gorm:"<-:create"` // allow read and create, but don't update
This tag helps to save created data from update.
In Gorm v2 Save(..) writes all fields to the database. As the CreatedAt field is the zero value, this is written to the database.
For you code to work, you can use a map:
err := r.writeDB.Model(&Record{Model:Model{ID:1}}).Update("name","AB").Error
Another option is to not fill in the Record struct, but just point to the table and use a Where clause:
err := r.writeDB.Model(&Record{}).Where("id = ?", 1).Update("name","AB").Error
If you have multiple updates you can use Updates(...) with a map, see here: https://gorm.io/docs/update.html
I was able to work around this problem using Omit() before save. Like this:
result := r.db.Omit("created_at").Save(item)
It omits the CreatedAt from the resulting update query, and updates everything else.

Route53: filter ListResourceRecordSets by RecordType

I can't get aws route53 service's ListResourceRecordSets to filter by StartRecord Type. Even with the StartRecordType filter, it returns all records (cname and A) instead of the type I select.
I also noticed I would get a validation error if StartRecordName was not included, so it seems if StartRecordType is used, then StartRecordName is required.
The code below returns all records, but does not filter as it should.
AWSLogin(instance)
svc := route53.New(instance.AWSSession)
listParams := &route53.ListResourceRecordSetsInput{
HostedZoneId: aws.String("Z2798GPJN9CUFJ"), // Required
StartRecordName: aws.String("subdomain.subdomain.mydomain.com"),
StartRecordType: aws.String(route53.RRTypeA),
// StartRecordType: aws.String(route53.RRTypeCname),
}
respList, err := svc.ListResourceRecordSets(listParams)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println("All Type "A" records:")
fmt.Println(respList)
I think you've misunderstood what StartRecordName and StartRecordType do. They do not filter the list, only specify where the list begins.
From the Service Documentation:
If you specify both Name and Type: The results begin with the first resource record set in the list whose name is greater than or equal to Name, and whose type is greater than or equal to Type.
So from your example I would expect all your records to be returned (up to a limit of 100), but the first record will be the A record for subdomain.subdomain.mydomain.com.
It will then proceed (and wrap) in alphabetical order by name/type.

Unable to update jsonb (ie subarray) column

I am just getting my feet wet with go-pg, and having some issues utilizing the JSONB capabilities of PostgreSQL DB.
I have defined my model structs as:
type Chemical struct {
Id int
ChemicalName string
ListingDetails []ListingDetail `sql:",array"`
ParentChemical *Chemical `pg:"fk_parent_chemical_id"`
ParentChemicalId int
}
type ListingDetail struct {
TypeOfToxicity string
ListingMechanism string
CasNo string
ListedDate time.Time
SafeHarborInfo string
}
In the Chemical struct - by using the tag sql:",array" on the ListingDetails field - my understanding is this should get mapped to a jsonb column in the destination table. Using go-pg's CreateTable - a chemicals table gets created with a listing_details jsonb column - so all is good in that regards.
However, I cant seem to update this field (in the db) successfully. When I initially create a chemical record in teh database - there are no ListingInfos - I go back up update them at a later time. Below is a snippet showing how I do that.
detail := models.ListingDetail{}
detail.TypeOfToxicity = strings.TrimSpace(row[1])
detail.ListingMechanism = strings.TrimSpace(row[2])
detail.CasNo = strings.TrimSpace(row[3])
detail.SafeHarborInfo = strings.TrimSpace(row[5])
chemical.ListingDetails = append(chemical.ListingDetails, detail)
err = configuration.Database.Update(&chemical)
if err != nil {
panic(err)
}
But I always get error message shown below. What am I doing wrong??
panic: ERROR #22P02 malformed array literal: "{{"TypeOfToxicity":"cancer","ListingMechanism":"AB","CasNo":"26148-68-5","ListedDate":"1990-01-01T00:00:00Z","SafeHarborInfo":"2"}}"

How to search for a specific value in Firebase using Golang?

I am using Golang and Firego for connecting to Firebase. I am trying to search an admin with Email: john#gmail.com. The following is my Database Structure
For this I have tried:
dB.Child("CompanyAdmins").Child("Info").OrderBy("Email").EqualTo("john#gmail.com").Value(&result)
but it does not produce expected result. How can I do this?
While #dev.bmax has the problem identified correctly, the solution is simpler. You can specify the path of a property to order on:
dB.Child("CompanyAdmins")
.OrderBy("Info/Email")
.EqualTo("john#gmail.com")
.Value(&result)
Update (2017-02-10):
Full code I just tried:
f := firego.New("https://stackoverflow.firebaseio.com", nil)
var result map[string]interface{}
if err := f.Child("42134844/CompanyAdmins").OrderBy("Info/Email").EqualTo("john#gmail.com").Value(&result); err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", result)
This prints:
map[-K111111:map[Info:map[Email:john#gmail.com]]]
Which is the exact place where I put the data.
Update 20170213:
This is the index I have defined:
"CompanyAdmins": {
".indexOn": "Info/Email"
}
If this doesn't work for you, please provide a similarly complete snippet that I can test.
Can you put Info data directly into CompanyAdmins structure? This way, your query will work.
CompanyAdmins
-id
-Email: "johndon#gmail.com"
-Settings:
- fields
The problem with your query, is that Info is not a direct child of CompanyAdmins.
You could use the email as the key instead of an auto-generated one when you insert values. That way, you can access the admin directly:
dB.Child("CompanyAdmins").Child("john#gmail.com").Child("Info")
Otherwise, you need to restructure the database. Your order-by field (email) should be one level higher, like Rodrigo Vinicius suggests. Then, your query will change to:
dB.Child("CompanyAdmins").OrderBy("Email").EqualTo("john#gmail.com")

golang gorp insert multiple records

Using gorp how can one insert multiple records efficiently? i.e instead of inserting one at a time, is there a batch insert?
var User struct {
Name string
Email string
Phone string
}
var users []Users
users = buildUsers()
dbMap.Insert(users...) //this fails compilation
//I am forced to loop over users and insert one user at a time. Error Handling omitted for brevity
Is there a better mechanism with gorp? Driver is MySQL.
As I found out on some other resource the reason this doesn't work is that interface{} and User{} do not have the same layout in memory, therefore their slices aren't of compatible types. Suggested solution was to convert []User{} into []interface{} in for loop, like shown here: https://golang.org/doc/faq#convert_slice_of_interface
There is still on caveat: you need to use pointers for DbMap.Insert()function.
Here's how I solved it:
s := make([]interface{}, len(users))
for i, v := range users {
s[i] = &v
}
err := dbMap.Insert(s...)
Note that &v is important, otherwise Insert will complain about non-pointers.
It doesn't look like gorp has anything that gives a wrapper for either raw SQL or multi value inserts (which is always SQL dialect dependent).
Are you worried about speed or transactions? If not, I would just do the inserts in a for loop.

Resources