Implementing filters in ORM golang - go

I am working on an api which takes in parameters for filters (as given below)
/api/endpoint?filter_key_1=filter_value_1&...
I've previously worked on spring where the criteria API allows for dynamically building SQL queries without much hassle. In golang I'm using gorm for handling the ORM operations. Is there anyway to build the queries with optional parameters without writing redundant code?.
For example:
If the request sent is:
/api/endpoint?fk_1=fv_1&fk_2=fv_2&fk_3=fv_3
Query generated should be :
select * from table where fk_1 = fv_1 AND fk_2 = fv_2 AND fk_3 = fv_3
but in case of :
/api/endpoint?fk_1=fv_1
Query generated should be:
select * from table where fk_1 = fv_1
Currently my approach is to check if each variable is present and build the query as a string :
query:="select * from table where "
if fk_1 != ""{
query += "fk_1 = fv_1"
}
... and so on
but this seems very awkward and error prone
Any help will be appreciated! Thanks
EDIT
Building on #bjornaer's answer what helped me was to get the map[string][]string in a form that I can send the same to gorm, map[string]interface{}.
This thread will help in the same.
Now there's no longer a need for redundant checks or string operations in the filters

so it seems to me your question has 2 parts:
you need to retrieve your query values from your url and
insert them to your db query
I don't see how you are handling your requests so let's assume you use the http package: from req.URL you get the URL object and from that calling the Query() method yields a map[string][]string of your query parameters, with those in a variable URLQuery let's pause and look at how you query with gorm:
db, err := gorm.Open(sqlite.Open("gorm.db"), &gorm.Config{
QueryFields: true,
})
here I open a sqlite, then you can pass a variable reference to fill with your query, for example:
result := db.Where(map[string]interface{}{"name": "jinzhu", "age": 20}).Find(&users)
now from the example above, replace your variable in:
result := db.Where(map[string]interface{}URLQuery).Find(&users)
you can find it in the docs

Related

Getting Second Order SQL Injection in Spring Hibernate

I am facing Second Order SQL Injection in the Spring-Hibernate application after scanning through the Checkmarx tool, I have gone through multiple questions in StackOverflow and in other platforms as well but did not get the right finding.
could you please look into the below code snip,
public String getOrderId(order_name){
String returnId= null;
Query query = entityManager.createNativeQuery("select order_id from order where order_name=?");
List<String> dataset = query.setParameter(1,order_name).getResultList();
if(dataset!=null){
returnId = dataset. Get(0);
}
return returnId;
}
In this above method, while calling getResultList(), getting a high vulnerability issue that, this method returns data flows through the code without being properly sanitized or validated, and eventually used in further database query in the method.
Earlier code was like this,
public String getOrderId(order_name){
String returnId= null;
String q = "select order_id from order where order_name="+order_name;
Query query = entityManager.createNativeQuery(q);
and directly it was used as a string append in query, which I have modified with set parameter,
Query query = entityManager.createNativeQuery("select order_id from order where order_name=?");
List<String> dataset = query.setParameter(1,order_name).getResultList();
but still after getting data from query.getResultSet(), it is asking for sanitizing and validating the data before use in further database query method.
and this return data is being used in further query like select * from return_Data where clause. (properly used in where clause to set parameter to avoid SQL injection).
and in the above query is used in another method where we pass return_Data as input to it.
could you please help here to know what checks and validation can be added to overcome this type of issue. Thanks in advance for prompt response.

How to read from Datastore using an Ancestor query and latest golang libraries

I want to read all entities from a Datastore kind (around 6 entities/records).
I have a Datastore that is key'ed on a weird type that I am trying to understand. I can't find any uniqueness on the a key to perform a query on.
The table looks like this:
GCP Datastore representing data I want to read into my Go app
When I click on a record, it looks like this:
Key literal exposed and used from here on out to try and get the records in the Go app
``I can perform an ancestor query in the console like this:```
GCP Datastore queried using Ancestor query
Great! So now I want to retrieve this data from my Golang App? But how?
I see a lot of solutions online about using q.Get(...) // where q is a *Query struct
Any of these solutions won't work because they import google.golang.org/appengine/datastore. I understand that this is legacy and deprecated. So I want a solution that imports cloud.google.com/go/datastore.
I tried something along these lines but didn't get much luck:
First try using GetAll and query
I tried this next:
Second try attempting to use ancestor query... not ready yet
Lastly I tried to get a single record directly:
Lastly I tried to get the record directly
In all cases, my err is not nil and the dts that should be populated from datastore query is also nil.
Any guidance to help me understand how to query on this key type? Am I missing something fundamental with the way this table is key'ed and queried?
Thank you
Then I tried this:
It seems you are just missing your Namespace
// Merchant Struct
type MerchantDetails struct {
MEID string
LinkTo *datastore.Key
Title string
}
// Struct array to store in
var tokens []MerchantDetails
// Ancestor Key to filter by
parentKey := datastore.NameKey("A1_1113", "activate", nil)
parentKey.Namespace = "Devs1"
// The call using the new datastore UI. Basically query.Run(), but datastore.GetAll()
keys, err := helpers.DatastoreClient.GetAll(
helpers.Ctx,
datastore.NewQuery("A1_1112").Ancestor(parentKey).Namespace("Devs1"),
&tokens,
)
if err != nil {
return "", err
}
// Print all name/id from the found values
fmt.Printf("keys: %v", keys)

How to run a Datastore GQL query in Golang?

I have the following query in GQL:
"SELECT * FROM Task WHERE Uuid = \"FOOBAR\" ORDER BY CreateTimeMs DESC LIMIT 1"
How can I directly run this query in Golang?
query := datastore.NewQuery("SELECT * FROM Task WHERE Uuid = \"FOOBAR\" ORDER BY CreateTimeMs DESC LIMIT 1") seems to be incorrect.
UPDATE:
I am very aware of the Query types. The thing is that I would like to use GQL directly instead of translating them into Query types.
I was searching to solve exactly the same problem when I came across this note in the GQL documentation:
Note: Google Cloud Client Library for Java and Google Cloud Client Library for Ruby support GQL, but other Google Cloud client libraries do not.
So that's a no-go ...
However, it should be feasible to implement it over the REST api projects.runQuery but then you have to unmarshal the result into structs by yourself.
Take a look at the Query type documentation, from looking at its methods and reading their respective documentation you should be able to understand how to translate your SQL into datastore.Query specific method calls.
For example your sql can be written as:
q := datastore.NewQuery("Task").Filter("Uuid =", "FOOBAR").Order("-CreateTimeMs").Limit(1)
This just builds your Query value, to get the actual result you need to run the query to get an iterator whose Next method can be used to read the result.
If you want to avoid the run-iterate-next dance you can use the GetAll query method.
var tt = []*Task{}
q := datastore.NewQuery("Task").Filter("Uuid =", "FOOBAR").Order("-CreateTimeMs").Limit(1)
if _, err := q.GetAll(tt); err != nil {
panic(err)
}
log.Println(tt[0]) // your task
Read more here:
Datastore Queries
Retrieving query results
Query Restrictions
Query Cursors

How to search for a specific value in Firebase using Golang?

I am using Golang and Firego for connecting to Firebase. I am trying to search an admin with Email: john#gmail.com. The following is my Database Structure
For this I have tried:
dB.Child("CompanyAdmins").Child("Info").OrderBy("Email").EqualTo("john#gmail.com").Value(&result)
but it does not produce expected result. How can I do this?
While #dev.bmax has the problem identified correctly, the solution is simpler. You can specify the path of a property to order on:
dB.Child("CompanyAdmins")
.OrderBy("Info/Email")
.EqualTo("john#gmail.com")
.Value(&result)
Update (2017-02-10):
Full code I just tried:
f := firego.New("https://stackoverflow.firebaseio.com", nil)
var result map[string]interface{}
if err := f.Child("42134844/CompanyAdmins").OrderBy("Info/Email").EqualTo("john#gmail.com").Value(&result); err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", result)
This prints:
map[-K111111:map[Info:map[Email:john#gmail.com]]]
Which is the exact place where I put the data.
Update 20170213:
This is the index I have defined:
"CompanyAdmins": {
".indexOn": "Info/Email"
}
If this doesn't work for you, please provide a similarly complete snippet that I can test.
Can you put Info data directly into CompanyAdmins structure? This way, your query will work.
CompanyAdmins
-id
-Email: "johndon#gmail.com"
-Settings:
- fields
The problem with your query, is that Info is not a direct child of CompanyAdmins.
You could use the email as the key instead of an auto-generated one when you insert values. That way, you can access the admin directly:
dB.Child("CompanyAdmins").Child("john#gmail.com").Child("Info")
Otherwise, you need to restructure the database. Your order-by field (email) should be one level higher, like Rodrigo Vinicius suggests. Then, your query will change to:
dB.Child("CompanyAdmins").OrderBy("Email").EqualTo("john#gmail.com")

golang gorp insert multiple records

Using gorp how can one insert multiple records efficiently? i.e instead of inserting one at a time, is there a batch insert?
var User struct {
Name string
Email string
Phone string
}
var users []Users
users = buildUsers()
dbMap.Insert(users...) //this fails compilation
//I am forced to loop over users and insert one user at a time. Error Handling omitted for brevity
Is there a better mechanism with gorp? Driver is MySQL.
As I found out on some other resource the reason this doesn't work is that interface{} and User{} do not have the same layout in memory, therefore their slices aren't of compatible types. Suggested solution was to convert []User{} into []interface{} in for loop, like shown here: https://golang.org/doc/faq#convert_slice_of_interface
There is still on caveat: you need to use pointers for DbMap.Insert()function.
Here's how I solved it:
s := make([]interface{}, len(users))
for i, v := range users {
s[i] = &v
}
err := dbMap.Insert(s...)
Note that &v is important, otherwise Insert will complain about non-pointers.
It doesn't look like gorp has anything that gives a wrapper for either raw SQL or multi value inserts (which is always SQL dialect dependent).
Are you worried about speed or transactions? If not, I would just do the inserts in a for loop.

Resources