Is their any way of increasing the wakanda client pageSize before performing the query to make itreturn us more than 40 value from the datastore.
I tried some query and getting the response to be 40 everytime not more than that.
The attribute of the returned object has a _count of 38654 value in it but i am getting only 40 of them.
You can set the pageSize of the query passing it as an option (query option definition). This options are equivalent to the old Datastore options (datastore query options).
Example:
ds.Company.query({
start: 0,
pageSize: 200
})
This will return the first 200 results of the query (query method definition). Please consult the collection documentation page to find further helpers (nextPage and more for instance) to explore the query results quickly.
It's possible through the REST API to change the number of returned entities. You can use $top or $limit properties.
GET /rest/Employee/$entityset/CB1BCC603DB0416D939B4ED379277F02?$skip=20&$top=10
Source : http://doc.wakanda.org/home2.en.html#/HTTP-REST/Manipulating-Data/toplimit.303-812598.en.html
Related
I have a requirement in which I just need a single row to be returned while querying a table in Dynamodb.
I can see a parameter in aws-cli named 'max-items' which apparently limits the result size of the query. Here is the sample query:
aws dynamodb query --table-name testTable --key-condition-expression "CompositePartitionKey = :pk"
--expression-attribute-values '{ ":pk": { "S": "1234_125" }, ":ps": { "S": "SOME_STATE" }
}'
--filter-expression 'StateAttribute IN (:ps) AND attribute_not_exists(AnotherAttribute)'
--index-name GSI_PK_SK --endpoint-url http://localhost:8000 --max-items 1
But I am not able to figure out any similar keyword/attribute in Go.
Here is what I could find relevant: How to set limit of matching items returned by DynamoDB using Java?
As you can see in DynamoDB's official description of the "Query" operation - https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html - the parameter you are looking for is "Limit". Please consult your Go library's documentation on how exactly to pass this parameter to the query.
By the way, note that Limit doesn't quite limit the number of returned results, but rather the number of rows read at the server side. If your query has a filter, it can return fewer than Limit results. I don't know whether this matters to you or not.
You might want to look into pagination. You can use page-size to control how much you can get in each query.
More details on pagination can be found on Paginating Table Query Results.
You need to paginate through DynamoDB's responses until your "user limit" is fulfilled.
I am trying to get all documents in an index, I tried the following-
1) getting the total number of records first and then setting /_search?size= parameter -doesn't work as size parameter is restricted to 10000
2)tried paginating by making multiple calls and used the parameters '?size=1000&from=9000'
-worked till 'from' was < 9000 but after it exceeds 9000 i again get this size restriction error-
"Result window is too large, from + size must be less than or equal to: [10000] but was [100000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting"
So how can I retrieve all documents in the index?I read some answers suggesting to use the scroll api and even the documentation states -
"While a search request returns a single “page” of results, the scroll API can be used to retrieve large numbers of results (or even all results) from a single search request, in much the same way as you would use a cursor on a traditional database."
But I couldn't find any sample query to get all records in a single request.
I have a total of 388794 documents in the index.
Also note, this is a one time call so I am not worried about performance concerns.
Figured out the solution-
Scroll api is the proper way to do it- here's how its working-
In the first call to fetch the documents, a size say 1000 can be provided and scroll parameter specifying the time in minutes after which search context times out.
POST /index/type/_search?scroll=1m
{
"size": 1000,
"query": {....
}
}
For all subsequent calls we can use the scroll_id returned in the response of the first call to get the nest chunk of records.
POST /_search/scroll
{
"scroll" : "1m",
"scroll_id" : "DnF1ZXJ5VGhIOLSJJKSVNNZZND344D123RRRBNMBBNNN==="
}
I have a user object with dozens of keys on it. Is it possible to give a query some parameters such that only certain fields on the user come back and I have to fetch the rest?
To limit the keys in the returned result set you can use 'keys' parameter in rest api.
Sample GET request:
https://demodomain/parse/classes/ClassName?keys=key1,key2,key3
For more:
https://docs.parseplatform.org/rest/guide/#query-constraints
I am trying to fetch more than 10000 doc with jest client.
I used scroll feature and use a query size of 50, but my program goes into an infinite loop and in every iteration returns the same 50 doc results.
I guess it is problem with scroll id which I am not passing can some body help.
Below is the call to be made to retrieve the first 50 records :
POST <host_name>:<port_num>/<index_name>/_search?scroll=1m&size=50
As shown above, the size is mentioned as 50 and scroll is 1m, this means that the scroll api will retrieve 50 records per hit and this scroll is available for 1 minute. Also, this api returns a scroll id, which should be used for further retrieval of records. Please find the sample below:
POST <host_name>:<port_num>/_search?scroll=1m&scroll_id=<scroll_id>
Note : For further scroll api calls, index name need not be mentioned. Only the scroll_id and scroll time is sufficient.
For more information, please refer to the elastic search documentation on scroll api : https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
I find myself having to write two db queries and I would like to know whether it's possible to reduce to one.
I am updating a document and then finding the document again in the database. Is it possible to update and receive the full updated document in the response of the update query? This would save a call to the db.
I should explain that the document I'm saving won't be a complete document which is why I'm having to retrieve it from the database to get all the fields.
Yes, it is possible. Please have a look at the documentation of the Query.Apply method, which runs the findAndModify MongoDB command.
Straight from the documentation, this example increments a counter and prints its new value:
change := mgo.Change{
Update: bson.M{"$inc": bson.M{"n": 1}},
ReturnNew: true,
}
info, err = col.Find(M{"_id": id}).Apply(change, &doc)
fmt.Println(doc.N)