I want to query all instances of a model by the most recently created.
Reading the official docs, they suggest a way of querying by the default timestamps (updatedAt/createdAt) but only when also querying by another key. So I know I could query a hypothetical User model by name and createdAt, but I can't query all instances of User by createdAt.
Is there an established way of doing this?
I have tried adding a #key directive to sort by updatedAt, but that results in an error because updatedAt is automatically added and not described in my schema. If I then add the timestamps to my schema this creates problems when mutating clients because it expects the timestamps to be added by me, which I obviously don't do because it's automatically added by DynamoDB.
Thanks
You could try using a Global Secondary Index on the field you want to query. In your AppSync resolver, you need to specify the index you want to use for the query.
Another way would be to run a scan operation against your DB (you don't need to specify a key in this case), although that would be way more inefficient than a GSI.
Related
I want to work with Elasticsearch to process some Whatsapp chats. So I am initially planning the data load.
The problem is that the data exported from Whatsapp, doesn't contain a real unique id per user but it only contains the name of the user taken from the contact directory of the device where the chat is exported (ie. a user can change the number or have two numbers in the same group).
Because of that, I need to create a custom explicit mapping table between the user names and a self-generated unique id, that gets populated in an additional column.
Then, my question is: "How can I implement such kind of explicit mapping in Elasticsearch to generate an additional unique column?". Alternatively, a valid answer could be a totally different approach to the problem.
PS. As I write, I think the solution could be in the ingestion process, like in a python script, but I still want to post the question to understand if this is something that Elasticsearch can do by itself.
yes, do it during the index process
if you had the data that maps the name and the id stored in a separate index you could do this with an enrich processor when you index the data to add whichever value you want to the document via a pipeline
also - Elasticsearch doesn't have columns, only fields
My app uses AppSync resolvers to fetch data from DDB and return it to our front-end. One table we have is for Notifications. A Notification can be either pending or default (non-pending). The table itself has a primary key of notification_id and we have a GSI called userIndex to grab the notifications for a user, with a sort key of timestamp.
In the app, I show all notifications in a list, pending first and then default. Given that a user may have many notifications, I'd like to implement pagination to fetch a batch at a time. The only way I've been able to do this is to
change the query to include a isPending parameter, which I use as a filter expression for the query to only return notifications that are isPending or isNotPending.
Store two "nextTokens", one for each isPending and isNotPending, along with corresponding lists.
Make separate queries for pending/non-pending, and use the filter to return to the appropriate list.
This is obviously inefficient and I am re-reading data from DynamoDB. My question is, given my DynamoDB table/requirements, is there a way I can paginate so that I can get all the pending notifications first (sorted by timestamp) and then all the default notifications next (sorted by timestamp) by using one query and one nextToken
I've seen the use of #model and #key, but I haven't been able to make it work in my app.
Thanks!
No, not really. There is a hard limit on returns for a Dynamodb query - and that cannot be bypassed. the only way to make use of nextToken is another query.
However, it is also worth noting that the FilterExpression happens after the data has already been retrieved and is filtered client side. It does not reduce the documents pulled from the query - only whats displayed. So the next token is still going to be (relatively) the same for each query. You can instead filter it yourself after the call before the next pagination query and save yourself a little bit in terms of multiple calls.
Hi I was building an app and was wondering on how I can convert this query into a derived query method without using the annotation #Query:
SELECT address, COUNT(*) address FROM `employee` GROUP BY address ORDER BY address DESC LIMIT 5
I have tried it here is how I did it
List<Employee> countByAddressGroupByAddressByOrderByAddressDescLimit5();
This throws the following error
Invalid derived query! No property groupByAddressBy found for type String! Traversed path: Employee.address.
I was wondering if what am I doing wrong here thank you in advance.
I don't think query derivation supports group by.
You can't.
Derived queries don't support GROUP BY.
Derived queries are intended only for simple queries, where the mapping between a normal method name that you might choose independently from Spring Data and the query needed for implementation is obvious.
For more complex cases like the one you describe other mechanisms are available, like annotated or named queries. Nobody wants to use a method name like countByAddressGroupByAddressByOrderByAddressDescLimit5 anyway.
As others said, derived queries do not support this. There might be other reasons besides simply not being a 'practical' solution.
AFAIK derived query methods are restricted to retrieving instances just for the entity type managed by the repository (you can retrieve MyEntity, Optional<MyEntity>, Collection<MyEntity>, etcetera) or projections that match the fields/columns used for the managed entity.
When you use 'Group By', you break with this resultset structure, you usually have to explicitly indicate which columns you want to retrieve or which operations you want to perform on the grouped columns (which would be impossible using just a method name).
TL;DR
You can't easily indicate the columns you want to include in a 'Grouped By' query result, so no, I don't think there is a way to use this technique with 'Group By'.
I'm using Collection.allow(options.insert) to validate the documents the users inserts into the collection. What I wonder is which validation tests I need to use on the _id property of the inserted doc (I use random strings as id, not the Mongo-style objectId).
Do I need to check that _id is a string looking like an id, or does the database refuse the document if the _id property is invalid? Should I also make sure that no other document in the database has that id?
Strictly speaking you do not need to do any validation tests of the _id. The database will refuse the insert if the _id you give is not unique but I think that is the only rule. Checking that the _id is unique could also be picked up afterwards if the insert errors.
Other checks are optional and are just to allow users to access, insert, or remove the documents you want them to.
As far as I know, all QuickBase API calls are called using the following syntax: http://<quickbase>/db/<dbid>?
Is there a way to get the dbid field without navigating to that database table within QuickBase?
If the above is not possible, would anyone recommend anything other than creating another table that stores the IDs of the tables you want?
With the latter method, I believe I would only need to store one dbid and could pull down the rest (which I believe would still need to be user entered, but would be better than requiring them to change the code).
Thanks for the help!
API_GetSchema will return the list of dbids.
https://www.quickbase.com/db/<yourApplicationId>?act=API_GetSchema&apptoken=<yourApplicationTokenId>&fmt=flat