The problem that I am facing is not that I'm not able to perform a sort, but rather a correct sort. That is, my objects that are stored as jsonb need to be sorted before getting displayed in a table. Part of the query that sorts is:
ORDER BY data ->> 'Name' ASC
However the problem is that at its current state, psql returns the list of people ordered by two clusters: upper and lower case. ASC sort returns sorted upcase + sorted downcase while DESC returns inverted sort downcase + inverted sort upcase.
Is there a trick behind sorting the data in a caseless order or does the data need to initially be stored in a particular case.
ORDER BY lower(data ->> 'Name') ASC
This does create a temporary fix, but I will be glad if there are other methods out there
Sorting by jsonb value works the same as sorting by simple text field. If you get case-sensitive sorting, you likely set incorrect collation to your database.
See this issue, answer by Michał Niklas.
Related
I am sorting on created field which is in the form of 2022-03-26T03:56:13.176+00:00 and is a representation of a java.time.LocalDateTime.
Sorting by only created is not consistent as the field is not unique, due to batch operations that run quickly enough to result in duplicates.
I've added a second sort, on _id, which is ObjectId.
It seems the second sort adds quite a bit of time to the query, more so than the first, which is odd to me.
Why does it more than double response time, and is there a more preferred way to ensure the order?
Using MongoTemplate, I sort like this:
query.with(Sort.by(Sort.Order.desc("createdOn"), Sort.Order.desc("_id")));
If your use case is always to sort in descending order on both fileds it is best to create the compound index in the expected sort order as follow:
db.collection.createIndex({ createdOn:-1,_id:-1 })
But in general the default _id field is containing the document insertion date and it is unique accross mongodb process so you may just sort based on _id , you most porbably don't need to sort additionally on createdOn date ...
I'm new to rethinkdb and i love it, but i found some problems when i tried to optimize my query and make it work on bigger datasets.
The problem is simple.
I need to filter my "event" table by timestamp (row.to) , by tag (row.tags), order by timestamp (row.from) and then slice for pagination.
row.tags has a multi index and works well!
row.from and row.to are start/end time of Event.
The slow query (testeded on 100k entries) is this:
r.db("test").table("event")
.getAll(r.args(["148a6e03-b6c3-4092-afa0-3b6d1a4555cd","7008d4b0-d859-49f3-b9e0-2e121f000ddf"]), {"index": "tags"})
.filter(function(row) {return row("to").ge(r.epochTime(1480460400));})
.orderBy(r.asc("from"))
.slice(0,20)
I created an index on 'from' and tried to do
.orderBy(r.asc("from"),{index:'from'})
but i get
e: Indexed order_by can only be performed on a TABLE or TABLE_SLICE in:
I already read about problems about index intersection in Rethinkdb, but maybe i miss something, maybe there is a way of doing this simple task.
Thank you.
The reason RethinkDB complains is this:
getAll returns a selection. When filter is applied to a selection it returns a selection. When orderBy is applied to a selection the index parameter can't be used (it can only be used when orderBy is applied to a table).
orderBy can be applied to a table, sequence or selection. Only when it's applied to table can the index parameter be used. This makes sense as the index is updated when rows are added and removed from the table.
In your case, you are applying orderBy on a result of filter which is a selection. In order to sort a selection the database needs to:
read all elements into memory (by default max is 100,000 elements)
sort them using the provided function or field
and it can't use index in this case.
The way to improve your query might be to sort the table first and then apply the filter. You will be able to use the index in this case.
If I filter records in a query using RethinkDB's between() operator on the primary key/index, do I need to explicitly sort the resulting record set by primary key, or is the resulting record set guaranteed to be sorted?
You need to explicitly order it. (This is because the data is sharded, so which means it's faster to return it in an unspecified order, so that's the default if you don't request an ordering.)
I am struck in a data operation where I want to sort results of a query by a Ref field.
Lets say I have the following Data Objects.
EmployeeDO {Long id, String name, Ref refCompany}
CompanyDO {Long id, String name}
Now i want to query employees arranged by company name.
I tried the query
Query<EmployeeDO> query = ofy().load().type(EmployeeDO.class).order("refCompany");
Obviously this did not sort results with company name, but this compiled successfully too.
Please suggest if such sorting is possible by this way or some other workaround can be tried?
You can order by refCompany if you #Index refCompany, but it won't sort by company name - it will index by the key (if you aren't using #Parent, just an id order).
There are two 'usual' choices:
Load the data into ram and sort there. This is what an RDBMS would do internally. It's not exactly true that GAE doesn't support joins; it's just that you're the query planner.
Denormalize and pre-index the companyName. Put #Index companyName in the EmployeeDO. This is what you would do with an RDBMS if you the magic sorting performed poorly (say, there are too many employees).
I got my lucene index with a field that needs to be sorted on.
I have my query and I can make my Sort object.
If I understand right from the javadoc I should be able to do query.SetSort(). But there seems to be no such method...
Sure I'm missing something vital.
Any suggestions?
There are actually two important points. First, the field must be indexed. Second, pass the Sort object into the overloaded search method.
Last time I looked, the docs didn't do a very good job of pointing out the indexing part, and certainly didn't explain why this is so. It took some digging to find out why.
When a field is sortable, the searcher creates an array with one element for each document in the index. It uses information from the term index to populate this array so that it can perform sorting very quickly. If you have a lot of documents, it can use a lot of memory, so don't make a field sortable unless there is a need.
One more caveat: a sortable field must have no more than one value stored in each field. If there are multiple values, Lucene doesn't know which to use as the sort key.
It looks like the actual method you want is e.g. Searcher.search(Query query, Filter filter, int n, Sort sort). setSort is a method of Sort.