Does the list returned by a CRUD repository method such as the following have a predictable order?
List<UserProfile> findAllByGroupKey (String groupKey);
The database is MySQL 5 and Spring Boot version is 2.2.7. I'd like the order of the items in the list to be the same as they were stored in the database, but I'm not sure if it's enough to omit `orderBy' to achieve this result. I couldn't find any documentation on this.
I did some testing with the specified configuration (MySQL 5 and Spring Boot 2.2.7). The answer is definitely "no", at least if the index is not a progressive number (in my case it is an UUID).
The order is in no way predictable based on how the data was entered.
The only way to get the rows in the same order as they were entered into the database is to add an additional sort variable (such as a sequence number or a timestamp) and explicitly sort the rows with "orderBy" based on that variable.
Related
I have a csv file of more than 1M records written in English + another language. I have to make a UI that gets a keyword, search through the document, and returns record where that key appears. I look for the key in two columns only.
Here is how I implemented it:
First, I made a postgres database for the data stored in the CSV file. Then made a classic website where the user can enter a keyword. This is the SQL query that I use(In spring boot)
SELECT * FROM table WHERE col1 LIKE %:keyword% OR col2 LIKE %:keyword%;
Right now, it is working perfectly fine, but I was wondering how to make search faster? was using SQL instead of classic document search better?
If the document is only searched once and thrown away, then it's overhead to load into a database. Instead can search the file directly using the nio parallel search feature which uses multiple threads to concurrently search the file:
List<Record> result = Files.lines("some/path")
.parallel()
.unordered()
.map(l -> lineToRecord(l))
.filter(r -> r.getCol1().contains(keyword) || r.getCol2().contains(keyword))
.collect(Collectors.toList());
NOTE: need to provide the lineToRecord() method and the Record class.
If the document is going to be searched over and over again, then can think about indexing the document. This means pre-processing the document to suit the search requirements. In this case it's keywords of col1 and col2. An index is like a map in java, eg:
Map<String, Record> col1Index
But since you have the "LIKE" semantics, this is not so easy to do as it's not as simple as splitting the string by white space since the keyword could match a substring. So in this case it might be best to look for some tool to help. Typically this would be something like solr/lucene.
Databases can also provide similar functionality eg: https://www.postgresql.org/docs/current/pgtrgm.html
For LIKE queries, you should look at the pg_trgm index type with the gin_trgm_ops operator class. You shouldn't need to change query at all, just build the index on each column. Or maybe one multi-column index.
I have a data JPA entity where it contains a "price" type Double. Now, the users need to able to filter the records based on that field (Between min and max). Now the problem is, the value in the DB can be null for some records. My data JPA repository uses a native query like "price BETWEEN :priceFrom AND :priceTo". Now, if the user does not specify anything in the filter conditions, all record including the ones where prices is null should be returned. However, this query does not return those record. I know, I can create a new method with query "price IS NULL" and check the filter values in my service layer and call the null version if nothing is specified. But, I have multiple fields with the same requirement then it results in a lot of duplicate methods to maintain. Is there a better approach to handle that situation?
It seems to me, that you can specify
(:priceFrom is null and :priceTo is null and price is null)
OR price between :priceFrom and :priceTo
if priceFrom and priceTo are entered, second part of OR will be used, otherwise it selects records where price is null
Since you are using Spring Data JPA - Specifications should solve this for you. This is a JPA Criteria based solution.
For any complex-enough API – searching/filtering your resources by very simple fields is simply not enough. A query language is more flexible and allows you to filter down to exactly the resources you need. Hence you should easily be able to program for NULL (in the scenario that you currently need) and anything else that you might need.
This is scalable for multiple fields and easy to code/configure. There are a few links which will give you more insight into it
Spring Blog
Tutorial 1
Tutorial 2
Hopefully, this is helpful.
Need help with Spring JPA. Say I want to find who checkout the car when gotten a traffic ticket. So that licenseNum and
citationDate is greater than checkoutDate and citationDate is less than returnDate; or
citationDate is greater than checkoutDate and returnDate is null.
Currently,
List<SomeEntity> findByLicenseNumAndCheckOutDatetimeBeforeAndReturnDatetimeAfterOrReturnDatetimeIsNullOrderByIdDesc()
produces #1 correctly but produces all licenseNum with returnDate null.
How do I write spring JPA statement with 'or' statement correctly?
You can actually think of using a Query annotation where you can specify a custom query. Yon can choose a more meaningful method name and easy to maintain/change.
I am working with a legacy system which requires doing a query for a partial match on a number field. So I want to do something like
where ID like concat(:num, '%') where num parameter is of the type Long. Is there a way to do that using Spring Data JPA derived queries. If the id was a string then I can simply do
findByIdContaining(String id)
I would appreciate any help.
No, this is not possible with derived queries (those where the query is derived from the method name). But it's easy with an annotated #Query annotation.
I am using elasticsearch with a custom script. I need to create a custom score based on another values defined in another index. Let's call this value "ranking". The score should look like this:
Math.log(_score * 2) + doc['ranking'].value
The ranking is dependent on the provider field in the type that is being scored. My problem is that since then ranking is in another index all together, it doesn't retrieve the value. What is the best way to conduct such a score? Also, how could it be done so that it could cache the results. For example, is there anyway in mvel to retrieve the value based on a redis key-value pair.
You must forget about mvel :)
You should use Groovy scripting
You could check this link for caching technique https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/LookupScript.java, but it's native java plugin. Though it could be translated to groovy script instance.
Another way (but it depends on your index relations, for example for parent/child), you could use has_child query and use score field as a transfer block between indexes, but this is too limited.