I want to implement filtering for multiple fields of an entity which is ordered by best match. By best match I mean that the more of the filtered fields match the higher in the order the result is listed. I want this to work dynamically, so I can add more filters later on.
I have been looking for a solution for a long time now and I didn't find an elegant way to do this with JPA.
My approach is to concatenate all my predicates with or and then order them by how many of the fields match. This is done by dynamically creating a CASE statement for each possible combination of the filters (this is a powerset and leads to a lot of CASE statements). Then I give every subset a rank (= size of the subset) and then I sort by the rank in descending order. This way subsets with more elements (= filters) are ranked higher.
From a few tests I can see that I already takes up to 10s for 4 filters, so that can't be a good solution.
Here is my code:
private fun orderByBestMatch(): Specification<User?> {
return Specification<User?> { root: Root<User?>, query: CriteriaQuery<*>, builder: CriteriaBuilder ->
val benefit = getExpressionForNestedClass<String>(root, "benefit")
val umbrellaTerm = getExpressionForNestedClass<String>(root, "umbrellaTerm")
val specialization = getExpressionForNestedClass<String>(root, "specialization")
val salaryExpectation = root.get<Number>("salaryExpectation")
val matcher: CriteriaBuilder.Case<Int> = builder.selectCase()
for (set in powerSetOfsearchedFields()) {
if(set.isNotEmpty()) {
var predicate: Predicate? = when(set.elementAt(0).key) {
"umbrellaTerm" -> builder.like(umbrellaTerm, set.elementAt(0).value.toString())
"specialization" -> builder.like(specialization, set.elementAt(0).value.toString())
"benefit" -> builder.like(benefit, set.elementAt(0).value.toString())
"salaryExpectation" -> builder.equal(salaryExpectation, set.elementAt(0).value.toString())
else -> null
}
for (i in 1 until set.size) {
predicate = when(set.elementAt(1).key) {
"umbrellaTerm" -> builder.and(predicate, builder.like(umbrellaTerm, set.elementAt(1).value.toString()))
"specialization" -> builder.and(predicate, builder.like(specialization, set.elementAt(1).value.toString()))
"benefit" -> builder.and(predicate, builder.like(benefit, set.elementAt(1).value.toString()))
"salaryExpectation" -> builder.and(predicate, builder.equal(salaryExpectation, set.elementAt(1).value.toString()))
else -> null
}
}
matcher.`when`(predicate, set.size)
}
}
matcher.otherwise(0)
query.orderBy(builder.desc(matcher))
query.distinct(true)
builder.isTrue(builder.literal(true))// just here for the function to have a return value
// result?.toPredicate(root, query, builder)
}
}
This function is used in a Builder class I implemented and is appended to the Specification with an and when building the Specification.
The Specification is then passed to UserRepository.findall().
Is there a better way (maybe even an out of the box way) to implement this behaviour?
Thanks in advance
Related
I expect to receive two elements after writing three to the database. The query returns a Flux, which returns the elements unsorted.
#Test
fun `SUCESSFULLY query pending purchases`() {
// arrange
val arrived = TestDataFactory.buchungssatzEntity(
sequentialId = 1,
arrived = true
)
val pending = TestDataFactory.buchungssatzEntity(
sequentialId = 2,
arrived = false
)
val next_pending = TestDataFactory.buchungssatzEntity(
sequentialId = 3,
arrived = false
)
// act
buchhaltungWriter.save(arrived)
buchhaltungWriter.save(pending)
buchhaltungWriter.save(next_pending)
val purchases = inventoryFinder.findAllPendingPurchases()
// assert
StepVerifier.create(purchases)
.expectNext(pending)
.expectNext(next_pending)
.verifyComplete()
}
This always throws
expected value: BuchungssatzEntity(... sequentialId=2 ...); actual value: BuchungssatzEntity(... sequentialId=3 ...)
I do not want to add sorting to the database query, it's a waste of time. How can I test Flux with 'all of these elements but in any order' or should I just go for a Mono in this case, return a list and do the rest myself? Then again, that's not why I'm using reactive.
PS: Goal of this test is to verify that my Spring query language statement filters correctly.
-------- UPDATE
I solved it by using
StepVerifier.create(inventoryFinder.findAllPendingPurchases())
.recordWith { listOf<PurchasesModel>() }
.thenConsumeWhile { i: Any? -> purchases.contains(i) }
.consumeRecordedWith{ }
.verifyComplete()
One way to do that is to use the thenConsumeWhile operator along with expectNextCount. If there is any element in the sequence that doesn't match, the StepVerifier will error.
In Java:
List<Item> pendingItems = List.of(pending, next_pending);
Flux<Item> items = inventoryFinder.findAllPendingPurchases();
StepVerifier.create(items)
.expectNextCount(2) //expecting 2 elements
.verifyComplete();
StepVerifier.create(items)
.thenConsumeWhile((i) -> pendingItems.contains(i)) // check if element was expected
.verifyComplete();
I have an array of object User(val name: String, val role: String). Role can be: leader, student, teacher, parent. And should be sorted in that order. I've read some article about Comparator but haven't figure out how they work. Can you please add some explanation too?
You can instantiate a HashMap for indicating roles priority:
private val roles: HashMap<String, Int> = hashMapOf(
"Leader" to 0,
"Student" to 1,
"Teacher" to 2,
"Parent" to 3
)
fun sortUsers(users: ArrayList<User>): ArrayList<User> {
val comparator = Comparator { o1: User, o2: User ->
return#Comparator roles[o1.role]!! - roles[o2.role]!!
}
val copy = arrayListOf<User>().apply { addAll(users) }
copy.sortWith(comparator)
return copy
}
In the comparator, if the result of the subtraction is negative, it will sort it in the order [o1, o2]. Otherwise, [o2, o1]. So in this case, it will sort your list in an ascending manner (since we have the highest priority as 0 - Leader as indicated in the HashMap).
For example, if we have o1 as Teacher and o2 as Leader,
the result of the comparator will be: 2 - 0 = 2 which is a positive integer. Hence, it is sorted as [o2 (Leader), o1 (Teacher)].
Reversing the roles however yields the opposite result: 0 - 2 = -2 and hence it will be ordered [o1 (Teacher), o2 (Leader)].
You can verify this result with:
fun main(args: Array<String>) {
val users = arrayListOf(
User("john", "Student"),
User("tim", "Teacher"),
User("nice", "Student"),
User("hey", "Leader"),
User("you", "Parent")
)
println(sortUsers(users))
}
Which prints out: [User(name=hey, role=Leader), User(name=john, role=Student), User(name=nice, role=Student), User(name=tim, role=Teacher), User(name=you, role=Parent)]
I believe the simplest way to solve your problem is to use enum classes, since they already implement the Comparable interface. You can't use String directly since it would use the String compareTo method, which sorts them alphabetically.
Essentially you would have to do something like this:
enum class Role { LEADER, STUDENT, TEACHER, PARENT }
data class User(val name: String, val role: Role)
fun main(args: Array<String>) {
val trumpino = User("trumpino", Role.STUDENT)
val obamino = User("barack-obamino", Role.PARENT)
val bushino = User("george-bushino", Role.TEACHER)
val kennedino = User("kennedino", Role.LEADER)
val mlist = listOf<User>(trumpino, obamino, bushino, kennedino)
val result = mlist.sortedBy { it.role }
println(result)
}
But that would require you to change other pieces of your code if you're already using strings. You could possibly add an extension method to parse the string into an enum or use the valueOf method which is explained in this other answer.
The comparator interface is meant to be used with the functional constructs from the kotlin library, in constrast with the comparable interface which is meant to represent the inherent ordering from your class (if it has one). If you need to order your data in a way that's different from its normal ordering (which would be defined by the comparable interface), you use comparator with some ordering method like sortedWith.
With comparator you could do something like, which specify a complex ordering in a simple manner:
mlist.sortedWith(compareByDescending { it.role }.thenBy { it.name })
In contrast to the comparable interface:
class User(val name: String, val role: Role) : Comparable {
override operator fun compareTo(other: User) : Int {
return when {
other.role > this.role -> 1
other.role < this.role -> -1
other.name > this.name -> 1
other.name < this.name -> -1
else -> 0
}
}
}
They both act in the same way in the sense that the value they return is meant to represent the ordering of between this and other, as explained by the kotlin documentation of compareTo:
Compares this object with the specified object for order. Returns zero if this object is equal to the specified other object, a negative number if it's less than other, or a positive number if it's greater than other.
But since comparator is a functional construct, it has a diferent interface from its OOP counterpart as it's meant to be used with lambda expressions.
abstract fun compare(a: T, b: T): Int
But it acts in the same way, as explained in the java documentation for Comparator
Returns a negative integer, zero, or a positive integer as the first argument is less than, equal to, or greater than the second.
its so simple if you use kotlin, you can use sort method. for example
you have list like
val listOfUser = listOf<User>(
User("raka", "teacher"),
User("adi", "student"),
User("steve", "student"),
User("mark", "student"),
User("jack", "operator")
)
if you want to sort, you just access your variable and use sort method. like this
listOfUser.sortedBy {it.role} // if you want to sort ASCENDING by roles
listOfUser.sortedByDescending { it.role } // if you want to sort DESC by roles
Let your array be
val listOfUser = listOf<User>(
User("Ben", "Teacher"),
User("Bheem", "Student"),
User("Steve", "Student"),
User("Fora", "Student"),
User("Jacker", "Operator")
)
To sort this array you can use sortedWith method and a comparator of your object whose return type is also list is as shown below
val sortedUsersList = listOfUser.sortedWith(compareBy { it.role })
If you use sortBy or sortByDescending methods their return will be Unit but not list.
Model:
public class AgencyMapping {
private Integer agencyId;
private String scoreKey;
}
public class AgencyInfo {
private Integer agencyId;
private Set<String> scoreKeys;
}
My code:
List<AgencyMapping> agencyMappings;
Map<Integer, AgencyInfo> agencyInfoByAgencyId = agencyMappings.stream()
.collect(groupingBy(AgencyMapping::getAgencyId,
collectingAndThen(toSet(), e -> e.stream().map(AgencyMapping::getScoreKey).collect(toSet()))))
.entrySet().stream().map(e -> new AgencyInfo(e.getKey(), e.getValue()))
.collect(Collectors.toMap(AgencyInfo::getAgencyId, identity()));
Is there a way to get the same result and use more simpler code and faster?
You can simplify the call to collectingAndThen(toSet(), e -> e.stream().map(AgencyMapping::getScoreKey).collect(toSet())))) with a call to mapping(AgencyMapping::getScoreKey, toSet()).
Map<Integer, AgencyInfo> resultSet = agencyMappings.stream()
.collect(groupingBy(AgencyMapping::getAgencyId,
mapping(AgencyMapping::getScoreKey, toSet())))
.entrySet()
.stream()
.map(e -> new AgencyInfo(e.getKey(), e.getValue()))
.collect(toMap(AgencyInfo::getAgencyId, identity()));
A different way to see it using a toMap collector:
Map<Integer, AgencyInfo> resultSet = agencyMappings.stream()
.collect(toMap(AgencyMapping::getAgencyId, // key extractor
e -> new HashSet<>(singleton(e.getScoreKey())), // value extractor
(left, right) -> { // a merge function, used to resolve collisions between values associated with the same key
left.addAll(right);
return left;
}))
.entrySet()
.stream()
.map(e -> new AgencyInfo(e.getKey(), e.getValue()))
.collect(toMap(AgencyInfo::getAgencyId, identity()));
The latter example is arguably more complicated than the former. Nevertheless, your approach is pretty much the way to go apart from using mapping as opposed to collectingAndThen as mentioned above.
Apart from that, I don't see anything else you can simplify with the code shown.
As for faster code, if you're suggesting that your current approach is slow in performance then you may want to read the answers here that speak about when you should consider going parallel.
You are collecting to an intermediate map, then streaming the entries of this map to create AgencyInfo instances, which are finally collected to another map.
Instead of all this, you could use Collectors.toMap to collect directly to a map, mapping each AgencyMapping object to the desired AgencyInfo and merging the scoreKeys as needed:
Map<Integer, AgencyInfo> agencyInfoByAgencyId = agencyMappings.stream()
.collect(Collectors.toMap(
AgencyMapping::getAgencyId,
mapping -> new AgencyInfo(
mapping.getAgencyId(),
new HashSet<>(Set.of(mapping.getScoreKey()))),
(left, right) -> {
left.getScoreKeys().addAll(right.getScoreKeys());
return left;
}));
This works by grouping the AgencyMapping elements of the stream by AgencyMapping::getAgencyId, but storing AgencyInfo objects in the map instead. We get these AgencyInfo instances from manually mapping each original AgencyMapping object. Finally, we're merging AgencyInfo instances that are already in the map by means of a merge function that folds left scoreKeys from one AgencyInfo to another.
I'm using Java 9's Set.of to create a singleton set. If you don't have Java 9, you can replace it with Collections.singleton.
Iterator<Rate> rateIt = rates.iterator();
int lastRateOBP = 0;
while (rateIt.hasNext())
{
Rate rate = rateIt.next();
int currentOBP = rate.getPersonCount();
if (currentOBP == lastRateOBP)
{
rateIt.remove();
continue;
}
lastRateOBP = currentOBP;
}
how can i use above code convert to lambda by stream of java 8? such as list.stream().filter().....but i need to operation list.
The simplest solution is
Set<Integer> seen = new HashSet<>();
rates.removeIf(rate -> !seen.add(rate.getPersonCount()));
it utilizes the fact that Set.add will return false if the value is already in the Set, i.e. has been already encountered. Since these are the elements you want to remove, all you have to do is negating it.
If keeping an arbitrary Rate instance for each group with the same person count is sufficient, there is no sorting needed for this solution.
Like with your original Iterator-based solution, it relies on the mutability of your original Collection.
If you really want distinct and sorted as you say in your comments, than it is as simple as :
TreeSet<Rate> sorted = rates.stream()
.collect(Collectors.toCollection(() ->
new TreeSet<>(Comparator.comparing(Rate::getPersonCount))));
But notice that in your example with an iterator you are not removing duplicates, but only duplicates that are continuous (I've exemplified that in the comment to your question).
EDIT
It seems that you want distinct by a Function; or in simpler words you want distinct elements by personCount, but in case of a clash you want to take the max pos.
Such a thing is not yet available in jdk. But it might be, see this.
Since you want them sorted and distinct by key, we can emulate that with:
Collection<Rate> sorted = rates.stream()
.collect(Collectors.toMap(Rate::getPersonCount,
Function.identity(),
(left, right) -> {
return left.getLos() > right.getLos() ? left : right;
},
TreeMap::new))
.values();
System.out.println(sorted);
On the other hand if you absolutely need to return a TreeSet to actually denote that this are unique elements and sorted:
TreeSet<Rate> sorted = rates.stream()
.collect(Collectors.collectingAndThen(
Collectors.toMap(Rate::getPersonCount,
Function.identity(),
(left, right) -> {
return left.getLos() > right.getLos() ? left : right;
},
TreeMap::new),
map -> {
TreeSet<Rate> set = new TreeSet<>(Comparator.comparing(Rate::getPersonCount));
set.addAll(map.values());
return set;
}));
This should work if your Rate type has natural ordering (i.e. implements Comparable):
List<Rate> l = rates.stream()
.distinct()
.sorted()
.collect(Collectors.toList());
If not, use a lambda as a custom comparator:
List<Rate> l = rates.stream()
.distinct()
.sorted( (r1,r2) -> ...some code to compare two rates... )
.collect(Collectors.toList());
It may be possible to remove the call to sorted if you just need to remove duplicates.
I have a very basic sql view which joins 3 tables: users, pictures, and tags.
How would one create the query below in a way that it won't list the same pictures more than once? In other words, I want to Group By pictures (I think) and return get the first insance of each.
I think this is very similar to the post Linq Query Group By and Selecting First Items, but I cannot figure out how to apply it in this case where the query is instantiating MyImageClass.
validPicSummaries = (from x in db.PicsTagsUsers
where x.enabled == 1
select new MyImageClass {
PicName = x.picname,
Username= x.Username,
Tag = x.tag }).Take(50);
To exclude duplicates, you can use the Distinct LINQ method:
validPicSummaries =
(from x in db.PicsTagsUsers
where x.tag == searchterm && x.enabled == 1
select new MyImageClass
{
PicName = x.picname,
Username= x.Username,
Tag = x.tag
})
.Distinct()
.Take(50);
You will need to make sure that the objects are comparable so that two MyImageClass objects that have the same PicName, Username, and Tag are considered equal (or however you wish to consider two of them as being equal).
You can write a small class that implements IEqualityComparer<T> if you would like to have a custom comparer for just this case. Ex:
private class MyImageClassComparer : IEqualityComparer<MyImageClass>
{
public bool Equals(MyImageClass pMyImage1, MyImageClass pMyImage2)
{
// some test of the two objects to determine
// whether they should be considered equal
return pMyImage1.PicName == pMyImage2.PicName
&& pMyImage1.Username == pMyImage2.Username
&& pMyImage1.Tag == pMyImage2.Tag;
}
public int GetHashCode(MyImageClass pMyImageClass)
{
// the GetHashCode function seems to be what is used by LINQ
// to determine equality. from examples, it seems the way
// to combine hashcodes is to XOR them:
return pMyImageClass.PicName.GetHashCode()
^ pMyImageClass.UserName.GetHashCode()
^ pMyImageClass.Tag.GetHashCode();
}
}
Then when you call distinct:
...
.Distinct(new MyImageClassComparer())
.Take(50);