I've been struggling with a problem with one of my lists of data because one of the requirements after generating it is to group some of them by some common parameters (more than 1)
What I should get at the end is a map where the value is a list of common objects. For example.
List<Cause> listToGroup = new ArrayList<>();
listToGroup.add(Similar);
listToGroup.add(Common);
listToGroup.add(Similar);
listToGroup.add(Similar);
listToGroup.add(Common);
In a weird way to represent one group (Similar) and the other (Common), those should be separated into two different lists (that list is generated by a request to other methods, in that case, I just added manually to show what could be the contained data in the list). My main problem is the criteria to group them because is based on a group of parameters that are shared, but not all (if the required parameters are equal, should belong to the same list) In the class shown below, that behaviour is seen because there are some parameters that are not being considered.
public class Cause extends GeneralDomain {
//parameters which must be equals between objects
private Long id;
private Date creationDate;
private Part origin;
private Part destination;
//parameters which are not required to be equal
private BigDecimal value
private Stage stageEvent
//omitted getters and setters
}
I've been seeing the comparator method and the groupingBy method provided in Java 8, but at the moment I just know how to perform that task considering just one parameter (for example grouping them by id) And I have no idea about how to group them using more than one parameter.
//this should be the code if the requirement would be just one parameter to groupby, but in my case are more than one.
Map<Long, List<Cause>> result = request.getList(criteria)
.stream()
.map(p -> parsin.createDto(p))
.collect(groupingBy(Cause ::getId));
I would be really glad for any suggestion. If my explanation is not clear, I'm so sorry. That became so complicated that even is hard for me to explain
Related
I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.
I came to a problem where I need to put ordering by multiplication of two columns of entity, for the sake of imagination entity is:
#Entity
public class Entity {
#Column(name="amount")
private BigDecimal amount;
#Column(name="unitPprice")
private BigDecimal unitPrice;
.
.
.
many more columns
}
My repo interface implements JpaRepository and QuerydslPredicateExecutor,
but I am struggling to find a way to order my data by "amount*unitPrice",
as I can't find a way to put it into
PageRequest (new Sort.Order(ASC, "amount * unitPrice"))
without having PropertyReferenceException: No property amount * unitPrice... thrown.
I can't user named query, as my query takes quite massive filter based on user inputs (can't put where clause into query, because if user hasn't selected any value, where clause can't just be in query).
To make it simple. I need something like findAll(Predicate, Pageable), but I need to force that query to order itself by "amount * unitPrice", but also have my Preditate (filter) and Pageable (offset, limit, other sortings) untouched.
Spring Sort can be used only for sorting by properties, not by expressions.
But you can create a unique sort in a Predicate, so you can add this sort-predicate to your other one before you call the findAll method.
I have in a Spring Repo something like this:
findTop10ItemsByCategIdInOrderByInsertDateDesc(List ids)
I want the first 10 items where category id in list of ids ordered by insert date.
Another similar query:
findTop10ItemsByDomainIdAndCategIdInOrderByInsertDateDesc(List ids, #Param Integer domainId)
Here I want that the domain id is equal to the given param and the categId to be in given list.
I managed to resolve it using #Query but I wonder if there is an one liner for the above queries.
thanks
EDIT
The top works fine. Initially I had findTop10ItemsByDomainIdAndCategIdOrderByInsertDateDesc. Now I want the results from a list of category ids. That's the new requirement.
SECOND EDIT
My query works for find the set o results where domain id is equal to a given param and categ id is contained in a given list. BUT I found out that HQL doesn't support a setMaxResult kind of thing as top or limit.
#Query("select i from Items i where i.domainId = :domainId and i.categId in :categoryIds order by i.insertDate desc")
The params for this method were (#Param("domainid") Integer domainid,List<Integer> categoryIds) but it seams that I'm alowed to use either #Param annotation to each parameter or no #Param at all ( except for Pageable return; not my case )
I still don't know how to achieve this think:
extract top n elements where field a eq to param, field b in set of param, ordered by another field.
ps: sorry for tags but there is no spring-crudrepository :)
The method to resolve your problem is:
List<MyClass> findTop10ByDomainIdAndCategIdInOrderByInsertDateDesc(Long domainId, List<Long> ids);
Top10 limits the results to first 10.
ByDomainId limits results to those that have passed domainId.
And adds another constraint.
CategIdIn limits results to those entries that have categId in the passed List.
OrderByInsertDateDesc orders results descending by insert date before limiting to TOP10.
I have tested this query on the following example:
List<User> findTop10ByEmailAndPropInOrderByIdDesc(String email, List<Long> props);
Where User is:
private Long id;
private String username;
private String password;
private String email;
private Long prop;
Currently I would recommend using LocalDate or LocalDateTime for storing dates using Spring Data JPA.
I am using method for finding structure articles:
List<JournalArticle> articles = JournalArticleLocalServiceUtil.getStructureArticles(groupId, "15566", 0, 10000, comparator);
comparator is instance of class:
public class OrderByEventsStartDate extends OrderByComparator{
public int compare(Object o1, Object o2){
System.out.println("comparator test");
return 1;
}
}
Message is never printed and results are not sorted... Any help, please?
results are not sorted...
Because you are always returning 1.
Now to create an OrderByComparator you need to do something more.
If you look at the source code as to how the OrderByComparator is used by the different persistence methods you would get some idea.
Also a look at some of Liferay's implemented Comparators will shed some light as to what you need to do:
Journal Comparators - You can check the ArticleTitleComparator
UserFirstNameComparator
A super-short tutorial on implementing OrderByComparator in liferay
You would require to create 3 fields:
public static final String ORDER_BY_ASC - Specifies the field with which you want to order the results. In the ascending manner.
public static final String ORDER_BY_DESC - Specifies the field with which you want to order the results. In descending order.
public static final String[] ORDER_BY_FIELDS - Specifies the different fields required for sorting (refer UserFirstNameComparator for multiple fields).
Implement the compare method on the field you would like to sort
Impement the getOrderBy() method - refer ArticleTitleComparator
Impement the getOrderByFields() method - refer ArticleTitleComparator
Impement the isAscending() method, required in compare and getOrderBy methods to decide whether it is asc or desc - refer ArticleTitleComparator
Message is never printed and
Because you have not implemented these methods, it ignores the OrderByComparator you have passed. You can check the source code of the method you are using JournalArticleLocalServiceImpl#getStructureArticles
Hope this helps.
I Have a list of string where i want to find particular value and return.
If i just want to search i can use Hashset instead of list
HashSet<string> data = new HashSet<string>();
bool contains = data.Contains("lokendra"); //
But for list i am using Find because i want to return the value also from list.
I found this methos is time consuming. The method where this code resides is hit more than 1000 times and the size of list is appx 20000 to 25000.This method takes time.Is there any other way i can make search faster.
List<Employee> employeeData= new List<Employee>();
var result = employeeData.Find(element=>element.name=="lokendra")
Do we have any linq or any other approach which makes retrievel of data faster from search.
Please help.
public struct Employee
{
public string role;
public string id;
public int salary;
public string name;
public string address;
}
I have the list of this structure and if the name property matches the value "lokendra".then i want to retrun the whole object.Consider list as the employee data.
I want to know the way we have Hashset to get faster search is there anyway we can search data and return fast other than find.
It sounds like what you actually want is a Dictionary<string, Employee>. Build that once, and you can query it efficiently many times. You can build it from a list of employees easily:
var employeesByName = employees.ToDictionary(e => e.Name);
...
var employee;
if (employeesByName.TryGetValue(name, out employee))
{
// Yay, found the employee
}
else
{
// Nope, no employee with that name
}
EDIT: Now I've seen your edit... please don't create struct types like this. You almost certainly want a class instead, and one with properties rather than public fields...
You can try with employeeData.FirstOrDefault(e => e == "lokendra"), but it still needs to iterate over collection, so will have performance list Find method.
If your list content is set only once and then you're searching it again and again you should consider implementing your own solution:
sort list before first search
use binary search (which would be O(log n) instead of O(n) for standard Find and Where)