Spring Data - Custom DTO Query with filtering - spring

I have a complexe application and I need to retrieve and filter 1000~5000 object for an xls export. Each object having multiple eager relationship (I need them for the export).
If I retrieve all the objects and their relationship as it is, I got some stackoverflow error.
Generaly when I need to make a big export, in order to make it efficient I use a DTO object with an #Query like this :
public interface myRepository extends JpaRepository<Car, Long> {
#Query("SELECT new com.blabla.myCustomObject(p.name, p.surname, c.model, c.number ...) "
+ "FROM Car c "
+ "LEFT JOIN c.person p "
+ "WHERE ... ")
List<myCustomObject> getExportCustomObject();
}
The problem is that the #Query is static and I want to add dynamic filter to my Query (Specifications, Criteria or some other system...)
How to do it ?

Specification cannot be used because this is only the where clause.
But you can use Criteria API. Here's an example. The BasicTeacherInfo is the DTO:
CriteriaQuery<BasicTeacherInfo> query = cb.createQuery(BasicTeacherInfo.class);
Root<Teacher> teacher = query.from(Teacher.class);
query.multiselect(teacher.get("firstName"),teacher.get("lastName"));
List<BasicTeacherInfo> results = em.createQuery(query).getResultList();

You can use #Param annotation to pass dynamic values to HQL, something like:
#Query("SELECT new com.blabla.myCustomObject(p.name, p.surname, c.model, c.number ...) "
+ "FROM Car c "
+ "LEFT JOIN c.person p "
+ "WHERE c.status = :status AND p.name = :name")
List<myCustomObject> getExportCustomObject(
#Param("status") Integer status,
#Param("name") String name
);

Below is one of the possible way where you can try to add offset and limit into your query you can make it dynamic with the help off placeholders.
Below is an sample pseudo code for reference:
Dao Layer:
#Query(value="SELECT e FROM tablename e WHERE condition_here ORDER BY e.id offset :offset limit:limit ")
public returnType yourMethod(String name, int offset, int limit);
Service Layer:
long count = number of records in db.
int a = // number of records to be fetched on each iterations
int num_iterations = count % a ;
int additionalrecords = count / a;
int start= 0;
while(num_iterations>0)
{
dao.yourMethod(start,a);
start = start+a;
count--;
// write your data to excel here
}
dao.yourMethod(start,additionalrecords);
Hope it is helpful.

Related

Pageable in spring for paging on List<Object> is not working

I have a list of object which contain 10000 records i am trying to split that records in each of 10,
But somehow it is not working.. can someone have a look
#Query("select a.applicantid,coalesce(to_char(a.createdon,'yyyy-MM-dd'),to_char(filing_date,'yyyy-MM-dd')) as dt1, \r\n" +
"coalesce(c.companyname,i.InstituteName,(u.firstname||' '||u.lastname),\r\n" +
"u.firstname) ,a.c_denomination,cc.crop_common_name,cs.crop_botanical_name,\r\n" +
"a.id,aps.status,a.cropid, \r\n" +
"(select mv.varietytype from VarietyType mv where mv.id= a.varirtytypeid),\r\n" +
"(select sv.subvarietytype from SubVarietyType sv,VarietyType mvr \r\n" +
" where a.subvarietytypeid = sv.id and mvr.id= sv.varietyid),a.formtype,mcg.crop_group \r\n" +
" from Applications a left join ApplicantRegistration ap on \r\n" +
" a.applicantid = ap.id left join CompanyRegistration c on ap.companyid = c.id \r\n" +
" left join InstitutionRegistration i on ap.institutionid = i.id \r\n" +
" left join Crops cc on a.cropid = cc.id left join CropSpecies cs \r\n" +
" on a.cropspeciesid =cs.id left join InternalUser u on ap.id = u.applicantid \r\n" +
" left join ApplicationStatus aps on a.application_current_status = aps.id "
+ "left join CropGroup mcg on cc.cropgroupid = mcg.id order by a.id desc")
List<Object[]> getapplication_adminview();
List<Object[]> admin_viewapplication=applicationrepository.getapplication_adminview();
int pageNumber = 0;
int size = 10;
Pageable pageable = PageRequest.of(pageNumber, size); // object of pageable
Page<Object> pages = new PageImpl(admin_viewapplication, pageable, admin_viewapplication.size());
List<Object> lpage = pages.getContent(); // here i am getting the lpage size as 10000 but as i enter pageable as of size 10 i am expecting 10 results only
where i am going wrong in this ?
if i am trying to add pagable object to query and run the code i will get the following error:
Cannot create TypedQuery for query with more than one return using requested result type [java.lang.Long]; nested exception is java.lang.IllegalArgumentException: Cannot create TypedQuery for query with more than one return using requested result type [java.lang.Long]
Page just represents one page of data . So page.getContent() only return all data in one page which is specified through constructor when you create this page instance . It has nothing to do with splitting the data in a page.
If you want to split a list , use Lists from Guava is the simplest way to go :
List<List<Object>> splittedList = Lists.partition(list, 10);
If you want to do pagination which split all the data stored in the database into different smaller pages , split it at the database level rather than getting the whole list to the memory to split which will be very inefficient when the entire list is large. See this for how to split it at the database level by declaring Pageable in the query method.
We can use PagedListHolder which can change the list in pages and we can than fetch a page by setting it's page size and page.
PagedListHolder<Object> page = new PagedListHolder(admin_viewapplicationpage);
page.setPageSize(50); // number of items per page
page.setPage(0); // set to first page
int totalPages = page.getPageCount(); // gives the totalpages according to the main list
List<Object> admin_viewapplication = page.getPageList(); // a List which represents the current page which is the sublist
the following tutorial helped me
-> https://www.baeldung.com/spring-data-jpa-query
At this point 4.3. Spring Data JPA Versions Prior to 2.0.4
VERY IMPORTANT to add \ n-- #pageable \ n
Without this I was wrong
Also the pagination setting must be without ordering
PageRequest paginaConf = new PageRequest ((param1 - 1)
, param2);
Finally to convert the Page <Object []>
Page <Object []> list = myQueryofRepo ();
List <XXXModel> lstReturn = myConversor (list.getContent ());
Page <XXXModel> ret = new PageImpl <XXXModel> (lstReturn, pageConf, param2);

Spring Data JPA: Parameterize #Query Annotation String, Refactor Two Similar #Query Methods

In Spring Data I have 2 very large queries which are essentially identical, but with small differences. I don't want to duplicate the methods for both queries. Suppose I have
Method 1
#Query(value = ".. " +
" .. " +
//... big query
"...")
public List<Bean> getResult(#Param("studyId") long studyId);
Method 2
#Query(value = ".. " +
" .. " +
//... big query, after WHERE:
" and (:startDate is null or :startDate = '' or r.recall_date >= to_date(cast(:startDate as TEXT) " +
"...")
public List<Bean> getResult(#Param("studyId") long studyId, #Param("startDate" String startDate);
My goal is:
1) Parameterize the #Query string so that it can either take or omit the optional additional WHERE as a sub-string.
2) Somehow refactor the methods so they don't call separate SQL. The only difference is the additional parameter in Method 2.
Is this possible?
Something like this should work
interface ReportTypeRepository extends PagingAndSortingRepository<ReportType,String> {
final String report = " select r from ReportType r ";
final String where = " where r.active=:active ";
final String sort = " order by r.id asc ";
#Query(report + sort) // <-- all with sort
List<ReportType> findByQuery();
#Query(report + where + sort) // <-- all with where and sort
List<ReportType> findByActiveQuery(#Param("active") boolean active);
}
Another (probably better) solution is to use Spring Data JPA with Querydsl or the JPA 2 Criteria API where you can define some Predicates and combine then to use multiple constraints.
You could also take a look on Specification if you plan to do dynamic queries.

Spring JPA - Issue while sorting on non entity column

I have requirement where I need to get the records based in join of three table with pagination(addition requirement are also there). So I have written a nativeQuery to get the records. Below is the sample query
#Query(value = "SELECT "
+ "a.GROUP_REF_ID as refId "
+ "count(case when c.STAT_CD in :userStatus then (c.grp_user_id) end) as numberOfUsers, "
+ "count(case when b.STAT_CD in :itemStatus then (b.grp_item_id) end) as numberOfItems "
+ "from grp a left join grp_item b on a.grp_id=b.grp_id left join grp_user c on a.grp_id=c.grp_id "
+ "where a.stat_cd in :status and a.co_id in :cids "
+ "group by a.GROUP_REF_ID,a.grp_nam,a.GRP_DESC,a.co_id,a.co_nam,a.CRTE_BY, "
+ "a.CRTE_DT,a.UPDT_BY,a.UPDT_DT ", countQuery = "select count(*) from grp where stat_cd in :status and co_id in :cids ", nativeQuery = true)
public Page<Object> findByStatusAndCompanyIdIn(#Param("status") String status, #Param("cids") List<Long> companyIds,
#Param("userStatus") List<GroupUserStatus> userStatus,
#Param("itemStatus") List<GroupItemStatus> itemStatus, Pageable pageable);
Now the requirement is also that these records are to be sorted on any of the column in select part. So, if user passes numberOfItems, the records are to be sorted on it. But I am facing an issue here because if I pass Sort parameter as numberOfItems, spring prepends an a. before numberOfItems which results in error that not able to find a.numberOfItems.
Is there a way I can stop spring from prepending table alias while creating a query with Sort, or should I write my logic in a different approach
Making my comment an answer so the question can be marked as answered:
Wrap the whole select in another one: select * from (<your current select>) x
I have solved the issue by creating a projection. (Kotlin was used but you’ll get the gist.)
class ShowRepository : JpaRepository<Int, Show> {
#Query("SELECT s AS show, (CASE WHEN (s.status = 'scheduled') THEN s.scheduledStartTime ELSE s.startTime END) AS time FROM Show s")
fun findShows(pageable: Pageable): Page<ShowWithTime>
}
interface ShowWithTime {
val show: Show,
val time: Date?
}
This allows Spring-Data to work its full magic, and using Sort.by(Order.desc("time")) works like a charm.
I’ve written it up with a little bit more detail here: Sorting by a Non-Entity Field.

Room database: how to retrieve column names into list of strings?

I have a Room table named "addresses" with 15 columns. I retrieve one row and want to get values into List< String >, not List< Addresses >. Is that possible?
#Query("SELECT * FROM addresses WHERE myid= :id")
List<String> getAddressAsList(int id);
Moreover, is it possible to retrieve database table column names together with values into list of map <"column name","value"> like this?
#Query("SELECT * FROM addresses WHERE myid= :id")
List<Map<String, String> getAddressAsList(int id);
You can use a SupportSQLiteDatabase but not a Dao.
Say you have, in an activity,
db = yourRoomDatabase.getInstance(this);
i.e. you'd typically then use something like yourDao = db.getYourDao(); and then
myAddressList = dao.getAddressList();
You can then do:-
SupportSQLiteDatabase sdb = db.getOpenHelper().getWritableDatabase();
Cursor csr = sdb.query("SELECT * FROM address WHERE myid=?",new String[]{String.value(id)}
Then to get the list of column names in the Cursor you can use:-
val columnNames = csr.columnNames
To get the Map of columnnames and values you could use :-
val columnsAndValues: MutableMap<String,String> = mutableMapOf()
while (csr.moveToNext()) {
for (i: Int in 0..csr.columnCount) {
columnsAndValues.put(csr.columnNames.get(i),csr.getString(i))
}
}
csr.close()

Spring Data JPA Repository returns Object[] instead of MyType while using Sum Function

Until today I was using this statement:
#Query(value = "select top 5 p.*, sum(po.quantity) as total_quantity from product p " +
"inner join productorder po " +
"on p.id = po.product_id " +
"group by p.id, p.name " +
"order by total_quantity desc", nativeQuery = true)
List<Product> findTopFiveBestSellerNative();
Here as i define the return type as a list of Products, i exactly get what i need. And the selected column total_quantity is simply ignored.
Lastly i needed to integrate pagination into this query. Since Spring does not support pagination handling with native queries, i wanted to first transform this query to JPQL (then i will add pagination). And now it looks like this:
#Query(value = "select p, sum(po.quantity) as total_quantity " +
"from Product p, ProductOrder po " +
"where p.id = po.pk.product " +
"group by p.id, p.name " +
"order by total_quantity desc")
List<Product> findTopFiveBestSeller();
The return type is now a list of object arrays, where the first element of array is Product, and second one is the total_quantity. (Although the method signature says List..)
How can i change my statement or somehow achieve this, so that i do not have to deal with array, and simply just get my Product objects?
EDIT: I had the idea to use this query as a subquery, and just select the products from the subquery. It turns out that the JPQL cannot do subqueries in the 'from' clause..
Fact is your query is not returning only the columns needed to build a Product object, so Spring JPA is mapping it to an object.
Create an aggregate entity that extends product and that contains the property totalQuantity and map your query to it (possibly in another repository)

Resources