i'm getting stuck in this problem.
This is my code.
public Mono<Pagination<T>> getAll(Pageable pageable) {
return db.select()
.from(entityClazz)
.page(pageable)
.fetch()
.all()
.map(this::convertToDTO)
.collectList()
.map(responses -> new Pagination<>(responses, pageable.getPageNumber(), pageable.getPageSize(),
total rows in db))
;
}
This is Pagination class:
public final class Pagination<T> {
#JsonProperty("total")
private final long total;
#JsonProperty("page")
private final int page;
#JsonProperty("size")
private final int size;
#JsonProperty("items")
private final List<T> items;
I want use DatabaseClient to get total rows number in database. How i do it?
I have a solution to get total number, but it have a poor performance:
total = getAll().count();
public Flux<T> getAll() {
Flux<E> entities = getRepository().findAll();
return convertToDTO(entities);
}
According to reactive specification there is no pagination at all.
However, if you want to implement it you should:
change your query to apply .limit() and .offset()
get total pages using .count() and not fetching whole data from database
See: https://docs.spring.io/spring-data/commons/docs/current/api/org/springframework/data/repository/reactive/ReactiveCrudRepository.html#count--
Related
I have a paginated endpoint which internally uses Hibernate Criteria to fetch certain objects and relations. The FetchMode is set as FetchMode.JOIN.
When I am trying to hit the endpoint, the request seems to work fine for few pages but is then erring out with :
could not initialize proxy - no Session
Method is as as below:
#Override
public Page<Person> findAllNotDeleted(final Pageable pageable)
{
final var criteria = createCriteria();
criteria.add(Restrictions.or(Restrictions.isNull(DELETED), Restrictions.eq(DELETED, false)));
criteria.setFetchMode(PERSON_RELATION, FetchMode.JOIN);
criteria.setFetchMode(DEPARTMENT_RELATION, FetchMode.JOIN);
criteria.setFirstResult((int) pageable.getOffset());
criteria.setMaxResults(pageable.getPageSize());
criteria.addOrder(asc("id"));
final var totalResult = getTotalResult();
return new PageImpl<>(criteria.list(), pageable, totalResult);
}
private int getTotalResult()
{
final Criteria countCriteria = createCriteria();
countCriteria.add(Restrictions.or(Restrictions.isNull(DELETED), Restrictions.eq(DELETED, false)));
return ((Number) countCriteria.setProjection(Projections.rowCount()).uniqueResult()).intValue();
}
Also, the call to findAllNotDeleted is done from a method anotated with #Transactional.
Not sure what is going wrong.
Any help would be highly appreciated.
EDIT
I read that FetchMode.Join does not work with Restrictions. So I tried implementing it using CriteriaBuilder but again stuck with the issue.
#Override
public Page<Driver> findAllNotDeleted(final Pageable pageable)
{
final var session = getCurrentSession();
final var builder = session.getCriteriaBuilder();
final var query = builder.createQuery(Person.class);
final var root = query.from(Driver.class);
root.join(PERSON_RELATION, JoinType.INNER)
.join(DEPARTMENT_RELATION,JoinType.INNER);
//flow does not reach here.....
var restrictions_1 = builder.isNull(root.get(DELETED));
var restrictions_2 = builder.equal(root.get(DELETED), false);
query.select(root).where(builder.or(restrictions_1,restrictions_2));
final var result = session.createQuery(query).getResultList();
return new PageImpl<>(result, pageable, result.size());
}
The flow does not seem to reach after root.join.
EDIT-2
The relations are as follows:
String PERSON_RELATIONSHIP = "person.address"
String DEPARTMENT_RELATION = "person.department"
and both person, address, department themselves are classes which extend Entity
I guess the associations you try to fetch i.e. PERSON_RELATION or DEPARTMENT_RELATION are collections? In such a case, it is not possible to directly do pagination on the entity level with Hibernate. You would have to fetch the ids first and then do a second query to fetch just the entities with the matching ids.
You could use Blaze-Persistence on top of Hibernate though which has a special pagination API that does these tricks for you behind the scenes. Here is the documentation about the pagination: https://persistence.blazebit.com/documentation/core/manual/en_US/index.html#pagination
There is also a Spring Data integration, so you could also use the Spring Data pagination convention along with Blaze-Persistence Entity-Views which are like Spring Data Projections on steroids. You'd use Page<DriverView> findByDeletedFalseOrDeletedNull(Pageable p) with
#EntityView(Driver.class)
interface DriverView {
Long getId();
String getName();
PersonView getPersonRelation();
DepartmentView getDepartmentRelation();
}
#EntityView(Person.class)
interface PersonView {
Long getId();
String getName();
}
#EntityView(Department.class)
interface DepartmentView {
Long getId();
String getName();
}
Using entity views will only fetch what you declare, nothing else. You could also use entity graphs though:
#EntityGraph(attributePaths = {"personRelation", "departmentRelation"})
Page<Driver> findByDeletedFalseOrDeletedNull(Pageable p);
I'm trying to persist a SparseArray in a Room database and can not get it to compile. I keep getting the "Not sure how to convert a Cursor to this method's return type" error message along with "The query returns some columns [plannerLineData] which are not use by android.util.SparseArray."
I have tried using a single field in the PlannerLine Entity alone with a separate PlannerLineData class.
I have data converters to convert SparseArray to String and to convert String back to SparseArray.
I have checked several questions on stackoverflow and have successfully used the Date to Long and the Long to Date converters in other projects, but I seem to be missing something somewhere.
Data Files:
#Entity
public class PlannerLine implements Serializable {
private static final long serialVersionUID = 1L;
#TypeConverters(Converters.class)
#PrimaryKey
#SerializedName("planner_line")
#NonNull
public SparseArray plannerLineData;
public SparseArray getPlannerLineData() {
return plannerLineData;
}
public void setPlannerLineData(SparseArray plannerLineData) {
this.plannerLineData = plannerLineData;
}
public class PlannerLineData implements Serializable {
#SerializedName("lineId")
public int lineId;
#SerializedName("plan_text")
public String planText;
public int getLineId() {
return lineId;
}
public void setLineId(int lineId) {
this.lineId = lineId;
}
public String getPlanText() {
return planText;
}
public void setPlanText(String planText) {
this.planText = planText;
}
}
DAO problem area:
#Dao
public interface PlannerDao {
#Query("SELECT * from PlannerLine")
public SparseArray getPlannerLine(); <---Doesn't like this line
I have also tried returning SparseArray<PlannerLine> and SparseArray<PlannerLineData>, but no joy.
Converters class:
public class Converters {
#TypeConverter
public static String sparseArrayToString(SparseArray sparseArray) {
if (sparseArray == null) {
return null;
}
int size = sparseArray.size();
if (size <= 0) {
return "{}";
}
StringBuilder buffer = new StringBuilder(size * 28);
buffer.append('{');
for (int i = 0; i < size; i++) {
if (i > 0) {
buffer.append("-,- ");
}
int key = sparseArray.keyAt(i);
buffer.append(key);
buffer.append("-=-");
Object value = sparseArray.valueAt(i);
buffer.append(value);
}
buffer.append('}');
return buffer.toString();
}
#TypeConverter
public static SparseArray stringToSparseArray(String string) {
if (string == null) {
return null;
}
String entrySeparator = "-=-";
String elementSeparator = "-,-";
SparseArray sparseArray = new SparseArray();
String[] entries = StringUtils.splitByWholeSeparator(string, elementSeparator);
for (int i = 0; i < entries.length; i++) {
String[] parts = StringUtils.splitByWholeSeparator(entries[i], entrySeparator);
int key = Integer.parseInt(parts[0]);
String text = parts[1];
sparseArray.append(key, text);
}
return sparseArray;
}
Suggestions would be appreciated. Thanks
Edit:
My original vision for this app was to store all the plan lines in a single SparseArray, along with two additional SparseIntArrays (which I did not mention before because the solution would be similar to the SparseArray) to hold info on how the plan lines interact with each other.
After reading through #dglozano's helpful responses, I have decided to re-design the app to just store regular DB files in Room and load the data into the SparseArray (and the two SparseIntArrays) at startup, use only the in memory SparseArray and SparseIntArrays while the app is active, then write changes in the Sparse Arrays to the DB during onStop(). I am also considering updating the DB in the background as I work through app.
Because the answers and suggestions provided by #dglozano led me to the re-design decision, I am accepting his answer as the solution.
Thanks for the help.
It seems that you are doing the Conversion properly. However, the problem is in your DAO Query:
#Query("SELECT * from PlannerLine") // This returns a List of PlannerLine, not a SparseArray
public SparseArray getPlannerLine(); // The return type is SparseArray, not a List of PlannerLine
Therefore, you can try two different things:
1 - Change the Query to #Query("SELECT plannerLineData FROM PlannerLine WHERE lineId == :lineId") , so that the query returns the SparseArray inside the PlannerLine with id lineId. You should change the method signature so it accepts the parameter lineId
#Query("SELECT plannerLineData FROM PlannerLine WHERE lineId == :lineId")
public SparseArray getPlannerLine(int lineId);
2 - If you want to return the full PlannerLine object and then access to its SparseArray field, then you should change the return type. You should also add the lineId parameter to return just one record and not a list of all the PlannerLine stored in the database table.
#Query("SELECT * FROM PlannerLine WHERE lineId == :lineId")
public PlannerLine getPlannerLine(int lineId);
UPDATE
If you want to get a List<PlannerLine> with all the PlannerLine stored in the database, use the following query in your Dao.
#Query("SELECT * FROM PlannerLine")
public List<PlannerLine> getAllPlannerLines();
Then you can access to the SparseArray of each PlannerLine in the list as usual.
I am having a bit of trouble with paging the results of an aggregation pipeline. After looking at In spring data mongodb how to achieve pagination for aggregation I came up with what feels like a hacky solution. I first performed the match query, then grouped by the field that I searched for, and counted the results, mapping the value to a private class:
private long getCount(String propertyName, String propertyValue) {
MatchOperation matchOperation = match(
Criteria.where(propertyName).is(propertyValue)
);
GroupOperation groupOperation = group(propertyName).count().as("count");
Aggregation aggregation = newAggregation(matchOperation, groupOperation);
return mongoTemplate.aggregate(aggregation, Athlete.class, NumberOfResults.class)
.getMappedResults().get(0).getCount();
}
private class NumberOfResults {
private int count;
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
}
This way, I was able to provide a "total" value for the page object I was returning:
public Page<Athlete> findAllByName(String name, Pageable pageable) {
long total = getCount("team.name", name);
Aggregation aggregation = getAggregation("team.name", name, pageable);
List<Athlete> aggregationResults = mongoTemplate.aggregate(
aggregation, Athlete.class, Athlete.class
).getMappedResults();
return new PageImpl<>(aggregationResults, pageable, total);
}
You can see that the aggregation to get the total count of results is not too different from the actual aggregation that I want to perform:
MatchOperation matchOperation = match(Criteria.where(propertyName).is(propertyValue));
SkipOperation skipOperation = skip((long) (pageable.getPageNumber() * pageable.getPageSize()));
LimitOperation limitOperation = limit(pageable.getPageSize());
SortOperation sortOperation = sort(pageable.getSort());
return newAggregation(matchOperation, skipOperation, limitOperation, sortOperation);
This definitely worked, but, as I was saying, it feels hacky. Is there a way to get the count for the PageImpl instance without essentially having to run the query twice?
your question has helped me get around the same problem of paging with aggregation and so I did a little digging and came up with a solution to your problem. I know it's a bit late but someone might get use out of this answer. I am in no way a Mongo expert so if what I am doing is bad practice or not very performant please don't hesitate to let me know.
Using group, we can add the root documents to a set and also count.
group().addToSet(Aggregation.ROOT).as("documents")
.count().as("count"))
Here is my solution for almost the exact same problem you were facing.
private Page<Customer> searchWithFilter(final String filterString, final Pageable pageable, final Sort sort) {
final CustomerAggregationResult aggregationResult = new CustomerAggregationExecutor()
.withAggregations(match(new Criteria()
.orOperator(
where("firstName").regex(filterString),
where("lastName").regex(filterString))),
skip((long) (pageable.getPageNumber() * pageable.getPageSize())),
limit(pageable.getPageSize()),
sort(sort),
group()
.addToSet(Aggregation.ROOT).as("documents")
.count().as("count"))
.executeAndGetResult(operations);
return new PageImpl<>(aggregationResult.getDocuments(), pageable, aggregationResult.getCount());
}
CustomerAggregationResult.java
#Data
public class CustomerAggregationResult {
private int count;
private List<Customer> documents;
public static class PageableAggregationExecutor {
private Aggregation aggregation;
public CustomerAggregationExecutor withAggregations(final AggregationOperation... operations) {
this.aggregation = newAggregation(operations);
return this;
}
#SuppressWarnings("unchecked")
public CustomerAggregationResult executeAndGetResult(final MongoOperations operations) {
return operations.aggregate(aggregation, Customer.class, CustomerAggregationResult.class)
.getUniqueMappedResult();
}
}
}
Really hope this helps.
EDIT: I had initially created a generic PageableAggregationResult with List but this returns a IllegalArgumentException as I pass PageableAggregationResult.class with no type for T. If I find a solution for this I will edit this answer as I want to be able to aggregate multiple collections eventually.
Am trying to get an object of count(*) and sum() through a hibernate query which actually uses another object. Here's what I'm doing :
String query = select new org.rangde.domain.AggregatedCount(count(*), sum(lps.loanAmount - lps.loanPledged - lps.loanRaised)) from LoanProfileSnapshot lps where lps.loanState in (:loanStates)
List<AggregatedCount> counts = getHibernateTemplate().findByNamedParam(query, params, values);
return counts.size() > 0 ? counts.get(0) : null;
and this is the class AggregatedCount (have removed getters and setters)
public class AggregatedCount {
private int id;
private BigInteger count;
private BigDecimal sum;
public AggregatedCount(){}
public AggregatedCount(BigInteger count, BigDecimal sum){
this.count = count;
this.setSum(sum);
}
}
This is the exception i get when i run the query.
Unable to locate appropriate constructor on class [org.domain.AggregatedCount] ... nested exception is org.hibernate.hql.ast.QuerySyntaxException: Unable to locate appropriate constructor on class [org.domain.AggregatedCount] ...
I tried having the sum as Double and BigInteger and still I'm getting the same exception.
Any help would be appreciated. I'd prefer to stick to findByNamedParam for this because there are a lot of conditions that I'm checking within the code before the final query is generated. Thanks.
Although my answer seems a little bit too late maybe it will help others ..
The necessary datatype is Long for the count and sum depends on the datatype in the entity.
It also depends on the hibernate version you are using:
https://community.jboss.org/wiki/HibernateCoreMigrationGuide35?_sscc=t
public class AggregatedCount {
private Long count;
private Long sum;
public AggregatedCount(Long count, Long sum){
this.count = count;
this.setSum(sum);
}
}
First experiments with Spring Data and MongoDB were great. Now I've got the following structure (simplified):
public class Letter {
#Id
private String id;
private List<Section> sections;
}
public class Section {
private String id;
private String content;
}
Loading and saving entire Letter objects/documents works like a charm. (I use ObjectId to generate unique IDs for the Section.id field.)
Letter letter1 = mongoTemplate.findById(id, Letter.class)
mongoTemplate.insert(letter2);
mongoTemplate.save(letter3);
As documents are big (200K) and sometimes only sub-parts are needed by the application: Is there a possibility to query for a sub-document (section), modify and save it?
I'd like to implement a method like
Section s = findLetterSection(letterId, sectionId);
s.setText("blubb");
replaceLetterSection(letterId, sectionId, s);
And of course methods like:
addLetterSection(letterId, s); // add after last section
insertLetterSection(letterId, sectionId, s); // insert before given section
deleteLetterSection(letterId, sectionId); // delete given section
I see that the last three methods are somewhat "strange", i.e. loading the entire document, modifying the collection and saving it again may be the better approach from an object-oriented point of view; but the first use case ("navigating" to a sub-document/sub-object and working in the scope of this object) seems natural.
I think MongoDB can update sub-documents, but can SpringData be used for object mapping? Thanks for any pointers.
I figured out the following approach for slicing and loading only one subobject. Does it seem ok? I am aware of problems with concurrent modifications.
Query query1 = Query.query(Criteria.where("_id").is(instance));
query1.fields().include("sections._id");
LetterInstance letter1 = mongoTemplate.findOne(query1, LetterInstance.class);
LetterSection emptySection = letter1.findSectionById(sectionId);
int index = letter1.getSections().indexOf(emptySection);
Query query2 = Query.query(Criteria.where("_id").is(instance));
query2.fields().include("sections").slice("sections", index, 1);
LetterInstance letter2 = mongoTemplate.findOne(query2, LetterInstance.class);
LetterSection section = letter2.getSections().get(0);
This is an alternative solution loading all sections, but omitting the other (large) fields.
Query query = Query.query(Criteria.where("_id").is(instance));
query.fields().include("sections");
LetterInstance letter = mongoTemplate.findOne(query, LetterInstance.class);
LetterSection section = letter.findSectionById(sectionId);
This is the code I use for storing only a single collection element:
MongoConverter converter = mongoTemplate.getConverter();
DBObject newSectionRec = (DBObject)converter.convertToMongoType(newSection);
Query query = Query.query(Criteria.where("_id").is(instance).and("sections._id").is(new ObjectId(newSection.getSectionId())));
Update update = new Update().set("sections.$", newSectionRec);
mongoTemplate.updateFirst(query, update, LetterInstance.class);
It is nice to see how Spring Data can be used with "partial results" from MongoDB.
Any comments highly appreciated!
I think Matthias Wuttke's answer is great, for anyone looking for a generic version of his answer see code below:
#Service
public class MongoUtils {
#Autowired
private MongoTemplate mongo;
public <D, N extends Domain> N findNestedDocument(Class<D> docClass, String collectionName, UUID outerId, UUID innerId,
Function<D, List<N>> collectionGetter) {
// get index of subdocument in array
Query query = new Query(Criteria.where("_id").is(outerId).and(collectionName + "._id").is(innerId));
query.fields().include(collectionName + "._id");
D obj = mongo.findOne(query, docClass);
if (obj == null) {
return null;
}
List<UUID> itemIds = collectionGetter.apply(obj).stream().map(N::getId).collect(Collectors.toList());
int index = itemIds.indexOf(innerId);
if (index == -1) {
return null;
}
// retrieve subdocument at index using slice operator
Query query2 = new Query(Criteria.where("_id").is(outerId).and(collectionName + "._id").is(innerId));
query2.fields().include(collectionName).slice(collectionName, index, 1);
D obj2 = mongo.findOne(query2, docClass);
if (obj2 == null) {
return null;
}
return collectionGetter.apply(obj2).get(0);
}
public void removeNestedDocument(UUID outerId, UUID innerId, String collectionName, Class<?> outerClass) {
Update update = new Update();
update.pull(collectionName, new Query(Criteria.where("_id").is(innerId)));
mongo.updateFirst(new Query(Criteria.where("_id").is(outerId)), update, outerClass);
}
}
This could for example be called using
mongoUtils.findNestedDocument(Shop.class, "items", shopId, itemId, Shop::getItems);
mongoUtils.removeNestedDocument(shopId, itemId, "items", Shop.class);
The Domain interface looks like this:
public interface Domain {
UUID getId();
}
Notice: If the nested document's constructor contains elements with primitive datatype, it is important for the nested document to have a default (empty) constructor, which may be protected, in order for the class to be instantiatable with null arguments.
Solution
Thats my solution for this problem:
The object should be updated
#Getter
#Setter
#Document(collection = "projectchild")
public class ProjectChild {
#Id
private String _id;
private String name;
private String code;
#Field("desc")
private String description;
private String startDate;
private String endDate;
#Field("cost")
private long estimatedCost;
private List<String> countryList;
private List<Task> tasks;
#Version
private Long version;
}
Coding the Solution
public Mono<ProjectChild> UpdateCritTemplChild(
String id, String idch, String ownername) {
Query query = new Query();
query.addCriteria(Criteria.where("_id")
.is(id)); // find the parent
query.addCriteria(Criteria.where("tasks._id")
.is(idch)); // find the child which will be changed
Update update = new Update();
update.set("tasks.$.ownername", ownername); // change the field inside the child that must be updated
return template
// findAndModify:
// Find/modify/get the "new object" from a single operation.
.findAndModify(
query, update,
new FindAndModifyOptions().returnNew(true), ProjectChild.class
)
;
}