MongoDB embedded Document Array: Get only one embedded document with a spezific attribute - spring

I want to get one Embedded Document with a specific field (version) from an array with mongodb and spring boot.
This is the data structure:
{
"_id": 5f25882d28e40663719d0b52,
"versions": [
{
"versionNr": 1
"content": "This is the first Version of some Text"
},
{
"versionNr": 2
"content": "This is the second Version of some Text"
},
...
]
...
}
Here are my entities:
#Data
#Document(collection = "letters")
public class Letter {
#Id
#Field("_id")
private ObjectId _id;
#Field("versions")
private List<Version> versions;
}
//There is no id for embedded documents
#Data
#Document(collection = "Version")
public class Version{
#Field("content")
private String content;
#Field("version")
private Long version;
}
And this is the query that doesn't work. I think the "join" isn't correct. But can't figure out the right way.
public Optional<Version> findByIdAndVersion(ObjectId id, Long version) {
Query query = new Query(Criteria.where("_id").is(id).and("versions.version").is(version));
return Optional.ofNullable(mongoTemplate.findOne(query,Version.class,"letters"));
}
}
EDIT: This is a working Aggregation, I'm sure it isn't a pretty solution but it works
#Override
public Optional<Version> findByIdAndVersion(ObjectId id, Long version) {
MatchOperation match = new MatchOperation(Criteria.where("_id").is(id).and("versions.version").is(version));
Aggregation aggregate = Aggregation.newAggregation(
match,
Aggregation.unwind("versions"),
match,
Aggregation.project()
.andInclude("versions.content")
.andInclude("versions.version")
);
AggregationResults<Version> aggregateResult = mongoTemplate.aggregate(aggregate, "letters", Version.class);
Version version = aggregateResult.getUniqueMappedResult();
return Optional.ofNullable(mongoRawPage);
}

Query query = new Query(Criteria.where("_id").is(id).and("versions.version").is(version));
return Optional.ofNullable(mongoTemplate.findOne(query,Version.class,"letters"));
You are querying the Letter document but your entity class is specified as Version.class, since findOne from MongoDB doesn't return the subdocument by itself but rather the whole document, you need to have Letter.class as return type and filter (project) what fields to get back. So you can either project the single version subdocument that you want to receive, like so:
Query query = new Query()
.addCriteria(Criteria.where("_id").is(id).and("versions.version").is(version))
.fields().position("versions", 1);
Optional.ofNullable(mongoTemplate.findOne(query, Letter.class))
.map(Letter::getVersions)
.findFirst()
.orElse(null);
or use aggregation pipeline:
newAggregation(
Letter.class,
match(Criteria.where("_id").is(id)),
unwind("versions"),
replaceRoot("versions"),
match(Criteria.where("version").is(version))),
Version.class)
Note -- I typed this on a fly.

Related

InvalidPathException while sorting with org.springframework.data.domain.Pageable

I am trying to sort my table's content on the backend side, so I am sending org.springframework.data.domain.Pageable object to controller. It arrives correctly, but at the repository I am getting org.hibernate.hql.internal.ast.InvalidPathException. Somehow the field name I would use for sorting gets an org. package name infront of the filed name.
The Pageable object logged in the controller:
Page request [number: 0, size 10, sort: referenzNumber: DESC]
Exception in repository:
Invalid path: 'org.referenzNumber'","logger_name":"org.hibernate.hql.internal.ast.ErrorTracker","thread_name":"http-nio-8080-exec-2","level":"ERROR","level_value":40000,"stack_trace":"org.hibernate.hql.internal.ast.InvalidPathException: Invalid path: 'org.referenzNumber'\n\tat org.hibernate.hql.internal.ast.util.LiteralProcessor.lookupConstant(LiteralProcessor.java:111)
My controller endpoint:
#GetMapping(value = "/get-orders", params = { "page", "size" }, produces = { MediaType.APPLICATION_JSON_VALUE })
public ResponseEntity<PagedModel<KryptoOrder>> getOrders(
#ApiParam(name = "searchrequest", required = true) #Validated final OrderSearchRequest orderSearchRequest,
#PageableDefault(size = 500) final Pageable pageable, final BindingResult bindingResult,
final PagedResourcesAssembler<OrderVo> pagedResourcesAssembler) {
if (bindingResult.hasErrors()) {
return ResponseEntity.badRequest().build();
}
PagedModel<Order> orderPage = PagedModel.empty();
try {
var orderVoPage = orderPort.processOrderSearch(resourceMapper.toOrderSearchRequestVo(orderSearchRequest), pageable);
orderPage = pagedResourcesAssembler.toModel(orderVoPage, orderAssembler);
} catch (MissingRequiredField m) {
log.warn(RESPONSE_MISSING_REQUIRED_FIELD, m);
return ResponseEntity.badRequest().build();
}
return ResponseEntity.ok(orderPage);
}
the repository:
#Repository
public interface OrderRepository extends JpaRepository<Order, UUID> {
static final String SEARCH_ORDER = "SELECT o" //
+ " FROM Order o " //
+ " WHERE (cast(:partnerernumber as org.hibernate.type.IntegerType) is null or o.tradeBasis.account.retailpartner.partnerbank.partnerernumber = :partnerernumber)"
+ " and (cast(:accountnumber as org.hibernate.type.BigDecimalType) is null or o.tradeBasis.account.accountnumber = :accountnumber)"
+ " and (cast(:orderReference as org.hibernate.type.LongType) is null or o.tradeBasis.referenceNumber = :orderReference)"
+ " and (cast(:orderReferenceExtern as org.hibernate.type.StringType) is null or o.tradeBasis.kundenreferenceExternesFrontend = :orderReferenceExtern)"
+ " and (cast(:dateFrom as org.hibernate.type.DateType) is null or o.tradeBasis.timestamp > :dateFrom) "
+ " and (cast(:dateTo as org.hibernate.type.DateType) is null or o.tradeBasis.timestamp < :dateTo) ";
#Query(SEARCH_ORDER)
Page<Order> searchOrder(#Param("partnerernumber") Integer partnerernumber,
#Param("accountnumber") BigDecimal accountnumber, #Param("orderReference") Long orderReference,
#Param("orderReferenceExtern") String orderReferenceExtern, #Param("dateFrom") LocalDateTime dateFrom,
#Param("dateTo") LocalDateTime dateTo, Pageable pageable);
}
Update:
I removed the parameters from the sql query, and put them back one by one to see where it goes sideways. It seems as soon as the dates are involved the wierd "org." appears too.
Update 2:
If I change cast(:dateTo as org.hibernate.type.DateType) to cast(:dateFrom as date) then it appends the filed name with date. instead of org..
Thanks in advance for the help
My guess is, Spring Data is confused by the query you are using and can't properly append the order by clause to it. I would recommend you to use a Specification instead for your various filters. That will not only improve the performance of your queries because the database can better optimize queries, but will also make use of the JPA Criteria API behind the scenes, which requires no work from Spring Data to apply an order by specification.
Since your entity Order is named as the order by clause of HQL/SQL, my guess is that Spring Data tries to do something stupid with the string to determine the alias of the root entity.

MongoDb Upsert in Spring removes properties set to null

Consider the following Java Class.
class Transaction {
#Id
public String id;
public String firstProperty;
public String secondProperty;
}
In Java following code is executed :
(mongoTemplate is of type Mongoperations)
Transaction transaction = new Transaction("T1");
transaction.setFirstProperty("first");
Query query = Query.query(Criteria.where("_id").is("T1"));
DBObject dbObject = new BasicDBObject();
mongoTemplate.getConverter().write(transactionInfo, dbObject);
Update update = Update.fromDBObject(dbObject);
mongoTemplate.upsert(query , update , TransactionInfo.class);
Following document is created.
{
_id:123,
firstProperty: "first"
}
If this piece of code is executed later :
Transaction transaction = new Transaction("T1");
transaction.setSecondProperty("second");
Query query = Query.query(Criteria.where("_id").is("T1"));
mongoTemplate.upsert(query , update , TransactionInfo.class);
Expected Document :
{
_id:123,
firstProperty: "first",
secondProperty: "second"
}
Actual Document:
{
_id:123,
secondProperty: "second"
}
From what I read in MongoDB docs I expect the document to be updated with "secondProperty" but it results in the removal of "firstProperty" . I think the document is getting created again, instead of getting updated. Please let me know if I am missing something.

Spring Data - mongodb unwind function does not give me correct result

I have 2 collections (articles & tags). Articles has 'tags' which is array of objectIds. I am trying to join articles and tags so as to get tagName from tags. Below is my mongodb query:
db.articles.aggregate([
{"$unwind": "$tags"},
{"$lookup": {
"localField": "tags",
"from": "tags",
"foreignField": "_id",
"as": "materialTags"
}
}
])
I converted this into spring Data as below
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
Aggregation aggregation = Aggregation.newAggregation(unwindOperation, lookupOperation1 );
}
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
My entity Article.class
....
private List<ObjectId> tags = new ArrayList<>(); //array of tag ids
#Transient
private List<Tag> materialTags = new ArrayList<>(); // array of all tag details
//getters and setters
...
Tag.class
#Id private ObjectId id;
private String tagName;
...
If I use mongodb query, I am getting 'materialTags' array list with tag object filled in. Through Spring Data, I do get the exact same result. After unwind I get multiple rows (1 row per tag array entry) which is correct. Through debug point I can see resultList -> rawResult contain 'materialTags'. But resultList -> mappedResults -> 'materialTags' is empty.
Why Spring Data does not give me proper result inside Mapped Result? What am I doing wrong ? Please help.
Thanks in advance.

Spring Mongodb findandModify fails to update entire document

I am new to mongodb and struggling to understand how document update works.
I have a document called 'menu':
{
"someId":"id123",
"someProperty":"property123",
"list" : [{
"innerProperty":"property423"
}]
}
which maps to my entity:
#Document(collection = "menu")
public class Menu {
#Id
private String id;
private String someid;
private String someProperty;
private List<SomeClass> list;
// accessors
}
when I try to find and update this document like this it does not update the document. It sure does find the menu as as it returns the original entity with Id:
#Override
public Menu update(Menu menu) {
Query query = new Query(
Criteria.where("someId").is(menu.getSomeId()));
Update update = Update.update("menu", menu);
return mongoOperations.findAndModify(query, update,
FindAndModifyOptions.options().returnNew(true), Menu.class);
}
But if I change it to this, it works:
#Override
public Menu update(Menu menu) {
Query query = new Query(
Criteria.where("someId").is(menu.getSomeId()));
Update update = new Update().set("someProperty", menu.getSomeProperty())
.set("list", menu.getList());
return mongoOperations.findAndModify(query, update,
FindAndModifyOptions.options().returnNew(true), Menu.class);
}
I don't really like this second method where each element of the document is individually set, as you might imagine I have a rather large document and is prone to errors.
Why does the first method not work? And what could be a better approach to update the document?
Check out the docs for findAndModify - it returns the version of the document before the fields were modified. If you do a new find() straight after, you will see that your changes were actually saved to MongoDB.

In spring data mongodb how to achieve pagination for aggregation

In spring data mongodb using mongotemplate or mongorepository, how to achieve pagination for aggregateion
This is an answer to an old post, but I'll provide an answer in case anyone else comes along while searching for something like this.
Building on the previous solution by Fırat KÜÇÜK, giving the results.size() as the value for the "total" field in the PageImpl constructor will not making paging work the way, well, you expect paging to work. It sets the total size to the page size every time, so instead, you need to find out the actual total number of results that your query would return:
public Page<UserListItemView> list(final Pageable pageable) {
long total = getCount(<your property name>, <your property value>);
final Aggregation agg = newAggregation(
skip(pageable.getPageNumber() * pageable.getPageSize()),
limit(pageable.getPageSize())
);
final List<UserListItemView> results = mongoTemplate
.aggregate(agg, User.class, UserListItemView.class)
.getMappedResults();
return new PageImpl<>(results, pageable, total);
}
Now, then, the best way to get the total number of results is another question, and it is one that I am currently trying to figure out. The method that I tried (and it worked) was to almost run the same aggregation twice, (once to get the total count, and again to get the actual results for paging) but using only the MatchOperation followed by a GroupOperation to get the count:
private long getCount(String propertyName, String propertyValue) {
MatchOperation matchOperation = match(Criteria.where(propertyName).is(propertyValue));
GroupOperation groupOperation = group(propertyName).count().as("count");
Aggregation aggregation = newAggregation(matchOperation, groupOperation);
return mongoTemplate.aggregate(aggregation, Foo.class, NumberOfResults.class).getMappedResults().get(0).getCount();
}
private class NumberOfResults {
private int count;
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
}
It seems kind of inefficient to run nearly the same query twice, but if you are going to page results, the pageable object must know the total number of results if you really want it to behave like paging. If anyone can improve on my method to get the total count of results, that would be awesome!
Edit: This will also provide the count, and it is simpler because you do not need a wrapper object to hold the result, so you can replace the entire previous code block with this one:
private long getCount(String propertyName, String propertyValue) {
Query countQuery = new Query(Criteria.where(propertyName).is(propertyValue));
return mongoTemplate.count(countQuery, Foo.class);
}
In addition to ssouris solution you can use Pageable classes for the results.
public Page<UserListItemView> list(final Pageable pageable) {
final Aggregation agg = newAggregation(
skip(pageable.getPageNumber() * pageable.getPageSize()),
limit(pageable.getPageSize())
);
final List<UserListItemView> results = mongoTemplate
.aggregate(agg, User.class, UserListItemView.class)
.getMappedResults();
return new PageImpl<>(results, pageable, results.size())
}
You can use MongoTemplate
org.spring.framework.data.mongodb.core.aggregation.Aggregation#skip
and
org.springframework.data.mongodb.core.aggregation.Aggregation#limit
Aggregation agg = newAggregation(
project("tags"),
skip(10),
limit(10)
);
AggregationResults<TagCount> results = mongoTemplate.aggregate(agg, "tags", TagCount.class);
List<TagCount> tagCount = results.getMappedResults();
As per the answer https://stackoverflow.com/a/39784851/4546949 I wrote code for Java.
Use aggregation group to get count and array of data with other paging information.
AggregationOperation group = Aggregation.group().count().as("total")
.addToSet(pageable.getPageNumber()).as("pageNumber")
.addToSet(pageable.getPageSize()).as("pageSize")
.addToSet(pageable.getOffset()).as("offset")
.push("$$ROOT").as("data");
Use Aggregation project to slice as per the paging information.
AggregationOperation project = Aggregation.project()
.andInclude("pageSize", "pageNumber", "total", "offset")
.and(ArrayOperators.Slice.sliceArrayOf("data").offset((int) pageable.getOffset()).itemCount(pageable.getPageSize()))
.as("data");
Use mongo template to aggregate.
Aggregation aggr = newAggregation(group, project);
CustomPage page = mongoTemplate.aggregate(aggregation, Foo.class, CustomPage.class).getUniqueMappedResult();
Create a CustomPage.
public class CustomPage {
private long pageSize;
private long pageNumber;
private long offset;
private long total;
private List<Foo> data;
}
Here is my generic solution:
public Page<ResultObject> list(Pageable pageable) {
// build your main stages
List<AggregationOperation> mainStages = Arrays.asList(match(....), group(....));
return pageAggregation(pageable, mainStages, "target-collection", ResultObject.class);
}
public <T> Page<T> pageAggregation(
final Pageable pageable,
final List<AggregationOperation> mainStages,
final String collection,
final Class<T> clazz) {
final List<AggregationOperation> stagesWithCount = new ArrayList<>(mainStages);
stagesWithCount.add(count().as("count"));
final Aggregation countAgg = newAggregation(stagesWithCount);
final Long count = Optional
.ofNullable(mongoTemplate.aggregate(countAgg, collection, Document.class).getUniqueMappedResult())
.map(doc -> ((Integer) doc.get("count")).longValue())
.orElse(0L);
final List<AggregationOperation> stagesWithPaging = new ArrayList<>(mainStages);
stagesWithPaging.add(sort(pageable.getSort()));
stagesWithPaging.add(skip(pageable.getOffset()));
stagesWithPaging.add(limit(pageable.getPageSize()));
final Aggregation resultAgg = newAggregation(stagesWithPaging);
final List<T> result = mongoTemplate.aggregate(resultAgg, collection, clazz).getMappedResults();
return new PageImpl<>(result, pageable, count);
}
To return a Paged Object with correct value of pageable object , I find this is the best and simple way.
Aggregation aggregation = Aggregation.newAggregation(Aggregation.match(Criteria.where("type").is("project")),
Aggregation.group("id").last("id").as("id"), Aggregation.project("id"),
Aggregation.skip(pageable.getPageNumber() * pageable.getPageSize()),
Aggregation.limit(pageable.getPageSize()));
PageableExecutionUtils.getPage(mongoTemplate.aggregate(aggregation, Draft.class, Draft.class).getMappedResults(), pageable,() -> mongoTemplate.count(Query.of(query).limit(-1).skip(-1), Draft.class));
Another approach would be to extend the PagingAndSortingRepository<T, ID> interface. Then, you can create an #Aggregation query method like this:
#Aggregation(pipeline = {
"{ $match: { someField: ?0 } }",
"{ $project: { _id: 0, someField: 1} }"
})
List<StuffAggregateModel> aggregateStuff(final String somePropertyName, final Pageable pageable);
Just call this from your business logic service class and construct the Pageable (which also contains sort options, if desired) and call the repo method. I like this approach because of the simplicity and the sheer minimization of the amount of code that you have to write. If your query (aggregation pipeline) is simple enough, this is probably the best solution. Maintenance coding for this approach is nearly effortless.
My answer with MongoDB $facet
// User(_id, first name, etc), Car (user_id, brand, etc..)
LookupOperation lookupStageCar = Aggregation.lookup(‘cars ’, ‘user_id’, ‘_id’, ‘car’);
MatchOperation matchStage = Aggregation.match(Criteria.where(‘car.user_id ‘).exists(true));
CountOperation countOperation = Aggregation.count().as("total");
AddFieldsOperation addFieldsOperation = Aggregation.addFields().addFieldWithValue("page", pageable.getPageNumber()).build();
SkipOperation skipOperation = Aggregation.skip(Long.valueOf(pageable.getPageNumber() * pageable.getPageSize()));
LimitOperation limitOperation = Aggregation.limit(pageable.getPageSize());
// here the magic
FacetOperation facetOperation = Aggregation.facet( countOperation, addFieldsOperation).as("metadata")
.and(skipOperation, limitOperation).as("data");
// users with car
List<AggrigationResults> map = mongoTemplate.aggregate(Aggregation.newAggregation( lookupStageCar, matchStage, facetOperation), "User", AggrigationResults.class).getMappedResults();
———————————————————————————
public class AggrigationResults {
private List<Metadata> metadata;
private List<User> data;
}
public class Metadata {
private long total;
private long page;
}
———————————————————————————
output:
{
"metadata" : [
{
"total" : 300,
"page" : 3
}
],
"data" : [
{
... original document ...
},
{
... another document ...
},
{
... etc up to 10 docs ...
}
]
}
see : How to use MongoDB aggregation for pagination?

Resources