A brief overview of the document I am working with:
#NoArgsConstructor
#AllArgsConstructor
#EqualsAndHashCode
#Data
#SuperBuilder(toBuilder = true)
public class BreachBrand {
#Id
private String id;
#CreatedDate
#Field("created_date")
#DiffIgnore
private Instant createdDate;
#LastModifiedDate
#Field("last_modified_date")
#DiffIgnore
private Instant lastModifiedDate;
}
What I am trying to do is compare the lastModifiedDate to the createdDate. So I created a criteria object like so:
criteria.add(Criteria.where("last_modified_date").gt("ISODate('created_date')"));
I've also tried:
criteria.add(Criteria.where("last_modified_date").gt("created_date"));
which is then used in the match operation of an Aggregation object. Using the first criteria code snippet, the aggregation looks like this:
{ "aggregate" : "__collection__", "pipeline" : [{ "$lookup" : { "from" : "brands", "localField" : "brand_dfp_id", "foreignField" : "dfp_id", "as" : "brand"}}, { "$match" : { "$and" : [{ "last_modified_date" : { "$gt" : "ISODate('created_date')"}}]}}, { "$sort" : { "date" : -1}}, { "$skip" : 0}, { "$limit" : 25}], "allowDiskUse" : true, "collation" : { "locale" : "en", "strength" : 1}}
The mongoTemplate object executes the aggregate method w/o error but no records are returned.
I'm suspecting that the gt(Object o) method is expecting an object that is an actual value to use to compare against. All is good when I use an actual date:
criteria.add(Criteria.where("last_modified_date").gt(Instant.parse("2019-05-18T17:07:25.333+00:00")));
As an interesting aside the following works in mongoshell:
db.breaches.find({$where: "this.last_modified_date>this.created_date"}).pretty();
And the following works in Compass (but the export to language button will not display the output):
/**
* $match operation
*/
{
last_modified_date: {$gt: ISODate('created_date')}
}
EDIT:
It appears I need to use a projection to determine if last_modified_date is greater than created date. I got this to work in compass:
[{
$project: {
greater: {
$gt: [
'$last_modified_date',
'$created_date'
]
},
doc: '$$ROOT'
}
}, {
$match: {
greater: true
}
}]
I'm having issues moving that into a projection though:
ProjectionOperation projectionOperation = project("last_modified_date", "created_date").andExpression("$gt", "$last_modified_date", "$created_date").as("greater");
I've also tried this:
ProjectionOperation projectionOperation = project("last_modified_date", "created_date").andExpression("$gt", Arrays.asList("$last_modified_date", "$created_date")).as("greater");
Results in an exception when creating the aggregation:
Aggregation aggregation = newAggregation(
lookup("brands", "brand_dfp_id", "dfp_id", "brand"),
projectionOperation,
matchOperation, //Criteria.where("greater").is(true)
sortOperation,
skipOperation,
limitOperation
)
.withOptions(AggregationOptions.builder()
.allowDiskUse(true)
.collation(Collation.of("en").strength(Collation.ComparisonLevel.primary())).build());
exception:
java.lang.IllegalArgumentException: Invalid reference 'date'!
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:114)
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:86)
at org.springframework.data.mongodb.core.aggregation.SortOperation.toDocument(SortOperation.java:74)
at org.springframework.data.mongodb.core.aggregation.AggregationOperation.toPipelineStages(AggregationOperation.java:55)
at org.springframework.data.mongodb.core.aggregation.AggregationOperationRenderer.toDocument(AggregationOperationRenderer.java:56)
at org.springframework.data.mongodb.core.aggregation.AggregationPipeline.toDocuments(AggregationPipeline.java:77)
at org.springframework.data.mongodb.core.aggregation.Aggregation.toPipeline(Aggregation.java:705)
at org.springframework.data.mongodb.core.AggregationUtil.createPipeline(AggregationUtil.java:95)
at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2118)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2093)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1992)
Related
My database contain products in a single table called Product, and each product might have certain fields with their specification, e.g.
//Bicycle product
{
"name" : "Bicycle",
"category" : "Bicycles"
"specification" : {
"color" : "red"
}
}
//Wheel product
{
"name" : "Wheel",
"category" : "Spare Parts",
"specification" : {
"diameter" : "7.5"
}
}
So i've come up with idea of making a field of type Map<String, String> (which creates a another table called specifications) in my Product entity to contain those specifications. But i don't like this approach, because all of the additional fields would be of String type, and because Spring will create a bean out of this field, I wont be able to specify the type of value as an abstract class (like this Map<String, Object>).
I thought of creating additional fields, like this:
#ElementCollection
private Map<String, String> string_features;
#ElementCollection
private Map<String, Double> double_features;
// ...
But it looks kind of ugly and I think there is a better way to do it. And also, if the specification field is of a different Entity type, I will have to create another map for that specific entity, e.g.
//Bicycle product
{
"name" : "Bicycle",
"category" : "Bicycles"
"specification" : {
"color" : "red",
"wheel" : {
"name" : "Wheel",
}
}
}
If the value can be only be numbers and strings, maybe you can save the value as strings and then use a regex to check if the string is a number before returning the value.
Otherwise, you need a way to recognize the type.
I think I would change it to this:
//Bicycle product
{
"name" : "Bicycle",
"category" : "Bicycles"
"specifications" : [
{ name: "color", value: "red", type: "string"},
{ name: "diameter", value: "7.5", type: "double"},
...
]
}
You can map it as:
#ElementCollection
private List<Specification> specifications;
...
#Embaddable
class Specification {
String name;
String value;
String type;
// ... getter/setter and everything else
#Transient
Object getConvertedValue() {
if ("double".equals(type)) {
return Double.parse(value);
}
// String is the default
return value;
}
}
The nice thing is that you can have as many types as you want.
I have a collection named results where each document has a token (unique) and a list of embedded documents (services).
Sample:
{
_id: ObjectId("61e7eed15b9df6f80f0164c0"),
token: '7683af2f-8f93-4387-9b6a-c89840d9525f',
providers: [
{
_id: 2,
companyName: 'Autopro'
}
],
services: [
{
_id: 1,
serviceTypeId: 103
},
{
_id: 5,
serviceTypeId: 103
},
{
_id: 6,
serviceTypeId: 103,
}
]
}
I want to extract using mongo template (spring boot) one service based on token and service id.
here is a code snippet:
UnwindOperation unwindServicesOp = Aggregation.unwind("services");
ProjectionOperation projectionOp = Aggregation.project(Fields.fields("services")).andExclude("_id");
HashSet<Criteria> set = new HashSet<>();
set.add(Criteria.where("token").is(token));
set.add(Criteria.where("services._id").is(serviceId));
Criteria c = new Criteria();
c.andOperator(set);
MatchOperation matchOp = Aggregation.match(c);
return mongoTemplate.aggregate(Aggregation.newAggregation(Arrays.asList(unwindServicesOp, matchOp, projectionOp)),"results", Service.class).getUniqueMappedResult();
Service class:
public class Service implements Serializable {
private static final long serialVersionUID = -7786799739639015883L;
#Id
private Integer id;
private Integer serviceTypeId;
//setter and getters omitted for simplicity
}
Above code execute this query:
[{ "$unwind" : "$services"}, { "$match" : { "$and" : [{ "token" : "7683af2f-8f93-4387-9b6a-c89840d9525d"}, { "services._id" : 1}]}}, { "$project" : { "services" : 1, "_id" : 0}}]
getUniqueMappedResult() method returns empty Service object!
Is there any changes to do in my aggregation in order to get a Service object?
There is one more stage (last one) to add in order to solve the issue:
ReplaceRootOperation replaceRootOp = Aggregation.replaceRoot("services");
Add it at last in the aggregation object.
I works.
I'm trying to use Mongo aggregation but I receive an error that I don't understand.
This is my domain:
#Document(collection = "tapes")
public class Tape {
#Id
private String id;
private String area;
private Integer tape;
private String tapeModel;
// follow getters e setters
The mongo shell command and the output is the following:
> db.tapes.aggregate([{ $group: { _id: { "area":"$area"}, tapes: {$push: {tape: "$tape"}}}} ])
{ "_id" : { "area" : "free" }, "tapes" : [ { "tape" : 1 }, { "tape" : 2 } ] }
{ "_id" : { "area" : "Qnap" }, "tapes" : [ { "tape" : 3 } ] }
The following is an attempt to re-create the aggregation in Spring:
AggregationOperation group = Aggregation.group("area").push("tape").as("tape");
Aggregation aggregation = Aggregation.newAggregation(group);
AggregationResults<Tape> results = mongoTemplate.aggregate(aggregation, "tapes", Tape.class);
//List<Tape> tapes = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Tape.class), Tape.class).getMappedResults();
List<Tape> tapes = results.getMappedResults();
System.out.println(tapes);
But I obtain the following error:
Cannot convert [3] of type class java.util.ArrayList into an instance of class java.lang.Integer! Implement a custom Converter<class java.util.ArrayList, class java.lang.Integer> and register it with the CustomConversions. Parent object was: it.unifi.cerm.cermadminspring.domain.Tape#2b84da07 -> null
org.springframework.data.mapping.MappingException: Cannot convert [3] of type class java.util.ArrayList into an instance of class java.lang.Integer! Implement a custom Converter<class java.util.ArrayList, class java.lang.Integer> and register it with the CustomConversions. Parent object was: it.unifi.cerm.cermadminspring.domain.Tape#2b84da07 -> null
I don't understand why, I searched for aggregation examples and all are more or less similar to mine.
Someone can help me?
First, for making life easier, a more simplified aggregation can be used:
> db.tapes.aggregate([ {$group: {_id: "$area", tapes: {$push: "$tape"}}} ])
Which should yield:
{ "_id" : "free", "tapes" : [ 1 , 2 ] }
{ "_id" : "Qnap", "tapes" : [ 3 ] }
Which should be matched by a change to the Java group aggregation operation:
AggregationOperation group = Aggregation.group("area").push("tape").as("tapes");
Note that I've changed to plural: as("tapes")
Then, notice that you are actually returning a document which doesn't have the same structure as the one you've mapped in the Tape class. That document contains two fields, the String id and a List<Integer> tapes fields.
This is the reason for the shorthand group aggregation I've suggested above, for making mapping easier:
public class TapesForArea {
private String id; // which is the area
private List<Integer> tapes;
// getters, setters ...
}
You don't need to map this class using spring-data-mongodb annotations.
Finally, have the aggregation results return the right type:
AggregationResults<TapesForArea> results =
mongoTemplate.aggregate(aggregation, "tapes", TapesForArea.class);
List<TapesForArea> tapes = results.getMappedResults();
BTW, the error comes from the fact that you try to map the single item tape array [ 3 ] into private Integer tape; property of Tape class.
I have an Array field replies in my document, which may or may not exist. If it exists, I want to return its size otherwise return 0. Below is my projection code. For documents that contain replies field with elements in it, it is returning 0.
ProjectionOperation project = Aggregation.project("title", "datePosted", "likes").
and(ConditionalOperators.ifNull(ArrayOperators.arrayOf("replies").length()).then(0)).as("repliesCount");
Reading the MongoDB reference doc, it is pretty straight forward but somehow it is not working.
Any suggestions are welcome.
Sample Data
{
"_id" : ObjectId("5e9e3873d022d154c54c2969"),
"title" : "Is this a nice place to meet some interesting people?",
"datePosted" : ISODate("2020-04-21T00:04:03.731Z"),
"views" : 0,
"likes" : 0,
"active" : true
},
{
"_id" : ObjectId("5e9e2f37d022d154c54c2961"),
"title" : "I am posting a quick discussion",
"datePosted" : ISODate("2020-04-20T23:24:39.768Z"),
"views" : 120,
"likes" : 4,
"active" : true,
"replies" : [
{
"_id" : ObjectId("5e9e2f69d022d154c54c2963"),
"reply" : "This is the first reply",
"datePosted" : ISODate("2020-04-20T23:25:29.608Z"),
"likes" : 0
},
{
"_id" : ObjectId("5e9e2f69d022ad4c54c2964"),
"reply" : "This is another reply",
"datePosted" : ISODate("2020-04-20T23:25:29.608Z"),
"likes" : 0
}
]
}
Root Cause
The solution provided by #valijon is correct. The problem lies in the POJO DiscussionDoc to which the result is mapped.
#Document(collection = "discussions")
public class DiscussionDoc {
#Id
private ObjectId id;
private String title;
private LocalDateTime datePosted;
private int views;
private int likes;
#Transient
private int repliesCount;
private boolean active;
#Field(value = "replies")
private List<ReplyDoc> replyList;
}
Removing #Transient it works. I am using #Transient on repliesCount because I do not want to persist this field when persisting a DiscussionDoc. There is no usefulness of persisting this field. But it will be used to hold total reply count when fetching a DiscussionDoc. But #Transient is somehow not letting repliesCount be set when a DiscussionDoc is fetched. I have removed #Transient to fix the problem. But what to do about repliesCount field now being persisted in DB?
You were almost close:
ProjectionOperation project = Aggregation.project("title", "datePosted", "likes").
and(ArrayOperators.arrayOf(ConditionalOperators.ifNull("replies").then(Collections.emptyList())).length()).as("repliesCount");
Equivalent to:
{
"$project": {
"title": 1,
"datePosted": 1,
"likes": 1,
"repliesCount": {
"$size": {
"$ifNull": [
"$replies",
[]
]
}
}
}
}
Workaround: You may use Gson serializer to get this calculated value:
private static final Gson g = new GsonBuilder().create();
...
List<Document> result = mongoTemplate.aggregate(aggregation,
mongoTemplate.getCollectionName(DiscussionDoc.class), Document.class)
.getMappedResults();
return result.stream()
.map(doc -> g.fromJson(doc.toJson(), DiscussionDoc.class))
.collect(Collectors.toList());
Hi a am try to do rest api in spring boot with mongodb to find group by count the input data look like. please share any logic, code, example link.
guys i am expecting spring boot logic. how mongodb aggregation framework integrating.
{
"_id" : "PRODUCT_01",
"productname" : "product1",
"value" : "codesoft"
},
{
"_id" : "PRODUCT_01",
"productname" : "product2",
"value" : "codesoft"
},
{
"_id" : "PRODUCT_01",
"productname" : "product1",
"value" : "codesoft"
}
expected output
{
product1 : 2,
product2 : 1
}
Any help is appreciated.
try this
db.testColln.aggregate(
{
$group : {_id : "$productname", total : { $sum : 1 }}
}
);
for Spring Boot
Aggregation agg = newAggregation(
group("productname").count().as("total")
project("productname").and("total"),
);
AggregationResults<Product> groupResults
= mongoTemplate.aggregate(agg, Product.class,Result.class);
List<Result> result = groupResults.getMappedResults();
public class Result {
private String productname;
private long total;
}
#GetMapping("/group")
public List<ProductCount> groupByName() {
// grouping by prductName
GroupOperation groupOperation =
Aggregation.group("productName").count().as("count");
// projection operation
ProjectionOperation projectionOperation =
Aggregation.project("count").and("productName").previousOperation();
// sorting in ascending
SortOperation sortOperation =
Aggregation.sort(Sort.by(Sort.Direction.ASC, "count"));
// aggregating all 3 operations using newAggregation() function
Aggregation aggregation =
Aggregation.newAggregation(groupOperation,projectionOperation
,sortOperation);
// putting in a list
// "products" is collection name
AggregationResults<ProductCount> result =
mongotemplate.aggregate(aggregation, "products",
ProductCount.class);
return result.getMappedResults();
}
$ make ProductCount class in model package
public class ProductCount {
private String productName;
private int count;
#getters
#setters