I have a Feed collection that has two fields: username and sentiment. I have to write a aggregation in spring to group by username and sentiment. I also have to display total number of times username comes in collection.
How to do that?
My code is:
db.feed.aggregate([
{$group: {_id : {username : '$username',sentiment:'$sentiment'}, total:{$sum :1}}},
{$project : {username : '$_id.username', sentiment : '$_id.sentiment', total : '$total', _id : 0}}
])
The first thing that comes to mind:
http://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo.aggregation
So in your case it would look something like:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
class Stat {
String username;
String sentiment;
int total;
}
Aggregation agg = newAggregation(
group("username", "sentiment").count().as("total"),
project().
and("_id.username").as("username").
and("_id.sentiment").as("sentiment").
and("total").as("total")
);
AggregationResults<Stat> results = mongoTemplate.aggregate(agg, "feed", Stat.class);
List<Stat> mappedResult = results.getMappedResults();
Related
A brief overview of the document I am working with:
#NoArgsConstructor
#AllArgsConstructor
#EqualsAndHashCode
#Data
#SuperBuilder(toBuilder = true)
public class BreachBrand {
#Id
private String id;
#CreatedDate
#Field("created_date")
#DiffIgnore
private Instant createdDate;
#LastModifiedDate
#Field("last_modified_date")
#DiffIgnore
private Instant lastModifiedDate;
}
What I am trying to do is compare the lastModifiedDate to the createdDate. So I created a criteria object like so:
criteria.add(Criteria.where("last_modified_date").gt("ISODate('created_date')"));
I've also tried:
criteria.add(Criteria.where("last_modified_date").gt("created_date"));
which is then used in the match operation of an Aggregation object. Using the first criteria code snippet, the aggregation looks like this:
{ "aggregate" : "__collection__", "pipeline" : [{ "$lookup" : { "from" : "brands", "localField" : "brand_dfp_id", "foreignField" : "dfp_id", "as" : "brand"}}, { "$match" : { "$and" : [{ "last_modified_date" : { "$gt" : "ISODate('created_date')"}}]}}, { "$sort" : { "date" : -1}}, { "$skip" : 0}, { "$limit" : 25}], "allowDiskUse" : true, "collation" : { "locale" : "en", "strength" : 1}}
The mongoTemplate object executes the aggregate method w/o error but no records are returned.
I'm suspecting that the gt(Object o) method is expecting an object that is an actual value to use to compare against. All is good when I use an actual date:
criteria.add(Criteria.where("last_modified_date").gt(Instant.parse("2019-05-18T17:07:25.333+00:00")));
As an interesting aside the following works in mongoshell:
db.breaches.find({$where: "this.last_modified_date>this.created_date"}).pretty();
And the following works in Compass (but the export to language button will not display the output):
/**
* $match operation
*/
{
last_modified_date: {$gt: ISODate('created_date')}
}
EDIT:
It appears I need to use a projection to determine if last_modified_date is greater than created date. I got this to work in compass:
[{
$project: {
greater: {
$gt: [
'$last_modified_date',
'$created_date'
]
},
doc: '$$ROOT'
}
}, {
$match: {
greater: true
}
}]
I'm having issues moving that into a projection though:
ProjectionOperation projectionOperation = project("last_modified_date", "created_date").andExpression("$gt", "$last_modified_date", "$created_date").as("greater");
I've also tried this:
ProjectionOperation projectionOperation = project("last_modified_date", "created_date").andExpression("$gt", Arrays.asList("$last_modified_date", "$created_date")).as("greater");
Results in an exception when creating the aggregation:
Aggregation aggregation = newAggregation(
lookup("brands", "brand_dfp_id", "dfp_id", "brand"),
projectionOperation,
matchOperation, //Criteria.where("greater").is(true)
sortOperation,
skipOperation,
limitOperation
)
.withOptions(AggregationOptions.builder()
.allowDiskUse(true)
.collation(Collation.of("en").strength(Collation.ComparisonLevel.primary())).build());
exception:
java.lang.IllegalArgumentException: Invalid reference 'date'!
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:114)
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:86)
at org.springframework.data.mongodb.core.aggregation.SortOperation.toDocument(SortOperation.java:74)
at org.springframework.data.mongodb.core.aggregation.AggregationOperation.toPipelineStages(AggregationOperation.java:55)
at org.springframework.data.mongodb.core.aggregation.AggregationOperationRenderer.toDocument(AggregationOperationRenderer.java:56)
at org.springframework.data.mongodb.core.aggregation.AggregationPipeline.toDocuments(AggregationPipeline.java:77)
at org.springframework.data.mongodb.core.aggregation.Aggregation.toPipeline(Aggregation.java:705)
at org.springframework.data.mongodb.core.AggregationUtil.createPipeline(AggregationUtil.java:95)
at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2118)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2093)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1992)
I'm trying to use Mongo aggregation but I receive an error that I don't understand.
This is my domain:
#Document(collection = "tapes")
public class Tape {
#Id
private String id;
private String area;
private Integer tape;
private String tapeModel;
// follow getters e setters
The mongo shell command and the output is the following:
> db.tapes.aggregate([{ $group: { _id: { "area":"$area"}, tapes: {$push: {tape: "$tape"}}}} ])
{ "_id" : { "area" : "free" }, "tapes" : [ { "tape" : 1 }, { "tape" : 2 } ] }
{ "_id" : { "area" : "Qnap" }, "tapes" : [ { "tape" : 3 } ] }
The following is an attempt to re-create the aggregation in Spring:
AggregationOperation group = Aggregation.group("area").push("tape").as("tape");
Aggregation aggregation = Aggregation.newAggregation(group);
AggregationResults<Tape> results = mongoTemplate.aggregate(aggregation, "tapes", Tape.class);
//List<Tape> tapes = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Tape.class), Tape.class).getMappedResults();
List<Tape> tapes = results.getMappedResults();
System.out.println(tapes);
But I obtain the following error:
Cannot convert [3] of type class java.util.ArrayList into an instance of class java.lang.Integer! Implement a custom Converter<class java.util.ArrayList, class java.lang.Integer> and register it with the CustomConversions. Parent object was: it.unifi.cerm.cermadminspring.domain.Tape#2b84da07 -> null
org.springframework.data.mapping.MappingException: Cannot convert [3] of type class java.util.ArrayList into an instance of class java.lang.Integer! Implement a custom Converter<class java.util.ArrayList, class java.lang.Integer> and register it with the CustomConversions. Parent object was: it.unifi.cerm.cermadminspring.domain.Tape#2b84da07 -> null
I don't understand why, I searched for aggregation examples and all are more or less similar to mine.
Someone can help me?
First, for making life easier, a more simplified aggregation can be used:
> db.tapes.aggregate([ {$group: {_id: "$area", tapes: {$push: "$tape"}}} ])
Which should yield:
{ "_id" : "free", "tapes" : [ 1 , 2 ] }
{ "_id" : "Qnap", "tapes" : [ 3 ] }
Which should be matched by a change to the Java group aggregation operation:
AggregationOperation group = Aggregation.group("area").push("tape").as("tapes");
Note that I've changed to plural: as("tapes")
Then, notice that you are actually returning a document which doesn't have the same structure as the one you've mapped in the Tape class. That document contains two fields, the String id and a List<Integer> tapes fields.
This is the reason for the shorthand group aggregation I've suggested above, for making mapping easier:
public class TapesForArea {
private String id; // which is the area
private List<Integer> tapes;
// getters, setters ...
}
You don't need to map this class using spring-data-mongodb annotations.
Finally, have the aggregation results return the right type:
AggregationResults<TapesForArea> results =
mongoTemplate.aggregate(aggregation, "tapes", TapesForArea.class);
List<TapesForArea> tapes = results.getMappedResults();
BTW, the error comes from the fact that you try to map the single item tape array [ 3 ] into private Integer tape; property of Tape class.
Hi a am try to do rest api in spring boot with mongodb to find group by count the input data look like. please share any logic, code, example link.
guys i am expecting spring boot logic. how mongodb aggregation framework integrating.
{
"_id" : "PRODUCT_01",
"productname" : "product1",
"value" : "codesoft"
},
{
"_id" : "PRODUCT_01",
"productname" : "product2",
"value" : "codesoft"
},
{
"_id" : "PRODUCT_01",
"productname" : "product1",
"value" : "codesoft"
}
expected output
{
product1 : 2,
product2 : 1
}
Any help is appreciated.
try this
db.testColln.aggregate(
{
$group : {_id : "$productname", total : { $sum : 1 }}
}
);
for Spring Boot
Aggregation agg = newAggregation(
group("productname").count().as("total")
project("productname").and("total"),
);
AggregationResults<Product> groupResults
= mongoTemplate.aggregate(agg, Product.class,Result.class);
List<Result> result = groupResults.getMappedResults();
public class Result {
private String productname;
private long total;
}
#GetMapping("/group")
public List<ProductCount> groupByName() {
// grouping by prductName
GroupOperation groupOperation =
Aggregation.group("productName").count().as("count");
// projection operation
ProjectionOperation projectionOperation =
Aggregation.project("count").and("productName").previousOperation();
// sorting in ascending
SortOperation sortOperation =
Aggregation.sort(Sort.by(Sort.Direction.ASC, "count"));
// aggregating all 3 operations using newAggregation() function
Aggregation aggregation =
Aggregation.newAggregation(groupOperation,projectionOperation
,sortOperation);
// putting in a list
// "products" is collection name
AggregationResults<ProductCount> result =
mongotemplate.aggregate(aggregation, "products",
ProductCount.class);
return result.getMappedResults();
}
$ make ProductCount class in model package
public class ProductCount {
private String productName;
private int count;
#getters
#setters
I am trying to implement a MongoDB query in java using Spring mongo.
This is the native MongoDB query:
db.ShieldReport.aggregate([
{$match:{"sellerCode":"e1aaf3"}},
{$project:{bucketName:"$bucketName", brandName: "$brandName", createdTime : "$createdTime", sellerCode : "$sellerCode"}},
{$sort:{"createdTime":-1}},
{$group:
{
_id: { sellerCode: "$sellerCode", bucketName: "$bucketName", brandName: "$brandName"},
itemsSold: { $first: { bucketName: "$bucketName", brandName: "$brandName", createdTime : "$createdTime"} }
}
},
{$sort:{"itemsSold.createdTime":-1}},
{$project : { _id : "$_id.sellerCode", bucketName :"$itemsSold.bucketName", brandName : "$itemsSold.brandName"}}
])
In my Spring java version, I have got it this far:
Aggregation agg = newAggregation(
match(Criteria.where("sellerCode").is(filterRequest.getSellerCode())),
Aggregation.project("bucketName")
.andInclude("brandName")
.andInclude("createdTime")
.andInclude("sellerCode"),
sort(Sort.Direction.DESC, "createdTime"),
group("sellerCode", "brandName", "bucketName")
);
But the problem is I am not able to create the itemSold field. How do I create that?
You could restructure your pipeline in Spring as follows:
Aggregation agg = Aggregation.newAggregation(
Aggregation.match(Criteria.where("sellerCode").is(filterRequest.getSellerCode())),
Aggregation.sort(Sort.Direction.DESC, "createdTime"),
Aggregation.group("sellerCode", "bucketName", "brandName")
.first("bucketName").as("bucketName")
.first("brandName").as("brandName")
.first("createdTime ").as("createdTime "),
Aggregation.sort(Sort.Direction.DESC, "createdTime"),
Aggregation.project("bucketName", "brandName")
.and("sellerCode").previousOperation()
);
As you can see some of the pipeline stages have been omitted because they are really not necessary. For instance, the $project pipeline before the $group step can be taken off since the $group pipeline operator will apply the accumulators
In the group step, you group the intermediate filtered documents by the three fields and store the other field values in new fields aided by the $first operator.
Sort the intermediate result by the createdTime field reference of the previous group operation.
Finally in the fourth step, select the "bucketName" and "brandName" fields from the previous group operation. Note that "sellerCode" again implicitly references an group-id field.
In my project I'm using SpringBoot 1.3.2 and org.springframework.data.mongodb.core.query.*
I'm trying to update element in array, in my main object i have array looking like this:
"sections" : [
{
"sectionId" : "56cc3c908f5e6c56e677bd2e",
"name" : "Wellcome"
},
{
"sectionId" : "56cc3cd28f5e6c56e677bd2f",
"name" : "Hello my friends"
}
]
Using Spring I want to update name of record with sectionId 56cc3c908f5e6c56e677bd2e
I was trying to to this like that but it didn't work
Query query = Query.query(Criteria
.where("sections")
.elemMatch(
Criteria.where("sectionId").is(editedSection.getId())
)
);
Update update = new Update().set("sections", new BasicDBObject("sectionId", "56cc3c908f5e6c56e677bd2e").append("name","Hi there"));
mongoTemplate.updateMulti(query, update, Offer.class);
It create something like:
"sections" : {
"sectionId" : "56cc3c908f5e6c56e677bd2e",
"name" : "Hi there"
}
But this above is object { } I want an array [ ], and I don't want it remove other elements.
Can any body help me how to update name of record with sectionId 56cc3c908f5e6c56e677bd2e using Spring
You essentially want to replicate this mongo shell update operation:
db.collection.update(
{ "sections.sectionId": "56cc3c908f5e6c56e677bd2e" },
{
"$set": { "sections.$.name": "Hi there" }
},
{ "multi": true }
)
The equivalent Spring Data MongoDB code follows:
import static org.springframework.data.mongodb.core.query.Criteria.where;
import static org.springframework.data.mongodb.core.query.Query;
import static org.springframework.data.mongodb.core.query.Update;
...
WriteResult wr = mongoTemplate.updateMulti(
new Query(where("sections.sectionId").is("56cc3c908f5e6c56e677bd2e")),
new Update().set("sections.$.name", "Hi there"),
Collection.class
);
Can use BulkOperations approach to update list or array of document objects
BulkOperations bulkOps = mongoTemplate.bulkOps(BulkMode.UNORDERED, Person.class);
for(Person person : personList) {
Query query = new Query().addCriteria(new Criteria("id").is(person.getId()));
Update update = new Update().set("address", person.setAddress("new Address"));
bulkOps.updateOne(query, update);
}
BulkWriteResult results = bulkOps.execute();
Thats my solution for this problem:
public Mono<ProjectChild> UpdateCritTemplChild(
String id, String idch, String ownername) {
Query query = new Query();
query.addCriteria(Criteria.where("_id")
.is(id)); // find the parent
query.addCriteria(Criteria.where("tasks._id")
.is(idch)); // find the child which will be changed
Update update = new Update();
update.set("tasks.$.ownername", ownername); // change the field inside the child that must be updated
return template
// findAndModify:
// Find/modify/get the "new object" from a single operation.
.findAndModify(
query, update,
new FindAndModifyOptions().returnNew(true), ProjectChild.class
)
;
}