I have the following aggregation pipeline to calculate the top rated brands from a collection of phones with their reviews embedded.
public Document findTopRatedBrands(int minReviews, int results) {
UnwindOperation unwindOperation = unwind("reviews");
GroupOperation groupOperation = group("$brand").avg("$reviews.rating")
.as("avgRating").count().as("numReviews");
MatchOperation matchOperation = match(new Criteria("numReviews").gte(minReviews));
SortOperation sortOperation = sort(Sort.by(Sort.Direction.DESC, "avgRating",
"numReviews"));
LimitOperation limitOperation = limit(results);
ProjectionOperation projectionOperation = project().andExpression("_id").as
("brand").andExpression("avgRating").as("rating").andExclude("_id")
.andExpression("numReviews").as("reviews");
Aggregation aggregation = newAggregation(unwindOperation, groupOperation, matchOperation,
sortOperation, limitOperation, projectionOperation);
AggregationResults<Phone> result = mongoOperations
.aggregate(aggregation, "phones", Phone.class);
return result.getRawResults();
}
An example of document in the phones collection is this:
{
"_id": {
"$oid": "61e1cc8f452d0aef89d9125f"
},
"brand": "Samsung",
"name": "Samsung Galaxy S7",
"releaseYear": 2016,
"reviews": [{
"_id": {
"$oid": "61d4403b86913bee0245c171"
},
"rating": 2,
"dateOfReview": {
"$date": "2019-12-24T00:00:00.000Z"
},
"title": "Won't do that again.",
"body": "I could not use with my carrier. Sent it back.",
"username": "bigrabbit324"
}]
}
I would like to sort by avgRating (rounded to the first decimal place), and secondly by the number of reviews. Now the average rating it's not rounded so it gives always different values, so I can't sort by number of reviews also. I have seen the ArithmeticOperators.Round class but I don't understand how to include it here if possible.
An example of result is the following:
[Document{{brand=Nokia, rating=3.25, reviews=4}}]
I would like to have 3.2 as rating.
This works in Mongo Compass:
$project: {
_id: 0,
brand: '$_id',
rating: { $round: ['$avgRating', 1] }
}
try
ProjectionOperation roundAverageRating = Aggregation.project("avgRating", "numReviews")
.and(ArithmeticOperators.Round.roundValueOf("avgRating").place(1))
.as("avgRatingRounded");
Related
I am trying to get Aggregation results from ElasticSearch index
For exmaple , values in my index
"_source": {
"ctry": "abc",
"totalentry": 1,
"entrydate": "2022-01-06"
},
"_source": {
"ctry": "abc",
"totalentry": 3,
"entrydate": "2022-01-07"
},
"_source": {
"ctry": "xyz",
"totalentry": 1,
"entrydate": "2022-01-08"
}
expected Results should be get totalentry based on country
ctry : abc
totalentry : 4
ctry : xyz
totalentry : 1
My Aggreagtion query
QueryBuilder querybuilder = QueryBuilders.boolQuery().must(QueryBuilders.rangeQuery("entrydate")
.gte("2022-01-01").lte ("2022-01-31"));
TermsAggregationBuilder groupBy = AggregationBuilders.terms("ctry").field("ctry");
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(querybuilder).addAggregation(groupBy)
.build();
List<Sample> records = elasticsearchRestTemplate.queryForList(searchQuery, Sample.class);
Above aggregation query returning 3 records instead of 2 aggregated results.
My index properties
"ctry": {
"type": "keyword"
How to change it to below , so that i hope i will get correct aggregation results
ctry": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
My java code
#Document(indexName="sample", createIndex=true, shards = 4)
public class Sample {
#Field(type = FieldType.Keyword)
private String ctry;
You are using an outdated version of Spring Data Elasticsearch. The queryForList variants were deprecated in 4.0 and have been removed in 4.2.
You need to use one of the search...() methods that return a SearchHits<Sample>> object. That will contain the documents for your query and the aggregations.
I have the following entity (getters, setters and constructor omitted)
public class Event {
#Id
private String id;
private String internalUuid;
private EventType eventType;
}
EventType is an enum containing arbitrary event types:
public enum EventType {
ACCEPTED,
PROCESSED,
DELIVERED;
}
My problem is that I have a table with a lot of events, some having the same internalUuid but different statuses. I need to get a list of Events with each Event representing the 'newest' status (ordering by EventType would suffice). Currently, I'm just fetching everything, grouping to separates lists in code, sorting the lists by EventType and then just creating a new list with the first element of each list.
Example would be as follows.
Data in table:
{ "id": "1", "internalUuid": "1", "eventType": "ACCEPTED" },
{ "id": "2", "internalUuid": "1", "eventType": "PROCESSED" },
{ "id": "3", "internalUuid": "1", "eventType": "DELIVERED" },
{ "id": "4", "internalUuid": "2", "eventType": "ACCEPTED" },
{ "id": "5", "internalUuid": "2", "eventType": "PROCESSED" },
{ "id": "6", "internalUuid": "3", "eventType": "ACCEPTED" }
Output of the query (any order would be ok):
[
{ "id": "3", "internalUuid": "1", "eventType": "DELIVERED" },
{ "id": "5", "internalUuid": "2", "eventType": "PROCESSED" },
{ "id": "6", "internalUuid": "3", "eventType": "ACCEPTED" }
]
It is not guaranteed that a "higher" status also has a "higher" ID.
How do I do that without doing the whole process by hand? I literally have no idea how to start as I'm very new to MongoDB but haven't found anything that helped me on Google. I'm using Spring Boot and Spring Data.
Thanks!
Okay I think I have figured it out (thanks to Joe's comment). I'm not a 100% sure that the code is correct but it seems to do what I want. I'm open to improvements.
(I had to add a priority field to Event and EventType because sorting by eventType obviously does String-based (alphabetic) sorting on the enum's name):
private List<Event> findCandidates() {
// First, 'match' so that all documents are found
final MatchOperation getAll = Aggregation.match(new Criteria("_id").ne(null));
// Then sort by priority
final SortOperation sort = Aggregation.sort(Sort.by(Sort.Direction.DESC, "priority"));
// After that, group by internalUuid and make sure to also push the full event to not lose it for the next step
final GroupOperation groupByUuid = Aggregation.group("internalUuid").push("$$ROOT").as("events");
// Get the first element of each sorted and grouped list (I'm not fully sure what the 'internalUuid' parameter does here and if I could change that)
final ProjectionOperation getFirst = Aggregation.project("internalUuid").and("events").arrayElementAt(0).as("event");
// We're nearly done! Only thing left to do is to map to our Event to have a usable List of Event in .getMappedResults()
final ProjectionOperation map = Aggregation.project("internalUuid")
.and("event._id").as("_id")
.and("event.internalUuid").as("internalUuid")
.and("event.eventType").as("eventType")
.and("event.priority").as("priority");
final Aggregation aggregation = Aggregation.newAggregation(getAll, sort, groupByUuid, getFirst, map);
final AggregationResults<InvoiceEvent> aggregationResults =
mongoTemplate.aggregateAndReturn(InvoiceEvent.class).by(aggregation).all();
return aggregationResults.getMappedResults();
}
I've got a lot of records like:
{
"id": "1000",
"lastSeen": "2018-02-26T18:49:21.863Z"
}
{
"id": "1000",
"lastSeen": "2017-02-26T18:49:21.863Z"
}
{
"id": "2000",
"lastSeen": "2018-02-26T18:49:21.863Z"
}
{
"id": "2000",
"lastSeen": "2017-02-26T18:49:21.863Z"
}
I'd like to get the most recent records for all ids. So in this case the output would be the following(most recent record for ids 1000 and 2000):
{
"id": "1000",
"lastSeen": "2018-02-26T18:49:21.863Z"
}
{
"id": "2000",
"lastSeen": "2018-02-26T18:49:21.863Z"
}
With N1QL, this would be
SELECT id, MAX(lastSeen) FROM mybucket GROUP BY id
How would I do this using a couchbase view and map/reduce?
Thanks!
I am far from a regular user of map/reduce, and there may be more efficient JavaScript, but try this:
Map
function (doc, meta) {
emit(doc.id, doc.lastSeen);
}
Reduce
function reduce(key, values, rereduce) {
var max = values.sort().reverse()[0];
return max;
}
Filter: ?limit=6&stale=false&connection_timeout=60000&inclusive_end=true&skip=0&full_set=true&group_level=1
The idea is to sort all the values being emitted (lastSeen). Since they are ISO 8601 and can be lexigraphically sorted, sort() works just fine. You want the latest, so that's what the reverse() is for (otherwise you'd get the oldest).
The filter has a group_level of 1, so it will group by the doc.id field.
You can query by descending and reduce to first one on list as below:
Map:
function (doc, meta) {
emit(doc.id, doc.lastSeen);
}
Reduce:
function reduce(key, values, rereduce) {
return values[0];
}
Filter:
?inclusive_end=true&skip=0&full_set=&group_level=1&descending=true
This will eliminate the overhead of sorting the grouped values inside reduce function.
I have articles & tags collection. Articles contain tags which is array of objectId. I want to fetch tagName as well, so I unwind (this gives me multiple rows - 1 per tag array entry) => lookup (joins with tabs collection) => group (combine it into original result set)
My mongodb query is as follows, which gives me correct result:
db.articles.aggregate([
{"$unwind": "$tags"},
{
"$lookup": {
"localField": "tags",
"from": "tags",
"foreignField": "_id",
"as": "materialTags"
}
},
{
"$group": {
"_id": "$_id",
"title": {"$first": "$title"},
"materialTags": {"$push": "$materialTags"}
}
}
])
My corresponding Spring code:
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
//I also want to add group operation but unable to find the proper syntax ??.
Aggregation aggregation = Aggregation.newAggregation(unwindOperation,
lookupOperation1, ??groupOperation?? );
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
I tried to play around with group operation but without much luck. How can I add group operations as per original query ?
Thanks in advance.
Group query syntax in Spring for
{
"$group": {
"_id": "$_id",
"title": {"$first": "$title"},
"materialTags": {"$push": "$materialTags"}
}
}
is
Aggregation.group("_id").first("title").as("title").push("materialTags").as("materialTags")
Final query
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
Aggregation aggregation = Aggregation.newAggregation(unwindOperation,
lookupOperation1, Aggregation.group("_id").first("title").as("title").push("materialTags").as("materialTags") );
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
To get more info please go thru the below references
http://www.baeldung.com/spring-data-mongodb-projections-aggregations
spring data mongodb group by
Create Spring Data Aggregation from MongoDb aggregation query
https://www.javacodegeeks.com/2016/04/data-aggregation-spring-data-mongodb-spring-boot.html
I have a collection with documents of the following structure:
{
"category": "movies",
"movies": [
{
"name": "HarryPotter",
"language": "english"
},
{
"name": "Fana",
"language": "hindi"
}
]
}
I want to query with movie name="fana" and the response sholud be
{
"category": "movies",
"movies": [
{
"name": "HarryPotter",
"language": "english"
}
]
}
How do I get the above using spring mongoTemplate?
You can try something like this.
Non-Aggregation based approach:
public MovieCollection getMoviesByName() {
BasicDBObject fields = new BasicDBObject("category", 1).append("movies", new BasicDBObject("$elemMatch", new BasicDBObject("name", "Fana").append("size", new BasicDBObject("$lt", 3))));
BasicQuery query = new BasicQuery(new BasicDBObject(), fields);
MovieCollection groupResults = mongoTemplate.findOne(query, MovieCollection.class);
return groupResults;
}
Aggregation based approach:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
import static org.springframework.data.mongodb.core.query.Criteria.where;
public List<BasicDBObject> getMoviesByName() {
Aggregation aggregation = newAggregation(unwind("movies"), match(where("movies.name").is("Fana").and("movies.size").lt(1)),
project(fields().and("category", "$category").and("movies", "$movies")));
AggregationResults<BasicDBObject> groupResults = mongoTemplate.aggregate(
aggregation, "movieCollection", BasicDBObject.class);
return groupResults.getMappedResults();
}
$unwind of mongodb aggregation can be used for this.
db.Collection.aggregate([{
{$unwind : 'movies'},
{$match :{'movies.name' : 'fana'}}
}])
You can try the above query to get required output.
Above approaches provides you a solution using aggregation and basic query. But if you dont want to use BasicObject below code will perfectly work:
Query query = new Query()
query.fields().elemMatch("movies", Criteria.where("name").is("Fana"));
List<Movies> movies = mongoTemplate.find(query, Movies.class);
The drawback of this query is that it may return duplicate results present in different documents, since more than 1 document may match this criteria. So you can add _id in the criteria like below:
Criteria criteria = Criteria.where('_id').is(movieId)
Query query = new Query().addCriteria(criteria)
query.fields().elemMatch("movies", Criteria.where("name").is("Fana"));
query.fields().exclude('_id')
List<Movies> movies = mongoTemplate.find(query, Movies.class);
I am excluding "_id" of the document in the response.