I´ve been trying to get multiple averages grouping by an identifier using the Spring boot Aggregation. The fields whose average I want are defined by an Array given by parameter, but I can´t retrieve more than one average.
The data is stored like :
{
"_id": {
"$oid": "XX"
},
"space_id": "AA",
"temperature": 50,
"temperatureUoMCode": 0,
"pm25": 50,
"pm25UoMCode": 0,
"co2": 0.0005,
"co2UoMCode": 0,
"co": 1,
"coUoMCode": 0,
"gas": 50,
"gasUoMCode": 0,
"humidity": 50,
"humidityUoMCode": 0,
"occupants": 6,
"maxCapacity": 15,
"_class": "Record"
}
The code that I am using:
List<AggregationOperation> aggregationOperations = new ArrayList<>();
aggregationOperations.add(Aggregation.match(
Criteria.where("space_id").is(space_id)
));
for(Variable var : concerns){
System.out.println(var.getField());
aggregationOperations.add(Aggregation.group(
space_id).
avg(var.getField()).as(var.getAvg()));
}
Aggregation aggregation = Aggregation.newAggregation(
aggregationOperations
);
AggregationResults<HashMap> results = mongoTemplate.aggregate(
aggregation, "records",HashMap.class);
Also, Variable class is an enum with two fields, the name of the field in the db (field) and the average alias (avg).
When I run it I get the following output:
co
pm25
temperature
humidity
gas
2021-09-22 11:34:17.046 ERROR 42971 --- [a1-0b2af6e69097] TConfig$$EnhancerBySpringCGLIB$$673c396b : Invalid reference 'pm25'!
java.lang.IllegalArgumentException: Invalid reference 'pm25'!
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:114)
at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:86)
at org.springframework.data.mongodb.core.aggregation.GroupOperation$Operation.getValue(GroupOperation.java:529)
at org.springframework.data.mongodb.core.aggregation.GroupOperation$Operation.toDocument(GroupOperation.java:507)
at org.springframework.data.mongodb.core.aggregation.GroupOperation.toDocument(GroupOperation.java:441)
at org.springframework.data.mongodb.core.aggregation.AggregationOperation.toPipelineStages(AggregationOperation.java:55)
at org.springframework.data.mongodb.core.aggregation.AggregationOperationRenderer.toDocument(AggregationOperationRenderer.java:56)
at org.springframework.data.mongodb.core.aggregation.AggregationPipeline.toDocuments(AggregationPipeline.java:81)
at org.springframework.data.mongodb.core.aggregation.Aggregation.toPipeline(Aggregation.java:716)
at org.springframework.data.mongodb.core.AggregationUtil.createPipeline(AggregationUtil.java:112)
at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2144)
at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2119)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2113)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2014)
at co.edu.javeriana.smartuj.EnvironmentalAlerts.services.AlertService.makeQuery(AlertService.java:131)
If someone is having a similar problem I managed to solve it by following this post's answer. I realized that instead of creating various GroupOperations, I could just create a single one and use it to store all the operations by saving the return of the "as" function in the same object:
MatchOperation matchOperation = Aggregation.match(Criteria.where("space_id").is(space_id));
GroupOperation groupOperation = null;
for(Variable var : concerns){
if (groupOperation == null){
groupOperation = Aggregation.group("space_id")
.avg(var.getField())
.as(var.getAvg());
}else
groupOperation = groupOperation
.avg(var.getField())
.as(var.getAvg());
}
Aggregation aggregation = Aggregation.newAggregation(
matchOperation,
groupOperation
);
AggregationResults<HashMap> results = mongoTemplate.aggregate(
aggregation, "records",HashMap.class);
return results.getUniqueMappedResult();
Related
In my Laravel-5.8, I have these four queries accessng the same model:
$allLeaves = HrLeaveRequest::where('company_id', $userCompany)->whereYear('created_at', date('Y'))->count();
$pendingLeaves = HrLeaveRequest::where('leave_status', 1)->where('company_id', $userCompany)->whereYear('created_at', date('Y'))->count();
$rejectedLeaves = HrLeaveRequest::where('leave_status', 3)->where('company_id', $userCompany)->whereYear('created_at', date('Y'))->count();
$approvedLeaves = HrLeaveRequest::where('leave_status', 4)->where('company_id', $userCompany)->whereYear('created_at', date('Y'))->count();
How do I combine the four queries as one
something similar to this
$gender_unpublished_record = HrEmployee::selectRaw('count(gender_code) as count,gender_code, if (gender_code = 1, "Male", "Female") as gender')->whereNotIn('employee_code', $publishedgoals)->where('company_id', $userCompany)->where('hr_status', 0)->groupBy('gender_code')->get();
The one above only have 0 or 1. But what I want to achive takes care of everything in the table, leave_status as 1, 3 and 4
Thank you
in your Company Model you should have the relation:
public function HrLeaveRequests()
{
return $this->hasMany(HrLeaveRequest::class,'company_id');
}
now you could use withCount:
$value=Company::where('company_id',$userCompany)->withCount(['HrLeaveRequests as allLeaves'=>function($query){
$query ->whereYear('created_at', date('Y'));
},
'HrLeaveRequests as pendingLeaves'=>function($query){
$query->where('leave_status', 1)->whereYear('created_at', date('Y'));
},
'HrLeaveRequests as rejectedLeaves'=>function($query){
$query->where('leave_status', 3)->whereYear('created_at', date('Y'));
},
'HrLeaveRequests as approvedLeaves'=>function($query){
$query->where('leave_status', 4)->whereYear('created_at', date('Y'));
},
])->get();
I've built this aggregation:
ProjectionOperation projectStage = Aggregation
.project("application", "uploadedRefs", "uploadedKb", "downloadedDocs", "downloadedKb")
.and(DateOperators.Year.yearOf("timestamp")).as("year")
.and(DateOperators.Month.monthOf("timestamp")).as("month")
.and(DateOperators.DayOfMonth.dayOfMonth("timestamp")).as("day")
.and(DateOperators.DateFromParts.dateFromParts()
.yearOf("timestamp")
.monthOf("timestamp")
.dayOf("timestamp")
).as("startIntervalTimestamp");
Aggregation aggregation = Aggregation
.newAggregation(
projectStage
);
System.out.println(aggregation.toString());
The output is:
[
{
"$project":{
"application":1,
"uploadedRefs":1,
"uploadedKb":1,
"downloadedDocs":1,
"downloadedKb":1,
"year":{
"$year":"$timestamp"
},
"month":{
"$month":"$timestamp"
},
"day":{
"$dayOfMonth":"$timestamp"
},
"startIntervalTimestamp":{
"$dateFromParts":{
"year":"timestamp",
"month":"timestamp",
"day":"timestamp"
}
}
}
}
]
The error message is:
Error: command failed: {
"ok" : 0,
"errmsg" : "'year' must evaluate to an integer, found string with value \"timestamp\"",
"code" : 40515,
"codeName" : "Location40515"
}
Solved:
Field timestampField = Fields.field("timestamp");
ProjectionOperation projectStage = Aggregation
.project("application", "uploadedRefs", "uploadedKb", "downloadedDocs", "downloadedKb")
.and(DateOperators.Year.year(timestampField)).as("year")
.and(DateOperators.Month.month(timestampField)).as("month")
.and(DateOperators.DayOfMonth.dayOfMonth(timestampField)).as("day")
.and(DateOperators.DateFromParts.dateFromParts()
.year(DateOperators.Year.year(timestampField))
.month(DateOperators.Month.month(timestampField))
.day(DateOperators.DayOfMonth.dayOfMonth(timestampField))
).as("startIntervalTimestamp");
Hi a am try to do rest api in spring boot with mongodb to find group by count the input data look like. please share any logic, code, example link.
guys i am expecting spring boot logic. how mongodb aggregation framework integrating.
{
"_id" : "PRODUCT_01",
"productname" : "product1",
"value" : "codesoft"
},
{
"_id" : "PRODUCT_01",
"productname" : "product2",
"value" : "codesoft"
},
{
"_id" : "PRODUCT_01",
"productname" : "product1",
"value" : "codesoft"
}
expected output
{
product1 : 2,
product2 : 1
}
Any help is appreciated.
try this
db.testColln.aggregate(
{
$group : {_id : "$productname", total : { $sum : 1 }}
}
);
for Spring Boot
Aggregation agg = newAggregation(
group("productname").count().as("total")
project("productname").and("total"),
);
AggregationResults<Product> groupResults
= mongoTemplate.aggregate(agg, Product.class,Result.class);
List<Result> result = groupResults.getMappedResults();
public class Result {
private String productname;
private long total;
}
#GetMapping("/group")
public List<ProductCount> groupByName() {
// grouping by prductName
GroupOperation groupOperation =
Aggregation.group("productName").count().as("count");
// projection operation
ProjectionOperation projectionOperation =
Aggregation.project("count").and("productName").previousOperation();
// sorting in ascending
SortOperation sortOperation =
Aggregation.sort(Sort.by(Sort.Direction.ASC, "count"));
// aggregating all 3 operations using newAggregation() function
Aggregation aggregation =
Aggregation.newAggregation(groupOperation,projectionOperation
,sortOperation);
// putting in a list
// "products" is collection name
AggregationResults<ProductCount> result =
mongotemplate.aggregate(aggregation, "products",
ProductCount.class);
return result.getMappedResults();
}
$ make ProductCount class in model package
public class ProductCount {
private String productName;
private int count;
#getters
#setters
I am trying to implement a MongoDB query in java using Spring mongo.
This is the native MongoDB query:
db.ShieldReport.aggregate([
{$match:{"sellerCode":"e1aaf3"}},
{$project:{bucketName:"$bucketName", brandName: "$brandName", createdTime : "$createdTime", sellerCode : "$sellerCode"}},
{$sort:{"createdTime":-1}},
{$group:
{
_id: { sellerCode: "$sellerCode", bucketName: "$bucketName", brandName: "$brandName"},
itemsSold: { $first: { bucketName: "$bucketName", brandName: "$brandName", createdTime : "$createdTime"} }
}
},
{$sort:{"itemsSold.createdTime":-1}},
{$project : { _id : "$_id.sellerCode", bucketName :"$itemsSold.bucketName", brandName : "$itemsSold.brandName"}}
])
In my Spring java version, I have got it this far:
Aggregation agg = newAggregation(
match(Criteria.where("sellerCode").is(filterRequest.getSellerCode())),
Aggregation.project("bucketName")
.andInclude("brandName")
.andInclude("createdTime")
.andInclude("sellerCode"),
sort(Sort.Direction.DESC, "createdTime"),
group("sellerCode", "brandName", "bucketName")
);
But the problem is I am not able to create the itemSold field. How do I create that?
You could restructure your pipeline in Spring as follows:
Aggregation agg = Aggregation.newAggregation(
Aggregation.match(Criteria.where("sellerCode").is(filterRequest.getSellerCode())),
Aggregation.sort(Sort.Direction.DESC, "createdTime"),
Aggregation.group("sellerCode", "bucketName", "brandName")
.first("bucketName").as("bucketName")
.first("brandName").as("brandName")
.first("createdTime ").as("createdTime "),
Aggregation.sort(Sort.Direction.DESC, "createdTime"),
Aggregation.project("bucketName", "brandName")
.and("sellerCode").previousOperation()
);
As you can see some of the pipeline stages have been omitted because they are really not necessary. For instance, the $project pipeline before the $group step can be taken off since the $group pipeline operator will apply the accumulators
In the group step, you group the intermediate filtered documents by the three fields and store the other field values in new fields aided by the $first operator.
Sort the intermediate result by the createdTime field reference of the previous group operation.
Finally in the fourth step, select the "bucketName" and "brandName" fields from the previous group operation. Note that "sellerCode" again implicitly references an group-id field.
I'm new with rethinkdb, I'm trying to write an update query preferred upsert which will set values to several fields if they don't exist or their values is less than the new value I want to set. Below is the way I did it with mongodb
collection.updateOne(new BasicDBObject(BookKeeperEvent.tenantPropertyName, bookKeeper.getTenantId()).append(BookKeeperEvent.timeLayerPropertyName, bookKeeper.getTimeLayer()),
new BasicDBObject("$max", new BasicDBObject(BookKeeperEvent.latestFullDataPropertyName, bookKeeper.getLatestFullData())
.append(BookKeeperEvent.latestRawDataPropertyName, bookKeeper.getLatestRawData())
.append(BookKeeperEvent.latestHealthPropertyName, bookKeeper.getLatestHealth())
.append(BookKeeperEvent.lastUpdatePropertyName, bookKeeper.getLastUpdate())
.append(BookKeeperEvent.firstFullDataPropertyName, bookKeeper.getFirstFullData()))
.append("$setOnInsert", new BasicDBObject(BookKeeperEvent.tenantPropertyName, bookKeeper.getTenantId()).append(BookKeeperEvent.timeLayerPropertyName, bookKeeper.getTimeLayer())),
new UpdateOptions().upsert(true))
This code set tenantId and timeLayer only if they are null, and for the other fields only if they are null or the value is less than the value I set in the query.
Is there a way to do the same thing in rethinkdb? and how?
I think the following query is equivalent to what I did in mongo but it fails.
r.table('book_keeper').filter({tenantId: '123', timeLayer: 4}).coerceTo('array').do(function (matches) {
return r.branch(
matches.isEmpty(),
r.table('book_keeper').insert({tenantId: '123', timeLayer: 4, latestRawData: 100, id: 4}),
r.table('book_keeper').get(4).update(function (doc) {
return r.branch(
doc.not(doc.hasFields('firstFullData')).or(doc('firstFullData').lt(100)),
{firstFullData : 100},
null)
}
))
})
The excption is:
e: Expected 1 argument but found 2 in:
r.table("book_keeper").filter({"tenantId": "123", "timeLayer": 4}).coerceTo("array").do(function(var_193) { return r.branch(var_193.isEmpty(), r.table("book_keeper").insert({"tenantId": "123", "timeLayer": 4, "latestRawData": 100, "id": 4}), r.table("book_keeper").get(4).update(function(var_194) { return r.branch(var_194.not(var_194.hasFields("firstFullData")).or(var_194("firstFullData").lt(100)), {"firstFullData": 100}, null); })); })
How can I fix it?
Thanks,
Daniela