Aggregation aggregation = newAggregation(
match(Criteria.where("moment").gte(from).lte(to)),
project("pass", "carrier", "route", "iso").and("iso").as("country")
.and("$moment").plus(14400000).as("shifted"),
project("shifted").and(DateOperators.DayOfMonth.dayOfMonth("$moment")).previousOperation(),
.andExpression("$moment").dateAsFormattedString("%Y-%m-%d").as("date"),
group("pass", "carrier", "route", "country","date").count().as("total"),
sort(Sort.Direction.DESC, "pass","total")
).withOptions(newAggregationOptions().
allowDiskUse(true).build());
I am trying to do this query in java (mongodb and spring boot) that works in console (the bottom one works fine) but i failed to pass the correct date shifted already (called shifted) to the DateOperators.DayOfMonth. I get projection field must not be null so I think i am using wrong previousOperation(). How should i properly use this previousOperation? or how can i correctly shifted the date i need? thank you very much in advance.
db.billing.aggregate(
[
{
$match: { "moment": { "$gte": ISODate("2020-11-29T00:00:00Z"), "$lt": ISODate("2020-11-29T23:59:59Z")}}
},
{
$project: { "pass" :1, "carrier": 1, "moment" : { $subtract: ["$moment", 14400000 ] } }
},
{
$group: { "_id": {"pass": "$pass","carrier": "$carrier", "day": { "$dayOfMonth": "$moment"},"total": { "$sum": 1 }}}
},
{
$sort: {"_id.day":1}
}
]
)
Related
I've got an employee collection that looks like:
{
_id
company_id
job_type
name
age
created_at
...
}
I've for a list of company_ids and for each of them I want to get the latest 3 employees for job_type="part_time" and the latest 3 for job_type="intern" (according to the created_at field)
How can I do such a thing using one call?
An answer with mongoTemplate.bulkOps will also be valid.
To get the latest you can use the auto created _id field it has a date embedded in it. Assuming the name of your collection is employee you can use the following aggregation:
db.employee.aggregate([
{
$facet: {
"part_time": [
{
$match: {
"job_type": "part_time"
}
},
{
$match: {
"company_id": "id_from_your_list"
}
},
{ $sort: { "_id": 1 } },
{
$limit: 3
}
],
"intern": [
{
$match: {
"job_type": "intern"
}
},
{
$match: {
"company_id": "id_from_your_list"
}
},
{ $sort: { "_id": 1 } },
{
$limit: 3
}
]
}
}])
I assume that you are using spring data mongodb you can execute this aggregation operation using MongoTemplate helper.
FacetOperation facetOperation = facet().and(match(Criteria.where("job_type").is("part_time")),match(Criteria.where("company_id").is("company_id_from_your_list")),
sort(Sort.Direction.ASC, "_id"),
limit(3)).as("part_time")
.and(match(Criteria.where("job_type").is("intern")),match(Criteria.where("company_id").is("company_id_from_your_list")),
sort(Sort.Direction.ASC, "_id"),
limit(3)).as("intern");
mongoTemplate.aggregate(Aggregation.newAggregation(facetOperation), "employee", Document.class).getUniqueMappedResult();
I have the following data:
{
"_id": ObjectID("5e2fa881c3a1a70006c5743c"),
"name": "Some name",
"policies": [
{
"cId": "dasefa-2738-4cf0-90e0d568",
"weight": 12
},
{
"cId": "c640ad67dasd0-92f981583568",
"weight": 50
}
]
}
I'm able to query this with Spring Mongo fine, however I want to be able to order the policies by weight
At the moment I get my results fine with:
return mongoTemplate.find(query, CArea::class.java)
However say I make the following aggregations:
val unwind = Aggregation.unwind("policies")
val sort = Aggregation.sort(Sort.Direction.DESC,"policies.weight")
How can I go and actually apply those to the returned results above? I was hoping that the dot annotation would do the job in my query however didnt do anything e.g. Query().with(Sort.by(options.sortDirection, "policies.weight"))
Any help appreciated.
Thanks.
I am not familier with Spring Mongo, but I guess you can convert the following aggregation to spring code.
db.collection.aggregate([
{
$unwind: "$policies"
},
{
$sort: {
"policies.weight": -1
}
},
{
$group: {
_id: "$_id",
"policies": {
"$push": "$policies"
},
parentFields: {
$first: "$$ROOT"
}
}
},
{
$replaceRoot: {
newRoot: {
$mergeObjects: [
"$parentFields",
{
policies: "$policies"
}
]
}
}
}
])
This will result:
[
{
"_id": "5e2fa881c3a1a70006c5743c",
"name": "Some name",
"policies": [
{
"cId": "c640ad67dasd0-92f981583568",
"weight": 50
},
{
"cId": "dasefa-2738-4cf0-90e0d568",
"weight": 12
}
]
}
]
Playground
Basically I am trying to use REST hreat to create two variables issuerId and sectionName. I am trying to get data from the URI mentioned below
http://ftc-wbpyrdb201:8080/statdata/InsStatData/_aggrs/getDataByIssuerAndSectionName?avars={"issuerId":19038},{"sectionName":"ASSETS"}
My aggregation definition is a below
{"$push" : {
"aggrs":
{
"type": "pipeline",
"uri": "getDataByIssuerAndSectionName",
"stages": [
{
"_$match": {
"_$and": [
{"issuerId": {"_$var": "issuerId" }},
{
"sectionName": {"_$var": "sectionName"}
}
]
}
},
{
"_$unwind": "$sections"
},
{
"_$unwind": "$sections.data"
},
{
"_$unwind": "$sections.data.values"
},
{
"_$match": {
"_$and": [
{"issuerId": {"_$var": "issuerId" }},
{
"sectionName": {"_$var": "sectionName"}
}
]
}
}
]
}
}
}
I am sure I am doing something wring in either defining the aggregation or while requesting the URI. There is not enough in the documentation. Please help me with this request.
db.flm_conversation.aggregate(
[
{$match: {"conversationRecords.isPrimary":true,"conversationRecords.commentTime":{'$gte': new Date('2016-12-18 00:00:00'), '$lte': new Date('2016-12-18 23:59:59')} } },
{$unwind:"$conversationRecords"},
{ $group: {
_id: {
"commentLevel": "$commentLevel",
"time":{"$add": [
{
"$subtract": [
{ "$subtract": [
"$conversationRecords.commentTime",
new Date(0)
]}
,
{ "$mod": [
{ "$subtract": [
"$conversationRecords.commentTime",
new Date(0)
]},
1000 * 60 * 30
]}
]
},
new Date(0)
]}
},
count: { "$sum": 1 }
}},
{ $group: {_id: "$_id.commentLevel",count: { "$sum": 1 },pointrecord:{$push: {time:"$_id.time",count:"$count"} } }},
{ $project: { _id: 1,count:1,pointrecord:1 } }
])
How to convert this query using Spring Mongodb Aggregation apis?
AggregationOperation match = Aggregation.match(Criteria.where("conversationRecords.isPrimary").is(true)
.and("conversationRecords.commentTime").gte(DateUtils.stringToDate("2016-12-18 00:00:00","yyyy-MM-dd HH:mm:ss")).lte(DateUtils.stringToDate("2016-12-18 23:59:59","yyyy-MM-dd HH:mm:ss")));
AggregationOperation unwind = Aggregation.unwind("conversationRecords");
AggregationOperation group = Aggregation.group("commentLevel");
Aggregation agg = newAggregation(match,unwind,group);
AggregationResults<SummaryRecordAggre> groupResults
= mongoTemplate.aggregate(agg, COLLECTION_NAME, SummaryRecordAggre.class);
I don't know group "$add" how to convert?I find by
http://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo.aggregation
You can try something like this. I removed the extra operators which I felt was not needed but we can easily put them back should you need.
The first group which is something you need to build using MongoDB objects as currently all aggregation operators are not supported out of box in spring mongo.
Similar way for push operator in second group as you cant push multiple values.
AggregationOperation match = Aggregation.match(Criteria.where("conversationRecords.isPrimary").is(true)
.and("conversationRecords.commentTime").gte(DateUtils.stringToDate("2016-12-18 00:00:00", "yyyy-MM-dd HH:mm:ss")).lte(DateUtils.stringToDate("2016-12-18 23:59:59", "yyyy-MM-dd HH:mm:ss")));
AggregationOperation unwind = Aggregation.unwind("conversationRecords");
AggregationOperation firstGroup = context -> context.getMappedObject(new BasicDBObject(
"$group", new BasicDBObject(
"_id", new BasicDBObject("commentLevel", "$commentLevel")
.append(
"time", new BasicDBObject(
"$subtract", new Object[]{"$conversationRecords.commentTime", new BasicDBObject(
"$mod", new Object[]{"$conversationRecords.commentTime", 1000 * 60 * 30})}))).append("count", new BasicDBObject("$sum", 1))));
AggregationOperation secondGroup = Aggregation.group("_id.commentLevel").count().as("count").push(new BasicDBObject("time", "$_id.time").append("count", "$count")).as("pointrecord");
AggregationOperation project = Aggregation.project("_id", "count", "pointrecord");
Aggregation agg = newAggregation(match, unwind, firstGroup, secondGroup, project);
This is equivalent of below query
[{
"$match": {
"conversationRecords.isPrimary": true,
"conversationRecords.commentTime": {
"$gte": {
"$date": "2016-12-27T14:46:50.896Z"
},
"$lte": {
"$date": "2016-12-27T14:46:50.896Z"
}
}
}
}, {
"$unwind": "$conversationRecords"
}, {
"$group": {
"_id": {
"commentLevel": "$commentLevel",
"time": {
"$subtract": ["$conversationRecords.commentTime", {
"$mod": ["$conversationRecords.commentTime", 1800000]
}]
}
},
"count": {
"$sum": 1
}
}
}, {
"$group": {
"_id": "$_id.commentLevel",
"count": {
"$sum": 1
},
"pointrecord": {
"$push": {
"time": "$_id.time",
"count": "$count"
}
}
}
}, {
"$project": {
"_id": 1,
"count": 1,
"pointrecord": 1
}
}]
}
I am currently trying to build this function score query with the Java API of elasticsearch:
{
"query": {
"function_score": {
"functions": [
{
"gauss": {
"location": {
"origin": {
"lat": 52.55,
"lon": 13.69
},
"offset": "30km",
"scale": "10km",
"decay": 0.9
}
}
},
{
"gauss": {
"createdAt": {
"origin": "2015-06-14T15:50:00",
"scale": "8h",
"offset": "4h",
"decay": 0.75
}
}
}
]
}
}
}
But I can't find any documentation regarding the java api and the function score queries. This is what I have so far:
elasticsearch.client
.prepareSearch(config.offerIndex.value)
.setQuery(
QueryBuilders.functionScoreQuery(
ScoreFunctionBuilders
.gaussDecayFunction("location", ???, ???).setDecay(0.9)
)
)
The second and the third parameter of the gaussDecayFunction are named origin and scale. But they have the any type and I have no idea how I have to provide my location and time values there.
And the next question is how I can provide to functions in the FunctionScore Builder
I have found this solution but I am not sure if this is the clean way to do it. Would appreciate if someone can approve it.
val lat = 52.52
val lon = 13.402
QueryBuilders
.functionScoreQuery(
ScoreFunctionBuilders.gaussDecayFunction("location", new GeoPoint(lat, lon), "10km")
.setDecay(0.9)
.setOffset("30km"))
.add(
ScoreFunctionBuilders.gaussDecayFunction("createdAt", new DateTime(), "8h")
.setDecay(0.75)
.setOffset("4h"))
)