I'm going crazy with this one...
I have this aggregation framework expression working like a charm in mongo shell:
{ $group :
{
_id : '$Code' ,
'Special' : { $sum : { $cond: [{ $eq: [ '$Special', 'Success']},1,0]}}
}
}
I need to do it in c#, I tried a lot of combinations but without success.
Has anyone any clue?
Thx
Give this a try:
var group = new BsonDocument
{
{
"$group",
new BsonDocument
{
{
"_id", "$Code"
},
{
"Special", new BsonDocument
{
{ "$sum", new BsonDocument
{
{"$cond", new BsonArray
{
new BsonDocument
{
{
"$eq", new BsonArray {"$Special", "Success"}
}
},
1,
0
}
}
}
}
}
}
}
}
};
Related
I'm trying to set an attribute of a document inside an array to uppercase.
This is a document example
{
"_id": ObjectId("5e786a078bc3b3333627341e"),
"test": [
{
"itemName": "alpha305102992",
"itemNumber": ""
},
{
"itemName": "beta305102630",
"itemNumber": "P5000"
},
{
"itemName": "gamma305102633 ",
"itemNumber": ""
}]
}
I already tried a lot of thing.
private void NameElementsToUpper() {
AggregationUpdate update = AggregationUpdate.update();
//This one does not work
update.set("test.itemName").toValue(StringOperators.valueOf(test.itemName).toUpper());
//This one also
update.set(SetOperation.set("test.$[].itemName").withValueOfExpression("test.#this.itemName"));
//And every variant in between these two.
// ...
Query query = new Query();
UpdateResult result = mongoTemplate.updateMulti(query, update, aClass.class);
log.info("updated {} records", result.getModifiedCount());
}
I see that Fields class in spring data is hooking into the "$" char and behaving special if you mention it. Do not seem to find the correct documentation.
EDIT: Following update seems to work but I do not seem to get it translated into spring-batch-mongo code
db.collection.update({},
[
{
$set: {
"test": {
$map: {
input: "$test",
in: {
$mergeObjects: [
"$$this",
{
itemName: {
$toUpper: "$$this.itemName"
}
}
]
}
}
}
}
}
])
Any solutions?
Thanks!
For now I'm using which does what i need. But a spring data way would be cleaner.
mongoTemplate.getDb().getCollection(mongoTemplate.getCollectionName(Application.class)).updateMany(
new BasicDBObject(),
Collections.singletonList(BasicDBObject.parse("""
{
$set: {
"test": {
$map: {
input: "$test",
in: {
$mergeObjects: [
"$$this",
{
itemName: { $toUpper: "$$this.itemName" }
}
]
}
}
}
}
}
"""))
);
i have this mongo query:
db.getCollection('My_collection_name').aggregate([
{ $project: { warehouses: { $objectToArray: "$outputVariables" } } },
{ $unwind: "$warehouses" },
{ $group: { _id: "$warehouses.k" }}
])
someone could help me to translate in spring mongoTemplate?
Thanks
The query should be write as below:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
...
AggregationOperation project = project()
.and(ObjectToArray.valueOfToArray("outputVariables")).as("warehouses");
AggregationOperation unwind = unwind("warehouses");
AggregationOperation group = Aggregation.group("warehouses.k");
Aggregation aggregation = Aggregation.newAggregation(
project,
unwind,
group);
String collectionName = "My_collection_name";
System.out.println("aggregation=" + aggregation);
this.mongoTemplate.aggregate(aggregation, collectionName, Output.class);
It generate output:
aggregation={ "aggregate" : "__collection__", "pipeline" : [{ "$project" : { "warehouses" : { "$objectToArray" : "$outputVariables" } } }, { "$unwind" : "$warehouses" }, { "$group" : { "_id" : "$warehouses.k" } }] }
I need some help in creating an AggregationDictionary from the following elasticsearch query
GET organisations/_search
{
"size": 0,
"aggs": {
"by_country": {
"nested": {
"path": "country"
},
"aggs": {
"by_country2": {
"filter": {
"bool": {
"must": [
{
"term": {
"country.isDisplayed": "true"
}
}
]
}
},
"aggs": {
"by_country3": {
"terms": {
"field": "country.displayName.keyword",
"size": 9999
}
}
}
}
}
}
}
}
I managed to write this horrible piece of code which I am pretty sure it is wrong, I am totally new to this.
AggregationDictionary aggs = new AggregationDictionary()
{
{
"countries_step1",
new NestedAggregation("countries_step1")
{
Path = "country",
Aggregations = new AggregationDictionary()
{
{
"countries_step2",
new FilterAggregation("countries_step2")
{
Filter = new BoolQuery
{
Must = new QueryContainer[] {
new NestedQuery
{
Query = new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
}
},
Aggregations = new AggregationDictionary
{
{
"countries_step3",
new TermsAggregation("countries_step3")
{
Field = "country.displayName.keyword",
Size = 9999
}
}
}
}
}
}
}
}
};
Can someone tell me if I am in the correct direction? I am using Nest 6.6.0. Is there any tool that helps with these translations?
What you have so far is pretty solid, but when you try to execute this aggregation with the following call
var searchAsync = await client.SearchAsync<Document>(s => s.Size(0).Aggregations(aggs));
you will get this error
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "query malformed, empty clause found at [14:22]"
}
],
"type" : "illegal_argument_exception",
"reason" : "query malformed, empty clause found at [14:22]"
},
"status" : 400
}
Checking request which was sent to elasticsearch give us the answer why it happened
{
"aggs": {
"countries_step1": {
"aggs": {
"countries_step2": {
"aggs": {
"countries_step3": {
"terms": {
"field": "country.displayName.keyword",
"size": 9999
}
}
},
"filter": {}
}
},
"nested": {
"path": "country"
}
}
},
"size": 0
}
filter clause is empty, this is because you tried to used nested query but you didn't pass path parameter. We don't need nested query here (as shown in your example query), we can simplify the whole query to
var aggs = new AggregationDictionary()
{
{
"countries_step1",
new NestedAggregation("countries_step1")
{
Path = "country",
Aggregations = new AggregationDictionary()
{
{
"countries_step2",
new FilterAggregation("countries_step2")
{
Filter = new BoolQuery
{
Must = new QueryContainer[]
{
new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
},
Aggregations = new AggregationDictionary
{
{
"countries_step3",
new TermsAggregation("countries_step3")
{
Field = "country.displayName.keyword",
Size = 9999
}
}
}
}
}
}
}
}
};
Now we have a valid request sent to elasticsearch.
There are a couple of things we can improve here:
1. Remove unnecessary bool query
Filter = new BoolQuery
{
Must = new QueryContainer[]
{
new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
},
to
Filter =
new TermQuery
{
Field = "country.isDisplayed",
Value = true
},
2. Replace string field names
Usually, when doing calls from .Net there is some kind of POCO type which is helping us with writing strongly-typed requests to elasticsearch which helps us managing clean code and refactoring. With this, we can change field definition from
"country.displayName.keyword"
to
Infer.Field<Document>(f => f.Country.FirstOrDefault().DisplayName.Suffix("keyword"))
my types definition
public class Document
{
public int Id { get; set; }
[Nested]
public List<Country> Country { get; set; }
}
public class Country
{
public bool IsDisplayed { get; set; }
public string DisplayName { get; set; }
}
3. Consider using a fluent syntax
With NEST you can write queries in two ways: using object initializer syntax (which you did) or with help of fluent syntax. Have a look. Trying to write above query with the fluent syntax you will get something like
var searchResponse = await client.SearchAsync<Document>(s => s
.Size(0)
.Aggregations(a => a.Nested("by_country", n => n
.Path(p => p.Country)
.Aggregations(aa => aa
.Filter("by_country2", f => f
.Filter(q => q
.Term(t => t
.Field(field => field.Country.FirstOrDefault().IsDisplayed)
.Value(true)))
.Aggregations(aaa => aaa
.Terms("by_country3", t => t
.Field(field => field.Country.FirstOrDefault().DisplayName.Suffix("keyword"))
.Size(9999)
)))))));
which I find a little bit easier to follow and write, maybe it will be better for you as well.
As a final note, have a look into docs and check how you can debug your queries.
Hope that helps.
I want to use conditional query.
Here is my query
db.projects.aggregate([
{
"$group": {
"_id": "$iecode",
"treatmentArms": { "$first": "$evaluationDTOList" }
}
},
{ "$unwind": "$treatmentArms" },
{
"$group": {
"_id": null,
"Package": {
"$sum": {
"$cond": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Package" ] },
1, 0
]
}
},
"Constraint-relaxing mechanisms": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Mechanism" ] },
{ "$eq": [ "$treatmentArms.mechanismTested1", "Constraint-relaxing mechanisms" ] }
]
},
1,
0 ]
}
},
"Delivery mechanisms": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Mechanism" ] },
{ "$eq": [ "$treatmentArms.mechanismTested1", "Delivery mechanisms" ] }
]
},
1,
0 ]
}
},
"Other": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Mechanism" ] },
{ "$eq": [ "$treatmentArms.mechanismTested1", "Other" ] }
]
},
1,
0 ]
}
}
}
}
])
Here is my java code
DBObject groupByIECode = new BasicDBObject("$group",
new BasicDBObject("_id", new BasicDBObject("iecode","$iecode")).append("treatmentArms",new BasicDBObject("$first","$evaluationDTOList")));
System.out.println("groupByIECode: "+groupByIECode.toString());
DBObject unwind = new BasicDBObject("$unwind","$treatmentArms");
System.out.println("unwind: "+unwind.toString());
DBObject finalCalculation = new BasicDBObject("$group",new BasicDBObject("_id",null))
.append(
"Package", new BasicDBObject(
"$sum", new BasicDBObject(
"$cond", new Object[]{
new BasicDBObject(
"$eq", new Object[]{ "$treatmentArms.mechanismOrPkg", "Package"}
),
1,
0
}
)
)
);
System.out.println("finalCalculation: "+finalCalculation);
final AggregationOutput output = projects.aggregate(match,groupByIECode,unwind,finalCalculation);
It gives me MongoException$DuplicateKey
Later I found out that $cond operator is not supported in spring mongotemplate. So how do I implememt this conditional query with spring mongotemplate.
This link has some explanation but it do not shows full implementation
From the documentation, a canonical example for using the Spring Data MongoDB support for the MongoDB Aggregation Framework looks as follows:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation agg = newAggregation(
pipelineOP1(),
pipelineOP2(),
pipelineOPn()
);
AggregationResults<OutputType> results = mongoTemplate.aggregate(agg,
"INPUT_COLLECTION_NAME", OutputType.class);
List<OutputType> mappedResult = results.getMappedResults();
Note that if you provide an input class as the first parameter to the
newAggregation method the MongoTemplate will derive the name of the
input collection from this class. Otherwise if you don’t specify
an input class you must provide the name of the input collection
explicitly. If an input-class and an input-collection is provided the
latter takes precedence.
For your query, create a workaround that implements the AggregationOperation interface to take in a DBObject that represents a single group operation in an aggregation pipeline with the $cond operator:
public class GroupAggregationOperation implements AggregationOperation {
private DBObject operation;
public GroupAggregationOperation (DBObject operation) {
this.operation = operation;
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
Then implement the $group operation as a DBObject in the aggregation pipeline that is the same as the one you have:
DBObject operation = (DBObject) new BasicDBObject("$group", new BasicDBObject("_id", null))
.append(
"Package", new BasicDBObject(
"$sum", new BasicDBObject(
"$cond", new Object[]{
new BasicDBObject(
"$eq", new Object[]{ "$treatmentArms.mechanismOrPkg", "Package"}
),
1,
0
}
)
)
);
which you can then use as:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
GroupAggregationOperation groupOp = new GroupAggregationOperation(operation);
Aggregation agg = newAggregation(
group("iecode").first("treatmentArms").as("treatmentArms"),
unwind("treatmentArms"),
groupOp
);
AggregationResults<Entity> results = mongoTemplate.aggregate(agg, Entity.class);
List<Entity> entities = results.getMappedResults();
I have the following list of objects
List<Obj> source = new List<Obj>();
source.Add(new Obj() { Name = "o1", Attributes = new Dictionary<string, string> { { "attA", "1" }, { "attB", "1" }, { "attC", "1" } } });
source.Add(new Obj() { Name = "o2", Attributes = new Dictionary<string, string> { { "attA", "1" }, { "attB", "2" }, { "attC", "1" } } });
source.Add(new Obj() { Name = "o3", Attributes = new Dictionary<string, string> { { "attA", "1" }, { "attB", "3" }, { "attC", "2" } } });
source.Add(new Obj() { Name = "o4", Attributes = new Dictionary<string, string> { { "attA", "1" }, { "attB", "4" }, { "attC", "2" } } });
source.Add(new Obj() { Name = "o5", Attributes = new Dictionary<string, string> { { "attA", "2" }, { "attB", "5" }, { "attC", "3" } } });
source.Add(new Obj() { Name = "o6", Attributes = new Dictionary<string, string> { { "attA", "2" }, { "attB", "6" }, { "attC", "3" } } });
source.Add(new Obj() { Name = "o7", Attributes = new Dictionary<string, string> { { "attA", "2" }, { "attB", "7" }, { "attC", "4" } } });
source.Add(new Obj() { Name = "o8", Attributes = new Dictionary<string, string> { { "attA", "2" }, { "attB", "8" }, { "attC", "4" } } });
so i need to group it by the values of a specific attribute(s), furthermore the names of these attributes are kept in a separate list, like:
List<string> groupBy = new List<string>() { "attA", "attC" };
i tried using
var groups =
from s in source
group s by s.Attributes["attA"];
this works fine, returning 2 groups:
"1" - "o1 o2 o3 o4"
"2" - "o5 o6 o7 o8"
but what actually I need to do is to group by "attA" and "attC" (or whatever is in the groupBy variable) and get the following four groups:
"1_1" - "o1 o2"
"1_2" - "o3 o4"
"2_3" - "o4 o5"
"2_4" - "o7 o8"
from c in source
group c by String.Join("_",groupBy.Select(gr=>c.Attributes[gr]).ToArray()) into gr
select new
{
AttrValues = gr.Key,
//Values = gr.Key.Split('_'),
Names = gr.Select(c=>c.Name).ToList()
};
The group key is the concatenated projection of the dictionary values obtained from groupBy list of keys.
You can group by multiple properties:
var groups = from s in source
group s by new
{
AttributeA = s.Attributes["attA"],
AttributeC = s.Attributes["attC"]
};
//shows 4 groups
foreach (var group in groups)
Console.WriteLine(group.Key.AttributeA + "_" + group.Key.AttributeC);