I want to use conditional query.
Here is my query
db.projects.aggregate([
{
"$group": {
"_id": "$iecode",
"treatmentArms": { "$first": "$evaluationDTOList" }
}
},
{ "$unwind": "$treatmentArms" },
{
"$group": {
"_id": null,
"Package": {
"$sum": {
"$cond": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Package" ] },
1, 0
]
}
},
"Constraint-relaxing mechanisms": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Mechanism" ] },
{ "$eq": [ "$treatmentArms.mechanismTested1", "Constraint-relaxing mechanisms" ] }
]
},
1,
0 ]
}
},
"Delivery mechanisms": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Mechanism" ] },
{ "$eq": [ "$treatmentArms.mechanismTested1", "Delivery mechanisms" ] }
]
},
1,
0 ]
}
},
"Other": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": [ "$treatmentArms.mechanismOrPkg", "Mechanism" ] },
{ "$eq": [ "$treatmentArms.mechanismTested1", "Other" ] }
]
},
1,
0 ]
}
}
}
}
])
Here is my java code
DBObject groupByIECode = new BasicDBObject("$group",
new BasicDBObject("_id", new BasicDBObject("iecode","$iecode")).append("treatmentArms",new BasicDBObject("$first","$evaluationDTOList")));
System.out.println("groupByIECode: "+groupByIECode.toString());
DBObject unwind = new BasicDBObject("$unwind","$treatmentArms");
System.out.println("unwind: "+unwind.toString());
DBObject finalCalculation = new BasicDBObject("$group",new BasicDBObject("_id",null))
.append(
"Package", new BasicDBObject(
"$sum", new BasicDBObject(
"$cond", new Object[]{
new BasicDBObject(
"$eq", new Object[]{ "$treatmentArms.mechanismOrPkg", "Package"}
),
1,
0
}
)
)
);
System.out.println("finalCalculation: "+finalCalculation);
final AggregationOutput output = projects.aggregate(match,groupByIECode,unwind,finalCalculation);
It gives me MongoException$DuplicateKey
Later I found out that $cond operator is not supported in spring mongotemplate. So how do I implememt this conditional query with spring mongotemplate.
This link has some explanation but it do not shows full implementation
From the documentation, a canonical example for using the Spring Data MongoDB support for the MongoDB Aggregation Framework looks as follows:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation agg = newAggregation(
pipelineOP1(),
pipelineOP2(),
pipelineOPn()
);
AggregationResults<OutputType> results = mongoTemplate.aggregate(agg,
"INPUT_COLLECTION_NAME", OutputType.class);
List<OutputType> mappedResult = results.getMappedResults();
Note that if you provide an input class as the first parameter to the
newAggregation method the MongoTemplate will derive the name of the
input collection from this class. Otherwise if you don’t specify
an input class you must provide the name of the input collection
explicitly. If an input-class and an input-collection is provided the
latter takes precedence.
For your query, create a workaround that implements the AggregationOperation interface to take in a DBObject that represents a single group operation in an aggregation pipeline with the $cond operator:
public class GroupAggregationOperation implements AggregationOperation {
private DBObject operation;
public GroupAggregationOperation (DBObject operation) {
this.operation = operation;
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
Then implement the $group operation as a DBObject in the aggregation pipeline that is the same as the one you have:
DBObject operation = (DBObject) new BasicDBObject("$group", new BasicDBObject("_id", null))
.append(
"Package", new BasicDBObject(
"$sum", new BasicDBObject(
"$cond", new Object[]{
new BasicDBObject(
"$eq", new Object[]{ "$treatmentArms.mechanismOrPkg", "Package"}
),
1,
0
}
)
)
);
which you can then use as:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
GroupAggregationOperation groupOp = new GroupAggregationOperation(operation);
Aggregation agg = newAggregation(
group("iecode").first("treatmentArms").as("treatmentArms"),
unwind("treatmentArms"),
groupOp
);
AggregationResults<Entity> results = mongoTemplate.aggregate(agg, Entity.class);
List<Entity> entities = results.getMappedResults();
Related
I have the following JS code:
$concatArrays: [
"$some.path",
[
{
"foo": "baz",
"x": 5
}
]
]
How do I do the same with ArrayOperators.ConcatArrays?
Toy can use like following. You can pass the aggregation expresion or reference field name in concat(). So what you can do is, you can add the new array before concat() like
db.collection.aggregate([
{
$addFields: {
secondArray: [
{ "foo": "baz", "x": 5 }
]
}
},
{
"$project": {
combined: {
"$concatArrays": [ "$path", "$secondArray" ]
}
}
}
])
Working Mongo playground
ArrayOperators.ConcatArrays concatArrays = ArrayOperators.ConcatArrays
.arrayOf("path")
.concat("secondArray");
Assuming the array field is defined as follows in a document:
{
"_id": ObjectId("60bdc6b228acee4a5a6e1736"),
"some" : {
"path" : [
{
"foo" : "bar",
"x" : 99
}
]
},
}
The following Spring Data MongoDB code returns the result- an array with two elements. The array field "some.path" and the input array myArr are concatenated using the ArrayOperators.ConcatArrays API.
String json = "{ 'foo': 'baz', 'x': 5 }";
Document doc = Document.parse(json);
List<Document> myArr = Arrays.asList(doc);
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "testDB");
Aggregation agg = newAggregation(
addFields()
.addFieldWithValue("myArr", myArr)
.build(),
addFields()
.addField("result").withValue(ConcatArrays.arrayOf("$some.path").concat("$myArr"))
.build()
);
AggregationResults<Document> results = mongoOps.aggregate(agg, "testColl", Document.class);
i have this mongo query:
db.getCollection('My_collection_name').aggregate([
{ $project: { warehouses: { $objectToArray: "$outputVariables" } } },
{ $unwind: "$warehouses" },
{ $group: { _id: "$warehouses.k" }}
])
someone could help me to translate in spring mongoTemplate?
Thanks
The query should be write as below:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
...
AggregationOperation project = project()
.and(ObjectToArray.valueOfToArray("outputVariables")).as("warehouses");
AggregationOperation unwind = unwind("warehouses");
AggregationOperation group = Aggregation.group("warehouses.k");
Aggregation aggregation = Aggregation.newAggregation(
project,
unwind,
group);
String collectionName = "My_collection_name";
System.out.println("aggregation=" + aggregation);
this.mongoTemplate.aggregate(aggregation, collectionName, Output.class);
It generate output:
aggregation={ "aggregate" : "__collection__", "pipeline" : [{ "$project" : { "warehouses" : { "$objectToArray" : "$outputVariables" } } }, { "$unwind" : "$warehouses" }, { "$group" : { "_id" : "$warehouses.k" } }] }
I need help in writing mongodb aggregation pipeline in spring.
Here is a sample of my data on which I want to execute the query (Please note that the following data is a result of Match aggregation pipeline in spring which has conditions for the eventDate to match the range and eventName equals to 'A'):
{
"_id": ObjectId("1234567890"),
"eventDate": ISODate("2017-01-29T03:56:04.297Z"),
"eventName": "A"
}
{
"_id": ObjectId("1234567890"),
"eventDate": ISODate("2017-01-29T03:57:04.887Z"),
"eventName": "A"
}
{
"_id": ObjectId("1234567890"),
"eventDate": ISODate("2017-01-29T04:05:00.497Z"),
"eventName": "A"
}
Now I want to write the Group aggregation pipeline on eventDate field where it should group on Year,Month, Day and Hour because what I'm looking for is hourly data.
Sample of the result I want it to look like which projects only eventDate and eventName fields:
{
"eventDate": ISODate("2017-01-29T03:00:00.000Z"),
"eventName": "A"
}
{
"eventDate": ISODate("2017-01-29T04:00:00.000Z"),
"eventName": "A"
}
So if you notice above that the data is in a group of hours and apart from that Minutes and seconds are now set to 0.
I tried this solution but didn't work for me:
MongoTemplate aggregate - group by date
Any kind of help appreciated. Thanks in advance.
You can use date arithmetic here to get the date time with minute, second and milliseconds part reset to 0.
You may need to make some adjustment based on your spring mongo version and java version.
Version : Mongo 3.2, Spring Mongo 1.9.5 Release and Java 8
Mongo Shell Query:
db.collection.aggregate(
[{
$group: {
_id: {
$subtract: ["$eventDate", {
$add: [{
$multiply: [{
$minute: "$eventDate"
}, 60000]
}, {
$multiply: [{
$second: "$eventDate"
}, 1000]
}, {
$millisecond: "$eventDate"
}]
}]
},
eventName: {
$addToSet: "$eventName"
}
}
}]
)
Spring Code:
AggregationOperation group = context -> context.getMappedObject(new BasicDBObject(
"$group", new BasicDBObject(
"_id",
new BasicDBObject(
"$subtract",
new Object[]{
"$eventDate",
new BasicDBObject("$add",
new Object[]{
new BasicDBObject("$multiply",
new Object[]{new BasicDBObject("$minute", "$eventDate"), 60000}),
new BasicDBObject("$multiply",
new Object[]{new BasicDBObject("$second", "$eventDate"), 1000}),
new BasicDBObject("$millisecond", "$eventDate")
}
)
}
)
).append("eventName", new BasicDBObject("$addToSet", "$eventName"))));
Update: Using $project
db.collection.aggregate(
[{
$project: {
eventDate: {
$subtract: ["$eventDate", {
$add: [{
$multiply: [{
$minute: "$eventDate"
}, 60000]
}, {
$multiply: [{
$second: "$eventDate"
}, 1000]
}, {
$millisecond: "$eventDate"
}]
}]
},
eventName: 1
}
}, {
$group: {
_id: "$eventDate",
eventName: {
$addToSet: "$eventName"
}
}
}]
)
Spring Code
AggregationOperation project = context -> context.getMappedObject(new BasicDBObject(
"$project", new BasicDBObject(
"eventDate",
new BasicDBObject(
"$subtract",
new Object[]{
"$eventDate",
new BasicDBObject("$add",
new Object[]{
new BasicDBObject("$multiply",
new Object[]{new BasicDBObject("$minute", "$eventDate"), 60000}),
new BasicDBObject("$multiply",
new Object[]{new BasicDBObject("$second", "$eventDate"), 1000}),
new BasicDBObject("$millisecond", "$eventDate")
}
)
}
)
).append("eventName", 1)));
AggregationOperation group = Aggregation.group("eventDate").addToSet("eventName").as("eventName");
Aggregation agg = Aggregation.newAggregation(project, group);
it seems that SpringData ES don't provide classes to fetch highlights returned by ES. Spring Data can return Lists of Objects but the highlights sections in the Json returned by ES is in a separated part that is not handled by the "ElasticSearchTemplate" class.
Code example :-
QueryBuilder query = QueryBuilders.matchQuery("name","tom");
SearchQuery searchQuery =new NativeSearchQueryBuilder().withQuery(query).
with HighlightFields(new Field("name")).build();
List<ESDocument> publications = elasticsearchTemplate.queryForList
(searchQuery, ESDocument.class);
I might be wrong, but I can't figure out to do only with SpringDataES. Someone can post an example of how we can get highlights with Spring Data ES ?
Thanks in advance !
From the test cases in spring data elasticsearch I've found solution to this :
This can be helpful.
#Test
public void shouldReturnHighlightedFieldsForGivenQueryAndFields() {
//given
String documentId = randomNumeric(5);
String actualMessage = "some test message";
String highlightedMessage = "some <em>test</em> message";
SampleEntity sampleEntity = SampleEntity.builder().id(documentId)
.message(actualMessage)
.version(System.currentTimeMillis()).build();
IndexQuery indexQuery = getIndexQuery(sampleEntity);
elasticsearchTemplate.index(indexQuery);
elasticsearchTemplate.refresh(SampleEntity.class);
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(termQuery("message", "test"))
.withHighlightFields(new HighlightBuilder.Field("message"))
.build();
Page<SampleEntity> sampleEntities = elasticsearchTemplate.queryForPage(searchQuery, SampleEntity.class, new SearchResultMapper() {
#Override
public <T> Page<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
List<SampleEntity> chunk = new ArrayList<SampleEntity>();
for (SearchHit searchHit : response.getHits()) {
if (response.getHits().getHits().length <= 0) {
return null;
}
SampleEntity user = new SampleEntity();
user.setId(searchHit.getId());
user.setMessage((String) searchHit.getSource().get("message"));
user.setHighlightedMessage(searchHit.getHighlightFields().get("message").fragments()[0].toString());
chunk.add(user);
}
if (chunk.size() > 0) {
return new PageImpl<T>((List<T>) chunk);
}
return null;
}
});
assertThat(sampleEntities.getContent().get(0).getHighlightedMessage(), is(highlightedMessage));
}
Spring Data Elasticsearch 4.0 now has the SearchPage result type, which makes things a little easier if we need to return highlighted results:
This is a working sample:
String query = "(id:123 OR id:456) AND (database:UCLF) AND (services:(sealer?), services:electronic*)"
NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
.withPageable(pageable)
.withQuery(queryStringQuery(query))
.withSourceFilter(sourceFilter)
.withHighlightFields(new HighlightBuilder.Field("goodsAndServices"))
.build();
SearchHits<Trademark> searchHits = template.search(searchQuery, Trademark.class, IndexCoordinates.of("trademark"));
SearchPage<Trademark> page = SearchHitSupport.searchPageFor(searchHits, searchQuery.getPageable());
return (Page<Trademark>) SearchHitSupport.unwrapSearchHits(page);
And this would be the response from Page object in json:
{
"content": [
{
"id": "123",
"score": 12.10748,
"sortValues": [],
"content": {
"_id": "1P0XzXIBdRyrchmFplEA",
"trademarkIdentifier": "abc234",
"goodsAndServices": null,
"language": "EN",
"niceClass": "2",
"sequence": null,
"database": "UCLF",
"taggedResult": null
},
"highlightFields": {
"goodsAndServices": [
"VARNISHES, <em>SEALERS</em>, AND NATURAL WOOD FINISHES"
]
}
}
],
"pageable": {
"sort": {
"unsorted": true,
"sorted": false,
"empty": true
},
"offset": 0,
"pageNumber": 0,
"pageSize": 20,
"unpaged": false,
"paged": true
},
"searchHits": {
"totalHits": 1,
"totalHitsRelation": "EQUAL_TO",
"maxScore": 12.10748,
"scrollId": null,
"searchHits": [
{
"id": "123",
"score": 12.10748,
"sortValues": [],
"content": {
"_id": "1P0XzXIBdRyrchmFplEA",
"trademarkIdentifier": "abc234",
"goodsAndServices": null,
"language": "EN",
"niceClass": "2",
"sequence": null,
"database": "UCLF",
"taggedResult": null
},
"highlightFields": {
"goodsAndServices": [
"VARNISHES, <em>SEALERS</em>, AND NATURAL WOOD FINISHES"
]
}
}
],
"aggregations": null,
"empty": false
},
"totalPages": 1,
"totalElements": 1,
"size": 20,
"number": 0,
"numberOfElements": 1,
"last": true,
"first": true,
"sort": {
"unsorted": true,
"sorted": false,
"empty": true
},
"empty": false
}
Actually, you could do the following, with a custom ResultExtractor:
QueryBuilder query = QueryBuilders.matchQuery("name", "tom");
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(query)
.withHighlightFields(new Field("name")).build();
return elasticsearchTemplate.query(searchQuery.build(), new CustomResultExtractor());
And then
public class CustomResultExtractor implements ResultsExtractor<List<MyClass>> {
private final DefaultEntityMapper defaultEntityMapper;
public CustomResultExtractor() {
defaultEntityMapper = new DefaultEntityMapper();
}
#Override
public List<MyClass> extract(SearchResponse response) {
return StreamSupport.stream(response.getHits().spliterator(), false)
.map(this::searchHitToMyClass)
.collect(Collectors.toList());
}
private MyClass searchHitToMyClass(SearchHit searchHit) {
MyElasticSearchObject myObject;
try {
myObject = defaultEntityMapper.mapToObject(searchHit.getSourceAsString(), MyElasticSearchObject.class);
} catch (IOException e) {
throw new ElasticsearchException("failed to map source [ " + searchHit.getSourceAsString() + "] to class " + MyElasticSearchObject.class.getSimpleName(), e);
}
List<String> highlights = searchHit.getHighlightFields().values()
.stream()
.flatMap(highlightField -> Arrays.stream(highlightField.fragments()))
.map(Text::string)
.collect(Collectors.toList());
// Or whatever you want to do with the highlights
return new MyClass(myObject, highlights);
}}
Note that I used a list but you could use any other iterable data structure. Also, you could do something else with the highlights. Here I'm simply listing them.
https://stackoverflow.com/a/37163711/6643675
The first answer does works,but I found some pageable problems with its returned result,which display with the wrong total elements and toalpages.Arter I checkout the DefaultResultMapper implementation, the returned statement shoud be return new AggregatedPageImpl((List<T>) chunk, pageable, totalHits, response.getAggregations(), response.getScrollId(), maxScore);,and then it works with paging.wish i could help you guys~
I'm going crazy with this one...
I have this aggregation framework expression working like a charm in mongo shell:
{ $group :
{
_id : '$Code' ,
'Special' : { $sum : { $cond: [{ $eq: [ '$Special', 'Success']},1,0]}}
}
}
I need to do it in c#, I tried a lot of combinations but without success.
Has anyone any clue?
Thx
Give this a try:
var group = new BsonDocument
{
{
"$group",
new BsonDocument
{
{
"_id", "$Code"
},
{
"Special", new BsonDocument
{
{ "$sum", new BsonDocument
{
{"$cond", new BsonArray
{
new BsonDocument
{
{
"$eq", new BsonArray {"$Special", "Success"}
}
},
1,
0
}
}
}
}
}
}
}
}
};