I have a mongo query which is used to unwind four arrays of objects and filter the data based on the matching criteria. How do i perform the same operation in Spring data mongodb
I have worked with single unwind but could not find any with multiple unwind and match operation.
db.generator.aggregate([
{ $unwind :'$eCList'},
{ $unwind :'$pcList'},
{ $unwind :'$cityList'},
{ $unwind :'$eATypeList'},
{ $match : {'eCList.eCCode': { $eq : 'enccode1'} }},
{ $match : {'pcList.pcCode': { $eq : 'pccode1'} }},
{ $match : {'cityList.cityCode': { $eq : 'citycode1'} }},
{ $match : {'eATypeList.eATypeCode': { $eq : 'eATypeCode1'} }},
{ $project : {generatorData : '$generatorData',pcList:'$pcList',
generatorCode: '$generatorCode', eCId : '$eCList.eCId',
eCCode : '$eCList.eCCode', eCKey : '$eCList.eCKey' } }
])
You can try below aggregation in 1.10.x spring mongo version.
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.unwind("eCList"),
Aggregation.unwind("pcList"),
Aggregation.unwind("cityList"),
Aggregation.unwind("eATypeList"),
Aggregation.match(Criteria.where("eCList.eCCode").is("enccode1")),
Aggregation.match(Criteria.where("pcList.pcCode").is("pccode1")),
Aggregation.match(Criteria.where("cityList.cityCode").is("citycode1")),
Aggregation.match(Criteria.where("eATypeList.eATypeCode").is("eATypeCode1")),
Aggregation.project("generatorData", "pcList", "generatorCode").
andInclude(Aggregation.bind("eCId", "eCList.eCId")).
andInclude(Aggregation.bind("eCCode", "eCList.eCCode")).
andInclude(Aggregation.bind("eCKey", "eCList.eCKey"))
);
List<BasicDBObject> results = mongoTemplate.aggregate( aggregation, "generator", BasicDBObject.class).getMappedResults();
Generated Shell Query:
{
"aggregate":"__collection__",
"pipeline":[
{"$unwind":"$eCList"},
{"$unwind":"$pcList"},
{"$unwind":"$cityList"},
{"$unwind":"$eATypeList"},
{"$match":{"eCList.eCCode":"enccode1"}},
{"$match":{"pcList.pcCode":"pccode1"}},
{"$match":{"cityList.cityCode":"citycode1"}},
{"$match":{"eATypeList.eATypeCode":"eATypeCode1"}},
{"$project":{
"generatorData":1,
"pcList":1,
"generatorCode":1,
"eCId":"$eCList.eCId",
"eCCode":"$eCList.eCCode",
"eCKey":"$eCList.eCKey"}}
]
}
With static imports
import org.springframework.data.mongodb.core.query.Criteria;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation aggregation = newAggregation(
unwind("eCList"),
unwind("pcList"),
unwind("cityList"),
unwind("eATypeList"),
match(where("eCList.eCCode").is("enccode1")),
match(where("pcList.pcCode").is("pccode1")),
match(where("cityList.cityCode").is("citycode1")),
match(where("eATypeList.eATypeCode").is("eATypeCode1")),
project("generatorData", "pcList", "generatorCode").
andInclude(bind("eCId", "eCList.eCId")).
andInclude(bind("eCCode", "eCList.eCCode")).
andInclude(bind("eCKey", "eCList.eCKey"))
);
Related
{
"version": "2018-05-29",
"operation": "Query",
"index": "sportLeague",
"query": {
"expression": "sportIdLeagueId = :sportIdLeagueId and gsSK = :gsSK",
"expressionValues": {
#set( $sportIdLeagueId = $util.dynamodb.toDynamoDBJson("fixture#${ctx.args.input.sportId}#${ctx.args.input.leagueId}") )
":sportIdLeagueId": $sportIdLeagueId,
}
},
"filter" : {
"expression" : "between(#title, :title)",
"expressionNames" : {
"#startDateUtc" : "startDateUtc"
}
"expressionValues" : {
":start" : $util.dynamodb.toDynamoDBJson($ctx.args.input.startDateUtc)
}
}
}
i need a between condition to satisfy the requirement
requirement: get data between two dates i.e start date and end date
You attribute names and values need to match what is in your expression. (:title needs to have a corresponding value in expressionNames and expressionValues)
Also the between operator syntax is BETWEEN :startDateUtc AND :endDateUtc
There is a tutorial for using DynamoDB Query in AppSync:
https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-dynamodb-resolvers.html#setting-up-the-allpostsbyauthor-resolver-ddb-query
DynamoDB expressions:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html
I haven't defined any analyzers and assume it's using the standard analyzer
ES version : 5.6
Lucene_version : 6.6.1
Normal query string results in case-insensitive search but if the query string is put inside nested query it results in case sensitive search
Non-nested query string
GET articles/_search
{
query : {
bool : {
must : [ {
query_string : {
query : "*ellow*
}}]}
}
Nested query string doesn't return case insensitive matches
GET articles/_search { query : {
bool : {
must : [ {
nested : {
path : "comments",
query : {
query_string : {
query : "*ellow*"
}}}} ]} }
Any help is greatly appreciated.
Thanks
After asking question to understand a bit more of the aggregation framework in MongoDB I finally found the way to do aggregation for my need (thanks to a StackExchange user)
So basically here is a document from my collection:
{
"_id" : ObjectId("s4dcsd5s4d6c54s6d"),
"items" : [
{
type : "TYPE_1",
text : "blablabla"
},
{
type : "TYPE_2",
text : "blablabla"
},
{
type : "TYPE_3",
text : "blablabla"
},
{
type : "TYPE_1",
text : "blablabla"
},
{
type : "TYPE_2",
text : "blablabla"
},
{
type : "TYPE_1",
text : "blablabla"
}
]
}
The idea was to be able to filter only some elements of my collections (avoiding Type 2 and 3). In fact I have more than 30 types and 6 are not allowed but for simplicity I made this example.
So the aggregation command in command line is this one:
db.history.aggregate([{
$match: {
_id: ObjectId("s4dcsd5s4d6c54s6d")
}
}, {
$unwind: '$items'
}, {
$match: {
'items.type': { '$nin': [ "TYPE_2" , "TYPE_3"] }
}
},
{ $limit: 10 }
]);
With this I am able to retrieve the 10 elements items of this document which do not match TYPE_2 and TYPE_3
However when I am using spring data there is no output. I looked a bit at the example to build mine but its still not working.
So I did:
Aggregation aggregation = newAggregation(
match(Criteria.where("id").is(myID)),
unwind("items"),
match(Criteria.where("items.type").nin(ignoreditemstype)),
limit(3),
skip(offsetLong)
);
AggregationResults<PersonnalHistory> results = mongAccess.getOperation().aggregate(query,
"items", PersonnalHistory.class);
PersonnalHistory is marked with annotation #Document(collection = "history") and id with the #id annotation
ignoreditemstype is a list containing TYPE_2 and TYPE_3
Here is what I have in the toString method of aggregation:
{
"aggregate" : "__collection__" ,
"pipeline" : [
{ "$match": { "id" : "s4dcsd5s4d6c54s6d"} },
{ "$unwind": "$items"},
{ "$match": { "items.type": { "$nin" : [ "TYPE_2" , "TYPE_3" ] } } },
{ "$limit" : 3},
{ "$skip" : 0 }
]
}
I tried a lot of stuff (to have at least an answer :) ) like removing id or the nin:
aggregation = newAggregation(
unwind("items"),
match(Criteria.where("items.type").nin(ignoreditemstype)),
limit(3),
skip(offsetLong)
);
aggregation = newAggregation(
match(Criteria.where("id").is(myid)),
unwind("items")
);
For information when I do a simple query like:
query.addCriteria(Criteria.where("id").is(myID));
My document is returned. However I have thousands of items. So I just want to have the 15 first (in fact the 15 first are the 15 last added)
Do you maybe see what I am doing wrong?
Yeah looks like you are passing simple String while it is expecting ObjectId
Aggregation aggregation = newAggregation(
match(Criteria.where("_id").is(new ObjectId(myID))),
unwind("items"),
match(Criteria.where("items.type").nin(ignoreditemstype)),
limit(3),
skip(offsetLong)
);
Now the question is why it works with simple query, my answer would be because spring-data driver is not that mature at least not with aggregation pipeline.
I have collection in following format.
{
id:____,
name:"Carlos",
city:"Mumbai"
},
{
id:____,
name:"Pravin",
city:"Mumbai"
},
{
id:_____,
name:"Gaurav",
city:"Ahmedabad"
}
I want whole document distinct by city. I tried the db.collection.distinct("city"). But it returns only distinct cities.
Current Output:
["Mumbai","Ahmedabad"]
Expected Output:
{
id:____,
name:"Carlos",
city:"Mumbai"
},
{
id:_____,
name:"Gaurav",
city:"Ahmedabad"
}
Above you can see there is only one record of "Mumbai". I need this kind of output.
Anyone know how we can get whole document with distinct in spring-mongodb?
You could try running an aggregation pipeline operation where you can include the the other fields inside the $group pipeline stage using the $first operator. Two examples that show this approach follow:
Mongo Shell:
pipeline = [
{
"$group": {
"_id": "$city",
"id": { "$first": "$_id" },
"name": { "$first": "$name" }
}
}
]
db.collection.aggregate(pipeline);
Spring Data MongoDB:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
MongoTemplate mongoTemplate = repository.getMongoTemplate();
Aggregation agg = newAggregation(
group("city")
.first("_id").as("id")
.first("name").as("name")
);
AggregationResults<OutputType> result = mongoTemplate.aggregate(agg,
"collection", OutputType.class);
List<OutputType> mappedResult = result.getMappedResults();
I am trying to fetch documents which are greater or lesser than specified date.
I am using the below searchQuery for this purpose.
QueryBuilder queryBuilder = QueryBuilders.boolQuery()
.must(QueryBuilders.rangeQuery("date")
.gt("2015-06-25T00:00:00")
.lt("2015-06-25T00:00:00"));
The query generated from the above querybuilder is like this.
{
"bool" : {
"must" : [ {
"range" : {
"date" : {
"from" : "2015-06-25T00:00:00",
"to" : "2015-06-25T00:00:00",
"include_lower" : false,
"include_upper" : false
}
}
}
} ]
}
Even when i use functions gt and lt of rangequery the query is generated as from and to.
What is the solution so that a query can be generated like this.
{
"bool" : {
"must" : [ {
"range" : {
"date" : {
"gt" : "2015-06-25T00:00:00",
"lt" : "2015-06-25T00:00:00",
"include_lower" : false,
"include_upper" : false
}
}
}
} ]
}
This is the test class i have written.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { ElasticSearchConfiguration.class }, loader = AnnotationConfigContextLoader.class)
public class ElasticSearchTest {
#Autowired
private ElasticsearchTemplate elasticsearchTemplate;
#Autowired
private Client client;
#Test
public void testAggregation(){
QueryBuilder querybuilder = QueryBuilders.boolQuery().must(QueryBuilders.rangeQuery("receiptdate").gte("2015-06-25T00:00:00").lte ("2015-07-25T00:00:00")));
final SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(qb)
.build();
final List<Test> records = elasticsearchTemplate.queryForList(searchQuery, Test.class);
}
}
Any suggestions on how to achieve this in Spring Data Elasticsearch would be helpful.
Your query would not return any results, since you're looking for dates strictly greater and strictly lower than the same date. You need to use gte and lte instead:
QueryBuilder queryBuilder = QueryBuilders.boolQuery()
.must(QueryBuilders.rangeQuery("date")
.gte("2015-06-25T00:00:00")
.lte("2015-06-25T00:00:00"));
The official parameters of the range query are gt, gte, lt and lte.
The from, to, include_lower and include_upper parameters are old deprecated parameters, which the RangeQueryBuilder is still using but can (and will) be removed at anytime.
Just know that:
from + include_lower: false is equivalent to gt
from + include_lower: true is equivalent to gte
to + include_upper: false is equivalent to lt
to + include_upper: true is equivalent to lte