I would like to run a bulk delete operation on a list of documents in MongoDB that have been selected by the user in the UI so I need to dynamically build a query that looks like the following (the or clause expands for every document selected):
{
$and: [
{
"contentType": "application/vnd.sometype"
},
{
$or: [
{
"metadata.name": "someName",
"metadata.version": "someVersion"
},
{
"metadata.name": "someOtherName",
"metadata.version": "someOtherVersion"
}
]
}
]
},
Fields: null,
Sort: null
Just now I'm using string concatenation to achieve this.
Is it possible to build this query with the Spring Data MongoDB Criteria Builder (org.springframework.data.mongodb.core.query.Criteria)?
Doesn't this work for you?
Criteria criteria = Criteria.where("contentType").is("application/vnd.sometype");
List<Criteria> docCriterias = new ArrayList<Criteria>(docs.size());
for (Document doc: docs) {
docCriterias.add(Criteria.where("metadata.name").is(doc.getName())
.and("metadata.version").is(doc.getVersion()));
}
criteria = criteria.orOperator(docCriterias.toArray(new Criteria[docs.size()]));
?
Here we need to build new query and embed the criteria to the built new query. And also, we have to create a list of criteria using some criteria for embed to the query. Here my example is providing a list of metadata and we don't know the name of parameter which will send for us. So, The solution is as given follow.
List<Criteria> criterias = new ArrayList<>();
for (MetaData metaData : newDoc.getMetaData()) {
Criteria dynamicCriteria = Criteria.where("metaData.key").is(metaData.getKey()).andOperator(Criteria.where("metaData.value").is(metaData.getValue()));
criterias.add(dynamicCriteria);
}
Criteria criteria = new Criteria().andOperator(criterias.toArray(new Criteria[criterias.size()]));
Query searchQuery = new Query(criteria);
List<Document> documents = mongoTemplate.find(searchQuery, Document.class);
Related
If I have a Documents defined as
[
{
"title": "title",
"tags": [ "cool", "amazing", "funny" ]
},
{
"title": "another title",
"tags": [ "nice", "amazing", "funny" ]
}
]
I'd like to be able to query with MongoTemplate in order to pass a list of values like ["cool","amazing"] and have in return the first collection above, not the second.
For what I mean to achieve, the $in condition doesn't seems enough.
I tried with the $all condition and from the Mongo console it works as I need, but in my code something isn't working and it takes forever for my query to elaborate. With the $in operator my code goes fast, instead.
My method in my repository (for other reasons, I have to do an aggregation as below):
public Page<MyDocument> findByProperties(String title, List<ObjectId> tags, Pageable page) {
final List<Criteria> criteria = new ArrayList<>();
Aggregation aggregation = null;
if (title != null && !title.isEmpty()) {
criteria.add(Criteria.where("title").is(title));
}
if (tags != null && !tags.isEmpty()) {
criteria.add(Criteria.where("MyBook.tags").all(tags));
}
if (!criteria.isEmpty()) {
aggregation = Aggregation.newAggregation(
Aggregation.lookup("from_collection", "_id", "idParent", "MyBook"),
Aggregation.unwind("MyBook"),
Aggregation.match(new Criteria().andOperator(criteria.toArray(new Criteria[0]))),
Aggregation.skip(page.getOffset()),
Aggregation.limit(page.getPageSize()),
Aggregation.sort(page.getSort())
);
} else {
aggregation = Aggregation.newAggregation(
Aggregation.lookup("from_collection", "_id", "idParent", "MyBook"),
Aggregation.unwind("MyBook"),
Aggregation.skip(page.getOffset()),
Aggregation.limit(page.getPageSize()),
Aggregation.sort(page.getSort())
);
}
List<MyDocument> results = mongoTemplate.aggregate(aggregation, "my_document", MyDocument.class).getMappedResults();
return PageableExecutionUtils.getPage(results, page,
() -> (long)results.size());
}
Looking at this answer, I tried with
criteria.add(Criteria.where("MyBook.tags").in(tags).all(tags));
But nothing changed, the query takes forever and not with the expected output.
Any ideas please? Thank you!
To find if tags array field contains all the members of youe array, your can use the bellow, if you need it to be in aggregation pipeline use the second.
In your query you have more stages, and more code, if you want to ask more if you can give example data and expected output in JSON, and what you want to do, so people can help.
Query1
find using $all operator
Test code here
db.collection.find({"tags": {"$all": ["cool","amazing"]}})
Query2
aggregation solution with set difference
Test code here
aggregate(
[{"$match":
{"$expr":
{"$eq": [{"$setDifference": [["cool", "amazing"], "$tags"]}, []]}}}])
We are using native search query builder as follows:
String[] includeFields = new String[]{"idDl", "clientName"};
String[] excludeFields = new String[]{"Address"};
Query searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withSourceFilter(new FetchSourceFilter(includeFields, excludeFields))
.build();
return elasticsearchRestTemplate.queryForObject((StringQuery) searchQuery, User.class);
I am able to get all the response data but first response object is all null fields, i want to exclude all null fields object in the final response. We are using we're using spring-data-elasticsearch 3.2.6.RELEASE and here is the sample response:
[{
"idDl": null,
"clientName": null,
"Address": null
},
{
"idDl": 19810008,
"clientName": "ABC",
"Address": "NYC"
}]
What you should do is to create a query that will exclude the documents with those fields having null values instead of doing a match_all. Change your query with this instead:
...
.withQuery(existsQuery("idDl"))
...
Also worth noting that source filtering alone will not look at the values of your fields to check if they are null or not, it simply returns the field by name if it's present in the source document.
I have articles & tags collection. Articles contain tags which is array of objectId. I want to fetch tagName as well, so I unwind (this gives me multiple rows - 1 per tag array entry) => lookup (joins with tabs collection) => group (combine it into original result set)
My mongodb query is as follows, which gives me correct result:
db.articles.aggregate([
{"$unwind": "$tags"},
{
"$lookup": {
"localField": "tags",
"from": "tags",
"foreignField": "_id",
"as": "materialTags"
}
},
{
"$group": {
"_id": "$_id",
"title": {"$first": "$title"},
"materialTags": {"$push": "$materialTags"}
}
}
])
My corresponding Spring code:
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
//I also want to add group operation but unable to find the proper syntax ??.
Aggregation aggregation = Aggregation.newAggregation(unwindOperation,
lookupOperation1, ??groupOperation?? );
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
I tried to play around with group operation but without much luck. How can I add group operations as per original query ?
Thanks in advance.
Group query syntax in Spring for
{
"$group": {
"_id": "$_id",
"title": {"$first": "$title"},
"materialTags": {"$push": "$materialTags"}
}
}
is
Aggregation.group("_id").first("title").as("title").push("materialTags").as("materialTags")
Final query
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
Aggregation aggregation = Aggregation.newAggregation(unwindOperation,
lookupOperation1, Aggregation.group("_id").first("title").as("title").push("materialTags").as("materialTags") );
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
To get more info please go thru the below references
http://www.baeldung.com/spring-data-mongodb-projections-aggregations
spring data mongodb group by
Create Spring Data Aggregation from MongoDb aggregation query
https://www.javacodegeeks.com/2016/04/data-aggregation-spring-data-mongodb-spring-boot.html
I have a collection with documents of the following structure:
{
"category": "movies",
"movies": [
{
"name": "HarryPotter",
"language": "english"
},
{
"name": "Fana",
"language": "hindi"
}
]
}
I want to query with movie name="fana" and the response sholud be
{
"category": "movies",
"movies": [
{
"name": "HarryPotter",
"language": "english"
}
]
}
How do I get the above using spring mongoTemplate?
You can try something like this.
Non-Aggregation based approach:
public MovieCollection getMoviesByName() {
BasicDBObject fields = new BasicDBObject("category", 1).append("movies", new BasicDBObject("$elemMatch", new BasicDBObject("name", "Fana").append("size", new BasicDBObject("$lt", 3))));
BasicQuery query = new BasicQuery(new BasicDBObject(), fields);
MovieCollection groupResults = mongoTemplate.findOne(query, MovieCollection.class);
return groupResults;
}
Aggregation based approach:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
import static org.springframework.data.mongodb.core.query.Criteria.where;
public List<BasicDBObject> getMoviesByName() {
Aggregation aggregation = newAggregation(unwind("movies"), match(where("movies.name").is("Fana").and("movies.size").lt(1)),
project(fields().and("category", "$category").and("movies", "$movies")));
AggregationResults<BasicDBObject> groupResults = mongoTemplate.aggregate(
aggregation, "movieCollection", BasicDBObject.class);
return groupResults.getMappedResults();
}
$unwind of mongodb aggregation can be used for this.
db.Collection.aggregate([{
{$unwind : 'movies'},
{$match :{'movies.name' : 'fana'}}
}])
You can try the above query to get required output.
Above approaches provides you a solution using aggregation and basic query. But if you dont want to use BasicObject below code will perfectly work:
Query query = new Query()
query.fields().elemMatch("movies", Criteria.where("name").is("Fana"));
List<Movies> movies = mongoTemplate.find(query, Movies.class);
The drawback of this query is that it may return duplicate results present in different documents, since more than 1 document may match this criteria. So you can add _id in the criteria like below:
Criteria criteria = Criteria.where('_id').is(movieId)
Query query = new Query().addCriteria(criteria)
query.fields().elemMatch("movies", Criteria.where("name").is("Fana"));
query.fields().exclude('_id')
List<Movies> movies = mongoTemplate.find(query, Movies.class);
I am excluding "_id" of the document in the response.
I'm trying to query pricing stats on products I am recording in my Elasticsearch Database by product number. The pricing may be for new, used or refurbished products, so I wish to filter on condition. The condition filter works as a JSON query in Marvel returning stats based on two price documents with condition new.
When I try to do similar using the Java API, I am getting stats based on 4 documents that includes 2 new and 2 refurbished.
Could anyone please identify what I am doing wrong in the Java code below?
Thanks.
Here's the working JSON Query:
GET /stats/price/_search
{
"query": {
"match_phrase": {"mpc": "MGTX2LL/A"}
},
"size": 0,
"aggs" : {
"low_price_stats" : {
"filter": {
"term" : { "condition" : "new"}
},
"aggs" : {
"price_stats" : { "extended_stats" : { "field" : "price" } }
}
}
}
}
And the problematic Java:
public Aggregations aggByManufacturerPartNumber(String mpn) {
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withIndices("stats")
.withTypes("price")
.withQuery(termQuery("mpn", mpn))
.withFilter(
FilterBuilders.termFilter("condition", "New")
)
.addAggregation(AggregationBuilders.extendedStats("stats_agg").field("price"))
.build();
Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
#Override
public Aggregations extract(SearchResponse response) {
return response.getAggregations();
}
});
return aggregations;
}
In your Java code you're only building the price_stats sub-aggregation without its parent filter aggregation. The call to withFilter will create a filter at the query level, not at the aggregation level. The correct Java code that matches your JSON query would be like this:
// build top-level filter aggregation
FilterAggregationBuilder lowPriceStatsAgg = AggregationBuilders.filter("low_price_stats")
.filter(FilterBuilders.termFilter("condition", "new"));
// build extended stats sub-aggregation
lowPriceStatsAgg.subAggregation(AggregationBuilders.extendedStats("stats_agg").field("price"));
// build query
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withIndices("stats")
.withTypes("price")
.withQuery(termQuery("mpn", mpn))
.addAggregation(lowPriceStatsAgg)
.build();
// then get the results
Aggregations aggs = response.getAggregations();
Filter lowPriceStats = aggs.get("low_price_stats");
ExtendedStats statsAgg = lowPriceStats.get("stats_agg");
Besides, also note that in your JSON query you have a match_phrase on the mpc field while in your Java code you have a term query on the mpn field. So you probably need to fix that, too, but the above code fixes the aggregation part only.