I am trying to get Aggregation results from ElasticSearch index
For exmaple , values in my index
"_source": {
"ctry": "abc",
"totalentry": 1,
"entrydate": "2022-01-06"
},
"_source": {
"ctry": "abc",
"totalentry": 3,
"entrydate": "2022-01-07"
},
"_source": {
"ctry": "xyz",
"totalentry": 1,
"entrydate": "2022-01-08"
}
expected Results should be get totalentry based on country
ctry : abc
totalentry : 4
ctry : xyz
totalentry : 1
My Aggreagtion query
QueryBuilder querybuilder = QueryBuilders.boolQuery().must(QueryBuilders.rangeQuery("entrydate")
.gte("2022-01-01").lte ("2022-01-31"));
TermsAggregationBuilder groupBy = AggregationBuilders.terms("ctry").field("ctry");
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(querybuilder).addAggregation(groupBy)
.build();
List<Sample> records = elasticsearchRestTemplate.queryForList(searchQuery, Sample.class);
Above aggregation query returning 3 records instead of 2 aggregated results.
My index properties
"ctry": {
"type": "keyword"
How to change it to below , so that i hope i will get correct aggregation results
ctry": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
My java code
#Document(indexName="sample", createIndex=true, shards = 4)
public class Sample {
#Field(type = FieldType.Keyword)
private String ctry;
You are using an outdated version of Spring Data Elasticsearch. The queryForList variants were deprecated in 4.0 and have been removed in 4.2.
You need to use one of the search...() methods that return a SearchHits<Sample>> object. That will contain the documents for your query and the aggregations.
Related
I have the following aggregation pipeline to calculate the top rated brands from a collection of phones with their reviews embedded.
public Document findTopRatedBrands(int minReviews, int results) {
UnwindOperation unwindOperation = unwind("reviews");
GroupOperation groupOperation = group("$brand").avg("$reviews.rating")
.as("avgRating").count().as("numReviews");
MatchOperation matchOperation = match(new Criteria("numReviews").gte(minReviews));
SortOperation sortOperation = sort(Sort.by(Sort.Direction.DESC, "avgRating",
"numReviews"));
LimitOperation limitOperation = limit(results);
ProjectionOperation projectionOperation = project().andExpression("_id").as
("brand").andExpression("avgRating").as("rating").andExclude("_id")
.andExpression("numReviews").as("reviews");
Aggregation aggregation = newAggregation(unwindOperation, groupOperation, matchOperation,
sortOperation, limitOperation, projectionOperation);
AggregationResults<Phone> result = mongoOperations
.aggregate(aggregation, "phones", Phone.class);
return result.getRawResults();
}
An example of document in the phones collection is this:
{
"_id": {
"$oid": "61e1cc8f452d0aef89d9125f"
},
"brand": "Samsung",
"name": "Samsung Galaxy S7",
"releaseYear": 2016,
"reviews": [{
"_id": {
"$oid": "61d4403b86913bee0245c171"
},
"rating": 2,
"dateOfReview": {
"$date": "2019-12-24T00:00:00.000Z"
},
"title": "Won't do that again.",
"body": "I could not use with my carrier. Sent it back.",
"username": "bigrabbit324"
}]
}
I would like to sort by avgRating (rounded to the first decimal place), and secondly by the number of reviews. Now the average rating it's not rounded so it gives always different values, so I can't sort by number of reviews also. I have seen the ArithmeticOperators.Round class but I don't understand how to include it here if possible.
An example of result is the following:
[Document{{brand=Nokia, rating=3.25, reviews=4}}]
I would like to have 3.2 as rating.
This works in Mongo Compass:
$project: {
_id: 0,
brand: '$_id',
rating: { $round: ['$avgRating', 1] }
}
try
ProjectionOperation roundAverageRating = Aggregation.project("avgRating", "numReviews")
.and(ArithmeticOperators.Round.roundValueOf("avgRating").place(1))
.as("avgRatingRounded");
I am looking for any examples that implement the parent-child relationship using the python interface.
I can define a mapping such as
es.indices.create(
index= "docpage",
body= {
"mappings": {
"properties": {
"my_join_field": {
"type": "join",
"relations": {
"my_document": "my_page"
}
}
}
}
}
)
I am then indexing a document using
res = es.index(index="docpage",doc_type="_doc",id = 1, body=jsonDict) ,
where jsonDict is a dict structure of document's text,
jsonDict['my_join_field']= 'my_document', and other relevant info.
Reference example.
I tried adding pageDict where the page is a string containing text on a page in a document, and
pageDict['content']=page
pageDict['my_join_field']={}
pageDict['my_join_field']['parent']="1"
pageDict['my_join_field']['name']="page"
res = es.index(index="docpage",doc_type="_doc",body=pageDict)
but I get a parser error:
RequestError(400, 'mapper_parsing_exception', 'failed to parse')
Any ideas?
This worked for me :
res=es.index(index="docpage",doc_type="_doc",body={"content":page,
"my-join-field":{
"name": "my_page",
"parent": "1"}
})
The initial syntax can work if the parent is also repeated in the "routing" key of the main query body:
res = es.index(index="docpage",doc_type="_doc",body=pageDict, routing=1)
I have articles & tags collection. Articles contain tags which is array of objectId. I want to fetch tagName as well, so I unwind (this gives me multiple rows - 1 per tag array entry) => lookup (joins with tabs collection) => group (combine it into original result set)
My mongodb query is as follows, which gives me correct result:
db.articles.aggregate([
{"$unwind": "$tags"},
{
"$lookup": {
"localField": "tags",
"from": "tags",
"foreignField": "_id",
"as": "materialTags"
}
},
{
"$group": {
"_id": "$_id",
"title": {"$first": "$title"},
"materialTags": {"$push": "$materialTags"}
}
}
])
My corresponding Spring code:
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
//I also want to add group operation but unable to find the proper syntax ??.
Aggregation aggregation = Aggregation.newAggregation(unwindOperation,
lookupOperation1, ??groupOperation?? );
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
I tried to play around with group operation but without much luck. How can I add group operations as per original query ?
Thanks in advance.
Group query syntax in Spring for
{
"$group": {
"_id": "$_id",
"title": {"$first": "$title"},
"materialTags": {"$push": "$materialTags"}
}
}
is
Aggregation.group("_id").first("title").as("title").push("materialTags").as("materialTags")
Final query
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
Aggregation aggregation = Aggregation.newAggregation(unwindOperation,
lookupOperation1, Aggregation.group("_id").first("title").as("title").push("materialTags").as("materialTags") );
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
To get more info please go thru the below references
http://www.baeldung.com/spring-data-mongodb-projections-aggregations
spring data mongodb group by
Create Spring Data Aggregation from MongoDb aggregation query
https://www.javacodegeeks.com/2016/04/data-aggregation-spring-data-mongodb-spring-boot.html
I have a collection with documents of the following structure:
{
"category": "movies",
"movies": [
{
"name": "HarryPotter",
"language": "english"
},
{
"name": "Fana",
"language": "hindi"
}
]
}
I want to query with movie name="fana" and the response sholud be
{
"category": "movies",
"movies": [
{
"name": "HarryPotter",
"language": "english"
}
]
}
How do I get the above using spring mongoTemplate?
You can try something like this.
Non-Aggregation based approach:
public MovieCollection getMoviesByName() {
BasicDBObject fields = new BasicDBObject("category", 1).append("movies", new BasicDBObject("$elemMatch", new BasicDBObject("name", "Fana").append("size", new BasicDBObject("$lt", 3))));
BasicQuery query = new BasicQuery(new BasicDBObject(), fields);
MovieCollection groupResults = mongoTemplate.findOne(query, MovieCollection.class);
return groupResults;
}
Aggregation based approach:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
import static org.springframework.data.mongodb.core.query.Criteria.where;
public List<BasicDBObject> getMoviesByName() {
Aggregation aggregation = newAggregation(unwind("movies"), match(where("movies.name").is("Fana").and("movies.size").lt(1)),
project(fields().and("category", "$category").and("movies", "$movies")));
AggregationResults<BasicDBObject> groupResults = mongoTemplate.aggregate(
aggregation, "movieCollection", BasicDBObject.class);
return groupResults.getMappedResults();
}
$unwind of mongodb aggregation can be used for this.
db.Collection.aggregate([{
{$unwind : 'movies'},
{$match :{'movies.name' : 'fana'}}
}])
You can try the above query to get required output.
Above approaches provides you a solution using aggregation and basic query. But if you dont want to use BasicObject below code will perfectly work:
Query query = new Query()
query.fields().elemMatch("movies", Criteria.where("name").is("Fana"));
List<Movies> movies = mongoTemplate.find(query, Movies.class);
The drawback of this query is that it may return duplicate results present in different documents, since more than 1 document may match this criteria. So you can add _id in the criteria like below:
Criteria criteria = Criteria.where('_id').is(movieId)
Query query = new Query().addCriteria(criteria)
query.fields().elemMatch("movies", Criteria.where("name").is("Fana"));
query.fields().exclude('_id')
List<Movies> movies = mongoTemplate.find(query, Movies.class);
I am excluding "_id" of the document in the response.
I'm using the following code to filter by elastic search java api,it works fine and return result if i use string query ,but If i use text with spaces or uppercase letters it don't return any data
if use
String query={"bool":{"should":[{"term":{"name":"test"}}]}}
return data
and if i use
String query={"bool":{"should":[{"term":{"name":"test airportone"}}]}}
or
String query={"bool":{"should":[{"term":{"name":"TEST"}}]}}
return no data
String query={"bool":{"should":[{"term":{"name":"test airport one"}}]}}
BoolQueryBuilder bool = new BoolQueryBuilder();
bool.must(new WrapperQueryBuilder(query));
SearchQuery searchQuery = new
NativeSearchQueryBuilder()
.withQuery(bool)
.build();
Page<Asset> asset =
elasticsearchTemplate.queryForPage(searchQuery,Asset.class);
return asset.getContent();
You have two options depending on your use-case.
First option: You can use match instead of term to search for a string if you want to get advantage of ElasticSearch full text search capabilities.
{
"bool": {
"should": [{
"match": {
"name": "test airportone"
}
}]
}
}
Second option: You can also specify that the name field is not analyzed when mapping your index so ElasticSearch will always store it as it is and always will get the exact match.
"mappings": {
"user": {
"properties": {
"name": {
"type": "string"
"index": "not_analyzed"
}
}
}
}