I use MongoDB and Spring Boot in a project. I used manual reference to point out a collection, My structure is as follows:
Reel collection
{
_id : "reel_id_1",
name: "reel 1",
category :[
{
_id : "category_id_1",
name: "category 1",
videos: ["video_id_1","video_id_2"]
}
]
}
Video collection
{
_id: "video_id_1", // first document
name: "mongo"
}
{
_id: "video_id_2", // seconddocument
name: "java"
}
Java classes are
#Document
#Data
public class Reel {
#Id
private ObjectId _id;
private String name;
List<Category> category;
}
#Data
public class Category {
#Id
private ObjectId _id=new ObjectId();
private String name;
Video videos;
}
#Document
#Data
public class Video {
#Id
private ObjectId _id = new ObjectId();
private String name;
}
I tried to join both document via mongoTemplate
public List<Reel> findById(ObjectId _id) {
LookupOperation lookupOperation = LookupOperation.newLookup()
.from("video")
.localField("category.videos")
.foreignField("_id")
.as("category.videos");
UnwindOperation unwindOperation = Aggregation.unwind("category");
Aggregation agg = newAggregation(unwindOperation,match(Criteria.where("_id").is(_id)),lookupOperation);
Aggregation aggregation = newAggregation(lookupOperation);
List<Reel> results = mongoTemplate.aggregate(aggregation, "reel", Reel.class).getMappedResults();
return results;
}
But it throws an error.
Failed to instantiate java.util.List using constructor NO_CONSTRUCTOR with arguments
But since I use "unwind", I created a new Entity UnwindReel and add Category category instead of List<Category> category. And used
List<UnwindReel> results = mongoTemplate.aggregate(aggregation, "reel", UnwindReel.class).getMappedResults();
It combines only first video (video_id_1) object. How can I get all objects inside videos array? Is there any method to fetch?
Your JSON stored in database has wrong structure. Your Reel class expects list of Category, but in database you have stored as nested object.
You need to add this stage just after $lookup
{
"$addFields": {
"category": {
"$map": {
"input": "$category.videos",
"in": {
"videos": "$$this"
}
}
}
}
}
Java code
public List<Reel> findById(String _id) {
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("_id").is(_id)),
Aggregation.lookup(mongoTemplate.getCollectionName(Video.class), "category.videos", "_id", "category.videos"),
new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext context) {
return new Document("$addFields",
new Document("category", new Document("$map", new Document("input", "$category.videos")
.append("in", new Document("videos", "$$this")))));
}
})
.withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
LOG.debug(
aggregation.toString().replaceAll("__collection__", mongoTemplate.getCollectionName(Reel.class)));
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Reel.class), Reel.class)
.getMappedResults();
}
Recomendations
Do not hard-code collection name, use better mongoTemplate.getCollectionName method
Always log aggregation pipeline before performing, it helps debugging.
If your collection will grow in the future, use {allowDiskUse: true} MongoDb aggregation option.
Related
I am writing RESTful API to search MongoDB collection named 'global' by criteria using TextQuery. My problem is that I cannot access nested object fields when doing query.
For example this works:
GET localhost:8080/search?criteria=name:'name'
And this does not:
GET localhost:8080/search?criteria=other.othername:'Other Name'
I have MongoDB json structure (imported from JSON into 'global' collection as whole nested objects)
[{
"name": "Name",
"desc": "Desc",
"other" {
"othername": "Other Name",
}
},
{
"name": "Name",
"desc": "Desc",
"other" {
"othername": "Other Name",
}
}
]
And classes (with getters & setters & etc):
#Document(collection="global")
public class Global{
#TextIndexed
String name;
#TextIndexed
String desc;
Other other;
...
}
public class Other{
String othername;
...
}
My controller has method
#GetMapping("/search")
public Iterable<Global> getByCriteria(#RequestParam("criteria") String criteria) {
...
}
And I am trying to write text search with
public Iterable<Global> findByCriteria(String criteria) {
TextCriteria criteria = TextCriteria.forDefaultLanguage().matching(criteria);
TextQuery query = TextQuery.queryText(criteria);
return mongoTemplate.find(query, Global.class);
}
You need to add #TextIndexed to your other field.
public class Global{
#TextIndexed
String name;
#TextIndexed
String desc;
#TextIndexed
Other other;
...
}
Note: All nested object fields are searchable
or you may add #TextIndexed for each nested object field:
public class Other {
#TextIndexed
String othername;
...
}
i reindex data from es 2.4 to 5.6.
data in es 2.4 have 2 types,and the 2 type is parent-child relation.
when reindex it to es 5.6,the index only contains single type,the parent-child relation by using join type to resolving.
the data above works ok.
the mapping example like this, it contains a join type:
"mappings": {
"doc": {
"properties": {
"my_join_field": {
"eager_global_ordinals": true,
"type": "join",
"relations": {
"question": "answer"
}
},
"name": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
}
}
}
how to mapping join type by using spring data elasticSearch:
in old version code es 2.4,i can mapping it like this:
#Document(indexName = ParentEntity.INDEX, type = ParentEntity.PARENT_TYPE, shards = 1, replicas = 0, refreshInterval = "-1")
public class ParentEntity {
public static final String INDEX = "parent-child";
public static final String PARENT_TYPE = "parent-entity";
public static final String CHILD_TYPE = "child-entity";
#Id
private String id;
#Field(type = FieldType.Text, store = true)
private String name;
public ParentEntity() {
}
public ParentEntity(String id, String name) {
this.id = id;
this.name = name;
}
public String getId() {
return id;
}
public String getName() {
return name;
}
#Override
public String toString() {
return new ToStringCreator(this).append("id", id).append("name", name).toString();
}
#Document(indexName = INDEX, type = CHILD_TYPE, shards = 1, replicas = 0, refreshInterval = "-1")
public static class ChildEntity {
#Id
private String id;
#Field(type = FieldType.Text, store = true)
#Parent(type = PARENT_TYPE)
private String parentId;
#Field(type = FieldType.Text, store = true)
private String name;
public ChildEntity() {
}
public ChildEntity(String id, String parentId, String name) {
this.id = id;
this.parentId = parentId;
this.name = name;
}
public String getId() {
return id;
}
public String getParentId() {
return parentId;
}
public String getName() {
return name;
}
#Override
public String toString() {
return new ToStringCreator(this).append("id", id).append("parentId", parentId).append("name", name).toString();
}
}
}
how can i Mapping join type by using spring data elasticSearch v3.0.10?
Today, i tried the entity below to working at spring data elasticSearch 3.0.10:
#Document(indexName = "join_index", type = "join_mapping")
#Data
public class JoinEntity {
#Id
private String id;
#Mapping(mappingPath = "/mappings/join_type.json")
private Map<String,String> relationType;
#Field(type = FieldType.Keyword)
private String name;
//#Parent(type = "question")
#Field(type = FieldType.Keyword)
private String parentId;
}
join_type.json below:
{
"type": "join",
"relations": {
"question": "answer"
}
}
it create index and put mapping work ok:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration("classpath:elasticsearch-template-test.xml")
public class ElasticsearchTemplateJoinTests {
#Autowired
private ElasticsearchTemplate elasticsearchTemplate;
#Before
public void before() {
clean();
elasticsearchTemplate.deleteIndex(JoinEntity.class);
elasticsearchTemplate.createIndex(JoinEntity.class);
elasticsearchTemplate.putMapping(JoinEntity.class);
elasticsearchTemplate.refresh(JoinEntity.class);
}
#Test
public void shouldCreateIndexAndMappingSuccess(){
Map mapping = elasticsearchTemplate.getMapping(JoinEntity.class);
assertThat(mapping, is(notNullValue()));
Map properties = (Map) mapping.get("properties");
assertThat(properties, is(notNullValue()));
assertThat(properties.containsKey("name"), is(true));
Map file = (Map) properties.get("relationType");
assertThat(file, is(notNullValue()));
assertThat(((String) file.get("type")), is("join"));
}
}
when index parent work ok too,but index child it throws exception:
#Test
public void shouldIndexParentAndChildSuccess(){
JoinEntity parenEntity = new JoinEntity();
parenEntity.setName("parent_name");
parenEntity.setRelationType(Collections.singletonMap("name","question"));
IndexQuery parentQuery = new IndexQueryBuilder().withId("11").withObject(parenEntity).build();
final String id = elasticsearchTemplate.index(parentQuery);
assertThat("11",is(id));
JoinEntity childEntity = new JoinEntity();
childEntity.setName("child_name");
Map<String,String> joinRelation = new HashMap<>(2);
joinRelation.put("name","answer");
joinRelation.put("parent", "11");
childEntity.setRelationType(joinRelation);
childEntity.setParentId("11");
IndexQuery childQuery = new IndexQueryBuilder().withId("22").withObject(childEntity).build();
elasticsearchTemplate.index(childQuery);
}
exception:
MapperParsingException[failed to parse
]; nested: IllegalArgumentException[[routing] is missing for join field [relationType]];
at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:171)
how can i resolve this problem or Mapping the new version Parent-child relation correctly?thks!!
Elasticsearch needs the parent document routing parameter when you index child document check this
This is because both parent and child documents must be indexed in same shard to join to work.
However I couldn't figure out a way to solve this using Spring data elasticsearch. The only way that worked was using RestHighLevelClient
The recent version of Spring Data ElasticSearch had added support for this doc
Your child indexing would be something like,
IndexRequest indexRequest = new IndexRequest();
indexRequest.source(objectMapper.writeValueAsString(childEntity),XContentType.JSON);
indexRequest.id("22"); //child doc id
indexRequest.index(INDEX_NAME);
indexRequest.type(INDEX_TYPE);
indexRequest.routing("11"); //parent doc id
restHighLevelClient.index(indexRequest);
Finally, i gived up the parent-child relation, i split them into two separate indexs. some advance feature should be used less if not neccessary.
For example, there are 2 classes:
#Document
public class WordSet {
#Id
private ObjectId id;
private Language language;
private ObjectId languageId;
}
#Document
public class Language {
#Id
private ObjectId id;
#Index(unique = true)
private String languageName;
}
Languages are stored in separate collection and WordSets are stored in another collection. BUT WordSet is stored only with languageId (language is always null when I save WordSet). language field in WordSet is needed, but it retrieved using aggregation
public Flux<WordSet> getAll() {
AggregationOperation[] joinLanguages = new AggregationOperation[] {
lookup(mongoTemplate.getCollectionName(Language.class), "languageId", "_id", "language"),
unwind("language")
};
Aggregation agg = newAggregation(joinLanguages);
return mongoTemplate.aggregate(agg, WordSet.class, WordSet.class);
}
So, WordSet in mongodb contains only languageId and language field is filled using aggregation (join from Language collection).
But if I want to add more than one WordSet to collection I have an error
com.mongodb.MongoWriteException: E11000 duplicate key error collection: langdope.wordSet index: language.language_name_index dup key: { : null }
How can I handle it? I need languageName to be unique, but also I want to save objects which contains Languages without this problems. I can't just mark language as #Transient (aggregation will not work).
I need to insert a new track into the existing event document following is my class structure
class Event
{
String _id;
List<Track> tracks;
}
class Track
{
String _id;
String title;
}
My existing document is
{
"_id":"1000",
"event_name":"Some Name"
}
document will look like after insertion
{
"_id":"1000",
"event_name":"Some name",
"tracks":
[
{
"title":"Test titile",
}
]
}
How can i insert that track into my existing document using mongoTemplate spring data mongodb?
First, you have to annotate Event class with #Document:
#Document(collection = "events")
public class Event
{
// rest of code
}
The code for adding an event should look like this:
#Repository
public class EventsDao {
#Autowired
MongoOperations template;
public void addTrack(Track t) {
Event e = template.findOne
(new Query(Criteria.where("id").is("1000")), Event.class);
if (e != null) {
e.getTracks().add(t);
template.save(e);
}
}
}
Note : You should change Event's class String _id; to String id; in order for this example to work (or change the query literal).
Edit update a track is also fairly easy. Suppose you want to change the first track's title:
Event e = template.findOne(new Query(Criteria.where("_id").is("1000")), Event.class);
if (e != null) {
e.getTracks().get(0).setTitle("when i'm 64");
template.save(e);
}
I'm using spring-data-mongodb (version 1.0.2.RELEASE) and mongodb (version 2.2).
I have an object A that contain a list of object Location. The classes are the following:
public class A {
#Id
private ObjectId id;
private List<Location> places;
//GETTER AND SETTER
}
public class Place {
private String name;
private String description;
#GeoSpatialIndexed
private double[] location;
//GETTER AND SETTER
}
I need to find all objects A with a specific location.
I tried to use together the operators $within and $elemMatch as following:
#Query(value = "{'places' : { $elemMatch: { location: {'$within' : {'$center' : [?0, ?1]} } }}}")
public List<A> findByLocation(Point location, double radius);
When i run this query, i receive the following exception:
org.springframework.data.mongodb.UncategorizedMongoDbException: can't find special index: 2d for: { places: { $elemMatch: { location: { $within: { $center: [ { x: 41.904159, y: 12.549132 }, 0.07000000000000001 ] } } } } }; nested exception is com.mongodb.MongoException: can't find special index: 2d for: { places: { $elemMatch: { location: { $within: { $center: [ { x: 41.904159, y: 12.549132 }, 0.07000000000000001 ] } } } } }
Any suggestions?
Regards
Apparently there's no 2d index on the location attribute, and MongoDB requires such an index to be present. I don't know if Spring should be creating that index by itself, but if it doesn't, you can create the index via
db.collection.ensureIndex({"location":"2d"})
where collection is replaced by the name of your collection.
I solved my problem changing A class. I replaced the list of Place with a list of org.springframework.data.mongodb.core.geo.Point as follow:
public class A {
#Id
private ObjectId id;
#GeoSpatialIndexed
private List<Point> places;
//GETTER AND SETTER
}
And i replaced old query with a new one as follow:
#Query(value = "{'places' : {'$within' : {'$center' : [?0, ?1]} } }")
public List<A> findByLocation(Point location, double radius);
And now it's work!
Thanks to all!