According to official elasticsearch, I'm understand that Nested required to reindexing the parent with all its children if add/delete/update operations, therefore is expensive when required a lot modification.
Example using Nested:
#Document(indexName = "test-index-person-multiple-level-nested", type = "user", shards = 1, replicas = 0, refreshInterval = "-1")
public class PersonMultipleLevelNested {
#Id
private String id;
private String name;
#Field(type = FieldType.Nested)
private List<GirlFriend> girlFriends;
//Getter, setter & constructor
}
And Parent & Child are better suite this kind situation, but how can I setup using Spring Data Elasticsearch? It is not yet supported? Seem can't find related documentation.
Not sure about documentation, but there is a unit test for this feature: https://github.com/spring-projects/spring-data-elasticsearch/blob/master/src/test/java/org/springframework/data/elasticsearch/entities/ParentEntity.java — see #Parent in particular.
Related
I am using Spring data for Elastic Search and am using the ReactiveCrudRepository for stuff like finding and deleting. I noticed that with attributes that are in root and are simple objects, the deletion works (deleteByAttributeName). However if I have nested objects then it does not work.
Here's my entities
Book
#Data
#TypeAlias("book")
#Document(indexName = "book")
public class EsBook{
#Field(type = FieldType.Long)
private Long id;
#Field(type = FieldType.Nested)
private EsStats stats;
#Field(type = FieldType.Date, format = DateFormat.date)
private LocalDate publishDate;
}
Stats
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
#EqualsAndHashCode
public class EsStats{
#Field(type = FieldType.Double)
private Double averageRating;
#Field(type = FieldType.Integer)
private Double totalRatings;
#Field(type = FieldType.Keyword)
private String category; //this can be null
}
Here is what I have tried and is working and not working
I used ReactiveCrudRepository to delete documents in index. For all the regular fields on Book Level like id or with id and publishDate deletion works perfectly. As soon as I use embedded object like Stats, it stops working. I see the documents and the stats that I am sending match atleast visually but never finds or deletes them.
I tried to use EqualsAndHashcode in the Stats assuming maybe iternally somehow does not consider equal for some reason. I also tried changing double data type to int, because on looking at the elastic search document, I see that average review if whole number like 3 is save as 3 but when we send it from Java, i see in the debug 3 being shown as 3.0, so I was doubting if that is the case, but does not seem so. Even changing the datatype to int deletion does not work.
public interface ReactiveBookRepository extends ReactiveCrudRepository<EsBook, String> {
Mono<Void> deleteById(long id); //working
Mono<Void> deleteByIdAndPublishDate(long id, LocalDate publishDate); //Nor working
Mono<Void> deleteByIdAndStats(long id, LocalDate startDate);
}
Any help will be appreciated
Have you verified that your Elasticsearch index mapping matches your Spring Data annotations?
Verify that the index mapping defines the stats field as a nested field type.
If not, then try changing your Spring annotation to:
#Field(type = FieldType.Object)
private EsStats stats;
I am using Spring Boot and sharing the same entity between an Elastic Search database and a MongoDB database. The entity is declared this way:
#Document
#org.springframework.data.elasticsearch.annotations.Document(indexName = "...", type = "...", createIndex = true)
public class ProcedureStep {
...
}
Where #Document is from this package: org.springframework.data.mongodb.core.mapping.Document
This works without any issue, but I am not able to use generic annotations to target Elastic Search only. For example:
#Transient
private List<Point3d> c1s, c2s, c3s, c4s;
This will exclude this field from both databases, Mongo and Elastic, whereas my intent was to apply it for Elastic Search only.
I have no issue in using Elastic specific annotations like this:
#Field(type = FieldType.Keyword)
private String studyDescription;
My question is:
what annotation can I use to exclude a field from Elastic Search only and keep it in Mongo?
I don't want to rewrite the class as I don't have a "flat" structure to store (the main class is composed with fields from other classes, which themselves have fields I want to exclude from Elastic)
Many thanks
Assumption: ObjectMapper is used for Serialization/Deserialization
My question is: what annotation can I use to exclude a field from
Elastic Search only and keep it in Mongo? I don't want to rewrite the
class as I don't have a "flat" structure to store (the main class is
composed with fields from other classes, which themselves have fields
I want to exclude from Elastic)
Please understand this is a problem of selective serialization.
It can be achieved using JsonViews.
Example:
Step1: Define 2 views, ES Specific & MongoSpecific.
class Views {
public static class MONGO {};
public static class ES {};
}
Step2: Annotate the fields as below. Description as comments :
#Data
class Product {
private int id; // <======= Serialized for both DB & ES Context
#JsonView(Views.ES.class) //<======= Serialized for ES Context only
private float price;
#JsonView(Views.MONGO.class) // <======= Serialized for MONGO Context only
private String desc;
}
Step 3:
Configure Different Object Mappers for Spring-Data-ES & Mongo.
// Set View for MONGO
ObjectMapper mapper = new ObjectMapper();
mapper.setConfig(mapper.getSerializationConfig().withView(Views.MONGO.class));
// Set View for ES
ObjectMapper mapper = new ObjectMapper();
mapper.setConfig(mapper.getSerializationConfig().withView(Views.ES.class));
I am new to elastic-search and i am trying to use spring data elastic search in the application. I have a requirement where in there are two separate indexes and i want to fetch documents from both indexes in one query based on some condition.
I would try to explain it with sample example with the same scenario.
There are two Different classes for individual indexes.
#Document(indexName = "Book", type = "Book")
public class Book {
#Id
private String id;
#Field(type = FieldType.String)
private String bookName;
#Field(type = FieldType.Integer)
private int price;
#Field(type = FieldType.String)
private String authorName;
//Getters and Setters
}
There is one more class Author
#Document(indexName = "Author", type = "Author")
public Class Author{
#Id
private String id;
#Field(type = FieldType.String)
private String authorName;
//Getters and setters
}
So there are two indexes one Book and Other Author.
I want to fetch all the documents where authorName in Book index is equal to authorName in Author index.
Can i get the details from both the index as a single document like merged result.
It would be very helpful if anyone can suggest solution for this usecase.
Thanks a lot for your answer
Your question is not clear. the document in the Author index is not the same document as the document in the Book index. they are two different documents. the closest thing I can think of is querying multiple indices with the same query -
just add multiple indices to the indexCoordinates.of method to search for the same filed\value in both indices.
for example:
elasticsearchTemplae.search(query, returnedObjectClass, indexCoordinates.of("Author", "Book")).
it will return all the search hits with both indices mentioned, which have to fulfill the query conditions, whichever they are.
In other words, the common Jackson markup is not enough for serializing the same entity for using as the REST request response to the Angular frontend and to pass the object to Elasticsearch via the Jest client. Say, I have an image in the Entity as a byte array, and I'd like it to be stored to DB and be passed to the frontend, but don't like it being indexed by Elasticsearch to reduce the costs or quotas.
Now I have to use Jackson's JsonView to markup the fields to use for the Spring Data Elasticsearch's ObjectMapper:
#Entity
#Table(name = "good")
#org.springframework.data.elasticsearch.annotations.Document(indexName = "good", shards = 1, replicas = 0, refreshInterval = "-1")
public class Good implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "sequenceGenerator")
#SequenceGenerator(name = "sequenceGenerator")
#org.springframework.data.elasticsearch.annotations.Field(type = FieldType.Keyword)
#JsonView(Views.Elasticsearch.class)
private Long id;
#NotNull
#Column(name = "short_name", nullable = false)
#JsonView(Views.Elasticsearch.class)
#Field(store = true, index = true, type=FieldType.Text)
private String shortName;
#NotNull
#Column(name = "description", nullable = false)
#JsonView(Views.Elasticsearch.class)
#Field(store = true, index = true, type=FieldType.Text)
private String description;
#Lob
#Column(name = "image", nullable = false)
#JsonView(Views.Rest.class)
private byte[] image;
#Column(name = "image_content_type", nullable = false)
#JsonView(Views.Rest.class)
private String imageContentType;
#NotNull
#Column(name = "price", nullable = false)
#JsonView(Views.Elasticsearch.class)
#Field(store = true, index = true, type=FieldType.Integer)
private Integer price;
...
I have a clss for Views:
public class Views {
public static class Rest{}
public static class Elasticsearch{}
}
And the ObjectMapper set up in the corresponding Bean:
#Override
public String mapToString(Object object) throws IOException {
log.trace("Object to convert to JSON : {}",object);
log.trace("Converting to json for elasticsearch >>> {}",objectMapper.writer().withView(Views.Elasticsearch.class).writeValueAsString(object));
//log.trace("Converting to json for elasticsearch >>> {}",objectMapper.writeValueAsString(object));
return objectMapper.writerWithView(Views.Elasticsearch.class).writeValueAsString(object);
//return objectMapper.writeValueAsString(object);
}
So, I have to markup all the fields except the ignored to Elasticsearch with #JsonView(Views.Elasticsearch.class) and this is the error prone. Also, this field still requires #Field usage if I like to pass some parameters there like store or value type. When I have #JsonView(Views.Elasticsearch.class), but don't have #Field on some, the fields are created in the index on a fly, that allows them to search, but not in desired way.
The latest is the reason why if I just leave #Field there and don't place it over fields I don't want to index into Elasticsearch, the initial index indeed ignores them, but later requests pass the undesired field when the entity is serialized exactly the same way as it is done for the REST. And the index property is created on a fly, making resources being spent for the large binary object indexing. So it looks like #Field is used for the initial index creation on the startup, but are not configured to be used with ObjectMapper of the Spring Data Elasticsearch.
So, I'd like to make this ObjectMapper take only fields with #Field above them into account, i.e serialize the fields marked with #Field only and use no #JsonView staff. How can I configure it?
These are known problems when using the Jackson Object Mapper in Spring Data Elasticsearch (which als is the default) and this is one of the reasons, why in Spring Data Elasticsearch as of version 3.2 (which currently is in RC2 and will be available as 3.2.0.GA in mid-september), there is a different mapper available, the ElasticsearchEntityMapper.
This mapper still has to be setup explicitly, the reference documentation of 3.2.0.RC2 shows how to do this. Using this mapper the Jackson annotations do not influence the data stored in and read from Elasticsearch. And you can use the org.springframework.data.annotation.Transient annotation on a field to not have it stored in Elasticsearch.
The #Field annotation is used to setup the initial Elasticsearch mapping, properties not having this annotation are automatically mapped by Elasticsearch when they are inserted.
I am in reference to Spring Data Elasticsearch's
org.springframework.data.elasticsearch.repository.ElasticsearchRepository
org.springframework.data.elasticsearch.core.ElasticsearchTemplate
It seems they are two different APIs that achieve the same goal but I am not sure what the differences are between those two types and more importantly when to use which.
Can someone please provide advice and guidance?
ElasticsearchRepository is intended to be used as a repository for your domain classes, as it's typed. It extends Spring interfaces for repositories so it can used as one of them. You'll feel very comfortable with it if you are used to Spring repositories.
All you need to start indexing your objects to Elasticsearch is to add the #Document annotation to them and create a Repository interface extending ElasticsearchRepository.
The indexable class:
#Document(
indexName = "customers",
type = "customer",
shards = 1,
replicas = 0,
refreshInterval = "-1"
)
public class Customer {
#Id
private Long id;
private String name;
public Customer() {
}
public Customer(String name) {
this.name = name;
}
//Getters and setters omited
}
The repostitory:
public interface CustomerRepository
extends ElasticsearchRepository<Customer, Long>{
}
With this you can, out of the box, make CRUD operations, index, search and other common operations.
ElasticsearchTemplate, by other hand, is an elasticsearch client for working with your indexes, and it's not typed or related to your domain classes. It's more powerful since you can do many tasks not available to the repository implementation, like deleting an index or making aggregated searchs.