In spring data elasticsearch one model class/entity represents or maps to an index and type.
eg :-
#Document(indexName = "myindex",type="mytype")
public class DocumentModel {
......
}
I have a use case in which I should index data in different es indexes with same structure. If that's the case how can I represent all of those indices with this model class ?
Spring Data ES supports using SpEL expressions in the index name of the #Document annotation, like this:
#Document(indexName = "myindex-#{userId}", type="mytype")
public class DocumentModel {
......
}
Hence, you have access to the whole context offered by SpEL in order to create your index names.
UPDATE
If you're using elasticsearchTemplate, there is a simpler variant, you can do it like this:
IndexQuery indexQuery = new IndexQueryBuilder()
.withId(docModel.getId())
.withObject(docModel)
.withIndex("myindex"+docModel.getUserId()).build();
the call to withIndex("...") will override whatever index name you have in the #Document annotation
Related
I am trying to make a specific query in a MongoDB collection with SpringBoot. There are lots of results with the query that can be seen here. I am trying to sort and filter the results as I did in MongoDB compass in the screenshot and get ONLY the latest entry in the query results.
TaskRepo.java
public interface TaskRepo extends MongoRepository<Task, String> { }
MongoRepository help you to query document with attributes of resource (entity), when you want to query with attributes of sub-resource (embedded entity) you can use #Query to declare finder queries directly on repository methods
public interface TaskRepo extends MongoRepository<Task, String> {
#Query("{'meta.idf' : ?0, 'method': ?1}).sort({'meta.date': -1}")
public List<Task> findBySubResourceAndSort(String id, String method);
}
Or you can use MongoTemplate (refer this document https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/MongoTemplate.html)
I am using below drivers.
implementation 'com.datastax.astra:astra-spring-boot-starter:0.3.0'
implementation 'com.datastax.oss:java-driver-core:4.14.1'
implementation 'com.datastax.oss:java-driver-query-builder:4.14.1'
implementation 'com.datastax.oss:java-driver-mapper-runtime:4.14.1'
implementation 'org.springframework.boot:spring-boot-starter-data-cassandra'
Here are my entities:
#NamingStrategy(convention = NamingConvention.SNAKE_CASE_INSENSITIVE)
#CqlName("engine_torque_by_last_miles")
#Entity
public class EngineTorqueByLastMiles {
private UUID id;
#PartitionKey(1)
private String vinNumber;
}
Here is my repository:
public interface EngineTorqueByLastMilesRepository extends CassandraRepository<EngineTorqueByLastMiles, String> {
List<EngineTorqueByLastMiles> findAllByVinNumberAndOrganizationId(String vinNumber, Integer organizationId);
}
The problem I am facing is the soring.data.jpa.cassandra does not map the Entity name or the attributes to snake_case even after using NamingStrategy or CqlName annotations from datastax drivers.
Does datastax provide any driver that supports jpa so that I can write my Entities and their attributes in typical java naming convention and cassandra tables or attributes with snake_case ?
Datastax provides indeed a way to map objects to your Cassandra Tables and it is called the Cassandra object mapper. The documentation is here https://docs.datastax.com/en/developer/java-driver/4.13/manual/mapper/ BUT YOU DO NOT NEED IT HERE.
Looking at your code it seems you want to use Spring Data Cassandra. This is totally fine. You are simply not using the proper set of annotations. You should the Spring data annotations.
Your bean becomes:
#Table("engine_torque_by_last_miles")
public class EngineTorqueByLastMiles {
#PrimaryKeyColumn(name = "vin_number", ordinal = 0, type = PrimaryKeyType.PARTITIONED)
private String vinNumber;
#Column("id")
#CassandraType(type = Name.UUID)
private UUID id;
// default constructor
// getters
// setters
}
Given the table name, it seems your partition key should be last_miles but it was not provided in your question.
You provided an id but it was not annotated also I assumed it was not part of the primary key. If you have a composite primary key with Partition key and cluster columns you need to create an ANOTHER internal bean for the PK and annotate it with #PrimaryKey (sample)
You can find a full-fledge working application here with multiple entities https://github.com/datastaxdevs/workshop-betterreads/tree/master/better-reads-webapp
If you edit or complete your question we could propose the exact beans needed.
Try setting the property:
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
since Hibernate 5 it's the default and you would get your snake-cased naming.
For more reference see the documentation here
I am using Spring Boot and sharing the same entity between an Elastic Search database and a MongoDB database. The entity is declared this way:
#Document
#org.springframework.data.elasticsearch.annotations.Document(indexName = "...", type = "...", createIndex = true)
public class ProcedureStep {
...
}
Where #Document is from this package: org.springframework.data.mongodb.core.mapping.Document
This works without any issue, but I am not able to use generic annotations to target Elastic Search only. For example:
#Transient
private List<Point3d> c1s, c2s, c3s, c4s;
This will exclude this field from both databases, Mongo and Elastic, whereas my intent was to apply it for Elastic Search only.
I have no issue in using Elastic specific annotations like this:
#Field(type = FieldType.Keyword)
private String studyDescription;
My question is:
what annotation can I use to exclude a field from Elastic Search only and keep it in Mongo?
I don't want to rewrite the class as I don't have a "flat" structure to store (the main class is composed with fields from other classes, which themselves have fields I want to exclude from Elastic)
Many thanks
Assumption: ObjectMapper is used for Serialization/Deserialization
My question is: what annotation can I use to exclude a field from
Elastic Search only and keep it in Mongo? I don't want to rewrite the
class as I don't have a "flat" structure to store (the main class is
composed with fields from other classes, which themselves have fields
I want to exclude from Elastic)
Please understand this is a problem of selective serialization.
It can be achieved using JsonViews.
Example:
Step1: Define 2 views, ES Specific & MongoSpecific.
class Views {
public static class MONGO {};
public static class ES {};
}
Step2: Annotate the fields as below. Description as comments :
#Data
class Product {
private int id; // <======= Serialized for both DB & ES Context
#JsonView(Views.ES.class) //<======= Serialized for ES Context only
private float price;
#JsonView(Views.MONGO.class) // <======= Serialized for MONGO Context only
private String desc;
}
Step 3:
Configure Different Object Mappers for Spring-Data-ES & Mongo.
// Set View for MONGO
ObjectMapper mapper = new ObjectMapper();
mapper.setConfig(mapper.getSerializationConfig().withView(Views.MONGO.class));
// Set View for ES
ObjectMapper mapper = new ObjectMapper();
mapper.setConfig(mapper.getSerializationConfig().withView(Views.ES.class));
Can anybody know how to achieve the pagination in Spring Data Cassandra ?
I have tried all the possible alternatives to implement pagination for my table.
One of the stackoverflow answers says it is not provided directly.
Paging SELECT query results from Cassandra in Spring Boot application
As per documentation (https://docs.spring.io/spring-data/cassandra/docs/current-SNAPSHOT/reference/html/#cassandra.repositories.queries) says CrudRepository provides method with Pageable.
org.springframework.data.repository.CrudRepository does not provide.
I am using Cassandra 3.11.3 and Spring Boot 1.5.1.RELEASE.
Can one provide me simple pagination demo with Spring Data Cassandra ?
The pagination in Spring Data Cassandra work in 'forward only' basis like the way iterable interface works.
#Autowired
PersonRepository personrepo;
#GetMapping("/all/{page}")
public List<Person> getPaginated(#PathVariable Integer page) {
int currpage = 0, size = 2;
Slice<Person> slice = personrepo.findAll(CassandraPageRequest.first(size));
while(slice.hasNext() && currpage < page) {
slice = personrepo.findAll(slice.nextPageable());
currpage++;
}
return slice.getContent();
}
Where your repository contains:
public interface PersonRepo extends CrudRepository<Person, Integer> {
Slice<Person> findAll(Pageable pr);
}
Hence, you keep querying the result until the desired location of the data is reached.
This is the simplest way to do pagination in reactive Cassandra or regular Cassandra with spring data
In your repository class create custom methods with Query annotation (key point here is for second method there should be a offset that needs to sent as a parameter) Assuming date created is a clustering key
user table primary key(user_id, date_created)
Ex:-
#Query("select * from user order by date_created asc Limit :count)
Flux<UserEntity> findFirstSetOfUsers(#Param(count) int noOfRecords);
Next set of records
#Query("select * from user where date_created > :dateCreated order by date_created asc Limit :count)
Flux<UserEntity> findNextSetOfUsers(#Param(count) int noOfRecords, #Param(dateCreated) Date dateCreated);
I have a ES Document like this
class User {
String name;
String describe;
List<String> items;
}
I'm using spring data to talk to ES by the Repository interface
interface UserRepository extends Repository<User, String> {
}
Now I need to build a rest interface which responses a JSON-format data like this
{"name": String, "firstItem": String}
Because the describe and items in User is very big, it's very expensive to retrieve all field from the ES.
I know the ES have a feature named "Response Filtering" which can fit my requirement, but I don't find a way to using it in Spring Data.
How to do this in spring data?
What you need is a mix of source filtering (for not retrieving heavy fields) and response filtering (for not returning heavy fields). However, the latter is not supported in Spring Data ES (yet)
For the former, you can leverage NativeSearchQueryBuilder and specify a FetchSourceFilter that will only retrieve the fields you need. The latter is not supported yet in Spring Data ES. What you can do is to create another field named firstItem in which you'd store the first element of items so that you can return it for this query.
private ElasticsearchTemplate elasticsearchTemplate;
String[] includes = new String[]{"name", "firstItem"};
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withSourceFilter(new FetchSourceBuilder(includes, null))
.build();
Page<User> userPage =
elasticsearchTemplate.queryForPage(searchQuery, User.class);