I have a requirement to read from three tables in spring boot batch and dynamically construct a new table and populate the data. So, how do i dynamically add new fields in the POJO?
A simple way is to create hashmap in the class and add as many values as you want, dynamically :
public Map<String, Object> attributes = new HashMap<>();
Related
I am using Spring Boot and sharing the same entity between an Elastic Search database and a MongoDB database. The entity is declared this way:
#Document
#org.springframework.data.elasticsearch.annotations.Document(indexName = "...", type = "...", createIndex = true)
public class ProcedureStep {
...
}
Where #Document is from this package: org.springframework.data.mongodb.core.mapping.Document
This works without any issue, but I am not able to use generic annotations to target Elastic Search only. For example:
#Transient
private List<Point3d> c1s, c2s, c3s, c4s;
This will exclude this field from both databases, Mongo and Elastic, whereas my intent was to apply it for Elastic Search only.
I have no issue in using Elastic specific annotations like this:
#Field(type = FieldType.Keyword)
private String studyDescription;
My question is:
what annotation can I use to exclude a field from Elastic Search only and keep it in Mongo?
I don't want to rewrite the class as I don't have a "flat" structure to store (the main class is composed with fields from other classes, which themselves have fields I want to exclude from Elastic)
Many thanks
Assumption: ObjectMapper is used for Serialization/Deserialization
My question is: what annotation can I use to exclude a field from
Elastic Search only and keep it in Mongo? I don't want to rewrite the
class as I don't have a "flat" structure to store (the main class is
composed with fields from other classes, which themselves have fields
I want to exclude from Elastic)
Please understand this is a problem of selective serialization.
It can be achieved using JsonViews.
Example:
Step1: Define 2 views, ES Specific & MongoSpecific.
class Views {
public static class MONGO {};
public static class ES {};
}
Step2: Annotate the fields as below. Description as comments :
#Data
class Product {
private int id; // <======= Serialized for both DB & ES Context
#JsonView(Views.ES.class) //<======= Serialized for ES Context only
private float price;
#JsonView(Views.MONGO.class) // <======= Serialized for MONGO Context only
private String desc;
}
Step 3:
Configure Different Object Mappers for Spring-Data-ES & Mongo.
// Set View for MONGO
ObjectMapper mapper = new ObjectMapper();
mapper.setConfig(mapper.getSerializationConfig().withView(Views.MONGO.class));
// Set View for ES
ObjectMapper mapper = new ObjectMapper();
mapper.setConfig(mapper.getSerializationConfig().withView(Views.ES.class));
I have a User entity that has as many properties as there are columns in the database and a transient one next to that. The user can be part of an Organisation, which is a onetoone relationship. In that entity the data type is an Organisation object.
The model is also used in the controller to check and contain the request data values, but this cannot be achieved with organisation, because the user passes the name as a string and in the entity, the organisation is an object as already mentioned.
So I would like to add another property that can contain the organisation name, so then I can convert it to an Organisation object in the Service. But I can't add a regular property because Spring JPA will think it should be a column and it can't be a transient property either, because it's ignored on the controller side for the request.
Currently I'm achieving this by adding another parameter with #RequestParam(name="organisationname"), and passing it together with the User object to the Service, but I have the feeling that this is not the proper way.
How can I approach this differently and in the User entity itself?
You can have your JSON serializer NOT ignore #Transient fields (I like this approach as transient is a DB term so it shouldn't be affecting your JSON serialization) - you can then use #JsonIgnore to skip JSON serialization properties.
So #Transient to ignore when interacting with the DB, #JsonIgnore to ignore when serializing/deserializing.
If using the Jackson ObjectMapper you can do the following:
#Bean
public ObjectMapper objectMapper() {
final ObjectMapper mapper = new ObjectMapper();
final Hibernate5Module hibernate5Module = new Hibernate5Module();
hibernate5Module.disable(Hibernate5Module.Feature.USE_TRANSIENT_ANNOTATION);
mapper.registerModule(hibernate5Module);
return mapper;
}
name of the table should be fixed but in my scenario the last part of the table name is profile based so in local it is X but in dev it is Y and so on till Prod. Is there way to add dynamically the value to the table name.
The question tries to implement a bad practice. Don't do that.
Currently, Spring, Hibernate, and JPA does not support your configuration type.
You can use Hibernate interceptors to change the table in the generated SQL statements.
For your case you can define your table class like this:
#Entity
#org.hibernate.annotations.Proxy(lazy=false)
#Table(name=TableNameReplacer.PLACEHOLDER, schema="MySchema")
#Inheritance(strategy=InheritanceType.SINGLE_TABLE)
public class ProfileData implements Serializable {
and define your Hibernate interceptor in a following way:
public class TableNameReplacer extends EmptyInterceptor {
public static final String TABLE_PLACEHOLDER = "{table_placeholder}";
#Override
public String onPrepareStatement(String sql) {
if (sql.contains(TABLE_PLACEHOLDER )) {
String replacement = "{your logic to fill proper table name}";
sql = sql.replace(TABLE_SUFFIX_PLACEHOLDER, replacement);
}
return super.onPrepareStatement(sql);
}
Using this approach you're free to modify generated SQL and replace the table name there as you wish.
I recommend to use good placeholder value which you're sure will not be a part of actual values being saved to the table (or you can only limit this to select statements if you only read the data).
I have a ES Document like this
class User {
String name;
String describe;
List<String> items;
}
I'm using spring data to talk to ES by the Repository interface
interface UserRepository extends Repository<User, String> {
}
Now I need to build a rest interface which responses a JSON-format data like this
{"name": String, "firstItem": String}
Because the describe and items in User is very big, it's very expensive to retrieve all field from the ES.
I know the ES have a feature named "Response Filtering" which can fit my requirement, but I don't find a way to using it in Spring Data.
How to do this in spring data?
What you need is a mix of source filtering (for not retrieving heavy fields) and response filtering (for not returning heavy fields). However, the latter is not supported in Spring Data ES (yet)
For the former, you can leverage NativeSearchQueryBuilder and specify a FetchSourceFilter that will only retrieve the fields you need. The latter is not supported yet in Spring Data ES. What you can do is to create another field named firstItem in which you'd store the first element of items so that you can return it for this query.
private ElasticsearchTemplate elasticsearchTemplate;
String[] includes = new String[]{"name", "firstItem"};
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withSourceFilter(new FetchSourceBuilder(includes, null))
.build();
Page<User> userPage =
elasticsearchTemplate.queryForPage(searchQuery, User.class);
In spring data elasticsearch one model class/entity represents or maps to an index and type.
eg :-
#Document(indexName = "myindex",type="mytype")
public class DocumentModel {
......
}
I have a use case in which I should index data in different es indexes with same structure. If that's the case how can I represent all of those indices with this model class ?
Spring Data ES supports using SpEL expressions in the index name of the #Document annotation, like this:
#Document(indexName = "myindex-#{userId}", type="mytype")
public class DocumentModel {
......
}
Hence, you have access to the whole context offered by SpEL in order to create your index names.
UPDATE
If you're using elasticsearchTemplate, there is a simpler variant, you can do it like this:
IndexQuery indexQuery = new IndexQueryBuilder()
.withId(docModel.getId())
.withObject(docModel)
.withIndex("myindex"+docModel.getUserId()).build();
the call to withIndex("...") will override whatever index name you have in the #Document annotation