Save composite key data to elastic search document - elasticsearch

I am using hibernate envers to audit the data for my tables and save in Oracle DB. This auditing data I am reading and saving to elastic search index through Java code using spring data elastic search. I have a composite key (id and rev) which defines a unique row to save data but for elastic search I can't provide a composite key. It is taking only rev (identifier) column and replacing data.
Hibernate envers background information:
rev is the default identifier that hibernate is providing and for list of records which modified at same time, it creates same rev id:
Eg: id rev comments
1 1 newly created
2 1 newly created
1 2 modified
2 2 modified
First 2 rows are created at same time and next time I modified both the rows and updated so hibernate envers creates same rev id for 1 save.
#Entity
#IdClass(MyEmbeddedId.class)
#Document(indexName = "#{#indexName}", type = "my-document", shards = 1, replicas = 0, refreshInterval = "-1")
#Getter #Setter
public class MyClassAudit {
#Id
private Long id;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#org.springframework.data.annotation.Id --> (this is for elastic search _id)
private Long rev;
}
#Getter #Setter
public class MyEmbeddedId implements Serializable {
private Long id;
private Long rev;
}
Java code:
List<MyClass> list = repository.findById(id);
elasticSearchRepository.saveAll(list)
Elastic search repository interface:
public interface MyElasticSearchRepository extends GenericSearchRepository<MyClassAudit, Long> {}
When I save data to elastic search then all 4 records should be saved as given in example but only 2 records are saved like below:
_id id rev comments
1 2 1 newly created
2 2 2 modified
It is because rev is taken as identifier in elastic search and 2nd record is being updated.
How to make elastic search to consider composite key to maintain unique records?
PS: _id is the elastic search identifier. since rev is having spring data annotation id, rev is considered as identifier in elastic search

Elasticsearch itself has no concept of a composite key. Spring Data Elasticsearch takes the #Id annotated element and uses it's toString() method to create the id entry for Elasticsearch (and stores the field as well in the source).
So - without haven't tried - you could use your MyEmbeddedId class as a field property of your MyClassAudit class and annotate it with #Id. But you have to have this property in your class, it will not be synthesized.
This will probably conflict with the annotations for hibernate, but I don't think it's a good idea to share one entity between storages and using mixed annotations.

Related

Not able to search data in redis cache using spring crud repository by passing list of values for a property of the model saved in cache

We have model class saved in Redis as mentioned below:-
#Data
#NoArgsConstructor
#AllArgsConstructor
#RedisHash("book")
public class Book implements Serializable {
private static final long serialVersionUID = 2208852329346517265L;
#Id
private Integer bookID;
#Indexed
private String title;
#Indexed
private String authors;
private String averageRating;
private String isbn;
private String languageCode;
private String ratingsCount;
private BigDecimal price;
}
We have title and authors as our indexed property.
Now we wanted to search all the records from Redis by passing title and a list of authors using the spring crud repository as mentioned below.
public interface BookSpringRepository extends CrudRepository<Book, String> {
List<Book> findAllByTitleAndAuthors(String title, List<String> authors);
}
Service layer:-
#Override
public Optional<List<Book>> searchBooksByTitleAndAuthorNames(String title, List<String>
autherNames) {
return Optional.ofNullable(bookSpringRepository.findAllByTitleAndAuthors(title,
autherNames));
}
Here we are getting below exception
Unable to fetch data from Spring data Redis cache using List of Integer or
String.
Getting error while fetching - "Resolved
[org.springframework.core.convert.ConversionFailedException: Failed to convert from type
[java.lang.String] to type [byte] for value 'Ronak';
nested exception is java.lang.NumberFormatException: For input string: "Ronak"]."
We would not want to convert the list of string/integer to byte as it is a time-consuming process and as we tried took so much amount of time. Also when the results are retrieved we will again have to convert back to normal integer or string values.
The other option is to loop through the list and pass a single value at a time to the Redis crud repository and this time Redis crud repository is happy but that will be a loop call to Redis and network latency.
We cannot add ID attributes on authors' property as these can be duplicate records.
Does the spring crud repository support the LIKE query in search that way we can create a unique id having these authors' names and make put ID annotation on that new derived property to search the records using spring crud repository using LIKE or contains kind of query.
Any suggestions here are highly appreciated!!
Try to add serialization to your redis key and value. This might help :
https://medium.com/#betul5634/redis-serialization-with-spring-redis-data-lettuce-codec-1a1d2bc73d26

Spring Data JDBC One-To-Many with Custom Column Name

I'm using spring-boot-starter-data-jdbc 2.4.2. In my domain aggregate I need to map a List of Strings that is populated from a column in another table. It is a legacy database so I have no control over the table and column names and need to use custom names. I see there is an #MappedCollection annotation, but can't see how to use it in this scenario. Below is my class:
#Data
#Table("NMT_MOVIE_THEATRE")
public class MovieTheatre {
#Id
#Column("MOVIE_THEATRE_ID")
private Long id;
#Column("ZIP_CODE")
private String zipCode;
// this comes from table NMT_CURRENT_MOVIE, column CM_ID, joined by MOVIE_THEATRE_ID
private List<String> currentMovieIds;
}
Using Spring Data JDBC, how can I create the one-to-many relation?
Wrap your String in a little entity.
#Table("NMT_CURRENTMOVIE")
class MovieId {
#Id
#Column("CM_ID")
final String id
// add constructor, equals and hashCode here or generate using Lombok
}
Then use it in the MovieTheatre. Since you don't have a column for an index, the proper collection to use is a Set
// ...
class MovieTheatre {
// ...
#MappedCollection(idColumn="MOVIE_THEATRE_ID")
Set<MovieId> currentMovieIds;
}
Note that equals and hashCode is important as well as the constructor taking all arguments used in those, since the entity is used in a Set.

Difference between MongoSpringData id and _id

I am using MongoDB community edition 4.2 on Windows 10 and inserting some documents. My requirement is to store some documents containing 'id' as field name while MongoDB is replacing it with '_id' and making it the primary key. Is it possible to disable this feature and store key '_id' as the separate autogenerated primary key column and store 'id' as well in separated column which is part of my document?
We are using Spring Data MongoDb Repository entities insertion into MongoDb. What we have observerd the entity having name 'id' as primary key get converted to _id while entities having other name like projectId or something stored as it is with another seperate column as _id generated by MongoDB as primary key. Suggestions are helpful.
Code:
#Autowired
TrackHistoryRepository trackHistoryRepository;
trackHistoryRepository.save(entity);
public interface TrackHistoryRepository extends MongoRepository<OdataEntity, String> {}
Entity:
public class CurrencyEntity implements OdataDeleteableEntity<Long> {
#Id
#GeneratedValue(strategy= GenerationType.IDENTITY)
#Column(name="id")
private Long id;
}
MongoDB can't do that. _id in mongodb is auto-incremented and auto-generated id key for specific record or data field. And _id is primary key by default set by mongodb. You can also enter your user defined key(value) to the _id but you cant change name of field.

Javers: Diff DB entities with data from external source without ids

I'm getting data from an external source with no ids and then I store it in my DB. I'm using Spring with JPA (Hibernate). The ids are autogenerated on the first commit.
The changes in the external data is to be updated every night.
The first commit is trivial. I just create new entities for each new data object and save.
But further on I need to track the changes and just update the existing DB entities that have changed (and add and remove).
It doesn't matter which existing entity is to be updated.
How would I need to approach this using Javers? Is there an example?
Currently I'm thinking about something like:
Get existing entities from the DB.
Create new entities from the external data.
?
The new entities have no id. I would like to use Javers get the diff without needing any id. Diff the collections only based on the values.
Following a sample entity. None of the values apart the id is unique. Only the comparison of all other values can be used to check the identity to an existing entity.
#Entity
#Data
public class TiwHier {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#NotNull
private Long id;
#NotBlank
private String prodId;
private String pnGuid;
private String parentPnGuid;
private String typ;
private String name;
private String text;
private String klasse;
}

Save object in database if it does not already exist (Hibernate & Spring)

I'm working on a Hibernate/Spring application to manage some movies.
The class movie has a many to many relationship with the class genre.
Both of these classes have generated id's using the GeneratedValue annotation.
The genre is saved through the movie object by using #Cascade(CascadeType.SAVE_UPDATE)
I have placed a unique constraint on the genre's type attribute (which is it's name; "Fantasy" for example).
What I would like to do now is have Hibernate check if there is already a genre with type "Fantasy" and if there is, use that genre's id instead of trying to insert a new record.
(The latter would obviously throw an error)
Finally what I need is something like select-before-update but more like select-before-save.
Is there such a function in Hibernate?
Some code:
Movie class
#Entity
public class Movie {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private int id;
private String name;
#Lob
private String description;
#Temporal(TemporalType.TIME)
private Date releaseDate;
#ManyToMany
#Cascade(CascadeType.SAVE_UPDATE)
private Set<Genre> genres = new HashSet<Genre>();
.... //other methods
Genre class
#Entity
public class Genre {
#Column(unique=true)
private String type;
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private int id
....//other methods
You may be over-thinking this. Any select-before-update/select-before-save option is going to result in 2 DB round trips, the first for the select, and the second for the insert if necessary.
If you know you won't have a lot of genres from the outset, you do have a couple of options for doing this in 1 RT most of the time:
The Hibernate second-level cache can hold many if not all of your Genres, resulting in a simple hashtable lookup (assuming a single node) when you check for existence.
You can assume all of your genres are already existing, use session.load(), and handle the new insert as a result of the row not found exception that gets thrown when you reference a genre that doesn't already exist.
Realistically, though, unless you're talking about a LOT of transactions, a simple pre-query before save/update is not going to kill your performance.
I haven't heard of such a function in Hibernate select-before-update/select-before-save
In situations like these you should treat Hibernate as if it was JDBC.
First if you want to know if you even have such a Genre you should query for it.
if you do. then the SAVE_UPDATE will not create a new one when you add it to a movie.
if you don't, Hibernate will create a new Genre row in the database and add the connection to the many_to_many table for you.

Resources