Im trying to use hibernate envers..
I have tables annotated with #Audited, but problem occurs with hibernate_sequence..
CREATE SEQUENCE IF NOT EXISTS custom_schema.hibernate_sequence START 1 INCREMENT 1;
So Im using custom_schema and obviously problem is
ERROR: relation "hibernate_sequence" does not exist
So Im asking if it is possible to somehow tell hibernate in which schema this sequence is?
Ok, solved by creating own revisionentity
#RevisionEntity
#Entity
#Table(name = "revinfo", schema = "custom_schema")
class RevInfo (
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#RevisionNumber
var id: Long? = null,
#RevisionTimestamp
var timestamp: Long = 0
)
Related
I am new in MongoDB and trying to execute a very simple query to save collection to the database, but receive the error Bulk write operation error on server.
Entity:
#Document("role")
#Data
#NoArgsConstructor
#RequiredArgsConstructor
#EqualsAndHashCode(exclude = "id") <--------- compare only by unique field `name`
public class Role {
#Id
private String id;
#NotBlank
#NonNull
#Indexed(unique = true)
private ERole name;
}
ERole:
public enum ERole {ROLE_USER, ROLE_MODERATOR, ROLE_ADMIN}
logic:
Collection<Role> rolesFromDb = repository.findAll(); < --------- (1)
Collection<Role> rolesFromEnumSet = Arrays.stream(ERole.values()).map(Role::new).collect(Collectors.toSet());
if (!(rolesFromDb.containsAll(rolesFromEnumSet) &&
rolesFromEnumSet.containsAll(rolesFromDb))
) {
rolesFromEnumSet.removeAll(rolesFromDb); < ------------------ (2)
repository.saveAll(rolesFromEnumSet); < --------------------- (3)
}
I have a single record in the database and receive the next collection in rolesFromDatabase variable (line (1):
Role(id=63dc16b253565a43cee65848, name=ROLE_USER)
In line (2) a rolesFromEnuSet has an expected set:
0 = {Role#9387} "Role(id=null, name=ROLE_ADMIN)"
1 = {Role#9388} "Role(id=null, name=ROLE_MODERATOR)"
So, the database doesn't contain remembered records.
Why do I receive a "Duplicate error" in line (3)?
Your EqualsAndHashCode is comparing by ID, not name. This means that the rolesFromEnumSet.removeAll(rolesFromDb)) code is not removing anything in rolesFromEnumSet
This means that inside your transaction, you are actually left with 4 records:
Role(id=63dc16b253565a43cee65848, name=ROLE_USER)
Role(id=null, name=ROLE_USER)
Role(id=null, name=ROLE_MODERATOR)
Role(id=null, name=ROLE_ADMIN)
You'll receive a duplicate key error because it tries to save an additional role with the name ROLE_USER
I am creating a service which writes directly to a snowflake database.
I am having a lot of trouble trying to get spring data jpa to work effectively with Snowflake. My main issue is that I am unable to save an entity to the Snowflake DB through Jpa Repository interface Save method. Because this application is being used to dump data into Snowflake, being able to leverage JPA would make life a lot easier.
I would prefer not to have to roll my own native queries so my question is whether it's possible to leverage Hibernate when working with Snowflake.
The main thing I want to be able to do is persist entities using the Jpa Repositories inbuild Save method.
Below is my current configuration. Any ideas on what could be improved in the configuration to get this working would be appreciated, or also any opinion on whether it is possible or not.
spring:
profiles:
active: local
application:
name: Service
datasource:
driverClassName: net.snowflake.client.jdbc.SnowflakeDriver
url: ${SPRING_DATASOURCE_URL}
username: ${SPRING_DATASOURCE_USERNAME}
password: ${SPRING_DATASOURCE_PASSWORD}
flyway:
locations: classpath:db/migration/common,classpath:db/migration/snowflake
jpa:
properties:
hibernate:
dialect: org.hibernate.dialect.SQLServerDialect
order_inserts: true
create sequence award_event_id_seq;
create table award_event
(
id INT NOT NULL DEFAULT award_event_id_seq.nextval PRIMARY KEY,
event_source_system varchar not null,
event_trigger VARCHAR NOT NULL,
event_triggered_by VARCHAR NOT NULL,
event_timestamp TIMESTAMP NOT NULL
)
#Entity(name = "award_event")
#SequenceGenerator(name = "award_event_id_seq", sequenceName = "award_event_id_seq", allocationSize = 1)
data class AwardEvent(
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
val id: Int = -1,
val eventTrigger: String,
val eventTriggeredBy: String,
val eventTimestamp: LocalDateTime,
val eventSourceSystem: String
)
override fun receiveMessage(message: String) {
logger.info("Receiving award event: $message")
val awardEvent: AwardEventMessage = message.toObject()
// This Save method does not work and throws an error specified below
awardEventRepository.save(awardEvent.toAwardEvent())
}
2021-01-08 10:49:28.163 ERROR 3239 --- [nio-9106-exec-1] o.hibernate.id.enhanced.TableStructure : could not read a hi value
net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
syntax error line 1 at position 50 unexpected 'with'.
syntax error line 1 at position 72 unexpected ')'.
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:124)
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:64)
at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:434)
at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:338)
at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:506)
at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:233)
at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:171)
at net.snowflake.client.core.SFStatement.execute(SFStatement.java:754)
at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:245)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:117)
Just as a follow up, I was unable to get the application up and running using the approach I outlined above. I am still unsure why but think it may have been to do with a lack of support for snowflake sequences as the generation type for the primary key in spring.
I changed the generation type to UUID and the application started to work as expected in turn. There was no requirements for what type of primary key was needed so this approach was satisfactory.
create sequence award_event_id_seq;
create table award_event
(
id varchar not null constraint award_event_pkey primary key,
event_source_system varchar not null,
event_trigger varchar not null,
event_triggered_by varchar not null,
event_timestamp timestamp not null
)
#Entity(name = "award_event")
data class AwardEvent(
#Id
#GeneratedValue
#Type(type = "uuid-char")
val id: UUID = UUID.randomUUID(),
val eventTrigger: String,
val eventTriggeredBy: String,
val eventTimestamp: LocalDateTime,
val eventSourceSystem: String
)
I need to retrieve a list of Category from the DB on the basis of value of column called owner. Here is my Category -
#Entity
#Table(name = "categories")
class Category(#Column(name = "category_id", nullable = false)
#Id #GeneratedValue(strategyGenerationType.AUTO)
var id: Long = 0,
#Column(name = "category_owner", nullable = false)
#field:NotNull(message = "Please assign an owner")
var owner: Long?,
#Column(name = "category_name", nullable = false)
#field:NotEmpty(message = "Please assign a name")
var name: String?)
Here is my interface which defines the function findByOwner -
interface CategoryRepository: JpaRepository<Category, Long> {
fun findByOwner(categoryOwner: Long): List<Category>
}
However, when I call the method, I get no response. I have made sure that the DB has correct data and I'm providing the correct owner Id. Have even invalidated the cache etc. What could be going wrong?
EDIT:
After spring.jpa.show-sql=true -
findAll()
Hibernate: select category0_.category_id as category1_0_, category0_.category_name as category2_0_, category0_.category_owner as category3_0_ from categories category0_
findByOwner()
Hibernate: select category0_.category_id as category1_0_, category0_.category_name as category2_0_, category0_.category_owner as category3_0_ from categories category0_ where category0_.category_owner=?
EDIT 2:
Turns out that my implementation was fine all along. The bug was in my service.
Create your named method according with the name of the column.
fun findByCategoryOwner(categoryOwner: Long): List<Category>
Or use #Query
#Query("SELECT * FROM categories WHERE category_owner = ?1", nativeQuery = true)
fun findByOwner(cateogryOwner: Long): List<Category
Can you put a breakpoint in org.springframework.data.jpa.repository.query.JpaQueryExecution class and when you execute findByOwner, it will come here.
When it reaches this breakpoint, select the query.createQuery(accessor).getResultList() and evaluate to see what value is returned by hibernate for spring-data-jpa to use
This post should help you. It appears to be happeing because of the parameter name mismatch.
Use camelCase to name your variables in Entity class then jpa will auto recognise the column name
findByCategoryOwner(String categoryOwner)
If you still wish to have underscore in your column names then try this
findByCategory_Owner(String categoryOwner)
I haven't tried the second option though
At least in java you need to provide the id in the method name:
**fun findByOwner_Id(categoryOwner: Long): List<Category>**
So change it from findByOwner -> findByOwnerId.
I've an entity defined like below. If I use save() Hibernate does not create a new index for newly created entity. Updating/modifying an existing entity works well and as expected.
I'm using kotling with spring boot 2.
#Entity(name = "shipment")
#Indexed
data class Shipment(
#Id #GeneratedValue(strategy = GenerationType.IDENTITY) val id: Long = -1,
#JoinColumn(name = "user") #ManyToOne() var user: User?,
#IndexedEmbedded
#JoinColumn(name = "sender") #ManyToOne(cascade = [CascadeType.ALL]) val sender: Contact,
#IndexedEmbedded
#JoinColumn(name = "sender_information") #ManyToOne(cascade = [CascadeType.ALL]) val senderInformation: ShipmentInformation,
) {}
Save function, I'm using this same function to update my entity and index is updated if index exists.
#Transactional
fun save(user: User, shipment: Shipment): Shipment {
shipment.user = user;
return this.shipmentRepository.save(shipment)
}
application.properties
spring.jpa.properties.hibernate.search.default.directory_provider=filesystem
spring.jpa.properties.hibernate.search.default.indexBase=./lucene/
spring.jpa.open-in-view=false
If I restart the server, indexing manually works too.
#Transactional
override fun onApplicationEvent(event: ApplicationReadyEvent) {
val fullTextEntityManager: FullTextEntityManager = Search.getFullTextEntityManager(entityManager)
fullTextEntityManager.createIndexer().purgeAllOnStart(true)
fullTextEntityManager.createIndexer().optimizeAfterPurge(true)
fullTextEntityManager.createIndexer().batchSizeToLoadObjects(15)
fullTextEntityManager.createIndexer().cacheMode(CacheMode.IGNORE)
fullTextEntityManager.createIndexer().threadsToLoadObjects(2)
fullTextEntityManager.createIndexer().typesToIndexInParallel(2)
fullTextEntityManager.createIndexer().startAndWait()
return
}
I tried to force to use JPA transaction manager but It did not help me.
#Bean(name = arrayOf("transactionManager"))
#Primary
fun transactionManager(#Autowired entityManagerFactory: EntityManagerFactory): org.springframework.orm.jpa.JpaTransactionManager {
return JpaTransactionManager(entityManagerFactory)
}
Update
I think I found why I don't get the results of newly inserted entities.
My search query has a condition on "pid" field which is declared:
#Field(index = Index.YES, analyze = Analyze.NO, store = Store.NO)
#SortableField
#Column(name = "id", updatable = false, insertable = false)
#JsonIgnore
#NumericField val pid: Long,
and query:
query.must(queryBuilder.keyword().onField("customer.pid").matching(user.customer.id.toString()).createQuery())
pid is not stored and so newly inserted values are not visible. Can this be the cause?
BTW: How can I query/search by nested indexed document id? In my case it is customer.id which is DocumentId. I've tried to change the query like below but don't get any result, should I create a new field to query?
query.must(queryBuilder.keyword().onField("customer.id").matching(user.customer.id.toString()).createQuery())
Update 2
I found a solution and now getting the newly inserted datas too. There was an error with definition of "pid" field and I've defined my Fields as below and it works as expected.
#Fields(
Field(name = "pid", index = Index.YES, analyze = Analyze.YES, store = Store.NO)
)
#SortableField(forField = "pid")
#Id #GeneratedValue(strategy = GenerationType.IDENTITY) val id: Long?,
Can we search and sort by id in an easy way or is it the best practice? I know that we should use native JPA functions to get results by id but in my case I need to search by an embedded id to restrict search results. (depends on role of user) so therefore it is not an option for me.
And I don't understand why manual indexing works...
BTW: How can I query/search by nested indexed document id? In my case it is customer.id which is DocumentId. I've tried to change the query like below but don't get any result, should I create a new field to query?
Normally you don't need to create a separate field if all you want is to perform an exact match.
Can we search and sort by id in an easy way
Searching, yes, at least in Hibernate Search 5.
Sorting, no: you need a dedicated field.
or is it the best practice?
The best practice is to declare a field alongside your #DocumentId if you need anything more complex than an exact match on the ID.
I know that we should use native JPA functions to get results by id
I'm not sure I understand what you mean by "native JPA functions".
but in my case I need to search by an embedded id to restrict search results. (depends on role of user)
Yes, this should work. That is, it should work if the id is properly populated.
And I don't understand why manual indexing works...
Neither do I, but I suppose the explanation lies in the "error in the definition of "pid" field". Maybe the ID wasn't populated properly in some cases, leading to the entity being considered as deleted by Hibernate Search?
If you need me to give you a definitive answer, the best way to get it would be to create a reproducer. You can use this as a template: https://github.com/hibernate/hibernate-test-case-templates/tree/master/search
This looks odd:
#Id #GeneratedValue(strategy = GenerationType.IDENTITY) val id: Long = -1,
I'd expect a nullable long, initialized to null (or whatever is the Kotlin equivalent).
I'm not sure this is the problem, but I imagine it could be, as a non-null ID is generally only expected from an already persisted entity.
Other than that, I think you're on the right track: if mass indexing works but not automatic indexing, it may have something to do with your changes not being executed in database transactions.
I'm importing historical football (or soccer, if you're from the US) data into a Neo4j database using a spring boot application (2.1.6.RELEASE) with the spring-boot-starter-data-neo4j dependency and a standalone, locally running 3.5.6 Neo4j database server.
But for some reason searching for an entity by a simple property and an attached, referenced entity, does not work, althought the relation is present in the database.
This is the part of the model, that is currently giving me a headache:
#NodeEntity(label = "Season")
open class Season(
#Id
#GeneratedValue
var id: Long? = null,
#Index(unique = true)
var name: String,
var seasonNumber: Long,
#Relationship(type = "IN_LEAGUE", direction = Relationship.OUTGOING)
var league: League?,
var start: LocalDate,
var end: LocalDate
)
#NodeEntity(label = "League")
open class League(
#Id
#GeneratedValue
var id: Long? = null,
#Index(unique = true)
var name: String,
#Relationship(type = "BELONGS_TO", direction = Relationship.OUTGOING)
var country: Country?
)
(I left out the Country class, as I'm pretty sure that it is not part of the problem)
To allow running the import more than once, I want to check if the corresponding entity is already present in the database and only import newer ones. So I added the following method SeasonRepository:
open class SeasonRepository : CrudRepository<Season, Long> {
fun findBySeasonNumberAndLeague(number: Long, league: League): Season?
}
But it is giving me a null result instead of the existing entity on consecutive runs, hence I get duplicates in my database.
I would have expected spring-data-neo4j to reduce the passed League to its Id and then have a generated query that looks somewhat like this:
MATCH (s:Season)-[:IN_LEAGUE]->(l:League) WHERE id(l) = {leagueId} AND s.seasonNumber = {seasonNumber} WITH s MATCH (s)-[r]->(o) RETURN s,r,o
but when I turn on finer logging on the neo4j package I see this output in the log file:
MATCH (n:`Season`) WHERE n.`seasonNumber` = { `seasonNumber_0` } AND n.`league` = { `league_1` } WITH n RETURN n,[ [ (n)-[r_i1:`IN_LEAGUE`]->(l1:`League`) | [ r_i1, l1 ] ] ], ID(n) with params {league_1={id=30228, name=1. Bundesliga, country={id=29773, name=Deutschland}}, seasonNumber_0=1}
So for some reason, spring-data seems to think, that the league property is a simple / primitive property and not a full releation, that needs to be resolved by the id (n.league= {league_1}).
I only got it to work, by passing the id of the league, and providing a custom query using the #Query annotation but I actually thought, that it would work with spring-data-neo4j out of the box.
Any help appreciated. Let me know if you need more details.
Spring Data Neo4j does not support objects as parameters at the moment. It is possible to query for properties on related entities/nodes e.g. findBySeasonNumberAndLeagueName if this is a suitable solution.