I'm trying to insert some data on H2 in memory database to make some tests and simulate my real Oracle11g but I'm facing an error NULL not allowed for column "ID". I've tried many solutions like putting Id and default on insert query, changing to Mode=LEGACY on jdb-url, but it didn't work.
data.sql
INSERT INTO CSM_SECURITY.csm_person(EMAIL, PERSON_NAME, SYS_ADMIN, DEFAULT_LANGUAGE, WSO2_ID, SYS_ADM_COUNTRY_ID, SYS_ADMIN_COUNTRY)
VALUES ('admin#wso2.com','admin','Y','es-ar',NULL,NULL,NULL);
Person Entity
#Entity
#Table(name = "csm_person", uniqueConstraints = #UniqueConstraint(columnNames = "email"))
public class Person implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "PERSON_SEQ")
#SequenceGenerator(sequenceName = "CSM_PERSON_ID_SEQ", allocationSize = 1, name = "PERSON_SEQ")
private Long id;
#Column(name = "email")
private String email;
#Column(name = "person_name")
private String personName;
...
src/test/resources/application.properties
# SQL properties
spring.jpa.defer-datasource-initialization=true
spring.jpa.database-platform = org.hibernate.dialect.H2Dialect
spring.jpa.show-sql = true
spring.jpa.properties.hibernate.format_sql = true
spring.jpa.properties.hibernate.default_schema = CSM_SECURITY
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.H2Dialect
spring.jpa.properties.hibernate.event.merge.entity_copy_observer = allow
spring.jpa.open-in-view = false
spring.datasource.driver-class-name = org.h2.Driver
spring.sql.init.platform = h2
spring.datasource.name = CSM_SECURITY
spring.datasource.url = jdbc:h2:mem:CSM_SECURITY;MODE=Oracle;DB_CLOSE_DELAY=-1;INIT=CREATE SCHEMA IF NOT EXISTS CSM_SECURITY
spring.sql.init.mode = embedded
spring.datasource.username = sa
pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
Error
Caused by: org.springframework.jdbc.datasource.init.ScriptStatementFailedException:
Failed to execute SQL script statement #1 of
URL [file:/home/ainacio/Development/csm-security/target/test-classes/data.sql]:
INSERT INTO CSM_SECURITY.csm_person(EMAIL, PERSON_NAME, SYS_ADMIN, DEFAULT_LANGUAGE, WSO2_ID, SYS_ADM_COUNTRY_ID, SYS_ADMIN_COUNTRY)
VALUES ('admin#wso2.com','admin','Y','es-ar',NULL,NULL,NULL);
nested exception is org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: NULL not allowed for column "ID"; SQL statement:
For such insertions you need to use identity columns (GenerationType.IDENTITY), but they aren't available in Oracle 11g, only 12c and newer versions support them.
You need to provide a value for ID column but this value must not conflict with values produced by a sequence, for example, you can change start value of a sequence generator to a 100, and use smaller values in your initialization scripts.
Alternatively you can fetch value of a sequence by itself, but Oracle 11g doesn't support sequence value expressions inside insert values or inside subqueries. So if you need to execute the same SQL in H2 and that historic version of Oracle, you need to execute something like that:
INSERT INTO CSM_SECURITY.CSM_PERSON
(ID, EMAIL, PERSON_NAME, SYS_ADMIN, DEFAULT_LANGUAGE,
WSO2_ID, SYS_ADM_COUNTRY_ID, SYS_ADMIN_COUNTRY)
SELECT CSM_SECURITY.CSM_PERSON_ID_SEQ.NEXTVAL,
'admin#wso2.com', 'admin', 'Y', 'es-ar', NULL, NULL, NULL FROM DUAL;
Make sure you're using Oracle compatibility mode of H2, because new versions of H2 don't accept NEXTVAL in Regular mode.
If you use this initilalization script only with H2, that trick with insert from query isn't required:
INSERT INTO CSM_SECURITY.CSM_PERSON
(ID, EMAIL, PERSON_NAME, SYS_ADMIN, DEFAULT_LANGUAGE,
WSO2_ID, SYS_ADM_COUNTRY_ID, SYS_ADMIN_COUNTRY)
VALUES (NEXT VALUE FOR CSM_SECURITY.CSM_PERSON_ID_SEQ,
'admin#wso2.com', 'admin', 'Y', 'es-ar', NULL, NULL, NULL);
Related
I am trying to use TestContainers for my integration tests. What I am trying to do is use sql script to populate with some data, and then also add new data using tests. Below is the test setup:
Integration test where I am trying to get the data inserted through sql scripts:
#SpringBootTest
#AutoConfigureMockMvc
#ExtendWith(SpringExtension.class)
#AutoConfigureTestDatabase(connection = EmbeddedDatabaseConnection.H2)
#TestPropertySource(value = {
"classpath:application-test.properties"
})
class EmployeeDatabaseApplicationTests
#Test
void getEmployeeByEmployeeId() throws Exception {
List<UUID> employeeIds = List.of();
this.mockMvc.perform(get("/admin/employees")
.accept(EmployeeProfileUtil.MEDIA_TYPE_JSON_UTF8)
.contentType(EmployeeProfileUtil.MEDIA_TYPE_JSON_UTF8)
.header("Employee-id", "cc95ccff-8169-4559-9806-1ca4a1db3a19"))
.andExpect(status().is2xxSuccessful());
}
application-test.properties:
spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver
spring.datasource.url=jdbc:tc:postgresql://employee
spring.jpa.hibernate.ddl-auto = create
spring.liquibase.contexts=test
spring.liquibase.change-log=classpath:/db-test.changelog/db.changelog-test-master.yaml
db.changelog-test-master.yaml
databaseChangeLog:
- includeAll:
path: classpath*:db-test.changelog/changes/
changes folder has 2 sql files, one creates the database schema and the other populates some of the created schema. ex:
CREATE TABLE IF NOT EXISTS employee (
id int8 generated by default as identity,
date_of_birth varchar(255),
deleted boolean not null,
employee_id uuid,
gender varchar(255),
phone varchar(255),
name_id int8,
primary key (id)
);
INSERT INTO employee (id, date_of_birth, deleted, employee_id, gender, phone, name_id) values (1, '2010-02-02', false, 'cc95ccff-8169-4559-9806-1ca4a1db3a19',
'female', '5561132977', 1);
The change log is getting picked as I can see in logs:
liquibase.changelog : Custom SQL executed
liquibase.changelog : ChangeSet db-test.changelog/changes/employee-create-tables-20220810.sql::raw::includeAll ran successfully in 22ms
liquibase.changelog : ChangeSet db-test.changelog/changes/employee-create-tables-20221010.sql::raw::includeAll ran successfully in 6ms
To summarize:
I want to query the database using employe_id- cc95ccff-8169-4559-9806-1ca4a1db3a19, as I(think)am inserting this to db, which currently doesn't return the data.
Thanks for the help!
Liquibase is already being used to create perform the scripts. For that reason the following property should be spring.jpa.hibernate.ddl-auto=none at application-tests.properties in order to override the one in application.properties.
I already saw you fixed in the code provided but #AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
We have a requirement where 3k to 4k records we need to save into database. User will add multiple items or select all to add to his watchlist after applying filter. I tried savellAndFlush(list) this is taking list of objects but while saving only one record is getting saved into database.
We are using Spring boot and JPA repository
Domain:
#Id
#GeneratedValue(generator = "system-uuid")
#GenericGenerator(name = "system-uuid", strategy = "org.hibernate.id.UUIDGenerator")
#Column(name = "PK_WISHLIST_ID", unique = true, nullable = false, precision = 30, scale = 0)
private String pkWishListId;
List<WatchList> watchList = new ArrayList<WatchList>();
watchListRepo.saveAllAndFlush(watchList);
This is executing without any issues but in database its not inserting all records, instead inserting only one record.
I am creating a service which writes directly to a snowflake database.
I am having a lot of trouble trying to get spring data jpa to work effectively with Snowflake. My main issue is that I am unable to save an entity to the Snowflake DB through Jpa Repository interface Save method. Because this application is being used to dump data into Snowflake, being able to leverage JPA would make life a lot easier.
I would prefer not to have to roll my own native queries so my question is whether it's possible to leverage Hibernate when working with Snowflake.
The main thing I want to be able to do is persist entities using the Jpa Repositories inbuild Save method.
Below is my current configuration. Any ideas on what could be improved in the configuration to get this working would be appreciated, or also any opinion on whether it is possible or not.
spring:
profiles:
active: local
application:
name: Service
datasource:
driverClassName: net.snowflake.client.jdbc.SnowflakeDriver
url: ${SPRING_DATASOURCE_URL}
username: ${SPRING_DATASOURCE_USERNAME}
password: ${SPRING_DATASOURCE_PASSWORD}
flyway:
locations: classpath:db/migration/common,classpath:db/migration/snowflake
jpa:
properties:
hibernate:
dialect: org.hibernate.dialect.SQLServerDialect
order_inserts: true
create sequence award_event_id_seq;
create table award_event
(
id INT NOT NULL DEFAULT award_event_id_seq.nextval PRIMARY KEY,
event_source_system varchar not null,
event_trigger VARCHAR NOT NULL,
event_triggered_by VARCHAR NOT NULL,
event_timestamp TIMESTAMP NOT NULL
)
#Entity(name = "award_event")
#SequenceGenerator(name = "award_event_id_seq", sequenceName = "award_event_id_seq", allocationSize = 1)
data class AwardEvent(
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
val id: Int = -1,
val eventTrigger: String,
val eventTriggeredBy: String,
val eventTimestamp: LocalDateTime,
val eventSourceSystem: String
)
override fun receiveMessage(message: String) {
logger.info("Receiving award event: $message")
val awardEvent: AwardEventMessage = message.toObject()
// This Save method does not work and throws an error specified below
awardEventRepository.save(awardEvent.toAwardEvent())
}
2021-01-08 10:49:28.163 ERROR 3239 --- [nio-9106-exec-1] o.hibernate.id.enhanced.TableStructure : could not read a hi value
net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
syntax error line 1 at position 50 unexpected 'with'.
syntax error line 1 at position 72 unexpected ')'.
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:124)
at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:64)
at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:434)
at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:338)
at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:506)
at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:233)
at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:171)
at net.snowflake.client.core.SFStatement.execute(SFStatement.java:754)
at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:245)
at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:117)
Just as a follow up, I was unable to get the application up and running using the approach I outlined above. I am still unsure why but think it may have been to do with a lack of support for snowflake sequences as the generation type for the primary key in spring.
I changed the generation type to UUID and the application started to work as expected in turn. There was no requirements for what type of primary key was needed so this approach was satisfactory.
create sequence award_event_id_seq;
create table award_event
(
id varchar not null constraint award_event_pkey primary key,
event_source_system varchar not null,
event_trigger varchar not null,
event_triggered_by varchar not null,
event_timestamp timestamp not null
)
#Entity(name = "award_event")
data class AwardEvent(
#Id
#GeneratedValue
#Type(type = "uuid-char")
val id: UUID = UUID.randomUUID(),
val eventTrigger: String,
val eventTriggeredBy: String,
val eventTimestamp: LocalDateTime,
val eventSourceSystem: String
)
With oracle JDBC driver (ojdbc7.jar), when I do x=Resultset.getString("COLUMN_DEF") for a column where the default value is 'N/A' in the database ('N/A' set at table creation, 'N/A' seen with DBeaver tool) the JDBC driver return x="'n/a'" (with postgres and mysql it returns x="N/A").
do you have an idea of why it is in lower case et why it is quoted inside the result string?
thanks in advance for any kind of help on this issue!
PS: how I use the database metadata object:
private static void readColumnMetaData(AMIDBLoader dbLoader, DatabaseMetaData metaData, String internalCatalog, String externalCatalog, String _table, Map<String, String> amiEntityMap, Map<String, String> amiFieldMap) throws SQLException
{
try(ResultSet resultSet = metaData.getColumns(internalCatalog, internalCatalog, _table, "%"))
{
while(resultSet.next())
{
String table = resultSet.getString("TABLE_NAME");
String name = resultSet.getString("COLUMN_NAME");
String type = resultSet.getString("TYPE_NAME");
int size = resultSet.getInt("COLUMN_SIZE");
int digits = resultSet.getInt("DECIMAL_DIGITS");
String def = resultSet.getString("COLUMN_DEF");
code for the table creation:
CREATE TABLE "router_locations" (
"id" NUMBER(*, 0),
"continentCode" VARCHAR2(3) DEFAULT 'N/A',
"countryCode" VARCHAR2(3) DEFAULT 'N/A'
);;
Jerome
We managed to find where the string was modified and the JDBC driver is ok .. thanks for your help.
I'm trying the following unit test:
#Test
#Transactional
public void thatFolderLocationAssociationTableIsWorking() {
Location l1 = new DomesticLocation();
l1.setDeptName("Test name 1");
Location l2 = new DomesticLocation();
l2.setDeptName("Test name 2");
KMLFolder k1 = new KMLFolder();
k1.setName("Test name 1");
KMLFolder k2 = new KMLFolder();
k1.setName("Test name 2");
List<Location> locations = new ArrayList<Location>();
locations.add(l1);
locations.add(l2);
k1.setLocations(locations);
kmlFolderServiceImpl.save(k1);
assertEquals("Test name 1", kmlFolderServiceImpl.find(1L).getLocations().get(0).getDeptName());
assertEquals("Test name 2", kmlFolderServiceImpl.find(1L).getLocations().get(1).getDeptName());
//The following line gets the NPE
assertEquals("Test name 1", locationServiceImpl.find(1L).getKmlFolderList().get(0).getName());
}
I'm getting NPEs on the laster assertions where I'm trying to retrieve KMLFolder.getName() from the Locations The other assertions are working, where I get the Location name from the KMLFolder.
Here are my JPA definitions:
#ManyToMany(mappedBy="kmlFolderList", cascade=CascadeType.ALL)
private List<Location> locations = new ArrayList<Location>();
#ManyToMany
#JoinTable(name="LOCATION_KMLFOLDER",
joinColumns={#JoinColumn(name="KMLFOLDER_ID", referencedColumnName="ID")},
inverseJoinColumns={#JoinColumn(name="LOCATION_ID", referencedColumnName="ID")}
)
The appropriate table is being created when I run the test. Here's the console output:
Hibernate:
create table project.LOCATION_KMLFOLDER (
KMLFOLDER_ID bigint not null,
LOCATION_ID bigint not null
) ENGINE=InnoDB
...
Hibernate:
alter table project.LOCATION_KMLFOLDER
add index FK_lqllrwb2t5cn0cbxxx3ms26ku (LOCATION_ID),
add constraint FK_lqllrwb2t5cn0cbxxx3ms26ku
foreign key (LOCATION_ID)
references project.KMLFolder (id)
Hibernate:
alter table .LOCATION_KMLFOLDER
add index FK_ckj00nos13yojmcyvtefgk9pl (KMLFOLDER_ID),
add constraint FK_ckj00nos13yojmcyvtefgk9pl
foreign key (KMLFOLDER_ID)
references project.Locations (id)
The console does not show inserts in to the LOCATION_KNLFOLDER table as I expect. Any thoughts on why this may be happening?
You're initializing the inverse side of the association, that Hibernate ignores, instead of (or in addition to) initializing the owner side of the association, that Hibernate doesn't ignore.
The owner side is the side without the mappedBy attribute.