I'm upgrading a Spring Boot 2.6.9 application to the 2.7.x line (2.7.1). The application tests use H2 with MS SQL Server compatibility mode.
I've created a simple sample project to reproduce this issue: https://github.com/codependent/boot-h2
Branches:
main: Spring Boot 2.7.1 - Tests KO
boot26: Spring Boot 2.6.9 - Tests OK
To check the behaviour just run ./mvnw clean test
These are the relevant parts of the code:
Test application.yml
spring:
r2dbc:
url: r2dbc:h2:mem:///testdb?options=MODE=MSSQLServer;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
schema.sql
CREATE SCHEMA IF NOT EXISTS [dbo];
CREATE TABLE IF NOT EXISTS [dbo].[CUSTOMER] (
id INTEGER GENERATED BY DEFAULT AS IDENTITY,
name VARCHAR(255) NOT NULL,
CONSTRAINT PK__otp__D444C58FB26C6D28 PRIMARY KEY (id)
);
Entity
#Table("[dbo].[CUSTOMER]")
#Data
#NoArgsConstructor
#AllArgsConstructor
public class CustomerEntity {
#Id
#Column("id")
private Integer id;
#Column("name")
private String name;
}
Data R2DBC Repository
public interface CustomerRepository extends ReactiveCrudRepository<CustomerEntity, Integer> {
}
The problem occurs when invoking customerRepository.findById(xxx), as can be seen in the following test
#SpringBootTest
#RequiredArgsConstructor
#TestConstructor(autowireMode = ALL)
class BootH2ApplicationTests {
private final CustomerRepository customerRepository;
#Test
void shouldSaveAndLoadUsers() {
CustomerEntity joe = customerRepository.save(new CustomerEntity(null, "Joe")).block();
customerRepository.findById(joe.getId()).block();
Exception:
Caused by: io.r2dbc.spi.R2dbcBadGrammarException:
Syntax error in SQL statement "SELECT [dbo].[CUSTOMER].* FROM [dbo].[CUSTOMER] WHERE [dbo].[CUSTOMER].id = $1 [*]LIMIT 2"; SQL statement:
SELECT [dbo].[CUSTOMER].* FROM [dbo].[CUSTOMER] WHERE [dbo].[CUSTOMER].id = $1 LIMIT 2 [42000-214]
The R2dbcEntityTemplate is limiting the selectOne query to 2 elements:
public <T> Mono<T> selectOne(Query query, Class<T> entityClass) throws DataAccessException {
return (Mono)this.doSelect(query.getLimit() != -1 ? query : query.limit(2), entityClass, this.getTableName(entityClass), entityClass, RowsFetchSpec::one);
}
And this is translated into a LIMIT N clause which is not supported by H2/SQL Server.
Not sure if it's some kind of H2/Spring Data bug or there's a way to fix this.
It will be solved in Spring Data 2.4.3 (2021.2.3)
Related
I'm trying to launch a SQL file at my database initialization.
Here is my configuration:
spring:
profiles: local
jpa:
properties:
hibernate.temp.use_jdbc_metadata_defaults: false
generate-ddl: true
hibernate:
ddl-auto: update
database: h2
show-sql: true
autoCommit: false
datasource:
platform: h2
url: jdbc:h2:mem:db;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=false;
driver-class-name: org.h2.Driver
initialization-mode: always
data: classpath:/sql/CreateGeographicZones.sql
My script is just this line (atm):
INSERT INTO GEOGRAPHIC_ZONE (name) VALUES ('EUROPE');
And the related entity:
#NoArgsConstructor
#Data
#Entity
#Table(name = "GEOGRAPHIC_ZONE")
public class GeographicZone {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "geo_zone_sequence")
#SequenceGenerator(name = "geo_zone_sequence", sequenceName = "geo_zone_id_seq", allocationSize = 1)
#Column(nullable = false)
private Long id;
...
}
The table is created as I can see in the logs:
Hibernate: create table geographic_zone (id bigint not null, name varchar(100) not null, primary key (id))
But I have an SQL error when the script is executed:
Table "GEOGRAPHIC_ZONE" not found; SQL statement:
INSERT INTO GEOGRAPHIC_ZONE (name) VALUES ('EUROPE')
In the logs I can see that my table is created before the script execution, so why it's not working ?
According with your entity's metadata Hibernate is querying geo_zone_id_seq sequence's next value and using it for the ID on each insert.
If you would like to use the same approach when inserting directly in your database then you will need to implement a H2 Trigger
Also you may use either the EntityManager bean or your Spring JPA Repository to insert your data after application startup via CommandLineRunner interface.
Using EntityManager:
#Bean
CommandLineRunner registerZonesDataRunner(EntityManager entityManager, TransactionTemplate transactionTemplate) {
return args -> transactionTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
// presuming that GeographicZone has a constructor expecting NAME
Stream.of("AFRICA", "EUROPE")
.map(GeographicZone::new)
.forEach(entityManager::persist);
}
});
Using Spring JPA Repository:
#Bean
CommandLineRunner registerZonesDataRunner(GeographicZoneRepository repository) {
// presuming that GeographicZone has a constructor expecting NAME
return args -> repository.saveAll(Stream.of("AFRICA", "EUROPE")
.map(GeographicZone::new)
.collector(Collectors.toList()));
}
minimal, reproducible example
You don't show how you've defined the id column but the schema indicates there is no auto-generation scheme. So, try:
INSERT INTO GEOGRAPHIC_ZONE (id, name) VALUES (1, 'EUROPE');
in your data file. If that works, you'll need to either manually set the id in your inserts or add something like #GeneratedValue(strategy = AUTO) to your #Id property.
I am using simple Entity with 2 cols only. In my table PK col is varchar in db but in actual values stored is numerical in that col and other col is int. DB is MS SQL Server. And this table has 165 million records.
Table structure:
SECURITY_TAB
varchar(30) SECID PK;
int VERSION;
Entity
#Entity
#Table(name = "SECURITY_TAB")
public class Security {
#Id
#Column
private String secid;
#Column
private int version;
//... getter n setter n toString
}
Repository
public interface SecurityRepository extends JpaRepository<Security, String> {
#Query("SELECT s from Security s where secid= :secid")
Security findSecurityBySecid(#Param("secid") String secid)); //this is slow
//below is fine.
//#Query("SELECT s from Security s where secid='1111'")
//Security findSecurityBySecid();
}
TestClass
#RunWith(SpringRunner.class)
#SpringBootTest
public class SecurityTests {
#Autowired
private SecurityRepository securityRepository;
#Test
public void testSecurityBySecid() {
Instant start = Instant.now();
Security security = securityRepository.findSecurityBySecid("1111");
System.out.println(Duration.between(start, Instant.now()));
System.out.println(security);
System.out.println(Duration.between(start, Instant.now()));
}
}
This simple query is taking more than 20 secs,
While when I run similar query MS SQL Server Mgmt Studio or hard code the value in the query result is returned in mill seconds. So what is going wrong?
was able to solve:
The solution was to setup this property in the jdbc url : sendStringParametersAsUnicode=false
Full example if you are using MS SQL official driver : jdbc:sqlserver://yourserver;instanceName=yourInstance;databaseName=yourDBName;sendStringParametersAsUnicode=false;
Solution: sql server query running slow from java
I have the following simple entity FileRegistry :
#Getter
#Setter
#Entity
#ToString
#Table(name = "file_store")
public class FileRegistry {
#Id
private String name;
/**
* Creation timestamp of the registry
* This value is automatically set by database, so setter method
* has been disabled
*/
#Setter(AccessLevel.NONE)
#Column(insertable = false, updatable = false)
private LocalDateTime creationDate;
}
The following FileRepository DAO:
#Repository
public interface FileRepository extends JpaRepository<FileRegistry, String> { }
and the following Spring Boot test :
#SpringBootTest(classes=PersistTestConfig.class, properties = { "spring.config.name=application,db"})
#ActiveProfiles("test")
#Transactional
public class FileRepositoryTest {
#Autowired
FileRepository fileRepository;
#Test
void insertFileTest() {
assertNotNull(fileRepository, "Error initializing File repository");
// Check registry before insertion
List<FileRegistry> allFiles = fileRepository.findAll();
assertNotNull(allFiles, "Error retrieving files from registry");
assertThat(allFiles.size(), is(0));
// Insert file
FileRegistry fileRegistry = new FileRegistry();
fileRegistry.setName("Test");
fileRepository.save(fileRegistry);
// Check that the insertion was successful
allFiles = fileRepository.findAll();
assertNotNull(allFiles, "Error retrieving files from registry");
assertThat(allFiles.size(), is(1));
assertEquals("File registry name mismatch", "Test", allFiles.get(0).getName());
System.out.println(allFiles.get(0));
}
}
Persistence configuration class defined as follows :
#Configuration
#EnableAutoConfiguration
#EnableTransactionManagement
#EnableJpaRepositories
public class PersistTestConfig {
}
The table file_store defined in H2 as :
CREATE TABLE file_store (name VARCHAR NOT NULL, creation_date TIMESTAMP(3) DEFAULT NOW() NOT NULL, CONSTRAINT file_store_pk PRIMARY KEY (name));
Everything works fine except that when I use #Transactional at test level (mainly to benefit from rollbacks i.e. db cleanup on each test) a null value is fetched for the creationDate field :
FileRegistry(name=Test, creationDate=null)
When I remove #Transactional from the test class, the fetched value contains the date as computed by H2 :
FileRegistry(name=Test, creationDate=2019-03-07T17:08:13.392)
I've tried to flush and merge manually the instance to no avail. To be honest, right now I'm a little bit lost on how #Transactional really works, in fact reading the docs and inspecting the code, the underlying JpaRepository implementation (SimpleJpaRepository) is annotated as #Transactional(readOnly = true).
A little help on this subject would be very appreciated.
Ok, figured it out.
Simply issuing a refresh entityManager.refresh(allFiles.get(0)); solves the issue.
I tested also using Hibernate's #Generated(INSERT) specific annotation in the entity creationDate field and it also worked fine.
By the way I've eventually decided to drop this thing in favor of using Spring Data's JpaAuditing features and annotating the field with #CreatedDate annotation to fill the value instead of relying on DB date (by the way, production-wise, you probably shouldn't rely on DB time). To me this is feels more, let's say, "correct" and springy way of doing things.
I am developing a new application using Spring Boot 2.0.0.M6 and Spring Data JPA. I am using MariaDB v10.
Below is my dev properties file.
spring.jpa.hibernate.ddl-auto=create-drop
spring.datasource.url=jdbc:mariadb://localhost:3306/testdb
spring.datasource.username=user
spring.datasource.password=
spring.jpa.show-sql=true
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
org.hibernate.dialect.Dialect=MariaDB53Dialect
spring.jooq.sql-dialect=MariaDB53Dialect
I get output:
Hibernate: create table hibernate_sequence (next_val bigint) engine=MyISAM
I am not able to change the storage engine. All the tables are being created using storage engine MyISAM.
I am able to create tables manually using other storage engines. But for some reason Spring or Hibernate falls back to MyISAM engine only.
With pure Hibernate-Java application, Hibernate uses InnoDB as default.
INFO: HHH000412: Hibernate Core {5.2.11.Final}
Hibernate: create table hibernate_sequence (next_val bigint) engine=InnoDB
Is there any way to override Database storage engine from the Spring Boot properties?
As described in Spring Boot's documentation, all properties prefixed with spring.jpa.properties are passed through to the underlying JPA provider (Hibernate in this case) with the prefix removed.
The Hibernate property to configure the dialect is hibernate.dialect and its value should be the fully qualified class name of the dialect that you want to use. In this case that's org.hibernate.dialect.MariaDB53Dialect.
Putting the above together, you could set the following property in your application.properties:
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MariaDB53Dialect
With this in place your Spring Boot-based application uses the MariaDB dialect:
2017-11-09 14:18:17.557 INFO 69955 --- [ost-startStop-1] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MariaDB53Dialect
With Hibernate 5.2.12, if I run the MySQLStoredProcedureTest while setting the dialect to MariaDB:
#RequiresDialect(MariaDB53Dialect.class)
public class MySQLStoredProcedureTest
extends BaseEntityManagerFunctionalTestCase {
#Override
protected Class<?>[] getAnnotatedClasses() {
return new Class<?>[] {
Person.class,
Phone.class,
};
}
...
}
The Post entity is mapped as follows:
#Entity
public class Person {
#Id
#GeneratedValue
private Long id;
private String name;
private String nickName;
private String address;
#Temporal(TemporalType.TIMESTAMP )
private Date createdOn;
#OneToMany(mappedBy = "person", cascade = CascadeType.ALL)
#OrderColumn(name = "order_id")
private List<Phone> phones = new ArrayList<>();
#Version
private int version;
//Getters and setter omitted for brevity
}
And, when I run the test on MariaDB, Hibernate generates the following schema:
create table Person (
id bigint not null,
address varchar(255),
createdOn datetime(6),
name varchar(255),
nickName varchar(255),
version integer not null,
primary key (id)
) engine=InnoDB
That's because MariaDB53Dialect extends the MariaDBDialect which uses the InnoDBStorageEngine:
public class MariaDBDialect extends MySQL5Dialect {
public MariaDBDialect() {
super();
}
public boolean supportsRowValueConstructorSyntaxInInList() {
return true;
}
#Override
protected MySQLStorageEngine getDefaultMySQLStorageEngine() {
return InnoDBStorageEngine.INSTANCE;
}
}
So, it's impossible to get MyISAM with MariaDB53Dialect when generating the schema with hbm2ddl.
However, you should only be using hbm2ddl to generate the initial script. In a production environment, you should use a tool like FlywayDB.
We actually wrote this in the Hibernate User Guide:
Although the automatic schema generation is very useful for testing
and prototyping purposes, in a production environment, it’s much more
flexible to manage the schema using incremental migration scripts.
We changed this in Hibernate 5.2.8, so I suppose you are using an older version instead, otherwise, there is no explanation why you'd see MyISAM in your hbm2ddl auto-generated schema.
Check the dependencies using:
mvn dependency:tree
and make sure you are really using Hibernate 5.2.12.
What I am trying to perform is the following:
I have some complex SQL (with SUM(distance) distanceSum as identifier for the returned columnd) that returns some values that should be parsed to a class (containing just the values needed for these columns).
However, I only need the result in memory and not as entity.
I already tried to create a repository to execute the SQL with a #Query annotation with native = true. However, the repository can't be autowired, probably because Repositories are only meant for entities.
So is there some way to tweak a repository for non-entities or is there a approach other than repositories that would let me execute SQL and parse the result automatically into an object.
Basically as #dunni said you can use the JdbcTemplate with your own mapper to convert the SQL result to Java POJO:
public CustomResult getCustomResult(){
final String complexSql = "SELECT SUM(distance) as distanceSum....";
final CustomResult customResult = (CustomResult) jdbcTemplate.queryForObject(complexSql, new CustomResultRowMapper());
return customResult;
}
public class CustomResultRowMapper implements RowMapper {
public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
CustomResult customResult = new CustomResult();
customResult.setDistanceSum(rs.getInt("distanceSum"));
...
return customResult;
}
}
Also in Spring Boot you don't need to do anything just add your jdbcTemplate to your Dao class:
#Autowired
private JdbcTemplate jdbcTemplate;