Upgraded Spring boot: Now getting "bad SQL grammar" errors from JDBC and JOOQ - spring-boot

So I upgraded Spring Boot starter to 2.3.1 from 2.2.8.
I kept JOOQ at its original 3.11.5 version, as I do not want to go to version 3.12 as it does not support MySQL 5_7 as a free option.
I am now getting bad SQL grammar errors from existing queries, I don't even know where to start debugging this as what library upgrade from the starter is causing the issue?
Full error is
`Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax\
; check the manual that corresponds to your MySQL server version \
for the right syntax to use near '-1) as `alias_13136459` on \
`ideas_service`.`challenge`.`id` = `id_alias` left ou' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:370)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:209)
at org.jooq.impl.Tools.executeStatementAndGetFirstResultSet(Tools.java:3483)
at org.jooq.impl.AbstractResultQuery.execute(AbstractResultQuery.java:268)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:350)
... 122 common frames omitted`
Here is the query that gives bad SQL error:
public List<ChallengeDetails> findAllChallengesDetails(
#NonNull Integer offset, #NonNull Integer limit, Integer competitionId) {
log.info(
"Find ChallengeEntity(s) (join ChallengeQuestionsEntity) with offset: {}, limit: {}, competitionId: {}",
offset,
limit,
competitionId);
var challengeIdAlias = CHALLENGE.ID.as("id_alias");
var filterSelect =
dsl.select(challengeIdAlias)
.from(CHALLENGE)
.where(findAllChallengesCondition(competitionId))
.orderBy(CHALLENGE.NAME.asc())
.limit(limit)
.offset(offset);
Result<Record> result =
dsl.select(CHALLENGE.fields())
.select(CHALLENGE_QUESTIONS.fields())
.from(CHALLENGE)
.innerJoin(filterSelect)
.on(CHALLENGE.ID.eq(challengeIdAlias))
.leftOuterJoin(CHALLENGE_QUESTIONS)
.on(CHALLENGE.ID.eq(CHALLENGE_QUESTIONS.CHALLENGE_ID))
.orderBy(CHALLENGE.ID.asc(), CHALLENGE_QUESTIONS.ID.asc())
.fetch();
return intoChallengeDetailsGroups(result)
.entrySet()
.stream()
.map(entry -> new ChallengeDetails(entry.getKey(), entry.getValue()))
.collect(toList());
}
private Condition findAllChallengesCondition(Integer competitionId) {
Condition condition = CHALLENGE.TENANT.eq(getCurrentTenant());
if (competitionId != null) {
condition = condition.and(CHALLENGE.COMPETITION_ID.eq(competitionId));
}
return condition;
}

The error can be reproduced in MySQL directly:
select *
from (select 1 a limit 2, -1) t
It produces:
[1064] [42000]: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-1) t' at line 2
You shouldn't pass negative numbers to LIMIT or OFFSET

Related

SpringBootTest in error with h2 v2.1.214 because of PageRequest

I have some tests in error after upgrading from Spring boot 2.5.6 to 2.7.3.
For information we use Oracle for the database and h2 for tests.
I have some tests in failure with the following error:
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException
In fact, the version of h2 was 1.4.200 before and is 2.1.214 now and a lot of things seem to have changed. The reason of the error is not always the same according to the test in error. Sometimes it is an error with a table not found (not solved yet), sometimes it is an error with "Values of types "BOOLEAN" and "INTEGER" are not comparable" (solved by updating a query where a comparison was done with a boolean column like this myBoolean = 0 and it has been updated to myBoolean = false) and I also have an error on a query done with a PageRequest.
For this last case, I have a Controller like this:
public Page<MyEntity> doSomething() {
final Sort sort = Sort.by(Order.desc("column1"));
final PageRequest pageRequest = PageRequest.of(0, 1000, sort);
return myEntityRepository.findAll(pageRequest);
}
But I have an error like that:
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "select myentity0_.id as id1_47_, myentity0_.column1 as column1_47_, myentity0_.column2 as column2_47_ from my_table myentity0_ order by myentity0_.column1 desc [*]limit ?"; SQL statement:
select myentity0_.id as id1_47_, myentity0_.column1 as column1_47_, myentity0_.column2 as column2_47_ from my_table myentity0_ order by myentity0_.column1 desc limit ? [42000-214]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:502)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:477)
at org.h2.message.DbException.get(DbException.java:223)
at org.h2.message.DbException.get(DbException.java:199)
at org.h2.message.DbException.getSyntaxError(DbException.java:247)
at org.h2.command.Parser.getSyntaxError(Parser.java:898)
at org.h2.command.Parser.prepareCommand(Parser.java:572)
at org.h2.engine.SessionLocal.prepareLocal(SessionLocal.java:631)
at org.h2.engine.SessionLocal.prepareCommand(SessionLocal.java:554)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1116)
at org.h2.jdbc.JdbcPreparedStatement.<init>(JdbcPreparedStatement.java:92)
at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:288)
at com.zaxxer.hikari.pool.ProxyConnection.prepareStatement(ProxyConnection.java:337)
at com.zaxxer.hikari.pool.HikariProxyConnection.prepareStatement(HikariProxyConnection.java)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:149)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:176)
... 205 more
If I change the Controller like this,the test is in success:
public Page<MyEntity> doSomething() {
List<MyEntity> result = myEntityRepository.findAll();
return new PageImpl<MyEntity>(result);
}
So It seems that the problem was due by the use of PageRequest.
Do you have an idea please?
Java persistence libraries are usually tested only with default Regular mode of H2 and may not work well with other modes.
Oracle doesn't support MySQL/PostgreSQL-style LIMIT, and H2 doesn't allow it in Oracle compatibility mode, but some libraries produce LIMIT instead of standard OFFSET / FETCH for H2.
Spring Data JDBC (spring-data-relational) added support of custom compatibility modes of H2 only about a month ago and version 2.4.3 with this fix isn't released yet.
Hibernate ORM 6.*.* should work well, but Hibernate ORM 5.6.* has a known issue:
https://hibernate.atlassian.net/jira/software/c/projects/HHH/issues/HHH-15318
You can enable LIMIT in Oracle compatibility mode of H2 as a temporary workaround. To do that, you need to execute the following Java code during initialization of your application:
org.h2.engine.Mode mode = org.h2.engine.Mode.getInstance("ORACLE");
mode.limit = true;

Atomikos with Hibernate(MySQL) have connection error

I setted below properties cause MySQL bug(https://www.atomikos.com/Documentation/KnownProblems#MySQL_XA_bug)
com.atomikos.icatch.serial_jta_transactions=false
pinGlobalTxToPhysicalConnection="true"
but when i test it with multiple request after upper properties set, it shows below error
XA resource 'shard0': suspend for XID '3139322E3136382E3231392E3131382E746D313634343537333034393734363030303031:3139322E3136382E3231392E3131382E746D31' raised -5: invalid arguments were given for the XA operation java.sql.SQLException: XAER_INVAL: Invalid arguments (or unsupported command)
at com.atomikos.datasource.xa.XAResourceTransaction.xaSuspend(XAResourceTransaction.java:700) [7 skipped]
at com.atomikos.datasource.xa.session.BranchEnlistedStateHandler.transactionSuspended(BranchEnlistedStateHandler.java:94)
at com.atomikos.datasource.xa.session.TransactionContext.transactionSuspended(TransactionContext.java:94)
at com.atomikos.datasource.xa.session.SessionHandleState.notifyBeforeUse(SessionHandleState.java:165)
at com.atomikos.jdbc.AtomikosConnectionProxy.enlist(AtomikosConnectionProxy.java:207)
... 140 common frames omitted
I wonder why my application throws 'UnexpectedTransactionContextException' from below code even i setted all properties.
//BranchEnlistedStateHandler.java(Atomikos)
TransactionContextStateHandler checkEnlistBeforeUse ( CompositeTransaction currentTx)
throws InvalidSessionHandleStateException, UnexpectedTransactionContextException
{
if ( currentTx == null || !currentTx.isSameTransaction ( ct ) ) {
//OOPS! we are being used a different tx context than the one expected...
//TODO check: what if subtransaction? Possible solution: ignore if serial_jta mode, error otherwise.
String msg = "The connection/session object is already enlisted in a (different) transaction.";
if ( LOGGER.isTraceEnabled() ) LOGGER.logTrace ( msg );
throw new UnexpectedTransactionContextException();
}
//tx context is still the same -> no change in state required
return null;
}java
It is too late but this might help someone:
Spring Boot 2.6.7
mysql-connector-java 8.0.29
According to the atomikos problems MySQL_XA_bug try to add following properties to spring boot.
spring.jta.atomikos.properties.serial-jta-transactions = false
spring.datasource.xa.properties.pinGlobalTxToPhysicalConnection = true

Function not found when generating code with DDL Database - jooq

I have a spring boot gradle project with a mysql database. Previously under jooq version 3.13.6 my sql was parsed without errors. When updating to a higher jooq version (3.14.X and 3.15.X) and generating/parsing the migrations with jooq, I get the following output:
SEVERE DDLDatabase Error: Your SQL string could not be parsed or
interpreted. This may have a variety of reasons, including:
The jOOQ parser doesn't understand your SQL
The jOOQ DDL simulation logic (translating to H2) cannot simulate your SQL
org.h2.jdbc.JdbcSQLSyntaxErrorException: Function "coalesce" not found;
A basic sql example where the error occurs is given below. Parsing the same view worked with jooq 3.13.6.
DROP VIEW IF EXISTS view1;
CREATE VIEW view1 AS
SELECT COALESCE(SUM(table1.col1), 0) AS 'sum'
FROM table1;
I am currently lost here. I don't see any related changes in the jooq changelog.
Any help or directions to further have a look into are highly appreciated.
Extended Stacktrace:
11:10:30 SEVERE DDLDatabase Error : Your SQL string could not be parsed or interpreted. This may have a variety of reasons, including:
- The jOOQ parser doesn't understand your SQL
- The jOOQ DDL simulation logic (translating to H2) cannot simulate your SQL
If you think this is a bug or a feature worth requesting, please report it here: https://github.com/jOOQ/jOOQ/issues/new/choose
As a workaround, you can use the Settings.parseIgnoreComments syntax documented here:
https://www.jooq.org/doc/latest/manual/sql-building/dsl-context/custom-settings/settings-parser/
11:10:30 SEVERE Error while loading file: /Users/axel/projects/service/./src/main/resources/db/migration/V5__create_view1.sql
11:10:30 SEVERE Error in file: /Users/axel/projects/service/build/tmp/generateJooq/config.xml. Error : Error while exporting schema
org.jooq.exception.DataAccessException: Error while exporting schema
at org.jooq.meta.extensions.AbstractInterpretingDatabase.connection(AbstractInterpretingDatabase.java:103)
at org.jooq.meta.extensions.AbstractInterpretingDatabase.create0(AbstractInterpretingDatabase.java:77)
at org.jooq.meta.AbstractDatabase.create(AbstractDatabase.java:332)
at org.jooq.meta.AbstractDatabase.create(AbstractDatabase.java:322)
at org.jooq.meta.AbstractDatabase.setConnection(AbstractDatabase.java:312)
at org.jooq.codegen.GenerationTool.run0(GenerationTool.java:531)
at org.jooq.codegen.GenerationTool.run(GenerationTool.java:237)
at org.jooq.codegen.GenerationTool.generate(GenerationTool.java:232)
at org.jooq.codegen.GenerationTool.main(GenerationTool.java:204)
Caused by: org.jooq.exception.DataAccessException: SQL [create view "view1" as select "coalesce"("sum"("table1"."col1"), 0) "sum" from "table1"]; Function "coalesce" not found; SQL statement:
create view "view1" as select "coalesce"("sum"("table1"."col1"), 0) "sum" from "table1" [90022-200]
at org.jooq_3.15.5.H2.debug(Unknown Source)
at org.jooq.impl.Tools.translate(Tools.java:2988)
at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:639)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:349)
at org.jooq.meta.extensions.ddl.DDLDatabase.load(DDLDatabase.java:183)
at org.jooq.meta.extensions.ddl.DDLDatabase.lambda$export$0(DDLDatabase.java:156)
at org.jooq.FilePattern.load0(FilePattern.java:307)
at org.jooq.FilePattern.load(FilePattern.java:287)
at org.jooq.FilePattern.load(FilePattern.java:300)
at org.jooq.FilePattern.load(FilePattern.java:251)
at org.jooq.meta.extensions.ddl.DDLDatabase.export(DDLDatabase.java:156)
at org.jooq.meta.extensions.AbstractInterpretingDatabase.connection(AbstractInterpretingDatabase.java:100)
... 8 more
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Function "coalesce" not found; SQL statement:
create view "view1" as select "coalesce"("sum"("table1"."col1"), 0) "sum" from "table1" [90022-200]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:576)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:429)
at org.h2.message.DbException.get(DbException.java:205)
at org.h2.message.DbException.get(DbException.java:181)
at org.h2.command.Parser.readJavaFunction(Parser.java:3565)
at org.h2.command.Parser.readFunction(Parser.java:3770)
at org.h2.command.Parser.readTerm(Parser.java:4305)
at org.h2.command.Parser.readFactor(Parser.java:3343)
at org.h2.command.Parser.readSum(Parser.java:3330)
at org.h2.command.Parser.readConcat(Parser.java:3305)
at org.h2.command.Parser.readCondition(Parser.java:3108)
at org.h2.command.Parser.readExpression(Parser.java:3059)
at org.h2.command.Parser.readFunctionParameters(Parser.java:3778)
at org.h2.command.Parser.readFunction(Parser.java:3772)
at org.h2.command.Parser.readTerm(Parser.java:4305)
at org.h2.command.Parser.readFactor(Parser.java:3343)
at org.h2.command.Parser.readSum(Parser.java:3330)
at org.h2.command.Parser.readConcat(Parser.java:3305)
at org.h2.command.Parser.readCondition(Parser.java:3108)
at org.h2.command.Parser.readExpression(Parser.java:3059)
at org.h2.command.Parser.parseSelectExpressions(Parser.java:2931)
at org.h2.command.Parser.parseSelect(Parser.java:2952)
at org.h2.command.Parser.parseQuerySub(Parser.java:2817)
at org.h2.command.Parser.parseSelectUnion(Parser.java:2649)
at org.h2.command.Parser.parseQuery(Parser.java:2620)
at org.h2.command.Parser.parseCreateView(Parser.java:6950)
at org.h2.command.Parser.parseCreate(Parser.java:6223)
at org.h2.command.Parser.parsePrepared(Parser.java:903)
at org.h2.command.Parser.parse(Parser.java:843)
at org.h2.command.Parser.parse(Parser.java:815)
at org.h2.command.Parser.prepareCommand(Parser.java:738)
at org.h2.engine.Session.prepareLocal(Session.java:657)
at org.h2.engine.Session.prepareCommand(Session.java:595)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1235)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:212)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:201)
at org.jooq.tools.jdbc.DefaultStatement.execute(DefaultStatement.java:102)
at org.jooq.impl.SettingsEnabledPreparedStatement.execute(SettingsEnabledPreparedStatement.java:227)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:414)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:335)
... 16 more
> Task :generateJooq FAILED
Jooq Configuration:
jooq {
version = "3.15.5"
edition = JooqEdition.OSS
configurations {
main {
generationTool {
generator {
name = 'org.jooq.codegen.KotlinGenerator'
strategy {
name = 'org.jooq.codegen.DefaultGeneratorStrategy'
}
generate {
relations = true
deprecated = false
records = true
immutablePojos = true
fluentSetters = true
daos = false
pojosEqualsAndHashCode = true
javaTimeTypes = true
}
target {
packageName = 'de.project.service.jooq'
}
database {
name = 'org.jooq.meta.extensions.ddl.DDLDatabase'
properties {
property {
key = 'scripts'
value = 'src/main/resources/db/migration/*.sql'
}
property {
key = 'sort'
value = 'semantic'
}
property {
key = 'unqualifiedSchema'
value = 'none'
}
property {
key = 'defaultNameCase'
value = 'lower'
}
}
}
}
}
}
}
You probably have the following configuration set:
<property>
<key>defaultNameCase</key>
<value>lower</value>
</property>
In jOOQ 3.15, this transforms all identifiers to lower case and quotes them before handing the SQL statement to H2 behind the scenes for DDL simulation, in order to emulate e.g. PostgreSQL behaviour, where unquoted identifiers are lower case, not upper case as in many other RDBMS.
There's a bug in the current implementation, which also quotes built-in functions, not just user defined objects. See:
https://github.com/jOOQ/jOOQ/issues/9931 (general problem related to "system names")
https://github.com/jOOQ/jOOQ/issues/12752 (DDLDatabase specific problem)
The only workaround I can think of would be to turn off that property again, and manually quote all identifiers to be lower case. Alternatively, instead of using the DDLDatabase, you can always connect to an actual database instead, e.g. by using testcontainers. This will be much more robust in many ways, anyway, than the DDLDatabase
In any case, this is quite the frequent problem, so, I've fixed this for the upcoming jOOQ 3.16. The above setting will no longer quote "system names", which are well known identifiers of built-in functions

jdbcTemplate: Invalid column type in delete query

I used jdbcTemplate in my projet spring boot, i tried to delete a row with the below script, it work fine
String ids1 = String.join(",", gourpIds.stream().map(id -> id+"").collect(Collectors.toList()));
String ids2 = String.join(",", deleteExamDTO.getRoomIds().stream().map(id -> id+"").collect(Collectors.toList()));
jdbcTemplateObject.update(String.format("DELETE FROM SUPERVISION_ROOM cl WHERE cl.FK_GROUP IN (%s)", ids1), gourpIds.toArray());
but when i add another param in my query like below i get this error java.sql.SQLException: Invalid column type
jdbcTemplateObject.update(String.format("DELETE FROM SUPERVISION_ROOM cl WHERE cl.FK_GROUP IN (%s) AND cl.FK_ROOM IN (%s)", ids1, ids2), gourpIds.toArray(), deleteExamDTO.getRoomIds().toArray());

Write custom H2 DB function Java

I am trying to run the below code using H2DB (via junit test), while doing so i get error message as below. I understand that, there are no function available as "days" in H2. So i am trying to write a custom function, but it does not work out, can any one help on writing this function.
SQLBuilder class code:
public String dummy() {
return new StringBuilder(new SQL() {
{
SELECT("date(CREATE_TMS)");
SELECT("CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_B");
FROM("Q.DUMMY");
}
}.toString().concat(" FOR READ ONLY WITH UR")).toString();
}
Error message:
org.springframework.jdbc.BadSqlGrammarException:
### Error querying database. Cause: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "SELECT DATE(CREATE_TMS), CASE WHEN DATE(CREATE_TMS) >= (CURRENT DATE - CAST('1' AS INTEGER) DAYS[*]) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR "; expected "[, ::, *, /, %, +, -, ||, ~, !~, NOT, LIKE, ILIKE, REGEXP, IS, IN, BETWEEN, AND, OR, ,, )"; SQL statement:
SELECT date(CREATE_TMS), CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR [42001-199]
For some reason days are converted to DAYS[*], we can see that in error message.
Customer method i tried in schema-db2.sql:
drop ALIAS if exists days;
CREATE ALIAS days as '
import java.lang.String;
#CODE
java.lang.String days() throws Exception {
return "days";
}
';
applicaiton.properties:
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false;Mode=DB2
DAYS is not a function and is not a something that other databases support. Db2 also uses non-standard interval literals.
If you can build H2 from its current sources, you can use cast('1' AS integer) day in it (not the days) and such construction is also supported by Db2. You can also simply use 1 DAY, it is supported by current H2 and Db2 too.
(CURRENT_DAY - 1 DAY)
Sources of H2 are available on GitHub:
https://github.com/h2database/h2database
Building instructions are here:
https://h2database.com/html/build.html#building
You need a jar target.
To compile H2 from the current sources you need JDK 8, 9, 10, 11, or 12. Compiled jar will be compatible with more recent versions.

Resources