I am trying to make ltree indexing work with Spring JPA. I have an entity that contains the path like this
#Type(type = "com.thomaster.ourcloud.model.filesystem.LTreeType")
#Column(columnDefinition="ltree")
private String relativePath;
The LTreeType class is a copy paste from here Getting error when mapping PostgreSQL LTREE column in hibernate.
The index is properly done in the database, the ltree extension is added. Still when I run the following query
#Query(value = "SELECT * FROM file_system_element WHERE relative_path ~ lquery(:pathToSearch)", nativeQuery = true)
List<FileSystemElement> findAllByPath(#Param("pathToSearch") String pathToSearch);
I get
org.hibernate.MappingException: No Dialect mapping for JDBC type: 1111.
When I change the query to
SELECT * FROM file_system_element WHERE relative_path ~ CAST(:pathToSearch AS lquery)
#OR TO THIS
SELECT * FROM file_system_element WHERE relative_path #> CAST(:pathToSearch AS ltree)
#OR TO THIS
SELECT * FROM file_system_element WHERE relative_path #> ltree(:pathToSearch)
#OR TO THIS
SELECT * FROM file_system_element WHERE ltree(relative_path) ~ lquery(:pathToSearch)
I get the same error.
For
SELECT * FROM file_system_element WHERE relative_path #> lquery(:pathToSearch)
I get
org.postgresql.util.PSQLException: ERROR: operator does not exist: ltree #> lquery
And for:
SELECT * FROM file_system_element WHERE ltree(relative_path) ~ ltree(:pathToSearch)
I get:
ERROR: operator does not exist: ltree ~ ltree
As you can see, I tried almost every combination, and still no result. I also tried the same queries as native queries on the entity manager, but it makes no difference. What am I missing here?
Solved it, it was my mistake. The query parameter was
"folder.anotherFolder"
instead of
"folder.anotherFolder.*"
Related
My Spring boot 2.7.5 app with PostgreSQL gives back an error when I try to run a nativeQuery:
#Query(value = """select import, export from
(select cast(max(sd.value) - min(sd.value) as float8)
from solar_data sd
where sd.device_address = :deviceAddress
and sd.tstamp between :from and now()) as import,
(select cast(max(sd.value) - min(sd.value) as float8)
from solar_data sd
where sd.device_address = :deviceAddress
and sd.tstamp between :from and now ()) as export
""", nativeQuery=true)
fun getMainScreenData(#Param("deviceAddress") deviceAddress:Long, #Param("from")
from:LocalDateTime) : List<Impex>
It gives back two floats as import and export.
My result interface is:
interface Impex {
fun getImport():Float
fun getExport():Float
}
Jpa error message is:
org.hibernate.MappingException: No Dialect mapping for JDBC type: 1111
org.hibernate.dialect.TypeNames.get(TypeNames.java:71)
org.hibernate.dialect.TypeNames.get(TypeNames.java:103)
org.hibernate.dialect.Dialect.getHibernateTypeName(Dialect.java:741)
I use dialect as
spring:
jpa:
databasePlatform: org.hibernate.dialect.PostgreSQL10Dialect
I tried casting as float and float8 in sql but the result is same.
How can I execute this native query?
Finally I found it.
The problem was that, postgresql result contains bracket for every value like this:
(3764.81494140625),(2461.824951171875)
That's why Hibernate couldn't parse it. I have to change sql like this:
select ri.val as import,
re.val as export
from
(select max(sd.value) - min(sd.value) as val
from solar_data sd
where sd.device_address=:deviceAddress
and sd.tstamp between :f and now ()
) ri ,
(select max(sd.value) - min(sd.value) as val
from solar_data sd
where sd.device_address=:deviceAddress
and sd.code_id= (select id from codes where code= '1-0:1.8.0')
and sd.tstamp between :f and now ()
) re
So I upgraded Spring Boot starter to 2.3.1 from 2.2.8.
I kept JOOQ at its original 3.11.5 version, as I do not want to go to version 3.12 as it does not support MySQL 5_7 as a free option.
I am now getting bad SQL grammar errors from existing queries, I don't even know where to start debugging this as what library upgrade from the starter is causing the issue?
Full error is
`Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax\
; check the manual that corresponds to your MySQL server version \
for the right syntax to use near '-1) as `alias_13136459` on \
`ideas_service`.`challenge`.`id` = `id_alias` left ou' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:370)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:209)
at org.jooq.impl.Tools.executeStatementAndGetFirstResultSet(Tools.java:3483)
at org.jooq.impl.AbstractResultQuery.execute(AbstractResultQuery.java:268)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:350)
... 122 common frames omitted`
Here is the query that gives bad SQL error:
public List<ChallengeDetails> findAllChallengesDetails(
#NonNull Integer offset, #NonNull Integer limit, Integer competitionId) {
log.info(
"Find ChallengeEntity(s) (join ChallengeQuestionsEntity) with offset: {}, limit: {}, competitionId: {}",
offset,
limit,
competitionId);
var challengeIdAlias = CHALLENGE.ID.as("id_alias");
var filterSelect =
dsl.select(challengeIdAlias)
.from(CHALLENGE)
.where(findAllChallengesCondition(competitionId))
.orderBy(CHALLENGE.NAME.asc())
.limit(limit)
.offset(offset);
Result<Record> result =
dsl.select(CHALLENGE.fields())
.select(CHALLENGE_QUESTIONS.fields())
.from(CHALLENGE)
.innerJoin(filterSelect)
.on(CHALLENGE.ID.eq(challengeIdAlias))
.leftOuterJoin(CHALLENGE_QUESTIONS)
.on(CHALLENGE.ID.eq(CHALLENGE_QUESTIONS.CHALLENGE_ID))
.orderBy(CHALLENGE.ID.asc(), CHALLENGE_QUESTIONS.ID.asc())
.fetch();
return intoChallengeDetailsGroups(result)
.entrySet()
.stream()
.map(entry -> new ChallengeDetails(entry.getKey(), entry.getValue()))
.collect(toList());
}
private Condition findAllChallengesCondition(Integer competitionId) {
Condition condition = CHALLENGE.TENANT.eq(getCurrentTenant());
if (competitionId != null) {
condition = condition.and(CHALLENGE.COMPETITION_ID.eq(competitionId));
}
return condition;
}
The error can be reproduced in MySQL directly:
select *
from (select 1 a limit 2, -1) t
It produces:
[1064] [42000]: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-1) t' at line 2
You shouldn't pass negative numbers to LIMIT or OFFSET
I am trying to run the below code using H2DB (via junit test), while doing so i get error message as below. I understand that, there are no function available as "days" in H2. So i am trying to write a custom function, but it does not work out, can any one help on writing this function.
SQLBuilder class code:
public String dummy() {
return new StringBuilder(new SQL() {
{
SELECT("date(CREATE_TMS)");
SELECT("CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_B");
FROM("Q.DUMMY");
}
}.toString().concat(" FOR READ ONLY WITH UR")).toString();
}
Error message:
org.springframework.jdbc.BadSqlGrammarException:
### Error querying database. Cause: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "SELECT DATE(CREATE_TMS), CASE WHEN DATE(CREATE_TMS) >= (CURRENT DATE - CAST('1' AS INTEGER) DAYS[*]) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR "; expected "[, ::, *, /, %, +, -, ||, ~, !~, NOT, LIKE, ILIKE, REGEXP, IS, IN, BETWEEN, AND, OR, ,, )"; SQL statement:
SELECT date(CREATE_TMS), CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR [42001-199]
For some reason days are converted to DAYS[*], we can see that in error message.
Customer method i tried in schema-db2.sql:
drop ALIAS if exists days;
CREATE ALIAS days as '
import java.lang.String;
#CODE
java.lang.String days() throws Exception {
return "days";
}
';
applicaiton.properties:
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false;Mode=DB2
DAYS is not a function and is not a something that other databases support. Db2 also uses non-standard interval literals.
If you can build H2 from its current sources, you can use cast('1' AS integer) day in it (not the days) and such construction is also supported by Db2. You can also simply use 1 DAY, it is supported by current H2 and Db2 too.
(CURRENT_DAY - 1 DAY)
Sources of H2 are available on GitHub:
https://github.com/h2database/h2database
Building instructions are here:
https://h2database.com/html/build.html#building
You need a jar target.
To compile H2 from the current sources you need JDK 8, 9, 10, 11, or 12. Compiled jar will be compatible with more recent versions.
I am trying to fetch historics data of the day and to print it.
the error i am getting is :
sqlite3 : Operational error near "(" : syntax error
import sqlite3 as sqlite
import sys
import time
conn = sqlite.connect('places.sqlite.db')
c = conn.cursor()
today = str(time.time())
here i am selecting the 10 first caracter because i want to search for a unix epoch match in seconds and not in milliseconds ( so only the first 10 are interesting to me)
c.execute("SELECT * FROM moz_places WHERE LEFT(last_visit_date, 10)='"+today+"'")
user1 = c.fetchone()
print(user1)
As mentionned earlier, i get "sqlite3 : Operational error near "(" : syntax error"
Do you what is wrong there ?
Here is how to convert moz_places.last_update_time into a string, 'YYYY-MM-DD HH:MM:SS':
UTC: datetime(last_visit_date/1000000, 'unixepoch')
Local time zone: datetime(last_visit_date/1000000, 'unixepoch','localtime')
Here is a link to SQLite doc on Date and Time Functions.
The today string created in python should match format exactly (because it will be doing a string comparison).
From the comments: the name of the places database in Firefox is places.sqlite (not places.sqlite.db). The database name should include full or relative path if it is not in your current working directory.
I've the following sql query in my kafka jdbc source connector properties file :
query=SELECT * FROM JENNY.WORKFLOW where ID = '565231'
If I run the same query in sql developer, it works fine and fetching the results. But if I use the same query in the "jdbc_workflow_connect.properties", getting the following error :
(io.confluent.connect.jdbc.source.JdbcSourceTaskConfig:223)
[2018-09-19 12:32:15,130] INFO WorkerSourceTask{id=Workflow-DB-source-0}
Source task finished initialization and start
(org.apache.kafka.connect.runtime.WorkerSourceTask:158)
[2018-09-19 12:32:15,328] ERROR Failed to run query for table
TimestampIncrementingTableQuerier{name='null', query='SELECT * FROM
JENNY.WORKFLOW where ID = '565231'', topicPrefix='workflow_data1',
timestampColumn='null', incrementingColumn='ID'}: {}
(io.confluent.connect.jdbc.source.JdbcSourceTask:247)
java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1017)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:655)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:249)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:566)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:215)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:58)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:776)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:897)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1034)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3820)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3867)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1502)
at io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier.executeQuery(TimestampIncrementingTableQuerier.java:201)
at io.confluent.connect.jdbc.source.TableQuerier.maybeStartQuery(TableQuerier.java:84)
at io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier.maybeStartQuery(TimestampIncrementingTableQuerier.java:55)
at io.confluent.connect.jdbc.source.JdbcSourceTask.poll(JdbcSourceTask.java:225)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:179)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:170)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:214)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Here is my JDBC source connector properties file content :
name=Workflow-DB-source
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
connection.password = ******
connection.url = jdbc:oracle:thin:#1.1.1.1:****/****
connection.user = *****
table.types=TABLE
query=SELECT * FROM JENNY.WORKFLOW where ID = '565231'
mode=incrementing
incrementing.column.name=ID
topic.prefix=workflow_data1
timestamp.delay.interval.ms=60000
transforms:createKey
transforms.createKey.type:org.apache.kafka.connect.transforms.ValueToKey
transforms.createKey.fields:ID
I'm using ojdbc7.jar
Observation :
If I remove the "WHERE" clause, the query is working fine(like below) :
SELECT * FROM JENNY.WORKFLOW
Please let me know if I'm doing something wrong or any modifications required for setting in the jdbc source connector.
Thanks in advance.
From the documentation of the JDBC Connect Configuration options you may read
If specified, the query to perform to select new or updated rows. Use this setting if you want to join tables, select subsets of columns in a table, or filter data. If used, this connector will only copy data using this query – whole-table copying will be disabled. Different query modes may still be used for incremental updates, but in order to properly construct the incremental query, it must be possible to append a WHERE clause to this query (i.e. no WHERE clauses may be used).
So if you realy want to consider only the part of the table with a given ID you must wrap the query as follows
select * from (SELECT * FROM JENNY.WORKFLOW where ID = '565231')
But please be sure you checked the documentation of the Configuration Options and you know the role of the query parameter.