Bad SQL grammar - Spring data R2DBC with Oracle - oracle

I'm using Spring Data R2DBC with Oracle 11g and I have the following error using method findById of R2dbcCrudRepository
executeMany; bad SQL grammar [SELECT GAME_PHASE.* FROM GAME_PHASE WHERE GAME_PHASE.ID = :P0_id FETCH FIRST 2 ROWS ONLY]
This is the repository declaration
public interface ReactiveGamePhaseRepository extends R2dbcRepository<GamePhase, Long> {
}
I don't understand why FETCH FIRST 2 ROWS ONLY is added to the query and it's the cause of problem.
I have the same problem writing the query using R2dbcEntityTemplate like below:
r2dbcEntityTemplate.selectOne(query(where("id").is(id)), GamePhase.class);
And these are the used dependancies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-r2dbc</artifactId>
<version>2.5.12</version>
</dependency>
<dependency>
<groupId>com.oracle.database.r2dbc</groupId>
<artifactId>oracle-r2dbc</artifactId>
<version>1.0.0</version>
</dependency>

I don't know anything about products you tagged (except Oracle) so I don't know whether this will help or not. Hopefully, it will).
FETCH FIRST 2 ROWS ONLY is added to the query and it's the cause of problem.
That's because - I presume - your Oracle database version doesn't support that syntax. I guess it is 11g or lower. See a demo:
Oracle 11g:
SQL> select banner from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE 11.2.0.4.0 Production
TNS for Linux: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production
SQL> SELECT *
2 FROM emp
3 WHERE deptno = 10
4 FETCH FIRST 2 ROWS ONLY;
FETCH FIRST 2 ROWS ONLY
*
ERROR at line 4:
ORA-00933: SQL command not properly ended --> that's your "bad grammar"
Oracle 18c:
SQL> select banner from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 18c Express Edition Release 18.0.0.0.0 - Production
SQL> SELECT *
2 FROM emp
3 WHERE deptno = 10
4 FETCH FIRST 2 ROWS ONLY;
EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO
---------- ---------- --------- ---------- -------- ---------- ---------- ----------
7782 CLARK MANAGER 7839 09.06.81 2450 10
7839 KING PRESIDENT 17.11.81 5000 10
SQL>
So, what can you do? Rewrite the query (if possible). For example:
SQL> WITH
2 temp
3 AS
4 (SELECT e.*, ROW_NUMBER () OVER (ORDER BY NULL) rn
5 FROM emp e
6 WHERE deptno = 10)
7 SELECT *
8 FROM temp
9 WHERE rn <= 2;
EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO RN
---------- ---------- --------- ---------- -------- ---------- ---------- ---------- ----------
7782 CLARK MANAGER 7839 09.06.81 2818 1 10 1
7839 KING PRESIDENT 17.11.81 5750 1 10 2
SQL>
(Note that - without the ORDER BY clause - you can't really know which rows will be returned. If you're happy with any two rows, then OK. If not, include ORDER BY).

I think Littlefoot did a great job explaining why this is causing problems for you. Let me explain why that clause is there in the first place.
selectOne has the following contract:
Execute a SELECT query and convert the resulting item to an entity ensuring exactly one result.
This means it needs to try to fetch at least 2 elements to ensure that there actually isn't a second element.
This could be done by selecting all matching rows.
But it would never look at anything beyond the second row, so all the other rows would be selected in vain.
And possibly worse, the execution plan might differ significantly when the database tries to load all rows vs when it only tries to fetch the first few.
Therefore the limit is applied to the query.
It uses the syntax that works for later versions of Oracle because those are the only ones supported by Spring Data JDBC.
You could avoid the problem by creating and registering your own Dialect that does not support any kind of limiting or paging and replaces ist with a noop.
Of course you need to make sure that you don't try to use such features in your application, since they obviously won't work.
Due to the interesting syntax required by older Oracle versions to implement this correctly it is not easily possible to properly implement paging and limiting for older Oracle versions.
The better option would be to migrate to a recent version of Oracle, although it is understood, that this often isn't easy for a variety of reasons.
Of course you always have the alternative to create your SQL statements in your code using either #Query annotations or custom methods.

If you're willing to use a third party library, you could translate the Oracle 12c compatible SQL using FETCH FIRST to Oracle 11g compatible SQL using jOOQ's ParsingConnection, which acts like an R2DBC Connection proxy, translating all incoming SQL to the target dialect. On Oracle 11g, the FETCH clause will be generated via ROWNUM filtering.
You could obviously also replace your Spring Data usage by jOOQ API usage to get the correct behaviour automatically. jOOQ supports R2DBC out of the box.
If that's not an option, then you'll have to write your SQL manually instead of relying on the DSL API of Spring Data or other third parties.
Disclaimer: I work for the company behind jOOQ.

Related

Oracle - denie columns in where clause

Is there a way to disable/restrict/alert-when-using some column in Oracle in a where clauses?
The reason that I'm asking this is because I have a very complex system (~30 services span cross millions of lines of code with thousends of sqls in it, in a sensitive production environment) working with an Oracle DB I need to migrate from using one column that is part of a key (and have a very not uniqu name) to another column.
Simple search is impossible....
The steps I'm having are:
populate new column
Add indexes on with the second column whenever there's an index with the first one.
Migrate all uses in where caluses from old to new column
Stop reading from the first column
Stop writing to the first column
Delete the column
I'm currently done step 3 and want to verify I've found all of the cases.
So, you're replacing one column with another. Which benefit do you expect once you're done? How will that improve overall experience with that application? I hope it is worth the effort.
As of your question: query user_source (or expand it to all_source or even dba_source, but you'll need additional privileges to do that) and see where's that very not unique name used. Something like this:
SQL> select * from user_source where lower(text) like '%empno%';
NAME TYPE LINE TEXT
--------------- ------------ ----- --------------------------------------------------------------------------------
P_RAISE PROCEDURE 22 WHERE empno = par_empno;
P_RAISE PROCEDURE 14 WHERE empno = par_empno;
P_RAISE PROCEDURE 1 PROCEDURE p_raise (par_empno IN emp.empno%TYPE)
GET_LIST FUNCTION 7 'select empno, ename, job, sal from emp where deptno = 10 order by '
SQL>

Is it possible to add a custom metadata field to Oracle Data Dictionary?

Is it possible to add a metadata field at column-level (in the Oracle Data Dictionary)?
The purpose would be to hold a flag identifying where individual data items in a table have been anonymised.
I'm an analyst (not a DBA) and I'm using Oracle SQL Developer which surfaces (and enables querying of) the COLUMN_NAME, DATA_TYPE, NULLABLE, DATA_DEFAULT, COLUMN_ID, and COMMENTS metadata fields of our Oracle DB (see pic).
I'd be looking to add another metadata field at this level (essentially, to add a second 'COMMENTS' field) to hold the 'Anonymisation' flag, to support easy querying of our flagged-anonymised data.
If it's possible (and advisable / supportable), I'd be grateful for any advice for describing the steps required to enable this, which I can then discuss with our Developer and DBA.
Short answer: NO.
But where could you keep that information?
In your data model.
Oracle provides a free data modeling solution, Oracle SQL Developer Data Modeler. It provides the ability to mark table/view columns as sensitive or PII.
Those same models can be stored back in your database so they can be accessed via SQL.
Once you've marked up all of your sensitive attributes/columns, and store it back into the database, you can query it back out.
Disclaimer: I work for Oracle, I'm the product manager for Data Modeler.
[TL;DR] Don't do it. Find another way.
If it's advisable
NO
Never modify the data dictionary; (unless Oracle support tells you to) you are likely to invalidate your support contract with Oracle and may break the database and make it unusable.
If it's possible
Don't do this.
If you really want to try it then still don't.
If you really, really want to try it then find a database you don't care about (the don't care about bit is important!) and log on as a SYSDBA user and:
ALTER TABLE whichever_data_dictionary_table ADD anonymisation_flag VARCHAR2(10);
Then you can test whether the database breaks (and it may not break immediately but at some point later), but if it does then you almost certainly will not get any support from Oracle in fixing it.
Did we say, "Don't do it"... we mean it.
As you already know, you shouldn't do that.
But, nothing prevents you from creating your own table which will contain such an info.
For example:
SQL> CREATE TABLE my_comments
2 (
3 table_name VARCHAR2 (30),
4 column_name VARCHAR2 (30),
5 anonymisation VARCHAR2 (10)
6 );
Table created.
Populate it with some data:
SQL> insert into my_comments (table_name, column_name)
2 select table_name, column_name
3 from user_tab_columns
4 where table_name = 'DEPT';
3 rows created.
Set the anonymisation flag:
SQL> update my_comments set anonymisation = 'F' where column_name = 'DEPTNO';
1 row updated.
When you want to get such an info (along with some more data from user_tab_columns, use (outer) join:
SQL> select u.table_name, u.column_name, u.data_type, u.nullable, m.anonymisation
2 from user_tab_columns u left join my_comments m on m.table_name = u.table_name
3 and m.column_name = u.column_name
4 where u.column_name = 'DEPTNO';
TABLE_NAME COLUMN_NAME DATA_TYPE N ANONYMISATION
---------- --------------- ------------ - ---------------
DEPT DEPTNO NUMBER N F
DSV DEPTNO NUMBER N
DSMV DEPTNO NUMBER Y
EMP DEPTNO NUMBER Y
SQL>
Advantages: you won't break the database & you'll have your additional info.
Drawbacks: you'll have to maintain the table manually.

REP-56048: Engine rwEng-0 crashed., job Id: 1398

Dear All,
I am working on oracle WebLogic forms & reports 12c version: (12.2.1.4)
While calling the reports with pass Lexical parameter from the parameter form the report terminated with error:
REP-56048: Engine rwEng-0 crashed., job Id: 2094.
If I pass the Lexical parameter to report locally in the report developer tool, then it works fine. When I pass only bind parameters to report, then it also works fine. The problem when passing the Lexical parameter from parameter form live.
Please help.
Well, looks like you hit a known problem. Oracle says
... check the Reports Server and engine trace files. A typical crash resembles the following in the Reports Server trace file:
<snip>
2005/6/1 3:38:36:147] State 56016 (JobManager:updateJobStatus): Job 17 status is:
Terminated with error:
REP-56048: Engine rwEng-0 crashed, job Id: 17
[2005/6/1 3:38:36:157] Debug 50103 (JobManager:notifyWaitingJobs): Master job 17
notify its duplicated jobs.
[2005/6/1 3:38:36:157] Debug 50103 (JobManager:updateJobStatus): Finished updating
job: 17
<snip>
Action to be done:
Identify the report that is causing the engine crash. You can do this by identifying the job ID. In the preceding examples, the engine crashed while running jobid 17. In the server trace file, search for the jobid = 17 Get command line string. This line contains the complete command line that includes the report name also. Enable tracing and engine diagnosis. Run the problematic report multiple times to reproduce the crash. When the crash is reproduced, pass on the trace files and diagnosis output to Oracle Support Services for analysis.
Alternatively, as you said
If I pass only bind parameters to report, then it works fine
then use bind parameters (if possible).
[EDIT], based on your comment:
If you are passing several values in the same parameter, then - if they are separated by comma - you can split that string into rows and use values as if they were returned by a subquery. It means that parameter's datatype would then be CHAR, not NUMBER.
I don't have your tables so I'm posting an example based on Scott's schema - passing EMPNO values and retrieving rows for those employees.
In Reports Builder, query would look like this:
select empno, job, ename
from emp
where empno in (select trim(regexp_substr(:par_empno, '[^,]+', 1, level))
from dual
connect by level <= regexp_count(:par_empno, ',') + 1
)
order by empno;
Testing, in SQL*Plus:
SQL> select empno, job, ename
2 from emp
3 where empno in (select trim(regexp_substr('&&par_empno', '[^,]+', 1, level))
4 from dual
5 connect by level <= regexp_count('&&par_empno', ',') + 1
6 )
7 order by empno;
Enter value for par_empno: 7369, 7499, 7521
EMPNO JOB ENAME
---------- --------- ----------
7369 CLERK SMITH
7499 SALESMAN ALLEN
7521 SALESMAN WARD
SQL>
To me, it looks as if you could use such an approach.
It could be that you are running into character set issues. Make sure you have set the correct character set in the reports server configuration file

edition$ in Oracle 11g

While checking v$sql in my database, I came across this query.
select p_obj#, flags, code, audit$ from edition$ where obj#=:1
Can anybody please explain what is this select statement for ?
I think the query is executed while gathering schema statistics by my application. But I cannot understand it.
Oracle actually stores meta data information on base tables(usually followed by $ sign on its name).
Its a base table for edition objects. DBA_EDITIONS is the view created for this base table.
An edition makes it possible to have two or more versions of the same editionable objects in the database.
SQL> select obj# from edition$;
OBJ#
----------
133
SQL> select object_type, object_name from all_objects where object_id=133;
OBJECT_TYPE OBJECT_NAME
---------------- -------------------
EDITION ORA$BASE

Know if a record is updated within Oracle?

Is there a option to see if existing table/record from a Oracle database is updated?
From a monitoring perspective (not intended to find previous changes), you have several options including but not limited to triggers, streams, and a column with a default value of sysdate. A trigger will allow you to execute a bit of programming logic (stored directly in the trigger or in an external database object) whenever the record changes (insert, update, delete). Streams can be used to track changes by monitoring the redo logs. One of the easiest may be to add a date column with a default value of sysdate.
Are you talking about within a transaction or outside of it?
Within our program we can use things like SQL%ROWCOUNT to see whether our DML succeeded...
SQL> set serveroutput on size unlimited
SQL> begin
2 update emp
3 set job = 'SALESMAN', COMM=10
4 where empno = 8083;
5 dbms_output.put_line('Number of records updated = '||sql%rowcount);
6 end;
7 /
Number of records updated = 1
PL/SQL procedure successfully completed.
SQL>
Alternatively we might test for SQL%FOUND (or SQL%NOTFOUND).
From outside the transaction we can monitor ORA_ROWSCN to see whether a record has changed.
SQL> select ora_rowscn from emp
2 where empno = 8083
3 /
ORA_ROWSCN
----------
83828715
SQL> update emp
2 set comm = 25
3 where empno = 8083
4 /
1 row updated.
SQL> commit
2 /
Commit complete.
SQL> select ora_rowscn from emp
2 where empno = 8083
3 /
ORA_ROWSCN
----------
83828780
SQL>
By default ORA_ROWSCN is set at the block level. If you want to track it at the lower level your need to create the table with the ROWDEPENCIES keyword.
These are ad hoc solutions. If you want to proactive monitoring then you need to implementing some form of logging. Using triggers to write log records is a common solution. If you have Enterprise Edition you should consider using Fine Grained Auditing: Dan Morgan's library has a useful demo of how to use FGA to track changes.
You can see if a table definition has change by querying the last_ddl_time from the user_objects view.
Without using triggers or materialized logs (which would be a total hack) there is no way I know of to see when any particular row in a table has been updated.

Resources