I got an error with the message "ORA-01722: invalid number" when I executed my query. As digged in, I found out, I will get that error when I use a numeric value for a column that expects a varchar.
In my case my query had a case statement like
CASE WHEN CHAR_COLUMN= 1 THEN 'SOME VALUE' END
But the behavior was different in different instances of same oracle version. The same query worked in one of our dev oracle server, but not in the other. I am curious if there is any configuration that allows oracle to use numeric value to be used in a character column.
PS: The oracle version that we are using is Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
I never rely on implicit data type conversion but always make it explicit:
CASE WHEN CHAR_COLUMN = TO_CHAR(1) THEN 'SOME VALUE' END
Or even not convert at all:
CASE WHEN CHAR_COLUMN = '1' THEN 'SOME VALUE' END
The reason is that Oracle tends to convert the character string to a number, not the other way round. See the example from the linked manual page:
Text Literal Example
The text literal '10' has data type CHAR. Oracle implicitly converts it to the NUMBER data type if it appears in a numeric expression as in the following statement:
SELECT salary + '10'
FROM employees;
To reproduce the issue:
create table foo (
CHAR_COLUMN varchar2(10)
);
create table bar (
CHAR_COLUMN varchar2(10)
);
insert into foo (CHAR_COLUMN) values ('1');
insert into foo (CHAR_COLUMN) values ('2');
insert into bar (CHAR_COLUMN) values ('1');
insert into bar (CHAR_COLUMN) values ('yellow');
Then, when querying against numeric 1, the first query table works and the second doesn't:
select * from foo where CHAR_COLUMN = 1;
select * from bar where CHAR_COLUMN = 1;
When you ask Oracle to resolve this comparison:
WHEN CHAR_COLUMN = 1
... Oracle converts the query internally to:
WHEN TO_NUMBER(CHAR_COLUMN) = 1
In my example, this can be spotted in the explain plan:
---------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Time |
---------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 7 | 3 | 00:00:01 |
| * 1 | TABLE ACCESS FULL | FOO | 1 | 7 | 3 | 00:00:01 |
---------------------------------------------------------------------
Predicate Information (identified by operation id):
------------------------------------------
* 1 - filter(TO_NUMBER("CHAR_COLUMN")=1)
Related
First of all, I am aware about basics.
select to_number('A231') from dual; --this will not work but
select to_char('123') from dual;-- this will work
select to_number('123') from dual;-- this will also work
Actually in my package, we have 2 tables A(X number) and B(Y varchar) There are many columns but we are worried about only X and Y. X contains values only numeric like 123,456 etc but Y contains some string and some number for eg '123','HR123','Hello'. We have to join these 2 tables. its legacy application so we are not able to change tables and columns.
Till this time below condition was working properly
to_char(A.x)=B.y;
But since there is index on Y, performance team suggested us to do
A.x=to_number(B.y); it is running in dev env.
My question is, in any circumstances will this query give error? if it picks '123' definitely it will give 123. but if it picks 'AB123' then it will fail. can it fail? can it pick 'AB123' even when it is getting joined with other table.
can it fail?
Yes. It must put every row through TO_NUMBER before it can check whether or not it meets the filter condition. Therefore, if you have any one row where it will fail then it will always fail.
From Oracle 12.2 (since you tagged Oracle 12) you can use:
SELECT *
FROM A
INNER JOIN B
ON (A.x = TO_NUMBER(B.y DEFAULT NULL ON CONVERSION ERROR))
Alternatively, put an index on TO_CHAR(A.x) and use your original query:
SELECT *
FROM A
INNER JOIN B
ON (TO_CHAR(A.x) = B.y)
Also note: Having an index on B.y does not mean that the index will be used. If you are filtering on TO_NUMBER(B.y) (with or without the default on conversion error) then you would need a function-based index on the function TO_NUMBER(B.Y) that you are using. You should profile the queries and check the explain plans to see whether there is any improvement or change in use of indexes.
Never convert a VARCHAR2 column that can contain non-mumeric strings to_number.
This can partially work, but will eventuelly definitively fail.
Small Example
create table a as
select rownum X from dual connect by level <= 10;
create table b as
select to_char(rownum) Y from dual connect by level <= 10
union all
select 'Hello' from dual;
This could work (as you limit the rows, so that the conversion works; if you are lucky and Oracle chooses the right execution plan; which is probable, but not guarantied;)
select *
from a
join b on A.x=to_number(B.y)
where B.y = '1';
But this will fail
select *
from a
join b on A.x=to_number(B.y)
ORA-01722: invalid number
Performance
But since there is index on Y, performance team suggested us to do A.x=to_number(B.y);
You should chalange the team, as if you use a function on a column (to_number(B.y)) index can't be used.
On the contrary, your original query can perfectly use the following indexes:
create index b_y on b(y);
create index a_x on a(x);
Query
select *
from a
join b on to_char(A.x)=B.y
where A.x = 1;
Execution Plan
--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 5 | 1 (0)| 00:00:01 |
| 1 | NESTED LOOPS | | 1 | 5 | 1 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN| A_X | 1 | 3 | 1 (0)| 00:00:01 |
|* 3 | INDEX RANGE SCAN| B_Y | 1 | 2 | 0 (0)| 00:00:01 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("A"."X"=1)
3 - access("B"."Y"=TO_CHAR("A"."X"))
I have a list of Oracle table and fields and I would like to define for each table, the maximum value of the field of the list.
Input:
+------+--------+
| TAB | FIELDS |
+------+--------+
| tab1 | field1 |
+------+--------+
| tab2 | field2 |
+------+--------+
Output:
+------+--------+-----------+
| TAB | FIELDS | Max value |
+------+--------+-----------+
| tab1 | field1 | 10 |
+------+--------+-----------+
| tab2 | field2 | 15 |
+------+--------+-----------+
I want to write a PL / SQL function to create the loop but I have very little knowledge in this language. Do you have any examples to show me?
The input table is dynamic, which is why I want to use a loop.
thanks in advance
The input is build with system table like all_column_tab The output must be store in a table.
It is indeed not a great design to store and retrieve data, but I presume something like this should work for you. I've used a VARCHAR2 variable for storing max value instead of a Numeric because to handle MAX for non-numeric fields. Your table that stores the max val should be defined as VARCHAR2 for it to work normally for such cases.
DECLARE
v_maxVal VARCHAR2(400);
begin
FOR rec IN
( SELECT table_name,column_name
FROM user_tab_columns where table_name IN ('TAB1','TAB2')
)
LOOP
EXECUTE IMMEDIATE
'SELECT MAX('||rec.column_name||') FROM '||rec.table_name
INTO v_maxVal ;
INSERT INTO fieldstab(tab,fields,max_val) VALUES
( rec.table_name,rec.column_name,v_maxVal);
END LOOP;
END;
/
DEMO
We need to implement a query rewrite with a bind variable because we don't have the option of modifying the web application source code. Example:
BEGIN
SYS.DBMS_ADVANCED_REWRITE.declare_rewrite_equivalence (
name => 'test_rewrite2',
source_stmt => 'select COUNT(*) from ViewX where columnA = :1',
destination_stmt => 'select COUNT(*) from ViewY where columnA = :1',
validate => FALSE,
rewrite_mode => 'recursive');
END;
The above command will result in error because there is a bind variable:
30353. 00000 - "expression not supported for query rewrite"
*Cause: The SELECT clause referenced UID, USER, ROWNUM, SYSDATE,
CURRENT_TIMESTAMP, MAXVALUE, a sequence number, a bind variable,
correlation variable, a set result, a trigger return variable, a
parallel table queue column, collection iterator, a non-deterministic
date format token RR, etc.
*Action: Remove the offending expression or disable the REWRITE option on
the materialized view.
I am reading here that there is a work around but I just cannot find the document anywhere online.
Could you please tell me what the work around is?
You can't specify the bind parameters, but it should already work as you wish. The key is the recursive parameter you passed as mode.
The recursive and general mode will intercept all statements that involve the table (or view), disregarding the filter, and transform them to target the second table (or view), adapting the filter condition from your original statement.
(If you had defined it as TEXT_MATCH, it would have checked the presence of the same filter in the original and target statement in order to trigger the transformation.)
In the example below one can see that even if we don't define any bind condition, the filter id = 2 is applied nervetheless; in other words it is actually transforming the SELECT * FROM A1 where id = 2 into SELECT * FROM A2 where id = 2
set LINESIZE 300
drop table A1;
drop view A2;
drop index A1_IDX;
EXEC SYS.DBMS_ADVANCED_REWRITE.drop_rewrite_equivalence (name => 'test_rewrite');
create table A1 (id number, name varchar2(20));
insert into A1 values(1, 'hello world');
insert into A1 values(2, 'hola mundo');
create index A1_IDX on A1(id);
select * from A1;
ALTER SESSION SET QUERY_REWRITE_INTEGRITY = TRUSTED;
CREATE OR REPLACE VIEW A2 AS
SELECT id,
INITCAP(name) AS name
FROM A1
ORDER BY id desc;
BEGIN
SYS.DBMS_ADVANCED_REWRITE.declare_rewrite_equivalence (
name => 'test_rewrite',
source_stmt => 'SELECT * FROM A1',
destination_stmt => 'SELECT * FROM A2',
validate => FALSE,
rewrite_mode => 'recursive');
END;
/
select * from A1;
ID NAME
---------- --------------------
2 Hola Mundo
1 Hello World
select * from A1 where id = 2;
ID NAME
---------- --------------------
2 Hola Mundo
explain plan for
select * from A1 where id = 2;
select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------
Plan hash value: 1034670462
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 25 | 2 (0)| 00:00:01 |
| 1 | VIEW | A2 | 1 | 25 | 2 (0)| 00:00:01 |
| 2 | TABLE ACCESS BY INDEX ROWID | A1 | 1 | 25 | 2 (0)| 00:00:01 |
|* 3 | INDEX RANGE SCAN DESCENDING| A1_IDX | 1 | | 1 (0)| 00:00:01 |
----------------------------------------------------------------------------------------
PLAN_TABLE_OUTPUT
---------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("ID"=2)
Note
-----
- dynamic sampling used for this statement (level=2)
- automatic DOP: Computed Degree of Parallelism is 1 because of parallel threshold
20 rows selected
As you can see
the engine is transparently applying the transformation and returning the filtered result
on top of that, the transformation on the filter is applied. The filter is correctly "pushed" into the source table, to extract the values from A1. It is not blindly extracting all values from A2 and then applying the filter, so the performance is preserved.
I have a table where two columns are of type VARCHAR2(3BYTE) and VARCHAR2(32BYTE). When I do a select query (where col1=10 and where col1='10') or (where col2=70001 or col2='70001') the number of records fetched are the same in each set of where clauses. How does this happen? How does Oracle treat string literals and numeric constants and compare to the data despite column data-type?
But this does not work for a column of type VARCHAR2(128BYTE). The query needed to be where col3='55555555001' to work and where col3=55555555001 throws ORA-01722 error.
As noted in the SQL Language Reference:
During SELECT FROM operations, Oracle converts the data from the column to the type of the target variable.
...
When comparing a character value with a numeric value, Oracle converts the character data to a numeric value.
Implicit conversion is performed on the table column when the types don't match. This can be seen by tracing in SQL*Plus, with some dummy data.
create table t42 (foo varchar2(3 byte));
insert into t42 (foo) values ('10');
insert into t42 (foo) values ('2A');
set autotrace on explain
This works:
select * from t42 where foo = '10';
FOO
---
10
Execution Plan
----------------------------------------------------------
Plan hash value: 3843907281
--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 3 | 3 (0)| 00:00:01 |
|* 1 | TABLE ACCESS FULL| T42 | 1 | 3 | 3 (0)| 00:00:01 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("FOO"='10')
Note
-----
- dynamic sampling used for this statement (level=2)
But this errors:
select * from t42 where foo = 10;
ERROR:
ORA-01722: invalid number
Execution Plan
----------------------------------------------------------
Plan hash value: 3843907281
--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 3 | 3 (0)| 00:00:01 |
|* 1 | TABLE ACCESS FULL| T42 | 1 | 3 | 3 (0)| 00:00:01 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(TO_NUMBER("FOO")=10)
Note the difference in the filter; filter("FOO"='10') versus filter(TO_NUMBER("FOO")=10). In the latter case, comparing against a number, a to_number() is being performed against every row in the table the the result of that conversion is compared against the fixed value. So if any of the character values cannot be converted, you'll get an ORA-01722. The function being applied will also stop an index being used, if one is present on that column.
Where it gets interesting is if you have more than one filter. Oracle may evaluate them in different orders at different times, so you might not always see the ORA-01722, and it'll pop up sometimes. Say you had where foo = 10 and bar = 'X'. If Oracle thought it could filter out the non-X values first, it would only apply the to_number() to what's left, and that smaller sample might not have non-numeric values in foo. But if you has and bar = 'Y', the non-Y values might include non-numerics, or Oracle might filter on foo first, depending on how selective it thinks the values are.
The moral is to never store numeric information as a character type.
I was looking for an AskTom reference to back up the moral, and the first one I looked at conveniently refers to the effect of "a change in the order of a predicate" as well as saying "don't store numbers in varchar2's".
If a numeric column or value and a character column are involved, Oracle converts the character column values to numbers and then converts numbers with numbers. It's as if you had written:
where to_number(col3) = 55555555001
That's why you get an ORA-01722: invalid number error if a single row contains a string (n col3) that cannot be converted to a numeric value.
For that reason we have the IS_NUMBER function in our Oracle database that doesn't cause an error but returns NULL for values that cannot be converted to numbers. Then you can safely write:
where is_number(col3) = 55555555001
The function is defined as:
CREATE OR REPLACE FUNCTION is_number (p_str IN VARCHAR2)
RETURN NUMBER
IS
l_num NUMBER;
BEGIN
l_num := to_number(p_str);
RETURN l_num;
EXCEPTION
WHEN others THEN
RETURN NULL;
END is_number;
Let's say users have 1 - n accounts in a system. When they query the database, they may choose to select from m acounts, with m between 1 and n. Typically the SQL generated to fetch their data is something like
SELECT ... FROM ... WHERE account_id IN (?, ?, ..., ?)
So depending on the number of accounts a user has, this will cause a new hard-parse in Oracle, and a new execution plan, etc. Now there are a lot of queries like that and hence, a lot of hard-parses, and maybe the cursor/plan cache will be full quite early, resulting in even more hard-parses.
Instead, I could also write something like this
-- use any of these
CREATE TYPE numbers AS VARRAY(1000) of NUMBER(38);
CREATE TYPE numbers AS TABLE OF NUMBER(38);
SELECT ... FROM ... WHERE account_id IN (
SELECT column_value FROM TABLE(?)
)
-- or
SELECT ... FROM ... JOIN (
SELECT column_value FROM TABLE(?)
) ON column_value = account_id
And use JDBC to bind a java.sql.Array (i.e. an oracle.sql.ARRAY) to the single bind variable. Clearly, this will result in less hard-parses and less cursors in the cache for functionally equivalent queries. But is there anything like general a performance-drawback, or any other issues that I might run into?
E.g: Does bind variable peeking work in a similar fashion for varrays or nested tables? Because the amount of data associated with every account may differ greatly.
I'm using Oracle 11g in this case, but I think the question is interesting for any Oracle version.
I suggest you try a plain old join like in
SELECT Col1, Col2
FROM ACCOUNTS ACCT
TABLE TAB,
WHERE ACCT.User = :ParamUser
AND TAB.account_id = ACCT.account_id;
An alternative could be a table subquery
SELECT Col1, Col2
FROM (
SELECT account_id
FROM ACCOUNTS
WHERE User = :ParamUser
) ACCT,
TABLE TAB
WHERE TAB.account_id = ACCT.account_id;
or a where subquery
SELECT Col1, Col2
FROM TABLE TAB
WHERE TAB.account_id IN
(
SELECT account_id
FROM ACCOUNTS
WHERE User = :ParamUser
);
The first one should be better for perfomance, but you better check them all with explain plan.
Looking at V$SQL_BIND_CAPTURE in a 10g database, I have a few rows where the datatype is VARRAY or NESTED_TABLE; the actual bind values were not captured. In an 11g database, there is just one such row, but it also shows that the bind value is not captured. So I suspect that bind value peeking essentially does not happen for user-defined types.
In my experience, the main problem you run into using nested tables or varrays in this way is that the optimizer does not have a good estimate of the cardinality, which could lead it to generate bad plans. But, there is an (undocumented?) CARDINALITY hint that might be helpful. The problem with that is, if you calculate the actual cardinality of the nested table and include that in the query, you're back to having multiple distinct query texts. Perhaps if you expect that most or all users will have at most 10 accounts, using the hint to indicate that as the cardinality would be helpful. Of course, I'd try it without the hint first, you may not have an issue here at all.
(I also think that perhaps Miguel's answer is the right way to go.)
For medium sized list (several thousand items) I would use this approach:
First:generate a prepared statement with an XMLTABLE in join with your main table.
For instance:
String myQuery = "SELECT ...
+" FROM ACCOUNTS A,"
+ "XMLTABLE('tab/row' passing XMLTYPE(?) COLUMNS id NUMBER path 'id') t
+ "WHERE A.account_id = t.id"
then loop through your data and build a StringBuffer with this content:
StringBuffer idList = "<tab><row><id>101</id></row><row><id>907</id></row> ...</tab>";
eventually, prepare and submit your statement, then fetch the results.
myQuery.setString(1, idList);
ResultSet rs = myQuery.executeQuery();
while (rs.next()) {...}
Using this approach is also possible to pass multi-valued list, as in the select statement
SELECT * FROM TABLE t WHERE (t.COL1, t.COL2) in (SELECT X.COL1, X.COL2 FROM X);
In my experience performances are pretty good, and the approach is flexible enough to be used in very complex query scenarios.
The only limit is the size of the string passed to the DB, but I suppose it is possible to use CLOB in place of String for arbitrary long XML wrapper to the input list;
This binding a variable number of items into an in list problem seems to come up a lot in various form. One option is to concatenate the IDs into a comma separated string and bind that, and then use a bit of a trick to split it into a table you can join against, eg:
with bound_inlist
as
(
select
substr(txt,
instr (txt, ',', 1, level ) + 1,
instr (txt, ',', 1, level+1) - instr (txt, ',', 1, level) -1 )
as token
from (select ','||:txt||',' txt from dual)
connect by level <= length(:txt)-length(replace(:txt,',',''))+1
)
select *
from bound_inlist a, actual_table b
where a.token = b.token
Bind variable peaking is going to be a problem though.
Does the query plan actually change for larger number of accounts, ie would it be more efficient to move from index to full table scan in some cases, or is it borderline? As someone else suggested, you could use the CARDINALITY hint to indicate how many IDs are being bound, the following test case proves this actually works:
create table actual_table (id integer, padding varchar2(100));
create unique index actual_table_idx on actual_table(id);
insert into actual_table
select level, 'this is just some padding for '||level
from dual connect by level <= 1000;
explain plan for
with bound_inlist
as
(
select /*+ CARDINALITY(10) */
substr(txt,
instr (txt, ',', 1, level ) + 1,
instr (txt, ',', 1, level+1) - instr (txt, ',', 1, level) -1 )
as token
from (select ','||:txt||',' txt from dual)
connect by level <= length(:txt)-length(replace(:txt,',',''))+1
)
select *
from bound_inlist a, actual_table b
where a.token = b.id;
----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 10 | 840 | 2 (0)| 00:00:01 |
| 1 | NESTED LOOPS | | | | | |
| 2 | NESTED LOOPS | | 10 | 840 | 2 (0)| 00:00:01 |
| 3 | VIEW | | 10 | 190 | 2 (0)| 00:00:01 |
|* 4 | CONNECT BY WITHOUT FILTERING| | | | | |
| 5 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 |
|* 6 | INDEX UNIQUE SCAN | ACTUAL_TABLE_IDX | 1 | | 0 (0)| 00:00:01 |
| 7 | TABLE ACCESS BY INDEX ROWID | ACTUAL_TABLE | 1 | 65 | 0 (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------
Another option is to always use n bind variables in every query. Use null for m+1 to n.
Oracle ignores repeated items in the expression_list. Your queries will perform the same way and there will be fewer hard parses. But there will be extra overhead to bind all the variables and transfer the data. Unfortunately I have no idea what the overall affect on performance would be, you'd have to test it.