Creating a mySomeTable table with 2 fields
create table mySomeTable (
IDRQ VARCHAR2(32 CHAR),
PROCID VARCHAR2(64 CHAR)
);
Creating an index on the table by the PROCID field
create index idx_PROCID on mySomeTable(trunc(PROCID));
Inserting records:
insert into mySomeTable values ('a', '1'); -- OK
insert into mySomeTable values ('b', 'c'); -- FAIL
As you can see, an error has been made in the index construction script and the script will try to build an index on the field using the trunc() function.
trunct() is a function for working with dates or numbers, and the field has the string type
This index building script successfully works out and creates an index without displaying any warnings and errors.
An index is created on the table using the TRUNC(TO_NUMBER(PROCID)) function
When trying to insert or change an entry in the table, if PROCID cannot be converted to a number, I get the error ORA-01722: invalid number, which is actually logical.
However, the understanding that I am working in a table with rows and adding string values to the table, and the error is about converting to a number, was misleading and not understanding what is happening...
Question: Why does Oracle change the index construction function, instead of giving an error? And how can this be avoided in the future?
Oracle version 19.14
Naturally, there was only one solution - to create the right index with the right script
create index idx_PROCID on mySomeTable(PROCID);
however, this does not explain, to me, this Oracle behavior.
Oracle doesn't know if the index declaration is wrong or the column data type is wrong. Arguably (though some may well disagree!) Oracle shouldn't try to second-guess your intentions or enforce restrictions beyond those documented in the manual - that's what user-defined constraints are for. And, arguably, this index acts as a form of pseudo-constraint. That's a decision for the developer, not Oracle.
It's legal, if usually ill-advised, to store a number in a string column. If you actually intentionally chose to store numbers as strings - against best practice and possibly just to irritate future maintainers of your code - then the index behaviour is reasonable.
A counter-question is to ask where it should draw the line - if you expect it to error on your index expression, what about something like
create index idx_PROCID on mySomeTable(
case when regexp_like(PROCID, '^\d.?\d*$') then trunc(PROCID) end
);
or
create index idx_PROCID on mySomeTable(
trunc(to_number(PROCID default null on conversion error))
);
You might actually have chosen to store both numeric and non-numeric data in the same string column (again, I'm not advocating that) and an index like that might then useful - and you wouldn't want Oracle to prevent you from creating it.
Something that obviously doesn't make sense and shouldn't be allowed to you is much harder for software to evaluate.
Interestingly the documentation says:
Oracle recommends that you specify explicit conversions, rather than rely on implicit or automatic conversions, for these reasons:
...
If implicit data type conversion occurs in an index expression, then Oracle Database might not use the index because it is defined for the pre-conversion data type. This can have a negative impact on performance.
which is presumably why it actually chooses here to apply explicit conversion when it creates the index expression (which you can see in user_ind_expressions - fiddle)
But you'd get the same error if the index expression wasn't modified - there would still be an implicit conversion of 'c' to a number, and that would still throw ORA-01722. As would some strings that look like numbers if your NLS settings are incompatible.
Related
I am trying to create an Oracle Spatial index but seeing strange behavior.
I have a table in my schema as follows:
CREATE TABLE "Event" (
"EventID" NUMBER(32,0) GENERATED ALWAYS AS IDENTITY INCREMENT BY 1 START WITH 1 NOT NULL,
"Name" NVARCHAR2(30),
"Location" "SDO_GEOMETRY" NOT NULL,
CONSTRAINT "PK_EVENT" PRIMARY KEY ("EventID")
) ;
This works fine and I know I have to create an entry in user_sdo_geom_metadata, that works as you would expect with the following:
insert into user_sdo_geom_metadata (table_name,column_name,diminfo,srid) values ('Event','Location',
sdo_dim_array(sdo_dim_element('X',-180.0,180.0, 0.005),sdo_dim_element('Y',-90.0,90.0, 0.005)), 4326);
This reports success and when I do a select on user_sdo_geom_metadata I see the row. However, when I try to create the spatial index with:
CREATE INDEX "EVINDEX" ON "Event" ("Location") INDEXTYPE IS MDSYS.SPATIAL_INDEX_V2
I get the following error:
SQL Error [29855] [99999]: ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-13203: failed to read USER_SDO_GEOM_METADATA view
ORA-13203: failed to read USER_SDO_GEOM_METADATA view
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_10I", line 10
The weird thing is the Index looks like it's been created.
select * from all_indexes where table_name='Event';
Shows the index??? The other odd thing is when I do a select * on ALL_SDO_GEOM_METADATA, no rows are returned??? I'm connecting as a user with almost every privilege and role but not as SYSDBA. I can't get my head around this one.
UPDATE
Incredibly, this seems to be a case sensitivity issue. If you change the table and column names to all UPPERCASE it works. It seems my neverending disappointment in Oracle has a whole new chapter. Going to try to struggle through this somehow, but like most things with Oracle, it's one unrelenting slog to get anything done :(
The documentation says:
The table name cannot contain spaces or mixed-case letters in a quoted string when inserted into the USER_SDO_GEOM_METADATA view, and it cannot be in a quoted string when used in a query (unless it is in all uppercase characters).
and
The column name cannot contain spaces or mixed-case letters in a quoted string when inserted into the USER_SDO_GEOM_METADATA view, and it cannot be in a quoted string when used in a query (unless it is in all uppercase characters).
However, it also says:
All letters in the names are converted to uppercase before the names are stored in geometry metadata views or before the tables are accessed. This conversion also applies to any schema name specified with the table name.
which you can see if you query the user_sdo_geom_metadata view after your insert; the mixed-case names have become uppercase EVENT and LOCATION.
But then:
Note: Letter case conversion does not apply if you use mixed case (“CamelCase”) names enclosed in quotation marks. However, be aware that many experts recommend against using mixed-case names.
And indeed, rather unintuitively, it seems to work if you include the quotes in the user_sdo_geom_metadata insert:
insert into user_sdo_geom_metadata (table_name,column_name,diminfo,srid)
values (
'"Event"',
'"Location"',
sdo_dim_array(sdo_dim_element('X',-180.0,180.0, 0.005),
sdo_dim_element('Y',-90.0,90.0, 0.005)), 4326
);
db<>fiddle
So it appears that the values from the view are at some point concatenated into a dynamic SQL statement, which would explain some of the behaviour.
I've been troubled by my lack of understanding about an issue that periodically emerges: Function-Determinicity.
From the docs, it seems fairly clear:
A DETERMINISTIC function may not have side effects.
A DETERMINISTIC function may not raise an unhandled exception.
As these are important core concepts with robust, central implementations in standard packages, I don't think there is a bug or anything (the fault lies in my assumptions and understanding, not Oracle). That being said, both of these requirements sometimes appear to have some idiosyncratic uses within the standard package and the DBMS_ and UTL_ packages.
I hoped to post a couple of examples of Oracle functions that raise some doubts for me in my use of DETERMINISTIC and the nuances in these restrictions, and see if anyone can explain how things fit together. I apologize this is something of a "why" question and it can be migrated if needed, but the response to this question: (Is it ok to ask a question where you've found a solution but don't know why something was behaving the way it was?) made me think it might be appropriate for SO.
Periodically in my coding, I face uncertainty whether my own UDFs qualify as pure, and at other times, I use Oracle functions that surprise me greatly to learn they are impure. If anyone can take a look and advise, I would be grateful.
As a first example, TO_NUMBER. This function seems pure, but it also throws exceptions. In this example I'll use TO_NUMBER in a virtual column (DETERMINISTIC should be required here)
CREATE TABLE TO_NUMBER_IS_PURE_BUT_THROWS (
SOURCE_TEXT CHARACTER VARYING(5 CHAR) ,
NUMERICIZATION NUMBER(5 , 0) GENERATED ALWAYS AS (TO_NUMBER(SOURCE_TEXT , '99999')) ,
CONSTRAINT POSITIVE_NUMBER CHECK (NUMERICIZATION >= 0)
);
Table TO_NUMBER_IS_PURE_BUT_THROWS created.
INSERT INTO TO_NUMBER_IS_PURE_BUT_THROWS VALUES ('0',DEFAULT);
INSERT INTO TO_NUMBER_IS_PURE_BUT_THROWS VALUES ('88088',DEFAULT);
INSERT INTO TO_NUMBER_IS_PURE_BUT_THROWS VALUES ('UH-OH',DEFAULT);
1 row inserted.
1 row inserted.
ORA-01722: invalid number
The ORA-01722 would seem to violate the unhandled-exception requirement. Presumably any function I create that casts via TO_NUMBER should handle this possibility to remain pure. But throwing the exception here seems appropriate, and reliable. It seems there is some debate about whether exceptions violate referential-transparency (Why is the raising of an exception a side effect?)
The second situation I encounter is System functions that seem like they should-be DETERMINISTIC but arent't. There must be some reason they are considered impure. In some cases, it seems unfathomable that the internals would be generating side-effects.
An extreme example of this could be DBMS_ASSERT.NOOP though there are many others. The function returns its input unmodified. How can it be nondeterministic?
CREATE TABLE HOW_IS_NOOP_IMPURE (
SOURCE_TEXT VARCHAR2(256 BYTE),
COPY_TEXT VARCHAR2(256 BYTE) GENERATED ALWAYS AS (DBMS_ASSERT.NOOP(SOURCE_TEXT)),
CONSTRAINT COPY_IS_NOT_NULL CHECK(COPY_TEXT IS NOT NULL)
);
Yields:
ORA-30553: The function is not deterministic
Presumably it violates the requirements for determinicity, but that is hard to imagine. I wondered what I'm missing in my presumption that functions like this would be deterministic.
EDIT In response to Lukasz's comment about session settings:
I can accept it if cross-session repeatability is the root cause of functions like NOOPnot being DETERMINISTIC, but TO_CHAR is deterministic/eligible for use in virtual columns et al. but appears to have sensitivity to session settings in its format masks:
ALTER SESSION SET NLS_NUMERIC_CHARACTERS = '._';
Session altered.
CREATE TABLE TO_CHAR_NLS(
INPUT_NUMBER NUMBER(6,0),
OUTPUT_TEXT CHARACTER VARYING(64 CHAR) GENERATED ALWAYS AS (TO_CHAR(INPUT_NUMBER,'999G999'))
);
Table TO_CHAR_NLS created.
INSERT INTO TO_CHAR_NLS VALUES (123456,DEFAULT);
INSERT INTO TO_CHAR_NLS VALUES (111222,DEFAULT);
SELECT INPUT_NUMBER, OUTPUT_TEXT FROM TO_CHAR_NLS ORDER BY 1 ASC;
1 row inserted.
1 row inserted.
INPUT_NUMBER OUTPUT_TEXT
111222 111_222
123456 123_456
The ORA-01722 would seem to violate the unhandled-exception
requirement. Presumably any function I create that casts via TO_NUMBER
should handle this possibility to remain pure.
Firstly, i must appreciate you for asking such a good question. Now, when you say you used TO_NUMBER, it should convert all the text inputted to the function but you should know that TO_NUMBER has some restrictions.
As per TO_NUMBER definition:
The TO_NUMBER function converts a formatted TEXT or NTEXT expression
to a number. This function is typically used to convert the
formatted numerical output of one application (which includes currency symbols, decimal markers, thousands group markers, and so
forth) so that it can be used as input to another application.
It clearly says,it used to cast the formatted numerical output of one application, that means TO_NUMBER itself expect a numerical input and when you write as below:
INSERT INTO TO_NUMBER_IS_PURE_BUT_THROWS VALUES ('UH-OH',DEFAULT);
You completely passed the unexpected input to TO_NUMBER function and hence it throws the error ORA-01722: invalid number as expected behavior.
Read more about TO_NUMBER.
Secondly,
An extreme example of this could be DBMS_ASSERT.NOOP though there are
many others. The function returns its input unmodified. How can it be
nondeterministic?
DBMS_ASSERT.NOOP function is can be used where someone passing actual piece of code through a variable and don't want it to be checked for SQL injection attacks.
This has to be nondeterministic as it just return what we input to the function.
I show you a example to demonstrate why this has to be non-deterministic.
Let's say i create a function years_from_today as deterministic.
CREATE OR REPLACE FUNCTION years_from_today
( p_date IN DATE )
RETURN NUMBER DETERMINISTIC IS
BEGIN
RETURN ABS(MONTHS_BETWEEN(SYSDATE, p_date) / 12);
END years_from_today;
/
Now i create a table and use this function in a query as below:
CREATE TABLE det_test AS
SELECT TO_DATE('01-JUL-2009', 'DD-MON-YYYY') AS date_value
FROM dual;
SELECT date_value, SYSDATE, years_from_today(date_value)
FROM det_test
WHERE years_from_today(date_value) < 2;
Output
DATE_VALU SYSDATE YEARS_FROM_TODAY(DATE_VALUE)
--------- --------- ----------------------------
01-JUL-09 20-SEP-10 1.21861774
Then i create a function-based index on the new table.
CREATE INDEX det_test_fbi ON det_test (years_from_today(date_value));
Now, to see the implications of our DETERMINISTIC choice, change the date on the server (in a test environment of course) to move ahead a full year. Even though the date has changed, running the query again will still return the same value as before from YEARS_FROM_TODAY, along with the same row, because the index is used instead of executing the function.
SELECT date_value, SYSDATE, years_from_today(date_value)
FROM det_test
WHERE years_from_today(date_value) < 2;
Output:
DATE_VALU SYSDATE YEARS_FROM_TODAY(DATE_VALUE)
--------- --------- ----------------------------
01-JUL-09 20-SEP-11 1.2186201
Without the WHERE clause, the query should return the following:
DATE_VALU SYSDATE YEARS_FROM_TODAY(DATE_VALUE)
--------- --------- ----------------------------
01-JUL-09 20-SEP-11 2.21867063
As is evident from the erroneous output, a function should never be created as deterministic unless it will ALWAYS return the same value given the same parameters.
And hence your assumption to make DBMS_ASSERT.NOOP doesnot stands true in all the cases.
Could someone please explain to me the difference between the below two Oracle queries? I know they look very similar but the first one returns results and the second one does not. My implementation of the function can be seen below as well.
--Returns results
SELECT *
FROM <TABLE_NAME>
WHERE ID = CAST(<UserID> AS CHAR(2000)); --ID is defined as CHAR(8) in the DB.
--Does not return results
SELECT *
FROM <TABLE_NAME>
WHERE ID = CAST_TO_CHAR(<UserID>); --ID is defined as CHAR(8) in the DB.
--Function definition
CREATE OR REPLACE FUNCTION CAST_TO_CHAR(varToPad IN VARCHAR2)
RETURN CHAR IS returnVal CHAR(2000);
BEGIN
SELECT CAST(varToPad AS CHAR(2000))
INTO returnVal
FROM DUAL;
RETURN returnVal;
END;
/
It almost seems to me that the type is not persisting when the value is retrieved from the database. From what I understand from CHAR comparisons in Oracle, it will take the smaller of the two fields and truncate the larger one so that the sizes match (that is why I am casting the second variable to length 2000).
The reason that I need to achieve something like this is because a vendor tool that we are upgrading from DB2 to Oracle defined all of the columns in the Oracle database as CHAR instead of VARCHAR2. They did this to make their legacy code more easily portable to a distributed environment. This is causing big issues in our web applications because compares are now being done against fixed length CHAR fields.
I thought about using TRIM() but these queries will be accessed a lot and I do not want them to do a full table scan each time. I also considered RPAD(, ) but I don't really want to hard code lengths in the application as these may change in the future.
Does anyone have any thoughts about this? Thank you in advance for your help!
I have similar problem. It turned out that these are the rules of implicit data conversion. Oracle Database automatically converts a value from one datatype to another when such a conversion makes sense.
If you change your select:
SELECT *
FROM <TABLE_NAME>
WHERE CAST(ID as CHAR(2000)) = CAST_TO_CHAR(<UserID>);
You will see that's works properly.
And here's another test script showing that the function works correctly:
SET SERVEROUTPUT ON --for DBMS_OUTPUT.PUT_LINE.
DECLARE
test_string_c CHAR(8);
test_string_v VARCHAR2(8);
BEGIN
--Assign the same value to each string.
test_string_c := 'string';
test_string_v := 'string';
--Test the strings for equality.
IF test_string_c = CAST_TO_CHAR(test_string_v) THEN
DBMS_OUTPUT.PUT_LINE('The names are the same');
ELSE
DBMS_OUTPUT.PUT_LINE('The names are NOT the same');
END IF;
END;
/
anonymous block completed
The names are the same
Here are some rules govern the direction in which Oracle Database makes implicit datatype conversions:
During INSERT and UPDATE operations, Oracle converts the value to
the datatype of the affected column.
During SELECT FROM operations, Oracle converts the data from the
column to the type of the target variable.
When comparing a character value with a numeric value, Oracle
converts the character data to a numeric value.
When comparing a character value with a DATE value, Oracle converts
the character data to DATE.
When making assignments, Oracle converts the value on the right side
of the equal sign (=) to the datatype of the target of the assignment
on the left side.
When you use a SQL function or operator with an argument of a
datatype other than the one it accepts, Oracle converts the argument
to the accepted datatype.
Complete list of datatype comparison rules you can explore here
I am trying to create a hash with employee_id (NUMBER(6,0)) as the key and salary (NUMBER(8,2)) as the value.
For that I have created a INDEX-OF table(associative array) in PL/SQL (Oracle 11g) using the following definition:
TYPE emp_title_hash IS TABLE OF employees.salary%type
INDEX BY employees.employee_id%type;
I am getting the following compilation error:
Error(22,28): PLS-00315: Implementation restriction: unsupported table index type
I am aware in this case that the only supported type for the index is STRING or PLS_INTEGER. This seems to really restrictive. Why exactly has this been imposed in Oracle ? Is there a work around to get the above done ?
Appreciate your comments / suggestions.
As someone already pointed out, you can use "index by pls_integer", since pls_integer can contain any number(6,0) value.
Surely it would be nice to be able to use any possible type to index a pl/sql associative array, but I have always managed a way of writing a function that builds a string that identifies the object instance I want to use as index value.
So, instead of writing:
type TMyAssociativeArray is table of MyDataType index by MyIndexType;
I write:
Type TMyAssociativeArray is table of MyDataType index by varchar2(30);
then I write a function that calculates a unique string from MyIndexType:
function GetHash(obj MyIndexType) return varchar2;
and, having written this function I can use it to simulate an associative array indexed by MyIndexType object:
so I can write
declare
arr TMyAssociativeArray;
obj TMyDataType;
idx TMyDataType;
begin
....
arr(GetHash(idx)) := obj;
end;
Here I am getting out of the strict question you asked, and giving you an advice about a possible other way of obtaining a quick customer->salary lookup cache... This is the final purpose of your associative-array question, as I can read from your comment, so maybe this could be useful:
If you are using associative arrays to build a fast look-up mechanism, and if you can use the oracle 11g R2 new features, a easier way of obtaining this caching is to rely on the native RESULT_CACHE feature which has been introduced both for queries (by using the RESULT_CACHE hint) and for pl/sql functions.
for pl/sql functions you can create a function whose result value is cached like this one:
create or replace function cached_employee_salary(employee number)
return number RESULT_CACHE is
result number;
begin
select salary
into result
from employees e
where e.code = employee;
return result;
end;
the RESULT_CACHE keyword instructs oracle to keep an in-memory cache of the result values and to reuse them on subsequent calls (when the function is called with the same parameters, of course).
this cache has these advantages, compared to the use of associative arrays:
it is shared among all sessions: cached data is not kept in the private memory allocated for each session, so it wastes less memory
oracle is smart enough to detect that the function calculates its results by accessing the employees table and automatically invalidates the cached results if the table data is being modified.
Of course I suggest you to run some tests to see if this optimization, in your case gives some tangible results. it mostly depends on how complex is the calculation you make in your function.
You can also rely on an analogous feature introduced for SQL queries, triggered by the /+RESULT_CACHE/ hint:
select /*+RESULT_CACHE*/
salary
from employees e
where e.code = employee;
return result
this hint instructs oracle to store and reuse (on subsequent executions) the result set of the query. it will be stored too in memory.
actually this hint has also the advantage that, being the hint -syntactically speaking- a special comment, this query will continue working without any modifications even on servers < 11gR2, whereas, for the function cache version, you should use some "conditional compilation" magic to make it compile also with previous server versions (for which it would be a normal function without any result caching)
I Hope this helps.
In my backend procedure i have a varchar2 parameter and i am using it in the SQL query to search with number column. Will this cause any kind of performance issues ?
for ex:
Proc (a varchar)
is
select * from table where deptno = a;
end
Here deptno is number column in table and a is varchar .
It might do. The database will resolve the differences in datatype by casting DEPTNO to a VARCHAR2. This will prevent the optimizer from using any (normal) index you have on that column. Depending on the data volumes and distribution, an indexed read may not always be the most efficient access path, in which case the data conversion doesn't matter.
So it does depend. But what are your options if it does matter (you have a highly selective index on that column)?
One solution would be to apply an explicit data conversion in your query:
select * from table
where deptno = to_number(a);
This will cause the query to fail if A contains a value which won't convert to a number.
A better solution would be to change the datatype of A so that the calling program can only pass a numeric value. This throws the responsibility for duff data where it properly belongs.
The least attractive solution is to keep the procedure's signature and the query as is, and build a function-based index on the column:
create index emp_deptchar_fbi on emp(to_char(deptno));
Read the documentation to find out more about function-based indexes.