In my backend procedure i have a varchar2 parameter and i am using it in the SQL query to search with number column. Will this cause any kind of performance issues ?
for ex:
Proc (a varchar)
is
select * from table where deptno = a;
end
Here deptno is number column in table and a is varchar .
It might do. The database will resolve the differences in datatype by casting DEPTNO to a VARCHAR2. This will prevent the optimizer from using any (normal) index you have on that column. Depending on the data volumes and distribution, an indexed read may not always be the most efficient access path, in which case the data conversion doesn't matter.
So it does depend. But what are your options if it does matter (you have a highly selective index on that column)?
One solution would be to apply an explicit data conversion in your query:
select * from table
where deptno = to_number(a);
This will cause the query to fail if A contains a value which won't convert to a number.
A better solution would be to change the datatype of A so that the calling program can only pass a numeric value. This throws the responsibility for duff data where it properly belongs.
The least attractive solution is to keep the procedure's signature and the query as is, and build a function-based index on the column:
create index emp_deptchar_fbi on emp(to_char(deptno));
Read the documentation to find out more about function-based indexes.
Related
Creating a mySomeTable table with 2 fields
create table mySomeTable (
IDRQ VARCHAR2(32 CHAR),
PROCID VARCHAR2(64 CHAR)
);
Creating an index on the table by the PROCID field
create index idx_PROCID on mySomeTable(trunc(PROCID));
Inserting records:
insert into mySomeTable values ('a', '1'); -- OK
insert into mySomeTable values ('b', 'c'); -- FAIL
As you can see, an error has been made in the index construction script and the script will try to build an index on the field using the trunc() function.
trunct() is a function for working with dates or numbers, and the field has the string type
This index building script successfully works out and creates an index without displaying any warnings and errors.
An index is created on the table using the TRUNC(TO_NUMBER(PROCID)) function
When trying to insert or change an entry in the table, if PROCID cannot be converted to a number, I get the error ORA-01722: invalid number, which is actually logical.
However, the understanding that I am working in a table with rows and adding string values to the table, and the error is about converting to a number, was misleading and not understanding what is happening...
Question: Why does Oracle change the index construction function, instead of giving an error? And how can this be avoided in the future?
Oracle version 19.14
Naturally, there was only one solution - to create the right index with the right script
create index idx_PROCID on mySomeTable(PROCID);
however, this does not explain, to me, this Oracle behavior.
Oracle doesn't know if the index declaration is wrong or the column data type is wrong. Arguably (though some may well disagree!) Oracle shouldn't try to second-guess your intentions or enforce restrictions beyond those documented in the manual - that's what user-defined constraints are for. And, arguably, this index acts as a form of pseudo-constraint. That's a decision for the developer, not Oracle.
It's legal, if usually ill-advised, to store a number in a string column. If you actually intentionally chose to store numbers as strings - against best practice and possibly just to irritate future maintainers of your code - then the index behaviour is reasonable.
A counter-question is to ask where it should draw the line - if you expect it to error on your index expression, what about something like
create index idx_PROCID on mySomeTable(
case when regexp_like(PROCID, '^\d.?\d*$') then trunc(PROCID) end
);
or
create index idx_PROCID on mySomeTable(
trunc(to_number(PROCID default null on conversion error))
);
You might actually have chosen to store both numeric and non-numeric data in the same string column (again, I'm not advocating that) and an index like that might then useful - and you wouldn't want Oracle to prevent you from creating it.
Something that obviously doesn't make sense and shouldn't be allowed to you is much harder for software to evaluate.
Interestingly the documentation says:
Oracle recommends that you specify explicit conversions, rather than rely on implicit or automatic conversions, for these reasons:
...
If implicit data type conversion occurs in an index expression, then Oracle Database might not use the index because it is defined for the pre-conversion data type. This can have a negative impact on performance.
which is presumably why it actually chooses here to apply explicit conversion when it creates the index expression (which you can see in user_ind_expressions - fiddle)
But you'd get the same error if the index expression wasn't modified - there would still be an implicit conversion of 'c' to a number, and that would still throw ORA-01722. As would some strings that look like numbers if your NLS settings are incompatible.
We have an accountnumber stored in a Clob field in a table...we'll call it tbl_accountdetail. I need to pull back all records from tbl_accountdetail if the account numbers are in the results from another query...we'll call it sourcequery.
I can do this individually for each account number with:
Select * from Tbl_accountdetail where REGEXP_INSTR(CLOB,'accountnumber')>0
Naturally, my first thought was to do a cursor and loop through each account number from the sourcequery.
Declare #accountnumber varchar(30)
Declare Err_Cursor Cursor for
Select accountnumber from ErrorTable;
Open Err_Cursor;
Fetch next from Err_Cursor into #accountnumber;
While ##Fetch_status = 0
Begin
Select * from Tbl_accountdetail where REGEXP_INSTR(CLOB,#accountnumber)>0
Fetch next from Err_Cursor into #accountnumber
End;
Close Err_Cursor;
Deallocate Err_Cursor;
The more I read the more I'm confused about the best/most efficient way to get my desired results.
References to cursors all seem to require them to be included in a stored procedure and based on the simplicity, you wouldn't think this needs to be added to a sp. References to macros all seem to be macros that need to update/insert,etc. which I don't need. All I need to do is return the rows from Tbl_accountdetail that have the accountnumber somewhere in the clob.
I'm new to Teradata and Clobs. Can someone help me with the best way to search the clob? And to do so for a list of values?
Any help/suggestions greatly appreciated.
How is your CLOB data structured? Is the accountnumber field stored in a way that you can extract it using a searchable pattern -- i.e. accountnumber=<10-digit-#>?
If you want to search for multiple accountnumber values, then I think the best way is to extract the accountnumber value(s) from each CLOB and then search on them. Something like:
SELECT *
FROM Tbl_accountdetail
WHERE <extracted_accountnumber> IN (
SELECT account_number
FROM account_table
)
You are right that cursors are only used in stored procedures. The main purpose for them is to process each row of a result set individually and perform any additional logic, which you don't seem to need in your case. You could either put the SQL into a macro or just run it as-is.
Update
Assuming you only have one accountnum value in your CLOB field with a format of "accountnum": "123456789", you should be able to do something like this:
SELECT *
FROM Tbl_accountdetail
WHERE REGEXP_SUBSTR(myclob, '"accountnum":\s+"([0-9]+)"', 1, 1, NULL) IN (
SELECT account_number
FROM account_table
)
This should extract the first accountnumber match in your CLOB field and see if that value also exists in the IN sub-query.
I don't have a TD system to test on, so you may need to fiddle with the arguments a bit. Just replace myclob with the name of your CLOB field and update the sub-query in the IN(). Give that a try and let me know.
SQL Fiddle (Oracle)
Regexp Tester
Teradata - REGEXP_SUBSTR
Could someone please explain to me the difference between the below two Oracle queries? I know they look very similar but the first one returns results and the second one does not. My implementation of the function can be seen below as well.
--Returns results
SELECT *
FROM <TABLE_NAME>
WHERE ID = CAST(<UserID> AS CHAR(2000)); --ID is defined as CHAR(8) in the DB.
--Does not return results
SELECT *
FROM <TABLE_NAME>
WHERE ID = CAST_TO_CHAR(<UserID>); --ID is defined as CHAR(8) in the DB.
--Function definition
CREATE OR REPLACE FUNCTION CAST_TO_CHAR(varToPad IN VARCHAR2)
RETURN CHAR IS returnVal CHAR(2000);
BEGIN
SELECT CAST(varToPad AS CHAR(2000))
INTO returnVal
FROM DUAL;
RETURN returnVal;
END;
/
It almost seems to me that the type is not persisting when the value is retrieved from the database. From what I understand from CHAR comparisons in Oracle, it will take the smaller of the two fields and truncate the larger one so that the sizes match (that is why I am casting the second variable to length 2000).
The reason that I need to achieve something like this is because a vendor tool that we are upgrading from DB2 to Oracle defined all of the columns in the Oracle database as CHAR instead of VARCHAR2. They did this to make their legacy code more easily portable to a distributed environment. This is causing big issues in our web applications because compares are now being done against fixed length CHAR fields.
I thought about using TRIM() but these queries will be accessed a lot and I do not want them to do a full table scan each time. I also considered RPAD(, ) but I don't really want to hard code lengths in the application as these may change in the future.
Does anyone have any thoughts about this? Thank you in advance for your help!
I have similar problem. It turned out that these are the rules of implicit data conversion. Oracle Database automatically converts a value from one datatype to another when such a conversion makes sense.
If you change your select:
SELECT *
FROM <TABLE_NAME>
WHERE CAST(ID as CHAR(2000)) = CAST_TO_CHAR(<UserID>);
You will see that's works properly.
And here's another test script showing that the function works correctly:
SET SERVEROUTPUT ON --for DBMS_OUTPUT.PUT_LINE.
DECLARE
test_string_c CHAR(8);
test_string_v VARCHAR2(8);
BEGIN
--Assign the same value to each string.
test_string_c := 'string';
test_string_v := 'string';
--Test the strings for equality.
IF test_string_c = CAST_TO_CHAR(test_string_v) THEN
DBMS_OUTPUT.PUT_LINE('The names are the same');
ELSE
DBMS_OUTPUT.PUT_LINE('The names are NOT the same');
END IF;
END;
/
anonymous block completed
The names are the same
Here are some rules govern the direction in which Oracle Database makes implicit datatype conversions:
During INSERT and UPDATE operations, Oracle converts the value to
the datatype of the affected column.
During SELECT FROM operations, Oracle converts the data from the
column to the type of the target variable.
When comparing a character value with a numeric value, Oracle
converts the character data to a numeric value.
When comparing a character value with a DATE value, Oracle converts
the character data to DATE.
When making assignments, Oracle converts the value on the right side
of the equal sign (=) to the datatype of the target of the assignment
on the left side.
When you use a SQL function or operator with an argument of a
datatype other than the one it accepts, Oracle converts the argument
to the accepted datatype.
Complete list of datatype comparison rules you can explore here
I have created an Oracle table with an indexed varchar2 column, called 'ID'.
A software I'm using is reading this table, but instead of running queries like
select * from table_name where ID='something'
it does (notice the extra "N" before the value)
select * from table_name where ID=N'something'
which is causing some kind of character conversion.
The problem is that, while the 1st query is performing a range scan, the 2nd is performing a full table scan.
Since I cannot modify the queries that this software is running, which data type should I use, instead of varchar2, so that the conversion performed by the 'N' function does not imply a full table scan?
The prefix N before the string is used to specify a NVARCHAR2 or NCHAR datatype.
When comparing NVARCHAR2s to VARCHAR2s, Oracle converts the VARCHAR2 variable to NVARCHAR2. This is why you are experiencing a FULL SCAN.
Use a NVARCHAR2 column instead of a VARCHAR2 in your table if you can't modify the query.
How do I optimize the following SQL query for performance?
select * from Employee where CNIC = 'some-CNIC-number'
Will using alias help making it a little faster?
I am using Microsoft SQL Server.
This is better if you tell us what RDBMS you are using, but...
1 - Don't do SELECT *. Specify which columns you need. Less data = faster query
2 - For indexing, make sure you have an index on CNIC. You also want a good clustered index on a primary key (preferably something like an ID number)
3 - You put the number in single quotes ' ' which indicates you may have it as a varchar column. If it will always be NUMERIC, it should be an int/bigint data type. This takes up less space and will be faster to retrieve and index by.
Create an index on CNIC:
CREATE INDEX ix_employee_cnic ON employee (cnic)
First thing, as I see this column will be used for storing Id card nos, then you can make your coulmn of type int rather than varchar or nvarchar as searching will faster on an integer type as compared to varchar or nvarchar.
Second, use with (no lock), like
select * from Employee with (nolock) where CNIC = 'some-CNIC-number'
This is to minimize the chances of a deadlock.