Efficient query to find invalid number's in varchar column - oracle

I have already asked almost the same question before:
Finding non-numeric values in varchar column
And I received some great answer's and idea's to make not only fix the issue in my query but also make it much more efficient.
Old Query:
Select * From Table_Name
Where
(NOT REGEXP_LIKE(COLUMN_NAME, '^-?[0-9.]+$')
OR
LENGTH(Function_To_Fetch_Left_Component(COLUMN_NAME)) > (Precision-Scale)
OR
LENGTH(Column_Name) - LENGTH(REPLACE(Column_Name,'.','')) > 1)
New Query:
Select * From Table_Name
WHERE (Translate(column_name,'x0123456789-.','x') is not null
OR column_name = '.'
OR column_name = '-'
OR column_name = '-.'
OR Instr(column_name,'-') > 1
OR column_name like '%-%-%'
OR column_name like '%.%.%'
OR Instr(column_name||'.','.')-1 > (precision-scale) ) ;
Known limitations which are okay:
1.) Doesn't handle scientific notation. Its by design as I want to reject those.
2.) Doesn't check total precision (decimal component) however as while inserting oracle itself round's it off I am fine with it.
3.) Doesn't check for leading zero's.
Please help to see if the above query cover's everything and I haven't missing anything.
Another great solution by one Stackoverflow member however increased execution time by 2.5 times:
Function:
CREATE OR REPLACE function temp_is_number(p_test_value varchar2,
p_scale NUMBER,
p_precision NUMBER) return varchar2 is
l_result varchar2(1);
begin
execute immediate 'DECLARE l_number NUMBER; ' ||
'l_test_number NUMBER(' || p_precision || ',' || p_scale || '); ' ||
'BEGIN ' ||
' l_number := cast(:b1 as number); ' ||
' l_test_number := cast(:b1 as number); ' ||
' IF (l_number = l_test_number) THEN ' ||
' :b2 := ''Y''; ' ||
' ELSE ' ||
' :b2 := ''N''; ' ||
' END IF; ' ||
'EXCEPTION ' ||
' WHEN OTHERS THEN ' ||
' :b2 := ''N''; ' ||
'END;'
using in p_test_value, out l_result;
return l_result;
end;
/
Select Count(1) from Table_Name Where temp_is_number (Column_Name,scale,precision) = 'N';

Related

Best strategy to reset Oracle sequence monthly

I'm looking for the best strategy to reset a sequence which is used to generate unique keys. These keys are build using a prefix value, also generated by a combination of month and year values, like the following example:
2020060100000001 - Year(4d)Month(2d)Sequence(8d)
The sequence value must be restarted at the first second of each new month.
Is there any Oracle event that allows calling a function or procedure based in this situation to do this job?
Can anyone help me with opinions and experiences?
Thanks a lot!
Rodrigo
If you really want to set the value of a sequence you can use something like the following:
PROCEDURE SET_SEQUENCE(pinSequence_owner IN VARCHAR2,
pinSequence_name IN VARCHAR2,
pinNew_next_value IN NUMBER,
pinDebug IN BOOLEAN := FALSE)
IS
strSQL VARCHAR2(4000);
nNext_number NUMBER;
nOriginal_increment NUMBER;
nNew_nextval NUMBER;
nNew_last_number NUMBER;
BEGIN
strSQL := 'SELECT s.LAST_NUMBER, INCREMENT_BY ' ||
'FROM DBA_SEQUENCES s ' ||
'WHERE s.SEQUENCE_OWNER = ''' || pinSequence_owner || ''' AND ' ||
's.SEQUENCE_NAME = ''' || pinSequence_name || '''';
EXECUTE IMMEDIATE strSQL INTO nNext_number, nOriginal_increment;
-- Note that DBA_SEQUENCES.LAST_NUMBER represents the *next* number which will be
-- returned by a call to NEXTVAL.
IF pinNew_next_value NOT IN (nNext_number-1, nNext_number)
THEN
strSQL := 'ALTER SEQUENCE ' || pinSequence_owner || '.' || pinSequence_name ||
' INCREMENT BY ' || TO_CHAR(pinNew_next_value - nNext_number) || ' NOCACHE';
EXECUTE IMMEDIATE strSQL;
strSQL := 'SELECT ' || pinSequence_owner || '.' || pinSequence_name || '.NEXTVAL FROM DUAL';
EXECUTE IMMEDIATE strSQL INTO nNew_nextval;
strSQL := 'ALTER SEQUENCE ' || pinSequence_owner || '.' || pinSequence_name ||
' INCREMENT BY ' || nOriginal_increment || ' NOCACHE';
EXECUTE IMMEDIATE strSQL;
strSQL := 'SELECT s.LAST_NUMBER FROM DBA_SEQUENCES s WHERE s.SEQUENCE_OWNER = ''' || pinSequence_owner ||
''' AND s.SEQUENCE_NAME = ''' || pinSequence_name || '''';
EXECUTE IMMEDIATE strSQL INTO nNew_last_number;
END IF;
END SET_SEQUENCE;
Resetting sequence every month is not a valid approach. Basically you are violating the utilization of sequence. From your question i could understand, you wanted to append year and month to your column. In this case you can simple concatenate while inserting,
year||month||sequence_name.nextval
Eg: 2020||02||sequence_name.nextval
If you still wanted to reset, you can create a trigger to reset the sequence every month,

Getting a ORA-00933: SQL command not properly ended error when I try to run an Oracle stored procedure that I wrote

As part of a project, I'm working on data mapping of the peoplesoft records and fields in use at our company. There are more than 25K fields that I have to document but it gets tedious and will take a much more time than I have if I did it normal way. So, I wrote a stored procedure to reduce some of the work in documenting translate values.
Here is the code of my stored procedure:
CREATE OR REPLACE PROCEDURE SP_DATAMAPPINGINFO AS
TYPE EmpCurTyp IS REF CURSOR;
newrow_cursor EmpCurTyp;
txtable_cursor EmpCurTyp;
recname varchar2(30);
recdescr varchar2(200);
fieldnum number(3);
fieldname varchar2(30);
fieldescr varchar2(2000);
keyflag varchar2(1);
fieldtype varchar2(20);
distinctcount number(10);
query1_str varchar(300);
query2_str varchar(300);
query3_str varchar(300);
fieldvalue varchar2(200);
hyphen varchar2(5);
txvalue varchar2(200);
fielduse varchar2(500);
tablename varchar2(40);
intertxtabname varchar2(30);
txtablename varchar2(40);
CURSOR get_fields is
select A.RECNAME as "Record", A.RECDESCR as "Record Description"
, B.FIELDNUM as "FieldNum", B.FIELDNAME as "Field", C.DESCRLONG as "Field Description", CASE WHEN
EXISTS(select K.FIELDNAME FROM PSRECFLDDBKEYVW K WHERE K.RECNAME = A.RECNAME AND K.FIELDNAME=B.FIELDNAME)
THEN 'Y' ELSE 'N' END as "Key (Y/N)", DECODE (C.FIELDTYPE,
0, 'Character',
1, 'Long Char',
2, 'Number',
3, 'Signed Number',
4, 'Date',
5, 'Time',
6, 'DateTime',
8, 'Image/ Attachment',
9, 'Image Reference',
'Unknown') as "FieldType"
FROM PSRECDEFN A, PSRECFIELDDB B LEFT JOIN PSDBFIELD C ON (B.FIELDNAME = C.FIELDNAME)
WHERE B.RECNAME = A.RECNAME
AND A.RECNAME IN (select R.RECNAME from PSRECDEFN R, DBA_TABLES T
WHERE ('PS_'||R.RECNAME=T.TABLE_NAME)
AND T.NUM_ROWS > 0
AND R.RECTYPE=0)
order by A.RECNAME, B.FIELDNUM;
BEGIN
OPEN get_fields;
LOOP
FETCH get_fields INTO recname, recdescr, fieldnum, fieldname, fieldescr, keyflag, fieldtype;
fielduse := '';
tablename := 'PS_' || recname;
hyphen := ' - ';
fieldvalue := '';
txvalue := '';
intertxtabname := '';
txtablename := '';
if (fieldname <> '%EMPLID%' and fieldname <> '%DESCR%' and fieldname <> '%COMMENT%') THEN
query1_str := 'select RI.EDITTABLE FROM PSRECDEFN RD, PSRECFIELDDB RI WHERE RD.RECNAME = RI.RECNAME
AND RD.RECNAME = ' || recname || 'AND RI.FIELDNAME = ' || fieldname;
OPEN txtable_cursor FOR query1_str;
FETCH txtable_cursor INTO intertxtabname;
CLOSE txtable_cursor;
query2_str := 'select count(distinct T.' || fieldname || ') FROM ' || tablename;
IF (intertxtabname IS NOT NULL) THEN
txtablename := 'PS_' || intertxtabname;
query3_str := 'select distinct T.' || fieldname || ', TR.DESCR FROM ' || tablename || ' T left join ' || txtablename || ' TR ON T.'
|| fieldname || ' = TR.' || fieldname || ' order by T.' || fieldname;
ELSE
txtablename := '';
query3_str := 'select distinct DT.' || fieldname || ', DTR.XLATLONGNAME FROM ' || tablename || ' DT left join PSXLATITEM DTR on
(DTR.FIELDNAME = ''' || fieldname || ''' and DT.' || fieldname || ' = DTR.FIELDVALE) order by DT.' || fieldname;
END IF;
execute immediate query2_str into distinctcount;
if(distinctcount > 150) THEN
fielduse := 'More than 150';
ELSE
OPEN newrow_cursor FOR query3_str USING 'fieldname';
LOOP
FETCH newrow_cursor INTO fieldvalue, txvalue;
fielduse := fieldvalue || ' - ' || txvalue;
EXIT WHEN newrow_cursor%NOTFOUND;
END LOOP;
CLOSE newrow_cursor;
END IF;
ELSE
fielduse := 'SKIPPING';
END IF;
dbms_output.put_line(recname || ',' || recdescr || ',' || fieldnum || ',' || fieldname || ',' || fieldescr || ',' || keyflag || ',' || fieldtype || ',' || fielduse);
END LOOP;
CLOSE get_fields;
NULL;
END SP_DATAMAPPINGINFO;
The stored proc compiles without any errors but when I execute it, I get the following error:
Error starting at line : 1 in command -
BEGIN SP_DATAMAPPINGINFO; END;
Error report -
ORA-00933: SQL command not properly ended
ORA-06512: at "SYSADM.SP_DATAMAPPINGINFO", line 69
ORA-06512: at line 1
00000 - "SQL command not properly ended"
*Cause:
*Action:
Line 69 in the stored proc is OPEN txtable_cursor FOR query1_str;
I have tried using a bind variable but I still get the error and I might have used it incorrectly.
I may have other issues in the code. It'll be great if you can point those out. We are currently at Oracle 12c
Line 69 in the stored proc is OPEN txtable_cursor FOR query1_str;.
Obviously we can't run your code as we don't have your schema so we can't compile and validate your dynamic SQL. However, looking at your assembled string, the striking thing is the concatenation of the boilerplate text and the columns:
AND RD.RECNAME = ' || recname || 'AND RI.FIELDNAME = ' || fieldname
If you look closely you will spot that there is no space after the quote in 'AND RI.FIELDNAME which means the value of recname is concatenated with AND. That's your ORA-00933 right there.
Once you fix that you'll probably run into ORA-00936: missing expression. From the variable declarations we know recname and fieldname are strings but you are not treating them as strings. The code doesn't include any quotes around the variables so the executed query will treat them as columns or something. So that's something you need to fix:
AND RD.RECNAME = ''' || recname || ''' AND RI.FIELDNAME = ''' || fieldname ||''''
Later on you open the cursor like this:
OPEN newrow_cursor FOR query3_str USING 'fieldname';
So you are passing a parameter to a dynamic query which doesn't include any parameter placeholders. That should throw
ORA-01006: bind variable does not exist
Dynamic SQL is hard because it turns compilation errors into runtime errors. You need to develop a cool eye for looking at your code to spot them. One thing which will help is using logging to record the dynamic statement (DBMS_OUTPUT is better than nothing). Usually when we have the actual statement in front of our eyes it is easy to spot our bloomers. But if we can't at least we can run the query and let the compiler point them out.

Insert into by chunks

I am trying to use insert into statement, but getting the error:
ORA-01628: max # extents (32765) reached for rollback segment _SYSSMU134_1882489978$
Increasing the UNDO tablespace is not an option, so I would like a way to insert those data by chunks (for example with 1 million rows at a time). Can someone help to rewrite this procedure in that way?
CREATE OR REPLACE PROCEDURE create_chunks (
p_source_table IN VARCHAR2,
p_table_name_chunk IN VARCHAR2,
p_chunks IN VARCHAR2
) AS
v_insert_sql CLOB;
BEGIN
v_insert_sql := 'INSERT INTO ' || p_table_name_chunk ||
' (rid, chunk_number) ' ||
'SELECT /*+ parallel(64) */ rowid rid,' ||
'mod( ora_hash(rowid), :p_chunks ) as chunk_number '
'FROM ' || p_source_table;
EXECUTE IMMEDIATE v_insert_sql USING p_chunks;
COMMIT;
END;
This v_insert_sql is failing with above mentioned error. I have working solution using the cursor fetching like that:
DECLARE
CURSOR v_cur IS SELECT /*+ parallel(64) */
rowid rid, mod( ora_hash(rowid), 20000 ) AS chunk_number
-- I need this table to be parametric name
FROM some_table;
TYPE t_sample IS TABLE OF v_cur%ROWTYPE;
v_sample t_sample;
v_row_limit CONSTANT NUMBER := 1000000;
BEGIN
OPEN v_cur;
LOOP
FETCH v_cur BULK COLLECT INTO v_sample LIMIT v_row_limit;
FORALL i IN v_sample.first .. v_sample.last
INSERT INTO chunk_table VALUES v_sample(i);
COMMIT;
EXIT WHEN v_cur%NOTFOUND;
END LOOP;
CLOSE v_cur;
END;
I can't move this cursor straight into the procedure as the table name is varying and I need it to be parametric as with cursor approach I have to repeat the same code for different tables. So the question is how to deal with this?
Basically, I was able to just past the whole chunks approach inside procedure as dynamic query, like that:
CREATE OR REPLACE PROCEDURE create_chunks (
p_source_table IN VARCHAR2,
p_table_name_chunk IN VARCHAR2,
p_chunks IN VARCHAR2
) AS
v_insert_sql CLOB;
BEGIN
v_insert_sql := '' ||
' DECLARE ' || CHR(10) ||
' CURSOR cur1 IS SELECT ' || CHR(10) ||
' /*+ parallel(64) full(tbn)*/ rowid rid,' || CHR(10) ||
' mod( ora_hash(rowid), :p_chunks ) AS chunk_number' || CHR(10) ||
' FROM ' || p_source_table || ' tbn;' || CHR(10) ||
' TYPE t_sample IS TABLE OF cur1%ROWTYPE;' || CHR(10) ||
' v_sample t_sample;' || CHR(10) ||
' v_row_limit CONSTANT NUMBER := 1000000;' || CHR(10) ||
' BEGIN' || CHR(10) ||
' OPEN cur1;' || CHR(10) ||
' LOOP' || CHR(10) ||
' FETCH cur1 BULK COLLECT INTO v_sample LIMIT v_row_limit;' || CHR(10) ||
' FORALL i IN v_sample.first .. v_sample.last' || CHR(10) ||
' INSERT INTO ' || p_table_name_chunk || ' VALUES v_sample(i);' || CHR(10) ||
' COMMIT;' || CHR(10) ||
' EXIT WHEN cur1%NOTFOUND;' || CHR(10) ||
' END LOOP;' || CHR(10) ||
' CLOSE cur1;' || CHR(10) ||
' END;';
EXECUTE IMMEDIATE v_insert_sql USING p_chunks;
COMMIT;
END;

Execute Immediate in oracle

I have below query which gives an error as encounter an symbol ( in the line where loop is used. I am trying to develop a function which takes dynamic paramater as table_name,column_name,table_id and used for other tables as well.
FUNCTION get_encryp_pass( table_name IN varchar2,column_name IN varchar2,table_id IN varchar2) RETURN VARCHAR2
IS
BEGIN
EXECUTE IMMEDIATE 'for c1 in (select * from' || table_name ||) loop
EXECUTE IMMEDIATE 'update ' || table_name || ' set ' || column_name = encrypt_val(c1.column_name) || ' where ' || table_id || ' = ' || c1.table_id and column_name is not null;
end loop;
END get_encrypt_pass;
this should work:
CREATE PROCEDURE get_encryp_pass(table_name IN varchar2,
column_name IN varchar2,
table_id IN varchar2) IS
BEGIN
EXECUTE IMMEDIATE 'begin for c1 in (select * from ' || table_name ||
') loop update ' || table_name || ' set ' ||
column_name || ' = encrypt_val(c1.' || column_name ||
') where ' || table_id || ' = c1.'||table_id||' and ' || column_name ||
' is not null; end loop; end;'
;
END;
But why not simply call update FTP_SFTP_SERVER set PASSWORD=encrypt_val(PASSWORD) where PASSWORD is not null ?
keep care of what is a variable and what is a string-literal and must be single-quoted therefore ... and string-variables mus be double-quoted:
EXECUTE IMMEDIATE 'update ' || table_name || ' set ' || column_name || ' = ''' || encrypt_val(c1.column_name) || ''' where ' || table_id || ' = ' || c1.table_id || ' and column_name is not null';
Best practice is to concatenate the statement in a varchar2-variable first and inspect this. If the content of the variable is syntactical correct and executable, the EXECUTE IMMEDIATE should work as well
declare
stmt varchar2(4000);
begin
stmt := 'update ' || table_name || ' set ' || column_name || ' = ''' || encrypt_val(c1.column_name) || ''' where ' || table_id || ' = ' || c1.table_id || ' and column_name is not null';
EXECUTE IMMEDIATE stmt;
end;
I think i have one alternative for your question. MERGE can be used in this case. Please try it i dont have workspaceso dint test it but it should work Let me know if it helps.
FUNCTION get_encryp_pass(
table_name IN VARCHAR2,
column_name IN VARCHAR2,
table_id IN VARCHAR2)
RETURN VARCHAR2
IS
BEGIN
EXECUTE IMMEDIATE 'MERGE INTO '||table_name||' t1 USING
(
SELECT * FROM '||table_name|| ')t2
ON
(
t1.table_id = t2.table_id
)
WHEN MATCHED THEN
UPDATE SET t1.'||column_name||' = encrypt_val(t2.'||column_name||')'
||' WHERE and t1.'||column_name||' IS NOT NULL';
END;

Procedure failed due to ORA-00920: invalid relational operator

Getting below error while executing in 12c oracle,but it is working fine when it was in oracle 10.2.0.4 :
Procedure proc_up_dts_product_cat_dynsql failed due to ORA-00920: invalid
relational operator
Here is the procedure :
CREATE OR REPLACE PROCEDURE SEAPROB.proc_up_dts_product_cat_dynsql(tablename IN varchar)
AS
dynamicsql varchar(8000);
ID int;
DTS_Segment_op varchar2(10);
DTS_Segment varchar2(15);
DTS_Segment_where varchar2(255);
DateEntered_op varchar2(10);
DateEntered varchar2(30);
DateEntered_where varchar2(255);
Svc_Name_op varchar2(10);
Svc_Name varchar2(100);
Svc_Name_where varchar2(255);
Product_Category varchar2(75);
Priority int;
combined_status_where varchar2(255);
refdate date ;
CURSOR PrivCursor
IS
SELECT
ID,
DTS_Segment_op,
Nvl(upper(trim(DTS_Segment)),' '),
DateEntered_op,
CASE WHEN dateentered='%' THEN dateentered
WHEN dateentered LIKE '%/%/____' THEN To_Char(To_Date(dateentered,'MM/DD/YYYY'),'YYYY-MM-DD')
WHEN dateentered LIKE '%/%/__' THEN To_Char(To_Date(dateentered,'MM/DD/YY'),'YYYY-MM-DD')
WHEN dateentered LIKE '''%/%/%'' % ''%/%/%''' THEN ''''||To_Char(To_Date(SubStr(dateentered,InStr(dateentered,'''',1,1)+1,InStr(dateentered,'''',1,2)-InStr(dateentered,'''',1,1)-1),'MM/DD/YY'),'YYYY-MM-DD')||''' and '''||To_Char(To_Date(SubStr(dateentered,InStr(dateentered,'''',1,3)+1,InStr(dateentered,'''',1,4)-InStr(dateentered,'''',1,3)-1),'MM/DD/YY'),'YYYY-MM-DD')||''''
ELSE dateentered END AS dateentered,
Svc_Name_op,
Nvl(upper(trim(Svc_Name)),' '),
Product_Category,
Priority
FROM tbl_dts_pt_lookup order by priority desc;
BEGIN
refdate := ADD_MONTHS(to_date(SYSDATE,'dd-mon-yy'),-6) ;
OPEN PrivCursor;
-- Loop through all the rows in the tbl_dts_category_lookup table
FETCH PrivCursor
INTO
ID,
DTS_Segment_op,
DTS_Segment,
DateEntered_op,
DateEntered,
Svc_Name_op,
Svc_Name,
Product_Category,
Priority;
WHILE PrivCursor%found
LOOP
-- Create dynamic SQL
--define case statements for where clause components
combined_status_where := ' where (DTS_Cm_DisputeStatus <>'|| '''C''' || ' OR ( DTS_Cm_DisputeStatus='|| '''C''' || ' AND DTS_CM_CLOSEDATE >= '''||refdate||'''))' ;
dts_segment_where := case when dts_segment='%' then ' and 1=1' else ' and NVL(trim(Replace(Upper(segment),chr(0),''' || ''')),''' || ' '') ' || dts_segment_op || ' ''' || dts_segment || '''' end;
svc_name_where := case when svc_name='%' then ' and 1=1' else ' and NVL(trim(Replace(Upper(dts_cm_servicename),chr(0),''' || ''')),''' || ' '') ' || svc_name_op || ' ''' || svc_name || '''' end ;
dateentered_where := case when dateentered='%' then ' and 1=1'
when dateentered_op='between' then ' and TO_CHAR(dts_cm_dateentered,''YYYY-MM-DD'') between ' || dateentered
else ' and TO_CHAR(dts_cm_dateentered,''YYYY-MM-DD'') ' || dateentered_op || ' ''' || dateentered || '''' end ;
dynamicsql := 'update '||tablename||' set product_cat_id=' || cast(id as varchar) ||', product_category =''' || product_category || '''';
--add where clause
dynamicsql := dynamicsql || combined_status_where || dts_segment_where || dateentered_where || svc_name_where;
EXECUTE IMMEDIATE dynamicsql;
COMMIT;
FETCH PrivCursor
INTO
ID,
DTS_Segment_op,
DTS_Segment,
DateEntered_op,
DateEntered,
Svc_Name_op,
Svc_Name,
Product_Category,
Priority;
END LOOP;
CLOSE PrivCursor;
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE('Procedure proc_up_dts_product_cat_dynsql failed due to '||SQLERRM);
END proc_up_dts_product_cat_dynsql;
/
The database hurls ORA-00920: invalid relational operator when we submit a SQL statement with a syntax error in a WHERE clause. Usually it's a typo. This is extremely easy to do in dynamic SQL, because we cannot see the whole statement except at run time.
You haven't posted the generated update statement. Well you can't, as your exception handling doesn't display it. Pro tip: when working with dynamic SQL always log or display the generated statement, in the exception block if nowhere else:
dbms_output.put_line('generated statement:'|| dynamicsql);
So this is just a guess but this line looks suspicious ...
when dateentered_op='between' then ' and TO_CHAR(dts_cm_dateentered,''YYYY-MM-DD'') between ' || dateentered
... because it appears to generate a BETWEEN statement with no AND clause. The BETWEEN operator requires two bounds.
But it could be many things. You may be running with an unusual combination of parameters which generates an invalid statement. You need better logging.

Resources