I need to copy stats from production to test database but all sample code I found use create_stat_table procedure, but I cannot create tables on source database. Is there any chance to do this different way? Maybe I can generate some INSERT statements from system views (like dba_tab_statistics) and run on destination db?
You can use the get_*_stats to fetch the details from prod, then pass these by calling set_*_stats in test.
For example, get the stats like this:
create table t as
select level tid, date'2022-01-01' + level c1
from dual
connect by level <= 100;
exec dbms_stats.gather_table_stats ( user, 't' ) ;
declare
numrows number;
numblks number;
avgrlen number;
begin
dbms_stats.get_table_stats (
ownname => user,
tabname => 'T',
numrows => numrows,
numblks => numblks,
avgrlen => avgrlen
);
dbms_output.put_line (
numrows || '; ' || numblks || '; ' || avgrlen
);
end;
/
100; 4; 11
Then load them like this (I'm dropping the table in the same DB for simplicity)
drop table t purge;
create table t ( tid integer, c1 date );
NUM_ROWS BLOCKS AVG_ROW_LEN
---------- ---------- -----------
<null> <null> <null>
select num_rows, blocks, avg_row_len
from user_tables
where table_name = 'T';
begin
dbms_stats.set_table_stats (
ownname => user,
tabname => 'T',
numrows => 10,
numblks => 4,
avgrlen => 11
);
end;
/
select num_rows, blocks, avg_row_len
from user_tables
where table_name = 'T';
NUM_ROWS BLOCKS AVG_ROW_LEN
---------- ---------- -----------
10 4 11
Copying the stats for all the columns and indexes this way is a faff though. If you need this, speak with whoever manages the database and get them to copy these over for you!
Related
I was asked to migrate this code from Sql server
It's used to count all rows with a cursor in a database and we need it to work both on sql server and oracle. The sql server part is working fine, but the oracle part i can't figure out.
DECLARE #tablename VARCHAR(500)
DECLARE #rowname VARCHAR(500)
DECLARE #sql AS NVARCHAR(1000)
IF OBJECT_ID('tempdb..#Temp1') IS NOT NULL
DROP TABLE #Temp1
CREATE TABLE #Temp1
(
tablename varchar(max) ,
rowCounter NUMERIC(38) ,
idColumn varchar(max),
SumIdCol NUMERIC(38)
);
DECLARE db_cursor CURSOR FOR
SELECT s.name + '.' + o.name as name, c.COLUMN_NAME
FROM sys.all_objects o
join sys.schemas s on o.schema_id=s.schema_id
join INFORMATION_SCHEMA.COLUMNS c on o.name=c.TABLE_NAME
WHERE type = 'U' and s.name not like 'sys' and c.ORDINAL_POSITION=1 and c.DATA_TYPE='int'
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #tablename,#rowname
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql =
'
insert into #Temp1 values('+
''''+#tablename+''','+
'(SELECT count(1) FROM '+#tablename+' ),'+
''''+#rowname+''''+','+
'(SELECT SUM(['+#rowname+']) FROM '+#tablename+' )'+
')'
EXEC sp_executesql #sql
FETCH NEXT FROM db_cursor INTO #tablename,#rowname
END
CLOSE db_cursor
DEALLOCATE db_cursor
select * from #Temp1
IF OBJECT_ID('tempdb..#Temp1') IS NOT NULL
DROP TABLE #Temp1
Into oracle. I decided to take on PL/SQL and wrote this:
DECLARE
v_tablename varchar2(400);
v_rowname varchar2(400);
cursor db_cursor is
select ALL_OBJECTS.OWNER || '.' || USER_TABLES.TABLE_NAME AS Tablename, USER_TAB_COLS.COLUMN_NAME AS columnName
from USER_TABLES
inner join user_tab_cols ON USER_TAB_COLS.TABLE_NAME=USER_TABLES.TABLE_NAME
inner join ALL_OBJECTS ON user_tab_cols.TABLE_NAME=ALL_OBJECTS.OBJECT_NAME
WHERE ALL_OBJECTS.OBJECT_TYPE = 'TABLE' AND USER_TAB_COLS.COLUMN_ID = 1;
begin
open db_cursor;
loop
fetch db_cursor into v_tablename,v_rowname;
EXIT WHEN db_cursor%NOTFOUND;
execute immediate 'INSERT INTO Temp1(tablename,rowCounter,idColumn,SumIdCol)
SELECT ''' || v_tablename || ''' ,COUNT(*), ''' || v_rowname ||''',SUM(''' || v_rowname ||''') FROM ' || v_tablename;
end loop;
CLOSE db_cursor;
END;
The issue i have is that i get Invalid table or view at line 42 (This line is EXIT WHEN db_cursor%NOTFOUND;)
I know this solution isn't the best but any help would be appreciated. Thanks
I'm not quite sure what are the last two columns in the temp1 table supposed to contain; the first column (why?) and some "sum" value, but - which one?
Anyway: in Oracle, you might start with this code (that compiles and does something; could/should be improved, perhaps):
Target table:
SQL> create table temp1
2 (tablename varchar2(70),
3 rowcounter number,
4 idcolumn varchar2(30),
5 sumidcol number
6 );
Table created.
Anonymous PL/SQL block; note that you need just all_tables and all_tab_columns as they contain all info you need. If you use all_objects, you have to filter tables only (so, why use it at all?).
As all_ views contain all tables/columns you have access to, I filtered only tables owned by user SCOTT because - if I left them all, I'd get various owners (such as SYS and SYSTEM - do you need them too?) and 850 tables.
SQL> select count(distinct owner) cnt_owner, count(*) cnt_tables
2 from all_tables;
CNT_OWNER CNT_TABLES
---------- ----------
19 850
SQL>
Maybe you'd actually want to use user_ views instead.
This code lists all tables, the first column and number of rows in each table:
SQL> declare
2 l_cnt number;
3 begin
4 for cur_r in (select a.owner,
5 a.table_name,
6 c.column_name
7 from all_tables a join all_tab_columns c on c.owner = a.owner
8 and c.table_name = a.table_name
9 where c.column_id = 1
10 and a.owner in ('SCOTT')
11 )
12 loop
13 execute immediate 'select count(*) from ' || cur_r.owner ||'.'||
14 '"' || cur_r.table_name ||'"' into l_cnt;
15
16 insert into temp1 (tablename, rowcounter, idcolumn, sumidcol)
17 values (cur_r.owner ||'.'||cur_r.table_name, l_cnt, cur_r.column_name, null);
18 end loop;
19 end;
20 /
PL/SQL procedure successfully completed.
Result is then:
SQL> select * from temp1 where rownum <= 10;
TABLENAME ROWCOUNTER IDCOLUMN SUMIDCOL
----------------------------------- ---------- --------------- ----------
SCOTT.ACTIVE_YEAR 1 YEAR
SCOTT.BONUS 0 ENAME
SCOTT.COPY_DEPARTMENTS 1 DEPARTMENT_ID
SCOTT.DAT 0 DA
SCOTT.DEPARTMENTS 1 DEPARTMENT_ID
SCOTT.DEPT 4 DEPTNO
SCOTT.DEPT_BACKUP 4 DEPTNO
SCOTT.EMP 14 EMPNO
SCOTT.EMPLOYEE 4 EMP_ID
SCOTT.EMPLOYEES 9 EMPLOYEE_ID
10 rows selected.
SQL>
You are getting the Table not found exception because you have lower-case table names and you are using unquoted identifiers, which Oracle will implicitly convert to upper-case and then because they are upper- and not lower-case they are not found.
You need to use quoted identifiers so that Oracle respects the case-sensitivity in the table names. You can also use an implicit cursor and can pass the values using bind variables (where possible) rather than using string concatenation:
BEGIN
FOR rw IN (SELECT t.owner,
t.table_name,
c.column_name
FROM all_tables t
INNER JOIN all_tab_cols c
ON (t.owner = c.owner AND t.table_name = c.table_name)
WHERE t.owner = USER
AND c.column_id = 1)
LOOP
EXECUTE IMMEDIATE 'INSERT INTO Temp1(tablename,rowCounter,idColumn)
SELECT :1, COUNT(*), :2 FROM "' || rw.owner || '"."' || rw.table_name || '"'
USING rw.table_name, rw.column_name;
END LOOP;
END;
/
fiddle
I would like to replace every string are null by 'n' and every number by 0.
Is there a way to do that?
With a polymorphic table function I can select all the columns of a certain type but I can't modify the value of the columns.
Yes, it is possible with PTF also. You may modify columns in case you've set pass_through to false for the column in the describe method (to drop it) and copy it into new_columns parameter of the describe.
Below is the code:
create package pkg_nvl as
/*Package to implement PTF*/
function describe(
tab in out dbms_tf.table_t
) return dbms_tf.describe_t
;
procedure fetch_rows;
end pkg_nvl;
/
create package body pkg_nvl as
function describe(
tab in out dbms_tf.table_t
) return dbms_tf.describe_t
as
modif_cols dbms_tf.columns_new_t;
new_col_cnt pls_integer := 0;
begin
/*Mark input columns as used and as modifiable for subsequent row processing*/
for i in 1..tab.column.count loop
if tab.column(i).description.type in (
dbms_tf.type_number,
dbms_tf.type_varchar2
) then
/*Modifiable*/
tab.column(i).pass_through := FALSE;
/*Used in the PTF context*/
tab.column(i).for_read := TRUE;
/* Propagate column to the modified*/
modif_cols(new_col_cnt) := tab.column(i).description;
new_col_cnt := new_col_cnt + 1;
end if;
end loop;
/*Return the list of modified cols*/
return dbms_tf.describe_t(
new_columns => modif_cols
);
end;
procedure fetch_rows
/*Process rowset and replace nulls*/
as
rowset dbms_tf.row_set_t;
num_rows pls_integer;
in_col_vc2 dbms_tf.tab_varchar2_t;
in_col_num dbms_tf.tab_number_t;
new_col_vc2 dbms_tf.tab_varchar2_t;
new_col_num dbms_tf.tab_number_t;
begin
/*Get rows*/
dbms_tf.get_row_set(
rowset => rowset,
row_count => num_rows
);
for col_num in 1..rowset.count() loop
/*Loop through the columns*/
for rn in 1..num_rows loop
/*Calculate new values in the same row*/
/*Get column by index and nvl the value for return column*/
if rowset(col_num).description.type = dbms_tf.type_number then
dbms_tf.get_col(
columnid => col_num,
collection => in_col_num
);
new_col_num(rn) := nvl(in_col_num(rn), 0);
elsif rowset(col_num).description.type = dbms_tf.type_varchar2 then
dbms_tf.get_col(
columnid => col_num,
collection => in_col_vc2
);
new_col_vc2(rn) := nvl(in_col_vc2(rn), 'n');
end if;
end loop;
/*Put the modified column to the result*/
if rowset(col_num).description.type = dbms_tf.type_number then
dbms_tf.put_col(
columnid => col_num,
collection => new_col_num
);
elsif rowset(col_num).description.type = dbms_tf.type_varchar2 then
dbms_tf.put_col(
columnid => col_num,
collection => new_col_vc2
);
end if;
end loop;
end;
end pkg_nvl;
/
create function f_replace_nulls(tab in table)
/*Function to replace nulls using PTF*/
return table pipelined
row polymorphic using pkg_nvl;
/
with a as (
select
1 as id, 'q' as val_vc2, 1 as val_num
from dual
union all
select
2 as id, '' as val_vc2, null as val_num
from dual
union all
select
3 as id, ' ' as val_vc2, 0 as val_num
from dual
)
select
id
, a.val_num
, a.val_vc2
, n.val_num as val_num_repl
, n.val_vc2 as val_vc2_repl
from a
join f_replace_nulls(a) n
using(id)
ID | VAL_NUM | VAL_VC2 | VAL_NUM_REPL | VAL_VC2_REPL
-: | ------: | :------ | -----------: | :-----------
3 | 0 | | 0 |
2 | null | null | 0 | n
1 | 1 | q | 1 | q
db<>fiddle here
Use COALESCE or NVL and list the columns you want to apply them to:
SELECT COALESCE(col1, 'n') AS col1,
COALESCE(col2, 0) AS col2,
COALESCE(col3, 'n') AS col3
FROM table_name;
The way I understood the question, you actually want to modify table's contents. If that's so, you'll need dynamic SQL.
Here's an example; sample data first, with some numeric and character columns having NULL values:
SQL> desc test
Name Null? Type
----------------------------------------- -------- ----------------------------
ID NUMBER
NAME VARCHAR2(20)
SALARY NUMBER
SQL> select * from test order by id;
ID NAME SALARY
---------- ---------- ----------
1 Little 100
2 200
3 Foot 0
4 0
Procedure reads USER_TAB_COLUMNS, checking desired data types (you can add some more, if you want), composes the update statement and executes it:
SQL> declare
2 l_str varchar2(1000);
3 begin
4 for cur_r in (select column_name, data_type
5 from user_tab_columns
6 where table_name = 'TEST'
7 and data_type in ('CHAR', 'VARCHAR2')
8 )
9 loop
10 l_str := 'update test set ' ||
11 cur_r.column_name || ' = nvl(' || cur_r.column_name ||', ''n'')';
12 execute immediate l_str;
13 end loop;
14
15 --
16
17 for cur_r in (select column_name, data_type
18 from user_tab_columns
19 where table_name = 'TEST'
20 and data_type in ('NUMBER')
21 )
22 loop
23 l_str := 'update test set ' ||
24 cur_r.column_name || ' = nvl(' || cur_r.column_name ||', 0)';
25 execute immediate l_str;
26 end loop;
27 end;
28 /
PL/SQL procedure successfully completed.
Result:
SQL> select * from test order by id;
ID NAME SALARY
---------- ---------- ----------
1 Little 100
2 n 200
3 Foot 0
4 n 0
SQL>
You can create a table macro to intercept the columns and replace them with nvl if they're a number or varchar2:
create or replace function replace_nulls (
tab dbms_tf.table_t
)
return clob sql_macro as
stmt clob := 'select ';
begin
for i in 1..tab.column.count loop
if tab.column(i).description.type = dbms_tf.type_number
then
stmt := stmt || ' nvl ( ' ||
tab.column(i).description.name || ', 0 ) ' ||
tab.column(i).description.name || ',';
elsif tab.column(i).description.type = dbms_tf.type_varchar2
then
stmt := stmt || ' nvl ( ' ||
tab.column(i).description.name || ', ''n'' ) ' ||
tab.column(i).description.name || ',';
else
stmt := stmt || tab.column(i).description.name || ',';
end if;
end loop;
stmt := rtrim ( stmt, ',' ) || ' from tab';
return stmt;
end;
/
with a (
aa1,aa2,aa3
) as (
select 1, '2hhhh', sysdate from dual
union all
select null, null, null from dual
)
select * from replace_nulls(a);
AA1 AA2 AA3
---------- ----- -----------------
1 2hhhh 27-JUN-2022 13:17
0 n <null>
I have the following query, which returns the number of rows per table in a schema.
Can this also be modified to RETURN the number of columns per table too?
CREATE table stats (
table_name VARCHAR2(128),
num_rows NUMBER,
num_cols NUMBER
);
/
DECLARE
val integer;
BEGIN
for i in (SELECT table_name FROM all_tables WHERE owner = 'Schema')
LOOP
EXECUTE IMMEDIATE 'SELECT count(*) from ' || i.table_name INTO val;
INSERT INTO stats VALUES (i.table_name,val);
END LOOP;
END;
/
You can use the ALL_TAB_COLS dictionary table:
DECLARE
val integer;
BEGIN
FOR i IN (
SELECT table_name,
COUNT(*) AS num_cols
FROM all_tab_cols
WHERE owner = 'Schema'
GROUP BY table_name
)
LOOP
EXECUTE IMMEDIATE 'SELECT count(*) from ' || i.table_name INTO val;
INSERT INTO stats VALUES (i.table_name,val, i.num_cols);
END LOOP;
END;
/
db<>fiddle here
One of my stored procedure recently took around 6 hours which usually takes about 3 hours to complete.
On checking, I found that the cursor is taking the time to execute.
Both the tables are present in my local DB instance.
I need to know what could be the possible reason for this and how the procedure can be fine tuned.
My stored procedure:
create or replace PROCEDURE VMS_DETAILS_D_1 IS
LOG_D1 VARCHAR2(20);
BEGIN
/* IDENTIFY PARTITION */
SELECT partition_name into LOG_D1 FROM all_tab_partitions a WHERE table_name = 'LOG' AND TABLE_OWNER='OWNER1' and partition_position IN
(SELECT MAX (partition_position-1) FROM all_tab_partitions b WHERE table_name = a.table_name AND a.table_owner = b.table_owner);
execute immediate 'DROP TABLE TAB1 PURGE';
COMMIT;
EXECUTE IMMEDIATE 'create table TAB1 Nologging as
select /*+ Parallel(20) */ TRANSACTIONID,TIME_STAMP from OWNER1.log partition('||LOG_D1||')
where ( MESSAGE = ''WalletUpdate| Request for Estel Update is Processed'' or MESSAGE = ''Voucher Core request processed'')';
EXECUTE IMMEDIATE 'CREATE INDEX IDX_TAB1 on TAB1(TRANSACTIONID)';
DBMS_STATS.GATHER_TABLE_STATS (ownname => 'OWNER2' , tabname => 'TAB1',cascade => true, estimate_percent => 10,method_opt=>'for all indexed columns size 1', granularity => 'ALL', degree => 1);
DECLARE
CURSOR resp_cur
IS
select TRANSACTIONID,to_char(max(TIME_STAMP),'DD-MM-YYYY HH24:MI:SS') TIME_STAMP from TAB1
where TRANSACTIONID in (select ORDERREFNUM from TAB2
where ORDERREFNUM like 'BV%') group by TRANSACTIONID;
BEGIN
FOR l IN resp_cur
LOOP
update TAB2
set TCTIME=l.TIME_STAMP
where ORDERREFNUM=l.TRANSACTIONID;
COMMIT;
END LOOP;
END;
end;
First off, DDL has an implicit commit, so you don't need a commit after your drop table.
Secondly, why are you dropping the table and recreating it instead of just truncating the table and inserting into it?
Thirdly, why loop around a cursor to do an update, when you can do it in a single update statement?
If you absolutely must store the data in a separate table, I would rewrite your procedure like so:
CREATE OR REPLACE PROCEDURE vms_details_d_1 IS
log_d1 VARCHAR2(20);
BEGIN
/* IDENTIFY PARTITION */
SELECT partition_name
INTO log_d1
FROM all_tab_partitions a
WHERE table_name = 'LOG'
AND table_owner = 'OWNER1'
AND partition_position IN (SELECT MAX(partition_position - 1)
FROM all_tab_partitions b
WHERE table_name = a.table_name
AND a.table_owner = b.table_owner);
EXECUTE IMMEDIATE 'TRUNCATE TABLE TAB1 reuse storage';
EXECUTE IMMEDIATE 'insert into TAB1 (transactionid, time_stamp)'||CHR(10)||
'select /*+ Parallel(20) */ TRANSACTIONID,TIME_STAMP from OWNER1.log partition(' || log_d1 || ')'||CHR(10)||
'where MESSAGE in (''WalletUpdate| Request for Estel Update is Processed'', ''Voucher Core request processed'')';
EXECUTE IMMEDIATE 'CREATE INDEX IDX_TAB1 on TAB1(TRANSACTIONID)';
dbms_stats.gather_table_stats(ownname => 'OWNER2',
tabname => 'TAB1',
cascade => TRUE,
estimate_percent => 10,
method_opt => 'for all indexed columns size 1',
granularity => 'ALL',
degree => 1);
MERGE INTO tab2 tgt
USING (SELECT transactionid,
max(time_stamp) ts
FROM tab1
GROUP BY transactionid) src
ON (tgt.transactionid = src.transactionid)
WHEN MATCHED THEN
UPDATE SET tgt.tctime = to_char(src.ts, 'dd-mm-yyyy hh24:mi:ss'); -- is tab2.tctime really a string? If it's a date, remove the to_char
COMMIT;
END vms_details_d_1;
/
If you're only copying the data to make it easier to do the update, you don't need to - instead, you can do it all in a single DML statement, like so:
CREATE OR REPLACE PROCEDURE vms_details_d_1 IS
log_d1 VARCHAR2(20);
BEGIN
/* IDENTIFY PARTITION */
SELECT partition_name
INTO log_d1
FROM all_tab_partitions a
WHERE table_name = 'LOG'
AND table_owner = 'OWNER1'
AND partition_position IN (SELECT MAX(partition_position - 1)
FROM all_tab_partitions b
WHERE table_name = a.table_name
AND a.table_owner = b.table_owner);
EXECUTE IMMEDIATE 'MERGE INTO tab2 tgt'||CHR(10)||
' USING (SELECT transactionid,'||CHR(10)||
' MAX(time_stamp) ts'||CHR(10)||
' FROM owner1.log partition(' || log_d1 || ')'||CHR(10)||
' GROUP BY transactionid) src'||CHR(10)||
' ON (tgt.transactionid = src.transactionid)'||CHR(10)||
'WHEN MATCHED THEN'||CHR(10)||
' UPDATE SET tgt.tctime = to_char(src.ts, ''dd-mm-yyyy hh24:mi:ss'')'; -- is tab2.tctime really a string? If it's a date, remove the to_char
COMMIT;
END vms_details_d_1;
/
If you know the predicate(s) which define the partition you're after, you can use those in your query, thus removing the need to find the partition name and therefore needing dynamic SQL.
Okay you procedure needs lots of enhancment:
In the below query you can use user_tab_partitions instead of all_tab_partitions.
SELECT partition_name
into LOG_D1
FROM all_tab_partitions a
WHERE table_name = 'LOG'
AND TABLE_OWNER = 'OWNER1'
and partition_position IN
(SELECT MAX(partition_position - 1)
FROM all_tab_partitions b
WHERE table_name = a.table_name
AND a.table_owner = b.table_owner);
You have to include a checking on the table tab1 incase if it doesnt exists and no need to commit here, its not a DML statement.
execute immediate 'DROP TABLE TAB1 PURGE';
COMMIT;
No need to update the statistics in the procedure, especially its a newly table created and the index is already created, and its only one index.
The above might slightly improve the performance but you have to check that there is an index on table log for column message ( but as i said its wrong modeling) also check query plan on tab2 if it needs a index.
This is the wrong approach, what you're doing is update TAB2 the number of times the records in cursor resp_cur, I would switch to a merge.
Oracle Database 11g Release 11.2.0.1.0 - 64bit Production.
I want to gather histogram statistics on a table with a BINARY_DOUBLE column.
However oracle seems to be unable to do so correctly.
After I changed the column to the NUMBER data type it worked as expected.
To reproduce the issue I created the following script:
create table a(
val binary_double not null
);
create table b(
val number not null
);
BEGIN
FOR i IN 0..49
LOOP
INSERT INTO a(val) (select dbms_random.value(10000, 50000) num from dual);
INSERT INTO a(val) (select dbms_random.value(100000, 300000) num from dual);
INSERT INTO b(val) (select dbms_random.value(10000, 50000) num from dual);
INSERT INTO b(val) (select dbms_random.value(100000, 300000) num from dual);
END LOOP;
END;
EXECUTE DBMS_STATS.GATHER_TABLE_STATS(OWNNAME => 'SYS', TABNAME => 'a', METHOD_OPT => 'FOR COLUMNS SIZE 4 val', estimate_percent => 100);
EXECUTE DBMS_STATS.GATHER_TABLE_STATS(OWNNAME => 'SYS', TABNAME => 'b', METHOD_OPT => 'FOR COLUMNS SIZE 4 val', estimate_percent => 100);
Checking DBA_HISTOGRAMS for table a
SELECT ENDPOINT_NUMBER, ENDPOINT_VALUE
FROM DBA_HISTOGRAMS
WHERE OWNER = 'SYS' AND TABLE_NAME = 'A';
Result
ENDPOINT_NUMBER | ENDPOINT_VALUE
0 0
1 0
2 0
3 0
4 0
Checking DBA_HISTOGRAMS for table b
SELECT ENDPOINT_NUMBER, ENDPOINT_VALUE
FROM DBA_HISTOGRAMS
WHERE OWNER = 'SYS' AND TABLE_NAME = 'B';
Result
ENDPOINT_NUMBER | ENDPOINT_VALUE
0 10976.7485097398
1 31489.048616045
2 49201.0216533027
3 200759.370719409
4 296991.623064998
Has someone an idea why this is happening and what to do about it? (preferably without changing the data type of the column)