I am looking towards unifying a number of columns across my data warehouse
My source system is Oracle based, and I would like to define my columns in my warehouse in such a way that a given column name can only have one data type and length
I am looking towards getting info on actual length used in the columns in my source system, and can identity the column name, datatype and length through this script
SELECT DISTINCT
column_name,
data_type,
data_length
FROM
all_tab_columns
ORDER BY
column_name
This does not however result in actual MAX(LENGTH()) of the individual columns
Is this possible to obtain, perhaps through a loop function?
Something like this, perhaps? Just an idea, modify it as you wish.
SQL> set serveroutput on;
SQL> declare
2 l_str varchar2(500);
3 l_maxlen number;
4 begin
5 for cur_r in (select table_name, column_name, data_type
6 from user_tab_columns
7 where table_name in ('EMP', 'DEPT')
8 order by table_name, column_name
9 )
10 loop
11 l_str := 'select max(length(' || cur_r.column_name || '))' ||
12 ' from ' || cur_r.table_name;
13 execute immediate l_str into l_maxlen;
14
15 dbms_output.put_line(cur_r.table_name ||'.'|| cur_r.column_name ||
16 ' - ' || cur_r.data_type ||': '|| l_maxlen);
17 end loop;
18 end;
19 /
DEPT.DEPTNO - NUMBER: 2
DEPT.DNAME - VARCHAR2: 10
DEPT.LOC - VARCHAR2: 8
EMP.COMM - NUMBER: 4
EMP.DEPTNO - NUMBER: 2
EMP.EMPNO - NUMBER: 4
EMP.ENAME - VARCHAR2: 6
EMP.HIREDATE - DATE: 8
EMP.JOB - VARCHAR2: 9
EMP.MGR - NUMBER: 4
EMP.SAL - NUMBER: 4
PL/SQL procedure successfully completed.
SQL>
Related
I'm expecting to write a query which takes second column if first column not found in table in oracle DB. In my case 'name' column is not present in table 'employees'
NOTE : I'm using reference cursor
I tried below,
query1:='select id or name,age from employees';
when I execute above statement, getting error
ORA-00904 "name": invalid identifier
ORA-06512 : at "employees", line 21
Explicitly, I don't think you can (as you saw).
Though, you can select * from employees and it'll work:
SQL> declare
2 l_rc sys_refcursor;
3 l_row dept%rowtype;
4 begin
5 open l_rc for select * from dept;
6 fetch l_rc into l_row;
7 end;
8 /
PL/SQL procedure successfully completed.
SQL>
Alternatively, did you consider creating a select statement dynamically, by querying user_tab_columns? Something like this:
SQL> declare
2 l_str varchar2(500);
3 l_rc sys_refcursor;
4 begin
5 for cur_r in (select column_name from user_tab_columns
6 where table_name = 'DEPT'
7 )
8 loop
9 l_str := l_str || ', '|| cur_r.column_name;
10 end loop;
11
12 l_str := 'select ' || ltrim(l_str, ', ') || ' from dept';
13
14 dbms_output.put_line(l_str);
15
16 open l_rc for l_str;
17 end;
18 /
select DEPTNO, DNAME, LOC from dept --> this is the SELECT statement
PL/SQL procedure successfully completed.
SQL>
As my permissions are limited, the following can not be created as procedure.
I need help on developing a Dynamic SQL that checks a table for duplicate unique IDs.
Also, is it possible to have more than one table to be checked for duplicates with the same query?
declare
table_name is table:= table_1
unique_id varchar2(100):= unique_1
begin
select unique_id,
count(unique_id) as count_unique
from table_name
having count(unique_id)>1
group by unique_id
end;
/
If you can't create a stored procedure (or a function), you're doomed to an anonymous PL/SQL block. Here's one that works in SQL*Plus (probably in SQL Developer as well). Read comments within code.
For Scott's EMP table, number of jobs is
SQL> select job, count(*) from emp group by job;
JOB COUNT(*)
--------- ----------
CLERK 4
SALESMAN 4
PRESIDENT 1
MANAGER 3
ANALYST 2
SQL>
You'd then
SQL> declare
2 l_table varchar2(30) := '&PAR_TABLE_NAME';
3 l_column varchar2(30) := '&PAR_COLUMN_NAME';
4 l_str varchar2(500);
5 l_rc sys_refcursor;
6 --
7 l_ret_column varchar2(30);
8 l_ret_cnt number;
9 begin
10 -- compose a SELECT statement
11 l_str := 'select ' || l_column || ', count(*) cnt ' ||
12 ' from ' || l_table ||
13 ' group by ' || l_column ||
14 ' having count(*) > 1';
15
16 -- use L_STR as a "source" for the L_RC (ref)cursor
17 open l_rc for l_str;
18
19 -- loop, fetch data, display what you've found
20 loop
21 fetch l_rc into l_ret_column, l_ret_cnt;
22 exit when l_rc%notfound;
23
24 dbms_output.put_line(l_table ||'.'|| l_column ||' = ' ||
25 l_ret_column ||', ' || l_ret_cnt || ' row(s)');
26 end loop;
27
28 close l_rc;
29 end;
30 /
Enter value for par_table_name: emp
Enter value for par_column_name: job
emp.job = CLERK, 4 row(s)
emp.job = SALESMAN, 4 row(s)
emp.job = MANAGER, 3 row(s)
emp.job = ANALYST, 2 row(s)
PL/SQL procedure successfully completed.
SQL>
I am new to ORACLE PL/SQL world. I am trying to figure out a way to calculate something as below.
Let's say you have a MASTER_TABLE as below :
SELECT * FROM MASTER_TABLE;
+----------+----------+------------------+-----------------------+
| SCHEMA | TABLE_NM | REQUIRED_COLUMNS | TABLE_FILTER |
+----------+----------+------------------+-----------------------+
| USER_SCH | A | A1,A2,A3 | EXAM_DT > SYSDATE - 1 |
| USER_SCH | B | B1,B2 | TRUNC(SYSDATE) |
+----------+----------+------------------+-----------------------+
I would like to generate SELECT query from above table such as below:
SELECT 'SELECT SCHEMA || '.' || TABLE_NM ||' WHERE '|| TABLE_FILTER FROM MASTER_TABLE;
Obviously, the result of above query would generate multiple select statements.
Now, I want to execute all such SELECT statements and send out the resultset via e-mail.
The tricky part is, the columns mentioned in the MASTER_TABLE varies (i.e. For table 'A' there can be 3 REQUIRED_COLUMNS to be selected, For table 'B' there can be 2 REQUIRED_COLUMNS to be selected - As shown in the MASTER_TABLE)
I have the e-mail utility ready which basically takes an argument as your_message and sends it out via e-mail.
Here is what I have tried :
Created CURSOR to generate such select statements.
Tried inserting the resultset (LIST OF SELECT QUERIES) to another temp table by concatenating the columns.
(i.e.
SELECT
'SELECT '
|| replace(required_columns, ',', '||'',''||')
|| ' AS MSG_BDY'
|| ' FROM '
|| schema
|| '.'
|| table_nm
|| ' WHERE '
|| table_filter
as my_select_stmt
FROM
master_table;
I am stuck after this.
Can you please help me out ? or is there any approach to achieve this ?.
Note : Tables mentioned in MASTER_TABLE can have 1 or more rows.
I don't have your tables so I used Scott's.
Master table:
SQL> select * From master_table;
SCHEM TABL REQUIRED_COLUMN TABLE_FILTER
----- ---- --------------- ----------------------
scott emp ename, job, sal hiredate < sysdate - 1
scott dept dname, loc deptno = 20
SQL>
Procedure which simulates your mailing procedure; I'll just display those values.
SQL> CREATE OR REPLACE PROCEDURE p_mail (par_result IN SYS.odcivarchar2list)
2 AS
3 BEGIN
4 FOR i IN par_result.FIRST .. par_result.LAST
5 LOOP
6 DBMS_OUTPUT.put_line (par_result (i));
7 END LOOP;
8 END;
9 /
Procedure created.
SQL>
Procedure you actually need; as you composed the select statement(s), now you only have to run them. In order to do so, use dynamic SQL (e.g. execute immediate):
SQL> SET SERVEROUTPUT ON
SQL>
SQL> DECLARE
2 retval SYS.odcivarchar2list;
3 BEGIN
4 FOR cur_r
5 IN (SELECT 'SELECT '
6 || REPLACE (required_columns, ',', '||'',''||')
7 || ' AS MSG_BDY'
8 || ' FROM '
9 || schema
10 || '.'
11 || table_nm
12 || ' WHERE '
13 || table_filter
14 AS my_select_stmt
15 FROM master_table)
16 LOOP
17 EXECUTE IMMEDIATE cur_r.my_select_stmt BULK COLLECT INTO retval;
18
19 -- you'd call your mailing procedure here
20 p_mail (retval);
21 END LOOP;
22 END;
23 /
SMITH,CLERK,920
ALLEN,SALESMAN,1600
WARD,SALESMAN,1250
JONES,MANAGER,2975
MARTIN,SALESMAN,1250
BLAKE,MANAGER,2850
CLARK,MANAGER,2450
SCOTT,ANALYST,3000
KING,PRESIDENT,10000
TURNER,SALESMAN,1500
ADAMS,CLERK,1300
JAMES,CLERK,950
FORD,ANALYST,3000
MILLER,CLERK,1300
RESEARCH,DALLAS
PL/SQL procedure successfully completed.
SQL>
[EDIT: what if you wanted to display 'null' for missing values?]
Well, that's a new moment - probably not very simple. See if this helps.
In order to help myself, I modified master_table and added ID column to uniquely identify every row. It'll be used to split required columns' list to rows, apply NVL to them, apply CAST to columns (because NVL complains if datatypes don't match), aggregate them back using listagg. As this is quite a lot to do, I'm going to create a view and use it instead of the table itself.
SQL> CREATE OR REPLACE VIEW v_master_table
2 AS
3 SELECT id,
4 schema,
5 table_nm,
6 LISTAGG ('NVL(cast(' || col || ' as varchar2(20)), ''null'')', '||'',''||')
7 WITHIN GROUP (ORDER BY lvl)
8 required_columns,
9 table_filter
10 FROM (SELECT id,
11 schema,
12 table_nm,
13 table_filter,
14 COLUMN_VALUE lvl,
15 TRIM (REGEXP_SUBSTR (required_columns,
16 '[^,]+',
17 1,
18 COLUMN_VALUE))
19 col
20 FROM master_table
21 CROSS JOIN
22 TABLE (
23 CAST (
24 MULTISET (
25 SELECT LEVEL
26 FROM DUAL
27 CONNECT BY LEVEL <=
28 REGEXP_COUNT (required_columns,
29 ',')
30 + 1) AS SYS.odcinumberlist)))
31 GROUP BY id,
32 schema,
33 table_nm,
34 table_filter;
View created.
For example, it now looks like this:
SQL> select * from v_master_table where id = 2;
ID SCHEM TABL REQUIRED_COLUMNS TABLE_FILTER
--- ----- ---- ------------------------------------------------------------------------------------- ------------
2 scott dept NVL(cast(dname as varchar2(20)), 'null')||','||NVL(cast(loc as varchar2(20)), 'null') deptno = 20
SQL>
The mailing procedure remains the same, no change.
Anonymous PL/SQL block is slightly changed - I removed REPLACE you previously used as view does it now; also, source is the view, not the table.
SQL> alter session set nls_date_format = 'dd.mm.yyyy';
Session altered.
SQL> DECLARE
2 retval SYS.odcivarchar2list;
3 BEGIN
4 FOR cur_r
5 IN (SELECT 'SELECT '
6 || required_columns
7 || ' AS MSG_BDY'
8 || ' FROM '
9 || schema
10 || '.'
11 || table_nm
12 || ' WHERE '
13 || table_filter
14 AS my_select_stmt
15 FROM v_master_table)
16 LOOP
17 EXECUTE IMMEDIATE cur_r.my_select_stmt BULK COLLECT INTO retval;
18
19 -- you'd call your mailing procedure here
20 p_mail (retval);
21 END LOOP;
22 END;
23 /
CLARK,09.06.1981,null
KING,17.11.1981,null
MILLER,23.01.1982,null
RESEARCH,DALLAS
PL/SQL procedure successfully completed.
SQL>
How to fetch odd columns in Oracle using a query when number of columns and name of columns are not known?
E.g.:
I need to get output in below format
Column1 column3 column5 column7
And so on....
You need to use the dynamic queries in the procedure as follows:
SQL> CREATE OR REPLACE PROCEDURE ODD_COLUMNS (
2 TABLE_NAME_P IN VARCHAR2,
3 DATAA OUT SYS_REFCURSOR
4 ) AS
5 V_SQL VARCHAR2(4000);
6 BEGIN
7 SELECT
8 'SELECT '
9 ||
10 LISTAGG(COLUMN_NAME, ',') WITHIN GROUP(
11 ORDER BY
12 COLUMN_ID
13 )
14 || ' FROM "'
15 || TABLE_NAME_P
16 || '"'
17 INTO V_SQL
18 FROM
19 USER_TAB_COLS
20 WHERE
21 TABLE_NAME = TABLE_NAME_P
22 AND MOD(COLUMN_ID, 2) = 1;
23
24 OPEN DATAA FOR V_SQL;
25
26 END ODD_COLUMNS;
27 /
Procedure created.
SQL>
Now, Let's test it:
SQL> variable rc refcursor;
SQL> exec ODD_COLUMNS('EMP',:rc);
PL/SQL procedure successfully completed.
SQL> print rc;
EMP_ID E
---------- -
10 N
20 Y
SQL>
SQL> exec ODD_COLUMNS('MY_TABLE1',:rc);
PL/SQL procedure successfully completed.
SQL> print rc;
ID REQ_QTY
---------- ----------
1001 10
1001 20
1001 30
1002 40
1003 10
1003 20
6 rows selected.
SQL>
Cheers!!
This cannot be done simply, but it is possible using the Oracle data dictionary and some dynamic SQL.
To find out the odd-numbered columns you need to look at the ALL_TAB_COLUMNS view. Column COLUMN_ID sequences the columns 1,2,3. So this will find all the odd-numbered columns in the SCOTT.EMP table:
select column_name, column_id
from all_tab_columns
where owner = 'SCOTT'
and table_name = 'EMP'
and mod(column_id,2) = 1
order by column_id;
This will return something like:
COLUMN_NAME COLUMN_ID
----------- ---------
EMPNO 1
JOB 3
HIREDATE 5
COMM 7
We can use the LISTAGG function to make that into a comma-separated list:
select listagg(column_name,',') within group (order by column_id) as result
from user_tab_columns
where table_name = 'EMP'
and mod(column_id,2) = 1;
RESULT
------
EMPNO,JOB,HIREDATE,COMM
Now we can add to that SQL to generate the select statement you want:
select 'select ' || listagg(column_name,',') within group (order by column_id) || ' from ' || table_name as sql
from user_tab_columns
where table_name = 'EMP'
and mod(column_id,2) = 1
group by table_name;
SQL
---
select EMPNO,JOB,HIREDATE,COMM from EMP
(Note I had to add a group by clause because table_name is not being aggregated by LISTAGG).
You could use that SQL within some PL/SQL code to populate a variable v_sql, then use the DBMS_SQL package to run it. But that is a complex topic in itself and I won't go into it here.
Example scenario:
5 tables are there and one common field among them is com_field (DATE data type). Now, I need to find the maximum of com_field in each of the five tables. Can someone give the logic?
I know UNION could be used but I need the flexibility not to miss any new table added to the OWNER.
The result I am expecting is like the below.
Table Max(com_field)
Tbl1 10/21/2019
Tbl2 10/18/2019
Tbl3 10/28/2019
Tbl4 09/30/2019
Tbl5 09/09/2019
Run this query:
SELECT
'SELECT '''||OWNER||'.'||TABLE_NAME ||''' AS TABLE_NAME , '||'MAX(COM_FIELD)AS COM_FIELD FROM ' ||OWNER||'.'||TABLE_NAME ||' UNION ALL'
FROM ALL_TAB_COLUMNS
WHERE COLUMN_NAME ='COM_FIELD'
then copy the outpu and delete last union all keyword. Then run the sql statement.
You can order it and see the max value
One option is to use dynamic SQL in a function that returns refcursor. Here's an example.
First, test case:
SQL> create table taba (com_field date);
Table created.
SQL> create table tabb (com_field date);
Table created.
SQL> create table tabc (com_field date);
Table created.
SQL> insert all
2 into taba values (sysdate)
3 into taba values (sysdate - 2)
4 into taba values (sysdate - 3)
5 into tabb values (sysdate + 2)
6 into tabc values (sysdate + 4)
7 into tabc values (sysdate + 5)
8 select * From dual;
6 rows created.
SQL>
Function:
SQL> create or replace function f_maxcom
2 return sys_refcursor
3 is
4 l_str varchar2(1000);
5 rc sys_refcursor;
6 begin
7 for cur_r in (select table_name
8 from user_tab_columns
9 where column_name = 'COM_FIELD'
10 )
11 loop
12 l_str := l_str ||
13 'select ' || chr(39) || cur_r.table_name || chr(39) || ', ' ||
14 'max(com_field) from ' || cur_r.table_name || ' union all ';
15 end loop;
16
17 l_str := rtrim(l_str, ' union all');
18
19 open rc for l_str;
20 return rc;
21 end;
22 /
Function created.
SQL>
Let's try it:
SQL> select f_maxcom from dual;
F_MAXCOM
--------------------
CURSOR STATEMENT : 1
CURSOR STATEMENT : 1
'TAB MAX(COM_FI
---- ----------
TABA 29.10.2019
TABB 31.10.2019
TABC 03.11.2019
SQL>
Add another table to see what happens; function will stay as is:
SQL> create table littlefoot (id number, com_field date);
Table created.
SQL> insert into littlefoot values (100, sysdate);
1 row created.
SQL> select f_maxcom from dual;
F_MAXCOM
--------------------
CURSOR STATEMENT : 1
CURSOR STATEMENT : 1
'LITTLEFOO MAX(COM_FI
---------- ----------
LITTLEFOOT 29.10.2019
TABA 29.10.2019
TABB 31.10.2019
TABC 03.11.2019
SQL>
Seems to be OK, eh?