Can I use Oracle sys tables to trace a path between two table, all the possibilities to go from X table to Y table.
The problem is:
I work on an enormous database, where it's really difficult to know rapidly, which tables are vital to make a join between two tables.
Can I do this?
First Need:
The problem with SQL Developer Data Modeler and the other tools, is the fact to have to select tables to rev_eng (So I should already know the tables to select) but for me, this is the major problem. In my case I have 800 tables and I can't select them all to trace the path. My desire is to submit as arguments two tables and then generate all the possible paths.
Second Need :
I have already try to query sys.all_constraints and the max I've done, is to detect the tables directly connected to a table X.
The query:
SELECT C1.TABLE_NAME,C2.TABLE_NAME
FROM ALL_CONSTRAINTS C1, ALL_CONSTRAINTS C2
WHERE C2.CONSTRAINT_NAME = C1.R_CONSTRAINT_NAME
AND UPPER(C1.OWNER) LIKE '**MY_SCHEMA**'
AND C1.CONSTRAINT_TYPE='R'
AND UPPER(C1.TABLE_NAME) LIKE '**X**'
ORDER BY C1.TABLE_NAME
So if somebody can help me to conceive at least the query to have this result:
Table1 | Table2 | JoinCollumnofTable1 | JoinCollumnofTable2
To have that, I surmise the other table to join to ALL_CONSTRAINTS is ALL_CON_COLUMNS
But the problem I've found is the composite primary_keys.
This is why Nature gave us data models: to assist in tasks like this.
If you don't have a data model then you can reverse engineer one from the data dictionary. See my answer to a question on reverse engineering.
Reverse engineering can only identify relationships which have been defined by foreign keys. This shouldn't need stating but let's say it anyway: if your database hasn't got constraints you have no chance of deriving a data model automatically.
"I have 800 tables and I can't select them all to trace the path. "
Hmmm, I suppose recommending you reverse engineer a data model is a bit like the punchline to the old joke about how to get to Cork: "Well I wouldn't start from here". The whole point about having a data model upfront is that we have it when when we really need it.
If primary and foreign key relationships are established in the database, you can use a tool like Oracle Developer with Data Modeler to reverse engineer the model and give a graphical representation of what the relationships are.
Tools like this read the Oracle dictionary to determine the relationships between tables. You can do this yourself by querying views such as sys.all_constraints.
I cobbled the following query together using Tim Hall's Generic Function Using a Ref Cursor, since I only have 10g here (you can use 11g's LISTAGG function if you've got 11g). It should get you close.
SELECT ac1.table_name "Table", ac2.table_name "Referencing Table"
, concatenate_list(CURSOR(SELECT acc.column_name
FROM all_cons_columns acc
WHERE acc.constraint_name = ac1.constraint_name
AND acc.owner = 'the_owner'
ORDER BY position)) "PK Columns"
, concatenate_list(CURSOR(SELECT acc.column_name
FROM all_cons_columns acc
WHERE acc.constraint_name = ac2.constraint_name
AND acc.owner = 'the_owner'
ORDER BY position)) "FK Columns"
FROM all_constraints ac1 JOIN all_constraints ac2
ON ac1.constraint_name = ac2.r_constraint_name
WHERE ac1.table_name = 'your_table'
AND ac1.owner = 'the_owner'
AND ac2.owner = 'the_owner'
AND ac1.constraint_type = 'P';
Also try schemaspy - an open source free alternative which uses the foreign keys to generate a relationship model!
Related
I'm looking for a way to generate a script that would generate an SQL query that would select all child tables columns from a parent table.
Let's say you have a table Class (teacher, room, program) and a table Student (firstname, lastname, age, score, email).
Let's say you want to get a select of all students in Class.
Sure you could write the query manually.
But now imagine you have a complex table with dozens of child tables, how do you do this efficiently/programmatically ?
This is something that all programmers would like to have, no ?
I can't believe no one has ever done that.
I understand the answer may depend on the DBMS vendor, I'm personally looking for a solution for Oracle.
Questions that are a bit similar :
Oracle: Easy way to find names and/or number of child record tables
Postgres: select data from parent table and all child tables
And here is an idea to solve this partially : use a tool such as PowerBi or Visual Studio to generate Model from database in ASP.NET MVC. You won't get the SQL query but you will get the data.
You can start with this POC:
select
juc.table_name as parent_table,
/*
uc.table_name as child_table, uc.constraint_name, uc.r_constraint_name,
juc.constraint_type,
uccc.column_name as parent_col_name, uccc.position as parent_col_position,
uccp.column_name as child_col_name, uccp.position as child_col_position,
*/
'SELECT c.* FROM ' || juc.table_name || ' p JOIN ' || uc.table_name || ' c ON '
||
LISTAGG( 'c.' || uccp.column_name || ' = p.' || uccc.column_name, ' AND ' ) WITHIN GROUP(order by uccc.position)
as sql
from user_constraints uc
join user_constraints juc on juc.constraint_name = uc.r_constraint_name
join user_cons_columns uccc on uccc.constraint_name = uc.r_constraint_name
join user_cons_columns uccp on uccp.constraint_name = uc.constraint_name and uccc.position = uccp.position
where uc.constraint_type = 'R'
group by uc.table_name, juc.table_name, uc.constraint_name
;
You can create your own entity relationship model metadata and write PL/SQL that will traverse it and assemble SQL intelligently. I've done this myself to avoid having to hard-code SQL in my front-end apps. But it is highly complex and involves a lot of coding, far more than can be shared in a forum like this. But to give you the general gist, I have the following metadata tables that describe my model:
sql_statements - associates a logical entity with a primary table, and specifies the PK column.
sql_statement_parents - defines the parent entity and the child attribute used to join to the parent's PK.
sql_attribute_dictionary - lists every available attribute for every statement, the source column, its datatype, plus optional derived column expressions.
attribute_dependencies - used for derived column expressions, specifies which attributes are needed by the derived attribute.
Then you write code that takes a sql_statement name and a list of desired attributes and a set of optional filters, and it builds a list of needed source tables/columns using the data relationships in the metadata, and then using the parent-child relationships recursively builds SQL (using nested query blocks) from the child to whatever parent ancestor(s) it needs to obtain the required columns, intelligently aliasing everything and joining in the write way to be performant. It can then pass back the finished SQL as a REF CURSOR which you can then parse, open and fetch from to get results. It works great for me, but it did take weeks of work to perfect, and that's with decades of experience in SQL and PL/SQL. This is no simple task, but it is doable. And of course there are always complex needs that defy the capabilities of our metadata model, and so for those we end up either creating views or pipeline functions, and registering those in our metadata so that generated SQL can invoke them when needed.
But in the end, however you do it, you will not get away from having to describe your data model in detail so that code can walk it.
I have Googled for a long time, but guessing that I struggle to find the right way to ask Google for my questions. I guess my question is pretty easy to solve, just as I need to know how ;)
I just started using PowerBI, and have established a connection to a Oracle database.
My challenge is:
I need to create some kind of "join" towards multiple tables, so I get the data I need.
Example:
Table 1
Table1_Id
Table1_FirstName
Table 2
Table2_Id
Table2_Table1_Id
Table2_LastName
Table 3
Table3_Id
Table3_Table2_Id
Table3_Email
etc....
And the user might have 100 emails, so there could be multiple rows here.
--
How do I do this? I've tried with "merge"/join I think, but maybe in the wrong way, as I get sh*t load of rows in return, more than I should.
I hope I'm clear, if not, please let me know and I will try to be more clear
Brgds
Kristian
I don't use PowerBi, but - from what you said - it looks like you didn't properly join all tables and there's - somewhere - cross join which results in too many rows to be returned.
If you'd write query yourself (I presume PowerBI lets you do that in a GUI), it would be something like this:
select a.first_name,
b.last_name,
c.email
from table1 a join table2 b on b.table1_id = a.table1_id --> this
join table3 c on c.table2_id = b.table2_id --> this
I marked joins you should be having.
I have 3 tables:
table1: id, person_code
table2: id, address, person_code_foreing(same with that one from table 1), admission_date_1
table3: id, id_table2, admission_date_2, something
(the tables are fictive)
I'm trying to make a view who takes infos from this 3 tables using left join, i'm doing like this because in the first table i have some record who don't have the person_code in the others tables but I want also this info to be returned by the view:
CREATE OR REPLACE VIEW schema.my_view
SELECT t1.name, t2.adress, t3.something
from schema.table1#ambient1 t1
left join schema.table2#ambient1 t2
on t1.person_code = t2.person_code_foreing
left join schema.table3#ambient1 t3
on t3.id_table2 = t2.id
and t1.admission_date_1=t2.admission_date_2;
This view needs to be created in another ambient (ambient2).
I tried using a subquery, there I need also a left join to use, and this thing is very confusing because I don't get it, the subquery and the left join are the big no-no?! Or just de left-join?!
Has this happened to anyone?
How did you risolved it?
Thanks a lot.
ORA-2019 indicates that your database link (#ambient1) does not exist, or is not visible to the current user. You can confirm by checking the ALL_DB_LINKS view, which should list all links to which the user has access:
select owner, db_link from all_db_links;
Also keep in mind that Oracle will perform the joins in the database making the call, not the remote database, so you will almost certainly have to pull the entire contents of all three tables over the network to be written into TEMP for the join and then thrown away, every time you run a query. You will also lose the benefit of any indexes on the data and most likely wind up with full table scans on the temp tables within your local database.
I don't know if this is an option for you, but from a performance perspective and given that it isn't joining with anything in the local database, it would make much more sense to create the view in the remote database and just query that through the database link. That way all of the joins are performed efficiently where the data lives, only the result set is pushed over the network, and your client database SQL becomes much simpler.
I managed to make it work, but apparently ambient2 doesn't like my "left-join", and i used only a subquery and the operator (+), this is how it worked:
CREATE OR REPLACE VIEW schema.my_view
SELECT t1.name, all.adress, all.something
from schema.table1#ambient1 t1,(select * from
schema.table3#ambient1 t3, schema.table2#ambient1 t2
where t3.id_table2 = t2.id(+)
and (t1.admission_date_1=t2.admission_date_2 or t1.admission_date is null))
all
where t1.person_code = t2.person_code_foreing(+);
I tried to test if a query in ambient2 using a right-join works (with 2 tables created there) and it does. I thought there is a problem with that ambient..
For me, there is no sense why in my case this kind of join retrieves that error.
The versions are different?! I don't know, and I don't find any official documentation about that.
Maybe some of you guys have any clue..
There is a mistery for me :))
Thanks.
I'm having some difficulty with joining a view to another table. This is on an Oracle RAC system running 11.2
I'll try and give as much detail as possible without going into specific table structures as my company would not like that.
You all know how this works. "Hey, can you write some really ugly software to implement our crazy ideas?"
The idea of what they wanted me to do was to make a view where the end user wouldn't know if they were going after the new table or the old table so one of the tables is a parameter table that will return "ON" or "OFF" and is used in the case statements.
There are some not too difficult but nested case statements in the select clause
I have a view:
create view my_view as
select t1.a as a, t1.b as b, t1.c as c,
sum(case when t2.a = 'xx' then case when t3.a then ... ,
case when t2.a = 'xx' then case when t3.a then ... ,
from table1 t1
join table t2 on (t1.a = t2.a etc...)
full outer join t3 on (t1.a = t3.a etc...)
full outer join t4 on (t1.a = t4.a etc...)
group by t1.a, t1.b, t2.c, and all the ugly case statements...
Now, when I run the query
select * from my_view where a='xxx' and b='yyy' and c='zzz'
the query runs great and the cost is 10.
However, when I join this view with another table everything falls apart.
select * from my_table mt join my_view mv on (mt.a = mv.a and mt.b=mv.b and mt.c=mv.c) where ..."
everything falls apart with a cost though the roof.
What I think is happening is the predicates are not getting pushed to the view. As such, the view is now doing full tables scans and joining everything to everything and then finally removing all the rows.
Every hint, tweak, or anything I've done doesn't appear to help.
When looking at the plan it looks like it has the predicates.
But this happens after everything is joined.
Sorry if this is cryptic but any help would be greatly appreciated.
Since you have the view with a "GROUP BY", predicates could not be pushed to the inner query
Also, you have the group by functions in a case statement, which could also make it worse for the optimizer
Oracle introduces enhancements to Optimizer every version/release/patch. It is hard to say what is supported in the version you're running. However, you can try:
See if removing the case from the GROUP BY function will make any difference
Otherwise, you have to take the GROUP BY and GROUP BY functions from the view to the outer most query
After many keyboard indentations on my forehead I may have tricked Oracle into pushing the predicates. I don't know exactly why this works but simplifying things may have helped.
I changed all my ON clauses to USING clauses and in this way the column names now match the columns from which I'm joining to. On some other predicates that were constants I added in a where clause to the view.
The end result is I can now join this view with another table and the cost is reasonable and the plan shows that the predicates are being pushed.
Thank you to everybody who looked at this problem.
At first, I seen the select statement on Oracle Docs.
I have some question about oracle select behaviour, when my query contain select,join,where.
see this below for information:
My sample table:
[ P_IMAGE_ID ]
IMAGE_ID (PK)
FILE_NAME
FILE_TYPE
...
...
[ P_IMG_TAG ]
IMG_TAG_ID (PK)
IMAGE_ID (FK)
TAG
...
...
My requirement are: get distinct of image when it's tag is "70702".
Method 1: Select -> Join -> Where -> Distinct
SELECT DISTINCT PID.IMAGE_ID
, PID.FILE_NAME
FROM P_IMAGE_ID PID
INNER JOIN P_IMG_TAG PTAG
ON PTAG.IMAGE_ID = PID.IMAGE_ID
WHERE PTAG.TAG = '70702';
I think the query behaviour should be like:
join table -> hint where cause -> distinct select
I use Oracle SQL developer to get the explain plan:
Method 1 cost 76.
Method 2: Select -> Where -> Where -> Distinct
SELECT DISTINCT PID.IMAGE_ID
, PID.FILE_NAME
FROM P_IMAGE_ID PID
WHERE PID.IMAGE_ID IN
(
SELECT PTAG.IMAGE_ID
FROM P_IMG_TAG PTAG
WHERE PTAG.TAG = '70702'
);
I think the second query behaviour should be like:
hint where cause -> hint where cause -> distinct select
I use Oracle SQL developer to get the explain plan too:
Method 2 cost 76 too. Why?
I believe when I try where cause first for reduce the database process and avoid join table that query performance should be better than the table join query, but now when I test it, I am confused, why 2 method cost are equal ?
Or am I misunderstood something ?
List of my question here:
Why 2 method above cost are equal ?
If the result of sub select Tag = '70702' more than thousand or million or more, use join table should be better alright ?
If the result of sub select Tag = '70702' are least, use sub select for reduce data query process is better alright ?
When I use method 1 Select -> Join -> Where -> Distinct mean the database process table joining before hint where cause alright ?
Someone told me when i move hint cause Tag = '70702' into join cause
(ie. INNER JOIN P_IMG_TAG PTAG ON PAT.IMAGE_ID = PID.IMAGE_ID AND PTAG.TAG = '70702' ) it's performance may be better that's alright ?
I read topic subselect vs outer join and subquery or inner join but both are for SQL Server, I don't sure that may be like Oracle database.
The DBMS takes your query and executes something. But it doesn't execute steps that correspond to SQL statement parts in the order they appear in an SQL statement.
Read about "relational query optimization", which could just as well be called "relational query implementation". Eg for Oracle.
Any language processor takes declarations and calls as input and implements the described behaviour in terms of internal data structures and operations, maybe through one or more levels of "intermediate code" running on a "virtual machine", eventually down to physical machines. But even just staying in the input language, SQL queries can be rearranged into other SQL queries that return the same value but perform significantly better under simple and general implementation assumptions. Just as you know that your question's queries always return the same thing for a given database, the DBMS can know. Part of how it knows is that there are many rules for taking a relational algebra expression and generating a different but same-valued expression. Certain rewrite rules apply under certain limited circumstances. There are rules that take into consideration SQL-level relational things like primary keys, unique columns, foreign keys and other constraints. Other rules use implementation-oriented SQL-level things like indexes and statistics. This is the "relational query rewriting" part of relational query optimization.
Even when two different but equivalent queries generate different plans, the cost can be similar because the plans are so similar. Here, both a HASH and SORT index are UNIQUE. (It would be interesting to know what the few top plans were for each of your queries. It is quite likely that those few are the same for both, but that the plan that is more directly derived from the particular input expression is the one that is offered when there's little difference.)
The way to get the DBMS to find good query plans is to write the most natural expression of a query that you can find.