Hare is the scenario:
Main DB Server: SQL Server 2008 R2 with a Linked Server to Oracle 11g.
I have a Stored Procedure that make a query like:
Select t1.a, t1.b, t2.c, t3.d
From LocalTable a inner join LinkedServerName..Schema.Tableb b on a.aNumber= b.id
inner join LinkedServerName..Schema.Tablec c on b.value = c.id
inner join LinkedServerName..Schema.Tabled d on a.someOtherNumber = d.Id
Where a.WhereValue1 = #Parameter1
and b.WhereValue2 = #Parameter2
That turns to be painfully slow. I cannot figure out how use OpenQuery to improve the query since the Where clauses uses parameters (if that is even possible to use).
Is there way to improve the data Retrieval? I'm retrieving millions of records from the Oracle DB.
Thanks you very much.
What I suggest you do is at least create a view on the Oracle side that joins tables b,c,d and join to that view. How many records are in LocalTable? If there are very few (under 10,000 or so) then you are better joining the entire thing on the Oracle side.
What is your overall objective? Are you building a report or trying to identifying differing records so you can merge the data? Are you able to make changes on the oracle side?
Related
I have 3 tables:
table1: id, person_code
table2: id, address, person_code_foreing(same with that one from table 1), admission_date_1
table3: id, id_table2, admission_date_2, something
(the tables are fictive)
I'm trying to make a view who takes infos from this 3 tables using left join, i'm doing like this because in the first table i have some record who don't have the person_code in the others tables but I want also this info to be returned by the view:
CREATE OR REPLACE VIEW schema.my_view
SELECT t1.name, t2.adress, t3.something
from schema.table1#ambient1 t1
left join schema.table2#ambient1 t2
on t1.person_code = t2.person_code_foreing
left join schema.table3#ambient1 t3
on t3.id_table2 = t2.id
and t1.admission_date_1=t2.admission_date_2;
This view needs to be created in another ambient (ambient2).
I tried using a subquery, there I need also a left join to use, and this thing is very confusing because I don't get it, the subquery and the left join are the big no-no?! Or just de left-join?!
Has this happened to anyone?
How did you risolved it?
Thanks a lot.
ORA-2019 indicates that your database link (#ambient1) does not exist, or is not visible to the current user. You can confirm by checking the ALL_DB_LINKS view, which should list all links to which the user has access:
select owner, db_link from all_db_links;
Also keep in mind that Oracle will perform the joins in the database making the call, not the remote database, so you will almost certainly have to pull the entire contents of all three tables over the network to be written into TEMP for the join and then thrown away, every time you run a query. You will also lose the benefit of any indexes on the data and most likely wind up with full table scans on the temp tables within your local database.
I don't know if this is an option for you, but from a performance perspective and given that it isn't joining with anything in the local database, it would make much more sense to create the view in the remote database and just query that through the database link. That way all of the joins are performed efficiently where the data lives, only the result set is pushed over the network, and your client database SQL becomes much simpler.
I managed to make it work, but apparently ambient2 doesn't like my "left-join", and i used only a subquery and the operator (+), this is how it worked:
CREATE OR REPLACE VIEW schema.my_view
SELECT t1.name, all.adress, all.something
from schema.table1#ambient1 t1,(select * from
schema.table3#ambient1 t3, schema.table2#ambient1 t2
where t3.id_table2 = t2.id(+)
and (t1.admission_date_1=t2.admission_date_2 or t1.admission_date is null))
all
where t1.person_code = t2.person_code_foreing(+);
I tried to test if a query in ambient2 using a right-join works (with 2 tables created there) and it does. I thought there is a problem with that ambient..
For me, there is no sense why in my case this kind of join retrieves that error.
The versions are different?! I don't know, and I don't find any official documentation about that.
Maybe some of you guys have any clue..
There is a mistery for me :))
Thanks.
I'm using entity framework core for mysql, and i've been running a complex linq query which i'm trying to optimise.
I turned on logging in the mysql server to view the resulting queries from the linq queries.
Oddly, none of it made sense as my complex query that joined 5 tables and performed multiple group bys, where, and order by clause was registered in the logs as 5 separate select all columns from table statements.
So, I tried a simple group by statement for one table. The resulting sql log produced "Select all_columns from table_name order by groupbyid".
Can anyone explain what happened here?
Thanks in advance.
More info as requested:
Sql query:
var queryCommand = (from p in _context.TableExtract group p by p.tableExtractPersonId);
queryCommand.ToList();
Resulting mysql log after:
SELECT .... [very long list of column names]
FROM TableExtract AS p
ORDER BY p.tableExtractPersonId
I've tried two different entity framework libraries: MySql.Data.EntityFrameworkCore(v8.0.17) and Pomelo.EntityFrameworkCore.MySql (v2.2.20) with the same results. I've tried .net core 3.0 and also received the same results. I'm going to try .net standard next.
Ok. I found it:
var queryCommand = (from p in _context.TableExtract group p by p.tableExtractPersonIdinto g select g.Key)
Forces linq to evaluate as a SQL group by. Otherwise apparently it does it's own thing with the group by.
There is a query having multiple inner joins. It involves two views, of which one view is based on four tables, and total there are four tables(including two views).
The same query with the same amount of data in the source tables runs in both, Oracle and DB2. In DB2, surprisingly, it takes 2 minutes to load 3 million records. While in Oracle, it is taking two hours. Same indexes are on all source tables in both the environments. Is the behavior of views (when used in joins) different in both environments (Oracle vs DB2)?
a dummy query I am sharing :-
INSERT INTO TABLE_A
SELECT
adf.column1,
adf.column2,
dd.column3,
SUM(otl.column4) column4,
SUM(otl.column5) column5,
(Case when SUM(otl.column5) = 0 then 0
else round(CAST(SUM(otl.column4) AS DECIMAL(19,2)) /abs(CAST(SUM(otl.column4) AS DECIMAL(18,2))),4)
end) taxl_unrlz_cgl_pct
FROM
view_a adf
INNER JOIN table_b hr on hr.hh_ref_id = adf.hh_ref_id
AND hr.col_typ_cd = 'FIRM'
AND hr.col_end_dt = TO_DATE('1/1/2900','MM/DD/YYYY')
INNER JOIN dw.table_c ar on ar.colb_id = adf.colb_id
AND ar.col_cd = '#'
AND ar.col_num BETWEEN 10000000 AND 89999999
AND ar.col_dt IS NULL
INNER JOIN table_d dd on dd.col_id = adf.col_id
INNER JOIN view2 otl ON otl.cola_id = ar.cola_id
GROUP BY adf.column1, adf.column2, dd.column3;
Technically, both DB2 and Oracle will try to rewrite the query in most efficient way possible using the base query that you have coded. But one of the common (but not frequent) issues that I have seen when using multi-table view is DBMS not being able to rewrite the query using underlying tables. So depending on complexity of the view itself and sometime the additional joins, DBMS may not be able to rewrite the query to use the underlying tables properly and hence resulting in not being able to use the indexes on the underlying tables used in the view. When this happens, the view itself acts like a materialized table (work table) and query goes for table scan on the materialized table.
There is no consistent pattern on when such issue can happen. So you will need to check on a case by case basis.
Since you are mentioning about 2 hrs vs 2 minutes, in most probability that might be the case. So you will need to check the access path on both Oracle and DB2. But you will also need to make sure that stats are updated and access path is based on latest stats on DBMS. Else it won't be apples to apples compare.
I understand that the performance of our queries is improved when we use EXISTS and NOT EXISTS in the place of IN and NOT IN, however, is performance improved further when we replace NOT IN with an OUTER JOIN as opposed to NOT EXISTS?
For example, the following query selects all models from a PRODUCT table that are not in another table called PC. For the record, no model values in the PRODUCT or PC tables are null:
select model
from product
where not exists(
select *
from pc
where product.model = pc.model);
The following OUTER JOIN will display the same results:
select product.model
from product left join pc
on pc.model = product.model
where pc.model is null;
Seeing as these both return the same values, which option should we use to better improve the performance of our queries?
The query plan will tell you. It will depend on the data and tables. In the case of OUTER JOIN and NOT EXISTS they are the same.
However to your opening sentence, NOT IN and NOT EXISTS are not the same if NULL is accepted on model. In this case you say model cannot be null so you might find they all have the same plan anyway. However when making this assumption, the database must be told there cannot be nulls (using NOT NULL) as opposed to there simply not being any. If you don't it will make different plans for each query which may result in different performance depending on your actual data. This is generally true and particularly true for ORACLE which does not index NULLs.
Check out EXPLAIN PLAN
I have been using SQL Server 2008 for a short time now and have never used Oracle before. I am able to access an Oracle table through SQL Server with the syntax
select * from [OracleDB1]..[OracleDB1].[Zips]
(where OracleDB1 is the oracle database and Zips is the table I require)
Is it possible to join a SQL Server table with this one in a Table-valued Function? Just using a normal join as I would with SQL Server tables gives an Invalid object name error on the Oracle table.
Can this be done directly (or at all) or is it possible to do this some other way such as table variables?
example query:
select * from dbo.Table1 t INNER JOIN [OracleDB1]..[OracleDB1].[Zips] z where t.zip = z.zip
I was performing the join wrong since I missed the ON clause. I was able to get it to work by declaring a temptable and joining on that.
declare #tempTable table{
ZIP nvarchar(5),
COUNTY nvarchar(10)
}
insert #tempTable select ZIP, COUNTY, from [OracleDB1]..[OracleDB1].[ZIPS]
select * from dbo.Table1 t INNER JOIN #tempTable z on t.ZIP = v.ZIP where t.AdmissionOn >= '08-08-2011' AND t.AdmissionOn <= ''09-08-2011'
This also worked in line as I had in the original question once I added the ON clause but the table variable suits my needs better since it only has to access the Oracle table once and not each comparison.