I have a view defined as below
CREATE VIEW DQ_DB.DQM_RESULT_VIEW
AS SELECT
res.ACTIVE_FL AS ACTIVE_FL,
res.VERSION as VERSION,
res.rule_constituents_tx,
nvl(ruletable.rule_desc,'N/A') AS rule_ds,
nvl(res.effective_dt, TO_DATE('31-dec-9999','dd-mon-yyyy')) AS effective_dt,
nvl(res.rule_id,'N/A') AS rule_id,
res.audit_update_ts AS rule_processed_at,
res.load_dt,
res.vendor_group_key,
nvl(res.vendor_entity_key,'N/A') AS vendor_entity_key,
res.vendor_entity_producer_nm,
(SELECT category_value_tx FROM dq_db.category_lookup_view WHERE category_nm = 'RESULT_STATUS_NB' AND category_value_cd = res.result_status_nb ) AS result,
--catlkp.category_value_tx as result,
res.entity_type,
nvl(rgrp.grp_nm,'N/A') AS rule_category,
nvl(ruletable.rule_nm,'N/A') AS rule_nm,
feedsumm.feed_run_nm AS file_nm,
res.application_id AS application,
res.data_source_id AS datasource,
res.entity_nm,
res.rule_entity_effective_dt,
res.result_id,
dim.dimension_nm,
dim.sub_dimension_nm,
ruletable.execution_env AS execution_env,
ruletable.ops_action AS ops_action,
rulefunctiontable.func_nm AS rule_func_nm,
-- nvl2(res.primary_dco_sid,dq_db.get_dco_name(res.primary_dco_sid),null) AS dco_primary,
-- nvl2(res.delegate_dco_sid,dq_db.get_dco_name(res.delegate_dco_sid),null) AS dco_delegate,
res.primary_dco_sid AS dco_primary,
res.delegate_dco_sid AS dco_delegate,
ruletable.data_concept_id AS data_concept_id,
res.latest_result_fl as latest_result_fl,
res.batch_execution_ts as batch_execution_ts
FROM
dq_db.dqm_result res
--LEFT OUTER JOIN dq_db.category_lookup_view catlkp on (catlkp.category_nm = 'RESULT_STATUS_NB' AND catlkp.category_value_cd = res.result_status_nb)
LEFT OUTER JOIN dq_db.feed_run_summary feedsumm ON res.vendor_group_key = feedsumm.batch_id
LEFT OUTER JOIN dq_db.dqm_rule ruletable ON res.rule_id = ruletable.rule_id
LEFT OUTER JOIN dq_db.dqm_rule_grp rgrp ON ruletable.rule_grp_id = rgrp.rule_grp_id
LEFT OUTER JOIN dq_db.dqm_rule_function rulefunctiontable ON ruletable.func_id = rulefunctiontable.func_id
LEFT OUTER JOIN dq_db.dq_dimension_view dim ON dim.dimension_id = ruletable.dimension_id
result column of view is a lookup value generated from subquery condition which translates code as
below.
0|PASS 1|ALERT 2|ERROR
My web application adds few predicates to this view which are pushed to base tables. But one particular predicate on view shown below is not pushed due to inline nature of the predicate.
select * from dqm_result_view where result IN ('ALERT','ERROR')
Right now this query applies filter after view JOINs are executed as there is no way to push predicate to DQM_RESULT table.
What is need is.. if we get that result predicate then apply code 0,1,2 instead of applying result predicate at end so that data is filtered ahead of time for JOINs from DQM_RESULT base table and thus improve performance. Any idea on how to achieve this?
Related
I have the following LINQ query (using EF Core 6 and MS SQL Server):
var resultSet = dbContext.Systems
.Include(system => system.Project)
.Include(system => system.Template.Type)
.Select(system => new
{
System = system,
TemplateText = system.Template.TemplateTexts.FirstOrDefault(templateText => templateText.Language == locale.LanguageIdentifier),
TypeText = system.Template.Type.TypeTexts.FirstOrDefault(typeText => typeText.Language == locale.LanguageIdentifier)
})
.FirstOrDefault(x => x.System.Id == request.Id);
The requirement is to retrieve the system matching the requested ID and load its project, template and template's type info. The template has multiple TemplateTexts (one for each translated language) but I only want to load the one matching the requested locale, same deal with the TypeTexts elements of the template's type.
The LINQ query above does that in one query and it gets converted to the following SQL query (I edited the SELECT statements to use * instead of the long list of columns generated):
SELECT [t1].*, [t2].*, [t5].*
FROM (
SELECT TOP(1) [p].*, [t].*, [t0].*
FROM [ParkerSystems] AS [p]
LEFT JOIN [Templates] AS [t] ON [p].[TemplateId] = [t].[Id]
LEFT JOIN [Types] AS [t0] ON [t].[TypeId] = [t0].[Id]
LEFT JOIN [Projects] AS [p0] ON [p].[Project_ProjectId] = [p0].[ProjectId]
WHERE [p].[SystemId] = #__request_Id_1
) AS [t1]
LEFT JOIN (
SELECT [t3].*
FROM (
SELECT [t4].*, ROW_NUMBER() OVER(PARTITION BY [t4].[ReferenceId] ORDER BY [t4].[Id]) AS [row]
FROM [TemplateTexts] AS [t4]
WHERE [t4].[Language] = #__locale_LanguageIdentifier_0
) AS [t3]
WHERE [t3].[row] <= 1
) AS [t2] ON [t1].[Id] = [t2].[ReferenceId]
LEFT JOIN (
SELECT [t6].*
FROM (
SELECT [t7].*, ROW_NUMBER() OVER(PARTITION BY [t7].[ReferenceId] ORDER BY [t7].[Id]) AS [row]
FROM [TypeTexts] AS [t7]
WHERE [t7].[Language] = #__locale_LanguageIdentifier_0
) AS [t6]
WHERE [t6].[row] <= 1
) AS [t5] ON [t1].[Id0] = [t5].[ReferenceId]
which is not bad, it's not a super complicated query, but I feel like my requirement can be solved with a much simpler SQL query:
SELECT *
FROM [Systems] AS [p]
JOIN [Templates] AS [t] ON [p].[TemplateId] = [t].[Id]
JOIN [TemplateTexts] AS [tt] ON [p].[TemplateId] = [tt].[ReferenceId]
JOIN [Types] AS [ty] ON [t].[TypeId] = [ty].[Id]
JOIN [TemplateTexts] AS [tyt] ON [ty].[Id] = [tyt].[ReferenceId]
WHERE [p].[SystemId] = #systemId and tt.[Language] = 2 and tyt.[Language] = 2
My question is: is there a different/simpler LINQ expression (either in Method syntax or Query syntax) that produces the same result (get all info in one go) because ideally I'd like to not have to have an anonymous object where the filtered sub-collections are aggregated. For even more brownie points, it'd be great if the generated SQL would be simpler/closer to what I think would be a simple query.
Is there a different/simpler LINQ expression (...) that produces the same result
Yes (maybe) and no.
No, because you're querying dbContext.Systems, therefore EF will return all systems that match your filter, also when they don't have TemplateTexts etc. That's why it has to generate outer joins. EF is not aware of your apparent intention to skip systems without these nested data or of any guarantee that these systems don't occur in the database. (Which you seem to assume, seeing the second query).
That accounts for the left joins to subqueries.
These subqueries are generated because of FirstOrDefault. In SQL it always requires some sort of subquery to get "first" records of one-to-many relationships. This ROW_NUMBER() OVER construction is actually quite efficient. Your second query doesn't have any notion of "first" records. It'll probably return different data.
Yes (maybe) because you also Include data. I'm not sure why. Some people seem to think Include is necessary to make subsequent projections (.Select) work, but it isn't. If that's your reason to use Includes then you can remove them and thus remove the first couple of joins.
OTOH you also Include system.Project which is not in the projection, so you seem to have added the Includes deliberately. And in this case they have effect, because the entire entity system is in the projection, otherwise EF would ignore them.
If you need the Includes then again, EF has to generate outer joins for the reason mentioned above.
EF decides to handle the Includes and projections separately, while hand-crafted SQL, aided by prior knowledge of the data could do that more efficiently. There's no way to affect that behavior though.
This LINQ query is close to your SQL, but I'm afraid of correctness of the result:
var resultSet =
(from system in dbContext.Systems
from templateText in system.Template.TemplateTexts
where templateText.Language == locale.LanguageIdentifier
from typeText in system.Template.Type.TypeTexts
where typeText.Language == locale.LanguageIdentifier
select new
{
System = system,
TemplateText = templateText
TypeText = typeText
})
.FirstOrDefault(x => x.System.Id == request.Id);
I hope anyone can help me :) My problem is when i run my code i am seeing this error " SQL command not properly ended" and "example" is red.
Thank you for your interested
(select mat.oid matoid
from lsn.material mat,
lsn.plan_def def,
lsn.unit meu
where math.plan_def_oid = def.oid
and math.unit_oid = meu.oid
)example left join lsn.plan_det det on det.def_oid = example .matoid;
I think you can just join all four tables together in one go:
select math.oid matoid
from lsn.material mat
inner join lsn.plan_def def
on math.plan_def_oid = def.oid
inner join lsn.unit meu
on math.unit_oid = meu.oid
left join lsn.plan_det det
on plan_def_oid = math.oid
Changes I made here include converting your implicit joins to explicit inner joins. As a general rule, you should avoid putting commas into the from clause. Also I moved your original incomplete subquery out to the main query.
In addition, it looks like there is a reference to a RESULTS table that is missing
At my job our main application was written long ago before n-tier was really a thing, ergo - it has tons and tons of business logic begin handled in stored procs and such.
So we have finally decided to bite the bullet and make it not suck so bad. I have been tasked with converting a 900+ line sql script to a .NET exe, which I am doing in C#/Linq. Problem is...for the last 5-6 years at another job, I had been doing Linq exclusively, so my SQL has gotten somewhat rusty, and some of thing I am converting I have never tried to do before in Linq, so I'm hitting some roadblocks.
Anyway, enough whining.
I'm having trouble with the following sql statement, I think due to the fact that he is joining on a temp table and a derived table. Here's the SQL:
insert into #processedBatchesPurgeList
select d.pricebatchdetailid
from pricebatchheader h (nolock)
join pricebatchstatus pbs (nolock) on h.pricebatchstatusid = pbs.pricebatchstatusid
join pricebatchdetail d (nolock) on h.pricebatchheaderid = d.pricebatchheaderid
join
( -- Grab most recent REG.
select
item_key
,store_no
,pricebatchdetailid = max(pricebatchdetailid)
from pricebatchdetail _pbd (nolock)
join pricechgtype pct (nolock) on _pbd.pricechgtypeid = pct.pricechgtypeid
where
lower(rtrim(ltrim(pct.pricechgtypedesc))) = 'reg'
and expired = 0
group by item_key, store_no
) dreg
on d.item_key = dreg.item_key
and d.store_no = dreg.store_no
where
d.pricebatchdetailid < dreg.pricebatchdetailid -- Make sure PBD is not most recent REG.
and h.processeddate < #processedBatchesPurgeDateLimit
and lower(rtrim(ltrim(pbs.pricebatchstatusdesc))) = 'processed' -- Pushed/processed batches only.
So that's raising an overall question first: how to handle temp tables in Linq? This script uses about 10 of them. I currently have them as List. The problem is, if I try to .Join() on one in a query, I get the "Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator." error.
I was able to get the join to the derived table to work using 2 queries, just so a single one wouldn't get nightmarishly long:
var dreg = (from _pbd in db.PriceBatchDetails.Where(pbd => pbd.Expired == false && pbd.PriceChgType.PriceChgTypeDesc.ToLower().Trim() == "reg")
group _pbd by new { _pbd.Item_Key, _pbd.Store_No } into _pbds
select new
{
Item_Key = _pbds.Key.Item_Key,
Store_No = _pbds.Key.Store_No,
PriceBatchDetailID = _pbds.Max(pbdet => pbdet.PriceBatchDetailID)
});
var query = (from h in db.PriceBatchHeaders.Where(pbh => pbh.ProcessedDate < processedBatchesPurgeDateLimit)
join pbs in db.PriceBatchStatus on h.PriceBatchStatusID equals pbs.PriceBatchStatusID
join d in db.PriceBatchDetails on h.PriceBatchHeaderID equals d.PriceBatchHeaderID
join dr in dreg on new { d.Item_Key, d.Store_No } equals new { dr.Item_Key, dr.Store_No }
where d.PriceBatchDetailID < dr.PriceBatchDetailID
&& pbs.PriceBatchStatusDesc.ToLower().Trim() == "processed"
select d.PriceBatchDetailID);
So that query gives the expected results, which I am holding in a List, but then I need to join the results of that query to another one selected from the database, which is leading me back to the aforementioned "Local sequence cannot be used..." error.
That query is this:
insert into #pbhArchiveFullListSaved
select h.pricebatchheaderid
from pricebatchheader h (nolock)
join pricebatchdetail d (nolock)
on h.pricebatchheaderid = d.pricebatchheaderid
join #processedBatchesPurgeList dlist
on d.pricebatchdetailid = dlist.pricebatchdetailid -- PBH list is restricted to PBD purge list rows that have PBH references.
group by h.pricebatchheaderid
The join there on #processedBatchesPurgeList is the problem I am running into.
So uh...help? I have never written SQL like this, and certainly never tried to convert it to Linq.
As pointed out by the comments above, this is no longer being rewritten as Linq.
Was hoping to get a performance improvement along with achieving better SOX compliance, which was the whole reason for the rewrite in the first place.
I'm happy with just satisfying the SOX compliance issues.
Thanks, everyone.
A little background on the query below. Cell has a 1:M to Container and a 1:M with Printer. I want a query that will retrieve all Cells and associated containers, if they exist, and associated printers, if they exist. Essentially I want to do a left outer join on both tables. Here is the query I have:
var query = from cell in Cell
join container in Container.Where (row => row.SerialNumber == "1102141") on cell.CellID equals container.CellID
into containers
join printer in Printer.Where (row => row.Name == "PG10RelWarrPrt3") on cell.CellID equals printer.CellID
into printers
select new { Cell = cell, Containers = containers, Printers = printers };
query.Dump();
This query works, but is not efficient. It does a left outer join on Container, but, for each Cell, it performs a separate query to retrieve any Printer rows, instead of also doing a left outer join on Printer.
How can I change this so that it also does a left outer join on the Printer table? BTW, I want a hierarchical result set. IOW, each Cell should have a list of containers and a list of printers. Each would be empty of course, if none existed for the cell.
Here's a query to produce a flat result set with correct left joins.
var query = from cell in Cell
join container in Container.Where (row => row.SerialNumber == "1102141") on cell.CellID equals container.CellID
into containers
from container2 in containers.DefaultIfEmpty()
join printer in Printer.Where (row => row.Name == "PG10RelWarrPrt3") on cell.CellID equals printer.CellID
into printers
from printer2 in printers.DefaultIfEmpty()
select new { Cell = cell, Container = container2, Printer = printer2 };
You'll have to post-process the results locally to get the hierarchical shape desired.
If you write this post-processing code, you'll understand why linq to sql doesn't process multiple sibling collections for you.
To make this clearer, suppose you had 3 sibling collections.
If all three sibling collections were empty for some parent record, you'd have just the parent record 1 time with a bunch of nulls.
If all three sibling collections had 100 records for some parent record, you'd have 1 million rows, each with a copy of the parent record. Every child record would be duplicated 10,000 times in the result.
It's always important to keep in mind with any ORM that it generates sql and gets back flat result sets, no matter what hierarchically shaped result it eventually present you with.
It's usually wrong to use join in LINQ to SQL.
Try:
var query = from cell in Cell
select new
{
Cell = cell,
Containers = cell.Containers
.Where (row => row.SerialNumber == "1102141"),
Printers = cell.Printers
.Where (row => row.Name == "PG10RelWarrPrt3")
};
I have the following SQL query:-
select distinct * from dbo.Profiles profiles
left join ProfileSettings pSet on pSet.ProfileKey = profiles.ProfileKey
left join PlatformIdentities pId on pId.ProfileKey = profiles.Profilekey
I need to convert it to a LinqToEntities expression. I have tried the following:-
from profiles in _dbContext.ProfileSet
let leftOuter = (from pSet in _dbContext.ProfileSettingSet
select new
{
pSet.isInternal
}).FirstOrDefault()
select new
{
profiles.ProfileKey,
Internal = leftOuter.isInternal,
profiles.FirstName,
profiles.LastName,
profiles.EmailAddress,
profiles.DateCreated,
profiles.LastLoggedIn,
};
The above query works fine because I haven't considered the third table "PlatformIdentities". Single left outer join works with what I have done above. How do I include PlatformIdentities (the 3rd table) ? I basically want to translate the SQL query I specified at the beginning of this post (which gives me exactly what I need) in to LinqToEntities.
Thanks
Let me know if you want to select something different, but a true join is below
from p in _dbContext.ProfileSet
join ps in _dbContext.ProfileSettings on p.ProfileKey = ps.ProfileKey into a
join pi in _dbContext.PlatformIdentities on p.ProfileKey = pi.ProfileKey into b
select new
{
profiles.ProfileKey,
profiles.FirstName,
profiles.LastName,
profiles.EmailAddress,
profiles.DateCreated,
profiles.LastLoggedIn,
PlatformSettings = a.Select(x=>x),
PlatformIdentities = b.Select(y=>y)
}