I'm using the VS2010 DataSet designer to make some select queries with optional parameters similar to this:
SELECT CustomerID, FirstName, JoinDate, etc
FROM tblCustomers
WHERE (
(#CustomerID IS NULL OR CustomerID = #CustomerID) AND
(#FirstName IS NULL OR FirstName = #FirstName) AND
(#JoinedBefore IS NULL OR JoinDate < #JoinedBefore) AND
(#JoinedAfter IS NULL OR JoinDate > #JoinedAfter) AND
.. etc ..
)
The inference for these properties data-types and allow DB null is almost always wrong. I end up with string types set for date time and vice versa. Over half the fields are always marked as non-null.
That obviously wreaks havoc on my queries. I can manually change these inference's, but every time I have to update the TableAdapter, it resets them all to what it thinks is best! Anyone know how to either a) get the inferences right, or b) override them in a permanent way?
It seems VS infers the data type based on the first occurrence of the parameter in the query. Because I put my #Parater IS NULL OR... first, that confused the designer and caused it to infer wrong a lot of the time. I swapped the order of my query and now it infers perfectly:
SELECT CustomerID, FirstName, JoinDate, etc
FROM tblCustomers
WHERE (
(CustomerID = #CustomerID OR #CustomerID IS NULL AND
(FirstName = #FirstName OR #FirstName IS NULL) AND
(JoinDate < #JoinedBefore OR #JoinedBefore IS NULL) AND
(JoinDate > #JoinedAfter OR #JoinedAfter IS NULL) AND
.. etc ..
)
Related
This is my query:
select count(*)
FROM TB_E2V_DOCUMENTOS_CICLO D
WHERE (D.TIPOCLIENTE = null or null is null)
AND (D.TIPODOCUMENTOCLIENTE = null or null is null)
AND (D.NUMDOCUMENTOCLIENTE = null or null is null)
AND (D.BA = null or null is null)
AND (D.FA = null or null is null)
AND (D.NOMBRECLIENTE = null or null is null)
AND (D.NUMTELEFONO = null or null is null)
AND (D.NUMSUSCRIPCION = null or null is null)
AND (D.TIPORECIBO in ('Recibo'))
AND (D.NUMRECIBO = null or null is null)
AND (TO_DATE(D.FECHAEMISION, 'yyyy/MM/dd') BETWEEN TO_DATE('2019-5-1', 'yyyy-MM-dd') AND TO_DATE('2020-2-18', 'yyyy-MM-dd'))
AND (D.MONTORECIBO = null or null is null)
AND (D.NUMPAGINAS = 0 or 0 = 0)
AND (D.NOMBREARCHIVO = null or null is null)
AND (D.NEGOCIO = null or null is null)
AND (D.NOMBREMETADATACARGA = null or null is null)
AND (D.FECHACARGA = TO_DATE(null) or TO_DATE(null) is null);
This query returns
And when I do a Xplain For:
The cost is very high, but this query uses the index. The query lasts 10 seconds approximately.
How can I improve the performance of the query?
I'm using Oracle 12c
Notes: All of the " and ( = null or null is null)" predicates will always evaluate to true; Oracle does not define null so null does not equal null, so instead if you want to check for null then use "is null"
select * from dual where null = null; -- returns no rows
select * from dual where not (null <> null); -- returns no rows
select * from dual where null is null; -- returns 1 row
select * from dual where not(null is not null); -- returns 1 row
As far as indexing goes, you need an index that is selective (i.e. return much fewer rows) and is present in the where clause predicate. In this case it looks like a function-based index on TO_DATE(D.FECHAEMISION, 'yyyy/MM/dd')
along with D.TIPORECIBO is in order. The INDEX SKIP SCAN is used in this case probably because D.TIPORECIBO is not the leading column; INDEX SKIP SCANs are slower then INDEX RANGE SCANs because it needs to read more index blocks.
There are a few factors involved here:
First, this query is using the second (or third) part of a composite index, resulting in the SKIP SCAN.
Take a look at all indexes on the table and see what kind of index is on TIPORECIBO.
It is likely that this isn't the leading column. You might improve the performance by creating an index with TIPORECIBO as leading column, but it is unlikely--this appears to be a "type" column that might have only a few values, and not a good candidate for an index.
The second issue is that Oracle uses the index to get a set of candidate rows, then goes to the data blocks themselves to get the rows for further filtering.
A select count(*) will perform much better if Oracle doesn't need to fetch the data blocks. This can be achieved by creating an index that contains all of the data needed for the filter.
In your case, an index on TIPORECIBO and FECHAEMISION would mean that Oracle could go to the index alone without needing to access the data blocks.
The third issue is that you are applying TO_DATE to the FECHAEMISION column. If this is a DATE datatype, then you don't need the conversion and it is causing you trouble. If you do need the conversion, an option would be a function-based index on TO_DATE(D.FECHAEMISION, 'yyyy/MM/dd').
To tune this particular query, you can try a function-based composite index:
CREATE INDEX TB_E2V_DOCUMENTOS_CICLO_FX1 ON TB_E2V_DOCUMENTOS_CICLO(FECHAEMISION, TO_DATE(D.FECHAEMISION, 'yyyy/MM/dd'))
Finally, this query is clearly being generated from code:
lines like AND (D.BA = null or null is null) seem to be a way of excluding portions of the WHERE clause when the front-end passes a NULL. This would possibly be AND (D.BA = 'X' or 'X' is null) if a value were provided for that parameter.
As such, be careful when tuning for the current set of parameters, as any change in what generated this query will impact the effectiveness of your tuning.
If you have a way to influence how this query is generated, it would be nice to simply exclude those non-event filters when the values are not provided, though Oracle ought to be able to handle them as-is.
I am filtering an IQueryable to return all entities that have the field UserId (a nullable int) set to null. The query generates the incorrect SQL and thus fails -- the statement is
as follows -
var filtered = certificates.Where(c => !c.UserId.HasValue).Select(c => c.SubjectName);
and the generated SQL is --
SELECT
CAST(NULL AS varchar(1)) AS [C1],
CAST(NULL AS int) AS [C2],
CAST(NULL AS datetime2) AS [C3],
CAST(NULL AS datetime2) AS [C4],
CAST(NULL AS bit) AS [C5],
CAST(NULL AS datetime2) AS [C6],
CAST(NULL AS int) AS [C7]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
WHERE 1 = 0
Any idea WTF is going on? The idea is simple I just want to return all the rows where the field UserId is false. UserId is nullable and the table being queried has three rows that match the condition described, however the LINQ query returns 0.
Thanks!
This is the kind of query that EF generates when it knows for sure that the query won't return any results. Such a query minimizes database processing.
How can EF be so sure? This can only be when for all it knows UserId in the database is not nullable. This, in turn, can only be when there's also a User reference in Certificate (the POCO class) that is mapped as required. Look for something like
HasRequired(t => t.User).WithMany(t => t.Certificates)
in an EntityTypeConfiguration<Certificate>, or in an override of OnModelCreating in your DbContext. (In code-first it is possible to have a required reference, while the accompanying primitive Id property is a nullable type. In an edmx file this doesn't validate).
So I think you have to map User as optional if in the database the foreign key is nullable.
Maybe you could try a more explicit option
var filtered = certificates.Where(c => c.UserId == null).Select(c => c.SubjectName);
I am adding this answer as I spent quite a while trying to diagnose this issue and maybe it will help someone.
I am using Entity Framework 6.1.3 and noticed that when I used the following query it would never return anything, even though there was items in the database that matched this condition.
dbContext.Items.Where(n => n.TypeID == null)
When I looked at the query it was executing it was using the same as the op:
SELECT
CAST(NULL AS int) AS [C1],
CAST(NULL AS int) AS [C2],
CAST(NULL AS varchar(100)) AS [C3],
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
WHERE 1 = 0
I knew that this meant that EF thought that this field could never be null and was returning an empty set and as I was using code first entity framework design I looked at the model declaration for tblItems but didnt see any issue here, I even tried adding .Optional() to the property definition with no luck.
It turned out that I had not configured the relationship between tblItems and tblType correctly on the tblType side and was using .HasRequired() instead of .HasOptional() when defining the relationship between the two tables.
Summary: If you are seeing this default query being used, and the table has relationships with other table(s), make sure both sides of the relationships are defined correctly in your model.
In my case this query also appeared in profiler when EF (v6.4.0) was sure that the result is empty. It was just much simpler case:
var productsIds = new List<int>();
var result = products.Where(p => productsIds.Contains(p.Id)).ToList();
EF must've checked if productsIds is empty and generate query that always return empty from db.
If you really want to get rid of such db queries you can just do your own check.
if (products.Any())
{
var result = products.Where(p => productsIds.Contains(p.Id)).ToList();
}
I wish EF could optimize it a bit further and don't do any db queries in such cases, but I guess it's better that nothing ;)
I believe the reason it is not working for you is that c.UserId has a value, it is just null. You should compare it to null instead:
var filtered = certificates.Where(c => c.UserId == null).Select(c => c.SubjectName);
edit: Accidently had the wrong if statement in there.
When doing an EF query in the code behind using LINQ, how does one go about retrieving only those items in a nullable column that actually have data?
for example;
Dim unit = (From d in ctx.Inventories
Where d.ProductId Is Not Null
Select d).ToList()
Where obviously that query doesnt work, how does one go about this?
Since ProductId is likely a nullable type, you should be able to do:
Dim unit = (From d in ctx.Inventories
Where d.ProductId.HasValue
Select d).ToList()
Using entity framwork with lazy loading - Have the following question on loading related entities when the entities
are null.
Say I have two tables employee and employeedetails. Assume in the above case not all employee entries have an entry in the employeedetails table.
If I want to look up a list of Employees
(from e in objectcontext.employees
select new EmployeeEntity
{
EmpID= e.EmployeeID,
FirstName = e.FirstName,
Address = e.employeedetails.Address
}).ToList();
EmployeeEntity is the data class into which we stuff the results.
The above code breaks if even one employee in the returned list
does not have a entry in table employeedetails. This is obvious since e.employeedetails will be null for those customers who do not have a details entry
What is the best way to rewrite the above query?
Would something like this be acceptable ?
(from e in objectcontext.employees
select new EmployeeEntity
{
EmpID= e.EmployeeID,
FirstName = e.FirstName,
Address = e.employeedetails == null ? "" : e.employeedetails.Address,
}).ToList();
I am not clear on the efficiency of this above query - Would this statment do the null check at DB level?
Should I instead do an explicit include like
objectcontext.include("employeedetails")...
And then loop through the results to check for null?
Yes, this statement would indeed perform a null check in the SQL query that is generated. Most likely, it will simply be a NVL or COALESCE.
That's the way you should be doing it.
I am trying to get customerId from Customer table.
However, when I try to make it, it has error said, 'Cannot implicitly convert type 'System.Linq.Iqueryable' to 'int''
How can I get customerId??
Thank you.
Here is my coding.
int customerId = from a in db.Customer
where a.userName == customerName
select a.customerId;
//it has error in here.
order.customerId = customerId;
It's possible that your query could return multiple rows. You need to tell LINQ that you only want the first (or a single) result:
int customerId = (from a in db.Customer
where a.userName == customerName
select a.customerId).First();
I like using the .Single() function when I know there should be only one row.
int customerId = (from a in db.Customer
where a.userName == customerName
select a.customerId).Single();
This throws an exception if more or less than one row is found, which is sometimes useful depending on your situation.
use it this way:
var customer = db.Customer.FirstOrDefault(c =>c.userName == customerName)
var id = customer.Id
Select returns an IEnumerable, so you can't convert it to int.
There is one more suggestion I can recommend for this question is to use First or default.
var customerId = (from a in db.Customer
where a.userName == customerName
select a.customerId).FirstOrDefault();
order.customerId = customerId; this should now work fine since it already knows it int