I am using automapping with Fluent NHibernate, and have used the following code to ensure that NHibernate does not strip away the milliseconds:
public class TimestampTypeConvention : IPropertyConvention, IPropertyConventionAcceptance
{
public void Accept(IAcceptanceCriteria<IPropertyInspector> criteria)
{
criteria.Expect(x => x.Type == typeof(DateTime) || x.Type == typeof(DateTimeOffset));
}
public void Apply(IPropertyInstance instance)
{
instance.CustomType<TimestampType>();
}
}
This works quite well, so the data is stored correctly in the database.
However, when I run the following LINQ query, I don't get a match where I would expect it:
bool isDuplicate = session.Query<TagData>()
.Any(x => x.TagName == message.EventTag.TagName
&& x.TimeStamp == message.EventTag.TimeStamp.UtcDateTime);
The resulting SQL looks like this, and explains why this doesn't work:
select tagdata0_."Id" as column1_0_, tagdata0_."TagName" as column2_0_,
tagdata0_."TimeStamp" as column3_0_, tagdata0_."Value" as column4_0_,
tagdata0_."QualityTimeStamp" as column5_0_, tagdata0_."QualitySubstatus" as column6_0_,
tagdata0_."QualityExtendedSubstatus" as column7_0_, tagdata0_."QualityLimit" as column8_0_,
tagdata0_."QualityDataSourceError" as column9_0_, tagdata0_."QualityTagStatus" as column10_0_,
tagdata0_."TagType" as column11_0_ from "TagData" tagdata0_
where tagdata0_."TagName"=:p0 and tagdata0_."TimeStamp"=:p1 limit 1;
:p0 = 'VALVE_HW_CMD' [Type: String (0)],
:p1 = 01.03.2013 16:51:30 [Type: DateTime (0)]
How can I force the generated query to use the full precision?
BTW, message.EventTag.TimeStamp is a DateTimeOffset
I was fooled by the logging output: The actual SQL (taken from the PostgreSQL log file) looks like this:
SELECT this_."Id" as column1_0_0_, this_."TagName" as column2_0_0_,
this_."TimeStamp" as column3_0_0_, this_."Value" as column4_0_0_,
this_."QualityTimeStamp" as column5_0_0_, this_."QualitySubstatus" as column6_0_0_,
this_."QualityExtendedSubstatus" as column7_0_0_, this_."QualityLimit" as column8_0_0_,
this_."QualityDataSourceError" as column9_0_0_, this_."QualityTagStatus" as column10_0_0_,
this_."TagType" as column11_0_0_ FROM "TagData" this_
WHERE this_."TimeStamp" = ((E'2013-03-01 16:51:30.509498')::timestamp)
That's the real reason that it didn't work quite as expected: The timestamp column in PostgreSQL only has microsecond accuracy, whereas the DateTimeDiff here has the value of 16:51:30.5094984, which means 1/10th of a microsecond accuracy. The only way to preserve the full accuracy seems to be to store the ticks in the database.
(Another reason for my confusion was that I received the duplicate messages from MassTransit more or less simultaneously on different threads, so checking the DB for duplicates of course didn't always work. O, the wonders of multithreading!)
Related
I have the following code:
DateTime timeStamp = Convert.ToDateTime(Request.QueryString["TimeStamp"]);
var result = (from rs in db.VRec
where
rs.TimeStamp == timeStamp &&
rs.Fixure == wFixture
select rs).ToList();
The result shows 0 even though the correct timeStamp is passed.
If I remove the part where I do the TimeStamp comparison:
rs.TimeStamp == timeStamp
The code works fine.
Any idea on why the datetime comparison may not be working?
DateTime has a pretty fine resolution - likely you are comparing timestamps that only differ in milliseconds, which will fail. You probably want something like:
DateTime now = DateTime.Now;
DateTime then = now.Add(TimeSpan.FromMilliseconds(1));
const int EPSILON_MS = 10;
if(now.Subtract(then).TotalMilliseconds < EPSILON_MS)
{
Console.WriteLine("More or less equal!");
}
Linq converts DateTime arguments to DateTime2 in the sql query executed.
That is, when you do the comparison the actual sql executed will compare a DateTime to a DateTime2. This comparison will "cast" the DateTime to a DateTime2 and the millisecond part will be expanded to a greater resolution (in an odd way in my opinion, please enlighten me).
Try to execute the following sql:
declare #d1 datetime = '2016-08-24 06:53:01.383'
declare #d2 datetime2 = '2016-08-24 06:53:01.383'
declare #d3 datetime2 = #d1
select #d1 as 'd1', #d2 'd2', #d3 'converted'
select (case when (#d1 = #d2) then 'True' else 'False' end) as 'Equal',
(case when (#d1 > #d2) then 'True' else 'False' end) as 'd1 greatest'
From the question, I do not know if you want to compare the date with time or only the date part. If you only want to compare date then following would work
var result = (from rs in db.VRec
where
rs.TimeStamp.Date == timeStamp.Date &&
rs.Fixure == wFixture
select rs).ToList();
Since you are using some reference to db, it gives me a feeling that you are fetching your records from database (which ORM you are using is not obvious from the question or tags). Assuming that you are using Entity framework the above query will fail with exception that .Date has no direct translation to sql. If so you can rewrite the query as following to make it work.
var result = (from rs in db.VRec
where
rs.TimeStamp.Day == timeStamp.Day &&
rs.TimeStamp.Month == timeStamp.Month &&
rs.TimeStamp.Year == timeStamp.Year &&
rs.Fixure == wFixture
select rs).ToList();
The benefit of this approach is that you can compare properties to arbitrary deep level i.e you can compare Hours, Minutes,Seconds etc. in your query. The second query is tested in Entity framework 5.
My Linq to entities query is written as below.
The datatype of DATECOLUMN1 in my ORACLE database is of string.
Datetime FilterStartDate = DateTime.Now;
var query = from c in db.TABLE1
join l in db.TABLE2 on c.FK equals l.PK
where (FilterStartDate >= DateTime.ParseExact(l.DATECOLUMN1, "dd/MM/yyyy", CultureInfo.InvariantCulture) : false) == true
select c;
Writing above query gives me an error of not supported. How can I convert DATECOLUMN1 into a datetime to compare it.
P.S. I do not have control over database schema, so changing datatype of column in Oracle database is not a feasible solution for me.
In you Model, add the following property to your partial class TABLE2:
public DateTime DATECOLUMN1_NEW
{
get
{
return DateTime.ParseExact(DATECOLUMN1, "dd/MM/yyyy", CultureInfo.InvariantCulture);
}
set { }
}
Then, in you LINQ query, use DATECOLUMN1_NEW (it's already in DateTime format) in place of DATECOLUMN1.
Erm.. I think the problem you are having is that you are putting ": false" in there.
It looks like you are trying to use a condtional operator (?:) but you forgot the "?".
I don't think you actually need this as you are just trying to determine if the date is greater or not. Also if ParseExact fails it will throw an exception (not what you want) so you should use TryParse instead and handle the true/false returned and the out value to determine whether or not the date is (a) Actually a date (b) less then FilterStartDate.
You can use two alternatives:
Use the function described in the answer here: How to I use TryParse in a linq query of xml data?
Use the following fluent syntax version which I think is more readable.
var query = db.Table1.Join(db.Table2, x => x.FK, y => y.PK, (x, y) => x).Where(x =>
{
DateTime Result;
DateTime.TryParse(x.Date, out Result);
return DateTime.TryParse(x.Date, out Result) && FilterStartDate >= Result;
});
I'm programming a search for a SQLite-database using C# and LINQ.
The idea of the search is, that you can provide one or more keywords, any of which must be contained in any of several column-entries for that row to be added to the results.
The implementation consists of several linq-queries which are all put together by union. More keywords and columns that have to be considered result in a more complicated query that way. This can lead to SQL-code, which is to long for the SQLite-parser.
Here is some sample code to illustrate:
IQueryable<Reference> query = null;
if (searchAuthor)
foreach (string w in words)
{
string word = w;
var result = from r in _dbConnection.GetTable<Reference>()
where r.ReferenceAuthor.Any(a => a.Person.LastName.Contains(word) || a.Person.FirstName.Contains(word))
orderby r.Title
select r;
query = query == null ? result : query.Union(result);
}
if (searchTitle)
foreach (string word in words)
{
var result = from r in _dbConnection.GetTable<Reference>()
where r.Title.Contains(word)
orderby r.Title
select r;
query = query == null ? result : query.Union(result);
}
//...
Is there a way to structure the query in a way that results in more compact SQL?
I tried to force the creation of smaller SQL-statments by calling GetEnumerator() on the query after every loop. But apparently Union() doesn't operate on data, but on the underlying LINQ/SQL statement, so I was generating to long statements regardless.
The only solution I can think of right now, is to really gather the data after every "sub-query" and doing a union on the actual data and not in the statement. Any ideas?
For something like that, you might want to use a PredicateBuilder, as shown in the chosen answer to this question.
After profiling my Entity Framework 4.0 based database layer I have found the major performance sinner to be a simple LINQ Any() I use to check if an entity is already existing in the database. The Any() check performs orders of magnitude slower than saving the entity.
There are relatively few rows in the database and the columns being checked are indexed.
I use the following LINQ to check for the existence of a setting group:
from sg in context.SettingGroups
where sg.Group.Equals(settingGroup) && sg.Category.Equals(settingCategory)
select sg).Any()
This generates the following SQL (additionally my SQL profiler claims the query is executed twice):
exec sp_executesql N'SELECT
CASE WHEN ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[SettingGroups] AS [Extent1]
WHERE ([Extent1].[Group] = #p__linq__0) AND ([Extent1].[Category] = #p__linq__1)
)) THEN cast(1 as bit) WHEN ( NOT EXISTS (SELECT
1 AS [C1]
FROM [dbo].[SettingGroups] AS [Extent2]
WHERE ([Extent2].[Group] = #p__linq__0) AND ([Extent2].[Category] = #p__linq__1)
)) THEN cast(0 as bit) END AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]',N'#p__linq__0 nvarchar(4000),#p__linq__1 nvarchar(4000)',#p__linq__0=N'Cleanup',#p__linq__1=N'Mediator'
Right now I can only think of creating stored procedures to solve this problem, but I would of course prefer to keep the code in LINQ.
Is there a way to make such an "Exist" check run faster with EF?
I should probably mention that I also use self-tracking-entities in an n-tier architecture. In some scenarios the ChangeTracker state for some entities is set to "Added" even though they already exist in the database. This is why I use a check to change the ChangeTracker state accordingly if updating the database caused an insert failure exception.
Try adding index to the database table "SettingGroups", by Group & Category.
BTW, does this produce similar sql?
var ok = context.SettingGroups.Any(sg => sg.Group==settingGroup && sg.Category==settingCategory);
The problem is Entity Framework (at least EF4) is generating stupid SQL. The following code seems to generate decent SQL with minimal pain.
public static class LinqExt
{
public static bool BetterAny<T>( this IQueryable<T> queryable, Expression<Func<T, bool>> predicate)
{
return queryable.Where(predicate).Select(x => (int?)1).FirstOrDefault().HasValue;
}
public static bool BetterAny<T>( this IQueryable<T> queryable)
{
return queryable.Select(x => (int?)1).FirstOrDefault().HasValue;
}
}
Then you can do:
(from sg in context.SettingGroups
where sg.Group.Equals(settingGroup) && sg.Category.Equals(settingCategory)
select sg).BetterAny()
or even:
context.SettingGroups.BetterAny(sg => sg.Group.Equals(settingGroup) && sg.Category.Equals(settingCategory));
I know it sounds a miserable solution, but what happens if you use Count instead of Any?
Have you profiled the time to execute the generated select statement against the time to execute the select what you would expect/like to be produced ? It is possible that it is not as bad as it looks.
The section
SELECT
1 AS [C1]
FROM [dbo].[SettingGroups] AS [Extent1]
WHERE ([Extent1].[Group] = #p__linq__0) AND ([Extent1].[Category] = #p__linq__1)
is probably close to what you would expect to be produced. It is quite possible that the query optimiser will realise the second query is the same as the first and hence it may add very little time to the overall query.
I'm having performance issues with Linq2Sql compared to raw ADO.NET which has led me down the path of compiled queries. I have got this far so far
public static readonly Func<MyDataContext, WebServices.Search.Parameters, IQueryable<image>>
Compiled_SelectImagesLinq =
CompiledQuery.Compile<MyDataContext, WebServices.Search.Parameters, IQueryable<image>>(
(dc, parameters) => from i in dc.images
join l in dc.links on i.image_id equals l.image_id
join r in dc.resolutions on i.image_id equals r.image_id
where i.image_enabled == true && i.image_rating >= parameters.MinRating
&& i.image_rating <= parameters.MaxRating
select i
);
However I can't figure out how to add the extra optional parameters to the query as I currently do
if (parameters.Country != null)
{
query = query.Where(x => x.image_country_id == parameters.Country);
}
if (parameters.ComponentId != null)
{
query = query.Where(x => x.links.Any(l => l.link_component_id == parameters.ComponentId));
}
etc, etc
I tried writing another function which does
var query = Compiled_SelectImagesLinq(parameters);
and then adding the extra parameters to the query and returning
return query.Distinct().Take(parameters.Results);
Bit this doesn't seem right and returns no results
Have a look at this article. It may not do what you need (especially since you are compiling your queries), but anytime someone mentions Dynamic and Linq in the same sentence, I refer them to this article:
Dynamic LINQ: Using the LINQ Dynamic Query Library
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
You'd have to benchmark your specific query, but often queries must be used 10-20 times before compiled query performance improvements equal the overhead. Also, how are you adding parameters to the where clause?
Additionally, dynamic compiled queries seems a bit of a mismatch. The Dynamic LINQ query library will do what you need but I don't think you'll get the compiled query performance improvement you want.