Calculate time interval between 2 dates on different rows - linq

I have login and logout events and i need to calculate time between them.
I guess i could group each 2 rows (each two messages) and then do the calculation, but how would you do that?
Example XML i need to query:
<Log>
<Message>
<DateTime>2009-12-02 14:38:41</DateTime>
<Priority>Local3.Info</Priority>
<Source_Host>192.168.0.100</Source_Host>
<MessageText>Dec 2 14:38:41 root: login,ng1,192.168.0.160,janis.veinbergs</MessageText>
</Message>
<Message>
<DateTime>2009-12-02 15:28:19</DateTime>
<Priority>Local3.Info</Priority>
<Source_Host>192.168.0.100</Source_Host>
<MessageText>Dec 2 15:30:33 root: logout,ng1,,janis.veinbergs</MessageText>
</Message>
<Message>
<DateTime>2009-12-02 15:29:11</DateTime>
<Priority>Local3.Info</Priority>
<Source_Host>192.168.0.100</Source_Host>
<MessageText>Dec 2 15:31:25 root: login,ng1,192.168.0.160,janis.veinbergs</MessageText>
</Message>
<Message>
<DateTime>2009-12-02 15:58:22</DateTime>
<Priority>Local3.Info</Priority>
<Source_Host>192.168.0.100</Source_Host>
<MessageText>Dec 2 16:00:37 root: logout,ng1,,janis.veinbergs</MessageText>
</Message>
</Log>
Thankyou.

Given SQL doesn't have any aggregate Diff method, I would suggest joining the table onto itself, and selecting each row that you're after from each side of the join.
Something like:
var diff = from a in db.Events
join b in db.Events on a.SessionId equals b.SessionId
where a.EventType == 'Login' && b.EventType == 'Logout'
select b.EventTime - a.EventTime;
Haven't tried this, but something along those lines should work.
EDIT: updated to suit new provided info.
Try the following. Could perhaps be more concise, but does the job. Have broken down into a couple of queries for easier reading.
var query = from a in (from log in data.Elements()
select new {
date = DateTime.Parse(log.Element("DateTime").Value),
msg = log.Element("MessageText").Value
})
select new {
a.date,
type = a.msg.Contains("login") ? "Login" : "Logout",
user = a.msg.Substring(a.msg.LastIndexOf(',') + 1)
};
var results = from a in query
join b in query on a.user equals b.user
where a.type == "Login" && b.type == "Logout"
&& b.date == (query.OrderBy(o => o.date).Where(d => d.date > a.date).FirstOrDefault().date)
select new {
a.user,
Login = a.date,
Logout = b.date
};

Related

LINQ - Previous Record

All:
Lets say I have the following table:
RevisionID, Project_ID, Count, Changed_Date
1 2 4 01/01/2016: 01:02:01
2 2 7 01/01/2016: 01:03:01
3 2 8 01/01/2016: 01:04:01
4 2 3 01/01/2016: 01:05:01
5 2 15 01/01/2016: 01:06:01
I am ordering the records based on Updated_Date. A user comes into my site and edits record (RevisionID = 3). For various reasons, using LINQ (with entity framework), I need to get the previous record in the table, which would be RevisionID = 2 so I can perform calculations on "Count". If user went to edit record (RevisionID = 4), I would need to select RevisionID = 3.
I currently have the following:
var x = _db.RevisionHistory
.Where(t => t.Project_ID == input.Project_ID)
.OrderBy(t => t.Changed_Date);
This works in finding the records based on the Project_ID, but how then do I select the record before?
I am trying to do the following, but in one LINQ statement, if possible.
var itemList = from t in _db.RevisionHistory
where t.Project_ID == input.Project_ID
orderby t.Changed_Date
select t;
int h = 0;
foreach (var entry in itemList)
{
if (entry.Revision_ID == input.Revision_ID)
{
break;
}
h = entry.Revision_ID;
}
var previousEntry = _db.RevisionHistory.Find(h);
Here is the correct single query equivalent of your code:
var previousEntry = (
from r1 in db.RevisionHistory
where r1.Project_ID == input.Project_ID && r1.Revision_ID == input.Revision_ID
from r2 in db.RevisionHistory
where r2.Project_ID == r1.Project_ID && r2.Changed_Date < r1.Changed_Date
orderby r2.Changed_Date descending
select r2
).FirstOrDefault();
which generates the following SQL query:
SELECT TOP (1)
[Project1].[Revision_ID] AS [Revision_ID],
[Project1].[Project_ID] AS [Project_ID],
[Project1].[Count] AS [Count],
[Project1].[Changed_Date] AS [Changed_Date]
FROM ( SELECT
[Extent2].[Revision_ID] AS [Revision_ID],
[Extent2].[Project_ID] AS [Project_ID],
[Extent2].[Count] AS [Count],
[Extent2].[Changed_Date] AS [Changed_Date]
FROM [dbo].[RevisionHistories] AS [Extent1]
INNER JOIN [dbo].[RevisionHistories] AS [Extent2] ON [Extent2].[Project_ID] = [Extent1].[Project_ID]
WHERE ([Extent1].[Project_ID] = #p__linq__0) AND ([Extent1].[Revision_ID] = #p__linq__1) AND ([Extent2].[Changed_Date] < [Extent1].[Changed_Date])
) AS [Project1]
ORDER BY [Project1].[Changed_Date] DESC
hope I understood what you want.
Try:
var x = _db.RevisionHistory
.FirstOrDefault(t => t.Project_ID == input.Project_ID && t.Revision_ID == input.Revision_ID -1)
Or, based on what you wrote, but edited:
_db.RevisionHistory
.Where(t => t.Project_ID == input.Project_ID)
.OrderBy(t => t.Changed_Date)
.TakeWhile(t => t.Revision_ID != input.Revision_ID)
.Last()

ajax and ruby script only returning 1 record instead of many

I am doing an Ajax call, using Ruby and Sinatra. The query should return multiple rows, it only returns one though.
The ajax script is:
$(document).ready(function() {
$(".showmembers").click(function(e) {
e.preventDefault();
alert('script');
var short_id = $('#shortmembers').val();
console.log(short_id);
$.getJSON(
"/show",
{ 'id' : short_id },
function(res, status) {
console.log(res);
$('#result').html('');
$('#result').append('<input type=checkbox value=' + res["email"] + '>');
$('#result').append( res["first"] );
$('#result').append( res["last"] );
$('#result').append( res["email"] );
});
});
});
and the Ruby script is:
get '/show' do
id = params['id']
DB["select shortname, first, last, email from shortlists sh JOIN shortmembers sm ON sm.short_id = sh.list_id JOIN candidates ca ON ca.id = sm.candidate_id where sh.list_id = ?", id].each do |row|
#shortname = row[:shortname]
#first = row[:first]
#last = row[:last]
#email = row[:email]
puts #shortname
puts #first
puts #last
puts #email
halt 200, { shortname: #shortname, first: #first, last: #last, email: #email }.to_json
end
end
If I run the query directly in the terminal on postgres I get 9 rows returned but, as above on my website, it just returns the first row only.
What's the problem? No error in the console, just one record.
You have halt 200 inside your loop. This will cause Sinatra to terminate the request processing and return the result back up the stack.
To return a full set of results, you will need to do something like the following:
get '/show' do
id = params['id']
results = DB["select shortname, first, last, email from shortlists sh
JOIN shortmembers sm ON sm.short_id = sh.list_id
JOIN candidates ca ON ca.id = sm.candidate_id
where sh.list_id = ?", id].map do |row|
{
:short_name => row[:shortname],
:first=>row[:first],
:last=>row[:last],
:email=>row[:email]
}
end
halt 200, results.to_json
end
This will return the selected fields from each row as an array of hashes.
In fact, as I look at the above, the solution might even be as simple as:
get '/show' do
id = params['id']
results = DB["select shortname, first, last, email from shortlists sh
JOIN shortmembers sm ON sm.short_id = sh.list_id
JOIN candidates ca ON ca.id = sm.candidate_id
where sh.list_id = ?", id]
halt 200, results.to_json
end
since you don't seem to be selecting anything but the columns you desire in the first place.

Entity Framework appears to be needlessly joining the same table twice

Update This may already be fixed: http://entityframework.codeplex.com/workitem/486
...
A fairly straightforward LINQ statement against my entities is resulting in unnecessarily complex SQL. More on that later, here's the setup:
Tables
Publication
PublicationId (pk)
TopicId (fk to a Topic table)
ReceiptCount (denormalized for query performance)
DateInserted
Receipt
ReceiptId (pk)
PublicationId (fk to the table above)
DateInserted
LINQ
var query = from r in context.Receipts.Include("Publication")
where r.DateInserted < lagDate
&& r.ReceiptId > request.AfterReceiptId
&& r.Publication.TopicId == topicEntity.TopicId
&& r.Publication.ReceiptCount > 1
select r;
SQL
exec sp_executesql N'SELECT TOP (25)
[Project1].[ReceiptId] AS [ReceiptId],
[Project1].[PublicationId] AS [PublicationId],
[Project1].[DateInserted] AS [DateInserted],
[Project1].[DateReceived] AS [DateReceived],
[Project1].[PublicationId1] AS [PublicationId1],
[Project1].[PayloadId] AS [PayloadId],
[Project1].[TopicId] AS [TopicId],
[Project1].[BrokerType] AS [BrokerType],
[Project1].[DateInserted1] AS [DateInserted1],
[Project1].[DateProcessed] AS [DateProcessed],
[Project1].[DateUpdated] AS [DateUpdated],
[Project1].[PublicationGuid] AS [PublicationGuid],
[Project1].[ReceiptCount] AS [ReceiptCount]
FROM ( SELECT
[Extent1].[ReceiptId] AS [ReceiptId],
[Extent1].[PublicationId] AS [PublicationId],
[Extent1].[DateInserted] AS [DateInserted],
[Extent1].[DateReceived] AS [DateReceived],
[Extent3].[PublicationId] AS [PublicationId1],
[Extent3].[PayloadId] AS [PayloadId],
[Extent3].[TopicId] AS [TopicId],
[Extent3].[BrokerType] AS [BrokerType],
[Extent3].[DateInserted] AS [DateInserted1],
[Extent3].[DateProcessed] AS [DateProcessed],
[Extent3].[DateUpdated] AS [DateUpdated],
[Extent3].[PublicationGuid] AS [PublicationGuid],
[Extent3].[ReceiptCount] AS [ReceiptCount]
FROM [dbo].[Receipt] AS [Extent1]
INNER JOIN [dbo].[Publication] AS [Extent2] ON [Extent1].[PublicationId] = [Extent2].[PublicationId]
LEFT OUTER JOIN [dbo].[Publication] AS [Extent3] ON [Extent1].[PublicationId] = [Extent3].[PublicationId]
WHERE ([Extent2].[ReceiptCount] > 1) AND ([Extent1].[DateInserted] < #p__linq__0) AND ([Extent1].[ReceiptId] > #p__linq__1) AND ([Extent2].[TopicId] = #p__linq__2)
) AS [Project1]
ORDER BY [Project1].[ReceiptId] ASC',N'#p__linq__0 datetime,#p__linq__1 int,#p__linq__2 int',#p__linq__0='2012-09-05 19:39:21:510',#p__linq__1=4458824,#p__linq__2=90
Problem
Publication gets joined twice:
LEFT OUTER JOIN because of .Include("Publication")
INNER JOIN because of the where.
If I remove [Extent2] from the SQL entirely, and change the WHERE bits to use [Extent3], I get the same results back. Since I'm not using Lazy Loading on my entities, I have to .Include("Publication")... is there any solution for this?
I was using EF4, but grabbed EF5 from NuGet to see if it was perhaps fixed, but it produces the same result (although I have no idea how to tell if my EDMX is really using EF5).
There is however, a work-around. It may not be the most elegant solution, but it does exactly what you want; it generates only one join.
Change:
var query = from r in context.Receipts.Include("Publication")
where r.DateInserted < lagDate
&& r.ReceiptId > request.AfterReceiptId
&& r.Publication.TopicId == topicEntity.TopicId
&& r.Publication.ReceiptCount > 1
select r;
To be:
var query = from r in context.Receipts
join pub in context.Publication on r.PublicationId equals pub.PublicationId
where r.DateInserted < lagDate
&& r.ReceiptId > request.AfterReceiptId
&& pub.TopicId == topicEntity.TopicId
&& pub.ReceiptCount > 1
select new {
Receipt = r,
Publication = pub
};
Note that we have removed the Include AND we are no longer using r.Publication.?? in the where clause. Instead we are using pub.??
Now when you loop through query, you will see that r.Publication is not null:
foreach ( var item in query)
{
//see that item.Publication is not null
if(item.Receipt != null && item.Receipt.Publication != null)
{
//do work based on a valid Publication
}
else
{
//do work based on no linked Publication
}
}
This behavior can be avoided by using temporary variables (eg, let pub = r.Publication).
var query = from r in context.Receipts
let pub = r.Publication // using a temp variable
where r.DateInserted < lagDate
&& r.ReceiptId > request.AfterReceiptId
&& pub.TopicId == topicEntity.TopicId
&& pub.ReceiptCount > 1
select new { r, pub };
I will optimize on previous person's answer by changing the code as below, which removes the need of join so you don't have to know what columns you need to join and also don't have to change LINQ when join criteria gets changed. This should have been unnecessary but MS is not focused on fixing their sql code generation right now.
var query = from pub in context.Publications
from r in pub.Reciepts
where r.DateInserted < lagDate
&& r.ReceiptId > request.AfterReceiptId
&& pub.TopicId == topicEntity.TopicId
&& pub.ReceiptCount > 1
select new {
Receipt = r,
Publication = pub
};

LINQ Performance Issue on only a few hundred records

in the following code i've commented on the line that SLOWS my page right down. I did some speed test to reveal the CONTAINS LINQ expression is the problem.
Does anyone know how to change this one line to be more efficient using something else instead. I'm also curious as to why its so slow.
Any ideas (thanks in advance):
var allWaste = _securityRepository.FindAllWaste(userId, SystemType.W);
var allWasteIndicatorItems = _securityRepository.FindAllWasteIndicatorItems();
// First get all WASTE RECORDS
var searchResults = (from s in allWaste
join x in allWasteIndicatorItems on s.WasteId equals x.WasteId
where (s.Description.Contains(searchText)
&& s.Site.SiteDescription.EndsWith(searchTextSite)
&& (s.CollectedDate >= startDate && s.CollectedDate <= endDate))
&& x.EWC.EndsWith(searchTextEWC)
select s).Distinct();
var results = searchResults.AsEnumerable();
if (hazardous != "-1")
{
// User has requested to filter on Hazardous or Non Hazardous only rather than Show All
var HazardousBoolFiltered = (from we in _db.WasteIndicatorItems
.Join(_db.WasteIndicators, wii => wii.WasteIndicatorId, wi => wi.WasteIndicatorId, (wii, wi) => new { wasteid = wii.WasteId, wasteindicatorid = wii.WasteIndicatorId, hazardtypeid = wi.HazardTypeId })
.Join(_db.HazardTypes, w => w.hazardtypeid, h => h.HazardTypeId, (w, h) => new { wasteid = w.wasteid, hazardous = h.Hazardous })
.GroupBy(g => new { g.wasteid, g.hazardous })
.Where(g => g.Key.hazardous == true && g.Count() >= 1)
select we).AsEnumerable(); // THIS IS FAST
// Now join the 2 object to eliminate all the keys that do not apply
if (bHazardous)
results = (from r in results join x in HazardousBoolFiltered on r.WasteId equals x.Key.wasteid select r).AsEnumerable(); //This is FAST
else
results = (from r in results.Where(x => !HazardousBoolFiltered
.Select(y => y.Key.wasteid).Contains(x.WasteId)) select r).AsEnumerable(); // This is DOG SLOW 10-15 seconds !--- THIS IS SLOWING EXECUTION by 10 times --!
}
return results.AsQueryable();
I suggest using a logging / tracing framework like smart inspect or log4net combined with a debug text writer.
http://www.codesprouts.com/post/View-LINQ-To-SQL-Statements-Using-A-Debug-TextWriter.aspx
another possibility is to use the sql server profiler and see what sql linq2sql produces.
also a very nice way is to use the mvc mini profiler in combination with the Profiled DB Connection and the SqlFormatters.SqlServerFormatter.
Try Any (MSDN)
Try this:
results = (from r in results
.Where(x => !HazardousBoolFiltered
.Any(y => y.Key.wasteid == r.WasteId)))
.AsEnumerable()
Or Count:
results = (from r in results
.Where(x => HazardousBoolFiltered
.Count(y => y.Key.wasteid == r.WasteId) == 0))
.AsEnumerable()

Error trying to exclude records with a JOIN to another object

In my code below, is there any way I can use the results in the object 'WasteRecordsExcluded' to join with searchResults, essentially excluding the WasteId's I don't want.
If I debug to the last line I get the error :
base {System.SystemException} = {"The query contains references to items defined on a different data context."}
Or if joining is impossible then i could change bHazardous from TRUE to FALSE and FALSE to TRUE and do some kind of 'NOT IN' comparison.
Going bananas with this one, anyone help? Kind Regards :
var allWaste = _securityRepository.FindAllWaste(userId, SystemType.W);
var allWasteIndicatorItems = _securityRepository.FindAllWasteIndicatorItems();
// First get all WASTE RECORDS
var searchResults = (from s in allWaste
join x in allWasteIndicatorItems on s.WasteId equals x.WasteId
where (s.Description.Contains(searchText)
&& s.Site.SiteDescription.EndsWith(searchTextSite)
&& (s.CollectedDate >= startDate && s.CollectedDate <= endDate))
&& x.EWC.EndsWith(searchTextEWC)
select s).Distinct();
var results = searchResults;
if (hazardous != "-1")
{
// User has requested to filter on Hazardous or Non Hazardous only rather than Show All
var WasteRecordsExcluded = (from we in _db.WasteIndicatorItems
.Join(_db.WasteIndicators, wii => wii.WasteIndicatorId, wi => wi.WasteIndicatorId, (wii, wi) => new { wasteid = wii.WasteId, wasteindicatorid = wii.WasteIndicatorId, hazardtypeid = wi.HazardTypeId })
.Join(_db.HazardTypes, w => w.hazardtypeid, h => h.HazardTypeId, (w, h) => new { wasteid = w.wasteid, hazardous = h.Hazardous })
.GroupBy(g => new { g.wasteid, g.hazardous })
.Where(g => g.Key.hazardous == bHazardous && g.Count() >= 1)
select we);
// Now join the 2 object to eliminate all the keys that do not apply
results = results.Where(n => WasteRecordsExcluded.All(t2 => n.WasteId == t2.Key.wasteid));
}
return results;
Maybe something like this:
.....
var results = searchResults.ToList();
.....
.....
.Where(g => g.Key.hazardous == bHazardous && g.Count() >= 1)
select we).ToList();
.....

Resources