I want to retrieve more recent X lines from elasticsearch using NEST, but when I use .Sort() it will return 0 documents. If I remove Sort() it will return documents but not the most recent ones.
Any idea how to retrieve most recent logs lines from elastic or what i am doing wrong?
var result = elk.Search<FileBeatDto>(x => x
.From(0)
.Index("filebeat-*")
.Type(type)
.Query(q =>
q.Match(qs => qs.Field("fields.asset_tag").Query(asset_tag)) &&
q.Match(qs => qs.Field("message").Query(filter))
)
.Take(lines)
.Sort(ss => ss.Descending(p => p.timestamp))
);
timestamp and version are retrieved as null. Not sure it is because in elastic they are #timestamp and #version.
public class FileBeatDto
{
public DateTime timestamp { get; set; }
public string datetime { get; set; }
public DateTime received_at { get; set; }
public int offset { get; set; }
public string version { get; set; }
public string input_type { get; set; }
public int count { get; set; }
public Beat beat { get; set; }
public string host { get; set; }
public string source { get; set; }
public string message { get; set; }
public string type { get; set; }
public object fields { get; set; }
public List<string> tags { get; set; }
}
type is well defined to the type of document that is not the problem if i remove the sort it returns the documents fine.
I get this error: No mapping found for [timestamp] in order to sort on
Related
I have the following object Job and Job sor are populated by reading in data from an XML file and Sor is populated from a database.
class Job
{
public int JobID { get; set; }
public string DepartmentCode { get; set; }
public string ClientReference { get; set; }
public string JobDescription { get; set; }
public List<JobSor> JobSorList { get; set; }
}
class JobSor
{
public int JobID { get; set; }
public string SorUserCode { get; set; }
public string SorNotes1 { get; set; }
public string SorNotes2 { get; set; }
}
class Sor
{
[Key]
public string code { get; set; }
public string description { get; set; }
public string contract { get; set; }
}
I want to write a linq query that will show me all the JobSors that do not exist in the Sor object.
This is what I have so far but I can’t reference the SorUserCode property?
var db = new dbContext();
var sor = db.Sors.Where(p => p.contract == "??");
var query =
from j in jobs
join p in sor on j.JobSorList.SorUserCode equals p.code into jp
from a in jp.DefaultIfEmpty()
select j;
How can I do this?
First get list of all JobSor from jobs list.
Then apply condition Where its SorUserCode value does not match with Any code value of the sor list.
Your query will be as below.
var query = jobs.SelectMany(x => x.JobSorList)
.Where(x => !sor.Any(y => y.code == x.SorUserCode));
I'm using NEST lib, and very basic setup according to the document. I've added index, mapping, etc.
But when I try to do search it always gives me, "An item with the same key has already been added". It confused me alot on what the error means.
var result = client.Search<Contact>(s => s
.AllIndices()
.From(0)
.Size(10)
.Query(q => q
.MatchAll())
);
Here is the Contact:
public class Contact
{
public Guid Id { get; set; }
public Guid TenantId { get; set; }
public string Title { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime ModifiedDate { get; set; }
public IList<string> Emails { get; set; }
}
You likely have a contact in elastic with the same email address twice. You could change the Email property to a string array perhaps?
I am trying to find one or more documents in RavenDB based on the values of a child collection.
I have the following classes
public class GoldenDocument
{
public GoldenDocument()
{
LinkedDocuments = new List<LinkedDocument>();
MergeMatchFields = new List<MergeMatchField>();
}
public string Id { get; set; }
public Guid SourceRowId { get; set; }
public List<MergeMatchField> MergeMatchFields { get; set; }
public List<LinkedDocument> LinkedDocuments { get; set; }
}
And the class that is in the collection MergeMatchFields
public class MergeMatchField
{
public string Id { get; set; }
public Guid OriginId { get; set; }
public string Name { get; set; }
public MatchType MatchType { get; set; }
public double MatchPerc { get; set; }
public string Value { get; set; }
}
In a List<MergeFields> mergeFields collection I have values that is not stored in RavenDB yet. Values are compared to values in a RavenDB document for find if it is a possible match by executing the following query:
using (var session = documentStore.OpenSession())
{
var docs = from gd in session.Query<GoldenDocument>()
from mf in gd.MergeMatchFields
from tf in mergeFields
where mf.Name == tf.Name
&& JaroWinklerCalculator.jaroWinkler(mf.Value, tf.Value) > .90d
&& !string.IsNullOrEmpty(mf.Value)
select gd;
}
I understand that ravenDB does not support SelectMany() so how would I go about getting the results from the Document store?
Create an index for this that would output the values you want to query on.
Note that you can't just execute arbitrary code the way you do here: JaroWinklerCalculator.jaroWinkler(mf.Value, tf.Value) > .90d
But you can use fuzzy queries, and they will do the same.
I am getting a weird behavior. I have a class that I created that is used to format data comping from a data entity into a data grid. I am a using a linq query to create a list of the class type from a list of the entity type. Some of the properties of the class are accessible from the linq query but other give me an error. (AMNotStartedPortalDisplay' does not contain a definition for 'ChecklistStatusID'). So my question is why can linq access some properties but not others? I see no reason why this should be happening.
Here is my class:
public class AMWOTPortalDisplay
{
public string DisplayName { get; set; }
public string LOB { get; set; }
public string DisplayProjectPackages { get; set; }
public string ChecklistStatus { get; set; }
public int ChecklistStatusID { get; set; }
public string InstallDate { get; set; }
public string dateToYellow { get; set; }
public string dateToRed { get; set; }
public string ApplicationManager { get; set; }
public string ApplicationManagerLanID { get; set; }
public int ApplicationManagerUserID { get; set; }
public string ImpersonatedManager { get; set; }
public string ImpersonatedManagerLanID { get; set; }
public int ImpersonatedManagerUserID { get; set; }
public string DelegateName { get; set; }
public string DelegateLanID { get; set; }
public int DelegateUserID { get; set; }
public string WOTAssignee { get; set; }
public int ChecklistID { get; set; }
public string DisplayLinkText { get; set; }
public string LinkTextURL { get; set; }
public string rowColor { get; set; }
public string rowTextColor { get; set; }
}
And here is the linq query as I have it so far:
var portaldisplay = checklists
.Select(c => new AMNotStartedPortalDisplay
{
DisplayName = string.Format("{0} ({1})", c.Application.Name, c.Application.ApplicationID),
LOB = c.Application.LOB,
ChecklistStatus = c.ChecklistStatusType.TypeName,
ChecklistStatusID = c.ChecklistStatusTypeID
});
Thanks,
Rhonda
Be careful with your types:
public class AMWOTPortalDisplay
And then:
Select(c => new AMNotStartedPortalDisplay { ... })
It looks like your query should probably be:
Select(c => new AMWOTPortalDisplay { ... })
I have two rows of test data and am able to pull back the first row with no problems but can't get the second row to return. Digging around and testing shows that this is probably due to the AppovedByID column being null in the row that is not being returned. I have looked but can't figure out how to modify my LINQ query so it will return all rows even if the child table can't be linked in due to a null value.
The Query:
public JsonResult ChangeOrders()
{
var ChangeOrdersList = _DbContext.ChangeOrders
.Include(co => co.ApprovalStatus)
.Include(co => co.ApprovedBy)
.Include(co => co.AssignedTo)
.Include(co => co.CreatedBy)
.Include(co => co.CurrentStatus)
.Include(co => co.Impact)
.Include(co => co.Priority)
.Include(co => co.ChangeType)
.Select(co => new ChangeOrderListVM()
{
ApprovalStatus = co.ApprovalStatus.Name,
ApprovedBy = string.Concat(co.ApprovedBy.FirstName, ' ', co.ApprovedBy.LastName),
AssignedTo = string.Concat(co.AssignedTo.FirstName, ' ', co.AssignedTo.LastName),
CreatedBy = string.Concat(co.CreatedBy.FirstName, ' ', co.CreatedBy.LastName),
CurrentStatus = co.CurrentStatus.Name,
DateApproved = co.DateApproved,
DateCompleated = co.DateCompleated,
DateCreated = co.DateCreated,
DateStarted = co.DateStarted,
EstimatedEndDate = co.EstimatedEndDate,
EstimatedStartDate = co.EstimatedStartDate,
ID = co.ID,
Impact = co.Impact.Name,
Name = co.Name,
Priority = co.Priority.Name,
Reason = co.ReasonForChange,
Type = co.ChangeType.Name
}).ToList();
return Json(ChangeOrdersList);
}
ChangeOrders:
public class ChangeOrder
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int ID { get; set; }
public short? ApprovedByUserID { get; set; }
public byte ApprovalStatusID { get; set; }
public short AssignedToUserID { get; set; }
public short CreatedByUserID { get; set; }
public byte CurrentStatusID { get; set; }
public DateTime? DateApproved { get; set; }
public DateTime? DateCompleated { get; set; }
public DateTime DateCreated { get; set; }
public DateTime? DateStarted { get; set; }
public DateTime EstimatedStartDate { get; set; }
public DateTime EstimatedEndDate { get; set; }
public byte ImpactID { get; set; }
public byte PriorityID { get; set; }
public byte TypeID { get; set; }
[Required]
public string Name { get; set; }
[Required]
public string ReasonForChange { get; set; }
[ForeignKey("ApprovalStatusID")]
public ChangeApprovalStatus ApprovalStatus { get; set; }
[ForeignKey("ApprovedByUserID")]
public User ApprovedBy { get; set; }
[ForeignKey("AssignedToUserID")]
public User AssignedTo { get; set; }
[ForeignKey("CreatedByUserID")]
public User CreatedBy { get; set; }
[ForeignKey("CurrentStatusID")]
public ChangeStatus CurrentStatus { get; set; }
[ForeignKey("ImpactID")]
public ChangeImpact Impact { get; set; }
[ForeignKey("PriorityID")]
public ChangePriority Priority { get; set; }
[ForeignKey("TypeID")]
public ChangeType ChangeType { get; set; }
}
Users:
public class User
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public short ID { get; set; }
[Required]
public string ADUserName { get; set; }
[Required]
public string FirstName { get; set; }
[Required]
public string LastName { get; set; }
[Required]
public string Email { get; set; }
[Required]
public string Phone { get; set; }
[Required]
public DateTime LastUpdated { get; set; }
}
EDIT:
This is apparently unique to Entity Framework 7 (AKA Core) as it works fine in EF 6. I am in fact using EF7 and as an additional test I updated a single line
ApprovedBy = string.Concat(co.ApprovedBy.FirstName, ' ', co.ApprovedBy.LastName),
and changed it to this
ApprovedBy = "",
and all the rows are returning so I then tried to do
ApprovedBy = (co.ApprovedByUserID.HasValue) ? string.Concat(co.ApprovedBy.FirstName, ' ', co.ApprovedBy.LastName) : "",
but that give a very odd error:
incorrect syntax near the keyword 'is'
This is a bug in the library. I have reported the bug for fixing in the next version on github