In Nest 1.7.1 Delete or DeleteByQuery nothing works - elasticsearch

In Nest 1.7.1 Delete or DeleteByQuery nothing works for me.
I am trying to delete below documents:
Article article1 = new Article()
{
Id = 1111,
Title = "Title - Test Elastic Search",
Summary = "Summary - Test Elastic Search",
Body = "Body - Test Elastic Search",
ArticleDate = _dateToday,
Author = new Author() { Id = 100, Name = "Mikey" },
};
Article article2 = new Article()
{
Id = 2222,
Title = "Title - Test Elastic Search",
Summary = "Summary - Test Elastic Search",
Body = "Body - Test Elastic Search",
ArticleDate = _dateToday,
Author = new Author() { Id = 100, Name = "Mikey" },
Published = true
};
I was expecting below queries would delete single document and all documents in an index but no query is deleting.
_elasticClient.Delete(article).Found;
_elasticClient.DeleteByQuery<Article>(q => q.Query(t => t.Term(m => m.OnField(f => f.Id).Value(articleId))))
.Found;
_elasticClient.DeleteByQuery<Article>(q => q.MatchAll()).IsValid;
Please correct me if i am doing anything wrong.

Here's a working example
void Main()
{
var settings = new ConnectionSettings(new Uri("http://localhost:9200"), "articles");
var client = new ElasticClient(settings);
if (client.IndexExists("articles").Exists)
{
client.DeleteIndex("articles");
}
client.CreateIndex("articles", c => c
.AddMapping<Article>(m => m
.MapFromAttributes()
)
);
var today = DateTime.Now.Date;
var article1 = CreateArticle(1111, today);
var article2 = CreateArticle(2222, today);
var article3 = CreateArticle(3333, today);
var article4 = CreateArticle(4444, today);
var bulkRequest = new BulkDescriptor();
bulkRequest.Index<Article>(i => i.Document(article1));
bulkRequest.Index<Article>(i => i.Document(article2));
bulkRequest.Index<Article>(i => i.Document(article3));
bulkRequest.Index<Article>(i => i.Document(article4));
bulkRequest.Refresh();
client.Bulk(bulkRequest);
var searchResponse = client.Search<Article>(q => q.MatchAll());
Console.WriteLine("Documents from search: {0}. Expect 4", searchResponse.Documents.Count());
client.Delete(article1, d => d.Refresh());
searchResponse = client.Search<Article>(q => q.MatchAll());
Console.WriteLine("Documents from search {0}. Expect 3", searchResponse.Documents.Count());
client.Delete(article2, d => d.Refresh());
searchResponse = client.Search<Article>(q => q.MatchAll());
Console.WriteLine("Documents from search {0}. Expect 2", searchResponse.Documents.Count());
client.DeleteByQuery<Article>(q => q.MatchAll());
searchResponse = client.Search<Article>(q => q.MatchAll());
Console.WriteLine("Documents from search {0}. Expect 0", searchResponse.Documents.Count());
}
private Article CreateArticle(int id, DateTime articleDate)
{
return new Article()
{
Id = id,
Title = "Title - Test Elastic Search",
Summary = "Summary - Test Elastic Search",
Body = "Body - Test Elastic Search",
ArticleDate = articleDate,
Author = new Author() { Id = 100, Name = "Mikey" },
Published = true
};
}
public class Article
{
public int Id { get; set;}
public string Title{ get; set;}
public string Summary { get; set;}
public string Body { get; set;}
public DateTime ArticleDate { get; set; }
public Author Author { get; set; }
public bool Published { get; set;}
}
public class Author
{
public int Id { get; set; }
public string Name { get; set;}
}
results in
Documents from search: 4. Expect 4
Documents from search 3. Expect 3
Documents from search 2. Expect 2
Documents from search 0. Expect 0
as expected.
Something to bear in mind is that Elasticsearch is eventually consistent meaning that a document that is indexed does not appear in search results until after a refresh interval (by default, 1 second); Likewise, with a delete query, a document marked for deletion will appear in search results until the refresh interval has elapsed.
A GET request on a given document with a given id will however return the document before the refresh interval.
If you need documents to be searchable (or to not show in search results after deletion), you can refresh the index after an operation as I did with the bulk and delete calls above, using .Refresh(). You might be tempted to call refresh after every operation, however I would recommend using it only when you really need to as it adds overhead to the cluster and called all the time would likely degrade performance.

I got it working finally.
The delete request which is sent via NEST in fiddler is DELETE /articlestest/article/_query and the query which worked in plugin is this DELETE /articlestest/articles/_query ( the document type name was misspelled in the code).That's the reason ,query was not deleting the documents via NEST.And the bad thing is ,it doesn't even complain about the non-existent document type :( It took me a while to found that issue.

Related

HotChocolate (GraphQL) schema first approach on complex type

I'm novice in HotChocolate and I'm trying to PoC some simple usage.
I've created very simple .graphql file:
#camera.graphql
type Camera {
id: ID!
name: String!
}
type Query {
getCamera: Camera!
}
And a very simple .NET code for camera wrapping:
public class QlCamera
{
public static QlCamera New()
{
return new QlCamera
{
Id = Guid.NewGuid().ToString(),
Name = Guid.NewGuid().ToString()
};
}
public string Id { get; set; }
public string Name { get; set; }
}
as well as such for schema creation:
public void CreateSchema()
{
string path = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var smBuilder = SchemaBuilder.New();
smBuilder.AddDocumentFromFile(path + "/GraphQL/camera.graphql");
smBuilder.AddResolver("Query", "getCamera", () => QlCamera.New());
var schema = smBuilder.Create();
}
On the last line however I do get an exception :
HotChocolate.SchemaException: 'Multiple schema errors occured:
The field Camera.id has no resolver. - Type: Camera
The field Camera.name has no resolver. - Type: Camera
'
I've tried to create :
public class QlCameraType : ObjectType<QlCamera>
{
protected override void Configure(IObjectTypeDescriptor<QlCamera> descriptor)
{
descriptor.Name("Camera");
descriptor.Field(t => t.Id).Type<NonNullType<StringType>>();
descriptor.Field(t => t.Name).Type<StringType>();
}
}
and to replace
smBuilder.AddResolver("Query", "getCamera", () => QlCamera.New());
with
smBuilder.AddResolver("Query", "getCamera", () => new QlCameraType());
But I continue to get the same exception.
Obviously I miss something here, But I cannot understand what exactly.
Could someone explain me what I do miss ?
(I've passed few times trough the documentation, but I cannot find relevant help there)
As exception clearly states - there are no revolvers bind for the particular fields ("id" and "name") of the "Camera" type/object.
So they just have to be added with :
smBuilder.AddResolver("Camera", "id", rc => rc.Parent<QlCamera>().Id);
smBuilder.AddResolver("Camera", "name", rc => rc.Parent<QlCamera>().Name);
And that is it.

Abstract object not mapped correctly in Elasticsearch using Nest 7.0.0-alpha1

I am using NEST (.NET 4.8) to import my data, and I have a problem getting the mapping to work in NEST 7.0.0-alpha1.
I have the following class structure:
class LinkActor
{
public Actor Actor { get; set; }
}
abstract class Actor
{
public string Description { get; set; }
}
class Person : Actor
{
public string Name { get; set; }
}
I connect to Elasticsearch this way:
var connectionSettings = new ConnectionSettings(new Uri(connection));
connectionSettings.DefaultIndex(indexName);
var client = new ElasticClient(connectionSettings);
The actual data looks like this:
var personActor = new Person
{
Description = "Description",
Name = "Name"
};
var linkActor = new LinkActor
{
Actor = personActor
};
And the data is indexed like this:
result = client.IndexDocument(linkActor);
Using NEST 6.6 I am getting the following data in Elasticsearch 6.5.2:
"actor": {
"name": "Name",
"description": "Description"
}
However when using NEST 7.0.0-alpha1 I get the following data in Elasticsearch 7.0.0:
"actor": {
"description": "Description"
}
So the data from the concrete class is missing. I am obviously missing / not understanding some new mapping feature, but my attempts with AutoMap has failed:
client.Map<(attempt with each of the above classes)>(m => m.AutoMap());
Is is still possible to map the data from the concrete class in NEST 7.0.0-alpha1?
I found a workaround using the NEST.JsonNetSerializer (remember to install this), which allows me to pass a JObject directly:
Connect to Elasticsearch using a pool so you can add the JsonNetSerializer.Default:
var pool = new SingleNodeConnectionPool(new Uri(connection));
var connectionSettings = new ConnectionSettings(pool, JsonNetSerializer.Default);
connectionSettings.DefaultIndex(indexName);
var client = new ElasticClient(connectionSettings);
Convert the linkActor object from above to a JObject (JsonSerializerSettings omitted for clarity, add them to get CamelCasing):
var linkActorSerialized = JsonConvert.SerializeObject(linkActor);
var linkActorJObject = JObject.Parse(linkActorSerialized);
result = client.IndexDocument(linkActorJObject);
This gives the desired result:
"actor": {
"name": "Name",
"description": "Description"
}
It is a workaround, hopefully someone will be able to explain the mapping in the question.

Nest Elasticsearch- searching front of some fields and back of others

I have a case where I need a partial match on the first part of some properties (last name and first name) and a partial match on the end of some other properties, and I'm wondering how to add both analyzers.
For example, if I have the first name of "elastic", I can currently search for "elas" and find it. But, if I have an account number of abc12345678, I need to search for "5678" and find all account numbers ending in that, but I can't have a first name search for "stic" find "elastic".
Here's a simplified example of my Person class:
public class Person
{
public string AccountNumber { get; set; }
[ElasticProperty(IndexAnalyzer = "partial_name", SearchAnalyzer = "full_name")]
public string LastName { get; set; }
[ElasticProperty(IndexAnalyzer = "partial_name", SearchAnalyzer = "full_name")]
public string FirstName { get; set; }
}
Here's the relevant existing code where I create the index, that currently works great for searching the beginning of a word:
//Set up analyzers on some fields to allow partial, case-insensitive searches.
var partialName = new CustomAnalyzer
{
Filter = new List<string> { "lowercase", "name_ngrams", "standard", "asciifolding" },
Tokenizer = "standard"
};
var fullName = new CustomAnalyzer
{
Filter = new List<string> { "standard", "lowercase", "asciifolding" },
Tokenizer = "standard"
};
var result = client.CreateIndex("persons", c => c
.Analysis(descriptor => descriptor
.TokenFilters(bases => bases.Add("name_ngrams", new EdgeNGramTokenFilter
{
MaxGram = 15, //Allow partial match up to 15 characters.
MinGram = 2, //Allow no smaller than 2 characters match
Side = "front"
}))
.Analyzers(bases => bases
.Add("partial_name", partialName)
.Add("full_name", fullName))
)
.AddMapping<Person>((m => m.MapFromAttributes()))
);
It seems like I could add another EdgeNGramTokenFilter, and make the Side = "back", but I don't want the first and last name searches to match back side searches. Can someone provide a way to do that?
Thanks,
Adrian
Edit
For completeness, this is the new decorator on the property that goes with the code in the accepted answer:
[ElasticProperty(IndexAnalyzer = "partial_back", SearchAnalyzer = "full_name")]
public string AccountNumber { get; set; }
You need to declare another analyzer (let's call it partialBack) specifically for matching from the back but you can definitely reuse the existing edgeNGram token filter, like this:
var partialBack = new CustomAnalyzer
{
Filter = new List<string> { "lowercase", "reverse", "name_ngrams", "reverse" },
Tokenizer = "keyword"
};
...
.Analyzers(bases => bases
.Add("partial_name", partialName)
.Add("partial_back", partialBack))
.Add("full_name", fullName))
)
The key here is the double use of the reverse token filter.
The string (abc12345678) is
first lowercased (abc12345678),
then reversed (87654321cba),
then edge-ngramed (87, 876, 8765, 87654, 876543, ...)
and finally the tokens are reversed again (78, 678, 5678, 45678, 345678, ...).
As you can see, the result is that the string is tokenized "from the back", so that a search for 5678 would match abc12345678.

load specific fields in Elasticsearch Nest query

the documentation seems to indicate i can return a subset of fields instead of the entire document. here's my code:
var result = client.Search<MyObject>(s => s
.Fields(f => f.Title)
.Query(q => q
.QueryString(qs => qs
.OnField("title")
.Query("the"))));
i'm searching on the word 'the' on the 'title' field and wanting to just return 'title'. my result.Documents object contains 10 objects that are each null.
i do see the values i want but it's deep in the search response:
result.Hits[0].Fields.FieldValues[0]...
is there a better way to get at the list of 'title' fields returned?
my mapping for the data (truncated) is this ...
{
"myidex": {
"mappings": {
"myobject": {
"properties": {
"title": {
"type": "string"
},
"artists": {
"properties": {
"id": {
"type": "string",
"index": "not_analyzed",
"analyzer": "fullTerm"
},
"name": {
"type": "string",
"index": "not_analyzed",
"analyzer": "fullTerm"
}
}
}
}
}
}
}
}
and my class objects are like this:
[Table("MyTable")]
[Serializable]
[ElasticType(Name="myobject")]
public class MyObject
{
[ElasticProperty]
public string Title { get; set; }
[JsonIgnore]
public string Artistslist { get; set; }
[ElasticProperty(Analyzer = "caseInsensitive")]
public List<Person> Artists { get; set; }
}
[Serializable]
public class Person
{
[ElasticProperty(Analyzer = "fullTerm", Index = FieldIndexOption.not_analyzed)]
public string Name { get; set; }
[ElasticProperty(Analyzer = "fullTerm", Index = FieldIndexOption.not_analyzed)]
public string Id { get; set; }
}
Artistslist comes from my data source (sql) then i parse it out into a new List object before indexing the data.
I think this deeply nested value is do to a change in Elasticsearch 1.0 and how partial fields are now returned as arrays (See 1.0 Breaking Changes - Return Values for details.). This is addressed in the NEST 1.0 Breaking Changes documentation; in the Fields() vs SourceIncludes() section. It shows an example of using a FieldValue helper method to gain access to these values. Based on that, try the following:
For all items:
foreach (var hit in result.Hits)
{
var title = hit.Fields.FieldValue<MyObject, string>(f => f.Title);
}
For a specific item:
var title = result.Hits.ElementAt(0)
.Fields.FieldValue<MyObject, string>(f => f.Title);
I know it is still a bit verbose but it should work for you and will handle the new array return formatting of Elasticsearch 1.0.
I found the solution in Nest's Github repo. They have created an issue about this problem. You should use FielddataFields instead of Fields.
https://github.com/elastic/elasticsearch-net/issues/1551
var result = client.Search<MyObject>(s => s
.FielddataFields(f => f.Title)
.Query(q => q
.QueryString(qs => qs
.OnField("title")
.Query("the"))));
and in response you see FieldSelections. You get the fields that you wanted.

Sorting Groups to display them by date of containing element (LINQ)

I'm not sure if this is possible.
My class I have a list of looks like this:
class Person
{
string Firstname
string Lastname
DateTime Timestamp
}
Now I would like to create groups by Firstname and Lastname.
John Deer, 3:12
John Deer, 6:34
John Deer, 11:12
Tom Kin, 1:12
Tom Kin, 3:49
Tom Kin, 4:22
Markus Fert, 11:23
Further more I would like to sort this groups by their Timestamp, the last should be first while the groups should stay to display them in a listView.
Markus Fert (Group Header)
11:23 (Content Element)
John Deer
11:12
6:34
Tom Kin
4:22
3:49
John Deer
3:12
Tom Kin
1:22
Hope any Linq genius can help me solving the problem :)
Thanks!!
Much Thanks to Sergey, worked like a charm!
Further I would like to create a custom Class for my group Key to display different additional things in my ListView headers. (not only a spliced together string)
I would like to assign my query to an IEnumerable like this:
IEnumerable<IGrouping<Header, Person>> PersonGroups
Where the header contains some other properties contained in each Person (e.g. there is also a Country, Age,... for each Person). Maybe you can help me there too?
Thanks again Sergey. Solved my problem by implementing an Header class which implements the ICompareable interface.
IEnumerable<IGrouping<Header, Person>> PersonGroups
public class Header: IComparable<Header>
{
public Header(string firstname, string lastname)
{
Firstname= firstname;
Lastname = lastname;
}
public string Firstname{ get; set; }
public string Lastname{ get; set; }
public int CompareTo(Header that)
{
if (this.Firstname == that.Firstname&& this.Lastname == that.Lastname)
return 0;
else
return -1;
}
}
My query now looks like this:
PersonGroups= persons.OrderByDescending(p => p.Timestamp)
.GroupConsecutive(p => new Header(p.Firstname, p.Lastname));
Actually you need to order results by timestamp first. And only then group this ordered sequence by consecutive people:
var query =
people.OrderByDescending(p => p.Timestamp.TimeOfDay)
.GroupConsecutive(p => String.Format("{0} {1}", p.Firstname, p.Lastname))
.Select(g => new {
Header = g.Key,
Content = String.Join("\n", g.Select(p => p.Timestamp.TimeOfDay))
});
You will need GroupConsecutive implementation, which creates groups of consecutive items based on same value of provided selector (full name in your case).
For your sample input result is:
[
{
"Header": "Markus Fert",
"Content": "11:23:00"
},
{
"Header": "John Deer",
"Content": "11:12:00\n06:34:00"
},
{
"Header": "Tom Kin",
"Content": "04:22:00\n03:49:00"
},
{
"Header": "John Deer",
"Content": "03:12:00"
},
{
"Header": "Tom Kin",
"Content": "01:12:00"
}
]
Here's an approach using the built-in link operator Aggregate.
First I need to order the list by descending timestamp and then I created a name formatter function.
var op = people
.OrderByDescending(p => p.Timestamp)
.ToArray();
Func<Person, string> toName = p =>
String.Format("{0} {1}", p.Firstname, p.Lastname);
Now I can build the query:
var query =
op
.Skip(1)
.Aggregate(new []
{
new
{
Name = toName(op.First()),
Timestamps = new List<string>()
{
op.First().Timestamp.ToShortTimeString(),
},
},
}.ToList(), (a, p) =>
{
var name = toName(p);
if (name == a.Last().Name)
{
a.Last().Timestamps.Add(p.Timestamp.ToShortTimeString());
}
else
{
a.Add(new
{
Name = name,
Timestamps = new List<string>()
{
p.Timestamp.ToShortTimeString(),
},
});
}
return a;
});
I got this result:

Resources