ElasticSearch Nest 2.x Indexing and Searching Nested Objects - elasticsearch

I'm having trouble figuring out how to index and search nested object.
I want to be able to search nested objects and return the parents - only the parents, without the list of Remarks, but I would like highlights from the remarks returned if possible.
My models:
[DataContract]
[ElasticsearchType(IdProperty = "CustomerId", Name = "CustomerSearchResult")]
public class SearchResult
{
[DataMember]
[String(Index = FieldIndexOption.NotAnalyzed)]
public int CustomerId { get; set; }
...
[Nested]
[DataMember]
public List<RemarkForSearch> Remarks { get; set; }
}
[ElasticsearchType(IdProperty = "RemarkId", Name = "RemarkForSearch")]
public class RemarkForSearch
{
[DataMember]
public int RemarkId { get; set; }
[DataMember]
public int CustomerId { get; set; }
[DataMember]
public string RemarkText { get; set; }
}
Index creation:
var customerSearchIdxDesc = new CreateIndexDescriptor(Constants.ElasticSearch.CustomerSearchIndexName)
.Settings(f =>
f.Analysis(analysis => analysis
.CharFilters(cf => cf
.PatternReplace(Constants.ElasticSearch.FilterNames.RemoveNonAlphaNumeric, pr => pr
.Pattern(#"[^a-zA-Z\d]") // match all non alpha numeric
.Replacement(string.Empty)
)
)
.TokenFilters(tf => tf
.NGram(Constants.ElasticSearch.FilterNames.NGramFilter, fs => fs
.MinGram(1)
.MaxGram(20)
)
)
.Analyzers(analyzers => analyzers
.Custom(Constants.ElasticSearch.AnalyzerNames.NGramAnalyzer, a => a
.Filters("lowercase", "asciifolding", Constants.ElasticSearch.FilterNames.NGramFilter)
.Tokenizer(Constants.ElasticSearch.TokenizerNames.WhitespaceTokenizer)
)
.Custom(Constants.ElasticSearch.AnalyzerNames.WhitespaceAnalyzer, a => a
.Filters("lowercase", "asciifolding")
.Tokenizer(Constants.ElasticSearch.TokenizerNames.WhitespaceTokenizer)
)
.Custom(Constants.ElasticSearch.AnalyzerNames.FuzzyAnalyzer, a => a
.Filters("lowercase", "asciifolding")
//.CharFilters(Constants.ElasticSearch.FilterNames.RemoveNonAlphaNumeric)
.Tokenizer(Constants.ElasticSearch.TokenizerNames.NGramTokenizer)
)
)
.Tokenizers(tokenizers => tokenizers
.NGram(Constants.ElasticSearch.TokenizerNames.NGramTokenizer, t => t
.MinGram(1)
.MaxGram(20)
//.TokenChars(TokenChar.Letter, TokenChar.Digit)
)
.Whitespace(Constants.ElasticSearch.TokenizerNames.WhitespaceTokenizer)
)
)
)
.Mappings(ms => ms
.Map<ServiceModel.DtoTypes.Customer.SearchResult>(m => m
.AutoMap()
.AllField(s => s
.Analyzer(Constants.ElasticSearch.AnalyzerNames.NGramAnalyzer)
.SearchAnalyzer(Constants.ElasticSearch.AnalyzerNames.WhitespaceAnalyzer)
)
.Properties(p => p
.String(n => n
.Name(c => c.ContactName)
.Index(FieldIndexOption.NotAnalyzed)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.String(n => n
.Name(c => c.CustomerName)
.Index(FieldIndexOption.NotAnalyzed)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.String(n => n
.Name(c => c.City)
.Index(FieldIndexOption.NotAnalyzed)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.String(n => n
.Name(c => c.StateAbbreviation)
.Index(FieldIndexOption.NotAnalyzed)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.String(n => n
.Name(c => c.PostalCode)
.Index(FieldIndexOption.NotAnalyzed)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.String(n => n
.Name(c => c.Country)
.Index(FieldIndexOption.NotAnalyzed)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.Number(n => n
.Name(c => c.AverageMonthlySales)
.Type(NumberType.Double)
.CopyTo(fs => fs.Field(Constants.ElasticSearch.CombinedSearchFieldName))
)
.String(n => n
.Name(Constants.ElasticSearch.CombinedSearchFieldName)
.Index(FieldIndexOption.Analyzed)
.Analyzer(Constants.ElasticSearch.AnalyzerNames.FuzzyAnalyzer)
.SearchAnalyzer(Constants.ElasticSearch.AnalyzerNames.FuzzyAnalyzer)
)
.Nested<ServiceModel.DtoTypes.Customer.RemarkForSearch>(s => s
.Name(n => n.Remarks)
.AutoMap()
)
)
)
);
var response = client.CreateIndex(customerSearchIdxDesc);
Loading the index:
var searchResults = Db.SqlList<DtoTypes.Customer.SearchResult>("EXEC [Customer].[RetrieveAllForSearch]");
var remarkResults = Db.SqlList<DtoTypes.Customer.RemarkForSearch>("EXEC [Customer].[RetrieveAllSearchableRemarks]");
foreach(var i in searchResults)
{
i.Remarks = remarkResults.Where(m => m.CustomerId == i.CustomerId).ToList();
}
var settings = new ConnectionSettings(Constants.ElasticSearch.Node);
var client = new ElasticClient(settings);
// Flush the index
var flushResponse = client.Flush(Constants.ElasticSearch.CustomerSearchIndexName);
// Refresh index
var indexResponse = client.IndexMany(searchResults, Constants.ElasticSearch.CustomerSearchIndexName);
Querying the Index:
var searchDescriptor = new SearchDescriptor<DtoTypes.Customer.SearchResult>()
.From(0)
.Take(Constants.ElasticSearch.MaxResults)
.Query(q => q
.Nested(c => c
.Path(p => p.Remarks)
.Query(nq => nq
.Match(m => m
.Query(query)
.Field("remarks.remarktext")
)
)
)
);
response = client.Search<DtoTypes.Customer.SearchResult>(searchDescriptor);
I don't know if I'm bulk loading the index properly and if its smart enough to know that the Remarks property is a nested property and to load those as well.
The search has no errors, but I get no results.
The search query is generating this json, which from what I can tell is OK:
{
"from": 0,
"size": 100,
"query": {
"nested": {
"query": {
"match": {
"remarks.remarktext": {
"query": "test"
}
}
},
"path": "remarks"
}
}
}
I do see the remark data when looking at json using a query string http://127.0.0.1:9200/customersearch/_search

I want to be able to search nested objects and return the parents -
only the parents, without the list of Remarks, but I would like
highlights from the remarks returned if possible.
What about this idea. Let's exclude nested object from source but leave highlight on nested field in place. What I mean.
public class Document
{
public int Id { get; set; }
[Nested]
public Nested Nested { get; set; }
}
var createIndexResponse = client.CreateIndex(indexName, descriptor => descriptor
.Mappings(map => map
.Map<Document>(m => m
.AutoMap()
)));
var items = new List<Document>
{
new Document
{
Id = 1,
Nested = new Nested {Name = "Robert" }
},
new Document
{
Id = 2,
Nested = new Nested {Name = "Someone" }
}
};
var bulkResponse = client.IndexMany(items);
client.Refresh(indexName);
var searchResponse = client.Search<Document>(s => s
.Source(so => so.Exclude(e => e.Field(f => f.Nested)))
.Highlight(h => h.Fields(f => f.Field("nested.name")).PostTags("<b>").PreTags("<b>"))
.Query(q => q
.Nested(n => n
.Path(p => p.Nested)
.Query(nq => nq.Match(m => m
.Query("Robert").Field("nested.name"))))));
And what elasticsearch returns is
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [{
"_index" : "my_index",
"_type" : "document",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"id" : 1
},
"highlight" : {
"nested.name" : ["<a>Robert<a>"]
}
}
]
}
}
What do you think?

Related

NEST API Default value for GeoShapes fields

We are using a filter as per following:
filters.Add(fq => fq
.Term(t => t
.Field(f => f.LocalityId)
.Value(locationParams[2])) || fq
.GeoShape(g => g
.Field("locationShape")
.Relation(GeoShapeRelation.Within)
.IndexedShape(f => f
.Id(searchCriteria.spLocationId)
.Index(indexName)
.Path("geometry")
)
)
);
However, if the geometry field is missing, Elasticsearch throws an exception.
Is there anyway to avoid this by using a default (Null Value) in the mapping or any other way.
It is not possible to avoid the exception in this case. Elasticsearch assumes that the parameters that the user provides to a pre-indexed shape are valid.
Ideally, the values supplied to the indexed shape should be constrained in a manner that prevents an end user from supplying invalid values. If that is unfeasible, you could run a bool query with filter clauses of exists query and ids query on the indexName index before adding the indexed shape geoshape query filter to the search request.
For example
private static void Main()
{
var documentsIndex = "documents";
var shapesIndex = "shapes";
var host = "localhost";
var pool = new SingleNodeConnectionPool(new Uri($"http://{host}:9200"));
var settings = new ConnectionSettings(pool)
.DefaultMappingFor<Document>(m => m.IndexName(documentsIndex))
.DefaultMappingFor<Shape>(m => m.IndexName(shapesIndex));
var client = new ElasticClient(settings);
if (client.Indices.Exists(documentsIndex).Exists)
client.Indices.Delete(documentsIndex);
client.Indices.Create(documentsIndex, c => c
.Map<Document>(m => m
.AutoMap()
)
);
if (client.Indices.Exists(shapesIndex).Exists)
client.Indices.Delete(shapesIndex);
client.Indices.Create(shapesIndex, c => c
.Map<Shape>(m => m
.AutoMap()
)
);
client.Bulk(b => b
.IndexMany(new [] {
new Document
{
Id = 1,
LocalityId = 1,
LocationShape = GeoWKTReader.Read("POLYGON ((30 20, 20 15, 20 25, 30 20))")
},
new Document
{
Id = 2,
LocalityId = 2
},
})
.IndexMany(new []
{
new Shape
{
Id = 1,
Geometry = GeoWKTReader.Read("POLYGON ((20 35, 10 30, 10 10, 30 5, 45 20, 20 35))")
}
})
.Refresh(Refresh.WaitFor)
);
var shapeId = 1;
var searchResponse = client.Search<Shape>(s => s
.Size(0)
.Query(q => +q
.Ids(i => i.Values(shapeId)) && +q
.Exists(e => e.Field("geometry"))
)
);
Func<QueryContainerDescriptor<Document>, QueryContainer> geoShapeQuery = q => q;
if (searchResponse.Total == 1)
geoShapeQuery = q => +q
.GeoShape(g => g
.Field("locationShape")
.Relation(GeoShapeRelation.Within)
.IndexedShape(f => f
.Id(shapeId)
.Index(shapesIndex)
.Path("geometry")
)
);
client.Search<Document>(s => s
.Query(q => +q
.Term(t => t
.Field(f => f.LocalityId)
.Value(2)
) || geoShapeQuery(q)
)
);
}
public class Document
{
public int Id { get; set; }
public int LocalityId { get; set; }
public IGeoShape LocationShape { get; set; }
}
public class Shape
{
public int Id { get; set; }
public IGeoShape Geometry { get; set; }
}
If var shapeId = 1; is changed to var shapeId = 2; then the geoshape query is not added to the filter clauses when searching on the documents index.

Not understanding the behavior of _id on ES

I have documents in a Mongo database, they use the _id field as an index.
I use Monstache to sync ES with Mongo's op log, so the documents in ES have the same _id field.
When searching a specific document, Kibana shows:
Tags:
tag1 testtag CreatedOn:
October 26th 2018, 14:25:57.053
_id:
FRaqDPIzWcVI2dl-oA9uUFHLVFQk8qIqqhySWSkM7Ds
_type:
testobject
_index:
test.object
_score:
0
but then a query with Nest, returns this in the Documents array:
_id = 0d5aa177-3066-4c6a-aaf5-9b887ae7297f
and when I look in the Hits array, I see:
Id = FRaqDPIzWcVI2dl-oA9uUFHLVFQk8qIqqhySWSkM7Ds
So in Documents, the _id is now an unrelated guid, but in Hits the _id is called Id and it has the right value.
Why is that, and is there a way to get the proper value for _id in Documents?
Edit: more info
This is the object; since it's shared by MongoDB and ES, it has attributes for both.
[Nest.ElasticsearchType, BsonIgnoreExtraElements]
public class TestObject
{
public string _id { get; set; }
public string OwnerId { get; set; }
public Flags Flags { get; set; }
[Nest.Text, BsonIgnoreIfDefault] public string Title { get; set; }
[Nest.Text] public string Tags { get; set; }
[Nest.Ignore] public string Hash { get; set; }
[Nest.Ignore, BsonIgnoreIfDefault] public string Link { get; set; }
}
This is the code creating the index:
private static void InitializeElasticSearch(string ConnectionString)
{
var Settings = new ConnectionSettings(new Uri(ConnectionString))
.DefaultIndex(_IndexName)
.DefaultFieldNameInferrer(_ => _)
.DefaultMappingFor<TestObject>(_ => _.Ignore(I => I._id));
_ElasticClient = new ElasticClient(Settings);
if (!_ElasticClient.IndexExists(_IndexName).Exists)
{
// create the index
var CreateIndexResponse = _ElasticClient.CreateIndex(_IndexName, C => C
.Settings(S => S
.Analysis(A => A
.CharFilters(Cf => Cf
.Mapping("expressions", E => E
.Mappings(TextLists.Expressions)
)
)
.TokenFilters(Tf => Tf
.Synonym("synonyms", Sy => Sy
.Synonyms(TextLists.Synonyms)
.Tokenizer("whitespace")
)
)
.Analyzers(An => An
.Custom("index", Ca => Ca
.CharFilters("expressions")
.Tokenizer("standard")
.Filters("standard", "synonyms", "stop")
)
)
)
)
.Mappings(M => M
.Map<TestObject>(Mm => Mm
.AutoMap()
.Properties(P => P
.Text(T => T
.Name(N => N.Title)
.Analyzer("index")
)
.Text(T => T
.Name(N => N.Tags)
.Analyzer("index")
)
)
)
)
);
Then, the query code:
var R = await _ElasticClient.SearchAsync<TestObject>(Sr => Sr
.Query(Q =>
{
// do we query 'all' ?
if (Terms == "*") return Q.MatchAll();
// or do we have a general query
return Q
.MultiMatch(Fu => Fu
.Fields(F => F
.Field(Ff => Ff.Tags)
.Field(Ff => Ff.Title)
)
.Query(Terms)
.Fuzziness(Fuzziness.EditDistance(2))
);
})
.Take(_MaxObjectReturned)
);

Elasticsearch NEST Library

We need to create facets in a field which is an array of int [1,2,3,4,5,6].
If we map this field as Text, Elasticsearch throws an exception. If we mark it as Keyword we don't get all the individual items in the facet.
How can this be achieved with the NEST library (version 5)
Many thanks
Map it as an integer. NEST's automapping will infer an integer mapping for it by default
Given the following type
public class Message
{
public int[] Integers { get; set; }
}
Automapping would look like
var client = new ElasticClient();
client.CreateIndex("messages", c => c
.Mappings(m => m
.Map<Message>(mm => mm
.AutoMap()
)
)
);
which produces the following mapping
{
"mappings": {
"message": {
"properties": {
"integers": {
"type": "integer"
}
}
}
}
}
You can use attribute mapping if you want to explicitly control the mapping
public class Message
{
[Number(NumberType.Integer)]
public int[] Integers { get; set; }
}
var client = new ElasticClient();
client.CreateIndex("messages", c => c
.Mappings(m => m
.Map<Message>(mm => mm
.AutoMap()
)
)
);
Or fluent mapping if you prefer to keep mapping separate from your POCOs
client.CreateIndex(messagesIndex, c => c
.Mappings(m => m
.Map<Message>(mm => mm
.AutoMap()
.Properties(p => p
.Number(n => n
.Name(nn => nn.Integers)
.Type(NumberType.Integer)
)
)
)
)
);
or using the scalar method
client.CreateIndex(messagesIndex, c => c
.Mappings(m => m
.Map<Message>(mm => mm
.AutoMap()
.Properties(p => p
.Scalar(n => n.Integers)
)
)
)
);
With the mapping in place, faceting on integer values would look like
client.IndexMany(new[] {
new Message { Integers = new [] { 1, 2, 3 } },
new Message { Integers = new [] { 3, 4, 5 } },
new Message { Integers = new [] { 1, 4, 5 } },
});
client.Refresh(messagesIndex);
var searchResponse = client.Search<Message>(s => s
.Size(0)
.Aggregations(a => a
.Terms("ints", t => t
.Field(f => f.Integers)
)
)
);
foreach (var bucket in searchResponse.Aggs.Terms<int>("ints").Buckets)
{
Console.WriteLine($"{bucket.Key}:{bucket.DocCount}");
}
which yields
1:2
3:2
4:2
5:2
2:1

How to index files with Elasticsearch, Ingest-Mapper and NEST

actually I am trying to index some files (most important PDFs) with Elasticsearch and NEST client from ASP.net Core 1.0.
I found some code snippets and tried to use it for my purpose.
Those are the code segments that I use to set up the Elasticsearch index and the Pipeline:
private void SetupElasticSearch()
{
ConnectionSettings settings = new ConnectionSettings(new Uri(_appSettings.ElasticSearchSettings.Url))
.MapDefaultTypeIndices(m => m
.Add(typeof(FSDocumentFile), _appSettings.ElasticSearchSettings.IndexName)
).OnRequestCompleted(response =>
{
_logger.LogInformation(String.Format("{0} {1}", response.HttpMethod, response.Uri));
if (response.RequestBodyInBytes != null)
{
_logger.LogInformation(Encoding.UTF8.GetString(response.RequestBodyInBytes));
}
// log out the response and the response body, if one exists for the type of response
_logger.LogInformation(String.Format("{0}", response.HttpStatusCode));
if (response.ResponseBodyInBytes != null)
{
_logger.LogInformation(Encoding.UTF8.GetString(response.ResponseBodyInBytes));
}
});
ElasticClient client = new ElasticClient(settings);
CreateElasticSearchIndex(client);
CreatePipeline(client);
}
private void CreateElasticSearchIndex(ElasticClient Client)
{
Client.DeleteIndex(_appSettings.ElasticSearchSettings.IndexName);
Client.CreateIndex(_appSettings.ElasticSearchSettings.IndexName, c => c
.Settings(s => s
.Analysis(a => a
.Analyzers(ad => ad
.Custom("windows_path_hierarchy_analyzer", ca => ca
.Tokenizer("windows_path_hierarchy_tokenizer")
)
)
.Tokenizers(t => t
.PathHierarchy("windows_path_hierarchy_tokenizer", ph => ph
.Delimiter('\\')
)
)
)
)
.Mappings(m => m
.Map<FSDocumentFile>(mp => mp
.AllField(all => all
.Enabled(false)
)
.Properties(ps => ps
.Number(n => n
.Name(nn => nn.Id)
)
.Text(s => s
.Name(n => n.Comment)
//.Analyzer("windows_path_hierarchy_analyzer")
)
//.Text(s => s
// .Name(n => n.Content)
//)
.Object<Attachment>(a => a
.Name(n => n.FileData)
.Properties(p => p
.Text(t => t
.Name(n => n.Name)
)
.Text(t => t
.Name(n => n.Content)
)
.Text(t => t
.Name(n => n.ContentType)
)
.Number(n => n
.Name(nn => nn.ContentLength)
)
.Date(d => d
.Name(n => n.Date)
)
.Text(t => t
.Name(n => n.Author)
)
.Text(t => t
.Name(n => n.Title)
)
.Text(t => t
.Name(n => n.Keywords)
)
)
)
)
)
)
);
}
private void CreatePipeline(ElasticClient Client)
{
Client.PutPipeline("attachments", p => p
.Description("Document attachment pipeline")
.Processors(pr => pr
.Attachment<FSDocumentFile>(a => a
.Field(f => f.FileData.Content)
.TargetField(f => f.Content)
)
.Remove<FSDocumentFile>(r => r
.Field(f => f.FileData)
)
)
);
}
This is the definition of the the FSFileInfo classe that is used for indexing:
[ElasticsearchType(Name = "FSDocumentFile")]
public class FSDocumentFile
{
public int Id { get; set; }
/// <summary>
/// FileData Base64 encoded
/// </summary>
public string Content { get; set; }
[Attachment(Store = true)]
public Attachment FileData { get; set; }
public string Comment { get; set; }
}
This is the code that I use to index the file:
FSDocumentFile fsFile = new FSDocumentFile()
{
Id = df.DocumentFileID,
FileData = new Attachment()
{
Content = Convert.ToBase64String(fd.FileBytes),
ContentType = "application/pdf",
ContentLength = fd.FileBytes.Count(),
Name = fileName
},
Comment = "TEst Comment" + df.DocumentFileID.ToString()
};
ElasticClient client = new ElasticClient(settings);
Result callResult = client.Index<FSDocumentFile>(fsFile, fi => fi.Pipeline("attachments")).Result;
It always results in an error on Elasticsearch that says:
[2016-11-28T15:40:28,311][ERROR][o.e.a.i.IngestActionFilter] [mU3hlQ7] failed to execute pipeline [attachments]
org.elasticsearch.ElasticsearchException: java.lang.IllegalArgumentException: ElasticsearchParseException[Error parsing document in field [fileData.content]]; nested: IllegalArgumentException[field [c
ontent] not present as part of path [fileData.content]];
at org.elasticsearch.ingest.CompoundProcessor.newCompoundProcessorException(CompoundProcessor.java:156) ~[elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:107) ~[elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.ingest.Pipeline.execute(Pipeline.java:58) ~[elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.ingest.PipelineExecutionService.innerExecute(PipelineExecutionService.java:166) ~[elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.ingest.PipelineExecutionService.access$000(PipelineExecutionService.java:41) ~[elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.ingest.PipelineExecutionService$1.doRun(PipelineExecutionService.java:65) [elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0-rc1.jar:5.0.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_112]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_112]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_112]
Caused by: java.lang.IllegalArgumentException: ElasticsearchParseException[Error parsing document in field [fileData.content]]; nested: IllegalArgumentException[field [content] not present as part of
path [fileData.content]];
Can someone please assist me? Or maybe point me to a good tutorial how to use Elasticsearch, Ingest-Mapper and NEST client together?
Thanks a lot in advance.

Elasticsearch suggestion is merging result of two types

I have 2 Types in one index and both have Suggest field (as required )
public class LegalAreaSearchModel : LegalAreaModel
{
public SuggestField Suggest
{
get
{
List<string> input = new List<string>();
string[] childArea = !string.IsNullOrEmpty(this.LegalArea) ? this.LegalArea.Split(' ') : new string[] { "" };
string[] ParentArea = !string.IsNullOrEmpty(this.Parent) ? this.Parent.Split(' ') : new string[] { "" };
input.AddRange(childArea);
input.AddRange(ParentArea);
return
new SuggestField
{
Input = input,
Output = this.LegalArea,
Payload = new
{
Id = this.RowId,
Name = !string.IsNullOrEmpty(this.Parent) ? this.Parent + "/" + this.LegalArea : this.LegalArea,
Type = "Specialization"
},
Weight = !string.IsNullOrEmpty(this.LegalArea) ? this.LegalArea.Length : 0
};
}
}
}
AND
public class LegalDocumentSearchModel : LegalDocumentModel
{
public LegalDocumentSearchModel()
{
//this = ObjectCopier.Clone<LegalDocumentSearchModel>(legalDocumentSearchModel);
SupplierDetails = new List<LegalDocumentSupplierDetailForSearch>();
CategoryDetails = new List<CategoryDetails>();
}
[Nested()]
public List<LegalDocumentSupplierDetailForSearch> SupplierDetails { get; set; }
[Nested()]
public List<CategoryDetails> CategoryDetails { get; set; }
public SuggestField Suggest
{
get
{
return
new SuggestField
{
Input = new List<string>(!string.IsNullOrEmpty(this.Name) ? this.Name.Split(' ') : new string[] { "" }) { this.Name },
Output = this.Name,
Payload = new
{
Id = this.RowId,
SEOFriendlyURLName = !string.IsNullOrEmpty(this.SEOFriendlyURLName) ? string.Concat(this.SEOFriendlyURLName) : string.Empty,
Name = this.Name,
ProductType = this.ProductType,
Description = this.Description
},
Weight = !string.IsNullOrEmpty(this.Description) ? this.Description.Length : 0
};
}
}
}
public class LegalDocumentSupplierDetailForSearch
{
public string PinCode { get; set; }
public string Lattitude { get; set; }
public string Longitude { get; set; }
}
public class CategoryDetails
{
public int CategoryId { get; set; }
public string CategoryName { get; set; }
}
now my search is as follows
List<AdvocateListingSuggestionModel> lsResult = new List<AdvocateListingSuggestionModel>();
var result = _searchProvider.Client.Suggest<LegalAreaSearchModel>(s => s
.Index(SearchConfigurationManager.DefaultSearchIndex)
.Completion("ml-la-suggestions", c => c
.Text(Search)
.Field(p => p.Suggest)
.Fuzzy(fz => fz
.Fuzziness(Fuzziness.Auto))
)
);
I am getting result from other type as well(mix) of both how can we restrict result from only one type
Type1.LegalAreaSearchModel
Type 2.LegalDocumentSearchModel
my index creation is as follows
public static bool CheckForIndex(ElasticSearchProvider searchProvider)
{
Nest.IndexExistsRequest idr = new Nest.IndexExistsRequest(Nest.Indices.Index(new Nest.IndexName() { Name = SearchConfigurationManager.DefaultSearchIndex }));
var indxres = searchProvider.Client.IndexExists(idr);
if (!indxres.Exists)
{
searchProvider.Client.CreateIndex(SearchConfigurationManager.DefaultSearchIndex, i => i
.Settings(s => s
.NumberOfShards(2)
.NumberOfReplicas(0)
.Analysis(analysis => analysis
.Tokenizers(tokenizers => tokenizers
.Pattern("ml-id-tokenizer", p => p.Pattern(#"\W+"))
)
.TokenFilters(tokenfilters => tokenfilters
.WordDelimiter("ml-id-words", wd => wd
.SplitOnCaseChange()
.PreserveOriginal()
.SplitOnNumerics()
.GenerateNumberParts(false)
.GenerateWordParts()
)
)
.Analyzers(analyzers => analyzers
.Custom("ml-id-analyzer", c => c
.Tokenizer("ml-id-tokenizer")
.Filters("ml-id-words", "lowercase")
)
.Custom("ml-id-keyword", c => c
.Tokenizer("keyword")
.Filters("lowercase")
)
)
)
));
}
return true;
}
Creatining Document Type as follows
public bool CreateDocumentIndex()
{
bool retVal = false;
if (Common.CheckForIndex(_searchProvider))
{
var res = _searchProvider.Client.Map<LegalDocumentSearchModel>(m =>
m.Index(SearchConfigurationManager.DefaultSearchIndex)
.Type(this.type)
.AutoMap()
.Properties(ps => ps
.String(s => s
.Name(p => p.Id)
.Analyzer("ml-id-analyzer")
.Fields(f => f
.String(p => p.Name("keyword").Analyzer("ml-id-keyword"))
.String(p => p.Name("raw").Index(FieldIndexOption.NotAnalyzed))
)
)
.Completion(c => c
.Name(p => p.Suggest)
.Payloads()
)
));
retVal = res.IsValid;
}
return retVal;
}
creating LegalArea Type as follows
public bool CreateLegalAreaIndex()
{
bool retVal = false;
if (Common.CheckForIndex(_searchProvider))
{
var res = _searchProvider.Client.Map<LegalAreaSearchModel>(m =>
m.Index(SearchConfigurationManager.DefaultSearchIndex)
.Type(this.type)
.AutoMap()
.Properties(ps => ps
.String(s => s
.Name(p => p.Id)
.Analyzer("ml-id-analyzer")
.Fields(f => f
.String(p => p.Name("keyword").Analyzer("ml-id-keyword"))
.String(p => p.Name("raw").Index(FieldIndexOption.NotAnalyzed))
)
)
.Completion(c => c
.Name(p => p.Suggest)
.Payloads()
)
));
retVal = res.IsValid;
}
return retVal;
}
now when i am running legal area suggestions as follows
public List<AdvocateListingSuggestionModel> LegalAreaSuggestion(string Search)
{
List<AdvocateListingSuggestionModel> lsResult = new List<AdvocateListingSuggestionModel>();
var result = _searchProvider.Client.Suggest<LegalAreaSearchModel>(s => s
.Index(SearchConfigurationManager.DefaultSearchIndex)
.Completion("ml-la-suggestions", c => c
.Text(Search)
.Field(p => p.Suggest)
.Fuzzy(fz => fz
.Fuzziness(Fuzziness.Auto))
)
);
if (result.IsValid)
{
lsResult = result.Suggestions["ml-la-suggestions"]
.FirstOrDefault()
.Options
.Select(suggest => suggest.Payload<AdvocateListingSuggestionModel>()).ToList();
}
return lsResult;
}
I am getting results of LegalDocumentSearchModel as well.pls advise
Thanks
Suggesters work at the index level, that is, one Finite State Transducer (FST) is created for a field per index so if you have two types with the same field name in the same index (both fields must be of the same type), both types will have data represented in the one FST.
We can see this with a simple example. Here we have two types with completion fields that we're going to index into the same index
public class Band
{
public int Id { get; set;}
public CompletionField<object> Suggest { get; set;}
}
public class Sport
{
public int Id { get; set;}
public CompletionField<object> Suggest { get; set; }
}
void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var defaultIndex = "default-index";
var connectionSettings = new ConnectionSettings(pool)
.DefaultIndex(defaultIndex);
var client = new ElasticClient(connectionSettings);
// just so we can re-run this example...
if (client.IndexExists(defaultIndex).Exists)
client.DeleteIndex(defaultIndex);
client.CreateIndex(defaultIndex, ci => ci
.Mappings(m => m
.Map<Band>(l => l
.AutoMap()
.Properties(p => p
.Completion(c => c
.Name(n => n.Suggest)
)
)
)
.Map<Sport>(l => l
.AutoMap()
.Properties(p => p
.Completion(c => c
.Name(n => n.Suggest)
)
)
)
)
);
var bands = new List<Band>
{
new Band { Id = 1, Suggest = new CompletionField<object> { Input = new [] {"Bowling for Soup"} } },
new Band { Id = 2, Suggest = new CompletionField<object> { Input = new [] {"Fastball"} } },
new Band { Id = 3, Suggest = new CompletionField<object> { Input = new [] {"Dropkick Murphys"} } },
new Band { Id = 4, Suggest = new CompletionField<object> { Input = new [] {"Yellowcard"} } },
new Band { Id = 5, Suggest = new CompletionField<object> { Input = new [] {"American Football"} } },
};
client.IndexMany(bands);
var sports = new List<Sport>
{
new Sport { Id = 1, Suggest = new CompletionField<object> { Input = new [] {"Bowling"} } },
new Sport { Id = 2, Suggest = new CompletionField<object> { Input = new [] {"Football"} } },
new Sport { Id = 3, Suggest = new CompletionField<object> { Input = new [] {"Baseball"} } },
new Sport { Id = 4, Suggest = new CompletionField<object> { Input = new [] {"Table Tennis"} } },
new Sport { Id = 5, Suggest = new CompletionField<object> { Input = new [] {"American Football"} } },
};
client.IndexMany(sports);
client.Refresh(defaultIndex);
var suggestResponse = client.Suggest<Band>(s => s
.Completion("suggestion", cs => cs
.Text("Bo")
.Field(f => f.Suggest)
)
);
}
We get back the following from our suggest call
{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"suggestion" : [ {
"text" : "Bo",
"offset" : 0,
"length" : 2,
"options" : [ {
"text" : "Bowling",
"score" : 1.0
}, {
"text" : "Bowling for Soup",
"score" : 1.0
} ]
} ]
}
We get a suggestion of both a Band type and a Sport type in the results. Perhaps not what we expected or wanted, but that's how suggesters work. Note that in the .Suggest<T>() call, the type T is used to provide strongly typed access to properties that may be used in the query. In this example, the .Suggest field of type Band.
The recommended solution here is to have separate indices for different types; you can still query across multiple indices when you need to and avoid the pitfalls that can come with having multiple types in the same index.
With that aside, you can get this to work by using a Context Suggester and taking advantage of category context for the completion type mapping. Ideally, you want to use a field that exists within the document for the category to afford you flexibility in the category, but you can however use a metadata field if you want to. I recommend investigating and researching approaches for your domain.
With our previous example, if we now include category context in the mapping (same code as before but with this updated mapping)
client.CreateIndex(defaultIndex, ci => ci
.Mappings(m => m
.Map<Band>(l => l
.AutoMap()
.Properties(p => p
.Completion(c => c
.Name(n => n.Suggest)
.Context(sc => sc
.Category("type", csc => csc
// recommend that you use another field on
// the document here instead of a metadata field
.Field("_type")
)
)
)
)
)
.Map<Sport>(l => l
.AutoMap()
.Properties(p => p
.Completion(c => c
.Name(n => n.Suggest)
.Context(sc => sc
.Category("type", csc => csc
// recommend that you use another field on
// the document here instead of a metadata field
.Field("_type")
)
)
)
)
)
)
);
And use category context whilst searching
var suggestResponse = client.Suggest<Band>(s => s
.Completion("suggestion", cs => cs
.Text("Bo")
.Field(f => f.Suggest)
.Context(d => d.Add("type", "band"))
)
);
then we get the desired result
{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"suggestion" : [ {
"text" : "Bo",
"offset" : 0,
"length" : 2,
"options" : [ {
"text" : "Bowling for Soup",
"score" : 1.0
} ]
} ]
}

Resources