I'm using the NEST client with the following syntax:
_server.Search<Document>(s => s.Index(_config.SearchIndex)
.Query(q => q.MatchAll(p => p)).Aggregations(
a => a
.Terms("type", st => st
.Field(p => p.Type)
)));
However I keep getting the following exception
A first chance exception of type 'System.NotSupportedException' occurred in mscorlib.dll
Additional information: Collection is read-only.
It only seems to occur when I'm using aggregations, the field of Type has the following mapping:
[Keyword(Name = "Type")]
public string Type { get; set; }
I would double check the versions of NEST and Elasticsearch.Net that you are using. I just tried the following example with Elasticsearch 5.1.2 and NEST 5.0.1 and don't see the issue
void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var defaultIndex = "default-index";
var connectionSettings = new ConnectionSettings(pool)
.DefaultIndex(defaultIndex);
var client = new ElasticClient(connectionSettings);
if (client.IndexExists(defaultIndex).Exists)
client.DeleteIndex(defaultIndex);
client.CreateIndex(defaultIndex, c => c
.Mappings(m => m
.Map<Document>(mm => mm.AutoMap())
)
);
var documents = Enumerable.Range(1, 100).Select(i =>
new Document
{
Type = $"Type {i % 5}"
}
);
client.IndexMany(documents);
client.Refresh(defaultIndex);
var searchResponse = client.Search<Document>(s => s
.Index(defaultIndex)
.Query(q => q.MatchAll(p => p))
.Aggregations(a => a
.Terms("type", st => st
.Field(p => p.Type)
)
)
);
foreach (var bucket in searchResponse.Aggs.Terms("type").Buckets)
Console.WriteLine($"key: {bucket.Key}, count {bucket.DocCount}");
}
public class Document
{
[Keyword(Name = "Type")]
public string Type { get; set; }
}
this outputs
key: Type 0, count 20
key: Type 1, count 20
key: Type 2, count 20
key: Type 3, count 20
key: Type 4, count 20
Related
We are using a filter as per following:
filters.Add(fq => fq
.Term(t => t
.Field(f => f.LocalityId)
.Value(locationParams[2])) || fq
.GeoShape(g => g
.Field("locationShape")
.Relation(GeoShapeRelation.Within)
.IndexedShape(f => f
.Id(searchCriteria.spLocationId)
.Index(indexName)
.Path("geometry")
)
)
);
However, if the geometry field is missing, Elasticsearch throws an exception.
Is there anyway to avoid this by using a default (Null Value) in the mapping or any other way.
It is not possible to avoid the exception in this case. Elasticsearch assumes that the parameters that the user provides to a pre-indexed shape are valid.
Ideally, the values supplied to the indexed shape should be constrained in a manner that prevents an end user from supplying invalid values. If that is unfeasible, you could run a bool query with filter clauses of exists query and ids query on the indexName index before adding the indexed shape geoshape query filter to the search request.
For example
private static void Main()
{
var documentsIndex = "documents";
var shapesIndex = "shapes";
var host = "localhost";
var pool = new SingleNodeConnectionPool(new Uri($"http://{host}:9200"));
var settings = new ConnectionSettings(pool)
.DefaultMappingFor<Document>(m => m.IndexName(documentsIndex))
.DefaultMappingFor<Shape>(m => m.IndexName(shapesIndex));
var client = new ElasticClient(settings);
if (client.Indices.Exists(documentsIndex).Exists)
client.Indices.Delete(documentsIndex);
client.Indices.Create(documentsIndex, c => c
.Map<Document>(m => m
.AutoMap()
)
);
if (client.Indices.Exists(shapesIndex).Exists)
client.Indices.Delete(shapesIndex);
client.Indices.Create(shapesIndex, c => c
.Map<Shape>(m => m
.AutoMap()
)
);
client.Bulk(b => b
.IndexMany(new [] {
new Document
{
Id = 1,
LocalityId = 1,
LocationShape = GeoWKTReader.Read("POLYGON ((30 20, 20 15, 20 25, 30 20))")
},
new Document
{
Id = 2,
LocalityId = 2
},
})
.IndexMany(new []
{
new Shape
{
Id = 1,
Geometry = GeoWKTReader.Read("POLYGON ((20 35, 10 30, 10 10, 30 5, 45 20, 20 35))")
}
})
.Refresh(Refresh.WaitFor)
);
var shapeId = 1;
var searchResponse = client.Search<Shape>(s => s
.Size(0)
.Query(q => +q
.Ids(i => i.Values(shapeId)) && +q
.Exists(e => e.Field("geometry"))
)
);
Func<QueryContainerDescriptor<Document>, QueryContainer> geoShapeQuery = q => q;
if (searchResponse.Total == 1)
geoShapeQuery = q => +q
.GeoShape(g => g
.Field("locationShape")
.Relation(GeoShapeRelation.Within)
.IndexedShape(f => f
.Id(shapeId)
.Index(shapesIndex)
.Path("geometry")
)
);
client.Search<Document>(s => s
.Query(q => +q
.Term(t => t
.Field(f => f.LocalityId)
.Value(2)
) || geoShapeQuery(q)
)
);
}
public class Document
{
public int Id { get; set; }
public int LocalityId { get; set; }
public IGeoShape LocationShape { get; set; }
}
public class Shape
{
public int Id { get; set; }
public IGeoShape Geometry { get; set; }
}
If var shapeId = 1; is changed to var shapeId = 2; then the geoshape query is not added to the filter clauses when searching on the documents index.
I want to use the UpdateByQuery method on the high level client but can't find any documentation for Nest. They have great documentation if I wanted to make a CURL request but nothing for NEST. https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html
If anyone has and example of them using it or can share documentation they have found that would be awesome!
Update By Query API is supported in NEST. Here's an example adapted from the integration tests. NEST Documentation for Index and Update APIs is planned :)
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool)
.DefaultMappingFor<Test>(m => m
.IndexName("tests")
.TypeName("test")
);
var client = new ElasticClient(settings);
var index = IndexName.From<Test>();
if (client.IndexExists(index).Exists)
client.DeleteIndex(index);
client.CreateIndex(index, c => c
.Mappings(m => m
.Map<Test>(map => map
.Properties(props => props
.Text(s => s.Name(p => p.Text))
.Keyword(s => s.Name(p => p.Flag))
)
)
)
);
client.Bulk(b => b
.IndexMany(new[] {
new Test { Text = "words words", Flag = "bar" },
new Test { Text = "words words", Flag = "foo" }
})
.Refresh(Refresh.WaitFor)
);
client.Count<Test>(s => s
.Query(q => q
.Match(m => m
.Field(p => p.Flag)
.Query("foo")
)
)
);
client.UpdateByQuery<Test>(u => u
.Query(q => q
.Term(f => f.Flag, "bar")
)
.Script("ctx._source.flag = 'foo'")
.Conflicts(Conflicts.Proceed)
.Refresh(true)
);
client.Count<Test>(s => s
.Query(q => q
.Match(m => m
.Field(p => p.Flag)
.Query("foo")
)
)
);
}
public class Test
{
public string Text { get; set; }
public string Flag { get; set; }
}
Observe that the count from the first Count API call is 1, and on the second Count API call after the Update By Query API call, it's 2.
I need to get the documents from ES using NEST client with multiple And/OR conditions on two fields.
My query is as:
SELECT * FROM Document WHERE (Year!=2012 && Year!=2013 ) AND (Format=".pdf" || Format=".prt" || Format=".jpeg")
below is my code:
var qc = new List<QueryContainer>();
foreach (var year in years)// years is the list of years that must not be included
{
qc.Add(Query<Document>.Match(m => m.OnField(p => p.Year).Query(year)));
}
var qF = new List<QueryContainer>();
foreach (var format in txtDocs)// txtDocs is the list of formats that should be included if available
{
qF.Add(Query<Document>.Match(m => m.OnField(p => p.Format).Query(format)));
}
var searchResults = client.Search<Document>(s => s.Index(defaultIndex).From(0).Size(50).
Query(
f => f.Bool(
b => b
.MustNot(qc.ToArray()).Should(qF.ToArray()))));
When I try this code it works for the years that must not appear in the results but for the formats that should be selected by user, it doesn't show those selected formats although they are available.
I also used "must" instead of "should", but then it does not retrieve anything at all.
Has anyone had such a similar problem?
public class Test
{
public int Year { get; set; }
[ElasticProperty(Index = FieldIndexOption.NotAnalyzed)]
public string Format { get; set; }
}
var searchResponse = client.Search<Test>(s => s.Query(q => q
.Bool(b => b
.MustNot(
m => m.Term(t => t.OnField(f => f.Year).Value(2012)),
m => m.Term(t => t.OnField(f => f.Year).Value(2013))
)
.Should(
should => should.Term(t => t.OnField(f => f.Format).Value(".pdf")),
should => should.Term(t => t.OnField(f => f.Format).Value(".prt")),
should => should.Term(t => t.OnField(f => f.Format).Value(".jpeg"))
)
)));
Hope it helps.
Here is the code for making a dynamic query:
QueryContainer qYear=null;
foreach (var year in years)
{
qYear |= new TermQuery() { Field = "year", Value = year };
}
QueryContainer qDoc=null;
foreach (var format in txtDocs)
{
qDoc|=new TermQuery() {Field="format", Value= format};
}
var searchResults = client.Search<Document>(s => s.Index(defaultIndex).From(0).Size(50).
Query(q => q.Bool(b => b.Should(qDoc).MustNot(qYear))));
I am trying to create some completion suggesters on some of my fields. My document class looks like this:
[ElasticType(Name = "rawfiles", IdProperty = "guid")]
public class RAW
{
[ElasticProperty(OmitNorms = true, Index = FieldIndexOption.NotAnalyzed, Type = FieldType.String, Store = true)]
public string guid { get; set; }
[ElasticProperty(OmitNorms = true, Index = FieldIndexOption.Analyzed, Type = FieldType.String, Store = true, IndexAnalyzer = "def_analyzer", SearchAnalyzer = "def_analyzer_search", AddSortField = true)]
public string filename { get; set; }
[ElasticProperty(OmitNorms = true, Index = FieldIndexOption.Analyzed, Type = FieldType.String, Store = true, IndexAnalyzer = "def_analyzer", SearchAnalyzer = "def_analyzer_search")]
public List<string> tags { get { return new List<string>(); } }
}
And here is how I am trying to create the completion fields
public bool CreateMapping(ElasticClient client, string indexName)
{
IIndicesResponse result = null;
try
{
result = client.Map<RAW>(
c => c.Index(indexName)
.MapFromAttributes()
.AllField(f => f.Enabled(false))
.SourceField(s => s.Enabled())
.Properties(p => p
.Completion(s => s.Name(n => n.tags.Suffix("comp"))
.IndexAnalyzer("standard")
.SearchAnalyzer("standard")
.MaxInputLength(20)
.Payloads()
.PreservePositionIncrements()
.PreserveSeparators())
.Completion(s2 => s2.Name(n=>n.filename.Suffix("comp"))
.IndexAnalyzer("standard")
.SearchAnalyzer("standard")
.MaxInputLength(20)
.Payloads()
.PreservePositionIncrements()
.PreserveSeparators())
)
);
}
catch (Exception)
{
}
return result != null && result.Acknowledged;
}
My problem is that this is only creating a single completion field named "comp". I was under the impression that this will create two completion fields, one named filename.comp and the other named tags.comp.
I then tried the answer on this SO question but this complicated the matter even worse as now my two fields were mapped as a completion field only.
Just to be clear, I want to create a multi-field (field) that has a data, sort and completion fileds. Much like the one in this example
This is how you can reproduce auto-complete example from attached by you article.
My simple class(we are going to implement auto-complete on Name property)
public class Document
{
public int Id { get; set; }
public string Name { get; set; }
}
To create multi field mapping in NEST we have to define mapping in such manner:
var indicesOperationResponse = client.CreateIndex(descriptor => descriptor
.Index(indexName)
.AddMapping<Document>(m => m
.Properties(p => p.MultiField(mf => mf
.Name(n => n.Name)
.Fields(f => f
.String(s => s.Name(n => n.Name).Index(FieldIndexOption.Analyzed))
.String(s => s.Name(n => n.Name.Suffix("sortable")).Index(FieldIndexOption.NotAnalyzed))
.String(s => s.Name(n => n.Name.Suffix("autocomplete")).IndexAnalyzer("shingle_analyzer"))))))
.Analysis(a => a
.Analyzers(b => b.Add("shingle_analyzer", new CustomAnalyzer
{
Tokenizer = "standard",
Filter = new List<string> {"lowercase", "shingle_filter"}
}))
.TokenFilters(b => b.Add("shingle_filter", new ShingleTokenFilter
{
MinShingleSize = 2,
MaxShingleSize = 5
}))));
Let's index some documents:
client.Index(new Document {Id = 1, Name = "Tremors"});
client.Index(new Document { Id = 2, Name = "Tremors 2: Aftershocks" });
client.Index(new Document { Id = 3, Name = "Tremors 3: Back to Perfection" });
client.Index(new Document { Id = 4, Name = "Tremors 4: The Legend Begins" });
client.Index(new Document { Id = 5, Name = "True Blood" });
client.Index(new Document { Id = 6, Name = "Tron" });
client.Index(new Document { Id = 7, Name = "True Grit" });
client.Index(new Document { Id = 8, Name = "Land Before Time" });
client.Index(new Document { Id = 9, Name = "The Shining" });
client.Index(new Document { Id = 10, Name = "Good Burger" });
client.Refresh();
Now, we are ready to write prefix query :)
var searchResponse = client.Search<Document>(s => s
.Query(q => q
.Prefix("name.autocomplete", "tr"))
.SortAscending(sort => sort.Name.Suffix("sortable")));
This query will get us
Tremors 2: Aftershocks
Tremors 3: Back to Perfection
Tremors 4: The Legend Begins
Tron
True Blood
True Grit
Hope this will help you.
Recently, guys from NEST prepared great tutorial about NEST and elasticsearch. There is a part about suggestions, it should be really useful for you.
I am writing a query in nest for elasticsearch that matches to a list of countries - it cutrrently matches whenever any of the countries in the list is present in ESCountryDescription (a list of countries). I only want to match when all of the countries in CountryList match ESCountryDescription. I believe that I need to use MinimumShouldMatch as in this example http://www.elastic.co/guide/en/elasticsearch/reference/0.90/query-dsl-terms-query.html
a.Terms(t => t.ESCountryDescription, CountryList)
But I cannot find a way of adding MinimumShouldMatch into my query above.
You can apply MinimumShouldMatch patameter in TermsDescriptor. Here is an example:
var lookingFor = new List<string> { "netherlands", "poland" };
var searchResponse = client.Search<IndexElement>(s => s
.Query(q => q
.TermsDescriptor(t => t.OnField(f => f.Countries).MinimumShouldMatch("100%").Terms(lookingFor))));
or
var lookingFor = new List<string> { "netherlands", "poland" };
var searchResponse = client.Search<IndexElement>(s => s
.Query(q => q
.TermsDescriptor(t => t.OnField(f => f.Countries).MinimumShouldMatch(lookingFor.Count).Terms(lookingFor))));
And this is the whole example
class Program
{
public class IndexElement
{
public int Id { get; set; }
[ElasticProperty(Index = FieldIndexOption.NotAnalyzed)]
public List<string> Countries { get; set; }
}
static void Main(string[] args)
{
var indexName = "sampleindex";
var uri = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(uri).SetDefaultIndex(indexName).EnableTrace(true);
var client = new ElasticClient(settings);
client.DeleteIndex(indexName);
client.CreateIndex(
descriptor =>
descriptor.Index(indexName)
.AddMapping<IndexElement>(
m => m.MapFromAttributes()));
client.Index(new IndexElement {Id = 1, Countries = new List<string> {"poland", "germany", "france"}});
client.Index(new IndexElement {Id = 2, Countries = new List<string> {"poland", "france"}});
client.Index(new IndexElement {Id = 3, Countries = new List<string> {"netherlands"}});
client.Refresh();
var lookingFor = new List<string> { "germany" };
var searchResponse = client.Search<IndexElement>(s => s
.Query(q => q
.TermsDescriptor(t => t.OnField(f => f.Countries).MinimumShouldMatch("100%").Terms(lookingFor))));
}
}
Regarding your problem
For terms: "netherlands" you will get document with Id 3
For terms: "poland" and "france" you will get documents with Id 1 and 2
For terms: "germany" you will get document with Id 1
For terms: "poland", "france" and "germany" you will get document
with Id 1
I hope this is your point.
Instead of doing
.Query(q => q
.Terms(t => t.ESCountryDescription, CountryList))
You can use the command below
.Query(q => q
.TermsDescriptor(td => td
.OnField(t => t.ESCountryDescription)
.MinimumShouldMatch(x)
.Terms(CountryList)))
See this for unit tests in elasticsearch-net Github repository.