I have got unique document ids (across all types) and I would like to check which document already exists in elasticsearch index. I try to search
var duplicateCheck = _elasticClient
.MultiGet(m => m.GetMany<object>(notices.Select(s => s.Id)).Fields("Id"));
but it returns wrong result - every document has set found property to false.
update
there is workaround here
var exisitngDocIds = _elasticClient.Search<ZPBase>(s => s
.AllTypes()
.Query(q => q.Ids(notices.Select(z=>z.Id)))
.Fields("Id")
.Take(notices.Count)
);
notices = notices.Where(q => !exisitngDocIds.Hits.Any(s => s.Id == q.Id)).ToList();
From the Multi Get API documentation I realized that you can use something similar to the following code to solve your problem:
var response = _elasticClient.MultiGet(m => m
.Index(MyIndex)
.Type("")
.GetMany<ZPBase>(noticeIds));
Note the empty string passed as the Type.
Related
I have this C# code which is expected to match 2 fields using multi-match Elastic Search type. I am using NEST package.
var response = await _elasticClient.SearchAsync<FileDocument>(
s => s.Query(q => q.MultiMatch(c => c
.Fields(f => f.Field(p => p.FileName).Field(query))
.Fields(f => f.Field(p => p.Metadata).Field(query))
)));
Problem is no matter what text I passed in, it returns all the result. Anything I miss out?
In order to efficiently debug these types of issue, you need to inspect the HTTP request going to Elasticsearch, ultimately your query builder will be converted to search JSON and will be executed against Elasticsearch.
I am not aware of nest but have written the answer for Java code, which prints the Elasticsearch query in JSON format.
Although my guess is that, you are not sending the correct HTTP method which should be POST and you might be sending it with GET, which is causing ES to ignore your search query and return all documents.
Solved after adding .Query(query)
var response = await _elasticClient.SearchAsync<FileDocument>(
s => s.Query(q => q.MultiMatch(c => c
.Fields(f => f.Field(p => p.FileName).Field(p=>p.Metadata))
.Query(query)
))
);
Reference - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/multi-match-usage.html
I cannot figure out how to search the documents in my index by the MatchPhrasePrefix. I want to match on the entire search term phrase, but allow prefixes on the last term.
This LINQ construction is giving me back everything with a score of 1.0. How would I construct this? On a side note, is there anyway to see the raw query that NEST is constructing? That would be very helpful!
var search = new SearchDescriptor<ObservationElasticModel>();
search = search
.Query(q => q.MatchPhrase(m => m.Query(term)));
var response = _client.Search<ObservationElasticModel>(search);
You can use the MatchPhrasePrefix method on the QueryDescriptor. You may also need to set the Operator to AND.
_client.Search<ObservationElasticModel>(s => s
.Query(q => q
.MatchPhrasePrefix(m => m
.Operator(Operator.And)
.Query(term)));
You can log raw queries to the debugging console by calling EnableTrace() and ExposeRawResponse() on the ConnectionSettings instance during setup.
var settings = new ConnectionSettings(new Uri(searchUri));
settings.EnableTrace();
settings.ExposeRawResponse();
_client = new ElasticClient(settings);
I need to add a timestamp path to my index using NEST, not sure how to make this happen:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-timestamp-field.html
I have been messing with NEST but I cannot figure this out.
I have been reading the docs here but have not found what i am looking for:
http://nest.azurewebsites.net/nest/quick-start.html
Using the fluent API, this can be done when creating your index:
var response = client.CreateIndex("myindex", c => c
.AddMapping<MyType>(m => m
.MapFromAttributes()
.TimestampField(t => t
.SetDisabled(false)
.SetPath(o => o.MyTimestampField)
)
)
);
Or updating an existing index:
var response = client.Map<MyType>(m => m
.TimestampField(t => t
.SetDisabled(false)
.SetPath(o => o.MyTimestampField)
)
);
Hope that helps.
i'm have a movies and music db. based on if the user is in movies or music, the facets need to change.
i've tried using .OnFields(string[]) to pass in an array (which changes based on movies/music) but i'm getting an error.
here's the code i'm using which generates the error. what am i missing?
string[] facetFields = new []{"genres","format","decades","price"};
var searchResult = client.Search<MyData>(s => s
.MatchAll()
.FacetTerm(t => t
.OnFields(facetFields)
.Order(TermsOrder.term)
.AllTerms()
));
the error is:
"Couldn't infer name for facet of type TermFacetDescriptor`1"
found the answer. you have to name the facet (i knew that but wasn't sure where) like so ...
see 'MyFacet' below...
var searchResult = client.Search<MyData>(s => s
.MatchAll()
.FacetTerm("MyFacet", t => t
.OnFields(facetFields)
.Order(TermsOrder.term)
.AllTerms()
));
if this is in the documentation somewhere - could someone point me to it?
Thanks!
I have a logstash/elasticsearch/kibana system set up and structured json type logs are getting into elastic search (not from a C# application) and visible in kibana.
I am using NEST because I would like to slice the data from logstash.
The following code is producing "hits" results that I can inspect in the debugger.
ElasticClient client = new ElasticClient(settings);
var searchResults = client.Search( s => s
.From(0)
.Size(100)
.AllIndices()
.SortDescending("#timestamp")
);
However, if I try to expand the search to include something I believe to be present in the log (visible in Kibana), I get now results.
var searchResults = client.Search( s => s
.From(0)
.Size(100)
.AllIndices()
.Query(q => q
.Term("MySpecialFieldName", "ValueThatAppears")
)
.SortDescending("#timestamp")
);
I would also like to take advantage of the type safety and other mechanisms shown in the samples. I am not sure if that is expected to be related.
( I am working on figuring that out separately: Adding a class like "client.Search( to the search seems to prevent the results as well, I am assuming that something about the class is not aligned with the data and is therefore unable to deserialize or is otherwise filtering...)
Found the correct settings to make this work:
The logstash system puts this into a type (log or logs I think) when it gets indexed by elastic search. Also the term needs to be suffixed with ".raw". So the working example ends up being:
var searchResults = client.Search<MyClass>( s => s
.From(0)
.Size(100)
.AllIndices()
.AllTypes()
.Query(q => q
.Term("MySpecialFieldName.raw", "ValueThatAppears")
)
.SortDescending("#timestamp")
);