I am looking for how to do an elasticsearch _count for nest :
in elastic seatch it would be:
i am looking for the equivalent of:
var request = new SearchRequest<type>()
{
Query = new BoolQuery
{
//Should = ...
//Must = ...
},
MinScore = 1
//....
};
var nbResult = client.Count(request);
If you know how to do it and if you have a tip for having a count of results with the fastest way it would help me a lot.
Use client.Count<T>( ... )
var request = new CountRequest<Document>
{
Query = new MatchAllQuery()
};
var nbResult = client.Count<Document>(request);
which yields the following request
POST http://localhost:9200/default-index/document/_count
{
"query": {
"match_all": {}
}
}
I found in sources. Its not solution because i cant test it locally but at least direction.
Look to this test and client source
Related
I am using Elasticsearch V6, NEST V6.
I am searching ES as below and I am using ScriptFields to calculate the distance and include it the result.
var searchResponse = _elasticClient.Search<MyDocument>(new SearchRequest<MyDocument>
{
Query = new BoolQuery
{
Must = new QueryContainer[] { matchQuery },
Filter = new QueryContainer[] { filterQuery },
},
Source = new SourceFilter
{
Includes = resultFields // fields to be included in the result
},
ScriptFields = new ScriptField
{
Script = new InlineScript("doc['geoLocation'].planeDistance(params.lat, params.lng) * 0.001") // divide by 1000 to convert to km
{
Lang = "painless",
Params = new FluentDictionary<string, object>
{
{ "lat", _center.Latitude },
{ "lng", _center.Longitude }
}
}
}
});
Now, I am trying to read the search result and I am not sure how to read the distance from the response, this is what I have tried:
// this is how I read the Document, all OK here
var docs = searchResponse.Documents.ToList<MyDocument>();
// this is my attempt to read the distance from the result
var hits = searchResponse.Hits;
foreach (var h in hits)
{
var d = h.Fields["distance"];
// d is of type Nest.LazyDocument
// I am not sure how to get the distance value from object of type LazyDocument
}
While debugging I can see the distance value, I am just not sure how to read the value?
I found the answer here
To read search document and distance:
foreach (var hit in searchResponse.Hits)
{
MyDocument doc = hit.Source;
double distance = hit.Fields.Value<double>("distance");
}
And if you are only interested in distance:
foreach (var fieldValues in searchResponse.Fields)
{
var distance = fieldValues.Value<double>("distance");
}
I am new to ElasticsearchTemplate. I want to get 1000 documents from Elasticsearch based on my query.
I have used QueryBuilder to create my query , and it is working perfectly.
I have gone through the following links , which states that it is possible to achieve big data sets using scan and scroll.
link one
link two
I am trying to implement this functionality in the following section of code, which I have copy pasted from one of the link , mentioned above.
But I am getting following error :
The type ResultsMapper is not generic; it cannot be parameterized with arguments <myInputDto>.
MyInputDto is a class with #Document annotation in my project.
End of the day , I just want to retrieve 1000 documents from Elasticsearch.
I tried to find size parameter but I think it is not supported.
String scrollId = esTemplate.scan(searchQuery, 1000, false);
List<MyInputDto> sampleEntities = new ArrayList<MyInputDto>();
boolean hasRecords = true;
while (hasRecords) {
Page<MyInputDto> page = esTemplate.scroll(scrollId, 5000L,
new ResultsMapper<MyInputDto>() {
#Override
public Page<MyInputDto> mapResults(SearchResponse response) {
List<MyInputDto> chunk = new ArrayList<MyInputDto>();
for (SearchHit searchHit : response.getHits()) {
if (response.getHits().getHits().length <= 0) {
return null;
}
MyInputDto user = new MyInputDto();
user.setId(searchHit.getId());
user.setMessage((String) searchHit.getSource().get("message"));
chunk.add(user);
}
return new PageImpl<MyInputDto>(chunk);
}
});
if (page != null) {
sampleEntities.addAll(page.getContent());
hasRecords = page.hasNextPage();
} else {
hasRecords = false;
}
}
What is the issue here ?
Is there any other alternative to achieve this?
I will be thankful if somebody could tell me how this ( code ) is working in the back end.
Solution 1
If you want to use ElasticsearchTemplate, it would be much simpler and readable to use CriteriaQuery, as it allows to set the page size with setPageable method. With scrolling, you can get next sets of data:
CriteriaQuery criteriaQuery = new CriteriaQuery(Criteria.where("productName").is("something"));
criteriaQuery.addIndices("prods");
criteriaQuery.addTypes("prod");
criteriaQuery.setPageable(PageRequest.of(0, 1000));
ScrolledPage<TestDto> scroll = (ScrolledPage<TestDto>) esTemplate.startScroll(3000, criteriaQuery, TestDto.class);
while (scroll.hasContent()) {
LOG.info("Next page with 1000 elem: " + scroll.getContent());
scroll = (ScrolledPage<TestDto>) esTemplate.continueScroll(scroll.getScrollId(), 3000, TestDto.class);
}
esTemplate.clearScroll(scroll.getScrollId());
Solution 2
If you'd like to use org.elasticsearch.client.Client instead of ElasticsearchTemplate, then SearchResponse allows to set the number of search hits to return:
QueryBuilder prodBuilder = ...;
SearchResponse scrollResp = client.
prepareSearch("prods")
.setScroll(new TimeValue(60000))
.setSize(1000)
.setTypes("prod")
.setQuery(prodBuilder)
.execute().actionGet();
ObjectMapper mapper = new ObjectMapper();
List<TestDto> products = new ArrayList<>();
try {
do {
for (SearchHit hit : scrollResp.getHits().getHits()) {
products.add(mapper.readValue(hit.getSourceAsString(), TestDto.class));
}
LOG.info("Next page with 1000 elem: " + products);
products.clear();
scrollResp = client.prepareSearchScroll(scrollResp.getScrollId())
.setScroll(new TimeValue(60000))
.execute()
.actionGet();
} while (scrollResp.getHits().getHits().length != 0);
} catch (IOException e) {
LOG.error("Exception while executing query {}", e);
}
I have a method that takes a query as parameter like:
public ISearchResponse<Object> SearchComponent(SearchDescriptor<Object> query)
{
...
}
In this query I want to add a source filter like:
public ISearchResponse<Object> SearchComponent(SearchDescriptor<Object> query)
{
query = query.Source(sf =>
sf.Exclude(e => e
.Field("SomeField")
));
...
}
But what happens if the query already have a source filter? This filter will override that filter right? How can I update the existing queries source filter?
This seems to work but it's not the most beautiful solution. Anyone that can come up with a better alternative?
public ISearchResponse<Object> SearchComponent(ISearchRequest query)
{
var excludeFields = new List<string>();
excludeFields.Add("SomeField");
if (query.Source == null)
{
query.Source = new SourceFilter {Include = "*", Exclude = excludeFields.ToArray()};
}
else if (query.Source.Exclude == null)
{
query.Source.Exclude = excludeFields.ToArray();
}
else
{
query.Source.Exclude.And(excludeFields.ToArray());
}
...
}
You are using an older version of Nest than what I have and this has changed a bit, but I figure you can do something along these lines:
var exclude = query.Source?.Exclude;
query.Source = new SourceFilter() { Excludes = (exclude ?? new Field[0]).Union(moreFields) };
Hi I am able to populate QueryContainer with DateRangeQuery array as shown below QueryContainer marriageDateQuerys = null;
if (!string.IsNullOrEmpty((item.marriage_date)))
{
DateRangeQuery query = new DateRangeQuery();
query.Field = "marriages.marriage_date";
query.Name = item.marriage_date;
query.GreaterThanOrEqualTo = item.marriage_date;
query.LessThanOrEqualTo = item.marriage_date;
marriageDateQuerys &= query;
}
But when I use QueryContainer to use MatchQuery/TermQuery to populate data it is not happening.
QueryContainer marriageSpouseFirstNameQuerys = null;
if (!string.IsNullOrEmpty((item.spouse_first_name)))
{
MatchQuery query = new MatchQuery();
query.Field = "marriages.spouse_first_name";
query.Name = item.spouse_first_name;
marriageSpouseFirstNameQuerys &= query;
}
Query object is created in last if condition but marriageSpouseFirstNameQuerys is not populated with the same. I even tried marriageSpouseFirstNameQuerys += query; but without any success
Didn't try it but you can try something like this
Query = new QueryContainer(new BoolQuery
{
Must = new List<QueryContainer>
{
new MatchQuery
{
//props
},
new TermQuery
{
Field = field
Value = value
},
}
})
Below code worked for me after making changes with eyildiz answer
if (!string.IsNullOrEmpty((item.spouse_last_name)))
{
marriageSpouseLastNameQuery = new QueryContainer(
new MatchQuery
{
Field = "marriages.spouse_last_name",
Query = item.spouse_last_name
});
lstmarriageSpouseLastNameQuerys.Add(marriageSpouseLastNameQuery);
}
I want to do a query with two facets with SolrNet, City And Category.
But each facet has diferrent properties. For example mincount for city is zero and for category is 1.
How can I do this with SolrNet?
I know FacetParametes has Queries property that can be an array of queries, but what about mincount?
Thanks.
solr.Query("something", new QueryOptions {
Facet = new FacetParameters {
Queries = new[] {
new SolrFacetFieldQuery("city") { MinCount = 0 },
new SolrFacetFieldQuery("category") { MinCount = 1 },
}
}
});