Elasticsearch NEST 5.x Search Query that is built dynamically - elasticsearch

I have the following query that I build piecemeal/dynamically using "&=". Elasticsearch 5.x and Nest 5.x.
QueryContainer qfilter = null;
qfilter = Query<ClassEL>.Term(m => m.OrderID, iOrderID);
qfilter &= Query<ClassEL>
.Range(r => r
.Field(f => f.Distance)
.GreaterThan(100))
&&
.Query<ClassEL>.Term(t => t.Active, true);
var searchDes = new SearchDescriptor<ClassEL>()
.From(0)
.Size(10)
.Query(qfilter); <===== *** ERROR HERE ***
In Visual Studio, it shows the following error message tip:
Error: Cannot convert from 'Nest.QueryContainer' to 'System.Func<Nest.QueryContainerDescriptor<ClassEL>, Nest.QueryContainer>'
The problem is I can't get the searchDescriptor to accept the query I built. Examples online show Search + Query rolled into one which differs from what I'm trying to accomplish. Below is common example that I want to avoid:
var response = client.Search<Tweet>(s => s
.From(0)
.Size(10)
.Query(q =>
q.Term(t => t.User, "kimchy")
|| q.Match(mq => mq.Field(f => f.User).Query("nest"))
)
);
EDIT: Using the Andrey's answer works just fine. A problem arises however when I try to get the results back from the search query:
List<ClassViewEL> listDocuments = response.Documents.ToList();
Visual Studio doesn't highlight it as an error immediately, but during compile time has a problem:
error CS0570:
'Nest.ISearchResponse.Documents' is
not supported by the language
Debugging and choosing to IGNORE the above error works fine, the code executes just as expected no problems. However the compile time error will prevent code deployments. How can this error be fixed?
Solution to EDIT: One of my dependencies in my projects (Newtonsoft.Json.dll) were targeting an older version causing the error to appear. Cleaning the solution and rebuilding fixes it.

You should use Func<SearchDescriptor<ClassEL>, ISearchRequest> or pass descriptor in separate method. For example:
var queryContainer = Query<ClassEL>.Term(x => x.Field(f => f.FirstName).Value("FirstName"));
queryContainer &= Query<ClassEL>.Term(x => x.Field(f => f.LastName).Value("LastName"));
Func<SearchDescriptor<ClassEL>, ISearchRequest> searchFunc = searchDescriptor => searchDescriptor
.Index("indexName")
.Query(q => queryContainer);
var response = _client.Search<ClassEL>(searchFunc);
or like this
ISearchRequest ExecuteQuery(SearchDescriptor<ClassEL> searchDescriptor, QueryContainer queryContainer)
{
return searchDescriptor.Index("indexName")
.Query(q => queryContainer);
}
public void GetResults()
{
var queryContainer = Query<ClassEL>.Term(x => x.Field(f => f.FirstName).Value("FirstName"));
queryContainer &= Query<ClassEL>.Term(x => x.Field(f => f.LastName).Value("LastName"));
var response = _client.Search<ClassEL>(s => ExecuteQuery(s, queryContainer));
}

Related

"Invalid NEST response built from a unsuccessful () low level call on POST"

The following code works most of the time but sometimes it throws an exception with this message:
Invalid NEST response built from a unsuccessful () low level call on POST: /queries2020-09/_search?typed_keys=true
var response = await client.SearchAsync<LogEntry>(s => s
.Query(q => q
.Bool(b => b
.Must(m => m.DateRange(r => r.Field(l => l.DateTimeUTC)
.GreaterThanOrEquals(new DateMathExpression(since))),
m => m.Term(term)
)))
.Aggregations(a => a
.Sum("total-cost", descriptor => descriptor
.Field(f => f.Cost)
.Missing(1)))
.Size(0));
if (!response.IsValid)
{
throw new Exception("Elasticsearch response error. " + response.ToString());
}
This seems to be a very generic message that pops up a lot on Q&A websites. How do I debug it to see the root cause?
Using NEST 7.6.1.
It may be better to write the debug information out rather than .ToString()
if (!response.IsValid)
{
throw new Exception("Elasticsearch response error. " + response.DebugInformation);
}
The debug information includes the audit trail and details about an error/exception, if there is one. It's a convenience method for collecting the pertinent information available on IResponse in a human readable form.
If a response is always checked for validity and an exception thrown, you may want to set ThrowExceptions() on ConnectionSettings to throw when an error occurs.

Elastic Search search_phase_execution_exception Reason: "all shards failed"

ES Query is as following :
var searchResponse = client.Search<ItemSearch>(s => s
.Query(q => q
.MultiMatch(mm => mm
.Query(searchQuery)
.Type(TextQueryType.BestFields)
.Fields(f => f.Field(p => p.Description).Field(p=>p.Comment).Field(p => p.CommentSmall)
.Field(p => p.DisplaySequence).Field(p => p.ImageUrl).Field(p => p.ItemBrandDescription)
.Field(p => p.ItemBrandSequence).Field(p => p.ItemCode)
.Field(p => p.ItemGroupID).Field(p => p.ItemGroupSpecification).Field(p => p.ItemMaximumOrderAmount)
.Field(p => p.ItemMinimumOrderAmount).Field(p => p.ItemSpecsDescription).Field(p => p.ItemSupplierCode)
.Field(p => p.PackSize).Field(p => p.PriceUnit).Field(p => p.Sequence).Field(p => p.Stock).Field(p => p.StockToday).Field(p => p.StockTomorrow)
.Field(p => p.SupplierCode).Field(p => p.UOM).Field(p => p.UOMTypeID).Field(p => p.UOMTypeDescription)
).Query(searchQuery)
)
));
getting Following error, Can anyone please help me out?
ServerError:ServerError: 400Type: search_phase_execution_exception Reason: "all shards failed"
DebugInformation:Invalid NEST response built from a unsuccessful low level call on POST: /myindex2-solvi/itemsearch/_search
# Audit trail of this API call:
- [1] BadResponse: Node: http://localhost:9200/ Took: 00:00:00.2664112
# ServerError: ServerError: 400Type: search_phase_execution_exception Reason: "all shards failed"
# OriginalException: System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData)
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
IsValid:False
OriginalException:System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData)
Solution if you have multiple websites.
Go to System > Store > Configuration > Catalog > Catalog Search > Elasticsearch Index Prefix
Change the prefix for each store
Save Settings, Clear Cache, and Run index again
Another Official Solution is
Base on the bug filed in my previous reply, I modified the following file to fix the search problem.
./vendor/magento/module-elasticsearch/Model/Adapter/FieldMapper/Product/FieldProvider/FieldType/Converter.php
private const ES_DATA_TYPE_DOUBLE = 'double';
--> private const ES_DATA_TYPE_FLOAT = 'float';
self::INTERNAL_DATA_TYPE_FLOAT => self::ES_DATA_TYPE_DOUBLE,
---> self::INTERNAL_DATA_TYPE_FLOAT => self::ES_DATA_TYPE_FLOAT,

Implementing debouncing batching queue with RxJS

I was trying to understand if RxJS would be a good fit for solving the problem that this node module performs: https://github.com/ericdolson/debouncing-batch-queue
It's description says: "A queue which will emit and clear its contents when its size or timeout is reached. Ideal for aggregating data for bulk apis where batching in a timely manner is best. Or anything really where batching data is needed."
If so, could someone walk me through how to implement the simple example in this npm module with RxJS? Ideally with ES5 if possible.
There's an operator for thatâ„¢: bufferwithtimeorcount. If you need it to be truly equivalent, the input stream would be a Subject, with group_by for namespaces, like the following:
var dbbq$ = new Subject();
dbbq$.group_by(function(v_ns) { return v_ns[1]; })
.flatMap(function(S) {
return S.bufferwithtimeorcount(1000, 2)
});
dbbq$.next([ 'ribs 0' ]);
dbbq$.next([ 'more ribs', 'bbq1' ]);
// is analogous to
var dbbq = new DBBQ(1000, 2);
dbbq.add('ribs 0');
dbbq.add('more ribs', 'bbq1');
No way I'm doing this with ES5 :)
const dataWithNamespace = (data, namespace) => ({data, namespace});
const source = [
dataWithNamespace('ribs 0'),
dataWithNamespace('ribs 1'),
dataWithNamespace('ribs 2'),
dataWithNamespace('ribs 3'),
dataWithNamespace('ribs 4'),
dataWithNamespace('more ribs', 'bbq1'),
dataWithNamespace('more ribs', 'bbq1'),
dataWithNamespace('brisket', 'best bbq namespace')
];
const DBBQ = (debounceTimeout, maxBatchSize) =>
source$ => source$
.groupBy(x => x.namespace)
.mergeMap(grouped$ => grouped$
.switchMap(x =>
Rx.Observable.of(x.data)
.concat(Rx.Observable.of(undefined)
.delay(debounceTimeout)
)
)
.bufferCount(maxBatchSize)
.filter(x => x.length == maxBatchSize)
.map(x => x.filter(x => x !== undefined))
);
const source$ = Rx.Observable.from(source);
DBBQ(1000, 2)(source$).subscribe(console.log)
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.6/Rx.js"></script>

Unable to use Filter_Path with NEST - ElasticSearch 5.0

On using Filter_Path to reduce the metadata returned from Elasticsearch, I get the following error:
: 'Method not found: 'Elasticsearch.Net.Search RequestParameters Elasticsearch.Net.Search RequestParameters.Filter Path(System.String)'.'
var respones = client.Search<dynamic>(s => s
.FilterPath("hits.hits._source")
.Index(Indices.Index(indices))
.Query(q => q.Bool(b => b.Must(mustQueries.ToArray()).Must(shouldQueries.ToArray()))).Source(o => source)
.Size(10000)
.AllTypes());
Any ideas how to use it?

Entity Framework 7 Include() Order

In MVC6/EF7, should there be a difference in an order I use Include() to include navigation properties into a query?
This query works
var vt = await db.VehicleTypes
.Include(t => t.Photos)
.Include(t => t.VehicleModels)
.ThenInclude(m => m.Units)
.Include(t => t.Rates)
.ThenInclude(r => r.DailyPrice.Currency)
.ToListAsync()
But this query throws an exception at ToListAsync()
var vt = await db.VehicleTypes
.Include(t => t.Photos)
.Include(t => t.Rates)
.ThenInclude(r => r.DailyPrice.Currency)
.Include(t => t.VehicleModels)
.ThenInclude(m => m.Units)
.ToListAsync()
The error is
ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
I understand it's Beta, there may be bugs. In this case - is it a bug or a designed behavior?
Looks like a bug; the order shouldn't matter. Would you mind creating an issue?

Resources