Is it possible to use ElasticSearch.Net or Nest for dynamic response - elasticsearch

Is there a client.Read(...) without generics? I have found none, neither in Nest nor ElasticSearch.Net.
Version 1.5 has an IDocument that might solve my problem but I cannot use that version with Elasticsearch5.5.
All examples, version 5 and 6, of ElasticSearch.Net and Nest require me to know the format of the response as generic at compile time. E.g. Read<Customer>(...)
My problem is that the we do not know the format of the database and we don't know the format of the output; but it should all be configurable.

You can use dynamic as the generic type if the response is truly dynamic.
In 5.x, this will be Json.NET's JObject type under the covers (so you could use JObject instead if you prefer).
In 6.x, dynamic will also work but the actual type will be an internal JObject type. If you would prefer to work with Json.NET's JObject type, you can hook up Json.NET as the serializer using the NEST.JsonNetSerializer nuget package, to use as the serializer for your documents and then use its JObject type as per 5.x.

(Feels strange to answer my own question but I want to show the resulting code for future reference.)
var settings = new ConnectionSettings(new Uri(#"http://localnhost:9200"))
.DefaultIndex("myindex");
var client = new ElasticClient(settings);
var res = client.Search<dynamic>(s => s
.AllTypes());
var rows = res.Documents;
Assert.IsTrue(rows.Count >= 1);
dynamic row = res.Documents.First();
Assert.AreEqual("50.7031526", row.POSITION.lat.ToString()); // It is case sensitive.
Assert.AreEqual(50.7031526, (double)row.POSITION.lat); // Convert to type explicitly.

Related

Elasticsearch NEST Document count for default index

I am using NEST for Elasticsearch 6 and would like to get the number of documents for the default index.
The documentation refers to the 1.1 version of the API which no longer seems to work.
I have created the connection settings using a default index:
var connectionSettings = new ConnectionSettings().DefaultIndex("test_docs");
When I try the code from the 1.1 api documentation:
var result = client.Count();
I get the following error:
The type arguments for method
'ElasticClient.Count(Func, ICountRequest>)'
cannot be inferred from the usage. Try specifying the type arguments
explicitly.
When I supply a type it is appended to the path. For example:
client.Count<TestDocument>();
Generates a URL of http://localhost:9200/test_docs/testdocument/_count when what I really need is http://localhost:9200/test_docs/_count
For those needing the new way of doing this (like myself). I would use the following to get a count from a specific index.
var countRequest = new CountRequest(Indices.Index("videos"));
long count = (await _ecClient.CountAsync(countRequest)).Count;
You can use
var countResponse = client.Count<TestDocument>(c => c.AllTypes());
which will call the API
GET http://localhost:9200/test_docs/_count

In Nest (Elasticsearch), how can I get the raw json mapping of an index?

I want to check the discrepancies between my current mapping (as in my C# code) and the mapping in the elasticsearch index.
With only:
var res = esClient.GetMapping<EsCompany>();
I get GetMappingResponse object in c#, I will have to compare field by field for equality. Even worse, each field has their own properties, I have to descend into those properties for further comparison.
In my application, I prefer obtaining the raw json of the mapping, and I can easily diff two json objects for equality.
I then tried this:
var res = esClient.Raw.IndicesGetMapping(myIndexName);
But when I read res.Response, I get an AmbiguousMatchException exception.
When you connect to Elasticsearch you can choose to expose the raw response like this:
var client = new ElasticClient(new ConnectionSettings().ExposeRawResponse());
Then you should be able to access the raw json via:
var json = res.ConnectionStatus.ResponseRaw;

Are ElasticClient.MapRaw and .CreateIndexRaw gone?

After updating to NEST 0.11.5, it appears as if the NEST.ElasticClient.MapRaw and .CreateIndexRaw methods aren't supported anymore. Have they been renamed or moved or are they completely gone?
In case they're gone, how can I define custom analysis settings on index creation? This is what I've tried:
var indexSettings = new IndexSettings()
{
NumberOfReplicas = 1,
NumberOfShards = 2,
Analysis = new AnalysisSettings() // doesn't work, no setter
{
// here's where my settings would go...
}
};
var response = elasticClient.CreateIndex(indexName, indexSettings);
Doesn't work since there's no setter defined for IndexSettings.Analysis.
The Raw calls have been pushed down to elasticClient.Raw.CreateIndexPost(...).
For the 0.11.5.0 release i created my own script that scans the elasticsearch source code to generate all the raw calls. Appearantly the elasticsearch dev's have also done this so the IRawElasticClient signatures might changes again in the 0.11.6.0 release as NEST will be compatible with the new low level client guidelines.
Also be sure to check out the MapFluent() call though
https://github.com/Mpdreamz/NEST/blob/master/src/Nest.Tests.Unit/Core/Map/FluentMappingFullExampleTests.cs
And CreateIndex() also exposes a fully mapped fluent variant
https://github.com/Mpdreamz/NEST/blob/master/src/Nest.Tests.Integration/Indices/Analysis/Analyzers/AnalyzerTests.cs#L19

FOSElasticaBundle order query

I am integrating FOSElasticaBundle in my Symfony 2.3 project and I need to sort the results by their price property.
Here is my code:
$finder = $this->container->get('fos_elastica.finder.website.product');
$fieldTerms = new \Elastica\Query\Terms();
$fieldTerms->setTerms('taxon_ids', $taxon_ids_array);
$boolQuery->addMust($fieldTerms);
$resultSet = $finder->find($boolQuery);
How I can do this?
Thanks
Try create a \Elastica\Query object which also contains the sorting information, then send this to the finder:
$finder = $this->container->get('fos_elastica.finder.website.product');
$fieldTerms = new \Elastica\Query\Terms();
$fieldTerms->setTerms('taxon_ids', $taxon_ids_array);
$boolQuery->addMust($fieldTerms);
$finalQuery = new \Elastica\Query($boolQuery);
$finalQuery->setSort(array('price' => array('order' => 'asc')));
$resultSet = $finder->find($finalQuery);
Have a look at the elasticsearch docs on the sort parameter to see how to use it properly.
NOTE: \Elastica\Query is quite different to \Elastica\Query\AbstractQuery, the first encapsulates everything you could send to the _search API endpoint (facets, sorting, explain, etc...) The AbstractQuery represents a base type for each of the individual query types (range, fuzzy, terms, etc...).

Translate odata uri to expression

I'd like to use an action filter to translate an Odata uri to a Linq expression. I'm doing this because i'm using the resulting expression to query nonSQL line of business systems. In the WCF web api this was trivial because the translated query was appended as a property of the the request object, as such:
var query = (EnumerableQuery)request.Properties["queryToCompose"];
That seems to have disappeared. Are there any public api's i can use to accomplish this?
I've been trying something similiar.. While not perfect, you can grab the OData expressions directly from the query string and build the LINQ expression manually:
var queryParams = HttpUtility.ParseQueryString( ControllerContext.Request.RequestUri.Query );
var top = queryParams.Get( "$top" );
var skip = queryParams.Get( "$skip" );
var orderby = queryParams.Get( "$orderby" );
And the apply that directly to your IQueryable or whatever you're using for the filtering. Not nearly as useful, but its a start.
So as it turns out the query has changed keys in the request property collection. It also seems that the internal filter that parses the query runs after the custom filters and thus doesn't add the query value. To get the translated query, call the following inside the controller action.
(EnumerableQuery<T>)this.Request.Properties["MS_QueryKey"];
Check out Linq2Rest. It solves this problem.

Resources