I need to search across multiple indices using OIS(Object Initializer Syntax).
I have seen examples of executing search across multiple indices with Fluent DSL, but I still do not know how to execute an equivalent search with OIS.
Here is my OIS search(Only searching against one index) :
var searchResult =
await _client.LowLevel.SearchAsync<string>(ApplicationsIndexName, "application", new SearchRequest()
{
From = (query.PageSize * query.PageNumber) - query.PageSize,
Size = query.PageSize,
Query = GetQuery(query),
Aggregations = GetAggregations()
});
Which modifications can be done, so I can search across multiple indices?
After some research, I found out how to search across multiple indices:
var searchResult =
await _client.LowLevel.SearchAsync<string>(new SearchRequest()
{
IndicesBoost = new Dictionary<IndexName, double>
{
{ "applications", 1.4 },
{ "attachments", 1.4 }
},
From = (query.PageSize * query.PageNumber) - query.PageSize,
Size = query.PageSize,
Query = GetQuery(query),
Aggregations = GetAggregations()
});
Related
I'm struggling to figure out how to include multiple indexes in a search using the Elastic low level client.
My understanding (right or wrong) is that I should be able to include multiple indexes by separation of commas, this doesn't work for me though. In the code example below, I find that the first index specified is still working and returning results, but the second one is ignored. Any ideas?
Appsettings.json file:
// System settings configured here for the WebApp. Applicable to all users.
"SystemSettings": {
// Sets the maximum number of distinct values returned by Elastic for a log property
"_distinctPropertyValuesLimit": 1000, // See LogPropertiesController.cs
// String for the list of Elastic Search indexes that are searched by default.
"indexesToSearch": "webapp-razor-*, systemconfig-api-*"
}
Query class:
_indexesToSearch = configuration.GetSection("SystemSettings").GetSection("indexesToSearch").Value;
var searchResponse = await _elasticLowLevelClient.SearchAsync<StringResponse>(_indexesToSearch, #"
{
""from"": """ + fromParameter + #""",
""size"": """ + rowsPerPage + #""",
""query"": {
""match"": {
""" + searchColumn + #""": {
""query"": """ + searchString + #"""
}
}
},
""sort"": [
{
""#timestamp"": {
""order"": ""desc""
}
}
]
}
");
Turns out that there must not be any spaces between the index names when multiple values are provided, see below:
I am trying to create an Elasticsearch MLT query using NEST's object initializer syntax. However, the final query when serialized, is ONLY missing the MLT part of it. Every other query is present though.
When inspecting the query object, the MLT is present. It's just not getting serialized.
I wonder what I may be doing wrong.
I also noticed that when I add Fields it works. But I don't believe fields is a mandatory property here that when it is not set, then the MLT query is ignored.
The MLT query is initialized like this;
new MoreLikeThisQuery
{
Like = new[]
{
new Like(new MLTDocProvider
{
Id = parameters.Id
}),
}
}
MLTDocProvider implements the ILikeDocument interface.
I expect the serialized query to contain the MLT part, but it is the only part that is missing.
This looks like a bug in the conditionless behaviour of more like this query in NEST; I've opened an issue to address. In the meantime, you can get the desired behaviour by marking the MoreLikeThisQuery as verbatim, which will override NEST's conditionless behaviour
var client = new ElasticClient();
var parameters = new
{
Id = 1
};
var searchRequest = new SearchRequest<Document>
{
Query = new MoreLikeThisQuery
{
Like = new[]
{
new Like(new MLTDocProvider
{
Id = parameters.Id
}),
},
IsVerbatim = true
}
};
var searchResponse = client.Search<Document>(searchRequest);
which serializes as
{
"query": {
"more_like_this": {
"like": [
{
"_id": 1
}
]
}
}
}
I am using Elasticsearch.NET (5.6) on ASP.NET API (.NET 4.6) on Windows, and try to publish to elasticsearch hosted on AWS (I have tried both 5.1.1 and 6, both same behaviour).
I have the following code which bulk index the documents to Elasticsearch. Image calling the below code block many times:
var node = new System.Uri(restEndPoint);
var settings = new ConnectionSettings(node);
var lowlevelClient = new ElasticLowLevelClient(settings);
var index = indexStart + indexSuffix;
var items = new List<object>(list.Count() * 2);
foreach (var conn in list)
{
items.Add(new { index = new { _index = index, _type = "doc", _id = getId(conn) } });
items.Add(conn);
}
try
{
var indexResponse = lowlevelClient.Bulk<Stream>(items);
if (indexResponse.HttpStatusCode != 200)
{
throw new Exception(indexResponse.DebugInformation);
}
return indexResponse.HttpStatusCode;
}
catch (Exception ex)
{
ExceptionManager.LogException(ex, "Cannot publish to ES");
return null;
}
It runs fine, can publish documents to Elasticsearch, but it only can run 80 times, after 80 times, it will always get exception:
# OriginalException: System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
at System.Net.HttpWebRequest.GetRequestStream()
at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Connection\HttpConnection.cs:line 148
The most interesting part is that: I have tried to change the bulk size to be 200 or 30, and it turned out to be 16000 and 2400, meaning both ends up at 80 times. (Each document size is very similar)
Any ideas? Thanks
There is a connection limit (Also refer to comments from #RussCam under the question). So the real issue is that the Stream in the response holding the connections.
So the fix is either indexResponse.Body.Dispose (I haven't tried this one) or use VoidResponse: reportClient.BulkAsync<VoidResponse>(items); which does not require the response stream. I've tried the second and it works.
Just find out about nest. I already insert some number of document in Elastic Search. Right now I want to search the data based on my type, subcriberId. I did run through curl and it works just fine. But when I tried using nest, no result found.
My curl which work:
http://localhost:9200/20160902/_search?q=subscribeId:aca0ca1a-c96a-4534-ab0e-f844b81499b7
My NEST code:
var local = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(local);
var elastic = new ElasticClient(settings);
var response = elastic.Search<IntegrationLog>(s => s
.Index(DateTime.Now.ToString("yyyyMMdd"))
.Type("integrationlog")
.Query(q => q
.Term(p => p.SubscribeId, new Guid("aca0ca1a-c96a-4534-ab0e-f844b81499b7"))
)
);
Can someone point what I did wrong?
A key difference between your curl request and your NEST query is that the former is using a query_string query and the latter, a term query. A query_string query input undergoes analysis at query time whilst a term query input does not so depending on how subscribeId is analyzed (or not), you may see different results. Additionally, your curl request is searching across all document types within the index 20160902.
To perform the exact same query in NEST as your curl request would be
void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var connectionSettings = new ConnectionSettings(pool)
// set up NEST with the convention to use the type name
// "integrationlog" for the IntegrationLog
// POCO type
.InferMappingFor<IntegrationLog>(m => m
.TypeName("integrationlog")
);
var client = new ElasticClient(connectionSettings);
var searchResponse = client.Search<IntegrationLog>(s => s
.Index("20160902")
// search across all types. Note that documents found
// will be deserialized into instances of the
// IntegrationLog type
.AllTypes()
.Query(q => q
// use query_string query
.QueryString(qs => qs
.Fields(f => f
.Field(ff => ff.SubscribeId)
)
.Query("aca0ca1a-c96a-4534-ab0e-f844b81499b7")
)
)
);
}
public class IntegrationLog
{
public Guid SubscribeId { get; set; }
}
This yields
POST http://localhost:9200/20160902/_search
{
"query": {
"query_string": {
"query": "aca0ca1a-c96a-4534-ab0e-f844b81499b7",
"fields": [
"subscribeId"
]
}
}
}
this specifies the query_string query in the body of the request which is analogous to using the q query string parameter to specify the query.
I'm using a design document to ensure that only owners can modify docs. How can I prevent couchdb from replicating this design document?
You can use the filter option in changes() and replicate(), e.g.
var opts = {
live: true,
filter: function(doc) {
return doc._id.indexOf('_design') !== 0;
}
};
var db = new PouchDB('todos');
db.replicate.to('http://localhost:5984/todos', opts);