Trying to achieve following,
Retrieve articles that match given id and genre
Retrieve selected fields for matching records
I have tried this with Sense(chrome plugin),
POST /d3acampaign/article/_search
{
"fields": ["title","genre"] ,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [{
"term": {
"id": "6"
}
},
{
"term": {
"genre": "metal"
}
}]
}
}
}
}
}
For C# code i am trying to build the query using following construct,
FilterContainer fc = null;
TermFilter title = new TermFilter()
{
Field = "id",
Value = "6",
};
TermFilter genre = new TermFilter()
{
Field = "genre",
Value = "metal",
};
fc = title & genre;
QueryContainer qc = new FilteredQuery() { Filter = fc };
var searchRequest = new SearchRequest
{
SearchType = Elasticsearch.Net.SearchType.QueryAndFetch,
Query = qc,
Indices = new IndexNameMarker[] {"journal"},
Types = new TypeNameMarker[] { "article" },
};
var r = client.SearchAsync<Article>(searchRequest);
var l = (List<Article>) r.Result.Documents;
I am able to run this query and get matching records, but i amnt sure how to specify selected fields to retrieve. Let me know what can be changed in C# code to specify necessary fields.
Thanks in advance.
Based on this answer you can modify your request object as follow:
var searchRequest = new SearchRequest
{
...
Fields = new List<PropertyPathMarker>
{
Property.Path<Article>(p => p.YourField)
}
};
or if you will decide to use source filtering:
var searchRequest = new SearchRequest
{
...
Source = new SourceFilter
{
Include = new []
{
Property.Path<Article>(p => p.YourField)
}
}
};
Hope it helps.
Related
I'm using NEST client for querying ES, but now I have a specific situation - I'm trying to proxy query to ES, but with specific query applied by default:
public IEnumerable<TDocument> Search<TDocument>(string indexName, string query, string sort, int page, int pageSize) where TDocument : class
{
var search = new SearchRequest(indexName)
{
From = page,
Size = pageSize,
Query = new RawQuery(query),
};
var response = this.client.Search<TDocument>(search);
return response.Documents;
}
Code above is just proxying query to ES, but what if I need to apply specific filter that should be always applied along with passed query?
So for example I'd want Active field to be true by default. How can I merge this raw query with some specific and always applied filter (without merging strings to formulate merged ES API call if possible).
Assuming that query is well formed JSON that corresponds to the query DSL, you could deserialize it into an instance of QueryContainer and combine it with other queries. For example
var client = new ElasticClient();
string query = #"{
""multi_match"": {
""query"": ""hello world"",
""fields"": [
""description^2.2"",
""myOtherField^0.3""
]
}
}";
QueryContainer queryContainer = null;
using (var stream = client.ConnectionSettings.MemoryStreamFactory.Create(Encoding.UTF8.GetBytes(query)))
{
queryContainer = client.RequestResponseSerializer.Deserialize<QueryContainer>(stream);
}
queryContainer = queryContainer && +new TermQuery
{
Field = "another_field",
Value = "term"
};
var searchResponse = client.Search<TDocument>(s => s.Query(q => queryContainer));
which will translate to the following query (assuming default index is _all)
POST http://localhost:9200/_all/_search?pretty=true&typed_keys=true
{
"query": {
"bool": {
"filter": [{
"term": {
"another_field": {
"value": "term"
}
}
}],
"must": [{
"multi_match": {
"fields": ["description^2.2", "myOtherField^0.3"],
"query": "hello world"
}
}]
}
}
}
I'm querying ElasticSearch using the Nest library for C#, to fetch graph data with multiple pivots. Each pivot is a nested TermsAggregation on a query, and everything works fine with one or two pivots. Once I get to three pivots, though, the SearchRequest object won't generate further aggregations.
The code to build the aggregations looks like this:
TermsAggregation topTermAgg = null;
TermsAggregation currentAgg = null;
foreach (var pivotName in activePivots)
{
newTermAgg = new TermsAggregation("pivot")
{
Field = pivot.ToString().ToLower()
};
if (topTermAgg == null)
{
topTermAgg = newTermAgg;
}
else
{
currentAgg.Aggregations = newTermAgg;
}
currentAgg = newTermAgg;
}
The SearchRequest itself is pretty straightforward:
var searchRequest = new SearchRequest(Indices.Index("a", "b", "c"))
{
Size = 0,
Aggregations = topTermAgg,
Query = query,
};
Unfortunately, the SearchRequest for 3 or more pivots, when converted to string, looks like this (via nestClient.Serializer.SerializeToString(searchRequest)):
{
"size": 0,
"query": {
"bool": <Fairly complex query, that works fine. It's the aggregation that has the problem.>
},
"aggs": {
"pivot": {
"terms": {
"field": "pivot1"
},
"aggs": {
"pivot": {
"terms": {
"field": "pivot2"
}
}
}
}
}
}
When I inspect the searchRequest object in the debugger, it quite definitely has 3 or more aggregations. What's going on here, and how can I get 3 or more nested terms aggregations to work properly?
I am using Nest version 5.01.
This must be related to the way in which you're building up the nested aggregations. Arbitrarily deep nested aggregations can be built with the client. Here's an example of a three deep nested aggregation
client.Search<Question>(s => s
.Aggregations(a => a
.Terms("top", ta => ta
.Field("top_field")
.Aggregations(aa => aa
.Terms("nested_1", nta => nta
.Field("nested_field_1")
.Aggregations(aaa => aaa
.Terms("nested_2", nnta => nnta
.Field("nested_field_3")
)
)
)
)
)
)
);
which serializes to
{
"aggs": {
"top": {
"terms": {
"field": "top_field"
},
"aggs": {
"nested_1": {
"terms": {
"field": "nested_field_1"
},
"aggs": {
"nested_2": {
"terms": {
"field": "nested_field_3"
}
}
}
}
}
}
}
}
You can also add values to AggregationDictionary directly
var request = new SearchRequest<Question>
{
Aggregations = new AggregationDictionary
{
{ "top", new TermsAggregation("top")
{
Field = "top_field",
Aggregations = new AggregationDictionary
{
{ "nested_1", new TermsAggregation("nested_1")
{
Field = "nested_field_1",
Aggregations = new AggregationDictionary
{
{ "nested_2", new TermsAggregation("nested_2")
{
Field = "nested_field_2"
}
}
}
}
}
}
}
}
}
};
client.Search<Question>(request);
is the same as the previous request. You can shorten this even further to
var request = new SearchRequest<Question>
{
Aggregations = new TermsAggregation("top")
{
Field = "top_field",
Aggregations = new TermsAggregation("nested_1")
{
Field = "nested_field_1",
Aggregations = new TermsAggregation("nested_2")
{
Field = "nested_field_2"
}
}
}
};
client.Search<Question>(request);
I got my code working by constructing the aggregation from the bottom-up, rather than from the top-down.
var terminalAggregation = <some aggregation. In my code, there's a lowest aggregation that's different from the rest. For the code I presented, you could just build the lowest pivot.>
TermsAggregation topTermAgg = null;
activePivots.Reverse();
foreach (var pivotName in activePivots)
{
newTermAgg = new TermsAggregation("pivot")
{
Field = pivot.ToString().ToLower(),
Aggregations = topTermAgg ?? terminalAggregation
};
topTermAgg = newTermAgg;
}
This looks like a bug in the Nest library; there are different classes like AggregationBase and BucketAggregationBase and AggregationDictionary that are all assignable to the "Aggregations" property, but it seems like there's some subtle flaw after the second assignment when you do this recursively.
The documentation is also not up-to-date: it claims that you can create an AggregationDictionary yourself, but since AggregationDictionary doesn't have a public Add() method, I really can't. Nor can I use the C#'s {}-after-insantiation syntax to populate its properties – again, because Add() is not public.
I'm trying to get an nGram filter to work with a fuzzy search, but it won't. Specifically, I'm trying to get "rugh" to match on "rough".
I don't know whether it's just not possible, or it is possible but I've defined the mapping wrong, or the mapping is fine but my search isn't defined correctly.
Mapping:
{
settings = new
{
index = new
{
number_of_shards = 1,
number_of_replicas = 1,
analysis = new
{
filter = new
{
edge_ngram_filter = new
{
type = "nGram",
min_gram = 3,
max_gram = 8
}
}, // filter
analyzer = new
{
analyzer_ngram = new
{
type = "custom",
tokenizer = "standard",
filter = new string[]
{
"lowercase",
"edge_ngram_filter"
}
}
} // analyzer
} // analysis
} // index
}, // settings
mappings = new
{
j_cv = new
{
properties = new
{
Text = new
{
type = "text",
include_in_all = false,
analyzer = "analyzer_ngram",
search_analyzer = "standard"
}
}
} // j_cv
} // mappings
}
Document:
{
Id = Guid.NewGuid(),
Name = "Jimmy Riddle",
Keyword = new List<string>(new string[] { "Hunting", "High", "Hotel", "California" }),
Text = "Rough Justice was a program on BBC some years ago. It was quite interesting. Will this match?"
}
Search:
{
query = new
{
query_string = new
{
fields = new string[] { "Text" },
fuzziness = "3",
query = "rugh"
}
}
}
Incidentally, "ugh" does match which is what you'd expect.
Thanks for any help you can give,
Adam.
The same analyzer should usually be applied at index and search time, so search_analyzer=standard is wrong, it should be working if you remove it.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-analyzer.html
Edit:
You forgot the fuzzy operator "~" in your query, if you add it to "rugh" it will work!
I am trying to do Elastic search with sort option. My query is like this:
var client = new ElasticClient(settings);
var query = new
{
query = new
{
term = new { title = "7-0 v Spurs" }
},
Sort = new List<ISort>
{
new SortField { Field = "releaseFrom", Order = SortOrder.Descending }
}
};
and my search is like this:
var stream = new MemoryStream();
client.Serializer.Serialize(query, stream);
var jsonQuery = System.Text.Encoding.UTF8.GetString(stream.ToArray());
var qRequest = new SearchRequest(jsonQuery);
var searchResponse = client.LowLevel.Search<SearchResponse<dynamic>>(IndexingService.IndexName, "article_en", qRequest);
I am getting the result, but it returns records which does not match the title and also it does not sort.
This is the query which is generated:
{ "query": { "term": { "title": "7-0 v Spurs" } }, "sort": [ { "releaseFrom": { "order": "desc" } } ] }
Anybody, with suggestion if I miss something here.
Found the solution.
Used ElasticLowLevelClient instead of ElasticClient.
Code is like this:
var lowlevelClient = new ElasticLowLevelClient(settings);
var stream = new MemoryStream();
lowlevelClient.Serializer.Serialize(query, stream);
var jsonQuery = System.Text.Encoding.UTF8.GetString(stream.ToArray());
var searchResponse = lowlevelClient.Search< SearchResponse<dynamic>>(IndexingService.IndexName, "article_en", jsonQuery);
one change in query also
match = new { title = "7-0 v Spurs" }
I am new to Elastic search,
I have object structure like,
class Student
{
int schoolId;
List<Subject> Subjects;
}
class Subject
{
string title;
string type;
}
I am trying to list down all the unique Subject titles. So far, I have the following Query that works in Sense and gives me desired result.
curl -XGET "http://localhost:9200/school-v0.1/student/_search?search_type=count&routing=5" -d '{
"filter":
{
"term": { "schoolId": 5 }
},
"aggs":
{
"SubjectsAggr":
{
"nested": { "path": "Subjects" },
"aggs":
{
"TitlesAggr":
{
"terms": { "field": "subject.title" }
}
}
}
}
}'
But I am not getting the NEST equivalent of the same. Refer below.
Am I missing something?
var result = this.ElasticClient.Search<Student>(q => q
.Index(this.ElasticClient.Index)
.Routing(schoolId)
.SearchType(SearchType.Count)
.Filter(q1 => q1.Term(a => a.schoolId, schoolId))
.Aggregations(student => student.Nested("SubjectsAggr", b => b.Path("subjects")
.Aggregations(sub => sub.Terms("TitlesAggr", s => s.Field("subject.title"))))));
//Below is what I hope to do, but the I get compilation errors because there seems to be something wrong with above NEST query that I have written
var subjectsAggregation = esResult.Aggs.Nested("SubjectsAggr");
var titlesAggregation = subjectsAggregation.Aggs.Terms("TitlesAggr");
var subjects = new List<string>();
foreach (var s in titlesAggregation.Items)
{
subjects.Add(s.Key);
}
Could you please help me find it? Thank you.
My NEST query was indeed correct. I was only not fetching the results out in appropriate way. I am now able to extract the needed information out of the search result as shown below.
var subjectsAggregation = result.Aggs.Nested("SubjectsAggr");
var titlesAggregation = subjectsAggregation.Aggregations["TitlesAggr"];
var titlesBucket = titlesAggregation as Bucket;
var titles = new List<string>();
if (titlesBucket != null)
{
foreach (var title in titlesBucket.Items)
{
titles.Add((((KeyItem) title).Key));
}
}