I'm using Elasticsearch 1.7.x (and NEST 1.7.2) and trying to take advantage of filtering with lookups as documented here: Terms Filter Lookup. I'm able to craft the JSON by hand for the request I desire and execute it using Sense. Works great, awesome feature! However, in the NEST library, I don't see a way to create such a terms clause. For example, from the link referenced above, I can do something like:
"terms" : {
"proteins" : {
"index" : "microarrays",
"type" : "experiment",
"id" : "experiment1234",
"path" : "upregulated_proteins"
},
"_cache_key" : "experiment_1234"
}
Is there a way to build this query using NEST? If not, is there a way I can inject some JSON into a NEST query as I'm building it? I don't know if NEST 2.x+ supports this but upgrading to ES 2.x is a longer term plan for us and I'd like to leverage functionality that is already available in ES 1.7.
Awesome, I've already received an answer from Greg Marzouka of Elastic! He says:
It's mapped as TermsLookup() or TermsLookupFilter in 1.x. Check out the unit tests for some examples.
client.Search<Paper>(s => s
.Query(q => q
.Filtered(fq => qf
.Filter(f => f
.CacheKey("experiment_1234")
.TermsLookup(t => t
.Lookup<Protein>(p => p.UnregulatedProteins, "experiment1234", "microarrays", "experiment")
)
)
)
));
In 2.x it's a little more aligned with the ES query DSL.
Related
I have different fields to make a query from, like a title, a text, and a location. I want to put more weight on the location field. I found the decay_function from Elastic Search, which exactly answers my needs according to this example : https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_detailed_example .
However the system i'm on uses Hibernate Search queries. I found the Spatial ( https://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#spatial ) filter, but is there a way to replace the "within()" by the decay_function ? Or by a personnalized function or something like that ?
I know I could use the within() but it means I would have to define specific ranges. It could work but it's not the optimal solution in my case.
Not using the DSL itself (for now, it will be possible in 6 to integrate native predicates in the DSL).
But you can create a native Elasticsearch query for this specific case as explained in our documentation: https://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#_queries :
FullTextEntityManager fullTextEm = Search.getFullTextEntityManager(entityManager);
QueryDescriptor query = ElasticsearchQueries.fromJson(
"{ 'query': { 'match' : { 'lastName' : 'Brand' } } }");
List<?> result = fullTextEm.createFullTextQuery(query, GolfPlayer.class).getResultList();
This way you can use whatever Elasticsearch construct you see fit. And you can still use the DSL for the other supported queries.
Unrelated to your question: I saw you are a student, are you using Hibernate Search for a school project? It would be nice having Search taught in a French school :).
I am using nest to create my Elasticsearch Index. I have two questions:
Question 1. How can I add the settings to use english analyzer with a fall back for Standard Analyzer?
This is how I am creating my Index:
Uri _node = new Uri("elasticUri");
ConnectionSettings _connectionSettings = new ConnectionSettings(_node)
.DefaultIndex("MyIndexName")
.DefaultMappingFor<POCO>(m => m
.IndexName("MyIndexName")
);
IElasticClient _elasticClient = new ElasticClient(_connectionSettings);
var createIndexResponse = _elasticClient.CreateIndex("MyIndexName", c => c
.Mappings(m => m
.Map<POCO>(d => d.AutoMap())
)
);
Looking at the examples Here, I am also not sure what should I pass for "english_keywords", "english_stemmer", etc
Question 2: If I use English Analyzer, will Elasticsearch automatically realize that the terms: "Barbecue" and "BBQ" are synonyms? Or do I need to explicitly pass a list of Synonyms to ES?
Take a look at the NEST documentation for configuring a built-in analyzer for an index.
The documentation for the english analyzer simply demonstrates how you could reimplement the english analyzer yourself, as a custom analyzer, with the built-in analysis components, if you need to customize any part of the analysis. If you don't need to do this, simply use english as the name for the analyzer for a field
client.CreateIndex("my_index", c => c
.Mappings(m => m
.Map<POCO>(mm => mm
.AutoMap()
.Properties(p => p
.Text(t => t
.Name(n => n.MyProperty)
.Analyzer("english")
)
)
)
)
);
Will use the built-in english analyzer for the MyProperty field on POCO.
The english analyzer will not perform automatic synonym expansion for you, you'll need to configure the synonyms that are relevant to your search problem. You have two choices with regards to synonyms
Perform synonym expansion at index time on the index input. This will result in faster search at the expense of being a relatively fixed approach.
Perform synonym expansion at query time on the query input. This will result in slower search, but affords the flexibility to more easily add new synonym mappings as and when you need to.
You can always take the approach of using both, that is, indexing the synonyms that you expect to be relevant to your search use case, and adding new synonyms at query time, as you discover them to be relevant to your use case.
Say we have an ElasticSearch instance and one index. I now want to search the whole index for documents that contain a specific value. It's relevant to the search for this query over multiple fields, so I don't want to specify every field to search in.
My attempt so far (using NEST) is the following:
var res2 = client.Search<ElasticCompanyModelDTO>(s => s.Index("cvr-permanent").AllTypes().
Query(q => q
.Bool(bo => bo
.Must( sh => sh
.Term(c=>c.Value(query))
)
)
));
However, the query above results in an empty query:
I get the following output, ### ES REQEUST ### {} , after applying the following debug on my connectionstring:
.DisableDirectStreaming()
.OnRequestCompleted(details =>
{
Debug.WriteLine("### ES REQEUST ###");
if (details.RequestBodyInBytes != null) Debug.WriteLine(Encoding.UTF8.GetString(details.RequestBodyInBytes));
})
.PrettyJson();
How do I do this? Why is my query wrong?
Your problem is that you must specify a single field to search as part of a TermQuery. In fact, all ElasticSearch queries require a field or fields to be specified as part of the query. If you want to search every field in your document, you can use the built-in "_all" field (unless you've disabled it in your mapping.)
You should be sure you really want a TermQuery, too, since that will only match exact strings in the text. This type of query is typically used when querying short, unanalyzed string fields (for example, a field containing an enumeration of known values like US state abbreviations.)
If you'd like to query longer full-text fields, consider the MultiMatchQuery (it lets you specify multiple fields, too.)
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html
Try this
var res2 = client.Search<ElasticCompanyModelDTO>(s =>
s.Index("cvr-permanent").AllTypes()
.Query(qry => qry
.Bool(b => b
.Must(m => m
.QueryString(qs => qs
.DefaultField("_all")
.Query(query))))));
The existing answers rely on the presence of _all. In case anyone comes across this question at a later date, it is worth knowing that _all was removed in ElasticSearch 6.0
There's a really good video explaining the reasons behind this and the way the replacements work from ElasticOn starting at around 07:30 in.
In short, the _all query can be replaced by a simple_query_string and it will work with same way. The form for the _search API would be;
GET <index>/_search
{
"query": {
"simple_query_string" : {
"query": "<queryTerm>"
}
}
}
The NEST pages on Elastic's documentation for this query are here;
I am working with elasticsearch 2.3.4 (can update to 5 but its still 1 week since release and waiting for reviews on how its working)
I am trying to create asearch within my .net class
ISearchResponse<ClassModel> apiResponse = client1.Search<ClassModel>(a =>
a.Query(q =>
q.Term(p => p.param1, Param1) &&
q.Term(p => p.const1, "const1") &&
q.Term(p => p.param2, param2)));
For some reason the const1 return no values (even if i run it alone without the other params) but with HD extension i get results, maybe i shouldnt use Term ? something else?
Thank you in advance
It sounds as though you might not have the correct mapping on the "const1" field.
Edit as per comment below: You can use a term query on an analyzed field but it's unlikely to work how you might expect. If your field "const1" contains multiple words, then a term query with search text equal to the string you indexed will not match.
"const1": {
"type": "string",
"index": "not_analyzed"
}
Previously with ES 1.x I could construct my nest query like this:
Nest.SearchDescriptor<object> query = new Nest.SearchDescriptor<object>()
.Type("my_type")
.Size(Int16.MaxValue)
.Query(q => q.Terms("my_ID", my_list));
Since upgrading Nest (and ES to 2.3.0) though Terms no longer takes 2 parameters, so I figured this is the new way to do it:
Nest.SearchDescriptor<object> query = new Nest.SearchDescriptor<object>()
.Type("my_type")
.Size(Int16.MaxValue)
.Query(q => q.Terms(t => t.Name("my_ID").Terms(my_list)));
If I look at the json this produces now I see it's just this:
{ "size": 32767 }
What am I doing wrong?