I ran into an issue where elasticsearch is not able to find indexed categories that contain an "-" in their slug. The category given is "wc-sitze". When I query for "wc", the category is found, and when I query for "sitze", it is also found, however when I query for the whole thing "wc-sitze", it is not found. I have checked with several other categories, and it seems that always the "-" is messing it up.
Any ideas?
Thanks in advance
Looks like you are using the text field with default analyzer standard which removes the special char like -. you need to change the analyzer to make it work.
Related
I am using ELK and I need to filter all the documents with an unmatched COUNTRY (from geoip)
Theses properties looks like:
'IPCOUNTRY': '??'
But I just can't filter on this special value...
I tried
IPCOUNTRY:?? => ? is evaluated > returns all records > normal case-
IPCOUNTRY:\?\? => Doesn't return any document... but lucene documentation says it should be the good way of achieving this...
IPCOUNTRY:"??" => doesnt work
IPCOUNTRY:'??' => doesnt work
EDIT:
This case doesn't work too
- IPCOUNTRY:/[^A-Z]{2}/
Simple but boring issue ^^
Thanx!
You could try :
!IPCOUNTRY:"?"
-IPCOUNTRY:"?"
NOT IPCOUNTRY:"?"
If you have an unanalyzed IPCOUNTRY field, you can do something like :
!IPCOUNTRY.raw:"??"
This is an elasticsearch mapping issue. Punctuation is dropped. You'll need to set your field to an analyzer that would keep ?. Maybe keyword? or not_analyzed?
extract from https://github.com/elastic/kibana/issues/6561#issuecomment-197951710
If all of your fields have documents same as 'IPCOUNTRY': '??', then you can directly filter this field which will exclude the field from matches.
To directly add a filter you can do it in the following 2 ways:-
In Discover page open the text and find the field. Click on + magnifier to add the field as a filter.
In Discover page, on the left side where fields are listed. Click on field name & select the value portaying as ?? to add it as a filter.
How can we do multiple field search in Elastic search.
for example I want to search subcategory and region, for one field it is working for multiple field search how we have to do.
Below link is working fine, since I am using one field only for search
http://34c512ba34534fffdfd12abfd69f2458.us-east-1.aws.found.io:9200/episodes/episode/_search?q=sub_cat_seo_url:english-news&sort=pubdate_timestamp:desc
but when I try to search multiple field for example sub_cat_seo_url and region it is not working
see this link (not working)
http://34c512ba34534fffdfd12abfd69f2458.us-east-1.aws.found.io:9200/episodes/episode/_search?q=sub_cat_seo_url:english-news,region:1&sort=pubdate_timestamp:desc
http://34c512ba34534fffdfd12abfd69f2458.us-east-1.aws.found.io:9200/episodes/episode/_search?q=sub_cat_seo_url:english-news®ion:1&sort=pubdate_timestamp:desc
According to documentation, it should work
See http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html
That being said, you can also use the following:
http://34c512ba34534fffdfd12abfd69f2458.us-east-1.aws.found.io:9200/episodes/episode/_search?q=%2Bsub_cat_seo_url%3Aenglish-news+%2Bregion%3A1&sort=pubdate_timestamp:desc
NOTE :
The existing mapping makes your field "sub_cat_seo_url" analyzed which is analyzed using standard analyzer. Hence, when you are searching for "english-news" it gets tokenized into "english", "news" which results in any document matching either english or news to be valid matches. For eg. "telugu-news" is a valid match for your query. Not sure if it is intentional.
In your mapping you need to mark it as "not_analyzed" for exact match.
Note : %2b is decoded as '+' whereas '+' is decoded as ' '
I have a use case which is a bit similar to the ES example of dynamic_template where I want certain strings to be analyzed and certain not.
My document fields don't have such a convention and the decision is made based on an external schema. So currently my flow is:
I grab the inputs document from the DB
I grab the approrpiate schema (same database, currently using logstash for import)
I adjust the name in the document accordingly (using logstash's ruby mutator):
if not analyzed I don't change the name
if analyzed I change it to ORIGINALNAME_analyzed
This will handle the analyzed/not_analyzed problem thanks to dynamic_template I set but now the user doesn't know which fields are analyzed so there's no easy way for him to write queries because he doesn't know what's the name of the field.
I wanted to use field name aliases but apparently ES doesn't support them. Are there any other mechanisms I'm missing I could use here like field rename after indexation or something else?
For example this ancient thread mentions that field.sub.name can be queried as just name but I'm guessing this has changed when they disallowed . in the name some time ago since I cannot get it to work?
Let the user only create queries with the original name. I believe you have some code that converts this user query to Elasticsearch query. When converting to Elasticsearch query, instead of using the field name provided by the user alone use both the field names ORIGINALNAME as well as ORIGINALNAME_analyzed. If you are using a match query, convert it to multi_match. If you are using a term query, convert it to a bool should query. I guess you get where I am going with this.
Elasticsearch won't mind if a field does not exists. This can be a problem if there is already a field with _analyzed appended in its original name. But with some tricks that can be fixed too.
I have just updated a website, the update adds new fields to elasticsearch.
In my dev environment, it all works fine. but on the live site, the new fields are not being found.
Eg. I have added a new field with the value : 1
However, when adding a filtered query of
{"field":1}
It does not find any matching results.
When I look in the documents, I can see docs with the field set to 1
Would the reason for this be that the new field was added after the mappings was set? I am not all that familiar with elasticsearch, So I am not really sure where to start looking to fix it.
Any help would be appreciated.
Update:
querying from URL shows nothing either
_search/?pretty=true&size=50&q=field1:*
however there is another field that was added at the same time which I can search on.
I can see field1 in the result set but it just wont allow me to search on it.
Only difference i see in the mapping is that the one that is working is set to type:long whereas the one not working is set as type:string
Is it a length issue on the ngram? what was your "min_gram" settings?
When you check on your index settings like this:
GET <host>/<index_name>/_settings
Does it work when you filter for a two digit field?
Are all the field values one digit?
It's OK to add a field after the mapping was set. ElasticSearch will guess the mapping for you. (in fact, it's one of their selling features --- no need to define the mapping, just throw the data at it)
There are a few things that can go wrong:
Verify that data is actually in the index. To do that, just navigate to the _search url with no parameters, you should see the field if it is indexed.
Look at your mapping. Could it be that the field is explicitly set not to be indexed?
Another possibility is that your query is wrong (but that is unlikely, since you're saying it works in the development environment)
I'm trying to get results of a facet query using solrj, but it seems it doesn't matter whether I add the facet query or not. I get the same document list anyway.
So this query returns the same document list...
query.setQuery(searchString);
query.setFacet(true);
query.addFacetField("CATNAME_STR");
query.addFacetQuery("CATNAME_STR:" + facetName);
...with this query
query.setQuery(searchString);
query.setFacet(true);
query.addFacetField("CATNAME_STR");
Only difference is I can get number of documents that matches the facet query with response.getFacetQuery();
I was expecting it to work like
http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=on&facet.field=CATNAME_STR&fq=CATNAME_STR:Erasmus
Any ideas?
Thanks.
By the way I'm using Solr Version 3.1.0 and solr-core-3.1.0
As it turns out fq=CATNAME_STR:Erasmus does not mean query.addFacetQuery("CATNAME_STR:Erasmus") but instead query.addFilterQuery("CATNAME_STR:Erasmus")