I'm querying ES to get me list of documents within some specific timestamp. Now I encountered a scenario where we had multiple versions of single documentId. By default ES returned me all the versions of that single documentId. My requirement is to get only one last version of all the docs.
Also I wanted to get all the ES response sorted in Ascending order of one the indexed timestamp field(called as streamingSegmentStartTime)
my current query looks like following :
{"size":25,"query":{"bool":{"must":[{"terms":{"streamingSegmentId":["00002933-be25-3b9c-9970-472b41aa53cc"],"boost":1.0}},{"range":{"streamingSegmentStartTime":{"from":1644480000000,"to":1647476658447,"include_lower":true,"include_upper":false,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},"_source":{"includes":["errorCount","benefitId","streamingSegmentStopTime", "fanoutPublishTimestamp", "search.version"],"excludes":[]},"sort":[{"streamingSegmentStartTime":{"order":"asc"}}, {"_timestamp": {"order": "desc"}}]}
try to use the collapse parameter to collapse search results based on field values
https://www.elastic.co/guide/en/elasticsearch/reference/8.1/collapse-search-results.html#collapse-search-results
Related
I am using olivere/elastic library for elasticsearch in my go app . I have list of values for a particular field (say fieldA) of elasticsearch document. I want to update a particular field of all document by searching on field fieldA .
This : Updating a record in ElasticSearch using olivere/elastic in google go
explains the update part. But in my case in don't have Id of documents to be updated . So, either i can make search call to retrieve document ids and then update them , or is there another way am missing? Thanks in Advance.
If you need to update a list of documents, you can use the Update By Query API. The unit tests give you a hint about how the syntax looks like. However, if you have individual values for individual documents, I guess there's no other way than updating them one by one. The fastest way to achieve that is by using the Bulk API.
I'm using Elastic Search 5.6.2 with Kibana and I'm currently facing a problem
My documents are indexed on the field timestamp which is normally an integer, however recently somebody has logged a document with a timestamp that is not an integer, and Kibana complains of conflicting type.
The discover panels display nothing and the following errors pop:
Saved "field" parameter is now invalid. Please select a new field.
Discover: "field" is a required parameter
How can I look for the document(s) causing these conflicts so that to find the service creating bad logs ?
The field type (either integer or text/keyword) is not defined on per document basis but rather on per index basis (in the mappings). I guess you are manipulating timeseries data, and you probably have un index per day (or per month or ...).
In Kibana Dev Tools:
List the created indices with GET _cat/indices
For each index (logstash-2017.09.28 in my example) do a GET logstash-2017.09.28/_mapping and check the type of the field in #timestamp
The field type is probably different between indices.
You won't be able to change the field type on created indices. Deleting document won't solve you're problem. The only solution is to drop the index or reindex the whole index with a new field type (in a specific mapping).
To avoid this problem on future indices, the solution is to create an index template with a mapping telling that the field #timestamp is of type date or whatever.
I'm using Solr to run a query on one of our cores. Suppose my documents have two fields: ID, and Name. I also have a separate list of IDs I'm grabbing from a database and passing into the query to boost certain results.
If the document gets returned in the query and the ID is in the list it goes to the top of the results, and if it gets returned in the query and the ID is not in the list then it goes below those that are in the list. The former is from the "boost". My query is something like this -
http://mysolrserver:8983/solr/MyCore/MyQueryHandler?q=Smith&start=0&rows=25&bq=Id%3a(36+OR+76+OR+90+OR+224+OR+391)
I am able to get the boost query working but I need the boosted results to be in alphabetical order by name, then the non boosted results under that also in alphabetical order by name. I need to know what to user for the &sort= parameter.
&sort=score%20desc,Name+asc does not work.
I've looked over a lot of documentation, but I still don't know if this even possible. Any help is appreciated. Thanks!
Solr version is 6.0.1. I am actually using SolrNet to interface with Solr, but I think I can figure out the SolrNet part if I know what the url's &sort= parameter value needs to be.
I figured it out, by doing away with the boost query. I added a sort query using the "exists" function and passing it a sub-query for the ID. The exists returns a boolean value to sort on, then I added the name as a second sort. It works perfect!!
The URL looks like this:
http://mysolrserver:8983/solr/MyCore/MyQueryHandler?q=Smith&start=0&rows=25&sort=exists(query({!v=%27Id:(36+OR+76+OR+90+OR+224+OR+391)%27}))%20DESC,%20Name%20ASC
The closest match to your requirement is the query elevation component[1] .
In your particular case I would first sort my Ids according to my requirements ( sorting them by name for example), then maintain them in the elevate.xml.
At query time you can use the "forceElevation" parameter to force the elevation and then sort the remaining results by name.
[1] https://cwiki.apache.org/confluence/display/solr/The+Query+Elevation+Component
I'm performing a multimatch search against an ElasticSearch index, and I want to get back the source object with fields in the same order as they were stored in.
However, when I get the response back from the ElasticSearch query, the fields are in alphabetical order (which is not particularly useful for what I'm doing). I'm fairly confident that it used to behave the desired way in a previous version of ES, but since I upgraded recently it is only returning the fields in alphabetical order.
Edit: Note that if I perform a standard match_all search, then I do get the fields back in the original order. I wonder if it has something to do with the multimatch query?
Edit 2: OK, I just ran it again and it returned the fields in a random order (not alphabetical). Maybe this is a bug in ElasticSearch?
You cannot guarantee any order in what is returned. The source document is a plain old JSON object and by definition:
An object is an unordered set of name/value pairs.
I'm trying to get results of a facet query using solrj, but it seems it doesn't matter whether I add the facet query or not. I get the same document list anyway.
So this query returns the same document list...
query.setQuery(searchString);
query.setFacet(true);
query.addFacetField("CATNAME_STR");
query.addFacetQuery("CATNAME_STR:" + facetName);
...with this query
query.setQuery(searchString);
query.setFacet(true);
query.addFacetField("CATNAME_STR");
Only difference is I can get number of documents that matches the facet query with response.getFacetQuery();
I was expecting it to work like
http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&facet=on&facet.field=CATNAME_STR&fq=CATNAME_STR:Erasmus
Any ideas?
Thanks.
By the way I'm using Solr Version 3.1.0 and solr-core-3.1.0
As it turns out fq=CATNAME_STR:Erasmus does not mean query.addFacetQuery("CATNAME_STR:Erasmus") but instead query.addFilterQuery("CATNAME_STR:Erasmus")