I'm trying to perform multiple concurrent search requests using Elasticsearch (version 6). Here is my queries file:
{"index" : "web"}
{"max_concurrent_searches": 64, "query": {"match": {"content": "school"}}}
{"index" : "web"}
{"max_concurrent_searches": 64, "query": {"match": {"content": "car"}}}
{"index" : "web"}
{"max_concurrent_searches": 64, "query": {"match": {"content": "cat"}}}
Here is the command I use to issue the bulk request:
curl -H "Content-Type: application/x-ndjson" -XGET ''$url'/_msearch'
--data-binary "#$file"; echo
However, I get the following error indicating my wrong usage of max_concurrent_searches parameter.
{"error":{"root_cause":[{"type":"parsing_exception","reason":"Unknown key for a VALUE_NUMBER in [max_concurrent_searches].","line":1,"col":29}],"type":"parsing_exception","reason":"Unknown key for a VALUE_NUMBER in [max_concurrent_searches].","line":1,"col":29},"status":400}
If I removed "max_concurrent_searches": 64, from the queries file above, everything works just perfect.
I want to know how can I use/set the max_concurrent_searches parameter, I couldn't find useful information in Elasticsearch documentation about this except the following:
The msearch’s max_concurrent_searches request parameter can be used to
control the maximum number of concurrent searches the multi search api
will execute. This default is based on the number of data nodes and
the default search thread pool size.
You should add it to the request itself:
Sample request: GET indexName/type/_msearch?max_concurrent_searches=1100
(where indexName and type is optional)
For you its should be like:
curl -H "Content-Type: application/x-ndjson" -XGET ''$url'/_msearch**?
max_concurrent_searches=1100**'
--data-binary "#$file"; echo
You can execute above using postman also. Just change the content-type as application/x-ndjson and dont forget to add a new line character in the end. This will give you the same error and you can correct it easily by different combinations. MultiSearch is an important feature.
Related
searchableAttributes, filterableAttributes, faceting.
I've read the documents, but a bit confused.
Please give some insights about:
Their differences.
Their relationships.
searchableAttributes are attributes where Meilisearch can search in for matching query words.
filterableAttributes are a list of attributes that can be used as filters to refine the search results. Given a movies dataset, if you want to filter movies by release_date, you need to first add the attribute release_date to the filterableAttributes list.
Both searchableAttributes and filterableAttributes are part of Meilisearch settings. An attribute doesn't necessarily have to be searchable for it to be filterable, so no relation between both.
facets is a search parameter, not a setting, it must be added to a search request. It gives information about the number of documents found per attribute value. If you want to know how many movies there are per genre for a given query, you can pass the "facets": ["genres"] as a parameter in the search query like so:
curl \
-X POST 'http://localhost:7700/indexes/movies/search' \
-H 'Content-Type: application/json' \
--data-binary '{
"q": "Batman",
"facets": ["genres"]
}'
The response should include a facetDistribution object with the information:
{
"hits": [
…
],
…
"facetDistribution": {
"genres": {
"action": 273,
"animation": 118,
"adventure": 132,
"fantasy": 67,
"comedy": 475,
"mystery": 70,
"thriller": 217
}
}
}
In order to have the facets information about an attribute, it must be first present in the filterableAttributes list.
I am trying to get all id's for a type, but I am pulling my hair out.
Please see my attacment.
HERE IS THE cURL call :
curl -XGET 'localhost:9200/_search?pretty' -H 'Content-Type: application/json'
-d'{ "query": { "wildcard" : { "id" : "Account[enter image description here][1]*" } }}'
cURL call with no results
I would guess there is an issue with the way your id-field is analyzed. You can retrieve the mapping by using the _mapping endpoint (described in the docs). Your id field should be analyzed as a string (with break characters, tokenizers and all) for the wildcard query to work. If it is not analyzed, as you might expect for an id-field, the wildcard query will not work. Then you would need to change the mapping and reindex your data to make it work.
I am looking for a way to pipeline multiple queries into Elastic search. My main problem is that when I receive the results I want to be able to know the which was the query that generated the result. In pseudo-code I would like to do something like following
query1="James Bond"
query2="Sean Connery"
query3="Charlie Chaplin"
pipeline=new ElasticSearchPipeline()
pipeline.add(query1);pipeline.add(query2);pipeline.add(query3)
pipeline.execute()
jamesBondResults=pipeline.getResultsForQuery(query1)
seanConneryResults=pipeline.getResultsForQuery(query2)
charleChaplinResults=pipeline.getResultsForQuery(query3)
The key feature is that I want to send avoid the overhead of sending multiple requests on the ES server, but still be able to treat the results as if I had sent the queries one by one.
The multi search API is exactly what you're looking for.
You can send many queries and the response will contain an array with the responses to each query in the same order:
curl -XPOST localhost:9200/_msearch -d '
{"index" : "test1"}
{"query" : {"match_all" : {}}, "from" : 0, "size" : 10}
{"index" : "test2",}
{"query" : {"match_all" : {}}}
'
The response array of the above multi search queries will contain two ES responses with the documents from the first and second queries.
I'm using Meteor (so Javascript, Node, NPM, etc) and would like to provide a simple text input for users to search via Elasticsearch. I would like to be able to use modifiers on the text like + and "" and search for a specific field. I'm looking for something that can convert a plain text input into Elasticsearch Query DSL.
These would be some example queries:
This query would mean that the keyword "tatooine" must exist:
stormtrooper +tatooine
This would mean that "death star" should be one keyword:
stormtrooper "death star"
This would search for the keyword "bloopers" only in the category field:
stormtrooper category=bloopers
Is there a library that can do this? Can a generic solution exist or is this why I can't find any existing answers to this?
simple_query_string would support your query syntax out of the box, except for category=bloopers which should be category:bloopers instead, but otherwise it should work:
curl -XPOST localhost:9200/your_index/_search -d '{
"query": {
"simple_query_string": {
"query": "stormtrooper category:bloopers"
}
}
}'
curl -XPOST localhost:9200/your_index/_search -d '{
"query": {
"simple_query_string": {
"query": "stormtrooper +tatooine"
}
}
}'
You can also send the query in the query string directly like this:
curl -XPOST localhost:9200/your_index/_search?q=stormtrooper%20%22death%20star%22"
I am fairly new to ElasticSearch and have a question on stop words. I have an index that contains state names for the USA....ex: New York/NY, California/CA,Oregon/OR. I believe Oregon's abbreviation, 'OR' is a stop word, so when I insert the state data into the index, I cannot search on 'OR'. Is there a way I can set up custom stopwords for this or am I doing something wrong?
Here is how I am building the index:
curl -XPUT http://localhost:9200/test/state/1 -d '{"stateName": ["California","CA"]}'
curl -XPUT http://localhost:9200/test/state/2 -d '{"stateName": ["New York","NY"]}'
curl -XPUT http://localhost:9200/test/state/3 -d '{"stateName": ["Oregon","OR"]}'
A search for 'NY', works fine. Ex:
curl -XGET 'http://localhost:9200/test/state/_search?pretty=1' -d '
{
"query" : {
"match" : {
"stateName" : "NY"
}
}
}'
But a search for 'OR', returns zero hits:
curl -XGET 'http://localhost:9200/test/state/_search?pretty=1' -d '
{
"query" : {
"match" : {
"stateName" : "OR"
}
}
}'
I believe this search returns no results because OR is stop word, but I don't know how to work around this. Thanks for you help.
You can (and definitely should) control the way you index data by modifying your mapping according to your data and the way you want to search against it.
In your case I would disable stopwords for that specific field rather than modifying the stopword list, but you could do the latter too if you wish to. The point is that you're using the default mapping which is great to start with, but as you can see you need to tweak it depending on your needs.
For each field, you can specify what analyzer to use. An analyzer defines the way you split your text into tokens (tokenizer) that will be indexed and also additional changes you can make to each token (even remove or add new ones) using token filters.
You can specify your mapping either while creating your index or update it afterwards using the put mapping api (as long as the changes you make are backwards compatible).