How do i prevent elasticsearch's _analyze from interpretting yml - elasticsearch

I'm trying to use the _analyze api with text that looks like this:
--- some -- text ---
This request works as expected:
curl localhost:9200/my_index/_analyze -d '--'
{"tokens":[]}
However, this one fails:
curl localhost:9200/medical_documents/_analyze -d '---'
---
error:
root_cause:
- type: "illegal_argument_exception"
reason: "Malforrmed content, must start with an object"
type: "illegal_argument_exception"
reason: "Malforrmed content, must start with an object"
status: 400
Considering the formatting of the response, i assume that elasticsearch tried to parse the request as yaml and failed.
If that is the case, how can i disable yml parsing, or _analyze a text that starts with --- ?

The problem is not the yaml parser. The problem is that you are trying to create a type.
The following is incorrect(will give you Malforrmed content, must start with an object error)
curl localhost:9200/my_index/medical_documents/_analyze -d '---'
This will give you no error, but is incorrect. Because it will tell elastic to create a new type.
curl localhost:9200/my_index/medical_documents/_analyze -d '{"analyzer" : "standard","text" : "this is a test"}'
Analyzers are created Index level. verify with:
curl -XGET 'localhost:9200/my_index/_settings'<br/>
So the proper way is:
curl -XGET 'localhost:9200/my_index/_analyze' -d '{"analyzer" : "your_analyzer_name","text" : "----"}'
Previously need to create the analyzer.

Related

What is causing the Elasticsearch bulk load to fail?

I can't figure out why I can't bulk load Elasticsearch with JSON. I've done this before but this time I am totally stumped.
I have processed a set of JSON documents into Elastic Bulk Load Format and am trying to bulk load the index I just created (verified created, can be queried, and is empty).
{"create": {"_id": "ef68e997-c616-4b0b-b08e-dfc09f8cb08f"}}
{"id": "ef68e997-c616-4b0b-b08e-dfc09f8cb08f", "title": "My document"}
... repeats for all records
The command I run uses a list of paths to the JSON bulk files and a loop to curl/POST them to Elastic using credentials:
while IFS= read -r "path" < "${DOC_LIST_PATH}"
do
echo "Submitting Elastic formatted docs at ${path} to Elastic index 'docs' ..."
curl \
-X POST \
-H "Content-Type: application/x-ndjson" \
"https://${ES_USER}:${ES_PASSWD}#${ES_HOSTNAME}:${ES_PORT}/docs/_bulk" \
--data-binary "#${path}"
done
I've done all this before and it should work but... it doesn't. I get this error instead:
Submitting Elastic formatted docs at data/docs.json/part-00000.json to Elastic index 'docs' ...
Warning: Couldn't read data from file
Warning: "data/docs.json/part-00000.json",
Warning: this makes an empty POST.
{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}
... repeats for all files
I have found that the problem is with this bash code, not the data or the bulk load request:
--data-binary "#${path}"
If I replace that with this, it works:
--data-binary "#data/docs.json/part-00000.json"
Making the full working command for a single file:
curl -X POST -H "Content-Type: application/x-ndjson" "https://${ES_USER}:${ES_PASSWD}#${ES_HOSTNAME}:${ES_PORT}/docs/_bulk" --data-binary "#data/docs.json/part-00000.json"
But I need to script this, so this is still maddening. Please help!
This example is also in a gist here

How to create index and type in elastic search?

I have installed elasticsearch version 2.3.2. I have to add index and type to that elasticsearch. Before I used sense plugin to achieve this. But the addon was removed from webstore. Please give suggestion.
Sense plugin is now a Kibana app. Please refer official reference for installation.
The answer of your question is, you can create index and type in Elasticsearch by running below curl command
curl -XPUT "http://localhost:9200/IndexName/TypeName"
You can use a Rest client like postman to do this. You can get the postman as a chrome extension.
The other way is to do an SSH into one of the nodes in your cluster and run the POST command using CURL.
`curl -X POST 'localhost:9200/bookindex/books' -H 'Content-Type: application/json' -d'
{
"bookId" : "A00-3",
"author" : "Sankaran",
"publisher" : "Mcgrahill",
"name" : "how to get a job"
}'
I will automatically create an index named 'bookindex' with type 'books' and index the data. If index and type already exist it will add the entry to the index.
All operations in Elasticsearch can be done via REST API calls.
To create an index use the index API
curl -XPUT 'localhost:9200/twitter?pretty' -H 'Content-Type: application/json' -d'{"settings" : {"index" : {"number_of_shards" : 3, "number_of_replicas" : 0 }}}'
To create the mapping the you can use the _mapping endpoint-
curl -XPUT http://localhost:9200/twitter/tweets/_mapping -d #"create_p4_schema_payload.json"
Here,mapping is provided via a json file name create_p4_schema_payload.json which contains the following-
{
"properties": {
"user_name": {
"type": "text"
}
}
}
All these can be run via any terminal which supports curl. For windows, you may install cygwin to run linux command from command prompt.
Like it was said above, you can access it through REST api calls. The command you need to run is:
curl -XPUT 'http://localhost:9200/IndexName?include_type_name=TypeName'
CURL is a raw text that can be imported into Postman, for example, or you can install it's CLI and simply run it. Simply put:
It's a PUT api call to the ElasticSearch/IndexName, adding the Query Parameter include_type_name.
The reference guide is at: Elastic Search - Create index API
Sense plugin is removed from chrome webstore. You could use Kibana which has sense like dev-tool to perform ElasticSearch queries.
Follow this link to install kibana.

Deleting a type in Elastic Search using curl

I am trying to delete a type in elastic search using curl script in bat file
ECHO Running Curl Script
curl -XDELETE "http://localhost/testing/" -d''
pause
The response that i got was No handler found for uri . I looked into documentation of Elastic Search and it says to use delete by query https://www.elastic.co/guide/en/elasticsearch/reference/5.0/docs-delete-by-query.html
How can i modify the my curl script to use this new api for ES 2.3
Thanks
If you want to use the delete-by-query API to delete all documents of a given type, you can do it like this:
curl -XDELETE "http://localhost/testing/_query?q=_type:typename"
However, you're better off deleting the index and recreating it so you can modify the mapping type as you see fit.
curl -XDELETE "http://localhost/testing/"
curl -XPUT "http://localhost/testing/" -d '{"settings": {...}, "mappings": {...}}'

Error on inserting data to InfluxDB

I am new to influxdb.
By learning from the official site of influx, I have entered the line
curl -i -XPOST "http://localhost:8086/write?db=mydb" --data-binary "cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000"
but I am getting error :
invalid character:"//" looking for beginning of value
any solution ?

msearch not working with bool must

Elasticsearch version - .90.1
The following works perfectly.
cat names
{"index":"events","type":"news"}
{"query":{"term":{"Type":"MarketEvent"}}}
{"query":{"term":{"Type":"MarketEvent"}}}
curl -XGET 'http://localhost:9200/_msearch' --data-binary #names
The following also works
{"index":"events","type":"news"}
{"query":{"bool":{"must":[{"query_string":{"query":"*","fields":["Events.Event"],"default_operator":"AND"}},{"term":{"Type":"MarketEvent"}}]}}}
But queries with more than 1 bool doesn't work -
cat names
{"index":"events","type":"news"}
{"query":{"bool":{"must":[{"query_string":{"query":"*","fields":["Events.Event"],"default_operator":"AND"}},{"term":{"Type":"MarketEvent"}}]}}}
{"query":{"bool":{"must":[{"query_string":{"query":"*","fields":["Events.Event"],"default_operator":"AND"}},{"term":{"Type":"MarketEvent"}}]}}}
curl -XGET 'http://localhost:9200/_msearch' --data-binary #names
{"error":"must doesn't support arrays"}
Am not seeing any log for this in logs ( not in DEBUG mode too)
Is this a bug ?
The _msearch query should have the following format:
header\n
body\n
header\n
body\n
In the first and the last queries the second header is missing. The error is not generated for the first query only because of the way the header is parsed. For this query to work the names file should be changed into
{"index":"events","type":"news"}
{"query":{"bool":{"must":[{"query_string":{"query":"*","fields":["Events.Event"],"default_operator":"AND"}},{"term":{"Type":"MarketEvent"}}]}}}
{"index":"events","type":"news"}
{"query":{"bool":{"must":[{"query_string":{"query":"*","fields":["Events.Event"],"default_operator":"AND"}},{"term":{"Type":"MarketEvent"}}]}}}

Resources