adding snapshot to elasticsearch Windows - elasticsearch

I have two snapshots that I want to insert into elasticsearch in path:
C:\Users\name\Downloads\book_backup\agg_example
C:\Users\name\Downloads\book_backup\search_example
which I properly listed in elasticsearch.yml
path.repo: ["C:\\Users\\olulo\\Downloads\\book_backup\\agg_example", "C:\\Users\\olulo\\Downloads\\book_backup\\search_example"]
my elasticsearch starts fine and creating a new index works too.
Now when I try to insert snapshot into my elasticsearch so I can work on it:
curl -XPUT "http://localhost:9200/movies/" -d '{"type":"fs", "settings":{"location":"C:\\Users\\name\\Downloads\\book_backup\\search_example", "compress":true}}'
It gives me:
curl: (1) Protocol "'http" not supported or disabled in libcurl
curl: (3) [globbing] unmatched brace in column 10
curl: (3) [globbing] unmatched close brace/bracket in column 14
In https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-snapshots.html it seems like they've added a name at the end of path in location so I did
curl -XPUT "http://localhost:9200/movies/" -d '{"type":"fs", "settings":{"location":"C:\\Users\\name\\Downloads\\book_backup\\search_example\\test", "compress":true}}'
but still gives me same error.
following https://superuser.com/questions/1322567/http-not-supported-or-disabled-in-libcurl I've change everything to double quotes:
curl -XPUT "http://localhost:9200/_snapshot/javacafe" -d "{"type":"fs", "settings":{"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example", "compress":true}}"
giving me :
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}
Tried adding curl option as suggested Content-Type header [application/x-www-form-urlencoded] is not supported on Elasticsearch by
curl -XPUT "localhost:9200/_snapshot/javacafe" -H 'Content-Type: application/json' -d "{
"type":"fs",
"settings"{"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test", "compress":true}
}"
which outputs similar error with added statement at the end:
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}curl: (6) Could not resolve host: application
SOLVED : for windows you need to use backslash infront of double quotation mark inside {}:
curl -XPUT "localhost:9200/_snapshot/javacafe" -H "Content-Type: application/json" -d "{
\"type\":\"fs\",
\"settings\"{\"location\":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test", \"compress\":true}
}"
From my understanding windows use \ to consider " as it is. If so why not add backslash to all curl commands as well?

Try this:
curl -XPUT "localhost:9200/_snapshot/javacafe" -H 'Content-Type: application/json' -d '{
"type":"fs",
"settings" : {
"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test",
"compress":true
}
}'

Related

Curl "(6) Could not resolve host: http"

I am trying to run this command
curl -XPUT -H "Content-Type: application/json" https://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
But this results in this error
curl: (6) Could not resolve host: http
curl: (3) unmatched close brace/bracket in URL position 5:
null}'
I got this command from here
https://stackoverflow.com/a/50609418/3259896
I'm am not yet very familiar with curl nor elasticseach, so I'm not sure how to pinpoint the error
If it makes a difference, I am using Windows.
I also tried
$ curl -X PUT -H "Content-Type: application/json" https://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
Edit2:
I tried
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
But now get this error
{"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#446f2dd5; line: 1, column: 2]"}],"type":"json_parse_exception","reason":"Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#446f2dd5; line: 1, column: 2]"},"status":400}curl: (3) unmatched close brace/bracket in URL position 5:
null}'
^
I then tried
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d "{"index.blocks.read_only_allow_delete": null}"
And I got
{"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unexpected character ('i' (code 105)): was expecting double-quote to start field name\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#696b36d3; line: 1, column: 3]"}],"type":"json_parse_exception","reason":"Unexpected character ('i' (code 105)): was expecting double-quote to start field name\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#696b36d3; line: 1, column: 3]"},"status":400}
First off, in that other SO question, the ES node was a live one so its URL included https.
Your localhost ist almost certainly not going to have https so change it to http:
curl -XPUT -H "Content-Type: application/json" \
http://localhost:9200/_all/_settings \
-d '{"index.blocks.read_only_allow_delete": null}'
Other than that, everything looks okay, provided that your ES is running.
When successful, you should receive a simple message:
{"acknowledged":true}
Edit: go with
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d "{\"index.blocks.read_only_allow_delete\": null}"

Ansible roles YAML errors

I'm try to execute this
curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d'{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'
And when i do this directly from the shell, it gives me right output
curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d'{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'
{
"acknowledged" : true,
"persistent" : {
"cluster" : {
"routing" : {
"allocation" : {
"enable" : "primaries"
}
}
}
},
"transient" : { }
}
and here is my ansible shell task
- name: Turn off shard reallocation
shell: "curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d'{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'"
register: response
failed_when: response.stdout.find('"acknowledged":true') == -1
and it executes with error
ERROR! Syntax Error while loading YAML.
did not find expected key
The offending line appears to be:
- name: Turn off shard reallocation
shell: "curl -XPUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d '{"persistent" : {\"cluster.routing.allocation.enable" : "primaries"}}'"
^ here
Double quotes inside other double quotes must be escaped.
shell: "curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d '{\"persistent\": {\"cluster.routing.allocation.enable\": \"primaries\"}}'"
In such cases, you can ease your life and make things more readable by using a yaml folded scalar block
shell: >-
curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty
-H 'Content-Type: application/json'
-d '{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'
Meanwhile have a look at #Matt Schuchard comment and consider using the uri module instead of curl in shell.

how to pass arguments in curl command

i am getting the below error while executing the curl command in unix. Not sure if this is the right way to put the arguments in curl. Can you please advise on this?
curl -X POST -H Authorization:'Bearer AAEAAJ6ZNfGbzJkSuJ1o5rXLBec5Q' -H 'Content-Type: application/json' -d {"Filter": {"ClientName": "ABC","WorkflowName": "sk_lask"},"SortingName": "StartDate","SortingOrder": "Desc"} http://someaddress.com/api/status/search HTTP/1.1
error
curl: (3) [globbing] unmatched brace at pos 13
curl: (6) Could not resolve host: ABC,WorkflowName; Unknown error
curl: (3) [globbing] unmatched close brace/bracket at pos 9
curl: (6) Could not resolve host: StartDate,SortingOrder; Unknown error
curl: (3) [globbing] unmatched close brace/bracket at pos 5
You need to quote your header, payload and also remove HTTP/1.1 from end as this command:
curl -X POST -H 'Authorization: Bearer AAEAAJ6ZNfGbzJkSuJ1o5rXLBec5Q' \
-H 'Content-Type: application/json' \
-d '{"Filter": {"ClientName": "ABC","WorkflowName": "sk_lask"},"SortingName": "StartDate","SortingOrder": "Desc"}' \
'http://someaddress.com/api/status/search'

Content-Type header [application/x-www-form-urlencoded] is not support error when import json data into elastic

MY Code below
curl -XPOST "http://localhost:9200/test/p
ost/_bulk/" -d #City_collection.json
error:
{"error":"Content-Type header [application/x-www-form-urlencoded] is not support
ed","status":406}
You need to specify the content-type header like this
curl -XPOST "http://localhost:9200/test/post/_bulk/" -H "Content-Type: application/json" -d #City_collection.json
^
|
add this

Error: index_not_found_exception

I use ELK stack to analyze my log file. I have tested last week and everything works well.
Today, I tested but I get this error when I typed "http://localhost:9200/iot_log/_count" (iot_log is my index pattern):
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no
such
index","resource.type":"index_or_alias","resource.id":"iot_log","index_uuid":"na","index":"iot_log"}],"type":"index_not_found_exception","reason":"no such
index","resource.type":"index_or_alias","resource.id":"iot_log","index_uuid":"na","index":"iot_log"},"status":404}
I really searched the forums but I have not found a solution, I want to know what is the cause of this problem please and how can I correct it?
Make sure index iot_log exist and create it if not:
curl -X PUT "localhost:9200/iot_log" -H 'Content-Type: application/json' -d'{ "settings" : { "index" : { } }}'
You need to set your action.auto_create_index parameter in elasticsearch.yml file.
Example:
action.auto_create_index: -l*,+z*
With this kind of configuration, indexes starting with "z" will be created automatically while indexes starting with "l" will not.
The best way to resolve it by using setting as follow
Allow Auto Create YourIndexName and index10 and not allowing any index name matching index1* and any other index matching ind*. The patterns are matched in the order they are given.
curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'{
"persistent": {
"action.auto_create_index": "YourIndexName,index10,-index1*,+ind*"
}
}'
Stop any Auto Indexing
curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'{
"persistent": {
"action.auto_create_index": "false"
}
}'
Allow any Index create automatically
curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'{
"persistent": {
"action.auto_create_index": "true"
}
}'
```
In my case, My all data is DELETED in elastic search automatically, After importing data again in elastic search my application working good.

Resources