I have an ElasticSearch index which has a name with . (Example: my_index-2020.11.06-001). When I use SQL to get the count of all documents, I am getting the following error
curl --location --request POST '127.0.0.1:9200/_opendistro/_sql'
--header 'Content-Type: application/json'
--data-raw '{
"query": "SELECT count(*) FROM my_index-2020.11.06-001"
}'
Failed to parse query due to offending symbol [.11] at: 'SELECT count(*) FROM my_index-2020.11.06-001 ...
I have also tried to use backtick (`) and single quotes (') in the index name, that is not helping either
curl --location --request POST '127.0.0.1:9200/_opendistro/_sql'
--header 'Content-Type: application/json'
--data-raw '{
"query": "SELECT count(*) FROM `my_index-2020.11.06-001`"
}'
{
"error": {
"reason": "Invalid SQL query",
"details": "Field [my_index-2020.11.06-001] cannot be found or used here.",
"type": "SemanticAnalysisException"
...
Is there any other way to resolve this issue?
It is a bug but there is a workaround
Disable semantic analyzer
curl --location --request PUT 'http://localhost:9200/_cluster/settings' \
--header 'Content-Type: application/json' \
--data-raw '{
"transient": {
"opendistro.sql.query.analysis.enabled": "false"
}
}'
Related
I'm try to execute this
curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d'{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'
And when i do this directly from the shell, it gives me right output
curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d'{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'
{
"acknowledged" : true,
"persistent" : {
"cluster" : {
"routing" : {
"allocation" : {
"enable" : "primaries"
}
}
}
},
"transient" : { }
}
and here is my ansible shell task
- name: Turn off shard reallocation
shell: "curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d'{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'"
register: response
failed_when: response.stdout.find('"acknowledged":true') == -1
and it executes with error
ERROR! Syntax Error while loading YAML.
did not find expected key
The offending line appears to be:
- name: Turn off shard reallocation
shell: "curl -XPUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d '{"persistent" : {\"cluster.routing.allocation.enable" : "primaries"}}'"
^ here
Double quotes inside other double quotes must be escaped.
shell: "curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty -H 'Content-Type: application/json' -d '{\"persistent\": {\"cluster.routing.allocation.enable\": \"primaries\"}}'"
In such cases, you can ease your life and make things more readable by using a yaml folded scalar block
shell: >-
curl -X PUT 192.168.1.11:9200/_cluster/settings?pretty
-H 'Content-Type: application/json'
-d '{"persistent": {"cluster.routing.allocation.enable": "primaries"}}'
Meanwhile have a look at #Matt Schuchard comment and consider using the uri module instead of curl in shell.
I'm trying to get a cURL script to input my variables in a --data section. I'm fairly new to this, but it's just inserting the variables names.
The background is this script is called to hit an API in our ticketing system to create a new job. I am finding the ticket that is created has the subject "${DESCRIPTION}" and not "problem description".
#!/bin/bash
# This will log a ticket in Ticketing System
DESCRIPTION='Problem Description'
SUBJECT='Problem Subject'
curl --location --request POST 'https://XXXXX.domain.com/helpdesk/tickets.json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic XXXXXXXXXXXXXXXXXXXXXXXXXXXXX' \
--data '{
"helpdesk_ticket":
{
"description": "${DESCRIPTION}",
"subject": "${SUBJECT}",
"email": "email#domain.com",
"priority": 1,
"status": 2
},
"cc_emails": ""
}'
As per always, got this working just after I posted....
The solution was to wrap the variable in "'".
#!/bin/bash
# This will log a ticket in Ticketing System
DESCRIPTION='Problem Description'
SUBJECT='Problem Subject'
curl --location --request POST 'https://XXXXX.domain.com/helpdesk/tickets.json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic XXXXXXXXXXXXXXXXXXXXXXXXXXXXX' \
--data '{
"helpdesk_ticket":
{
"description": "'"$DESCRIPTION"'",
"subject": "'"$SUBJECT"'",
"email": "email#domain.com",
"priority": 1,
"status": 2
},
"cc_emails": ""
}'
I have two snapshots that I want to insert into elasticsearch in path:
C:\Users\name\Downloads\book_backup\agg_example
C:\Users\name\Downloads\book_backup\search_example
which I properly listed in elasticsearch.yml
path.repo: ["C:\\Users\\olulo\\Downloads\\book_backup\\agg_example", "C:\\Users\\olulo\\Downloads\\book_backup\\search_example"]
my elasticsearch starts fine and creating a new index works too.
Now when I try to insert snapshot into my elasticsearch so I can work on it:
curl -XPUT "http://localhost:9200/movies/" -d '{"type":"fs", "settings":{"location":"C:\\Users\\name\\Downloads\\book_backup\\search_example", "compress":true}}'
It gives me:
curl: (1) Protocol "'http" not supported or disabled in libcurl
curl: (3) [globbing] unmatched brace in column 10
curl: (3) [globbing] unmatched close brace/bracket in column 14
In https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-snapshots.html it seems like they've added a name at the end of path in location so I did
curl -XPUT "http://localhost:9200/movies/" -d '{"type":"fs", "settings":{"location":"C:\\Users\\name\\Downloads\\book_backup\\search_example\\test", "compress":true}}'
but still gives me same error.
following https://superuser.com/questions/1322567/http-not-supported-or-disabled-in-libcurl I've change everything to double quotes:
curl -XPUT "http://localhost:9200/_snapshot/javacafe" -d "{"type":"fs", "settings":{"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example", "compress":true}}"
giving me :
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}
Tried adding curl option as suggested Content-Type header [application/x-www-form-urlencoded] is not supported on Elasticsearch by
curl -XPUT "localhost:9200/_snapshot/javacafe" -H 'Content-Type: application/json' -d "{
"type":"fs",
"settings"{"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test", "compress":true}
}"
which outputs similar error with added statement at the end:
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}curl: (6) Could not resolve host: application
SOLVED : for windows you need to use backslash infront of double quotation mark inside {}:
curl -XPUT "localhost:9200/_snapshot/javacafe" -H "Content-Type: application/json" -d "{
\"type\":\"fs\",
\"settings\"{\"location\":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test", \"compress\":true}
}"
From my understanding windows use \ to consider " as it is. If so why not add backslash to all curl commands as well?
Try this:
curl -XPUT "localhost:9200/_snapshot/javacafe" -H 'Content-Type: application/json' -d '{
"type":"fs",
"settings" : {
"location":"C:\\Users\\olulo\\Downloads\\book_backup\\search_example\\test",
"compress":true
}
}'
I would like to make cURL/Elasticsearch understand HTTP query parameters passed as normal strings while being url encoded by the command.
If I run this HTTP GET via cURL to submit the query to Elasticsearch:
curl \
-H 'Content-Type: application/json' \
-XGET '127.0.0.1:9200/movies/movie/_search?q=%2Byear%3A%3E1980+%2Btitle%3Astar%20wars&pretty'
Then I am able to retrieve the expected documents.
However if I run this cURL query:
curl \
-H 'Content-Type: application/json' \
--data-urlencode "pretty" \
--data-urlencode "q=+year:>1980 +title:star wars&pretty" \
-XGET '127.0.0.1:9200/movies/movie/_search'
Then I get this error:
{
"error": {
"root_cause": [{
"type": "json_parse_exception",
"reason": "Unrecognized token 'pretty': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#7856627; line: 1, column: 8]"
}],
"type": "json_parse_exception",
"reason": "Unrecognized token 'pretty': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput#7856627; line: 1, column: 8]"
},
"status": 500
}
I am using:
cURL version 7.47.0 which should understand the command parameter --data-urlencode
Elasticsearch 6.3.1
--data-urlencode will send a POST and URL encode the body. You have to use -G or --get in order to send a GET request & append the data specified with --data-urlencode in the URL :
curl -G -v \
-H 'Content-Type: application/json' \
--data-urlencode "pretty=true" \
--data-urlencode "q=+year:>1980 +title:star wars" \
'127.0.0.1:9200/movies/movie/_search'
I know how to store a string as a variable, for example: API="http://localhost:4741"
However, for the sake of a CURL request I would like to be able to store on object as a variable that I can access values on, something like OBJ="{name : Joe}". Is this possible?
Right now my CURL request looks like this:
curl --include --request POST localhost:3000/scrape \
--header "Content-Type: application/json" \
--data '{
"url": "http://www.oddsshark.com/stats/gamelog/basketball/nba/20736",
"team": "LA Clippers"
}'
I would like to be able to do something like this, using a dictionary or an object:
TEAM=( ["Clippers"]="http://www.oddsshark.com/stats/gamelog/basketball/nba/20736" )
curl --include --request POST localhost:3000/scrape \
--header "Content-Type: application/json" \
--data '{
"url": "http://www.oddsshark.com/stats/gamelog/basketball/nba/20736",
"team": "${TEAM[Clippers]}"
}'