How to pass Authorisation token for all elasticsearch requests - elasticsearch

I've setup a local elasticsearch node with kibana and have set
xpack.security.enabled: true
I require this as i plan to further add roles and I want to add document level security.
Both Kibana & Elastic search are up and running at their respective ports. I am struggling to understand how to I add a new document operating via console.
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'
This fails with
{
"error" : {
"root_cause" : [
{
"type" : "security_exception",
"reason" : "missing authentication credentials for REST request [/customer/_doc/1?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
}
}
],
"type" : "security_exception",
"reason" : "missing authentication credentials for REST request [/customer/_doc/1?pretty]",
"header" : {
"WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\""
}
},
"status" : 401
}
Fair , I understand I need to get an auth token to pass to the curl. Which API retrives this and where can i find an example. I've tried this unsucessfully.
curl -X POST "localhost:9200/_security/oauth2/token?pretty" -H 'Content-Type: application/json' -d'
{
"grant_type" : "client_credentials"
"username" : "elastic",
"password" : "VKZjNLBVBSeLS08sHDIN"
}
'
The password was generated automatically while bringing up the node.
Thanks!

By curl:
curl -X PUT --basic -u {user}:{password} http://localhost:9200/customer/_doc/1?pretty
By http request in program:
1. Make the the base64 encoding for the string: ${user}:${password}
2. Add http header
key:"Authorization"
value:"Basic ${encode value in step 1}"
By curl with header:
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Authorization:Basic ********' -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
******** is the value of base64 encoding for the string: ${user}:${password}

Related

Why do I have to PUT new documents to a nested URI, if mapping types have been removed?

I'm on Elasticsearch 7.14.0 where mapping types have been removed.
If I run the following:
curl -X PUT "localhost:9200/products/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Toast"
}
'
I get
{
"error" : "Incorrect HTTP method for uri [/products/1?pretty] and method [PUT], allowed: [POST]",
"status" : 405
}
It seems that elastic wants me PUT it in an /index/type/ URI:
curl -X PUT "localhost:9200/pop/products/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Toast"
}
'
{
"_index" : "pop",
"_type" : "products",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
I am wondering why I must have a nested URI indicating a type, if mapping types have been removed?
You have to add _doc to your put request call as shown below
curl -X PUT "localhost:9200/products/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Toast"
}
'
As mentioned in elasticsearch official documentation after mapping types were removed in 7.x, you need to add , _doc (which does not represent a document type rather it represents the endpoint name) for the document index, get, and delete APIs

Hasura: Change permissions/schema in Hasura on runtime (dynamically) through API call

I'm trying to change permissions in Hasura on runtime (dynamically) through API call as follows:
curl --location --request POST 'https://hasura-0ccfcde0.nhost.app/v1/query' \
--header 'x-hasura-role: admin' \
--header 'Content-Type: application/json' \
--data-raw '{
"type" : "create_update_permission",
"args" : {
"table" : "customers",
"role" : "users",
"permission" : {
"check" : {
"user_id" : {
"_ne": ""
}
},
"filter" : {
"user_id" : "X-Hasura-User-Id"
},
"set":{
"name":"X-Hasura-User-Id"
},
"columns":["name","email"]
}
}
}'
But it returns with
{
"path": "$.args",
"error": "restricted access : admin only",
"code": "access-denied"
}
I'm following: https://hasura.io/docs/1.0/graphql/core/api-reference/schema-metadata-api/permission.html#create-update-permission
as you can see in the screenshot as well
It's not enough for you to just put admin in the x-hasura-role Header. If you think about it, this would be a huge vulnerability as anyone could issue this request against your backend. You've even shared the endpoint here in your example CURL request!
Instead, you need to pass the x-hasura-admin-secret Header with the value you've configured for your deployment. This value should be extremely guarded as anyone who has access to it has pretty wide open access to Hasura and potentially your entire Database at that point depending on what your Hasura configuration looks like
curl --location --request POST 'https://hasura-0ccfcde0.nhost.app/v1/query' \
--header 'x-hasura-admin-secret: whatever-your-configured-admin-secret-is' \
--header 'Content-Type: application/json' \
--data-raw '{
"type" : "create_update_permission",
"args" : {
"table" : "customers",
"role" : "users",
"permission" : {
"check" : {
"user_id" : {
"_ne": ""
}
},
"filter" : {
"user_id" : "X-Hasura-User-Id"
},
"set":{
"name":"X-Hasura-User-Id"
},
"columns":["name","email"]
}
}
}
Alternatively, making a call with a JWT that is signed with the admin role could also work if you don't want to directly use your Hasura secret. In this case you would set it in the Authorization header with the x-hasura-role: admin claim set

Using Curl to put data into ES and got Unexpected character ('n' (code 110))

I'm using Curl to put data into ES. I have already created a customer index.
The following command is from ES document.
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'
When I do this, I get an error.
{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "failed to parse"
}
],
"type" : "mapper_parsing_exception",
"reason" : "failed to parse",
"caused_by" : {
"type" : "json_parse_exception",
"reason" : "Unexpected character ('n' (code 110)): was expecting double-quote to start field name\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper#1ec5236e; line: 3, column: 4]"
}
},
"status" : 400
}
I think, the below is the main reason of my error.
reason" : "Unexpected character ('n' (code 110)): was expecting double-quote to start field name
I have a feeling that I need to use (backslash) to escape. However, my attempt \' is not working great. Any advice?
I made it work like the below.
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d '
{
\"name\": \"John Doe\" <==== I used "backslash" in front of all the "
}
'
Answer without my comment:
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d '
{
\"name\": \"John Doe\"
}
'

Error while sending data into Elasticsearch

While using Elasticsearch to load datasets with curl command->
curl -H "Content-Type: application/x-ndjson" -XPOST "localhost:9200/shakespeare/doc/_bulk?pretty" --data-binary #$shakespeare_6.0
Following warning is encountered->
Warning: Couldn't read data from file "$shakespeare_6.0", this makes an empty
Warning: POST.
{
"error" : {
"root_cause" : [
{
"type" : "parse_exception",
"reason" : "request body is required"
}
],
"type" : "parse_exception",
"reason" : "request body is required"
},
"status" : 400
}
My data is:
{"index":{"_index":"shakespeare","_id":0}}
{"type":"act","line_id":1,"play_name":"Henry IV", "speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"}
What is the root cause of this warning? I am using 64 bit Windows 10.
Also, Please let me know what are the different ways to send the data into the elasticsearch? I am a noob.
You provided a wrong file name. The name of that file is shakespeare_6.0.json, not $shakespeare_6.0. This is the correct command:
curl -H "Content-Type: application/x-ndjson" -XPOST "localhost:9200/shakespeare/doc/_bulk?pretty" --data-binary #shakespeare_6.0.json
This assumes that the file is in the current directory.

Set up Flume HTTP agent on a URL?

I've been able to set up my HTTP source on a host and port like:
agent.sources=s1
...
agent.sources.s1.type=http
agent.sources.s1.bind=0.0.0.0
agent.sources.s1.port=5140
And I can, for example, POST a json document to it via:
curl -X POST -H 'Content-Type: application/json; charset=UTF-8' -d '[{
"headers" : { "ip" : "192.168.1.102", "host" :
"random_host.example.com" }, "body" : "random_body" }, { "headers" : {
"ip" : "192.168.1.102", "host" : "random_host.example.com" }, "body" :
"really_random_body" }]' http://hostname:port
However I would like to be able to POST a Json document to http://hostname.com:port/a/b/c/
How may I do this?

Resources