ES 7.17 cloud is hosted in https://xxx.elastic-cloud.com:9243]
Using Java High Level REST client, I was able to connect to server using spring data ES 4.1.5(ES client 7.9.3). Now I update to spring data ES 4.4.1 (ES client 7.17.4). But it gives me permission issue.
"root_cause":[{"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user [xxxx] with roles
Why does it need this permission?
It's because the client pings the cluster using the / endpoint and it's actually why the cluster:monitor/main privilege is required.
I believe this was added after the 7.10.2 Opensearch fork.
Val is right. / endpoint requests are being sent out.
I found out more details for my issue.
HEAD / and GET / in spring data ES 4.4.1
Related
we have a security requirement to use private link when connecting to Elastic cluster in elastic cloud.
However when using private link our old solution using Cloud Id connection won't work anymore. We tested it and it is confirmed in this discussion - link
We could switch to using Uri connection however I noticed this sentence in the official docs - link
We recommend using a Cloud ID whenever possible because your client
will be automatically configured for optimal use with Elastic Cloud,
including HTTPS and HTTP compression.
This would be the connection setting we would be using
ConnectionSettings settings =
new ConnectionSettings(new Uri("https://someUri.privatelink.westeurope.azure.elastic-cloud.com:443"))
.DefaultIndex(DefaultIndex)
.EnableDebugMode()
.MaximumRetries(MaxRetries)
.MaxRetryTimeout(MaxRetryTimeout)
.ApiKeyAuthentication(new ApiKeyAuthenticationCredentials("someApiKey"))
.EnableApiVersioningHeader();
Now I'm not sure what the authors in the official docs meant by optimal use with Elastic cloud.
Are there any settings you can think of we should add here for production ready scenarios ? Are there some things that might not work if we go with out the use of Cloud Id ?
I am currently getting backwards incompatibility issue when hitting endpoint of /update using elastic.v7 as client to Elasticsearch 6.x server.
I noticed that for update, the 7.x endpoint is POST /<index>/_update/<_id>, while the 6.x is expecting POST /<index>/_doc/<_id>/_update.
This results in that when I make update operation from v7 client to 6.x Elasticsearch server, I am seeing the error below
{"error":{"root_cause":[{"type":"invalid_type_name_exception","reason":"Document mapping type name can't start with '_', found: [_update]"}],"type":"invalid_type_name_exception","reason":"Document mapping type name can't start with '_', found: [_update]"},"status":400}
I found this difference when I enabled client side tracelog.
Version of Elasticsearch used
elastic.v7 as client (Elasticsearch 6.x server as a way to test backwards compatibility)
Expected behavior
Successful update with POST /<index>/_update/<_id> from v7 client to both Elasticsearch 6.x and 7.x servers.
Actual behavior
Error described above with POST /<index>/_update/<_id> endpoint from v7 client to Elasticsearch 6.x server.
Steps to reproduce the behavior
Spin up a ES 6 server
Use elastic v7 as client
Create an index
Insert a document
Update the document with POST /<index>/_update/<_id> endpoint from elastic.v7
Is this a suggestion of not to use v7 client for 6.x server? Originally I am thinking of sticking this plan as it would be easier to use one client to request to both Elasticsearch servers (v6 and v7).
Is it possible to write DB connector using Knox Provider extension?
I want Knox to expose endpoint in response to which Knox would put to or get from database records.
Depends on how you want to connect and what functionality you want to add. Currently, Knox support HTTP and Websockets protocols so if you are using JDBC to connect then that will not work. If you are using JDBC over HTTP (e.g. beeline) then that might work. Here are few links to Knox documentation
User Guide
Dev Guide
I've started learning elasticsearch using AWS I followed this article to create elasticsearch and Kibana setup
After completing the setup, tried to log in using my user from user pool, after login I'm getting error "Missing Role"
url https://search-mytestdomain-xxxxxxxxxxxxxxxxxxxxxx.us-east-2.es.amazonaws.com/_plugin/kibana/customerror?type=missingRole#?_g=()
Was looking all over for the fix after following the same guide you linked. Came across this post during my searching and wanted to update my fix for that specific guide is to use Elasticsearch 6.2 when creating the domain.
I haven't tried any other versions but when I switched from ES 7.4 (latest) to 6.2 (which is mentioned in the guide) it worked with no issues.
What worked for me is below on ES 7.4,
Open Identity Pool
Click "Edit identity pool"
Authentication Providers
Authenticated Role Section
Choose role from token
Role resolution: DENY
This document had the answer that worked for me:
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/fgac.html#fgac-walkthrough-iam
The real issue was that the role I was using on the identity pool and elasticsearch cluster did not have a trust relationship policy to allow the identity pool to assume the role. So login was succeeding, but the pool wasn't able to assume the role, and therefore there really was a missing role on the token that Kibana was receiving. Adding the trust relationship policy shown in step 4 of the doc in addition to the policy allowing kibana to assume the role, fixed the missing role issue for me and successfully passed my authenticated user into kibana's main interface.
Basic username/password authentication for ELK, i was able to achieve using searchguard on windows platform.
Now i am trying to establish secure communication. I have performed the following changes,
In elasticsearch.yml
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks
searchguard.ssl.http.keystore_password: 221749a2add117cf889f
searchguard.ssl.http.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks
searchguard.ssl.http.truststore_password: 6d6cf1cc017dc874960b
searchguard.authcz.admin_dn:
- CN=sgadmin
searchguard.ssl.transport.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks
searchguard.ssl.transport.keystore_password: 221749a2add117cf889f
searchguard.ssl.transport.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks
searchguard.ssl.transport.truststore_password: 6d6cf1cc017dc874960b
In Kibana.yml
elasticsearch.url: "https://localhost:9200"
elasticsearch.username: "admin"
elasticsearch.password: "admin"
If i login to kibana, using http://localhost:5601, it asks for username and password. I dont know what credentails to enter here. i tried admin/admin. Its not working. Before i gave searchguard.ssl.http.... configurations, admin/admin was working fine.
After i added all the searchguard.ssl.http related configuration, the credentails are not working.
I am sure there is some other configuration in kibana.yml w.r.t searchguard configuration. I am not able to find it online. Can any one please help me here on what is missing.
Enabling TLS on the REST layer does not have any impact on user authentication/authorisation. The only difference is:
If you enable TLS on REST layer, only HTTPS access is allowed. You will see an error message if you try to access ES with HTTP
If you disable TLS on REST layer, only HTTP access is allowed. You will see an error message if you try to access ES with HTTPS.
Search Guard will authenticate the credentials against the configured authentication backend in sg_config.yml. If you use the default configuration that ships with Search Guard, it will use the internal user database and the users defined in internal_users.yml. The default "admin" user has full access to all indices and types, so you should be able to use this user to log into Kibana. If you need a user with limited access, the corresponding demo user is "kibanaro". Please refer to internal_users.yml to see all demo users.
Since ES 5.0.2, you need to install the Search Guard Kibana plugin for full authentication and session support. You can find the plugin on the corresponding GitHub repository. You install it like any other Kibana plugin, for example:
bin/kibana-plugin install https://github.com/floragunncom/search-guard-kibana-plugin/releases/download/v5.4.3-3/searchguard-kibana-5.4.3-3.zip
If you do not need advanced stuff like multi-tenancy or JWT Single Sign-on, it should start and work out of the box.
If this does not help, please post the output of your Elasticsearch log files when trying to log in.
Disclaimer: I work for floragunn, makers of Search Guard.