I am facing an issue while connecting filebeats with my trial version of elastic cloud.
I am getting below error
ERROR pipeline/output.go:100 Failed to CLOUD_ID): 401 Unauthorized:
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user
[USER_EMAIL] for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\"
charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":
"unable to authenticate user ...
I have attached my filebeat.yml too. Please look.
I crossed check my username and password both are correct.
please help!
Here my cloud credentials don’t look right. The syntax for cloud.auth is user:password
It needs to be the cluster credentials, not the cloud account credentials. (Default username is elastic, and you’re presented a randomly generated password when the cluster is created)
Related
I'm running two keycloak docker instances and configured cluster as specified here https://hub.docker.com/r/jboss/keycloak/
I can able see logs related to clustering and two records in JGROUPSPING table. Also it works when authenticate(openid-connect) through Host1 and get access token/refresh token and able to retrieve new access_token using refresh token via Host2, which means I believe clustering setup is working.
But Im getting 401 error when I make API call to Host2 either using the access token I received from Host1 or access_token I got from Host1's refresh token. It works only when I use access_token received from same host.
My understanding is that these access_tokens doesn't related to cookie it should be working seamlessly. But it fails.
I had a problem with the verification of the access token signature.
The access token are signed by Keycloak with a keystore. If you don't have a certificate and key mounted in the docker, this keystore will be different between the nodes in your cluster, and a token generated by one node will not be valid for another node.
So you have to follow the "Setting up TLS(SSL)" part of the documentation of the docker.
I am trying to use spring config server with cross account as I am deploying config server in kubernetise with aws backed.
but due to DefaultAWSCredentialsProviderChain I am unable to get connected to s3 bucket and gets 403 error.
In DefaultAWSCredentialsProviderChain as per logs WebIdentityTokenCredentialsProvider try to get credentials get 403 error.
but when I am try to connect with my awss3 client with STSAssumeRoleSessionCredentialsProvider it gets connect.
Is there any way so that I can provide STSAssumeRoleSessionCredentialsProvider instead of DefaultAWSCredentialsProviderChain
I Use Kibana 7.11.1 stack. this problem is related to elasticsearch alone.
I added xpack.security.enabled: true to elasticsearch.yml and ran elasticsearch-setup-passwords auto
to set the default users password. but now when I start elasticsearch I keep seeing the message:
[o.e.x.s.a.AuthenticationService] [SERVER_NAME] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
i see this only when elasticsearch service is up, so I assumed that I need to set the new elastic user password in elasticsearch.yml file, but i have no idea how..
any ideas ?
thanks
The error message indicates that the credentials with which you are trying to authenticate do not match the saved username and password.
After turning on the xpack.security.enabled setting, you can set the built-in user credentials, using the below command
bin/elasticsearch-setup-passwords interactive
I missed one service that is part of the kibana stack. the elastic-agent.
so the database wasn't trying to connect to itself! :) elastic-agent was trying to connect to elasticsearch without the proper credentials.
once i resolved that and restart elastic-agent, the problem resolved.
Is it possible configure filebeat to communicate with an Elastic Cloud instance using token authentication?
According to the docs, if I'm using a cloud instance I should configure cloud.id and cloud.auth in filebeat.yml
cloud.id: "..."
cloud.auth: "filebeat_setup:YOUR_PASSWORD"
The docs say that cloud.auth should be a username and password from my Elastic Cloud instance. I'd like to use an api_key instead. However, when I configure an API key
output.elasticsearch:
# Authentication credentials - either API key or username/password.
api_key: "key-id:key-value"
and attempt to test my connection,
$ sudo filebeat test output
elasticsearch: https://...:443...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 52.202.123.120, 18.214.74.184, 50.19.154.221
dial up... OK
TLS...
security: server's certificate chain verification is enabled
handshake... OK
TLS version: TLSv1.2
dial up... OK
talk to server... ERROR 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}
It appears that filebeat doesn't acknowledge my API Key
"root_cause":[
{
"type":"security_exception",
"reason":"missing authentication credentials for REST request [/]"
/* ... */
I've had success connecting to my cloud instance with the #elastic/elasticsearch javascript package using this token.
Before I continue to debug this probiem -- is it even possible to use token authentication to connection to Elasticsearch via filebeat? Or does filebeat only support username/password authentication?
The answer to this question turned out to be: Yes, you can use an api_key with filebeat, even if you're using elastic cloud.
While the error message received during my config test
missing authentication credentials for REST request
indicated the authentication was missing, the real problem was the key I had had previous success with had recently expired. I presume filebeat tried the API key, was rejected, and then fell back to trying the user credentials. When those credentials were missing, it gave the above error.
Basic username/password authentication for ELK, i was able to achieve using searchguard on windows platform.
Now i am trying to establish secure communication. I have performed the following changes,
In elasticsearch.yml
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks
searchguard.ssl.http.keystore_password: 221749a2add117cf889f
searchguard.ssl.http.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks
searchguard.ssl.http.truststore_password: 6d6cf1cc017dc874960b
searchguard.authcz.admin_dn:
- CN=sgadmin
searchguard.ssl.transport.keystore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\CN=localhost-keystore.jks
searchguard.ssl.transport.keystore_password: 221749a2add117cf889f
searchguard.ssl.transport.truststore_filepath: D:\Softwares\ELK\elasticsearch-5.4.0\elasticsearch-5.4.0\config\truststore.jks
searchguard.ssl.transport.truststore_password: 6d6cf1cc017dc874960b
In Kibana.yml
elasticsearch.url: "https://localhost:9200"
elasticsearch.username: "admin"
elasticsearch.password: "admin"
If i login to kibana, using http://localhost:5601, it asks for username and password. I dont know what credentails to enter here. i tried admin/admin. Its not working. Before i gave searchguard.ssl.http.... configurations, admin/admin was working fine.
After i added all the searchguard.ssl.http related configuration, the credentails are not working.
I am sure there is some other configuration in kibana.yml w.r.t searchguard configuration. I am not able to find it online. Can any one please help me here on what is missing.
Enabling TLS on the REST layer does not have any impact on user authentication/authorisation. The only difference is:
If you enable TLS on REST layer, only HTTPS access is allowed. You will see an error message if you try to access ES with HTTP
If you disable TLS on REST layer, only HTTP access is allowed. You will see an error message if you try to access ES with HTTPS.
Search Guard will authenticate the credentials against the configured authentication backend in sg_config.yml. If you use the default configuration that ships with Search Guard, it will use the internal user database and the users defined in internal_users.yml. The default "admin" user has full access to all indices and types, so you should be able to use this user to log into Kibana. If you need a user with limited access, the corresponding demo user is "kibanaro". Please refer to internal_users.yml to see all demo users.
Since ES 5.0.2, you need to install the Search Guard Kibana plugin for full authentication and session support. You can find the plugin on the corresponding GitHub repository. You install it like any other Kibana plugin, for example:
bin/kibana-plugin install https://github.com/floragunncom/search-guard-kibana-plugin/releases/download/v5.4.3-3/searchguard-kibana-5.4.3-3.zip
If you do not need advanced stuff like multi-tenancy or JWT Single Sign-on, it should start and work out of the box.
If this does not help, please post the output of your Elasticsearch log files when trying to log in.
Disclaimer: I work for floragunn, makers of Search Guard.