Elasticsearch Shield Plugin - Users can not be authenticated, even users with Admin Privileges - elasticsearch

Question

I'm having an issue with the Shield plugin for Elasticsearch. I have installed elasticsearch so that it runs as a service, and I can see that it is running on port 9200.
If I enter curl http://localhost:9200 in my Mac terminal I get the correct elasticsearch output...
Recently I have been trying to upgrade from basic authentication to shield authentication. I installed shield and license using the following commands:
/usr/share/elasticsearch/bin/plugin install license/latest
/usr/share/elasticsearch/bin/plugin install shield/latest
This successfull installs shield and license plugins. They are contained within my /usr/share/elasticsearch/plugins directory on my Vagrant box.
So, now if I try to use curl -XGET http://localhost:9200 again, I get the following response:
{"error":{"root_cause":[{"type":"security_exception","reason":"missing
authentication token for REST request
[/]","header":{"WWW-Authenticate":"Basic
realm=\"shield\""}}],"type":"security_exception","reason":"missing
authentication token for REST request
[/]","header":{"WWW-Authenticate":"Basic
realm=\"shield\""}},"status":401}
This is good. I can not reach elasticsearch without proper authentication. So now I want to create a user to authenticate to elasticsearch. I use the command:
sudo ./esusers useradd testuser -p password -r admin
This creates a user 'testuser' with password 'password' with admin privileges.
I can verify that this user has been created using the following command:
sudo ./esusers list
This returns:
testuser : admin
So I now try to run the curl command again with this user:
curl -u testuser:password -XGET http://localhost:9200
But I get the same error message as before when I tried without the admin user.
What is the issue here? Why is my admin user not authenticating?

Answer 1

You said that your verified the user is created using sudo ./esusers list but still the curl command fails. Chances are that you are using custom directory for elasticsearch even though the home dir is /usr/share/elasticsearch. And due to this, the esusers creates the users in /etc/elasticsearch/shield/ directory but they need to be copied to the custom dir, in case you are using one.
I know you've solved this differently but I'm answering this in the hope that it helps someone else in future. Can you confirm regarding the custom dir? For e.g in my case, the custom dir was /data/elasticsearch and the home dir was /usr/share/elasticsearch