I'm following the Hadoop official tutorial to run Hadoop on a my machine in a pseudo-distributed mode.
I can use ssh to login in localhost without password:
admin#mycomputer:/usr/local/hadoop/hadoop-2.6.0$ ssh localhost
Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-45-generic x86_64)
* Documentation: https://help.ubuntu.com/
4 packages can be updated.
0 updates are security updates.
Last login: Mon Feb 9 12:31:17 2015 from localhost
admin#mycomputer:~$
And I can also format the namenode without error, but I cannot start Hadoop with start-dfs.sh:
admin#mycomputer:/usr/local/hadoop/hadoop-2.6.0$ sudo sbin/start-dfs.sh
Starting namenodes on [localhost]
root#localhost's password:
localhost: Permission denied, please try again.
Why I'm still asked to provide root password while I can ssh into localhost without it?
I also tried:
sudo passwd
to reset the password, but later encounter the same permission denied error, it seems to me that this password is not the password for root#localhost. How can I solve this problem?
I think you didn't change the permission for the hadoop-2.6.0 folder. Give admin user permission to this folder and try to start.
Follow my below blog link : I provided steps in detail installing in Ubuntu by enriching from another blog.
http://gubendran.blogspot.com/2015/01/install-hadoop-in-single-node-linux.html
Related
When I try to ssh into localhost, I am prompted for password. See below
"
ssh connection to localhost:
[hadoop#mftrhel74 sbin]$ ssh localhost
hadoop#localhost's password:
Last login: Fri Aug 23 15:44:08 2019 from mah"
---The above statement means, passwordless connection is not setup----
But when I try to start Hadoop nodes as below, it doesn't prompt for password.
And the nodes are not starting, I see below message
I think it should prompt me to enter the password for the user just like as SSH connection is to be established.
[hadoop#mftrhel74 ~]$ start-dfs.sh
Starting namenodes on [mftrhel74]
mftrhel74: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Starting datanodes
localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Starting secondary namenodes [mftrhel74]
mftrhel74: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
************I DO NOT WANT A PASSWORDLESS CONNECTION*****
I suspect you are able to log in to one of the nodes with SSH, however probably you have not set up passwordless ssh between the nodes, so the steps you try to execute from the node will fail.
Here is some documentation that should explain that you need to set up passwordless ssh or otherwise install an ambari client (assuming you work on HDP).
https://ambari.apache.org/1.2.2/installing-hadoop-using-ambari/content/ambari-chap1-5-2.html
I am using ubuntu local machine with below hostname and trying to setup cloudera Hadoop Distribution CDH5.
chaithu#localhost:~$ hostname
localhost
chaithu#localhost:~$ hostname -f
localhost
chaithu#localhost:~$ ssh chaithu#localhost
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.8.0-36-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
449 packages can be updated.
232 updates are security updates.
Last login: Mon Dec 18 22:44:30 2017 from 127.0.0.1
Failed as Failed to detect root privileges with below error:
/tmp/scm_prepare_node.qkAAjdTz
using SSH_CLIENT to get the SCM hostname: 127.0.0.1 35708 22
opening logging file descriptor
Starting installation script...
Acquiring installation lock...
BEGIN flock 4
END (0)
Detecting root privileges...
effective UID is 1000
BEGIN which pbrun
END (1)
BEGIN sudo -S id
[sudo] password for chaithu:
END (1)
need root privileges but sudo requires password, exiting
closing logging file descriptor
Screen shot for where I am stuck with CDH installation.
Looks like you are missing sudo or passwordless sudo for the user which you are using for installation.
Configure sudo for the user which is used for set up.
Make sure passwordless sudo is configured for that user.
I am using: CentOS 7 with Ambari 2.1.1 to try and setup a single node setup on a VM. I want to do this to install vanilla hadoop etc instead of installing a prepackaged VM with some modified version of hadoop.
I am logged in as root. I have created a ssh key pair. I also ran:
"cat id_rsa.pub > authorized_keys"
"chmod 700 .ssh/"
"chmod 640 ./ssh/authorized_keys"
I have edited /etc/ssh/sshd_config to: permit empty passwords, allow root login and also to state where the authorized_keys file is.
Without a password I can run "ssh root#localhost" and log in fine.
I have ran "ambari-server setup" successfully and logged in at localhost:8080 with user: admin pass: admin.
In "Install Options" FQDN I typed "localhost.test" and have selected a copy of my private key for the Host Registration Information.
But not matter what I do I am unable to get the components install under the confirmed hosts part and thus can't get any further.
Can someone please point out what I am missing here?
Thanks to Yusaku on HortonWorks forum for the help.
Ok I ran:
hostname -f
and got localhost
python -c ‘import socket; print socket.getfqdn()’
and got localhost.localdomain
By entering localhost.localdomain into the FQDN I was able to get the install working.
I have a hadoop installation on the Amazon Elastic MapReduce , whenever I try to restart the cluster I get the following error:
/stop-all.sh
no jobtracker to stop
The authenticity of host 'localhost (::1)' can't be established. RSA key fingerprint is
Are you sure you want to continue connecting (yes/no)? yes
localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
localhost: Permission denied (publickey).
no namenode to stop
localhost: Permission denied (publickey).
localhost: Permission denied (publickey).
Any idea on how to restart hadoop?
Following hack worked for me.
I have replaced "ssh" command in sbin/slaves.sh & sbin/hadoop-daemon.sh with "ssh -i ~/.ssh/keyname"
I'm using hadoop version 2.4 and this worked for me:
export HADOOP_SSH_OPTS="-i /home/hadoop/mykey.pem"
For the stop-all.sh script to work, you probably need to have the same user in all the machines as the user with which you are executing the stop-all.sh script.
Moreover, it seems you do not have a password less ssh setup from the machine you are executing stop-all.sh to rest of the machines that will spare you from manually entering the password for each machine separately. Passwords might be different for the same user for different machines, please don't forget that.
I'm installing greenplum database on my desktop computer following the official installation guide. When I'm executing
# gpseginstall -f hostfile_exkeys -u gpadmin -p P#$$word
it asks me to provide cluster password access:
[root#sm403-08 greenplum-db-4.2.1.0]# gpseginstall -f hostfile_exkeys -uyang -par0306
20120506:05:59:33:012887 gpseginstall:sm403-08:root-[INFO]:-Installation Info:
link_name None
binary_path /usr/local/greenplum-db-4.2.1.0
binary_dir_location /usr/local
binary_dir_name greenplum-db-4.2.1.0
20120506:05:59:33:012887 gpseginstall:sm403-08:root-[INFO]:-check cluster password access
*** Enter password for localhost-2:
*** Enter password for localhost-2:
*** Enter password for localhost-2:
*** Enter password for localhost-2:
*** Enter password for localhost-2:
This is what my hostfile_exkeys file looks like:
localhost
localhost-1
localhost-2
since I only have one machine.
A similar post on the web (http://www.topix.com/forum/com/greenplum/TSDQHMJ6M7I9D0A44) says:
"I had the same error and I discovered that it was because I had set sshd to refuse root login. You must edit your sshd configuration and permit root login for gpseginstall to work. Hope that helps!"
But I have tried to modify my /etc/ssh/sshd_config file to let it permit root login:
# Authentication:
#LoginGraceTime 2m
PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10
and restarted sshd:
Stopping sshd: [FAILED]
Starting sshd: [ OK ]
but nothing works; the gpseginstall program is still asking for password.
I have tried all the passwords I can ever think of, root, gpadmin, my own user's password, but none of them works. What am I expected to do to get it work?
Update: It seems that the problem lies in installing the Greenplum community edition on a single node. Is there anyone who has some experience with this?
Thanks in advance!
It seems that I'm installing Greenplum database on a single node, so don't have to do the gpseginstall step. This is used to install Greenplum on all segments from the master host.
You need to enable password auth.
sudo nano /etc/ssh/sshd_config
PermitRootLogin yes
PasswordAuthentication yes
Then service sshd restart
I will be glad if it helps someone who is trying to install greenplum in cluster mode.
#installing greenplum cluster steps
# first add entires for all servers and interfaces in your /etc/hosts
# gpdb01- master
# gpdb02 - secondary master
# gpdb03 , gpdb04 - data nodes
#setup ssh between all machines
ssh-keygen
ssh-copy-id gpdb02
ssh-copy-id gpdb03
ssh-copy-id gpdb04
# also add entries for the interfaces
vi /etc/hosts
172.12.13.14 gpdb01
172.12.13.14 gpdb01-1
172.12.13.14 gpdb01-2
172.12.13.15 gpdb02
172.12.13.15 gpdb02-1
172.12.13.15 gpdb02-2
172.12.13.16 gpdb03
172.12.13.16 gpdb03-1
172.12.13.16 gpdb03-2
172.12.13.17 gpdb04
172.12.13.17 gpdb04-1
172.12.13.17 gpdb04-2
# enable RootLogin and PasswordAuthentication on all servers
vi /etc/ssh/sshd_config
service sshd restart
#create your hostkey file
gpdb01
gpdb01-1
gpdb01-2
gpdb02
gpdb02-1
gpdb02-2
gpdb03
gpdb03-1
gpdb03-2
gpdb04
gpdb04-1
gpdb04-2
# run the gpseg installer
gpseginstall -f hostfile_exkeys -u gpadmin -p P#$$word