How to set up a Chef Node on an ec2 instance? - amazon-ec2

I am trying to use knife bootstrap command from mac terminal
"knife bootstrap node_domain_or_IP -x username -P password -N name_for_node --sudo"
But the problem is, I don't have user name and password, instead the ec2-user has this private key stored in my local work station which helps me to connect to the server.
I find lot of examples for debian os, but hard to find for RHEL on ec2.
OS: RHEL 6
Chef: 11.1.6
Kindly let me know if any details you need to help me better.


Even with host keys you will have a username. Typically, if you are using
ssh -i somekey user#host_or_ip
to ssh to your node, then you would use
knife bootstrap node_domain_or_IP -x username -i sameKey -N chef_name_you_want --sudo
Notice, I use -i rather than -P. That's all there is to it.
An even better option for most people is to use knife ec2 server create to create your node in the first place. This will create the node in AWS and bootstrap it, all in one command.

Related

I am using Windows and I want to use aws MySQL database by using the following command

ssh -i "key_pair.pem" ec2-user#ec2-xy-xyz-201-00.ap-southeast-1.compute.amazonaws.com
and I want use DB like cms and I want to use this by using the command
-L 3000:xyz.ap-southeast-1.rds.amazonaws.com:3306 -N
I am totally confused.

To do local forward(from ec2 to your system)
ssh -i <pem key> -NfL localhost:<your local port>:<instanceip>:<your sql port> <user>#<instance ip>
To do remote forward(from your local system to EC2)
Make sure AllowTCPForwarding yes present in /etc/ssh/sshd_config file by default it's not enabled
Ref: Remote port forwarding failed on Amazon EC2
ssh -i keyPair.pem -R <remote_EC2_port>:localhost:<localhost_port> ec2-user#address.amazonaws.com
Also, make sure your EC2 security group allows access to MySql port default(3306)

How to move my files stored under my personal computer to my Amazon EC2 instance?

My question may sound funny to some of you since I am new to Terminal/Linux/Amazon AWS.
But how can I move (copy) the file that is stored under my personal computer to my Amazon EC2 instance?
Thank you,

You can use scp.
scp -i privateKey.pem -r ./localDir ec2-user#ip:~
To copy from ec2, reverse the command
scp -i privateKey.pem -r ec2-user#ip:~/remote-dir ./localDir
Please ec2-user with Ubuntu if you are using Ubuntu. This will copy to the home folder of the instance
Note: Please note your private key file should not have too open permissions.
chmod 400 /path/to/privateKeyFile.pem
hope this helps.

Move is different to copy. I'll assume you mean copy. The simplest way to copy files from your local machine to a running EC2 instance is to scp them. That requires you to be able to SSH to the EC2 instance. Something like this:
scp -i mykey.pem myfile.png ec2-user#ec2-01-02-03-04.compute-1.amazonaws.com:~
This assumes you're using Amazon Linux on which the default user is ec2-user. Change this as appropriate, for example to ubuntu.

Laravel homestead after.sh unable to auto setup custom configuration

I`m using laravel/homestead package and im trying tu set some configuration on "after.sh" file, but with no success.
After run the command "vagrant up" i want to make the followings statments:
1- Install sshpass dependency without asking (y/n)
sudo apt-get -y install sshpass
2- Create a ssh key without asking (file/password)
ssh-keygen -b 2048 -f ~/.ssh/mykeyname-t rsa -N ''
3- Copy the ssh id to domain without asking (password)
sshpass -p mypassword ssh-copy-id -i ~/.ssh/mykeyname user#domain
And when vagrant is up i want to be able to connect with comand:
ssh -i ~/.ssh/mykey user#domain
If all those thing i do it manually it works properly, but i want to know if is possible to create script in "after.sh" to automate it and ready to use when machine is deployed.
I tryied to write those lines on "after.sh" but it is not ready to use when machine is deployed.
It seems to work just to step 2 and never work on step 3.
Can anyone help me, or tell me if this can be done?
Thanks for awnsers, and sorry my english.

i just find the answer.
Between step 2 and 3, creating sshkey and copysshid i need to first add the domain to known_hosts with the following command
ssh-keyscan domain>> ~/.ssh/known_hosts
And then after deploy machine via vagrant up i was able to login shh without password with the following command:
ssh -i ~/.ssh/mykey user#domain
Thanks everyone.

Hadoop - requestion for network lan password during starting cluster

I can't understant what password is expected by hadoop.
I configured it according to tutorial. I do:
sudo su
#bash start-dfs.sh
And now it expects someting like password lan's network. I have no idea what should I write.
As you can see, I run script as root. Of course master (from that I run script) may ssh to slaves as root without password (I configured and tested it).
Disclaimer: It is possbile that I give incorrect name (for example for script name - it is beacause of I don't understand exactly now. However I am sure that it was about something like lan's network password)
Help me please, for which a password is it?
Edit: I was using http://backtobazics.com/big-data/setup-multi-node-hadoop-2-6-0-cluster-with-yarn/

It seems you may not setup passwordless-ssh. Passwordless-ssh is required to run hadoop services (daemons). So try to setup ssh among nodes
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys
Then ssh user#hostname

Using scp to copy a file to Amazon EC2 instance?

I am trying to use my Mac Terminal to scp a file from Downloads (phpMyAdmin I downloaded online) to my Amazon EC2 instance.
The command I used was:
scp -i myAmazonKey.pem phpMyAdmin-3.4.5-all-languages.tar.gz hk22#mec2-50-17-16-67.compute-1.amazonaws.com:~/.
The error I got:
Warning: Identity file myAmazonKey.pem not accessible: No such file or directory.
Permission denied (publickey).
lost connection
Both my myAmazonkey.pem and phpMyAdmin-3.4.5-all-languages.tar.gz are in Downloads, so then I tried
scp -i /Users/Hello_Kitty22/Downloads/myAmazonKey.pem /Users/Hello_Kitty22/Downloads/phpMyAdmin-3.4.5-all-languages.tar.gz hk22#mec2-50-17-16-67.compute-1.amazonaws.com:~/.
and the error I got:
Warning: Identity file /User/Hello_Kitty22/Downloads/myAmazonkey.pem not accessible: No such file or directory.
Permission denied (publickey).
lost connection
Can anyone please tell me how to fix my problem?
p.s. there is a similar post: scp (secure copy) to ec2 instance without password
but it doesn't answer my question.

Try specifying the user to be ec2-user, e.g.
scp -i myAmazonKey.pem phpMyAdmin-3.4.5-all-languages.tar.gz ec2-user#mec2-50-17-16-67.compute-1.amazonaws.com:~/.
See Connecting to Linux/UNIX Instances Using SSH.

second directory is your target destination, don't use server name there. In other words, you don't need to mention machine name for the machine you're currently in.
scp -i /path/to/your/.pemkey -r /copy/from/path user#server:/copy/to/path
-r if it's a directory.

Your key must not be publicly viewable for SSH to work. Use this command if needed:
chmod 400 yourPublicKeyFile.pem

You should be on you local machine to try the above scp command.
On your local machine try:
scp -i ~/Downloads/myAmazonKey.pem ~/Downloads/phpMyAdmin-3.4.5-all-languages.tar.gz hk22#mec2-50-17-16-67.compute-1.amazonaws.com:~/.

Here are the details of what works for an EC2 instance:
scp -i /path/to/whatever.pem /users/me/path-to-file ec2-user#ec2-55-55-555-555.compute-1.amazonaws.com:~
Few notes for beginning:
Note the spaces between the three parameters given after the -i
scp stands for secure copy protocol. Knowing the words makes it easier to remember the command.
-i dictates that you need to give the .pem file as the next param. If there is no -i, than you do not need a .pem.
Note the :~ at the end of the destination for the EC2 instance.

I had exactly same problem, my solution was to
scp -i /path/pem -r /path/file/ ec2-user#public aws dns name: (leave it blank here)
once you done this part, get into ssh server and mv file to desired location

This just worked for me. I used a combination of two other answers to this question.
scp -i /Users/me/documents/myKP.pem -r /Users/me/desktop/testDir \
ec2-user#ec2-11-111-11-11.compute-1.amazonaws.com:/home/ec2-user/remoteDir
The "ec2-user#ec2-11-111-11-11.compute-1.amazonaws.com" is copy-and-pasted from your ec2 instance's public DNS.

Send file from Local to Server:
scp -i .ssh/awsinstance.pem my_local_file
ubuntu#XX.XXX.XXX.XXX:/home/ubuntu
Download file from Server to Local:
scp -i .ssh/awsinstance.pem
ubuntu#XX.XXX.XXX.XXX:/home/ubuntu/server_file .

scp -i ~/path to pem file/file.pem -r(for directory) /PATH OF LOCAL/localfile user#hostname:PATH OF SERVER/serverdirectory

Below SCP format works for me
scp -i /path/my-key-pair.pem ec2-user#ec2-198-51-100-1.compute-1.amazonaws.com:~/SampleFile.txt ~/SampleFile2.txt
SampleFile.txt: It will be the path from your root directory(In my case, /home/ubuntu). in my case the file which I wanted to download was at /var/www
SampleFile2.txt: It will be path of your machine's root path(In my case, /home/MyPCUserName)
So, I have to write below command
scp -i /path/my-key-pair.pem ec2-user#ec2-198-51-100-1.compute-1.amazonaws.com:~/../../var/www/Filename.zip ~/Downloads

Public DNS
scp -i /path/my-key-pair.pem /path/my-file.txt ec2-user#my-instance-public-dns-name:path/
(IPv6)
scp -i /path/my-key-pair.pem /path/my-file.txt ec2-user#\[my-instance-IPv6-address\]:path/

SCP Commend
Send File from Local To Remote Server
sudo scp -i ../Downloads/new_bb_key.pem ./dump.zip ubuntu#13.127.124.129:~/.
Send File from Remote Server To Local
sudo scp -i ~/Downloads/new_bb_key.pem ubuntu#13.127.124.129:/home/ubuntu/LatestDBdump.zip Downloads/

try to use this command
if your instance is using ubuntu
scp -i myAmazonKey.pem phpMyAdmin-3.4.5-all-languages.tar.gz ec2-user#mec2-50-17-16-67.compute-1.amazonaws.com:~/.
you can get more info about your instance from here
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/connection-prereqs.html

The process of using SCP to copy files from a local machine to an AWS EC2 Linux instance is covered step-by-step (including the points mentioned below) in this video.
To correct this particular issue with using SCP:
You need to specify the correct Linux user. From Amazon:
For Amazon Linux, the user name is ec2-user.
For RHEL, the user name is ec2-user or root.
For Ubuntu, the user name is ubuntu or root.
For Centos, the user name is centos.
For Fedora, the user name is ec2-user.
For SUSE, the user name is ec2-user or root.
Otherwise, if ec2-user and root don't work, check with your AMI provider.
Your private key must not be publicly visible. Run the following command so that only the root user can read the file.
chmod 400 /path/to/yourKeyFile.pem

Check the permissions on the .pem file...openssh usually doesn't like world-readable private keys, and will fail (iir, scp doesn't do a great job of providing this feedback to the user).
Can you simply ssh with that key to your AWS host?

First you should change the mode of .pem file from read and write mode to read only mode. This can be done just by a single command in terminal sudo chmod 400 your_public_key.pem

I tried all the suggestions mentioned above and nothing worked. I terminated the current instance, launched another one and repeated the same exact process. This time no problems. Sometimes it might be the remote ami's fault.

I would use:
scp -i "path to .pem file" "file to be copeide from local machine" username#amazoninstance: 'destination folder to copy file on remote machine'

Resources