I booted the DataStax AMI for Amazon EC2, logged in via SSH, but the terminal hangs on "Installation Started":
Cluster started with these options:
--clustername CassandraDev --totalnodes 1 --version enterprise --username **** --password ****
Installation started.
"Installation started" keeps going through suffixes consisting of one, two, and three dots. But nothing happens, I can't quit the installation process, and I can't access any log files to see what might be going on (or I don't know how).
Tried on two separate m3.large instances operating in a VPC subnet, at the us-east-1 region. The exact AMI is datastax_clustering_ami_2.5.1_hvm.manifest.xml (ami-ada2b6c4). On the first instance, I waited about an hour and a half. The second instance I just left online all night, with the same results.
Because this is a VPC, all outbound traffic goes through a NAT server. Security groups allow outbound traffic only on ports 80, 443, and 123. Might there be another outbound port that needs to be opened? Inbound ports do not matter, as the server is not public-facing, but within the subnet I have allowed all traffic on all ports.
Someone else has had a similar issue, but without answers so far: DataStax AMI hangs on
Any help would be appreciated!
Since there were a few tickets that came up recently around the same issue, it seems as though something recently changed within the AMI provisioning side in EC2, or this specific configuration of VPCs had never been used before, which seems a bit unlikely.
The current fix is to add an additional entry into /etc/hostname to get rid of the stderr output that occurs after each sudo command. This in turn doesn't get flagged as an error on the provisioning side.
This has been fixed and patched as documented on this ticket:
https://github.com/riptano/ComboAMI/issues/51.
If you spot any additional issues, feel free to create another ticket there.
Going forward, just launch another set of instances using the same user-data and you should be up and running.
Related
Has anyone encountered the NLA error after changing instance type for an EC2 instance?
Getting this error after upgrading a domain joined instance (t3.2xlarge) to the recommended instance type according to Computer Optimizer (m6i.2xlarge).
Cannot RDP using the local administrator account either, same NLA error.
Also made a re:post question but no answer yet.
Kind regards,
Ken
Things I have tried:
Changing back to t3.2xlarge, connected using domain credentials OK
Changing to m5.2xlarge, connected using domain crendentials OK
Added another NIC when it was on m6i.2xlarge, NLA error on the second interface.
(Don't think this matters, the instance is HVM) Upgraded to the latest PV driver, changing instance type to m6i.2xlarge, NLA error.
Launched a m6i.2xlarge instance in a different subnet(AZ), joined domain OK, connected using domain crendentials OK; changed to t3.2xlarge, NLA error; changed back to m6i.2xlarge, connected using domain crendentials OK
Launched another m6i.2xlarge instance in the same subnet as the t3.2xlarge, swapped the root volume, NLA error. Swapped back the volumes, connected OK.
All these tests leads me to think there is incompatibility b/w gen 5 Xeon processors and gen 6, which is strange, I thought at first it was a network card issue.
Copy pasted from my own answer on RE:POST
Managed to isolate the cause after performing some rescuing via SSM.
The issue seems to stem from the upgrade from the CPU generation leap.
I had always thought each component, Storage, Compute, and Networking are separate, but the ENI config was lost during the upgrade, so the server had trouble (i.e. did not know where the DNS server is) contacting the DCs for authentication.
Without this link to the DCs, NLA will never be met.
So if you are going to upgrade to the latest generation.
While on the current instance type (while you can RDP to the EC2 instance), navigate to System Properties and go to the Remote tab.
Untick the NLA option and apply and save the change.
Shutdown the instance and change to the desired instance type.
RDP to the instance using the Administrator account.
Here you will see that the network interface configuration is empty, so add your DNS server IP address back in here.
Confirm you have a connection to the DCs by pinging or something of the sort, then repeat step 1, but this time enable the NLA option and save.
Reboot and Voilà, you should now have access to the EC2 instance using your domain logins again.
I trying to setup AWS Cloud9 and am running into a wall each time I try to setup my environments. Once I create the environment and start following this guide https://docs.aws.amazon.com/cloud9/latest/user-guide/sample-lamp.html to configure the LAMP server, through the Cloud9 IDE terminal, the environment will just stop responding. Once I try to reload the IDE I get the follow error;
Cannot connect to instance error message.
Rebooting the instance doesn't seem to resolve the error message. But any time I make a fresh instance it will let me work from anywhere to 30 seconds to 90 seconds before it stops responding.
I have looked through my VPC port settings, as well as security group settings, and they both appear to the correct.
VPC inbound rules VPC outbound rules Security Group inbound
rules Security Group outbound rules
Additionally, I was using the default t2.micro instance until I read this post AWS Cloud9: Cannot open environment and have tried with the t2.small but I am still getting the same results.
Any help with where else to look or what else to try would be much appreciated!
Edit: It appears to be random when it stops and freezes, for example when making a m4.large instance. It froze while I was setting up the sudo mysql_secure_installation.
Once I typed "Y" it wouldn't let me press enter. Reloading the IDE gave me the VPC error.
Welcome to SO! When I use cloud9 I tend to use m4.large for anything that's non-trivial. If you're running Apache and MySQL on the same host I would definitely try the m4.large instance. It's $0.10/hr (pricing) so you could try it out fairly cheaply. I'm guessing that's the root of the issue. If you're still having the issue please repost here and we can check further.
Just to confirm:
- You can connect to the instance at least once (even if for a few seconds)
- You see the IDE and can type for 30-60 seconds before it stops responding
If you can't connect that's likely a different issue.
I'm trying to setup Realm Object Server on Amazon EC2.
I've used the public AMI on North Virginia, and I have a running instance. I'm doing all this from Europe as most of my users are in the USA.
Now I'm trying to access ec2-xx-xx-xx-xx.compute-1.amazonaws.com:9080.
I've tried to open the different ports as indicated but I feel that what I've done is incorrect.
I've also tried to open all traffic but I still have a timeout on the page. I'm probably doing something wrong here, I'm not sure what.
Thanks for your help!
Thanks for trying out our AWS AMI! It would be helpful to know the AMI ID that you ran, as that can help us track down problems for others. In fact, we've released new AMIs this morning. Check our website for the latest available AMI IDs.
In the meantime, can you check if the realm-object-server service is running? You can check this via SSH and by running:
sudo service realm-object-server status
So I managed to make things work!
I guess my issue is that I was somehow on the wrong security group.
When looking at your running instances, be sure to hit your security group at the right of the instance row, in order to be able to configure the correct one.
Then, configure a Custom TCP Rule with port 9080.
That's it!
After installing Tigase on an AWS EC2 instance I keep getting the error message 'connection refused' when I try to connect to it using an xmpp client.
The instance is attached to a security group with rules to allow traffic to the necessary ports (tigase needs 5223 primarily and some others for more exotic features). I've also tried it with rules allowing all traffic to all ports from all sources but I still get the same message.
I've also checked iptables because I noticed some people needed to configure those as well in specific cases, I made sure it allows all connections but still I can't connect to Tigase.
Yes Tigase is running, there are no relevant errors in the Tigase logs
SSH (port 22) and HTTP (port 80) work fine
Enabling ICMP (ping) on all ports works fine
I've tried several xmpp clients, same problem
I've deleted and recreated instances several times
Re-installed Tigase on fresh instances several times with various configuration options
Tried using domain name associated with Elastic IP, normal IP and tried public DNS directly.
Configured the DNS in the way necessary for Tigase as described here
I've looked everywhere and have not been able to find anything to fix this. Networking isn't my main area of expertise and I'd really appreciate any advice.
Wow, in case anyone runs into the same problem in the future, turns out that this was related to the AMI. I was using an Amazon Linux AMI and switched to Ubuntu Server 14.04 LTS. I wish I tried this sooner but I didn't really consider this a possible solution earlier. Apparently Amazon Linux doesn't play well with Tigase.
Ok, I am quite a newbie of the salt-stack world, but after 2 days being stuck with this issue I'm starting to feel a bit stupid too.
I would like to have a simple 1:1 configuration:
[Master] Vagrant/VirtualBox/Ubuntu with salt & salt-cloud installed
[Minion] Amazon EC2 machine, conveniently provisioned with state files I have in [Master].
I have reached the step where I am able to create the Minion instance thanks to salt-cloud, but I am stuck at the next step: I don't know how I can
Transfer .sls files to the Minion
Run the top.sls at Minion side to perform the provision
The fact is that any salt-cloud command seems to work (I am able to create, list, delete the Amazon EC2 instance by command line), but I cannot connect to the Minion with any salt command, I just get timeout ("Minion did not return").
Moreover I am not comfortable with this architecture because the Minion could receive Master's requests, but on the other end it doesn't have visibility of the Master, since the Master is not publicly reachable (and I don't want to).
What am I missing to be able to have an architecture as simple as this?
The problem you're running into is that the Salt Minion probably can't find a route back to your laptop. Your laptop is most likely behind a nat firewall.
Your Salt Minion must be able to reach your Salt Master on ports 4505 and 4506. Once you've got that working, you should be fine. You're probably going to want to have your Salt Master on EC2 or somewhere that can be reached easily by your minion on EC2.