I booted up an EC2 instance months ago and at that time I had the .pem key stored in my PC. The PC since crashed and I had to reinstall Windows on it and I don't have a backup of the .pem file
Is there any way at all for me to get to the prompt on the EC2 instance- any way to ssh or otherwise get to the instance?
Not easily.
Guidance on the forums indicate that you can kind of do it by generating a new key pair and then bringing up a new instance and mounting the volume from the original instance and installing the keypair in that volume. After that the original instance should be able to use the new keypair.
According to this post on AWS Developer Forums it can be accomplished via:
Creating a new keypair downloads the private key to your machine and the public key is stored in your AWS account. When you launch a new (linux) instance the public key is placed into the /root/.ssh/authorized_keys file (or /home/ubuntu/.ssh/authorized_keys for Canonical Ubuntu AMIs), allowing your private key access to the instance.
Losing the private key can be fixed by logging in to the instance via another linux account if you have set one up.
Alternatively, if you're running an EBS-backed instance then you can Stop it, attach the root EBS volume to another instance, and place a new public key into the authorized_keys file, then return the volume to the original (stopped) instance and Start the instance again.
If none of these work, then - sorry - you're out of luck. It's called a key because you can't get in without it.
You have to create an image of an current instance and then use that image for launching new instance. All files and data would copy from that instance and use different key when launching instance, then access it with same key
When we loose private key, You can't login to that machine. However, there is an another way to access that machine by generating a new key-pair
Please follow the below steps to recover the key.
Step 1) Detach your root volume from your machine using AWS console.
Step 2) Launch a fresh EC2 instance(Not from your old machine AMI)
Step 3) Attach your old volume to new EC2 machine
Step 4) Now login to new ec2 machine and mount the old EBS volume
Step 5) Now go to that partition then visit home directory inside that machine and go to .ssh folder.
Step 6) Now generate a new private and public key. Then paste public key into authorized_keys file.
Step 7) Once you done with above steps, detach that volume from this ec2 machine.
Step 8) Now attach this volume to your old machine as root volume
Step 9) Now try to login to your old machine with the newly generated key.
Hope it helps !!
General idea: Use the AWS instance user-data to write a new ssh-rsa public key to /root/.ssh/authorized_keys. The cloud-init package installed on the linux instance needs to support the bootcmd directive. It worked for me with Ubuntu 16.04 and 18.04.
User-data example:
#cloud-config
bootcmd:
- echo 'ssh-rsa AAAAB3NzaC1... key-comment' > /root/.ssh/authorized_keys
This can be done manually, e.g. generate a new key with PuTTYgen, and set the user-data on the EC2 instance via AWS console.
Or automated, e.g. with Java, using the AWS EC2 Java SDK and Bouncy Castle:
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.interfaces.RSAPublicKey;
import java.text.SimpleDateFormat;
import java.util.Base64;
import java.util.Date;
import org.bouncycastle.openssl.PEMKeyPair;
import org.bouncycastle.openssl.PEMParser;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;
import org.bouncycastle.openssl.jcajce.JcaPEMWriter;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.ec2.AmazonEC2;
import com.amazonaws.services.ec2.AmazonEC2ClientBuilder;
import com.amazonaws.services.ec2.model.DescribeInstancesRequest;
import com.amazonaws.services.ec2.model.InstanceStateName;
import com.amazonaws.services.ec2.model.ModifyInstanceAttributeRequest;
import com.amazonaws.services.ec2.model.StartInstancesRequest;
import com.amazonaws.services.ec2.model.StopInstancesRequest;
public class RecoverAwsEc2RootSshAccess {
public static void main(String[] args) throws Exception {
// use the AWS console to create an AWS IAM user with ec2 permissions for your region, and generate security credentials
String awsAccessKey = "...";
String awsSecretKey = "...";
Regions region = Regions.US_EAST_1;
String instanceId = "i-...";
File pemKeyFile = new File("private.key.pem");
String keyComment = "key-generated-" + new SimpleDateFormat("yyyyMMdd-HHmmss").format(new Date());
KeyPair keyPair;
if(pemKeyFile.exists()) {
System.out.println("reusing existing RSA private key: " + pemKeyFile.getAbsolutePath());
try(PEMParser pemParser = new PEMParser(new FileReader(pemKeyFile))) {
keyPair = new JcaPEMKeyConverter().getKeyPair((PEMKeyPair) pemParser.readObject());
}
}
else {
System.out.println("generating new RSA private key: " + pemKeyFile.getAbsolutePath());
KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(2048);
keyPair = keyGen.generateKeyPair();
try(JcaPEMWriter pemWriter = new JcaPEMWriter(new FileWriter(pemKeyFile))) {
pemWriter.writeObject(keyPair.getPrivate());
}
}
String authorized_keys = to_authorized_keys_line(keyPair, keyComment);
String userdata = "#cloud-config";
userdata += "\n" + "bootcmd:";
userdata += "\n" + " - echo '" + authorized_keys + "' > /root/.ssh/authorized_keys";
String userdataBase64 = Base64.getEncoder().encodeToString(userdata.getBytes(StandardCharsets.UTF_8));
System.out.println("AWS instance user-data (can also be set manually via the AWS console):");
System.out.println(userdata);
AmazonEC2 ec2 = AmazonEC2ClientBuilder.standard().withRegion(region) //
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(awsAccessKey, awsSecretKey))).build();
// stop, set userdata, start
ec2.stopInstances(new StopInstancesRequest().withInstanceIds(instanceId));
waitForInstanceState(ec2, instanceId, InstanceStateName.Stopped);
ec2.modifyInstanceAttribute(new ModifyInstanceAttributeRequest().withInstanceId(instanceId).withUserData(userdataBase64));
ec2.startInstances(new StartInstancesRequest().withInstanceIds(instanceId));
waitForInstanceState(ec2, instanceId, InstanceStateName.Running);
// optional: stop, clear userdata, start
System.out.println("new IP: " + ec2.describeInstances(new DescribeInstancesRequest().withInstanceIds(instanceId)).getReservations()
.get(0).getInstances().get(0).getPublicIpAddress());
// next step: automate DNS update
}
private static void waitForInstanceState(AmazonEC2 ec2, String instanceId, InstanceStateName desiredState) throws Exception {
String currentState = "?";
while(!currentState.equals(desiredState.toString())) {
Thread.sleep(3_000);
currentState = ec2.describeInstances(new DescribeInstancesRequest().withInstanceIds(instanceId)).getReservations().get(0)
.getInstances().get(0).getState().getName();
System.out.println("instance state: " + currentState + " (waiting for " + desiredState + ")");
}
}
/** https://stackoverflow.com/questions/3706177/how-to-generate-ssh-compatible-id-rsa-pub-from-java */
private static String to_authorized_keys_line(KeyPair key, String keyComment) throws IOException {
byte[] exponent = ((RSAPublicKey) key.getPublic()).getPublicExponent().toByteArray();
byte[] modulus = ((RSAPublicKey) key.getPublic()).getModulus().toByteArray();
ByteArrayOutputStream out = new ByteArrayOutputStream();
out.write(new byte[] { 0, 0, 0, 7, 's', 's', 'h', '-', 'r', 's', 'a' });
out.write(toUInt32(exponent.length));
out.write(exponent);
out.write(toUInt32(modulus.length));
out.write(modulus);
return "ssh-rsa " + Base64.getEncoder().encodeToString(out.toByteArray()) + " " + keyComment;
}
private static byte[] toUInt32(int value) {
return new byte[] { (byte) (value >>> 24 & 0xff), (byte) (value >>> 16 & 0xff), (byte) (value >>> 8 & 0xff), (byte) (value & 0xff) };
}
}
Finally test the connection, e.g. with FileZilla:
Related
I am trying to run the performance script which has API which takes the payload as file from S3 and submit to target system. As the JSSR preprocessor i have added the code.(Jmeter-How to copy files from one AWS S3 bucket to another bucket?)
This will copy file upto 5 GB. It will not allow to copy larger files where as i need to test for 15 GB. Please suggest how to perform for more than 5 GB file
Looking into Q: How much data can I store in Amazon S3?
Upload an object in a single operation using the AWS SDKs, REST API, or AWS CLI—With a single PUT operation, you can upload a single object up to 5 GB in size.
from the same page:
Upload an object in parts using the AWS SDKs, REST API, or AWS CLI—Using the multipart upload API, you can upload a single large object, up to 5 TB in size.
So you need to switch to Multipart Upload, there is even a piece of example code:
import com.amazonaws.AmazonServiceException;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.TransferManagerBuilder;
import com.amazonaws.services.s3.transfer.Upload;
import java.io.File;
public class HighLevelMultipartUpload {
public static void main(String[] args) throws Exception {
Regions clientRegion = Regions.DEFAULT_REGION;
String bucketName = "*** Bucket name ***";
String keyName = "*** Object key ***";
String filePath = "*** Path for file to upload ***";
try {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion(clientRegion)
.withCredentials(new ProfileCredentialsProvider())
.build();
TransferManager tm = TransferManagerBuilder.standard()
.withS3Client(s3Client)
.build();
// TransferManager processes all transfers asynchronously,
// so this call returns immediately.
Upload upload = tm.upload(bucketName, keyName, new File(filePath));
System.out.println("Object upload started");
// Optionally, wait for the upload to finish before continuing.
upload.waitForCompletion();
System.out.println("Object upload complete");
} catch (AmazonServiceException e) {
// The call was transmitted successfully, but Amazon S3 couldn't process
// it, so it returned an error response.
e.printStackTrace();
} catch (SdkClientException e) {
// Amazon S3 couldn't be contacted for a response, or the client
// couldn't parse the response from Amazon S3.
e.printStackTrace();
}
}
}
In case of JMeter's Groovy you either need to invoke this main() function explicitly or completely remove the class and the main function declaration.
I have created REST API to create EC2 instance using AWS JAVA SDK provided.
Now I am trying to connect to created EC2 instance and then need to install software's in the instance again through java. I didn't find any appropriate article for this. Is there any possible way to do this? I don't want to use SSH client like putty.. Thanks..
Sounds like you're looking for a java ssh client.
You should set up key authentication and use the ssh client library from java to execute the installation for you.
See this post: for a list of solutions
public static void connectToEC2(){
try{
JSch jsch=new JSch();
String user = "User-name";
String host = "host";
int port = 22;
File directory = new File(".");
String privateKey = directory.getCanonicalPath() + File.separator + "pem file path";
jsch.addIdentity(privateKey);
System.out.println("identity added ");
Session session = jsch.getSession(user, host, port);
System.out.println("session created.");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect();
Channel channel=session.openChannel("shell");
channel.setInputStream(System.in);
channel.setOutputStream(System.out);
channel.connect(3*1000);
}
catch(Exception e){
System.out.println(e);
}
}
I am using aws sdk to retrieve cloud data from aws sdk.
I get all ec2 related data but I am not able to find out how to connect instance.
Here is code of retrieve instance from amazon cloud:
IAmazonEC2 ec2Client = new AmazonEC2Client(accesskey,secretkey, new AmazonEC2Config
{
Timeout = TimeSpan.FromSeconds(300),
MaxErrorRetry = 3,
RegionEndpoint = RegionEndpoint.GetBySystemName(regionName)
});
var instanceRequest = new DescribeInstancesRequest();
DescribeInstancesResponse ec2Response = ec2Client.DescribeInstances(instanceRequest);
First of you will need a key/pair file that you've used while creating an instance as it is needed to retrieve windows password.
Following are the steps to retrieve windows instance password using AWS SDK:
#1. You need to pass instanceId as well as RSA key from your .pem file to the following code.
IAmazonEC2 ec2Client = new AmazonEC2Client(accesskey, secretkey, new AmazonEC2Config
{
Timeout = TimeSpan.FromSeconds(300),
MaxErrorRetry = 3,
RegionEndpoint = region,
});
var passwordRequest = new GetPasswordDataRequest();
passwordRequest.InstanceId = instanceId;
var passwordResponse = ec2Client.GetPasswordData(passwordRequest);
var password = passwordResponse.GetDecryptedPassword(rsaKey);
return password;
});
Note: You have to wait at least 4 minutes after launching an instance to get the windows password.
Using Java API, I am trying to access Public key stored in Luna HSM. Even though I am able to print the corresponding public key label name, but when I am trying to get the public key, I am not able to get the reference to that public key. Here is the code snippet:
KeyStore ks = KeyStore.getInstance("Luna");
ks.load(null, null);
lunaProvider = ks.getProvider();
publicKey = (PublicKey) ks.getKey(alipayImpl.getHsmKeyStorePublicEntryName(), null);
// ****************************************************************************
// ** If the private keystore is not found, return original barcode string. **
// ****************************************************************************
if (publicKey == null) {
throw new Exception("Unable to acquire the Public Key " + alipayImpl.getHsmKeyStorePublicEntryName() + ", Hash will not be verified.");
}
// ***********************************************************
// ** Create a Signature Object and sign the encrypted text **
// ***********************************************************
Signature signatureObject = Signature.getInstance(alipayImpl.getAlipaySignAlgorithm(), lunaProvider);
signatureObject.initVerify(publicKey);
signatureObject.update(signedMessage
.getBytes(AlipayConstants.INPUT_CHARSET_VALUE));
isValidSign = signatureObject.verify(Base64.decode(hash));
I am logging to HSM properly. While Accessing Private Key, I didnt have any issues. Is there any restriction on Luna HSM that access to public key is given only through Certificates?
Thanks in advance.
The correct answer is >
LunaKey lk= LunaKey.LocateKeyByAlias("publicKeyName");
But it is advisable to make the key persistent before querying HSM.
Did you try something like this :
final KeyStore keyStore = KeyStore.getInstance("Luna");
keyStore.load(null, null);
final Certificate certificate = keyStore.getCertificate(alias);
if (certificate == null) {
throw new IllegalArgumentException(String.format("Certificate '%s' does not exists", alias));
}
final PublicKey publicKey = certificate.getPublicKey();
// TODO Working with the public key...
In Java keystore there is no PublicKeyEntry and that's the reason why you're not able to access your public key.
https://docs.oracle.com/javase/8/docs/api/java/security/KeyStore.Entry.html
Java's KeyStore.KeyEntry interface has three implementing classes.
PrivateKeyEntry
SecretKeyEntry
TrustedCertificateEntry.
Java expects you to get retrieve a public key out of your certificate. Sebastien Vanmechelen has given you the perfect example on how to do that.
If by any chance, your Luna HSM partition does not have a X509 certificate then using LunaKey is the only alternative.
I am getting the following error "astyanax.connectionpool.exceptions.PoolTimeoutException:" when trying to use client Astyanax to connect to Cassandra on a EC2 instance. Need help
Following is my code snippet.
import org.mortbay.jetty.servlet.Context;
import com.netflix.astyanax.AstyanaxContext;
import com.netflix.astyanax.Keyspace;
import com.netflix.astyanax.MutationBatch;
import com.netflix.astyanax.connectionpool.NodeDiscoveryType;
import com.netflix.astyanax.connectionpool.OperationResult;
import com.netflix.astyanax.connectionpool.exceptions.ConnectionException;
import com.netflix.astyanax.connectionpool.impl.ConnectionPoolConfigurationImpl;
import com.netflix.astyanax.connectionpool.impl.ConnectionPoolType;
import com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor;
import com.netflix.astyanax.impl.AstyanaxConfigurationImpl;
import com.netflix.astyanax.model.Column;
import com.netflix.astyanax.model.ColumnFamily;
import com.netflix.astyanax.model.ColumnList;
import com.netflix.astyanax.model.CqlResult;
import com.netflix.astyanax.serializers.StringSerializer;
import com.netflix.astyanax.thrift.ThriftFamilyFactory;
public class MetadataRS {
public static void main(String args[]){
AstyanaxContext<Keyspace> context = new AstyanaxContext.Builder()
.forCluster("ClusterName")
.forKeyspace("KeyspaceName")
.withAstyanaxConfiguration(new AstyanaxConfigurationImpl()
.setDiscoveryType(NodeDiscoveryType.RING_DESCRIBE)
.setConnectionPoolType(ConnectionPoolType.ROUND_ROBIN)
)
.withConnectionPoolConfiguration(new ConnectionPoolConfigurationImpl("MyConnectionPool")
.setPort(9042)
.setMaxConnsPerHost(40)
.setSeeds("<EC2-IP>:9042")
.setConnectTimeout(5000)
)
.withConnectionPoolMonitor(new CountingConnectionPoolMonitor())
.buildKeyspace(ThriftFamilyFactory.getInstance());
context.start();
Keyspace keyspace = context.getEntity();
System.out.println(keyspace);
ColumnFamily<String, String> CF_USER_INFO =
new ColumnFamily<String, String>(
"Standard1", // Column Family Name
StringSerializer.get(), // Key Serializer
StringSerializer.get()); // Column
OperationResult<ColumnList<String>> result = null;
try {
result = keyspace.prepareQuery(CF_USER_INFO)
.getKey("user_id_hash")
.execute();
} catch (ConnectionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
ColumnList<String> columns = result.getResult();
// Lookup columns in response by name
String uid = columns.getColumnByName("user_id_hash").getStringValue();
System.out.println(uid);
// Or, iterate through the columns
for (Column<String> c : result.getResult()) {
System.out.println(c.getName());
}
}
}
Error
com.netflix.astyanax.thrift.ThriftKeyspaceImpl#1961f4
com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException: PoolTimeoutException: [host=():9042, latency=5001(5001), attempts=1] Timed out waiting for connection
at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.waitForConnection(SimpleHostConnectionPool.java:201)
at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.borrowConnection(SimpleHostConnectionPool.java:158)
at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.borrowConnection(RoundRobinExecuteWithFailover.java:60)
at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:50)
at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:229)
at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$1.execute(ThriftColumnFamilyQueryImpl.java:180)
at com.rjil.jiodrive.rs.MetadataRS.main(MetadataRS.java:57)
Exception in thread "main" java.lang.NullPointerException
at com.rjil.jiodrive.rs.MetadataRS.main(MetadataRS.java:62)
Since you are running cassandra on EC2 instance, check that the cassandra's port no. (which you have choosen as 9042) is in the allowed list of ec2 security group and that you have access to it. If not add the port no. in the inbound list of the ec2 security group and set the ip ranges as 0.0.0.0.
Alos check the firewall on the ec2 is turned off. By default its false, but its good to check it anyway.
If you have done this, then your client might be behind a firewall that prevents outbound traffic to your choosen port (9042).
Lastly if you have not used any elastic ip, its better to use the ec2 instance dns name, both in your setSeeds section and in the rpc_address of cassandra.yaml
Your issue is not in your code. You have a connectivity problem to the node you specified as your seed. So either that node isn't running, or you can't reach it from the machine running your client.
I finally upgraded libthrift to 0.9 and changed my code to the following nd it working fine now.
public Keyspace getDBConnection() {
if (poolConfig == null) {
poolConfig = new ConnectionPoolConfigurationImpl(
"CassandraPool").setPort(port).setMaxConnsPerHost(1)
.setSeeds((new StringBuilder(seedHost).append(":").append(port).toString()))
.setLatencyAwareUpdateInterval(latencyAwareUpdateInterval) // Will resort hosts per
// token partition every
// 10 seconds
.setLatencyAwareResetInterval(latencyAwareResetInterval) // Will clear the latency
// every 10 seconds. In
// practice I set this
// to 0 which is the
// default. It's better
// to be 0.
.setLatencyAwareBadnessThreshold(latencyAwareBadnessThreshold) // Will sort hosts if a host
// is more than 100% slower
// than the best and always
// assign connections to the
// fastest host, otherwise
// will use round robin
.setLatencyAwareWindowSize(latencyAwareWindowSize) // Uses last 100 latency
// samples. These samples are in
// a FIFO q and will just cycle
// themselves.
.setTimeoutWindow(60000)
;
}
AstyanaxContext<Keyspace> context = new AstyanaxContext.Builder()
.forCluster(clusterName)
.forKeyspace(keyspaceName)
.withAstyanaxConfiguration(
new AstyanaxConfigurationImpl().setDiscoveryType(
NodeDiscoveryType.NONE)
.setConnectionPoolType(
ConnectionPoolType.ROUND_ROBIN)
.setCqlVersion("3.0.0")
.setTargetCassandraVersion("2.0"))
.withConnectionPoolConfiguration(poolConfig)
.withConnectionPoolMonitor(new CountingConnectionPoolMonitor())
.buildKeyspace(ThriftFamilyFactory.getInstance());
context.start();
return context.getClient();
}