How can I create an HTML snapshot, server side, to provide it to Google for rendering(Crawling) and I have to do it on AWS EC2 instance which is the headless server.
So how should I go about this?
Here is the answer for it:
String Xport = System.getProperty("lmportal.xvfb.id", ":99");
final File firefoxPath = new File(System.getProperty("lmportal.deploy.firefox.path", "/opt/firefox/firefox"));
FirefoxBinary firefoxBinary = new FirefoxBinary(firefoxPath);
firefoxBinary.setEnvironmentProperty("DISPLAY", Xport);
FirefoxDriver driver = new FirefoxDriver(firefoxBinary,null);
driver.get(url);
Thread.sleep(1500);
res.setContentType("text/html;charset=UTF-8");
PrintWriter out = res.getWriter();
String str = driver.getPageSource();
out.println(str);
out.close();
driver.quit();
And install the xvfb and firefox on your ec2 instance thats inportant.....
Related
I have created REST API to create EC2 instance using AWS JAVA SDK provided.
Now I am trying to connect to created EC2 instance and then need to install software's in the instance again through java. I didn't find any appropriate article for this. Is there any possible way to do this? I don't want to use SSH client like putty.. Thanks..
Sounds like you're looking for a java ssh client.
You should set up key authentication and use the ssh client library from java to execute the installation for you.
See this post: for a list of solutions
public static void connectToEC2(){
try{
JSch jsch=new JSch();
String user = "User-name";
String host = "host";
int port = 22;
File directory = new File(".");
String privateKey = directory.getCanonicalPath() + File.separator + "pem file path";
jsch.addIdentity(privateKey);
System.out.println("identity added ");
Session session = jsch.getSession(user, host, port);
System.out.println("session created.");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect();
Channel channel=session.openChannel("shell");
channel.setInputStream(System.in);
channel.setOutputStream(System.out);
channel.connect(3*1000);
}
catch(Exception e){
System.out.println(e);
}
}
I am using aws sdk to retrieve cloud data from aws sdk.
I get all ec2 related data but I am not able to find out how to connect instance.
Here is code of retrieve instance from amazon cloud:
IAmazonEC2 ec2Client = new AmazonEC2Client(accesskey,secretkey, new AmazonEC2Config
{
Timeout = TimeSpan.FromSeconds(300),
MaxErrorRetry = 3,
RegionEndpoint = RegionEndpoint.GetBySystemName(regionName)
});
var instanceRequest = new DescribeInstancesRequest();
DescribeInstancesResponse ec2Response = ec2Client.DescribeInstances(instanceRequest);
First of you will need a key/pair file that you've used while creating an instance as it is needed to retrieve windows password.
Following are the steps to retrieve windows instance password using AWS SDK:
#1. You need to pass instanceId as well as RSA key from your .pem file to the following code.
IAmazonEC2 ec2Client = new AmazonEC2Client(accesskey, secretkey, new AmazonEC2Config
{
Timeout = TimeSpan.FromSeconds(300),
MaxErrorRetry = 3,
RegionEndpoint = region,
});
var passwordRequest = new GetPasswordDataRequest();
passwordRequest.InstanceId = instanceId;
var passwordResponse = ec2Client.GetPasswordData(passwordRequest);
var password = passwordResponse.GetDecryptedPassword(rsaKey);
return password;
});
Note: You have to wait at least 4 minutes after launching an instance to get the windows password.
I was trying to create an EMR cluster from Eclipse using Java. I was able to output the job id but when I viewed the EMR web console. There wasn't any cluster created. What's wrong ?
My code:
AWSCredentials credentials = new BasicAWSCredentials("xxx", "xxx");
AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials);
StepFactory stepFactory = new StepFactory();
StepConfig enableDebugging = new StepConfig()
.withName("Enable Debugging")
.withActionOnFailure("TERMINATE_JOB_FLOW")
.withHadoopJarStep(stepFactory.newEnableDebuggingStep());
StepConfig installHive = new StepConfig()
.withName("Install Hive")
.withActionOnFailure("TERMINATE_JOB_FLOW")
.withHadoopJarStep(stepFactory.newInstallHiveStep());
StepConfig hiveScript = new StepConfig().withName("Hive Script")
.withActionOnFailure("TERMINATE_JOB_FLOW")
.withHadoopJarStep(stepFactory.newRunHiveScriptStep("s3://mywordcountbuckett/binary/WordCount2.jar"));
RunJobFlowRequest request = new RunJobFlowRequest()
.withName("Hive Interactive")
.withReleaseLabel("emr-4.1.0")
.withSteps(enableDebugging, installHive)
.withJobFlowRole("EMR_DefaultRole")
.withServiceRole("EMR_EC2_DefaultRole")
//.withSteps(enableDebugging, installHive)
.withLogUri("s3://mywordcountbuckett/")
.withInstances(new JobFlowInstancesConfig()
.withEc2KeyName("mykeypair")
.withHadoopVersion("2.4.0")
.withInstanceCount(5)
.withKeepJobFlowAliveWhenNoSteps(true)
.withMasterInstanceType("m3.xlarge")
.withSlaveInstanceType("m1.large"));//.withAmiVersion("3.10.0");
RunJobFlowResult result = emr.runJobFlow(request);
System.out.println(result.toString());
My output was:
{JobFlowId: j-3T7H65FOSKHDQ}
What could be the reason that I was unable to create the cluster ?
I have changed the ports that Azure Storage Emulator runs on from 10000,10001,10002 to 10003,10004,10005 from the config file at "C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator\WAStorageEmulator.exe.config"
Now when I try to access Development Storage from Server Explorer in Visual Studio 2013 it fails to access the updated ports. I tried to manually add external storage and specify the endpoints to reflect the updated ports with the following info default storage account information:
DefaultEndpointsProtocol=http
AccountName=devstoreaccount1
AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
BlobEndpoint=http://127.0.0.1:10003/devstoreaccount1
QueueEndpoint=http://127.0.0.1:10004/devstoreaccount1
TableEndpoint=http://127.0.0.1:10005/devstoreaccount1
but that still does not allow it to connect. I also tried the same endpoints but without the storage account suffix. It even reverts the ports to 10000,10001,10002 when I refresh the External Storage. I assume it is reading from some config somewhere but I cannot seem to google any answer as to where this is being read from.
So how can I configure Server Explorer to reflect the updated ports?
The ports are hard coded into the CloudStorageAccount class so no you can't modifiy them:
private static CloudStorageAccount GetDevelopmentStorageAccount(Uri proxyUri)
{
UriBuilder uriBuilder = proxyUri != (Uri)null ? new UriBuilder(proxyUri.Scheme, proxyUri.Host) : new UriBuilder("http", "127.0.0.1");
uriBuilder.Path = "devstoreaccount1";
uriBuilder.Port = 10000;
Uri uri1 = uriBuilder.Uri;
uriBuilder.Port = 10001;
Uri uri2 = uriBuilder.Uri;
uriBuilder.Port = 10002;
Uri uri3 = uriBuilder.Uri;
uriBuilder.Path = "devstoreaccount1-secondary";
uriBuilder.Port = 10000;
Uri uri4 = uriBuilder.Uri;
uriBuilder.Port = 10001;
Uri uri5 = uriBuilder.Uri;
uriBuilder.Port = 10002;
Uri uri6 = uriBuilder.Uri;
CloudStorageAccount cloudStorageAccount = new CloudStorageAccount(new StorageCredentials("devstoreaccount1", "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="), new StorageUri(uri1, uri4), new StorageUri(uri2, uri5), new StorageUri(uri3, uri6), (StorageUri)null);
cloudStorageAccount.Settings = (IDictionary<string, string>)new Dictionary<string, string>();
cloudStorageAccount.Settings.Add("UseDevelopmentStorage", "true");
if (proxyUri != (Uri)null)
cloudStorageAccount.Settings.Add("DevelopmentStorageProxyUri", proxyUri.ToString());
cloudStorageAccount.IsDevStoreAccount = true;
return cloudStorageAccount;
}
Unfortunately, there is no support for changing the Azure Storage Emulator ports.
Below code is for donwnloading file from azure blob. I have problem with .docx,.xlsx files and that too after deployment only, mean in local machine it is working fine.
The problem is after downloading .xlsx or .docx, when i open that file showing file corrupted popup.
public void DownloadBlob(string blobName)
{
//You have to get values for below items from azure
string accountName = "MyAccName";
string accountPrimaryKey = "MyKey";
string blobContainer = "ContainerName";
CloudStorageAccount account = new CloudStorageAccount(new StorageCredentialsAccountAndKey(accountName, accountPrimaryKey), false);
CloudBlobClient blobClient = account.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(blobContainer);
CloudBlob blob = container.GetBlobReference(blobName);
MemoryStream memStream = new MemoryStream();
blob.DownloadToStream(memStream);
Response.ContentType = blob.Properties.ContentType;
Response.AddHeader("Content-Disposition", "Attachment; filename=" + blobName.ToString());
Response.AddHeader("Content-Length", (blob.Properties.Length - 1).ToString());
Response.BinaryWrite(memStream.ToArray());
Response.End();
}
I am sure you have issue with your code as As Steve suggested you are setting length incorrectly.
I worked on similar issue sometime last year and documented the solution in my blog as below:
http://blogs.msdn.com/b/avkashchauhan/archive/2011/04/05/downloading-word-and-excel-files-from-windows-azure-storage-in-a-asp-net-web-role.aspx