I am still learning my way around AWS. I am trying to connect an AWS Batch(Fargate)job to MongoDB while it is running. I know that I need to use AWS privatelink between AWS and Mongo but from there I have tried finding a clear and concise way of establishing a connection.
I have tried creating my own vpc with the privatelink and internet gateway then tried configuring that Batch Job to the correct subnets but I still either a ServerSelectionTimeOut(i.e I'm not on the whitelist) and other times its Cannotpullcontainererror. I really am at a lost and I just need a clear heres how you do it from the ground up
Related
Is there a way to host Moqui on AWS? I was trying to host Moqui using a EC2 instance but couldn't figure out a way to connect them.
The Run and Deploy document on moqui.org has a section for a simple recommended deployment using ElasticBeanstalk and RDS:
https://www.moqui.org/m/docs/framework/Run+and+Deploy#AWSElasticBeanstalkandRDS
With more details about how you want to set things up on AWS the answer to how might vary from this.
For clustered setups things get more involved to get the right settings for Hazelcast AWS discovery and it is best to use an external ElasticSearch server like an AWS ElasticSearch instance and configure Moqui using environment variables to use the Java REST Client mode instead of the Embedded Node mode. Settings for the moqui-hazelcast and moqui-elasticsearch components can be seen in the MoquiConf.xml file in each component.
I launched one RDS instance,s3 and EC2 in AWS and its is triggered properly using lambda. Now I wish to change the change the RDS and EC2 from AWS to local machine. My lambda is triggered from s3.
How do I connect the local database through lambda in AWS?
It appears that your requirement is:
You wish to run an AWS Lambda function
Within the function, you wish to connect to a database running on your own computer (outside of AWS)
Firstly, I would not recommend this strategy. To maintain good performance, you should always have an application as close as possible to the database. This means on the same network, in the same location and not going across remote network connections or the Internet.
However, if you wish to do this, then here's some things you would need to do:
Your database will need to be accessible on the Internet, so that you can connect to it remotely. To test this, try accessing it from an Amazon EC2 instance.
The AWS Lambda function should either be configured without VPC connectivity (which means that it is connected to the Internet) or, if you have configured it for VPC connectivity, it needs to be in a Private Subnet with a NAT Gateway enabling Internet access.
(Optional) For added security, you could lock-down your database to only accept connections from a known IP address. To achieve this, you would need to use the VPC + NAT Gateway so that all traffic is coming from the Elastic IP address assigned to the NAT Gateway.
I agree with John Rotenstein that connecting your local machine to a Lambda running on AWS is probably a bad idea.
If your intention is to develop or test locally, I recommend the serverless framework, and the serverless-offline plugin. It will allow you to simulate Lambda locally, and you can pass database config values through as environment variables.
See: Running AWS Lambda and API Gateway locally: serverless-offline
I have created an instance of marklogic via Market place and can't connect to 8001 port. Also tried to create ti via Cloud Formation and the same problem can't connect to 8001. I used the guides https://developer.marklogic.com/products/cloud/aws
I double checked several time Security groups and make sure 22 and 8001 are available publicly, but still no luck to connect. Could you advise me what could be the problem?
Believe it or not, but it looks like the answer is in the little note in brackets (NB: MarkLogic AMIs will not run on micro instances). Therefore, I simply tried to create a small instance instead of micro and all worked.
I am using the official Docker image for RethinkDB. I am trying to use AWS EC2 Container Services to create a RethinkDB cluster. I can easily get stand alone instances to run, but have had no luck creating a RethinkDB cluster.
I have tried various security group settings. I even made everything wide open, but no luck. When I launch the Docker image, I pass in --bind all and --join [ip]:29015, but nothing.
Has anyone got this to work?
The default networking for docker on amazon ECS is the docker0 bridge. This means multiple containers on the same EC2 instance can talk to each other through the bridge but not to other EC2 instances and containers across the ECS cluster.
You could set the networkMode in your task definition to 'host' which should then let you use the network on your EC2 instances directly and use the security groups you have defined See http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#network_mode.
The alternative is to setup an overlay network using something like flannel, weave, openvswitch etc. See https://aws.amazon.com/blogs/apn/architecting-microservices-using-weave-net-and-amazon-ec2-container-service/ for an example using weave.
I am planning to migrate from Ec2 classic to EC2 VPC. My application reads messages from SQS, download assets from S3 and perform actions mentioned in the SQS messages and then updates RDS. I have following queries
Is it beneficial for me to migrate to Amazon VPC from Classic
I create my EC2 machines using ruby scripts, and deploy code on them using capistrano. In classic mode I used the IP address to deploy code using capistrano. But in VPC there is a concept of private IP address and you cannot access a machine inside a subnet.So my question is:
How should I deploy code on the EC2 instances or rather how should I connect to them?
Thank You.
This questions is pretty broad but I'll take stab at it:
Is it beneficial for me to migrate to Amazon VPC from Classic
It's beneficial if you care about security of your data in transit and at rest. In a VPC none of your traffic is exposed to the outside and you can chose which components you want to expose in case you want to receive traffic/data from the outside. i.e Your ELB or ELBs.
I create my EC2 machines using ruby scripts, and deploy code on them using capistrano. In classic mode I used the IP address to deploy
code using capistrano. But in VPC there is a concept of private IP
address and you cannot access a machine inside a subnet. So my question
is: How should I deploy code on the EC2 instances or rather how should
I connect to them?
You can actually assign a public IP to your EC2 machines in a VPC if you choose to. You can use that IP to deploy your code from the outside.
You can read about it here: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-ip-addressing.html
If you want more security you can always deploy from a machine in your VPC (that has SSH access to the outside). You can ssh to that machine and then run cap deploy from there.