I'm trying to connect my local standalone MySQL with Cloud Fusion to create and test a data pipeline. I have deployed the driver successfully.
Also, I have configured the pipeline properties with correct values of jdbc string, user name and password but connectivity isn't getting established.
Connection String: jdbc:mysql://localhost:3306/test_database
I have also tried to test the connectivity via data wrangling option but that is also not getting succeeded.
Do I need to bring both the environments under same network by setting up some VPC and tunneling?
In your example, I see that you specified localhost in your Connection String. localhost is only advertised to other services running local to your machine, and Cloud Data Fusion (running in GCP) will not be able to reach the MySQL instance (running on your machine). Hence you're seeing the connectivity issue.
I highly recommend looking at this answer on SO that will help you setup a quick proof-of-concept.
I think that your question is more related to the way how to connect some on-premise environments to GCP networking system that gathering Google cloud instances or resources throughout VPC connection model.
Admitting the fact that GCP is actually leveraging different approaches for connection methods within a Hybrid cloud concepts, I would encourage you to learn some fundamental principles of Cloud VPN as a essential part of performing secure connection between particular VPN Peer Gateway and Cloud VPN Gateway and further creating a VPN tunnel between parties.
I guess there is even dedicated chapter in GCP documentation about Data Fusion VPC peering implementation that might be helpful in your user case.
Azure Connect is a service found on the older Azure.com portal and allows connectivity between on-premise and cloud servers/roles/resources. It creates a virtual IP (overlay) network - pretty much a VPN.
Azure Virtual Network (found on the new Azure portal) is ALSO touted as a VPN solution for also the same purpose however the configuration seems a lot twisted (although with a pretty UI).
I'm confused how these two product stack up against each other. Googling and searching MSDN didn't reveal much information either.
What are the differences between them and the target use-cases? Are they expected to be merged into one product down the road?
The use case for us is a WebRole that's running as a cloud service, whose REST/Web API services are consumed by machines on a private network. Azure Connect or Azure Virtual Network would (should?) provide the underlying connectivity between them.
Azure Connect allows users to connect Azure applications with on-premise servers in a super simple and quick way. It does not require VPN devices, it does not require user to have network knowledge, it does not require/assume user have access to network infrastructure (e.g. ability to configure the firewall at company's edge firewall). You express your connectivity intent (e.g. Azure service x should connect to a set of machines (machine group) y on-premise) in the management portal, Azure Connect does the rest for you. It is also very flexible in that you can change the network and connectivity policy at any time via the portal, without requiring redeployment of your app or any change on-premise. e.g. you can make Azure service x to connect to machine group z on-premise instead of y, once you make that change in portal, the rest happens automatically, machines in y are not long accessible to/from Azure. Azure Connect uses endpoint software to manage all the network connectivity for users, so you do have to install endpoint software. But it supports many different automatic deployment options including using Microsoft Update.
Azure Virtual Network allows user to extend part of their on-premise infrastructure to your Azure virtual network via standard site-to-site IPSEC connection. You must have an internet facing VPN device at on-premise side. The solution also assumes you have network knowledge - you will be asked to specify the network address range you will be using at both Azure and on-premise sides, you will must launch a VPN gateway at Azure side and manage the IPSEC connection. It does not require install endpoint software on servers, you are responsible for setting up routes to route the traffic from VPN device to servers and vice versa.
The two technologies complement each other, they are suitable for different scenarios.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
Currently moving to Amazon EC2 from another VPS provider. We have your typical web server / database server needs. Web servers in front of our database servers. Database servers are not directly accessible from the Internet.
I am wondering if there is any reason to put these servers into an AWS Virtual Private Cloud (VPC) instead of just creating the instances and using security groups to firewall them off.
We are not doing anything fancy just a typical web app.
Any reason to use a VPC or not using a VPC?
Thanks.
NOTE: New accounts in AWS launch with a "default VPC" enabled immediately, and make "EC2-Classic" unavailable. As such, this question and answer makes less sense now than they did in August 2012. I'm leaving the answer as-is because it helps frame differences between "EC2-Classic" and the VPC product line. Please see Amazon's FAQ for more details.
Yes. If you're security conscious, a heavy CloudFormation user, or want complete control over autoscaling (as opposed to Beanstalk, which abstracts certain facets of it but still gives you complete access to the scaling parameters), use a VPC. This blog post does a great job summarizing both the pros and cons. Some highlights from the blog post (written by kiip.me):
What’s Wrong with EC2?
All nodes are internet addressable. This doesn’t make much sense for nodes which have no reason to exist on the global internet. For example: a database node should not have any public internet hostname/IP.
All nodes are on a shared network, and are addressable to each other. That means an EC2 node launched by a user “Bob” can access any of EC2 nodes launched by a user “Fred.” Note that by default, the security groups disallow this, but its quite easy to undo this protection, especially when using custom security groups.
No public vs private interface. Even if you wanted to disable all traffic on the public hostname, you can’t. At the network interface level each EC2 instance only has one network interface. Public hostnames and Elastic IPs are routed onto the “private” network.
What's Great About the VPC
First and foremost, VPC provides an incredible amount of security compared to EC2. Nodes launched within a VPC aren’t addressable via the global internet, by EC2, or by any other VPC. This doesn’t mean you can forget about security, but it provides a much saner starting point versus EC2. Additionally, it makes firewall rules much easier, since private nodes can simply say “allow any traffic from our private network.” Our time from launching a node to having a fully running web server has gone from 20 minutes down to around 5 minutes, solely due to the time saved in avoiding propagating firewall changes around.
DHCP option sets let you specify the domain name, DNS servers, NTP servers, etc. that new nodes will use when they’re launched within the VPC. This makes implementing custom DNS much easier. In EC2 you have to spin up a new node, modify DNS configuration, then restart networking services in order to gain the same effect. We run our own DNS server at Kiip for internal node resolution, and DHCP option sets make that painless (it just makes much more sense to type east-web-001 into your browser instead of 10.101.84.22).
And finally, VPC simply provides a much more realistic server environment. While VPC is a unique product to AWS and appears to “lock you in” to AWS, the model that VPC takes is more akin to if you decided to start running your own dedicated hardware. Having this knowledge beforehand and building up the real world experience surrounding it will be invaluable in case you need to move to your own hardware.
The post also lists some difficulties with the VPC, all of which more or less relate to routing: Getting an internet gateway or NAT instance out of the VPC, communicating between VPCs, setting up a VPN to your datacenter. These can be quite frustrating at times, and the learning curve isn't trivial. All the same, the security advantages alone are probably worth the move, and Amazon support (if you're willing to pay for it) is extremely helpful when it comes to VPC configuration.
Currently VPC has some useful advantages over EC2, such as:
multiple NICs per instance
multiple IP's per NIC
'deny'-rules in security-groups
DHCP options
predictable internal IP ranges
moving NICs and internal IPs between instances
VPN
Presumably Amazon will upgrade EC2 with some of those features as well, but currently they're VPC-only.
VPCs are useful if your app needs to access servers outside of EC2, e.g. if you have a common service that's hosted in your own physical data center and not accessible via the internet. If you're going to put all of your web and DB servers on EC2, there's no reason to use VPC.
Right now VPC is the only way to have internal load balancers
If you choose RDS to provide your database services, you can configure DB Security Groups to allow database connections from a given EC2 Security Groups, then even if you have dynamic IP addresses in your EC2 cluster, the RDS will automatically create the firewall rules to allow connections only from your instances, reducing the benefit of a VPS in this case.
VPS in the other hand is great when your EC2 instances have to access your local network, then you can establish a VPN connection between your VPS and your local network, controlling the IP range, sub networks, routes and outgoing firewall rules, which I think is not what you are looking for.
I would also highly recommend trying the Elastic Beanstalk, which will provide a console that makes easy to setup your EC2 cluster for PHP, Java and .Net applications, enabling Auto Scaling, Elastic Load Balancer and Automatic Application Versioning, allowing easy rollback from bad deployments.
You have raised a good concern here.
I would like to focus on the viability in terms of cost...
What about the cost factor?
I think You will be paying for that server per hour. Even if you pick $20-$50 dollars a month instance it will be something you will pay the rest of your server life. The VPN server is something you can easily set on old hardware very cheap and even free for open source solution.
Adding VPN to existing AWS servers park make sense, setting a solo VPN server on AWS doesn't. I don't think it is the best cost-effective option but that just my opinion.
Thanks,
Alisa
I sell a product that runs on Amazon EC2. A company now wants to purchase and install it within their perimeter... This also implies the use of a VPN connection to the EC2 datacenter.
I want to test my product using Amazon VPN (VPC) before handing over the code. Must I change my code to make it work across VPC? If I run on Windows, then wants the quickest and easiest desktop VPN client avaialable that will allow me to connect across VPN to the Amazon datacenter?
Make sure you setup NAT servers and set your routes in the AWS console. Your client can have some security infrastructure for extending their data center to the cloud - firewall rules at the VPC level etc. Disable firewall rules on the server you deploy to since your VPC already takes care of this. As root execute the following command. service iptables stop (you probably already know this I am guessing)
Is it important for your app to run across VPCs?
Depending on how large the company you are selling to is, their security team may give them the run around to have VPC to VPC communication. Is it important for your software to span across VPCs?
I would like to setup an Amazon's VPC gateway to my server. I do not have a Cisco or Juniper router, but found the OpenSolaris VPC Gateway, supposedly addressing this use case (see their wiki for details).
Anyone tried to build this on Linux?
I ended up using openvpn access server ami in a vpc:
http://openvpn.net/index.php?option=com_content&id=493
and
http://sysextra.blogspot.com/2011/01/creating-virtual-private-cluster-with.html
for setting the iptables or using your own openvpn server
Using an Amazon Virtual Private Cloud (VPC) gateway without advanced/expensive hardware routers is meanwhile much easier, because AWS has just dropped the requirement to establish Border Gateway Protocol (BGP) peerings in order to use the built in VPN connectivity, see Amazon VPC - Additional VPN Features:
You can now create Hardware VPN connections to your VPC using static
routing. This means that you can establish connectivity using VPN
devices that do not support BGP such as Cisco ASA and Microsoft
Windows Server 2008 R2. You can also use Linux to establish a
Hardware VPN connection to your VPC. In fact, any IPSec VPN
implementation should work. [emphasis mine]
The outlined reason for this change specifically highlights BGP as a previous barrier to adoption of this otherwise very appealing VPN connectivity to a VPC:
First, BGP can be difficult to set up and to manage, [...]. Second, some firewalls and entry-level
routers support IPSec but not BGP. These devices are very popular in
corporate branch offices. As I mentioned above, this change
dramatically increases the number of VPN devices that can be used to
connect to a VPC. [...]
I couldn't agree more - accordingly, if so desired, you could drop OpenVPN now in favor of a connection between the built in Linux IPSec stack (or a dedicated package like Openswan/strongSwan) and the respective built in VPC IPSec functionality.