spring-boot mongodb ssh tunnel - spring

I've created a mongodb setup using the AWS cloudplatform quickstart found here: https://docs.aws.amazon.com/quickstart/latest/mongodb/deployment.html. I can successfully connect to my from my EC2 instance, so it has been set up properly.
Since I need to connect to it through my spring application, I need to set up an SSH tunnel through the EC2 instance to the mongodb nodes.
The only related answers I could find were using mysql and some sort of DataSource object that required a schema parameter (which I don't have, database will be full of different sorts of entities), so I was unsure of how to proceed.
My questions:
How do I set up the SSH tunnel? How do I pass in my .pem key file?
How will my application.properties look like for connecting to DB?

I setup a bastion in my private subnet and using ssh tunnel to access my MongoDb Cluster through git-bash:
~/.ssh/config
Host bastion-host-mongo-forward
User ec2-user
Port 22
IdentityFile ~/admin.pem
HostName ec2-xx-xxx-xx-xxx.compute-1.amazonaws.com
Localforward 3307 ip-10-50-2-24.ec2.internal:27017
Localforward 3308 ip-10-50-3-143.ec2.internal:27017
Localforward 3309 ip-10-50-3-103.ec2.internal:27017
start the tunnel:
ssh bastion-host-mongo-forward
After that you can setup your application.yaml and target the uri:
spring:
profiles: local
data:
mongodb:
uri: mongodb://user:password#localhost:3307,localhost:3308,localhost:3309/databaseName

Related

Access public PostgreSQL server (Amazon RDS) from personal computer through proxy

I'm new to Amazon Web Service (AWS).
I already created a PostgreSQL from AWS RDS:
Endpoint: database-1.XXX.rds.amazonaws.com
Port: 5432
Public accessibility: Yes
Availablity zone: ap-northeast-1c
After that, I will push my application that using the database to AWS (maybe deploy to EKS).
However, I want to try testing the database server from my local computer first.
I haven't tried testing from my laptop PC at home yet, but I think it will connect OK because my laptop PC is not using the HTTP proxy to connect to the network.
The problem is that I want to try testing from my company PC, which needs setup the HTTP proxy to connect to the internet. The PC spec:
Windows 10
Installed PostgreSQL 10
Firstly, I tried using psql command-line:
psql -h database-1.XXXX.rds.amazonaws.com -U postgre
> Unknown host
set http_proxy=http://user:password#my_company_proxy:3128
set https_proxy=http://user:password#my_company_proxy:3128
psql -h database-1.XXXX.rds.amazonaws.com -U postgre
> Unknown host
set http_proxy=http://my_second_company_proxy:3128
set https_proxy=http://my_second_company_proxy:3128
psql -h database-1.XXXX.rds.amazonaws.com -U postgre
> Unknown host
Then, I tried using the pgAdmin tool.
As from the internet post, it said that we can use "SSH Tunnel" for inputing proxy:
However, the error message will be shown:
So, anyone can help suggest if we can connect to the public PostgreSQL server through HTTP proxy?
I think problem is Postgres uses plain TCP/IP protocol and you are trying to use HTTP proxy. Also you're trying to create SSH tunnel against your HTTP proxy server which won't work.
So I'd suggest following solutions:
Use TCP proxy instead of HTTP proxy
Create an EC2 or any instance that has SSH access from your company network and has access to public internet. So that you can create SSH tunnel through that instance to achieve your goal.
NOTE: Make sure you PostgreSQL is accessible from public internet (although this is usually bad idea, but it's out of scope this question) sometimes security group configs prevent it to connect from public internet.
Just add all ports(5432,3128...) in the Security Group from your RDS and specify your IP. Don't forget "/32"
Let me add that "unknown host" is usually an indication that you're not resolving the DNS hostname. Also, your HTTP proxy should not interfere with connections to databases since they aren't on port 80 or 443. A couple of things you can try (assuming you're on windows) sub in your actual url:
nslookup database-1.XXXX.rds.amazonaws.com
telnet database-1.XXXX.rds.amazonaws.com 5432
You should also check the security group that is attached to your RDS and make sure you've opened up the ip address that you're originating from on port TCP/5432.
Lastly check that your VPC has DNS and Hostnames enabled. https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html#vpc-dns-updating

Unable to connect MongoDB Compass to AWS DocumentDB using SSH tunnel

I am new to mongoDB trying to setup tools for my new project. Most of my infrastructure run on AWS so i prefer to use AWS documentDB. I manage to connect to documentDB from EC2 both via mongo client or NodeJS aplication. but it would be good to mange documentDB from my Windows workstation using MongoDB Compass.
As we know, we can not direct connect any mongo client from outside AWS to DocumentDB Connecting to an Amazon DocumentDB Cluster from Outside an Amazon VPC
so we need SSH tunnel through EC2. I try many options but still fail... below are most likely 2 options:
Option 1: Connect using MongoDB Compass SSH tunnel
Error: unable to get local issuer certificate
both RDS-COMBINED-CA-BUNDLE.PEM and SSH Key already supplied so which one unable to get?
as red highlight on SSH port, I also tried to open another SSHD port on server and tried to connect using second port but still failed.
Option 2: Connect using Putty SSH tunnel
Error: Hostname/IP does not match certificate's altnames...
since MongoDB Compass need to connect to locathost to get into tunnel and i still can not find the way to supply --sslAllowInvalidHostnames options.
So, what i can do to get around this ?
MongoDB Compass: 1.25.0
I am done with Compass.
successful established "robo3t" connection to AWS DocumentDB using this guild.
https://docs.aws.amazon.com/documentdb/latest/developerguide/robo3t.html
As of Jan 2022 MongoDB Compass does not support sslInvalidHostNameAllowed=true in the connection builder form, this is the parameter you are missing in order to connect to AWS DocumentDB while ssh tunneling to a machine inside the same VPC of the database itself.
I used Studio 3T and it worked perfectly. You could create the connection string yourself or try other GUI.
Edit Jan 2023:
I just gave a try to compass again and it seems they now support sslInvalidHostNameAllowed flag through the UI, you could still change manually the connection string but then any UI interaction would overwrite it.
If you edit the connection string directly in MongoDB Compass you can set options that may not be accessible in the user interface.
Below is an example with tweaked parameters to connect without using TLS:
mongodb://xxxx:yyyy#localhost:27017/?authSource=admin&connectTimeoutMS=10000&readPreference=primary&authMechanism=SCRAM-SHA-1&serverSelectionTimeoutMS=5000&appname=MongoDB%20Compass&ssl=false
For Hostname, are you using DocumentDB endpoint? In one screenshot, I see you are using localhost.
I have managed to connect with option 1.
The workaround can be by establish connection using SSH Tunnel (port forward) and so that SSH tunnel opens a port on your local system that connects through to another port at the other end of the tunnel.
Using the below command establishes a tunnel on terminal and later you can use this channel/connection to connect MongoDB using MongoDB Compass.
For example:
ssh user#aws-ec2-ip-address -L 35356:127.0.0.1:27017 -N
where -L as the Local listening side
Port 35356 is listening on localhost (that is in this case your EC2) and port forwards through to port 27017 on remote server.
Note - Add identity file in .ssh/config
Ex - On Mac
Host XXXXXXX
HostName 52.xx.xx.xx
User ubuntu
IdentityFile ./path/prod.pem

How to setup SSH tunnel to connect to my ElasticSearch and MongoDB residing on AWS EC2 server?

I'm trying SSH tunneling for the first time hence, I'm expecting some level of guidance (with explanation) to setup an SSH tunnel so that I from my Windows client machine can connect to things like ElasticSearch and MongoDB that are residing on AWS EC2 Windows Server.
Here is how you can make tunnel to server for MongoDB,
ssh -L 9999:127.0.0.1:27017 user#serverip -NnT
Now you are able to access your remote mongodb through tunnel on port 9999 so you can now connect to mongodb server from local like,
mongo --host 127.0.0.1 --port 9992
The same way you can also create your own tunnel for elasticsearch also by specifying Port of elastic search like below,
ssh -L 9200:127.0.0.1:9200 user#serverip -NnT
Not have more knowledge of accessing elastic through port but this might help.

DBVisualizer Hive SSH Tunnel Ignoring Server Information

I'm attempting to setup a connection to our Hadoop cluster via DBVisualizer.
In order to connect I need to SSH into a server on the domain and then I need to run the command to a remote server (I've not ssh'd onto the Hadoop cluster directly)
I have (figuratively)
Database Server: abcd.efg
Database Port: 12345
Database: Hello
configured for the Database section
SSH Host: hijk.efg
SSH Port: 678
When I attempt a connection, it returns
Could not open client transport with JDBC Uri:
jdbc:hive2://127.0.0.1:-----
Where 127.0.0.1 and ----- appear to be the defaults instead of what I entered.
Any idea how I get the SSH tunnel to use the server configuration I specify?
The SSH Tunnel is set up locally on the client, so connecting to the port on localhost tunnels you to the SSH Host/Port, which then sets up a connection to the database server/port you have specified. This page may help:
http://confluence.dbvis.com/display/UG100/Using+an+SSH+Tunnel
Best Regards,
Hans

ssh to amazon EC2 over proxy

I have some problem connecting to my amazon EC2 server over ssh over proxy.
I have my username and password for http proxy port 8080.(dont have control over proxy)
Also I have my connection string which would work without proxy
ssh -i key.pem root#xx.compute.amazonaws.com
when I am trying to connect I am getting "No route to host" error
I tried to use putty, configured proxy + authentication file, But then I getting this error
"Unable to use this key file (OpenSSH SSH-2 private key)"
Also I dont know how putty inserts my proxy config, into ssh connection string, so I could try it in terminal
I was facing the same problem and this is what I used to connect, using corkscrew. My config file looks like this
Host AWS
Hostname <Public DNS>
Port 443
#Write the appropriate username depending on your AMI, eg : ubuntu, ec2-user
User ubuntu
IdentityFile </path to key file>
ProxyCommand /usr/bin/corkscrew 10.10.78.61 3128 %h %p
then I simply use this command to connect
ssh AWS
and it works flawlessly.
Note : You must edit your sshd_config file on the server to listen to ssh connections on port 443 (in addition to 22) and restart the ssh daemon.
Are you sure you can login as root? Try logging in as ec2-user instead.
Also, if you have assigned an elastic IP to your instance, the public DNS has probably changed. Log in to the aws console, and select your instance. Scroll down to look at the public DNS again and double check you are using the correct xx.compute.amazonaws.com addr.

Resources