I have an Elastic Stack (8.0.1) consisting of Elasticsearch, Logstash, and Kibana all running within Docker containers deployed to private subnet using AWS EC2. For now, I have the entire Elastic Stack running in a single AWS EC2 instance (this is just for our initial small test environment; I know this is not the way Elasticsearch is intended to be run).
I have Nginx sitting in a public subnet acting as a proxy to the various Elastic Stack components which are all separated by their port numbers: :9200 (elasticsearch), :8080 (logstash HTTP plugin), :5601 (kibana).
Since Kibana 8+ is configured by default using TLS, I also installed Let's Encrypt with Nginx to create signed certificates that would work with browsers and maintain the SSL connectivity all the way back.
What I have works perfect for the default URL works because Nginx redirects port 80 to 443 and then to port 5601 for Kibana interaction.
What I want to do now is allow HTTPS connectivity for other ports: e.g. :9200 and :8080 (logstash http plugin). For example, I want to be able to interact with Logstash via cURL at port 8080:
`curl -0 -v XPUT --user elastic: 'https://elastic.example.com:8080//<doc#> -H 'Content-Type: text/csv; charset=utf-8' --data-binary "#/filename.txt"
(NOTE: running this cURL command locally on my Dockerized Elastic Stack without Nginx works great)
I don't understand how to modify the Nginx configuration that was created by Let's Encrypt to also forward traffic on to other ports (:9200 and :8080).
Here's the nginx.conf that is currently loaded into /etc/nginx/nginx.conf
http {
server {
server_name elastic.example.com;
location / {
proxy_pass https://10.6.101.20:5601;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/elastic.example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/elastic.example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = elastic.example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
server_name elastic.example.com;
listen 80;
return 404; # managed by Certbot
}
}
What confuses me is that it appears as if I need nested "listen" lines, one for 443 SSL and others for the :8080 and :9200 e.g.
http {
server {
listen 443 ssl; # managed by Certbot
listen 8080; # <-- Nested somehow?
server_name elastic.example.com;
location / {
proxy_pass https://10.6.101.20:8080; # <-- Proxy forward to IP and Port
}
...
}
After reading more, I realized that "ssl" did not have to be limited to port 443 and I could use "ssl" on any of the ports. (Therefore, I didn't need to "nest" the nginx config) Duh! So, all I needed to do for nginx.conf to be able to be the SSL reverse proxy for elasticsearch was to add the following:
http {
server {
server_name elastic.example.com;
location / {
proxy_pass https://10.6.101.20:5601;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/elastic.example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/elastic.example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
server_name elastic.example.com;
location / {
proxy_pass https://10.6.101.20:9200;
}
listen 9200 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/elastic.example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/elastic.example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = elastic.example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
server_name elastic.example.com;
listen 80;
return 404; # managed by Certbot
}
}
So, I'm now able to do things like:
curl --user elastic:<password> -XGET "https://elastic.example.com:9200/_cluster/state?pretty"
And get back some very basic cluster information.
BUT, adding another block to the nginx.conf to be able to do something similar via the HTTP plugin for Logstash failed to work. I suspect it's something with the Nginx "location /" specification because I continue to get permission denied error, but maybe this is a question for another SO?
Related
I have a spring boot app running on 8080 (not https as I am not sure if this also need https enabled)
There is an nginx server that redirects requests from 80 (or 443/8443) to 8080
The nginx is secured using letsencrypt. I see this domain file in sites-enabled folder
created certificate using
sudo certbot --nginx -d {dom}.co.uk -d www.{dom}.co.uk
server {
root /var/www/{mydomain}.co.uk/html;
index index.html index.htm index.nginx-debian.html;
server_name {mydomain}.co.uk www.{mydomain}.co.uk;
location / {
#try_files $uri $uri/ =404;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_pass "http://127.0.0.1:8080";
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/{mydomain}.co.uk/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/{mydomain}.co.uk/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = www.{mydomain}.co.uk) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = {mydomain}.co.uk) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
listen [::]:80;
server_name {mydomain}.co.uk www.{mydomain}.co.uk;
return 404; # managed by Certbot
}
OAuth 2 settings
In OAuth Credentials
Authorized Javascript urls (For use with requests from a browser)
https://{dom}.co.uk
Authorized redirect URIs (For use with requests from a web server)
https://{dom}.co.uk/login/oauth2/code/google
Configured redirect URL
private static API_BASE_URL = "https://{dom}.co.uk/";
private static OAUTH2_URL = AppConstants.API_BASE_URL + "oauth2/authorization/";
Question:
How to fix my
Authorisation Error
Error 400: redirect_uri_mismatch
Do I need to make my spring app also https enabled
(OR)
Any config issue nginx or redirect url etc ?
The redirect_uri you send to Google when initiating the flow must match what you put in the console.
Here you have:
https://example.co.uk/oauth2/authorization/ in the code and
https://example.co.uk/login/oauth2/code/google in the console.
Change either one to match the other. I suggest that you change your code to avoid waiting a good 5 minutes for the changes in the console to propagate.
User must verify their email address so I use laravel email verification.
I configured the project on Ubuntu20.04 and with nginx. Verification link works when I use let's encrypt certificate.
I followed all the steps and configured cloudflare and I followed digitalocean tutorial for adding cloudflare ssl certificate.
This is the nginx configuration for domain
server {
listen 80;
listen [::]:80;
server_name ishtap.az www.ishtap.az;
return 302 https://$server_name$request_uri;
}
server {
# listen 80;
# ssl(created in cloudflare) configuration follwing digitalocean tutorial
# https://www.digitalocean.com/community/tutorials/how-to-host-a-website-using-cloudflare-and-nginx-on-ubuntu-20-04
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
ssl_client_certificate /etc/ssl/cloudflare.crt;
ssl_verify_client on;
server_name ishtap.az www.ishtap.az;
root /var/www/ishtap.az/public;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options "nosniff";
index index.php;
charset utf-8;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location = /favicon.ico { access_log off; log_not_found off; }
location = /robots.txt { access_log off; log_not_found off; }
error_page 404 /index.php;
location ~ \.php$ {
fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
include fastcgi_params;
}
location ~ /\.(?!well-known).* {
deny all;
}
# listen 443 ssl; # managed by Certbot
# ssl_certificate /etc/letsencrypt/live/ishtap.az/fullchain.pem; # managed by Certbot
# ssl_certificate_key /etc/letsencrypt/live/ishtap.az/privkey.pem; # managed by Certbot
# include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
# ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
So when I click email verification link I get error in the attached
image. If cloudflare paused it works no problem but at some route like
where you input sensitive data like password chrome not makes request
and throws "your connection to this site is not fully secured"
In this case, you will have to repair the certificate since the certificate chain is unable to see where is the exact location or the cert is unable to be decrypted. In other words, you can use this tool
To use this tool, you will have to use a Windows machine.
This tool is from DigiCert, you can open it and you can click on SSL, and you can select the cert and click on repair. Also, you can create a new CSR, and you can reissue the cert once again to be able to upload it one more time to your server.
Let me know if you have any other questions or concerns, and I would be more than happy to help you.
The email verification notification is sent in queue and I use supervisor in ubuntu. I find out that there is something wrong with laravel .env file
This is the steps:
fixed APP_URL in .env file to https version of domain
php artisan cache:clear
php artisan config:clear
sudo systemctl reload nginx
php artisan config:cache
supervisorctl restart all restarts all workers
https://www.greenhousemarketplace.com
After freshly installing certbot and forcing HTTPS redirect, my CSS and JS no longer loads, even though it is accessible via direct URL.
I'm not sure why, I've updated the links to the CSS and JS files, and set my config.toml to include the https prefix.
sites-enabled/ghm-landing-page
server {
root /var/www/ghm-landing-page/public/;
index index.html index.htm index.nginx-debian.html;
server_name greenhousemarketplace.com;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/greenhousemarketplace.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/greenhousemarketplace.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = greenhousemarketplace.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80 default_server;
listen [::]:80 default_server;
server_name greenhousemarketplace.com;
return 404; # managed by Certbot
config.toml
# Site settings
baseurl = "https://www.greenhousemarketplace.com/"
languageCode = "en-us"
title = "Greenhouse Marketplace"
theme = "hugo-highlights-theme"
The Javascript is not loading because you are loading mixed content. The script tags at the bottom of the page should use the https:// scheme.
The CSS is not loading because of a SSL_ERROR_BAD_CERT_DOMAIN error. You have the content loading on www., which is a domain not listed on your certificate. Using your cert issuer, be sure to add both the www. and non-www. domains of your domain.
I have a React application running with NGINX which handles traffic on one port (www.domain.com - https) and I also have a back-end Spring Boot application which runs on a different port (www.domain.com:7080 - http).
Now NGINX serves 80, 443 ports and loads up my React application. My react application is hard-coded to send requests to www.domain.com:7080, however all requests fail. In the browser's console I can see the following error:
The page at 'https:// domain.com/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http:// domain.com:7080/auth/login'. This request has been blocked; the content must be served over HTTPS.
My NGINX configuration:
server {
listen 443 ssl; # managed by Certbot
root /var/www/ui;
server_name www.domain.com domain.com;
ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
location / {
index index.html;
}
}
server {
listen 80;
if ($host = domain.com) {
return 301 $host$request_uri;
} # managed by Certbot
server_name www.domain.com domain.com;
return 301 https://$host$request_uri; # managed by Certbot
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE-HOST $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
My back-end application is served over http and I'd like to permit the front-end to talk to the back-end service.
I couldn't locate a similar question or tutorial on how I would go about solving this therefore I'm hoping to get some answers here :3
create api endpoint in your domain i.e. www.domain.com/api and configure nginx to pass traffic from that endpoint to your backend with proxy_pass directive. You'll have secure connection from your users and won't need to change anything in your backend server.
When I head over to my public DNS and attempt to load an Auth route, it sends me to http://localhost:8080/ExoscapeWebsite/public/login. Inside of my .env file, I have altered the APP_URL to my public DNS:
APP_URL=http://exoscape.co.uk
I am using NGINX to load Laravel by proxy_pass since I couldn't get try_files working so I stuck with using Apache2. My NGINX configuration looks like this:
server {
server_name exoscape.co.uk;
location / {
proxy_pass http://localhost:8080/ExoscapeWebsite/public/;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/exoscape.co.uk/fullchain.pem; # manag$
ssl_certificate_key /etc/letsencrypt/live/exoscape.co.uk/privkey.pem; # man$
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Any ideas on how I could change the hostname in Laravel so it continues with the correct domain?