How to run bash script from Nginx - bash

1) I have static site and wand to set "autopull" from bitbucket.
2) I have webhook from bitbucket.
3) I have bash script which do "git pull"
How can I run this script when nginx catch request?
server {
listen 80;
server_name example.ru;
root /path/to/root;
index index.html;
access_log /path/to/logs/nginx-access.log;
error_log /path/to/logs/nginx-error.log;
location /autopull {
something to run autopull.sh;
}
location / {
auth_basic "Hello, login please";
auth_basic_user_file /path/to/htpasswd;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
}
}
I tried lua_block and fastcgi service, but both are failed.
lua does not run os.execute("/path/to/script") and does not write the log.
fastcgi is more successful, but it has not permissions, because my www-data user doesn't have ssh-key in my bitbuchet repo.

Problem solved.
I didnt want to use any script/process on another port because I have several sites and I need port for each.
My final configuration is:
server {
listen 80;
server_name example.ru;
root /path/to/project;
index index.html;
access_log /path/to/logs/nginx-access.log;
error_log /path/to/logs/nginx-error.log;
location /autopull {
content_by_lua_block {
io.popen("bash /path/to/autopull.sh")
}
}
location / {
auth_basic "Hello, login please";
auth_basic_user_file /path/to/htpasswd;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
}
}
Problem was in permission of www-data user and its ssh-kay in repo.

Based on this, create py script
#!/usr/bin/python
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
from subprocess import call
PORT_NUMBER = 8080
autopull = '/path/to/autopull.sh'
command = [autopull]
#This class will handles any incoming request from
#the browser
class myHandler(BaseHTTPRequestHandler):
#Handler for the GET requests
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
# Send the html message
self.wfile.write("runing {}".format(autopull))
call(command)
return
try:
#Create a web server and define the handler to manage the
#incoming request
server = HTTPServer(('', PORT_NUMBER), myHandler)
print 'Started httpserver on port ' , PORT_NUMBER
#Wait forever for incoming htto requests
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down the web server'
server.socket.close()
Run it and in nginx config add
location /autopull { proxy_pass http://localhost:8080; }

Related

Should I change the localhost to be my public IP DNS address inside Nginx server config file?

My front-end in running on port 3000 and my back-end on port 8000.
I've deployed my app using EC2 instance and I've installed Nginx and PM2.
When I try to open the app on the browser using my public DNS address the app appears for a short period of time then it breaks. It shows me the error
net::ERR_CONNECTION_REFUSED
GET http://localhost:8000/api/info/countByType net::ERR_CONNECTION_REFUSED
Here is where my front-end is making the request
export const countByCity = createAsyncThunk("info/countByCity", async() => {
try{
const res = await axios.get("http://localhost:8000/api/info/countByCity");
return res.data;
}catch(error){
console.log(error.response)
}
});
and here is my /etc/nginx/sites-available/default file configuration
server {
listen 80;
listen [::]:80;
root /usr/share/nginx/booking.com;
index index.html index.htm index.nginx-debian.html;
server_name _;
access_log /var/log/nginx/example.com.access.log;
error_log /var/log/nginx/example.com.error.log;
location / {
try_files $uri /index.html;
}
location /api {
proxy_pass http://ec2-54-167-89-197.compute-1.amazonaws.com:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
I have changed the proxy_pass to be my Public IP address. The previous configuration was http://localhost:8000.
My question is : My front end is trying to reach the localhost:8000 endpoint. Should I change it to be my Public DNS address instead? Example : http://ec2-54-167-89-197.compute-1.amazonaws.com/api/info/countByCity ?
This code in your front end is running in the user's web browser, not on the EC2 server:
const res = await axios.get("http://localhost:8000/api/info/countByCity");
When that code tries to access localhost it is trying to access a server running on your local laptop/PC, not a server in AWS. You need to change that localhost address to be the public IP of your EC2 server.
This proxy configuration on your Nginx server:
proxy_pass http://ec2-54-167-89-197.compute-1.amazonaws.com:8000;
Is going all the way out to the Internet and back, just to access a service that is running on a different port of the same server. You need to change that to be localhost for both security and efficiency.

Setting up Varnish on CentOS 7, Nginx and PHP-FPM with SSL

I've not used Varnish before but I need to install it on our Magento site to help speed things up.
I've found lots of articles on how to set up Varnish on Centos 7, PHP-FPM etc but none that runs with CentOS7, Nginx, PHP-FPM AND SSL.
As I understand it, Varnish doesn't natievly work with SSL so you need to do some Nginx jiggery-pokery to get things working.
This is also a multi-store Magento site so that adds another layer of complication.
Does anybody have any information to help with this?
I will show you my own Nginx config files to make this works. This is Debian 9 not Centos 7, but Nginx should works in the same way.
If someone have a better configuration, or advices, i will listen carfully... I am a Magento dev not a system admin. I have a lot to learn about Nginx & Varnish.
Here, Varnish is listening port 6081.
I created a Varnish Proxy to redirect HTTPS requests to HTTP varnish. In /etc/nginx/sites-available/proxy.website.com :
## HTTPS termination & Varnish proxy
server {
server_name en.website.com fr.website.com es.website.com de.website.com;
listen 443 ssl http2;
access_log /var/www/log/varnish-proxy.log;
error_log /var/www/log/varnish-proxy.error.log;
include /etc/nginx/conf/ssl.conf;
keepalive_timeout 300s;
location / {
#BYPASS VARNISH
#proxy_pass http://127.0.0.1:611;
#VARNISH ENABLED
proxy_pass http://127.0.0.1:6081;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header X-Secure on;
proxy_set_header X-Magento-Debug 1;
}
}
Then, my vhost in /etc/nginx/sites-available/website.com :
upstream fastcgi_backend { # USE YOUR OWN CONFIG HERE
# use tcp connection
# server 127.0.0.1:9000;
# or socket
server unix:/var/run/php7.1-fpm.sock;
}
map $http_host $MAGE_RUN_CODE_GLOBAL { # USE YOUR OWN CONFIG HERE
en.website.com en;
fr.website.com fr;
es.website.com es;
de.website.com de;
}
# Redirect to https
server {
server_name en.website.com fr.website.com es.website.com de.website.com;
listen 80;
location ~ /.well-known {
allow all;
}
return 301 https://$http_host$request_uri;
}
# Redirect to https
server {
server_name _;
listen 611;
set $MAGE_ROOT /var/www/magento;
set $MAGE_MODE developer;
set $MAGE_RUN_TYPE store;
set $MAGE_RUN_CODE $MAGE_RUN_CODE_GLOBAL;
set $HTTPS_FORWARD on;
set $FPM_USER www-data;
access_log /var/www/log/website.com.access.log;
error_log /var/www/log/website.com.error.log error;
include /var/www/magento/nginx.conf.sample;
}
Enable your vhosts
sudo ln -s /etc/nginx/sites-available/proxy.website.com /etc/nginx/sites-enabled/
sudo ln -s /etc/nginx/sites-available/website.com /etc/nginx/sites-enabled/
Restart nginx. -t will test your configuration files, -s reload will reload Nginx config without interupting the service :
nginx -t && nginx -s reload
EDIT :
Edit Varnish startup config :
CentOS 6: /etc/sysconfig/varnish
CentOS 7: /etc/varnish/varnish.params
Debian/Ubuntu: /etc/default/varnish
...
## Alternative 2, Configuration with VCL
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,1024m \
-p workspace_backend=256 \
-p http_resp_hdr_len=42000"
...
In Magento admin :
set Stores > Configuration > Advanced > System > Full Page Cache > Caching Application to Varnish Cache
Now clic on the new "Varnish Configuration" Filed
Set Access list and Backend host to localhost. I don't know what are the other options.
Save configuration changes
Clic Export VCL according to your Varnish's version
Upload the Magento VCL
Backup the default varnish VCL /etc/varnish/default.vcl to /etc/varnish/default.vcl.bkp
Put the magento VCL in a new /etc/varnish/default.vcl file.
Edit the first lines :
vcl 4.0; import std;
backend default {
.host = "127.0.0.1";
.port = "404";
}
backend mywebsite {
.host = "127.0.0.1";
.port = "611";
}
acl purge {
"localhost";
}
sub vcl_recv {
if (req.http.host ~ "website.com") {
set req.backend_hint = mywebsite;
} else {
set req.backend_hint = default;
}
...
Sometimes, you will have to handle special cases like disabling Varnish for some URLs.
Go to your /etc/varnish/default.vcl and edit this like you need. It's quite obscur the first time you see the VCL, but in the end it's not that hard to understand.
Or edit your varnish proxy that way :
## HTTPS termination & Varnish proxy
server {
...
location ^~ /sitemap {
#BYPASS VARNISH
proxy_pass http://127.0.0.1:611;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header X-Secure on;
}
...
}

Sinatra, Unicorn and Nginx - Proxy multiple Sinatra Apps

I have multiple Sinatra apps on unicorn + nginx and I want to proxy the second Sinatra app to be on a /app path.
root
root/app
Here is my nginx configuration file:
upstream root {
# Path to Unicorn SOCK file, as defined previously
server unix:/tmp/unicorn.root.com.sock fail_timeout=0;
}
upstream app {
# Path to Unicorn SOCK file, as defined previously
server unix:/tmp/unicorn.app.io.sock fail_timeout=0;
}
server {
listen 80;
# Set the server name, similar to Apache's settings
server_name root.com www.root.com;
# 301 redirect http://root.com$requesturi;
# Application root, as defined previously
root /var/www/root.com/public;
try_files $uri/index.html $uri #root;
location #root {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://root;
}
location /app {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://app;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
access_log off;
}
Using the configuration above I get a 404 from the app application.
How can I achieve that?

Nginx lots of 'hanging' / pending requests on Windows

The config works fine when on a Linux server, but when on Windows, it randomly hangs -- like 50% of the time the pages will load, else it will be in a pending state for over a minute. (pending request can be seen in chrome developer tool
#user nobody;
worker_processes 1;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
server {
listen 80;
server_name 127.0.0.1;
location / {
proxy_pass http://127.0.0.1:9090;
proxy_set_header Host $host;
}
location ~ /(lang|js|css|pic|uploads) {
root "C:/Users/Dan/Desktop/Projects/FF_Main/public";
}
}
I tried Nginx 1.7 and 1.6.
Removing the line proxy_set_header Host $host; fixed the issue.

nginx - rewrite domain.com:port to sub.domain.com

How can i rewrite a domain with a port to a subdomain?
e.q.: domain.com:3000 to sub.domain.com ?
thanks for your help! :)
greetz
If you actually want to redirect (301 or 302) your web traffic
You create a server {} section listening on port 3000 and you just redirect it to another server {} section that is listening on port 80. In each server {} section set the listen property appropriately.
I guess you are trying to handle the redirection within à single server section and according to this page the listen directive applies to a server context
If you want to use nginx as a proxy
Then what you are looking for is the proxy_pass directive. Here is a sample configuration extracted from an config I have to use nginx as a proxy for my rails app (thin). Basically my app runs locally (but it would also work on a remote host) on port 3200 and the relevant nginx config part looks as follow:
upstream my-app-cluster
{
server localhost:3200;
}
server
{
listen 80;
server_name mydomain.com;
root /root/to/public/folder;
access_log /my/app/log/folder/myapp.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://my-app-cluster;
break;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
You could use Passenger in nginx to delivery the Ruby app - that's the method we are currently using.
http://www.modrails.com/

Resources