I use printf to setup my Nginx settings, and it works fine, but it's awkward to read and alter; is there a better way to improve readability?
config.vm.provision "shell", inline: <<-SHELL
sudo rm /etc/nginx/sites-enabled/default
sudo printf '%s\n' 'server {' 'root /home/vagrant/web;' 'index index.php index.html index.htm;' 'location / {' 'try_files $uri $uri/ /index.php?$args ;' '}' 'location ~ \.php$ {' 'fastcgi_split_path_info ^(.+\.php)(/.+)$;' 'fastcgi_pass unix:/var/run/php5-fpm.sock;' 'fastcgi_index index.php;' 'include fastcgi_params;' '}' '}' >> /etc/nginx/sites-enabled/default
sudo service nginx start
SHELL
Assuming that there is Bash on your guest machine, you could try and use nested bash here documents. There is no guarantee that will work with Vagrantfile but it's worth a shot anyway.
config.vm.provision "shell", inline: <<-SHELL
sudo /bin/bash << 'SCRIPT'
rm /etc/nginx/sites-enabled/default;
cat << 'EOF' >> /etc/nginx/sites-enabled/default
%s\n
server {
root /home/vagrant/web;
index index.php index.html index.htm;
location / {
try_files $uri $uri/ /index.php?$args ;
}
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}
}
EOF
service nginx start
SCRIPT
SHELL
Note that EOF, SCRIPT and SHELL must be placed in the very beginning of the line. There must not be any tabs or whitespaces in front of these words.
Related
This is how I create a file (nginx.conf) via shell.
As there are $ characters in the filecontent I'm using EOF.
if [ $type == "nginx" ]; then
cat > ${path}/nginx.conf <<'EOF'
server {
listen 3000;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html =404;
}
include /etc/nginx/extra-conf.d/*.conf;
}
EOF
fi
Now I have to use a dynamic port value, so instead of listen 3000 I need to use listen $port.
But this won't work, as in the content there is also $uri, which should be handled as text, not as a variable.
Using only the delimiter itself, either all parameters are expanded or none. You'll have to allow expansion, but escape the dollar signs for $uri to inhibit their expansion.
if [ "$type" = "nginx" ]; then
cat > "${path}/nginx.conf" <<EOF
server {
listen $port;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files \$uri \$uri/ /index.html = 404;
}
include /etc/nginx/extra-conf.d/*.conf;
}
EOF
fi
The here document behaves like a double-quoted string:
$ foo=bar
$ echo "$foo"
bar
$ echo "\$foo"
$foo
I'm very new to NGINX (and bash) and I'm attempting to write a bash script to automate the creation of a new site (so adding a server block) to a webserver. However for some reason my script appears to be putting me into a redirect loop. Any ideas?
cd /var/www/
git clone git#bitbucket.org:wardy484/portfolio.git
mv portfolio kimward.co.uk
sudo chmod -R 755 kimward.co.uk
FILE="/etc/nginx/sites-available/kimward.co.uk"
/bin/cat <<EOM >$FILE
server {
listen 80;
listen [::]:80;
root /var/www/kimward.co.uk/public;
index index.php index.html index.htm;
server_name kimward.co.uk www.kimward.co.uk;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location ~ \.php$ {
try_files $uri /index.php =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php7.0-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
include fastcgi_params;
}
}
EOM
sudo nano /etc/nginx/sites-available/kimward.co.uk
sudo ln -s /etc/nginx/sites-available/kimward.co.uk /etc/nginx/sites-enabled/
sudo service nginx restart
cd /var/www/kimward.co.uk
composer install
composer update
$uri, $url, $query_string, etc. are nginx variables and needs to be escaped or they will be expanded by the shell:
location / {
try_files \$uri \$uri/ /index.php?\$query_string;
}
Same might be the case with other special characters. Instead of having to escape them all you should use << 'EOM' which will treat the here document as a single quoted string.
file="/etc/nginx/sites-available/kimward.co.uk"
/bin/cat <<'EOM' >"$file"
server {
listen 80;
listen [::]:80;
...
...
EOM
I also lower cased $FILE since all uppercase names are reserved for environment variables.
I am trying to automate the installation in Ubuntu 14.04 of a development environment, part of this requires that I uncomment a block of code for the Nginx vHost configuration:
#location ~ \.php$ {
# fastcgi_split_path_info ^(.+\.php)(/.+)$;
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
#
# # With php5-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php5-fpm:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
# fastcgi_index index.php;
# include fastcgi_params;
#}
I tried sed:
sed -i "s/#location ~ \.php$ {
# fastcgi_split_path_info ^(.+\.php)(/.+)$;
# # NOTE: You should have \"cgi.fix_pathinfo = 0;\" in php.ini
#
# # With php5-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php5-fpm:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
# fastcgi_index index.php;
# include fastcgi_params;
#}/location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(\/.+)$;
# NOTE: You should have \"cgi.fix_pathinfo = 0;\" in php.ini
# With php5-cgi alone:
fastcgi_pass 127.0.0.1:9000;
# With php5-fpm:
fastcgi_pass unix:\/var\/run\/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}/g" /etc/nginx/sites-available/nginx.dev;
But this returns:
sed: -e expression #1, char 22: unterminated `s' command
Which I assume relates to syntax errors, I tried to escape the / and " characters but I think that this is not enough / right at all.
I found this: https://unix.stackexchange.com/a/26289/138115
Which suggested that perl might be a good solution here and as it is installed with Ubuntu I tried it:
perl -0777 -i.original -pe 's/#location ~ \.php$ {\n # fastcgi_split_path_info ^(.+\.php)(\/.+)$;\n # # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini\n #\n # # With php5-cgi alone:\n # fastcgi_pass 127.0.0.1:9000;\n # # With php5-fpm:\n # fastcgi_pass unix:\/var\/run\/php5-fpm.sock;\n # fastcgi_index index.php;\n # include fastcgi_params;\n #}/slocation ~ \.php$ {\n fastcgi_split_path_info ^(.+\.php)(\/.+)$;\n NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini\n With php5-cgi alone:\n fastcgi_pass 127.0.0.1:9000;\n With php5-fpm:\n fastcgi_pass unix:\/var\/run\/php5-fpm.sock;\n fastcgi_index index.php;\n include fastcgi_params;\n}/igs' /etc/nginx/sites-available/nginx.dev;
But this gives a whole host of syntax errors:
syntax error at -e line 1, near "(."
Unknown regexp modifier "/v" at -e line 1, within string
Unknown regexp modifier "/r" at -e line 1, within string
Unknown regexp modifier "/h" at -e line 1, within string
Unknown regexp modifier "/5" at -e line 1, within string
Not enough arguments for index at -e line 1, near "index."
syntax error at -e line 1, near "n}"
Execution of -e aborted due to compilation errors.
I have written quite a few scripts like this before for various environments but I have tried always to avoid replacing multiple lines of text because i've never been able to get it right. Today, i've spent 3 hours and I still have no real understanding on how to make this work out. If anyone could share some input / insight into this and how it could be accomplished then it would be greatly appreciated, thanks!
Edit 1:
Simple escape with square brackets:
#!/bin/bash
#/etc/nginx/sites-available/test.sh
file=$(<default.file);
search=$(<nginx_search.txt);
replace=$(<nginx_replace.txt);
$file =~ s[$search][$replace]g;
echo "$file" > "/etc/nginx/sites-available/test.file";
# Outputs notice: ./test.sh: line 6: #: command not found
The test.file is created but it contains the original values of default.file without the amendment.
After testing in perl I receive:
syntax error at ./perl.perl line 6, near "(."
Unknown regexp modifier "/v" at ./perl.perl line 6, within string
Unknown regexp modifier "/r" at ./perl.perl line 6, within string
Unknown regexp modifier "/h" at ./perl.perl line 6, within string
Unknown regexp modifier "/5" at ./perl.perl line 6, within string
syntax error at ./perl.perl line 9, near ";
}"
Execution of ./perl.perl aborted due to compilation errors.
And line 6 reads:
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
Note - you mention (and tag) perl. The below is from a perl perspective. Some of it may be applicable to conventional shell, but I can't say for sure exactly what. perl does support some regex things that are above and beyond the base POSIX spec.
The problem with patterns like that, is that you've got delimiters in your pattern. Your initial is failing because it'll be treating this slash:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
As the 'split point' in the pattern. You can handle this by escaping the delimiter, but actually a better trick is - use delimiters that don't exist elsewhere. In the above, I'd suggest you can use square brackets:
my $str = "some fish";
$str =~ s[some][more]g;
print $str;
Although as an alternative - you can use a range operator which tests as 'true' if it's within two designated delimitors:
while ( <> ) {
if ( m|\#location.*php\$ \{| .. m|^\s*#\}| ) {
s/#//;
}
print ;
}
E.g.:
#!/usr/bin/perl
use strict;
use warnings;
while ( <DATA> ) {
if ( m|\#location.*php\$ \{| .. m|^\s*#\}| ) {
#note - no g modifier, so we only do the first per line
s/#//;
}
print ;
}
__DATA__
# Some stuff
we don't care about this line
#and this shouldn't be changed
#but after this point, it should be!
#location ~ \.php$ {
# fastcgi_split_path_info ^(.+\.php)(/.+)$;
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
#
# # With php5-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php5-fpm:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
# fastcgi_index index.php;
# include fastcgi_params;
#}
# and now we stop, and leave this bit alone.
more stuff;
here;
# and another comment line
This conditionally applies the transform if you're between two delimiters (location php and 'close squiggly brackets' in the above).
You can one-liner-ify this:
perl -ne 'if ( m|\#location.*php\$ \{| .. m|^\s*#\}| ) { s/#//g; } print' myfile
(and add -i if you want to edit in place).
These long string replacements are always finicky, I think, and it is often best to try and avoid dealing with the content as much as possible. I came up with this, which simply captures the string, takes away the first # and reprints the line:
use strict;
use warnings;
my $search = q|#location ~ \.php$ {
# fastcgi_split_path_info ^(.+\.php)(/.+)$;
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
#
# # With php5-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php5-fpm:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
# fastcgi_index index.php;
# include fastcgi_params;
#}
|;
local $/; # slurp the file
while (<DATA>) {
s|(\Q$search\E)| my $x = $1; $x =~ s/^\s*#//mg; $x; |e;
print;
}
__DATA__
# stuff
#location ~ \.php$ {
# fastcgi_split_path_info ^(.+\.php)(/.+)$;
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
#
# # With php5-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php5-fpm:
# fastcgi_pass unix:/var/run/php5-fpm.sock;
# fastcgi_index index.php;
# include fastcgi_params;
#}
#comment
Note the use of \Q ... \E to avoid the regex meta characters in the string to mess you up.
With this method, you should technically be able also to read the search strings from a file if you would want that.
I want to run a shell script every time my nginx server receives any HTTP request. Any simple ways to do this?
You can execute a shell script via Lua code from the nginx.conf file to achieve this. You need to have the HttpLuaModule to be able to do this.
Here's an example to do this.
location /my-website {
content_by_lua_block {
os.execute("/bin/myShellScript.sh")
}
}
I found the following information online at this address: https://www.ruby-forum.com/topic/2960191
This does expect that you have fcgiwrap installed on the machine. It is really as simple as:
sudo apt-get install fcgiwrap
Example script (Must be executable)
#!/bin/sh
# -*- coding: utf-8 -*-
NAME=`"cpuinfo"`
echo "Content-type:text/html\r\n"
echo "<html><head>"
echo "<title>$NAME</title>"
echo '<meta name="description" content="'$NAME'">'
echo '<meta name="keywords" content="'$NAME'">'
echo '<meta http-equiv="Content-type"
content="text/html;charset=UTF-8">'
echo '<meta name="ROBOTS" content="noindex">'
echo "</head><body><pre>"
date
echo "\nuname -a"
uname -a
echo "\ncpuinfo"
cat /proc/cpuinfo
echo "</pre></body></html>"
Also using this as an include file, not restricted to only shell
scripts.
location ~ (\.cgi|\.py|\.sh|\.pl|\.lua)$ {
gzip off;
root /var/www/$server_name;
autoindex on;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
include /etc/nginx/fastcgi_params;
fastcgi_param DOCUMENT_ROOT /var/www/$server_name;
fastcgi_param SCRIPT_FILENAME /var/www/$server_name$fastcgi_script_name;
}
I found it extremely helpful for what I am working on, I hope it help you out with your RaspberryPI project.
Install OpenResty (OpenResty is just an enhanced version of Nginx by means of addon modules ) Refer https://openresty.org/en/getting-started.html for this
Configure aws cli on the instance
Write a shell script which download a file from specified S3 bucket
Do the required changes in nginx.conf file
Restart the nginx server
I have tested the http request using curl and file gets download in /tmp directory of respective instance:
curl -I http://localhost:8080/
OutPut:
curl -I http://localhost:8080/
HTTP/1.1 200 OK
Server: openresty/1.13.6.2
Date: Tue, 14 Aug 2018 07:34:49 GMT
Content-Type: text/plain
Connection: keep-alive
Content of nginx.conf file:
worker_processes 1;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
server {
listen 8080;
location / {
default_type text/html;
content_by_lua '
ngx.say("<p>hello, world</p>")
';
}
location / {
content_by_lua_block{
os.execute("sh /tmp/s3.sh")
}
}
}
}
If you prefer full control in Python:
Create /opt/httpbot.py:
#!/usr/bin/env python3
from http.server import HTTPServer, BaseHTTPRequestHandler
import subprocess
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self._handle()
def do_POST(self):
self._handle()
def _handle(self):
try:
self.log_message("command: %s", self.path)
if self.path == '/foo':
subprocess.run(
"cd /opt/bar && GIT_SSH_COMMAND='ssh -i .ssh/id_rsa' git pull",
shell=True,
)
finally:
self.send_response(200)
self.send_header("content-type", "application/json")
self.end_headers()
self.wfile.write('{"ok": true}\r\n'.encode())
if __name__ == "__main__":
HTTPServer(("127.0.0.1", 4242), Handler).serve_forever()
No concurrency/parallelism here, so httpbot runs one command at a time, no conflicts.
Run apt install supervisor
Create /etc/supervisor/conf.d/httpbot.conf:
[program:httpbot]
environment=PYTHONUNBUFFERED="TRUE"
directory=/opt
command=/opt/httpbot.py
autorestart=true
redirect_stderr=true
stdout_logfile=/var/log/httpbot.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
Add to your nginx server:
location /foo {
proxy_pass http://127.0.0.1:4242/foo;
}
Run:
chmod u+x /opt/httpbot.py
service supervisor status
# If stopped:
service supervisor start
supervisorctl status
# If httpbot is not running:
supervisorctl update
curl https://example.com/foo
# Should return {"ok": true}
tail /var/log/httpbot.log
# Should show `command: /foo` and the output of shell script
You can also use the nginx mirror module and poxy_pass it to a web script that runs whatever, in my case I just added this to my main site location {...
mirror /mirror;
mirror_request_body off;
and then a new location called mirror that I had run a php script that executed whatever...
location = /mirror {
internal;
proxy_pass http://localhost/run_script.php;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
}
https://nginx.org/en/docs/http/ngx_http_mirror_module.html
You can use nginx's perl module which is usually part of a repo and can be easily installed. Sample to call system curl command:
location /mint {
perl '
sub {
my $r = shift;
$r->send_http_header("text/html");
$r->print(`curl -X POST --data \'{"method":"evm_mine"}\' localhost:7545`);
return OK;
}
';
}
This is a well discussed issue of www.domain.com vs domain.com on nginx. For some reason it doesn't work. Here is my nginx conf file:
server{
server_name www.xyz.com;
return 301 $scheme://xyz.com$request_uri;
}
server {
server_name xyz.com;
access_log /home/access_logs/shiv/access.log;
error_log /home/access_logs/shiv/error.log;
root /home/xyz;
location / {
try_files $uri $uri/ /index.php;
index index.html;
}
location ~ \.php$ {
include /opt/nginx/conf/fastcgi.conf;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /home/xyz$fastcgi_script_name;
}
}
Please point out as to whats wrong with this config !!
QUESTION
xyz.com opens just fine.
www.xyz.com just doesn't open
MY DNS ZONE FILE
$TTL 1800
# IN SOA ns1.abc.com. hostmaster.xyz. (
1376913474 ; last update: 2013-08-19 11:57:54 UTC
3600 ; refresh
900 ; retry
1209600 ; expire
1800 ; ttl
)
IN NS ns1.cpp.com.
NS ns2.cpp.com.
NS ns3.cpp.com.
MX 0 9d209d3837fd2a499a12e566975cce.pamx1.hotmail.com.
# IN A 192.xxx.xxx.154
www IN A 192.xxx.xxx.154
I think your issue is that you didn't define that www subdomain, go to your dns manager and make sure that www points to the same ip or name server