Automate nginx installation via passenger - ruby

Every time we reinstall nginx using passenger, we have to provide to it a bunch of options to define the paths, modules and options we'd like to activate, which can be a bit of pain.
Does anybody know of a way to automate this using some kind of answer file?

Yes you can use a provisioning tool like Chef or Puppet but in some cases those can be overkill. I just have a Bash script with some variables at the top that get changed as necessary, I then just transfer the script to the target server and execute it.
The actual packages for nginx & passenger I keep in my own S3 bucket.
#!/bin/sh
# setup download location
DOWNLOAD_DIR=/tmp
NGINX="nginx-1.4.1"
NGINX_DIR=${DOWNLOAD_DIR}/${NGINX}
PASSENGER_VERSION="4.0.3"
S3HOST="your-s3-host"
# download nginx source
curl http://${S3HOST}.s3.amazonaws.com/software/${NGINX}.tar.gz -o $DOWNLOAD_DIR/${NGINX}.tar.gz
tar xzf $DOWNLOAD_DIR/${NGINX}.tar.gz --directory $DOWNLOAD_DIR
# download headers module
curl http://${S3HOST}.s3.amazonaws.com/software/headers-more-nginx-module-v0.16.zip -o $DOWNLOAD_DIR/headers-more-nginx-module-v0.16.zip
unzip $DOWNLOAD_DIR/headers-more-nginx-module-v0.16.zip -d $DOWNLOAD_DIR
# install passenger
apt-get install -y gcc g++ build-essential bison openssl libreadline6 lsof
apt-get install -y libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libpcre3-dev
apt-get install -y libyaml-dev libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev libcurl4-openssl-dev
apt-get install -y default-jre python-software-properties imagemagick
apt-get install -y ruby1.9.3 ruby1.9.1-dev libruby1.9.1 rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 s3cmd
ln -nfs /var/lib/gems/1.9.1/bin/* /usr/bin/
gem install passenger -v ${PASSENGER_VERSION} --no-ri --no-rdoc
gem install bundler sass --no-ri --no-rdoc
# postgres client libraries
aptitude install software-properties-common -y
add-apt-repository -y ppa:pitti/postgresql
apt-get update
apt-get install -y postgresql-client-9.2 postgresql-server-dev-9.2
wget http://${S3HOST}.s3.amazonaws.com/software/passenger-enterprise-server-${PASSENGER_VERSION}.gem
gem install ./passenger-enterprise-server-${PASSENGER_VERSION}.gem --no-ri --no-rdoc
/var/lib/gems/1.9.1/gems/passenger-enterprise-server-${PASSENGER_VERSION}/bin/passenger-install-nginx-module \
--auto --prefix=/usr --nginx-source-dir=${NGINX_DIR} \
--extra-configure-flags="--conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --http-client-body-temp-path=/var/tmp/nginx/client --http-proxy-temp-path=/var/tmp/nginx/proxy --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi --with-md5-asm --with-md5=/usr/include --with-sha1-asm --with-sha1=/usr/include --without-http_fastcgi_module --with-http_stub_status_module --with-http_ssl_module --add-module=${DOWNLOAD_DIR}/headers-more-nginx-module-v0.16"
# make proper folders
mkdir -p /etc/nginx/sites-enabled
mkdir -p /etc/nginx/conf.d
mkdir -p /var/tmp/nginx
mkdir -p /etc/nginx/ssl/rapidssl
mkdir -p /var/www/shared/log/old_logs
chown -R vino:vino /var/www
# cleanup the stupid files
rm /etc/nginx/*.default
# start from init script
update-rc.d nginx defaults
# cleanup
rm -rf $DOWNLOAD_DIR/nginx*

passenger-install-apache2-module supports non-interactive, automatic, headless installs or upgrades through command line options. You can use those options to automate answers. This is documented in the manual: http://www.modrails.com/documentation/Users%20guide%20Nginx.html#_non_interactive_automatic_headless_installs_or_upgrades
Alternatively, you can install Phusion Passenger as if it's a normal Nginx module, and use the normal Nginx installation automation scripts: http://www.modrails.com/documentation/Users%20guide%20Nginx.html#_installing_as_a_normal_nginx_module_without_using_the_installer

You could use a provisioning tool such as chef or puppet. It may be overkill to use it for this case on its own, but you could use it to prepare entire servers or workstations from the ground up.

Related

How do I merge 2 directories with the same name in shell script

I am programming something and one of the things I need to do is to merge 2 directories. How do I do that? Rsync does not work. I tried compiling but after a few seconds it gave me errors.
These are the commands I ran (I ran these because the INSTALL.md said so):
sudo apt install -y gcc g++ gawk autoconf automake python3-cmarkgfm
sudo apt install -y acl libacl1-dev
sudo apt install -y attr libattr1-dev
sudo apt install -y libxxhash-dev
sudo apt install -y libzstd-dev
sudo apt install -y liblz4-dev
sudo apt install -y libssl-dev
./configure
make
sudo make install
a simple and efficient way is with cp
cp --force --archive --update --link ./img1/. ./img2
copy all files of ./img1 directory to ./img2 directory and replaces older items.
the --link option is for merging directories without moving any data.

pip3 installs inside virtual environment with python3.6 failing due to ssl module not available

(py36venv) vagrant#pvagrant-dev-vm:/vagrant/venvs$ pip3 install pep8
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting pep8 Could not fetch URL
https://pypi.python.org/simple/pep8/: There was a problem confirming
the ssl certificate: Can't connect to HTTPS URL because the SSL module
is not available. - skipping
Could not find a version that satisfies the requirement pep8 (from
versions: ) No matching distribution found for pep8
Background information - Trying to move to python 3.6.
Installed python3.6 using the below commands:
wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz
tar -xvf Python-3.6.0.tgz
cd Python-3.6.0
./configure --enable-optimizations
make -j8 sudo
make altinstall python3.6
Created virtualenv by:
python3.6 -m venv py36venv
source py36venv/bin/activate
Tried to install pep8
(py36venv) pip3 install pep8
pip is configured with locations that require TLS/SSL, however the ssl
module in Python is not available.
Collecting pep8
Could not fetch URL https://pypi.python.org/simple/pep8/: There was a problem
confirming the ssl certificate: Can't connect to HTTPS URL because the
SSL module is not available. - skipping Could not find a version
that satisfies the requirement pep8 (from versions: ) No matching
distribution found for pep8
I followed the below steps for python3.6 installation in ubuntu 14.04 and virtualenv pip installs works fine.
Python 3.6 Installation:
sudo apt-get install python3-dev libffi-dev libssl-dev
wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz
tar xvf Python-3.6.0.tgz
cd Python-3.6.0
./configure --enable-optimizations
make -j8
sudo make altinstall
python3.6
If seeing the following error --
zipimport.ZipImportError: can't decompress data; zlib not available
make: *** [altinstall] Error 1
try:
sudo apt-get install zlib1g-dev
Validation:
Create virtualenv in python3.6:
python3.6 -m venv testenv
source testenv/bin/activate
pip install pep8
using pip:
(testenv) vagrant#pvagrant-dev-vm:~$ pip install pep8
*Collecting pep8
Downloading pep8-1.7.0-py2.py3-none-any.whl (41kB)
100% |████████████████████████████████| 51kB 4.1MB/s
Installing collected packages: pep8
Successfully installed pep8-1.7.0*
(testenv) vagrant#pvagrant-dev-vm:~$ pip list
pep8 (1.7.0)
pip (9.0.1)
setuptools (28.8.0)
I stumbled upon the same issue when I tried to create a virtual environment utilising python3.6.0. Here is my solution for Mac OS X 10.12.2 (Py_minion comment was pretty close):
Setup
I created the environment by the following steps:
downloading python3.6.0
running
./configure --prefix=<some_path>`
make
make install
mkvirtualenv --python=<some_path/bin/python3.6> foo
So basically similar to: https://stackoverflow.com/a/11301911/1286093
An indication if you have the same issue as I had would be a similar line when running make
The necessary bits to build these optional modules were not found: _ssl
Solution
Install openssl
brew install openssl
brew unlink openssl && brew link openssl --force
Change Module/Setup or Module/Setup.dist
You can find those files in the directory of the downloaded Python version.
Comment in and, if necessary change, lines 209 - 211 (I had to change the SSL variable to my openssl location).
SSL=/usr/local/opt/openssl <---- THIS DEPENDS ON YOUR INSTALLATION
_ssl _ssl.c \
-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-L$(SSL)/lib -lssl -lcrypto
Given that this was the location of openssl
Set environment variables
export CFLAGS="-I$(brew --prefix openssl)/include"
export LDFLAGS="-L$(brew --prefix openssl)/lib"
make and install again
Running
./configure --prefix=<some_path>`
make
make install
mkvirtualenv --python=<some_path/bin/python3.6> foo
again did the trick for me
Running make reported to me in the shell output:
The necessary bits to build these optional modules were not found:
_bz2 _dbm _gdbm
_sqlite3 _ssl _tkinter
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
What solved the problem in my case (Linux Mint 18.1, openssl already installed) was editing the setup.py in the Python-3.6.0 folder adding there the path to where the openssl installation put the ssl.h file on my system into ( /usr/include/openssl/ssl.h ). Here the section in which I have added the line '/usr/include':
# Detect SSL support for the socket module (via _ssl)
search_for_ssl_incs_in = [
'/usr/local/ssl/include/',
'/usr/contrib/ssl/include/',
'/usr/include/'
]
ssl_incs = find_file('openssl/ssl.h', inc_dirs,
search_for_ssl_incs_in
)
I have solved this problem on Ubuntu-16.04.1.
First you need to install necessary libraries. To install open Terminal (Ctrl+Alt+T), then type;
sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
After that go the directory where your python file is then reconfigure and reinstall python3.6 .
cd /opt/Python3.6/
./configure
make
sudo make install
NOTE
If you installed Python3.6 via ppa, then reinstall it again;
sudo apt-get install python3.6
Now you should be able to use pip3.6
I ran into the same error when building Python 3.6.1 from source under CentOS 7.
For CentOS7, I had to first:
sudo yum install openssl-dev
Then:
./configure --enable-optimizations
make altinstall
Now pip3.6 works :-)
A complete script can be found HERE
Install Prerequisites
For RHEL/CentOS
sudo yum -y install gcc gcc-c++ zlib zlib-devel libffi-devel openssl-devel wget
For Ubuntu/Debian
sudo apt-get -y install build-essential python-dev python-setuptools python-pip
python-smbus libncursesw5-dev libgdbm-dev libc6-dev zlib1g-dev libsqlite3-dev
tk-dev libssl-dev openssl libffi-dev wget
Download Python
Modify for the version of python you want
Python Versions
cd /var/tmp
sudo wget https://www.python.org/ftp/python/3.x.x/Python-x.x.x.tgz
sudo tar xf Python-3.*.tgz
cd Python-3*
Configure/Make/Install
sudo ./configure --enable-optimizations --enable-shared --prefix=/usr/local
sudo make && make altinstall
Cleanup Shared Library & Add to Path
Stripping the shared library of debugging symbols can speed up execution when running parallel scripts.
sudo make && make altinstall
sudo strip /usr/local/lib/libpython3.7m.so.1.0
sudo echo 'export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib' >> /etc/profile.d/python.sh
sudo echo 'export PATH=${PATH}:~/usr/local/bin/' >> /etc/profile.d/python.sh
sudo echo '/usr/local/lib' >> /etc/ld.so.conf
sudo ldconfig
Reference
Gist
Stack Overflow
Stack Overflow
Daniel Erikson
Unix StackExchange
TLDP

Problems Installing Metasploit Framework on Ubuntu

I have encountered 3 problems while following this guide to installing Metasploit Framework on Ubuntu and Debian:
1) After installing proper version of ruby, there is a command given for installing Ruby libraries: sudo gem install bundler. When I typed that in and hit enter, it says "command not found". After some researching on the internet, I decided to remove sudo and try again. So I've entered gem install bundler and it worked fine. Will this cause any problems?
2) After installing Metasploit Framework, there is a command given to install the required gems and versions:
cd metasploit-framework
bundle install
But when I tried that, I got this error:
rbenv: version '2.1.6' is not installed
I really don't know what to do next (But I moved on).
3) Probably due to the previous problems, in the end when I executed msfconsole command, it said
could not find rake-10.4.2 in any of the sources.
run bundle install to install the missing gems
But the bundle install doesn't help either, because when I run that it says
Could not locate Gemfile or .bundle/directory
Any help would be appreciated.
EDIT: When I asked the person himself who had written the guide, regarding this, he told me to select the version of ruby I had just installed with either rebind or RVM before installing bundler, and then to run the bundle install. Could you please list the commands to do so? I am really new to Linux!
step 1 :
sudo apt-get update
sudo apt-get upgrade
step 2 :
Install dependent packages
sudo apt-get install build-essential libreadline-dev libssl-dev libpq5 libpq-dev libreadline5 libsqlite3-dev libpcap-dev openjdk-7-jre git-core autoconf postgresql pgadmin3 curl zlib1g-dev libxml2-dev libxslt1-dev vncviewer libyaml-dev curl zlib1g-dev
step 3 :
Install Ruby
curl -L https://get.rvm.io | bash -s stable
source ~/.rvm/scripts/rvm
echo "source ~/.rvm/scripts/rvm" >> ~/.bashrc
source ~/.bashrc
rvm install 2.1.6
rvm use 2.1.6 --default
ruby -v
step 4 :
Install and configure postgresql
sudo apt-get install postgresql
sudo -s
su postgres
createuser msf -P -S -R -D
createdb -O msf msf
exit
Step 5 :
Install Metasploit Framework
cd /opt
sudo git clone https://github.com/rapid7/metasploit-framework.git
sudo chown -R `whoami` /opt/metasploit-framework
cd metasploit-framework
cd metasploit-framework
sudo bash -c 'for MSF in $(ls msf*); do ln -s /opt/metasploit-framework/$MSF /usr/local/bin/$MSF;done'
Still if you are facing any issue, you can download the iso or vm image from kali wesbite
I have this problem in kali after install metasploit-framework, and i can't run msfconsole. I solved the problem using the following method:
At change your directory to /usr/share/metasploit-framework:
cd /usr/share/metasploit-framework
and after that, install bundler with gem:
gem install bundler
then:
bundle install
at the end you need to update:
gem update --system
and it will work!
This command fixes it:
apt-get install ruby2.1-dev &&\
apt-get install libsqlite3-dev &&\
bundle install
What it does: install ruby development items

installing ghostscript with vagrant

I would like to install ghostscript when creating my vagrant environment but i have no idea how to achieve this. My provisioning script looks as following:
#!/usr/bin/env bash
sudo apt-get update
sudo apt-get install -y python-software-properties
sudo add-apt-repository -y ppa:ondrej/php5
sudo apt-get update
sudo apt-get install -y wget php5 apache2
# Apache stuff
sudo a2enmod rewrite
rm -rf /var/www
ln -fs /vagrant /var/www
sudo service apache2 restart
Inside your shell provisioning script you must define the following commands
wget http://downloads.ghostscript.com/public/ghostscript-9.10.tar.bz2
tar xjf ghostscript-9.10.tar.bz2
cd ghostscript-9.10
rm -rf zlib && ./configure --prefix=/usr --disable-compile-inits \ --enable-dynamic --with-system-libtiff && make
make so
su root
make install
gs -v
What this basically does is:
download the package
extract the contents
compiles ghostscript
installs ghostscript
checks if it installed correctly
Above explanation has to send you on it's way to install it

RVM- Multiple ruby versions for different projects on same server

I am configuring a new server for multiple rails 2.x/3.x projects, with multiple ruby versions 1.8.7/1.9.2 using RVM.
I want to know how to set a project to use particular version of ruby and another project to use another version of ruby?
Thanks,
Pravin
Assuming your project directories are:
/var/www/project1
/var/www/project2
Let's move to the multiple rails version management:
Run the below commands:
sudo apt-get update
sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties
sudo apt-get install libgdbm-dev libncurses5-dev automake libtool bison libffi-dev
curl -L https://get.rvm.io | bash -s stable
source ~/.rvm/scripts/rvm
echo "source ~/.rvm/scripts/rvm" >> ~/.bashrc
rvm install 1.8.7
rvm use 1.8.7 --default
ruby -v
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs
gem install rails
rails -v
Now change directory to:
$: var/www/project1 rails s
Move to second project:
$: var/www/project2
rvm install 1.9.1
rvm use 1.9.1 --default
gem install rails
and finally start rails:
$: var/www/project2 rails s -p 3003

Resources