web.py deployment on amazon ec2 server using mod_wsgi, and apache - amazon-ec2

For the last three months, I have been development a website using web.py. After successfully building it locally on my machine,
we are trying to deploy it on an amazon ec2 server under their linux ami. However, we have had problems getting it to run,
so we followed your cookbook examples.
We are using Apache 2.2.25 and python 2.6.8 on the Linux AMI, so we are following the Red Hat deployment example (http://webpy.org/cookbook/mod_wsgi-apache).
We had no luck successfully deploying our site, so we took a step back and tried to deploy the simplest program possible (the example listed in the link above).
Once again, it was not working. We have tried following several tutorials, but to no avail.
ex:
http://theadaptiveweb.org/2011/11/04/how-to-setup-mod_wsgi-on-ec2-amazon-ami/
http://www.thefourtheye.in/2013/03/deploying-webpy-application-in-apache.html
We have modified these tutorials to see if we can get it working. these are the steps we have taken:
Setting up the Server
0. Set the Time
sudo rm /etc/localtime
cd /user/share/zoneinfo
ln -s /user/share/zoneinfo/America/Edmonton /etc/localtime
1. Get Pip
wget http://pypi.python.org/packages/source/p/pip/pip-1.1.tar.gz#md5=62a9f08dd5dc69d76734568a6c040508
tar -xvf pip*.gz
cd pip*
sudo python setup.py install
1.1 Or Easy install
(Included in directory)
2. Get Web.py
yum install python-webpy (fedora)
or: easy_install web.py
we added a file call app.py into /var/www/html/app.py
3. Get Apache
sudo yum install httpd httpd-devel
4. Get mod_wsgi
sudo yum install mod_wsgi
4.1: Configure mod_wsgi
sudo vim /etc/httpd/conf/httpd.conf
Add this: *http://webpy.org/cookbook/mod_wsgi-apache*
LoadModule wsgi_module modules/mod_wsgi.so
WSGIScriptAlias /app /var/www/html/app.py
Alias /app/static /var/www/html/static/
AddType text/html .py
<Directory /var/www/html/>
Order deny,allow
Allow from all
Options Indexes FollowSymLinks Multiviews
</Directory>
4.2 Change Permissions
cd /var/run
sudo mkdir wsgi
sudo chown -R ec2-user wsgi
sudo chgrp -R ec2-user wsgi
sudo chmod -R 775 wsgi
cd /var/www
sudo chown -R ec2-user html
sudo chgrp -R ec2-user html
sudo chmod -R 775 html
5. Run Apache
sudo /sbin/service httpd start
5.1 To view error logs
sudo su
cd /var/log/httpd/
tail -n 300 error_log
When we run sudo /sbin/service httpd start, we get Starting httpd:
but there is no
[ OK ] (which usually is displayed if you it is successfully running).
*Note: Options Indexes FollowSymLinks Multiviews was added to prevent this error: Directory Index Forbidden by Options directive: /var/www/html/"
Also note we changed /var/www/html/app.py/ to /var/www/html/app.py
We know apache is working. Going to http://ec2-54-226-136-215.compute-1.amazonaws.com shows the their standard main page.
However, going to http://ec2-54-226-136-215.compute-1.amazonaws.com/app only shows "not found".
Checking the error logs shows no problems. Accessing the website only displays "not found".
We are lost, have no information on how to debug and don't no how to proceed.

Related

Elasticsearch won't start on centos, permission denied

I installed elasticsearch this way :
retrieved the tar.gz on windows, extracted it and put it on a zip
uploaded the zip over ssh on a server and use 'unzip file' to unzip it.
I modified the elasticsearch file so that it has this line
export JAVA_HOME = "/home/xxx/project1/jdk1.8.0_73_linux64"
just below
#!/bin/sh
now when I go into /home/xxx/project1/elasticsearch/bin and type
./elasticsearch
I get
-bash: ./elasticsearch: Permission denied
What could I do to get more information about the problem?
I'm logged as user xxx
Thanks.
The problem is, that you've installed elastic being logged in as root. Now you're surely logged in as not root. If you're working as a user with root privileges, you potentially could start elastic, but it tells you, you can't run in as root. The owner of your elastic folder is root (he installed it). So, all you need to do is to change the owner of a folder to your xxx-user (to let him run elastic). You can easily do it executing a command:
chown -R new_owner path
for example, user is xxx and path is /opt/elasticsearch-2.3.5:
chown -R xxx /opt/elasticsearch-2.3.5
"-R" parameter ensures, that not only folder, but all files inside will recursively change the owner you provide.
Try to install using the package Yum Install for elasticsearch.
Else try: Its sounds like the user permission to access the files.
Ensure that you have downloaded the linux distribution (Optional) Since you have mentioned its tar.gz.
If you have extracted using sudo command then you need to change the user permission for elasticsearch folder to logged in user OR start the elasticsearch using sudo command
sudo ./elasticsearch -d
Check the execution permission if not please do the same by following command
sudo chmod +x /home/xxx/project1/elasticsearch/bin/elasticsearch.sh
Try this things if not please create a chat window and invite me.
Looking ahead I will immediately say that the ES app doesn't work using the sudo
In my case with ES 6.8.1 version on ubuntu i solved the problem using this steps:
Add the current user(or any other) to ES group:
sudo adduser <yourName> elasticsearch
Add possibility to execute ES:
sudo chmod +x /usr/share/elasticsearch/bin/elasticsearch
Then i changed owner of some paths**, in my case these were:
sudo chown -R <yourName>:elasticsearch /etc/default/elasticsearch
sudo chown -R <yourName>:elasticsearch /etc/elasticsearch
sudo chown -R <yourName>:elasticsearch /var/log/elasticsearch
**I started the application several times and then looked where I had no permission
Then i started ./usr/share/elasticsearch/bin/elasticsearch and saw the long-awaited JSON on port 9200 :)
Hope this will help someone.

Composer: Can't write to cache file

I am new to Symfony and Composer.
I recently installed Symfony 3 with the installer (on Linux) just as explained.
Afterwards I also installed Composer.
When I try to run an update with Composer it gives errors like:
post-update-cmd: Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap
PHP Fatal error: Uncaught exception 'RuntimeException' with message 'Failed to write cache
file "/home/dev/var/bootstrap.php.cache".' in
/home/dev/vendor/symfony/symfony/src/Symfony/Component/ClassLoader/ClassCollectionLoader.php:236
and
[RuntimeException]
Unable to write in the cache directory (/home/dev/var/cache/dev)
Script Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache handling the
post-install-cmd event terminated with an exception
[RuntimeException]
An error occurred when executing the "'cache:clear --no-warmup'" command:
[RuntimeException]
Unable to write in the cache directory (/home/dev/var/cache/dev)
Script Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap handling the
post-install-cmd event terminated with an exception
[RuntimeException]
An error occurred when generating the bootstrap file.
I found some issues but they were old and seem to have a specific reason. I just cannot figure out what is going wrong.
I set the owner/group of /var to www-data which is the Apache 2 user. I set the write permissions for the group. Even tried 777 on /var/cache and bootstrap.php.cache.
It won't help.
When I delete the cache-dir it is created normally by Symfony. /var/cache/dev and /prod are also created and loaded with dirs and files.
I can't find bootstrap.php.cache anymore now.
How can I fix this?
If you are running Centos Linux, by default SELinux (/etc/sysconfig/selinux) is set to "enforcing" mode. This will prevent apache user or httpd process from writing to Symfony cache and log paths. Most people take the easy way out by setting to "permissive" mode and rebooting the OS.
The right approach is to leave SELinux in "enforcing" mode and use policycoreutils-python to change some permissioning. These instructions assumes you have used setfacl as per Symfony2 docs.
sudo yum install policycoreutils-python
sudo setsebool -P httpd_can_network_connect 1
sudo setsebool -P httpd_enable_cgi 1
sudo setsebool -P httpd_builtin_scripting 1
sudo setsebool -P httpd_unified 1
sudo setsebool -P httpd_read_user_content true
sudo setsebool allow_httpd_anon_write true
sudo setsebool -P httpd_can_network_connect_db 1
sudo restorecon -rv /var/www

Homebrew postgres broken

I installed Postgresql 9.4.0 installed on my Mac (10.10.1/Yosemite) using homebrew. It does not work.
I have created the softlink to /usr/local/opt/postgresql/homebrew.mxcl.postgresql.plist in ~/Library/LaunchAgents.
If I try to manually load postgres I get the message that the "Operation is in progress"
> launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
/usr/local/Cellar/postgresql/9.4.0/homebrew.mxcl.postgresql.plist: Operation already in progress
However postgres does not appear to be running.
> ps auxw | grep post
billmcn 670 0.0 0.0 2424272 452 s000 R+ 10:12PM 0:00.01 grep post
and I cannot connect with the command line client.
> psql
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
To my knowledge I have tried all the fixes suggested on other Stackoverflow threads discussing this problem. Specifically:
I have uninstalled and reinstalled postgres and the accompanying Ruby gem. There is no postgres 8.0 version on my machine.
I have verified that the psql client program is the 9.4.0 version installed by Homebrew and not a Mac system binary.
I have verified that the /usr/local/var/postgres/postmaster.pid does not exist.
I have rebooted the machine.
I did have Homebrew postgres working on this machine earlier. I think what broke it is upgrading from version 8 to version 9 but I'm not sure.
I don't have any databases I need to preserve. I'm willing to start clean with postgres; I just need to get it to work now. Any ideas?
The issue appears to have been permissions on the /usr/local/var/postgres directory. Here is what my var directory looked like when things weren't working.
ll /usr/local/var/
drwxr-xr-x 3 billmcn admin 102 Dec 20 12:44 cache
drwxr--r-- 2 root admin 68 Dec 29 21:37 postgres
(whoami = "billmcn")
I deleted /usr/local/var/postgres, uninstalled and reinstalled postgres, and now it looks like this.
ll /usr/local/var/
drwxr-xr-x 3 billmcn admin 102 Dec 20 12:44 cache
drwx------ 23 billmcn admin 782 Dec 30 10:51 postgres
Not sure how it got into this state because I don't remember futzing with the permissions on this directory, but no matter. It works now.
I had the same problem installing postgres using homebrew on a freshly installed Yosemite.
First off my brew config looks like this:
HOMEBREW_VERSION: 0.9.5
ORIGIN: https://github.com/Homebrew/homebrew
HEAD: 9f6926265f8e4be7cc80dfe9042f2cd3c1e8dc9e
Last commit: 64 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
CPU: quad-core 64-bit sandybridge
OS X: 10.10.1-x86_64
Xcode: 6.1.1
Clang: 6.0 build 600
X11: N/A
System Ruby: 2.0.0-481
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: ~/.rvm/rubies/ruby-2.1.1/bin/ruby
First thing i noticed was that I had no write permission to /usr/local/var/postgres. This was easily changed issuing sudo chown -R `whoami` /usr/local/var/postgres then I reinstalled postgresql and did
cat /usr/local/var/postgres/server.log
which revealed:
postgres cannot access the server configuration file "/usr/local/var/postgres/postgresql.conf": No such file or directory
So I removed the directory /usr/local/var/postgres and issued the command to initialize the database.
initdb -D /usr/local/var/postgres/
This seemed to have done the trick and postgres is running fine.
I had this same problem. The primary issue here is that the initdb step of installation will create the directory with root ownership instead of as the user on a Mac. To solve this issue:
Create the data directory before running initdb and set permissions of 0700
rm -rf /usr/local/var/postgres # in case this is not your first try
mkdir /usr/local/var/postgres
chmod 0700 /usr/local/var/postgres
Then run initdb and it will respect the permissions of the data directory.
initdb -D /usr/local/var/postgres
For grins and giggles, create a test db named after your user:
createdb `whoami`
Login to test:
psql
After trying to install postgresql with Homebrew, I got this:
Warning: postgresql-9.5.2 already installed, it's just not linked
So I tried:
brew link postgresql
And got this error:
Linking /usr/local/Cellar/postgresql/9.5.2...
Error: Could not symlink share/man/man3/SPI_connect.3
/usr/local/share/man/man3 is not writable.
It seemed to be a write permission matter, so I did:
sudo chown -R `whoami` /usr/local/share/man/
It did the trick because, then I was able to do (without error):
brew link postgresql
In case anyone upgraded from a previous version, dont forget to:
brew postgresql-upgrade-database
That will solve the problem by upgrading your existing databases to the version you upgraded postgres to.
Please note that their is a thread on Homebrew's github dealing with this issue: https://github.com/Homebrew/homebrew/issues/35240
I have had a similar issue. James answer helped me solve it. But I then ran into the issue jbk is mentioning (after having deleted /usr/local/var/postgres, it kept on being recreated).
The issue is that if you have created the symlink:
ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgents
and launched the process:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
you should first unload it:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
before running James's commands.
rm -rf /usr/local/var/postgres # in case this is not your first try
mkdir /usr/local/var/postgres
chmod 0700 /usr/local/var/postgres
In addition, if, like me, you have an admin user managing homebrew and a regular user who will be using pgsl for development purpose, James command should be run as super user:
sudo -s
and ownership over the postgres directory should be given to your dev user:
chown my-dev-user /usr/local/var/postgres
The following command, run as the dev user, should then properly populate the directory:
createdb `whoami`
Running:
psql -l
should show you the tables and user permissions in postgre after such manipulations.
Hope this helps.
I had to delete the .pid file after seeing this in the logs
/usr/local/var/log/postgres.log
2021-10-10 19:05:27.468 BST [41868] FATAL: lock file "postmaster.pid" already exists
2021-10-10 19:05:27.468 BST [41868] HINT: Is another postmaster (PID 820) running in data directory "/usr/local/var/postgres"?
rm /usr/local/var/postgres/postmaster.pid
I installed it using brew
based on #James answer this is what I did on my M1 Monterey machine. For me the directory differed.
DANGER: In the comments it has been pointed out that my script deletes the database.
In terminal:
#to fix postgresql of existing installation
cd /opt/homebrew/var
rm -rf postgres
mkdir postgres
chmod 0700 postgres
initdb -D postgres
#install postgres
echo "installing postgres..."
brew install postgresql
brew services restart postgresql
createuser postgres -s
I then could brew install --cask pgadmin4 and run pgadmin from Applications and connect to 127.0.0.1.
I recently had a problem which began when I upgraded some brew updates / upgrades, mainly python versions etc. What worked for me.
brew uninstall postgres
brew install postgresql#9.5
echo 'export PATH="/usr/local/opt/postgresql#9.5/bin:$PATH"' > ~/.zshrc
# you may need > ~/.bashrc if you use bash
I needed pg_dump, pg_restore etc so to get that working I did
brew install libpq
Start the service
brew services start postgresql#9.5
From here I would have expected everything to work but still all rails db commands were giving error that server was not running. This final bit was the missing piece of the puzzle which finally solved it for me.
gem uninstall pg
gem install pg -v 0.20.0 # which was set in Gemfile
# could also just probably do bundle install instead.
For posterity, I had this issue and wanted to note what worked for me.
I am running postgres 11.2 on High Sierra. I had recently upgraded from postgres 10 with brew postgresql-upgrade-database.
I kept getting the error psql: could not connect to server: No such file or directory, and my server.log indicated is another postmaster (PID 5894) running in data directory "/usr/local/var/postgres"?
I tried several solutions including restarting my computer, deleting postmaster.pid, using brew services restart postgres, but to no avail. I eventually stumbled on the solution:
brew unlink postgresql && brew link postgresql
No idea why this worked, but putting it here mostly so I can reference it myself in the future! Throw stuff at the wall till it sticks!
Check #leo_chaz_maltrait for fixing errors the error Could not symlink share/man/man3/SPI_connect.3
Another error that might show up is:
Error: Could not symlink lib/pkgconfig/libecpg.pc
sudo chown -R `whoami` /usr/local/lib/pkgconfig
brew link postgresql
Please read and follow the instructions.
Check postgres logs to see what the issue is.
tail -f /usr/local/var/log/postgres.log
tail -f /opt/homebrew/var/log/postgres.log
tail -f /usr/local/var/postgres/server.log
In my case it this was the error.
2022-07-19 21:16:12.095 IST [2138] FATAL: data directory "/usr/local/var/postgres" has invalid permissions
[3472] FATAL: lock file "postmaster.pid" already exists
Added the required permission and issue got fixed.
sudo chown -R vikas /usr/local/var/postgres
rm /usr/local/var/postgres/postmaster.pid
That's it.
I'd this issue after shutting down the computer due power outage.
# This initialize your database with the current data and settings
initdb -D postgres
# This will start database service
pg_ctl -D postgres -l logfile start

Install GitLab on Subdomain

I am trying to install GitLab on a subdomain. I am not very familiar with web servers and stuff, only very little knowledge. I am currently connected to the subdomain via ssh admin#mysite.com on the Mac's Terminal. Then I ls to the subdomain's folder. Question is if I execute:
curl -O https://downloads-packages.s3.amazonaws.com/centos-6.5/gitlab-7.4.3_omnibus.5.1.0.ci-1.el6.x86_64.rpm
sudo yum install openssh-server
sudo yum install postfix
sudo yum install cronie
sudo service postfix start
sudo chkconfig postfix on
sudo rpm -i gitlab-7.4.3_omnibus.5.1.0.ci-1.el6.x86_64.rpm
on the current folder. Will GitLab be installed or downloaded only in the current folder or will other subdomains and folders of the site be affected?
Is there a way to test yum install first on the current directory?
The Omnibus/RPM version of Gitlab will install gitlab to the /opt/gitlab directory.
If you want to check the files that will be installed by an RPM package you can do so with:
rpm -qlp gitlab-7.4.3_omnibus.5.1.0.ci-1.el6.x86_64.rpm
After you install the RPM with "rpm -ivh gitlab*.rpm" you configure the URL/subdomain using the:
/etc/gitlab/gitlab.rb
file.
After amending the gitlab.rb file you can start gitlab with:
gitlab-ctl start
Or restart it with:
gitlab-ctl restart
You can configure the web address & port that gitlab/nginx listens on using the external_url parameter in the gitlab.rb file:
external_url "https://gitlab.mydomain.com"
If you already have a webserver (such as Apache) using port 80 or port 443 then you might want to ask gitlab/nginx to listen on a port other than 80 or 443 with:
external_url "https://gitlab.mydomain.com:8443"
You can check if anything is listening on particular ports using:
netstat -luntap | grep LISTEN

Clamav installation steps for linux centos

I have tried to install clamav in linux centos, but not able to succeed, so anybody provide me the procedure to install clamav antivirus and test the same.
Thanks in advance
First thing to do is install the program sudo apt-get install clamav
It´s necessary install the deamon program as well sudo apt-get install clamav-daemon
Once that we have the program properly installed, will be necessary configure to make it works with our app. /etc/clamav/clamd.conf
TCPAddr 127.0.0.1
TCPSocket 3310
To configure by console the antivirus just type sudo dpkg-reconfigure clamav-freshclam
Restart the service sudo /etc/init.d/clamv-* restart
For CentOS 6
Install ClamAV via yum epel repo:
yum install clamd
Edit config file:
vim /etc/clamd.conf
ExtendedDetectionInfo yes
ExcludePath /usr/local/maldetect/
DetectPUA yes
Set to start on boot:
chkconfig clamd on
Update ClamAV prior to a scan:
freshclam
Start ClamAV:
service clamd start
Add the cron for daily automatic scans:
vim /etc/cron.daily/daily-clamscan
#!/bin/bash
SCAN_DIR="/"
LOG_FILE="/var/log/clamav/daily-clamscan.log"
/usr/bin/clamscan -i -r $SCAN_DIR >> $LOG_FILE
Set proper cron permissions:
sudo chmod +x /etc/cron.daily/daily-clamscan
Update the logrotate file to look for all logs via wildcard
vim /etc/logrotate.d/daily-clamscan
/var/log/clamav/daily-clamscan.log {
missingok
notifempty
create 644 clam clam
}
Thats how I installed it on my machine. Here are a few good quick commands or you can use the cron to manually run it. I also installed Linux Malware Detect (LMD) with it so I can do more for my scans.
Full System Scan that only displays infected files and runs in the background with bell
Use jobs command to view status
clamscan -r --bell -i / &

Resources