Wednesday, May 02, 2012

Adventures in Apache/Python Configuration: Getting Python running in an Apache Shared Hosting environment

Recently I have been doing some development with Python/Django. I have been working with Alan Viars of Videntity and working with his Restcat open source activity tracker.

One thing I wanted to do was to get my own copy of Restcat running on my bluehost account. I have used Bluehost for many years. It has proven to be a reliable shared hosting environment for me offering good value for money.

One of the challenges with getting Python running with Apache is that in a shared hosting environment you can't necessarily get at the Apache Configuration file or the virtual server configuration file. This leaves you with a more limited subset of options that can be applied using the .htaccess file.

After a lot of experimentation I finally succeeded in getting Python running and I thought I would document the setup here. I am doing this for two reasons:

1. Other may find this useful in tackling a similar challenge.
2. I do not profess to be an expert in Apache configuration so I am sure that other experts can point out improvements to the configuration I ended up with.

First, let's start with an outline of the configuration.

The web documents used by apache are stored in {User Account}/public_html

I didn't want my Python files in the web document folders, so I created a Python Apps folder. I placed this in {User Account}/pyapps

I followed the instructions from Bluehost to install Python in my user account. I also used VirtualEnv to be able to isolate the setup for individual python apps. This resulted in python being installed in {User Account}/python. Python 2.7.2 is the version that is installed.

My virtualenv configuration placed python code in {User Account}/my_python_env1/ 

RESTCat was installed in to {User Account}/pyapps/RESTCat

Bluehost offers SSH Access so you can connect securely to your account and check that your python application works.

The next step is to get the python application to be run when you access via the Apache web server. This was the fun part!

To get this working I brought a number of elements together. 

I used cPanel to setup a subdomain. in this case http://restcat.ekive.com

In the Apache web documents directory {User Account}/public_html I created an apps directory. ie. {User Account}/public_html/apps

My plan is to create an application folder in side this apps directory for each Python Application I want to install. So for Restcat we have:
{User Account}/public_html/apps/restcat

I configured the subdomain redirection to point to a Document Root of {Home}/public_html/apps/restcat

The next step was to mess with .htaccess and use the FCGI Daemon that is configured in Apache. 

I setup .htaccess in {User Account}/public_html/apps/restcat

ie. {User Account}/public_html/apps/restcat/.htaccess

In the restcat directory I also placed a symlink to the location where the static files for the application are stored.

ln -s {User Account}/public_html/static/restcat/mainstatic/   static

This file is used to control how Apache handles content.

AddHandler fcgid-script .fcgi
Options +SymLinksIfOwnerMatch

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]

ErrorDocument 500 '<h2>(Restcat) Application Error!</h2>Application Failed to load correctly'

I think the .htaccess file still needs optimizing in order to point to the location of the Static files and admin media files before getting to the python/fcgi stage but this gets the app going. 

What is in .htaccess?

AddHandler tells Apache to use the FCGI daemon when it encounters a file with a .fcgi extension.

I will leave experts in Apache configuration to explain most of this file but the last two lines are important.

The ErrorDocument 500 line will write something to the browser if the .htaccess file is executed but the application doesn't run (in this case the mysite.fcgi file)

The RewriteRule basically passes anything that hasn't been executed up to this point and passes it to the mysite.fcgi file. It is the mysite.fcgi file that will execute the python application. So let's look at that next. 

Getting this file working was troublesome because you can get different errors between Apache executing the file and running the file directly from the SSH command line. In the end this mostly seems to come down to getting all the necessary components added to the PATH before Apache tries to execute them. Once again I am sure there are Python and Shell experts out there that can tell me far more efficient ways to accomplish this, but at least this worked!

#!/usr/bin/python
import sys, os

sys.path.insert(0, "{USER ACCOUNT}/python")
sys.path.insert(0, "{USER ACCOUNT}/my_python_env1/lib/python2.7/site-packages")
sys.path.insert(0, "{USER ACCOUNT}/my_python_env1/lib/python2.7/site-packages/flup-1.0.2-py2.7.egg")
sys.path.insert(0, "{USER ACCOUNT}/my_python_env1")
sys.path.insert(0, "{USER ACCOUNT}/pyapps/python-omhe")

base = os.path.dirname(os.path.abspath(__file__))
sys.path.append(base)
sys.path.append(base + '/..')

sys.path.append('{USER ACCOUNT}/pyapps')
sys.path.insert(0,"{USER ACCOUNT}/pyapps/RESTCat")

os.environ['DJANGO_SETTINGS_MODULE'] = 'RESTCat.settings'

# switching to FCGI
from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

So #!/usr/bin/python is there to execute in python

We then import the os and sys components. 

Next we add a series of directories to the PATH so that this routine can find all the components it needs. For me this involved pointing to the python installation, site-packages - so django and other components could be found, the flup egg file (Bluehost recommends flup as a fcgi to wsgi interface and who am I to argue). I also needed to point to the python-omhe installation which is needed by RESTCat.

I also added the location of my Python apps and RESTCat specifically to the PATH.

I then pointed to the RESTCat settings file.

The final step was to load the fastCGI interface from the django libraries and execute. 

When you put all these pieces together you can point at a sub-domain that is hosted on an Apache Shared Host have it execute a python application.

If I want to add another Python Application I can create an additional folder in the {USER ACCOUNT}/public_html/apps directory and tweak the mysite.fcgi file in that folder to point to a different python application. 

I welcome any suggestions to optimize this setup, in particular to see if it is possible to write an Apache rule in .htaccess to point to static files in a directory located elsewhere in the Apache document root, rather than being a sub-directory of the apps/{python app} folder in the web document area.

In other words:

Have http://restcat.ekive.com run from 

ekive.com/apps/restcat and use static files in

The same also goes for the Django static files used in the admin library.

This configuration successfully runs python from Apache wthout placing the python code in the Apache web document folder structure.

Posted via email from ekivemark: pre-blogspot