My fab file

There aren't enough examples online of Fabric scripts as used in the real world. The documentation is good but a real example is always better.

Maybe that is because Fabric scripts tend to be closely tied to the environment they are working on. To be able to post mine, I have had to change some of the details, too. There are also constraints that I deal with that most people won't have, like I am not allowed to ssh directly as the service account the program runs under. There are a few additional steps needed to make sure the permissions are set properly in the script below.

I save this as fabfile.py in my project directory. The first step is to set the host names for the deployment. I have a method setup to handle that by manually hardcoding the server names. In my environment, those rarely ever change. Others might get a list of active servers from a cloud host, for example.

The deploy method is the main entry point, but I have broken each deployment step down into a function so they can be called individually if needed. If I needed to restart memcached, for example, I can run fab setup:prod restart_memcached instead of logging into the server manually.

This script handles the deployment for a Django web project. It works by creating a new virtualenv for each deployment, pulling the source from git, installing all the pip requirements (which are hosted locally on a private Chishop server), and finally restarting uwsgi.

This scheme works great:

  • Easy rollback, just replace the symlink
  • You always know that checking out the source results in a working project
  • New project requirements are automatically applied, but don't interfere with rolling back
  • I love getting an email containing the commits between versions
  • Easily handle adding new servers, since the Fabric script handles setting up the project

Still, I have no idea how people have been using Fabric. Everybody says they are using Fabric but there are few examples online. For all I know, I could be doing it wrong. Please post your own fabfile.py so we can all learn or leave a comment.

from __future__ import with_statement

from datetime import datetime
import smtplib
from email.mime.text import MIMEText
import os
import pwd

from fabric.api import run, sudo, cd, env

GIT_URL = "ssh://mygitserver/project"
DEPLOYED = '/opt/project/deployed/{}'
DEPLOYED_DIR = '/opt/project/deployed'

def setup(name):
    if name == 'uat':
        env.hosts = fab_hosts = someUATservers
    elif name == 'prod':
        env.hosts = fab_hosts = somePRODservers
    else:
        raise ValueError('Invalid name')
    env.envname = name.upper()
    env.user = pwd.getpwuid(os.getuid())[0]
    env.deploy = DEPLOYED.format(datetime.today().isoformat().replace(':', '_'))
    env.project = '{}/project'.format(env.deploy)

def current_env():
    """
    Get the path to the virtualenv currently in use
    """
    current = DEPLOYED.format('current')
    link = run('readlink -f {}'.format(current))
    print 'current env', env.deploy
    return link

def symlink_current():
    current = DEPLOYED.format('current')
    run('rm -f {}'.format(current))
    run('ln -s {} {}'.format(env.deploy, current))

def create_virtualenv():
    run('virtualenv -p /opt/apps/local/bin/python ' \
        '--prompt=project{envname} ' \
        '--no-site-packages --distribute {deploy}'.format(**env))

def cleanup_old_deployed():
    current = os.path.basename(current_env())
    old_entries = set(run('ls -1 {}'.format(DEPLOYED_DIR)).splitlines()) - {'current', current}
    if old_entries:
        with cd(DEPLOYED_DIR):
            sudo('rm -rf {}'.format(' '.join(old_entries)))

def virtualenv(command):
    with cd(env.project):
        run('source {deploy}/bin/activate && {command}'.format(
            command=command,
            deploy=env.deploy))

def clone_source():
    run('git clone {} {project}'.format(GIT_URL, **env))
    run('cp /opt/project/site/local_settings.py {project}'.format(**env))

def install_requirements():
    run('{deploy}/bin/pip install -r {project}/requirements/core.txt'.format(**env))

def syncdb():
    virtualenv('python manage.py syncdb --migrate')

def django_tests():
    virtualenv('python manage.py test')

def chown_virtualenv():
    sudo('chown -R projectaccount.project {deploy}'.format(**env))

def restart_uwsgi():
    sudo('kill -HUP `cat /opt/project/conf/project.pid`')

def restart_memcached():
    # flush memcached or start it
    run('echo "flush_all" | nc localhost 11211 || memcached -d')

def get_git_changes():
    # first find the current hash for the latest commit in the current production env
    current = current_env()
    with cd('{}/project'.format(current)):
        lastcommit = run("git show-ref --heads | awk '{print $1}'")
    if not lastcommit:
        return ''
    with cd(env.project):
        diff = run('git log --no-merges {}..HEAD | head -600'.format(lastcommit))
    return diff

def deploy():
    create_virtualenv()
    clone_source()
    changes = get_git_changes()
    install_requirements()
    syncdb()

    symlink_current()
    chown_virtualenv()

    restart_uwsgi()
    restart_memcached()

    if changes:
        message = MIMEText(str(changes))
        message['Subject'] = 'Deployed project {envname}'.format(**env)
        message['From'] = me              # configure
        message['To'] = ', '.join(to)     # configure

        smtp = smtplib.SMTP('smtprelay')
        smtp.sendmail(me, to, message.as_string())
        smtp.quit()

Discussions:

blog comments powered by Disqus