Deploying Django with Salt Stack

  • Mon 11 March 2013

    • What is Salt Stack?

    Some light reading (go read these and come back):

    I'm gonna ease into this, I don't want to throw a bunch of code snippets at you, and make you feel like you're drowning.

    Salt can be ran in a master/minion (server/client) environment, or a masterless (client-only) type environment. What I will show you here can be used in either environment.

    I will be using Ubuntu 12.04 32-bit.

    What's Needed?

    Let's think about what's absolutely necessary to install Django and start using it?

    • Python & dependancies
    • virtualenv (because virtualenvs are awesome)
    • install latest version of Django via Pip
    • configure Django to use local Sqlite DB

    That's it! Obviously it gets a lot more complex if you're doing anything more than simple development or the Django tutorial, but that's for another post.

    So let's get started!

    A note I'm skipping how to install Salt. That's covered thoroughly in Salt's own documentation and Linux Journal's article.

    I'm creating a top.sls file in /srv/salt directory that will outline which hosts will get which states.

    base:
    '*':
        - requirements
        - django

    The above example is applying a requirements and django to all (*) hosts. I will put any global requirements that are not specific to any application in requirements and I will put Django specific requirements in django.

    Above we mentioned that we needed Python and it's dependancies, often python-dev is sufficient. The next thing we need is virtualenv. Ubuntu's package name for virtualenv is python-virtualenv so I create an init.sls file in the requirements directory, I tell it I need the python-virtualenv and python-dev packages installed.

    packages:
    pkg.installed:
        - names:
            - pthon-dev
            - python-virtualenv

    That's it! When Salt runs that, it will install the packages python-virtualenv and python-dev.

    Don't believe me?

    Neat right?!

    The important part to take away from this is the following:

    [default] local:
----------
    State: - pkg
    Name:      python-dev
    Function:  installed
        Result:    True
        Comment:   The following package(s) were installed/updated: python-dev.
        Changes:   libpython2.7: {'new': '2.7.3-0ubuntu3.1', 'old': ''}
                   python2.7: {'new': '2.7.3-0ubuntu3.1', 'old': '2.7.3-0ubuntu3'}
                   python2.7-minimal: {'new': '2.7.3-0ubuntu3.1', 'old': '2.7.3-0ubuntu3'}
                   python2.7-dev: {'new': '2.7.3-0ubuntu3.1', 'old': ''}
                   libexpat1-dev: {'new': '2.0.1-7.2ubuntu1.1', 'old': ''}
                   libexpat-dev: {'new': '1', 'old': ''}
                   libexpat1: {'new': '2.0.1-7.2ubuntu1.1', 'old': '2.0.1-7.2ubuntu1'}
                   python-dev: {'new': '2.7.3-0ubuntu2', 'old': ''}

----------
    State: - pkg
    Name:      python-virtualenv
    Function:  installed
        Result:    True
        Comment:   The following package(s) were installed/updated: python-virtualenv.
        Changes:   python-pip: {'new': '1.0-1build1', 'old': ''}
                   python-virtualenv: {'new': '1.7.1.2-1', 'old': ''}
                   python-setuptools: {'new': '0.6.24-1ubuntu1', 'old': ''}
                   python-distribute: {'new': '1', 'old': ''}

    It's basically saying the packages python-dev and python-virtualenv were installed

    (Result: True)

    and their dependancies were also installed:

    Comment:   The following package(s) were installed/updated: python-dev.
        Changes:   libpython2.7: {'new': '2.7.3-0ubuntu3.1', 'old': ''}
                   python2.7: {'new': '2.7.3-0ubuntu3.1', 'old': '2.7.3-0ubuntu3'}
                   python2.7-minimal: {'new': '2.7.3-0ubuntu3.1', 'old': '2.7.3-0ubuntu3'}
                   python2.7-dev: {'new': '2.7.3-0ubuntu3.1', 'old': ''}
                   libexpat1-dev: {'new': '2.0.1-7.2ubuntu1.1', 'old': ''}
                   libexpat-dev: {'new': '1', 'old': ''}
                   libexpat1: {'new': '2.0.1-7.2ubuntu1.1', 'old': '2.0.1-7.2ubuntu1'}
                   python-dev: {'new': '2.7.3-0ubuntu2', 'old': ''}

Comment:   The following package(s) were installed/updated: python-virtualenv.
        Changes:   python-pip: {'new': '1.0-1build1', 'old': ''}
                   python-virtualenv: {'new': '1.7.1.2-1', 'old': ''}
                   python-setuptools: {'new': '0.6.24-1ubuntu1', 'old': ''}
                   python-distribute: {'new': '1', 'old': ''}

    OK...I know, that's fine, but still no Django!

    Let's install Django!

    In my Django directory I'm going to create an init.sls file that will contain the following:

    include:
- requirements

/home/vagrant/learning-salt/venv:
    virtualenv.managed:
        - no_site_packages: True
        - runas: vagrant
        - requirements: salt://django/requirements.txt
        - require:
            - pkg: python-virtualenv

    The Pieces:

    include:
    - requirements

    This is necessary for us, because we need a package from requirements/init.sls as you'll see below:

    /home/vagrant/learning-salt/venv:
virtualenv.managed:
    - no_site_packages: True
    - runas: vagrant
    - requirements: salt://django/requirements.txt
    - require:
        - pkg: python-virtualenv

    What this is saying is

    • we want to create a virtualenv in /home/vagrant/learning-salt called venv
    • we don't want site-packages
    • run as the vagrant user (by default, this would run as root -- could cause nasty permission problems for you later)
    • we've defined a requirements.txt file
    • we can't create a virtual env, without the python-virtualenv package, so don't run this state until python-virtualenv is installed. Salt's docs on requisites

    My requirements.txt file simply looks like:

    Django==1.5

    My requirements.txt file simply says "Hey! Install Django version 1.5".

    See!!

    The important parts to take away:

    [default] local:

----------
    State: - virtualenv
    Name:      /home/vagrant/learning-salt/venv
    Function:  managed
        Result:    True
        Comment:   Created new virtualenv
        Changes:   new: Python 2.7.3
                   packages: {'new': ['Django==1.5'], 'old': ''}

    What that's saying:

    • Created new virtualenv venv in /home/vagrant/learning-salt
    • Django==1.5 was installed

    A little review. From the list above:

    • Python & dependancies DONE
    • virtualenv (because virtualenvs are awesome) DONE
    • install latest version of Django via Pip DONE
    • configure Django to use local Sqlite DB

    Now you're saying...Fine...Django is installed...but there is no project, I can't use it. What good does this do me?

    So let's create a project!

    I'm going to work smarter, not harder here, and leverage the power of Django. I'm going to bootstrap Django by defining a project from a git source, rather than using an empty django-project. While this might seem like cheating, it is not. I am using Two Scoops of Django's project layout, available here

    The code:

    I'm appending this to django/init.sls.

    create Django project:
    cmd.run:
        - user: vagrant
        - name: ". venv/bin/activate && django-admin.py startproject
      --template=https://github.com/twoscoops/django-twoscoops-project/zipball/master
        --extension=py,rst,html icecream && pip install -Ur
        icecream/requirements/local.txt && python icecream/icecream/manage.py
            syncdb --noinput"
        - cwd: /home/vagrant/learning-salt/
        - require:
            - virtualenv: /home/vagrant/learning-salt/venv
            - pkg: python-dev

    The important pieces:

    • We're again running as the vagrant user, rather than root with - user: vagrant
    • We're requiring the virtualenv venv exist in /home/vagrant/learning-salt before we run this command
    • We're also requiring the package python-dev before we run this command, as this system package provides necessary C libarires for some of the Django packages we're installing.

    The actual command (and it's "magic"):

    ". venv/bin/activate && django-admin.py startproject
      --template=https://github.com/twoscoops/django-twoscoops-project/zipball/master
        --extension=py,rst,html icecream && pip install -Ur
        icecream/requirements/local.txt && python icecream/icecream/manage.py
            syncdb --noinput"

    We're simply activating our virtualenv, running django-admin.py startproject, pointing it to a git repo, and after we've created the project, we're pip installing the packages as defined by this project with pip install -Ur icecream/requirements/local.txt. Then finally, we're running our initial syncdb.

    The important part to take away from this is the following:

    Successfully installed bpython django-braces django-model-utils logutils
South coverage django-discover-runner django-debug-toolbar Sphinx pygments
Jinja2 docutils

    It simply reports that via Salt Stack, we installed the requirements defined in our icecream/requirements/local.txt file. I should also note, previously I copied and pasted the output of Salt, the above example is essentially the stdout from the command, in the end, the only thing I care about.

    Questions you may have

    • How do I use this? You'll want to activate your virtualenv. I'm using Salty Vagrant and forwarding my local port 8000 to remote port 8000. Once you've activated your virtualenv, you'll want to run:
    python manage.py runserver 0.0.0.0:8000

    The 0.0.0.0 says to listen on all adapters. The 8000, is the default port.

    Then open your favorite browser and browse to 'localhost:8000' and you should see Django project's index.html

    • Vagrant? What's that? Vagrant is awesome!
    • Salt Stack + Vagrant = Salty Vagrant == Even MORE awesome!
    • Wondering how to install Postgresql for a production database server?
    • Wondering how to create a database and user in Postgresql for Django?
    • Wondering how to serve static files from Nginx?
    • Wonder how to serve Django behind uWSGI?
    • Want Memcache?

    I created a Github repo: https://github.com/esacteksab/learning-salt this will allow you to see the source in it's entirety. You can also git-clone it, and use Salty Vagrant to play with this further.

    More posts will (hopefully) come outlining how to do all the above (and maybe more!).

    Part II: http://www.barrymorrison.com/2013/Apr/21/deploying-django-with-salt-now-with-postgresql/

    Any questions/comments/advice/suggestions/etc. please feel free to comment below! Thanks!

    Comments !

    social