Django settings by environment

One thing not explained in Django Doc is how to make your project to live accross environments. Some settings need to have different values when used in production or testing, and testers might want to set their own values. A clear example is I wanna use a MySQL in production and SQLite in testing.

What is environment

In this post I'll name environment where a project instance is installed and configured. It also define third services as frontend, static file server, etc. In my previous cases I saw the following environment:

• Production: Real application runtime accessible by everyone
• Preproduction or staging: Installation similar to production, for test in real environment, only for testers
• Testing or developement:Used by dev for their job
• Test: Used by CI or unittest

Here I'll only use production and testing.

Adapt a Django project

I considere my project as a Python package, it must be installed in your PYTHONPATH and importable with import myblog. See here the setup.py file used for packaging.

Like many projects, it needs to be launched with a configuration which matches with the different environment described on top. And otherwise I need to be able to change quickly some part of my settings.

The best way I found are:

1. Shell vars

Unix shell includes environment variable which can be retrieve in Python as following:

$ export MYVAR=foo
$ python
>>> import os; os.environ['MY_VAR']
'foo'

The idea is to use hold variables which begin by BLOG_ and use it in Django settings. This quick example, this will let me make an atomic setting of STATIC_URL to '/foo/':

BLOG_STATIC_URL='/foo/' ./manage.py runserver

2. Configuration file

A basic Python configuration file is used to set settings in hard way. By default it is /etc/myblog.cfg, but devs will be able to change it with Unix variable $BLOG_CONFIG_FILE. Below, an example of myblog.cfg:

[DEFAULT]
env = testing
debug = True
server_url = myblog.net
static_url = /static/

Split settings.py in several files

All parameters are defined by the methods above, but some settings are inherent to some environment, so I choose to split settings.py, make one file per idea:

• common.py: Parameter for all environment
• env.py: Configuration and environment variable parser
• __init__.py: Real settings module which contains the the configuration variable
• prod.py | testing.py: Parameters by environment

Let's do this:

cd myblog # Go near manage.py
mkdir settings
touch settings/__init__.py
mv settings.py settings/common.py

All the needed modifications are described below but it is not a full description, there are only files' samples and explanation.

env.py

This file contains a SafeConfigParser for read the desired configuration file, it also jails BLOG_* variables into the ones used in config file. Create the file settings/env.py with a content like below:

import os
from ConfigParser import SafeConfigParser
# Get env vars
ENV_VARS = {
    k.replace('BLOG_', '').lower(): v
    for k, v in os.environ.items()
    if k.startswith('BLOG_')
}
# Default config values
DEFAULT_CONFIG = {
    'env': 'prod', # String
    'debug': 'False', # Boolean
    'static_root': '/static/',
    'internal_ips': '', # List
    ...
}
CONFIG_FILE = os.environ.get('BLOG_CONFIG_FILE', '/etc/myblog.cfg')
# Read config
CONFIG = SafeConfigParser(defaults=DEFAULT_CONFIG, allow_no_value=True)
CONFIG.read(CONFIG_FILE)
# Overide config by env vars
map(lambda i: CONFIG.set('DEFAULT', i[0], i[1]),
    ENV_VARS.items())
ENV = CONFIG.get('DEFAULT', 'env') # Shortcut for this value

testing.py

Testing imports common settings and add configuration, for example configure Django Debug toolbar. Here's my settings/testing.py:

import os
from .common import * # Get common settings
from .env import CONFIG # Get configParser

DEBUG = CONFIG.getboolean('DEFAULT', 'debug') # Get bool from str
TEMPLATE_DEBUG = True
SECRET_KEY = '&qaeg(m5s0rpdj-wx@azdzadb)v@@n$if67ba-4e9&cplq+j$$c+'

# Debug toolbar
try:
    __import__('imp').find_module('debug_toolbar')
    INSTALLED_APPS += ('debug_toolbar',)
    MIDDLEWARE_CLASSES = ('debug_toolbar.middleware.DebugToolbarMiddleware',) + MIDDLEWARE_CLASSES
    # Get list from str
    INTERNAL_IPS = CONFIG.get('DEFAULT', 'internal_ips').split(',') # list from str
except ImportError:
    pass

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
} 

common.py

It gathers all common settings and imports some of them from env.CONFIG. Like default settings.py, it contains static definition of INSTALLED_APPS, MIDDLEWARE_CLASSES or TEMPLATE_CONTEXT_PROCESSORS.

import os
from .env import CONFIG # Get ConfigParser

ALLOWED_HOSTS = CONFIG.get('DEFAULT', 'allowed_hosts').split(',') # List from str
SERVER_URL = CONFIG.get('DEFAULT', 'server_url')
SITE_ID = CONFIG.getint('DEFAULT', 'site_id')
SECRET_KEY = CONFIG.get('DEFAULT', 'secret_key')
ROOT_URLCONF = 'urls'
WSGI_APPLICATION = 'wsgi'
INSTALLED_APPS = (
    ...
)

MIDDLEWARE_CLASSES = (
    ...
)

TEMPLATE_CONTEXT_PROCESSORS = (
    ...
)

TEMPLATES = [
    ...
]

 I changed ROOT_URLCONF and WSGI_APPLICATION to point to modules at root of my project.

__init__.py

settings/__init__.py is simple file which imports objects related to the environment. In facts, it is launched by Django and must contain all your project settings (by importing).

from __future__ import absolute_import
import sys
from .env import ENV

if ENV == 'prod':
    from .prod import *
elif ENV == 'testing':
    from .testing import *
else:
    from django.core.exceptions import ImproperlyConfigured
    raise ImproperlyConfigured('Choose a great environment to launch.')

manage.py and wsgi.py

This files contain the path of your settings module. Like I design it, I must set this path to the folder 'settings/' (which uses 'settings/__init__.py'). Django uses an environment variable DJANGO_SETTINGS_MODULE for define settings path, by default it is 'myblog.settings', and I changed it to 'settings'.

In both files, DJANGO_SETTINGS_MODULE is reached, generaly not found and set to the default value. Change only the following line on files:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

Done

Your project directory should look like something like below:

$ tree .
├── manage.py
├── settings
│   ├── __init__.py
│   ├── common.py
│   ├── env.py
│   └── testing.py
├── urls.py
└── wsgi.py

References & links

• stackoverflow: Django Local Settings