Server side MVC sucks. Server side template rendering is obsolete.

And I’m going to actually contradict myself on this!

But, several hours earlier…

The Beginning

To develop a Django application for the Heroku platform, beginners should follow this excellent guide: django-skel . I’m using that exact pattern, learned at the very same place, it’s simple, easy and works wonderfully. Once you get into the habit of deployment the Heroku way, you kinda like the automation, all the tools at your service doing all the job, and they all work out-of-the-box. Concentrating on the frontend code, django-storages allows us to easily deploy all the static content to Amazon S3 using the built-in collectstatic command, and there’s django-compressor which minifies both our JS and CSS files.

Sounds like an ideal solution:

  1. write the templates
  2. use the appropriate template tags to mark the compressable static content
  3. deploy to S3.

And this may function up to the point where we realize that the application grew so large, that it’s no longer a server-side driven webapp, enhanced with JavaScript, but a complex, dynamic, interactive JavaScript application, attached to a web service, with a huge JavaScript codebase. So, in order to produce healthy, easily maintainable code, jQuery by itself is not an option anymore. Putting all the code in a $(document).ready scope would result in huge JS files, unorganized and ugly code. Frameworks like Backbone.js, Knockout.js (the chosen one for this project), Ember.js, etc. are essential now, but still, they won’t solve code modularization by themselves, and for that task, RequireJS comes to the rescue. Unfortunately, that choice will make django-compressor almost useless, but luckily, RequireJS got it’s own tools for that job.

Rethinking the problem, an ideal solution would be to develop the frontend and backend code totally separately, implementing the backend as a REST service, and do the frontend with whatever fancy technology we wish to use. Pure server-side MVC makes the backend and frontend code too tightly coupled, which makes it harder to switch the frontend at a later time to something new if needed. Unfortunately, the Django world will certainly put us sometimes in a situation where the apps we wish to use are written in MVC fashion. That works against the ideal solution we want to achieve, so let’s grab the best of the two worlds. Those who don’t like this may write most of the stuff from scratch, but I’ve got deadlines.

Getting the skeleton project

Clone the sample application from this repo, and you’ll see this structure:

.
|-- build.js
|-- djmodjs
|   |-- apps
|   |   |-- __init__.py
|   |   +-- treasuremap
|   |       |-- admin.py
|   |       |-- __init__.py
|   |       |-- models.py
|   |       |-- urls.py
|   |       +-- views.py
|   |-- assets
|   |   |-- common.js
|   |   |-- shared
|   |   |   |-- css
|   |   |   |   +-- main.css
|   |   |   +-- js
|   |   |       |-- handlers
|   |   |       |   +-- getapiurl.js
|   |   |       +-- utils.js
|   |   +-- treasuremap
|   |       +-- js
|   |           |-- islands.js
|   |           |-- map.js
|   |           +-- models
|   |               |-- island.js
|   |               +-- treasure.js
|   |-- __init__.py
|   |-- libs
|   |   +-- __init__.py
|   |-- settings
|   |   |-- common.py
|   |   |-- dev.py
|   |   |-- __init__.py
|   |   +-- prod.py
|   |-- templates
|   |   |-- 404.html
|   |   |-- 500.html
|   |   |-- base.html
|   |   +-- treasuremap
|   |       |-- islands.html
|   |       +-- map.html
|   +-- urls.py
|-- fabfile.py
|-- gunicorn.py.ini
|-- manage.py
|-- Procfile
|-- reqs
|   |-- common.txt
|   |-- dev.txt
|   +-- prod.txt
|-- requirements.txt
+-- wsgi.py

Following regular Django conventions, the apps folder contains our application(s), but only the Python modules. All templates and static files are put into the templates and assets folders respectively, rather than to remain under the application folder they belong to. This allows us to maintain the client-side code much easier, and eventually switch the whole thing.

Django templates (The Best Of) feat. Knockout / RequireJS

We’ve kept using Django’s template inheritance, because it’s simply awesome. By peeking into the djmodjs/templates/base.html file, you’ll surely see this:

{% block head_javascript %}
<script type="text/javascript">
    var require = {
        baseUrl: '{{ STATIC_URL }}'
    };
</script>
<script type="text/javascript" src="http://cdn.jsdelivr.net/requirejs/2.1.4/require.js"></script>
<script type="text/javascript">
    require(['common'], function (common) {
        {% block require_javascript %}
        {% endblock require_javascript %}
    });
</script>
{% endblock head_javascript %}

The first script block is essential, as RequireJS would use the application’s domain from where it’s originated to look for the JavaScript modules, and that would fail in production, where the static contents are deployed to Amazon S3. So that’s why we’re overriding the default configuration, by putting Django’s STATIC_URL as the baseUrl, and that way it will work in both the local development, and in the production environment. The next tag just loads RequireJS, and after it’s loaded, it will get djmodjs/assets/common.js, which is the global RequireJS configuration file. After common.js is loaded, the callback function will be executed, and that’s the interesting part, as we’ve put this code in our base.html file, we actually allow the body of that callback function to be different for all pages. One just have to override the require_javascript block in a child template.

Knocking out the youngling templates

By looking into djmodjs/templates/treasuremap/islands.html, we use the previously described technique to load the required JavaScript module for the page:

{% block require_javascript %}
    require(['treasuremap/js/islands']);
{% endblock require_javascript %}

We’ve built our templates in a way that no actual data from the database will be rendered on the server side into the HTML. That’s something what the client-side knockout template engine will do, after it makes an AJAX request to get the data. Django just renders a simple HTML page that contains the necessary knockout templates, and additionally, in order to avoid hardcoding API URLs into the JavaScript code, we are using Django’s url template tag to pass the URLs to the client-side code, which are used to make those AJAX requests to get the actual data. These URL’s are put into hidden input fields like this:

<input type="hidden" value="{% url 'treasuremap_get_islands' %}" data-bind="saveURL: loadIslands" />

And we’ve got a custom binding handler at the same place, called saveURL. This binding handler can be found in the djmodjs/assets/shared/js/handlers/getapiurl.js file:

ko.bindingHandlers.saveURL = {
    init: function (element, valueAccessor) {
        var url = $(element).val();
        var load_function = valueAccessor();
        load_function(url);
    }
};

It’s purpose is to read the value attribute of the input field, which is the actual URL rendered with Django’s url template tag, and then invoke a function with the found url parameter. The function in question was passed to the binding handler as parameter in the HTML template (data-bind="saveURL: loadIslands"). When knockout renders the template and applies the saveURL binding handler, it will try to find the function by the name we passed to it in it’s current context, and invoke it. As the djmodjs/assets/treasuremap/js/islands module is currently loaded, the current context is in it’s BrowseIslandsViewModel view:

function BrowseIslandsViewModel() {
    var self = this;
    self.islands = ko.observableArray([]);

    self.loadIslands = function (url) {
        $.ajax({
            url: url,
            type: 'GET',
            cache: false,
            success: function (result) {
                self.islands(utils.mapTo(result.islands, IslandModel));
            }
        });
    };
}

This view contains the loadIslands function, which receives the url parameter from our custom binding handler, and initiates an AJAX request to get the data from the server. Once the AJAX request is completed, and the data is successfully loaded into the models, knockout will render our island-template(s), where we actually have a similar, but a bit more complicated URL resolving situation:

<a data-bind="'attr': {'href': getURL('{% url 'treasuremap_load_island' '%d' %}') }">{% trans "Open Map" %}</a>

The href attribute is calculated when the knockout template is rendered, by calling the getURL function, which can be found in the djmodjs/assets/treasuremap/js/models/island.js file (the current context is no longer the root view, as we’re iterating over the BrowseIslandsViewModel.islands array, to render the knockout template). getURL receives an initial parameter called baseURL, which was put there on the server side by Django when it served the HTML file. That baseURL needs some modification, as it’s purpose is to load an island by getting it through the island’s primary key. So we just need to replace the %d part of the URL with the primary key of the island in question, and that data is already available, because we got it when we made the AJAX request.

self.getURL = function (baseURL) {
    return baseURL.replace('%d', self.pk());
};

This seems to be elegant, no hardcoded values anywhere. By the way, knockout was chosen primarly because it’s template language syntax, as we’re mixing Django templates with knockout templates. Now let’s see how can we deploy this…

Deployment

Clearly, node.js and RequireJS are needed to minify and glue the JavaScript modules together. After node.js is intalled, by running:

$ npm install requirejs

we will have RequireJS available as a new command line tool. Now, as we’re deploying to Heroku, we’re certainly already using git. Assuming all the development so far was done on the ‘master’ branch, we need a separate branch for deployment, let’s call it ‘deploy’. This is needed, as we don’t want to commit the minified / optimized static files to our ‘master’ branch, but we need them on Heroku. So by creating a separate branch, we will add and commit them there, and push that branch to Heroku as ‘master’. After deployment, we checkout the ‘master’ branch again, and continue the development there, and once we need to deploy again, just merge the changes from the ‘master’ to the ‘deploy’ branch, build the static files, and deploy.

Assuming that you followed the instuctions from the ultimate django-heroku-guide, and the Heroku application is already created, the git repository initialized, and the heroku remote is added, make sure your current working directory is the project root (manage.py and build.js are located there), and follow:

Verify that we’re on the master branch:

$ git branch
* master

Start tracking all files by git:

$ git add .

Commit everything into ‘master’:

$ git commit -m "initial commit"

Now create the new ‘deploy’ branch from the current state of ‘master’:

$ git checkout -b deploy

Proof that we’re on the ‘deploy’ branch:

$ git branch
* deploy
master

Permission granted to run the optimization process on the static files, which will create the assets-built folder:

$ r.js -o build.js

A new assets-built folder will appear as a direct neighbour to the assets folder. It will include the minified static files, and that’s the actual folder we want to put into production. Start tracking the newly generated files in the ‘deploy’ branch:

$ git add djmodjs/assets-built/

Commit the newly added files into ‘deploy’:

$ git commit -m "add optimized static files"

Push the ‘deploy’ branch to Heroku as ‘master’:

$ git push heroku deploy:master

After it’s deployed, return to the ‘master’ branch and continue with the development(naturally, the assets-built folder will not be present there):

$ git checkout master

Make some changes to the code, and if you’re ready to re-deploy:

$ git commit -am "some changes applied"
$ git checkout deploy
$ git merge master

Now all the changes from the ‘master’ branch are applied to the ‘deploy’ branch, you may continue from the step above where we ran the optimization using the r.js command-line tool.

As the ultimate django-heroku-guide clearly explains how to upload static content to Amazon S3 as well, we won’t bother with that, just note the important changes that we introduced. Django’s collectstatic will pick up the project’s static files by looking at the paths specified under STATICFILES_DIRS in the settings file (and in a couple of other places, but they’re not important for us now). By opening djmodjs/settings/common.py you can see that it specifies the assets folder as the source for the static files:

STATICFILES_DIRS = (
    normpath(join(DJANGO_ROOT, 'assets')),
)

However, djmodjs/settings/prod.py changes that option to assets-built:

STATICFILES_DIRS = (
    normpath(join(DJANGO_ROOT, 'assets-built')),
)

What this minor change allowed us is that when you work in your local development environment, Django will use and serve the uncompressed static files from the assets folder, while the collectstatic command on Heroku will send the optimized static files from the assets-built folder to Amazon S3. Perfect.

And that would be all, the entire solution is derived from these sources, and anything not covered in this article can be found somewhere here:

Now tell me this is actually a bad-design, that would be sooo…

Discussion

comments powered by Disqus