Download to read offline
![Django Documentation, Release 1.5.1
manage.py syncdb
The syncdb command looks at all your available models and creates tables in your database for whichever tables
don’t already exist.
2.1.3 Enjoy the free API
With that, you’ve got a free, and rich, Python API to access your data. The API is created on the fly, no code generation
necessary:
# Import the models we created from our "news" app
>>> from news.models import Reporter, Article
# No reporters are in the system yet.
>>> Reporter.objects.all()
[]
# Create a new Reporter.
>>> r = Reporter(full_name=’John Smith’)
# Save the object into the database. You have to call save() explicitly.
>>> r.save()
# Now it has an ID.
>>> r.id
1
# Now the new reporter is in the database.
>>> Reporter.objects.all()
[<Reporter: John Smith>]
# Fields are represented as attributes on the Python object.
>>> r.full_name
’John Smith’
# Django provides a rich database lookup API.
>>> Reporter.objects.get(id=1)
<Reporter: John Smith>
>>> Reporter.objects.get(full_name__startswith=’John’)
<Reporter: John Smith>
>>> Reporter.objects.get(full_name__contains=’mith’)
<Reporter: John Smith>
>>> Reporter.objects.get(id=2)
Traceback (most recent call last):
...
DoesNotExist: Reporter matching query does not exist. Lookup parameters were {’id’: 2}
# Create an article.
>>> from datetime import date
>>> a = Article(pub_date=date.today(), headline=’Django is cool’,
... content=’Yeah.’, reporter=r)
>>> a.save()
# Now the article is in the database.
>>> Article.objects.all()
[<Article: Django is cool>]
8 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-12-2048.jpg)
![Django Documentation, Release 1.5.1
# Article objects get API access to related Reporter objects.
>>> r = a.reporter
>>> r.full_name
’John Smith’
# And vice versa: Reporter objects get API access to Article objects.
>>> r.article_set.all()
[<Article: Django is cool>]
# The API follows relationships as far as you need, performing efficient
# JOINs for you behind the scenes.
# This finds all articles by a reporter whose name starts with "John".
>>> Article.objects.filter(reporter__full_name__startswith="John")
[<Article: Django is cool>]
# Change an object by altering its attributes and calling save().
>>> r.full_name = ’Billy Goat’
>>> r.save()
# Delete an object with delete().
>>> r.delete()
2.1.4 A dynamic admin interface: it’s not just scaffolding – it’s the whole house
Once your models are defined, Django can automatically create a professional, production ready administrative inter-face
– a Web site that lets authenticated users add, change and delete objects. It’s as easy as registering your model in
the admin site:
# In models.py...
from django.db import models
class Article(models.Model):
pub_date = models.DateField()
headline = models.CharField(max_length=200)
content = models.TextField()
reporter = models.ForeignKey(Reporter)
# In admin.py in the same directory...
import models
from django.contrib import admin
admin.site.register(models.Article)
The philosophy here is that your site is edited by a staff, or a client, or maybe just you – and you don’t want to have to
deal with creating backend interfaces just to manage content.
One typical workflow in creating Django apps is to create models and get the admin sites up and running as fast as
possible, so your staff (or clients) can start populating data. Then, develop the way data is presented to the public.
2.1.5 Design your URLs
A clean, elegant URL scheme is an important detail in a high-quality Web application. Django encourages beautiful
URL design and doesn’t put any cruft in URLs, like .php or .asp.
2.1. Django at a glance 9](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-13-2048.jpg)
![Django Documentation, Release 1.5.1
The next obvious steps are for you to download Django, read the tutorial and join the community. Thanks for your
interest!
2.2 Quick install guide
Before you can use Django, you’ll need to get it installed. We have a complete installation guide that covers all
the possibilities; this guide will guide you to a simple, minimal installation that’ll work while you walk through the
introduction.
2.2.1 Install Python
Being a Python Web framework, Django requires Python. It works with any Python version from 2.6.5 to 2.7. It also
features experimental support for versions 3.2 and 3.3. All these versions of Python include a lightweight database
called SQLite so you won’t need to set up a database just yet.
Get Python at http://www.python.org. If you’re running Linux or Mac OS X, you probably already have it installed.
Django on Jython
If you use Jython (a Python implementation for the Java platform), you’ll need to follow a few additional steps. See
Running Django on Jython for details.
You can verify that Python is installed by typing python from your shell; you should see something like:
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
2.2.2 Set up a database
This step is only necessary if you’d like to work with a “large” database engine like PostgreSQL, MySQL, or Oracle.
To install such a database, consult the database installation information.
2.2.3 Remove any old versions of Django
If you are upgrading your installation of Django from a previous version, you will need to uninstall the old Django
version before installing the new version.
2.2.4 Install Django
You’ve got three easy options to install Django:
• Install a version of Django provided by your operating system distribution. This is the quickest option for those
who have operating systems that distribute Django.
• Install an official release. This is the best approach for users who want a stable version number and aren’t
concerned about running a slightly older version of Django.
• Install the latest development version. This is best for users who want the latest-and-greatest features and aren’t
afraid of running brand-new code.
12 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-16-2048.jpg)
![Django Documentation, Release 1.5.1
project since the last time you ran syncdb. syncdb can be called as often as you like, and it will only ever create the
tables that don’t exist.
Read the django-admin.py documentation for full information on what the manage.py utility can do.
2.3.4 Playing with the API
Now, let’s hop into the interactive Python shell and play around with the free API Django gives you. To invoke the
Python shell, use this command:
python manage.py shell
We’re using this instead of simply typing “python”, because manage.py sets the DJANGO_SETTINGS_MODULE
environment variable, which gives Django the Python import path to your mysite/settings.py file.
Bypassing manage.py
If you’d rather not use manage.py, no problem. Just set the DJANGO_SETTINGS_MODULE environment variable
to mysite.settings and run python from the same directory manage.py is in (or ensure that directory is on
the Python path, so that import mysite works).
For more information on all of this, see the django-admin.py documentation.
Once you’re in the shell, explore the database API:
>>> from polls.models import Poll, Choice # Import the model classes we just wrote.
# No polls are in the system yet.
>>> Poll.objects.all()
[]
# Create a new Poll.
# Support for time zones is enabled in the default settings file, so
# Django expects a datetime with tzinfo for pub_date. Use timezone.now()
# instead of datetime.datetime.now() and it will do the right thing.
>>> from django.utils import timezone
>>> p = Poll(question="What’s new?", pub_date=timezone.now())
# Save the object into the database. You have to call save() explicitly.
>>> p.save()
# Now it has an ID. Note that this might say "1L" instead of "1", depending
# on which database you’re using. That’s no biggie; it just means your
# database backend prefers to return integers as Python long integer
# objects.
>>> p.id
1
# Access database columns via Python attributes.
>>> p.question
"What’s new?"
>>> p.pub_date
datetime.datetime(2012, 2, 26, 13, 0, 0, 775217, tzinfo=<UTC>)
# Change values by changing the attributes, then calling save().
>>> p.question = "What’s up?"
>>> p.save()
20 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-24-2048.jpg)
![Django Documentation, Release 1.5.1
# objects.all() displays all the polls in the database.
>>> Poll.objects.all()
[<Poll: Poll object>]
Wait a minute. <Poll: Poll object> is, utterly, an unhelpful representation of this object. Let’s fix that by
editing the polls model (in the polls/models.py file) and adding a __unicode__() method to both Poll and
Choice:
class Poll(models.Model):
# ...
def __unicode__(self):
return self.question
class Choice(models.Model):
# ...
def __unicode__(self):
return self.choice_text
It’s important to add __unicode__() methods to your models, not only for your own sanity when dealing with the
interactive prompt, but also because objects’ representations are used throughout Django’s automatically-generated
admin.
Why __unicode__() and not __str__()?
If you’re familiar with Python, you might be in the habit of adding __str__() methods to your classes, not
__unicode__() methods. We use __unicode__() here because Django models deal with Unicode by default.
All data stored in your database is converted to Unicode when it’s returned.
Django models have a default __str__() method that calls __unicode__() and converts the result to a UTF-8
bytestring. This means that unicode(p) will return a Unicode string, and str(p) will return a normal string, with
characters encoded as UTF-8.
If all of this is gibberish to you, just remember to add __unicode__() methods to your models. With any luck,
things should Just Work for you.
Note these are normal Python methods. Let’s add a custom method, just for demonstration:
import datetime
from django.utils import timezone
# ...
class Poll(models.Model):
# ...
def was_published_recently(self):
return self.pub_date >= timezone.now() - datetime.timedelta(days=1)
Note the addition of import datetime and from django.utils import timezone, to reference
Python’s standard datetime module and Django’s time-zone-related utilities in django.utils.timezone,
respectively. If you aren’t familiar with time zone handling in Python, you can learn more in the time zone support
docs.
Save these changes and start a new Python interactive shell by running python manage.py shell again:
>>> from polls.models import Poll, Choice
# Make sure our __unicode__() addition worked.
>>> Poll.objects.all()
[<Poll: What’s up?>]
2.3. Writing your first Django app, part 1 21](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-25-2048.jpg)
![Django Documentation, Release 1.5.1
# Django provides a rich database lookup API that’s entirely driven by
# keyword arguments.
>>> Poll.objects.filter(id=1)
[<Poll: What’s up?>]
>>> Poll.objects.filter(question__startswith=’What’)
[<Poll: What’s up?>]
# Get the poll that was published this year.
>>> from django.utils import timezone
>>> current_year = timezone.now().year
>>> Poll.objects.get(pub_date__year=current_year)
<Poll: What’s up?>
# Request an ID that doesn’t exist, this will raise an exception.
>>> Poll.objects.get(id=2)
Traceback (most recent call last):
...
DoesNotExist: Poll matching query does not exist. Lookup parameters were {’id’: 2}
# Lookup by a primary key is the most common case, so Django provides a
# shortcut for primary-key exact lookups.
# The following is identical to Poll.objects.get(id=1).
>>> Poll.objects.get(pk=1)
<Poll: What’s up?>
# Make sure our custom method worked.
>>> p = Poll.objects.get(pk=1)
>>> p.was_published_recently()
True
# Give the Poll a couple of Choices. The create call constructs a new
# Choice object, does the INSERT statement, adds the choice to the set
# of available choices and returns the new Choice object. Django creates
# a set to hold the "other side" of a ForeignKey relation
# (e.g. a poll’s choices) which can be accessed via the API.
>>> p = Poll.objects.get(pk=1)
# Display any choices from the related object set -- none so far.
>>> p.choice_set.all()
[]
# Create three choices.
>>> p.choice_set.create(choice_text=’Not much’, votes=0)
<Choice: Not much>
>>> p.choice_set.create(choice_text=’The sky’, votes=0)
<Choice: The sky>
>>> c = p.choice_set.create(choice_text=’Just hacking again’, votes=0)
# Choice objects have API access to their related Poll objects.
>>> c.poll
<Poll: What’s up?>
# And vice versa: Poll objects get access to Choice objects.
>>> p.choice_set.all()
[<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>]
>>> p.choice_set.count()
3
22 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-26-2048.jpg)
![Django Documentation, Release 1.5.1
# The API automatically follows relationships as far as you need.
# Use double underscores to separate relationships.
# This works as many levels deep as you want; there’s no limit.
# Find all Choices for any poll whose pub_date is in this year
# (reusing the ’current_year’ variable we created above).
>>> Choice.objects.filter(poll__pub_date__year=current_year)
[<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>]
# Let’s delete one of the choices. Use delete() for that.
>>> c = p.choice_set.filter(choice_text__startswith=’Just hacking’)
>>> c.delete()
For more information on model relations, see Accessing related objects. For more on how to use double underscores
to perform field lookups via the API, see Field lookups. For full details on the database API, see our Database API
reference.
When you’re comfortable with the API, read part 2 of this tutorial to get Django’s automatic admin working.
2.4 Writing your first Django app, part 2
This tutorial begins where Tutorial 1 left off. We’re continuing the Web-poll application and will focus on Django’s
automatically-generated admin site.
Philosophy
Generating admin sites for your staff or clients to add, change and delete content is tedious work that doesn’t require
much creativity. For that reason, Django entirely automates creation of admin interfaces for models.
Django was written in a newsroom environment, with a very clear separation between “content publishers” and the
“public” site. Site managers use the system to add news stories, events, sports scores, etc., and that content is displayed
on the public site. Django solves the problem of creating a unified interface for site administrators to edit content.
The admin isn’t intended to be used by site visitors. It’s for site managers.
2.4.1 Activate the admin site
The Django admin site is not activated by default – it’s an opt-in thing. To activate the admin site for your installation,
do these three things:
• Uncomment "django.contrib.admin" in the INSTALLED_APPS setting.
• Run python manage.py syncdb. Since you have added a new application to INSTALLED_APPS, the
database tables need to be updated.
• Edit your mysite/urls.py file and uncomment the lines that reference the admin – there are three lines in
total to uncomment. This file is a URLconf; we’ll dig into URLconfs in the next tutorial. For now, all you need
to know is that it maps URL roots to applications. In the end, you should have a urls.py file that looks like
this:
from django.conf.urls import patterns, include, url
# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()
2.4. Writing your first Django app, part 2 23](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-27-2048.jpg)
![Django Documentation, Release 1.5.1
2.4.6 Customize the admin form
Take a few minutes to marvel at all the code you didn’t have to write. By registering the Poll model with
admin.site.register(Poll), Django was able to construct a default form representation. Often, you’ll want
to customize how the admin form looks and works. You’ll do this by telling Django the options you want when you
register the object.
Let’s see how this works by re-ordering the fields on the edit form. Replace the admin.site.register(Poll)
line with:
class PollAdmin(admin.ModelAdmin):
fields = [’pub_date’, ’question’]
admin.site.register(Poll, PollAdmin)
You’ll follow this pattern – create a model admin object, then pass it as the second argument to
admin.site.register() – any time you need to change the admin options for an object.
This particular change above makes the “Publication date” come before the “Question” field:
This isn’t impressive with only two fields, but for admin forms with dozens of fields, choosing an intuitive order is an
important usability detail.
And speaking of forms with dozens of fields, you might want to split the form up into fieldsets:
2.4. Writing your first Django app, part 2 27](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-31-2048.jpg)
![Django Documentation, Release 1.5.1
class PollAdmin(admin.ModelAdmin):
fieldsets = [
(None, {’fields’: [’question’]}),
(’Date information’, {’fields’: [’pub_date’]}),
]
admin.site.register(Poll, PollAdmin)
The first element of each tuple in fieldsets is the title of the fieldset. Here’s what our form looks like now:
You can assign arbitrary HTML classes to each fieldset. Django provides a "collapse" class that displays a
particular fieldset initially collapsed. This is useful when you have a long form that contains a number of fields that
aren’t commonly used:
class PollAdmin(admin.ModelAdmin):
fieldsets = [
(None, {’fields’: [’question’]}),
(’Date information’, {’fields’: [’pub_date’], ’classes’: [’collapse’]}),
]
28 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-32-2048.jpg)
![Django Documentation, Release 1.5.1
class PollAdmin(admin.ModelAdmin):
fieldsets = [
(None, {’fields’: [’question’]}),
(’Date information’, {’fields’: [’pub_date’], ’classes’: [’collapse’]}),
]
inlines = [ChoiceInline]
admin.site.register(Poll, PollAdmin)
This tells Django: “Choice objects are edited on the Poll admin page. By default, provide enough fields for 3
choices.”
Load the “Add poll” page to see how that looks, you may need to restart your development server:
It works like this: There are three slots for related Choices – as specified by extra – and each time you come back
to the “Change” page for an already-created object, you get another three extra slots.
At the end of the three current slots you will find an “Add another Choice” link. If you click on it, a new slot will be
added. If you want to remove the added slot, you can click on the X to the top right of the added slot. Note that you
can’t remove the original three slots. This image shows an added slot:
30 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-34-2048.jpg)
![Django Documentation, Release 1.5.1
By default, Django displays the str() of each object. But sometimes it’d be more helpful if we could display
individual fields. To do that, use the list_display admin option, which is a tuple of field names to display, as
columns, on the change list page for the object:
class PollAdmin(admin.ModelAdmin):
# ...
list_display = (’question’, ’pub_date’)
Just for good measure, let’s also include the was_published_recently custom method from Tutorial 1:
class PollAdmin(admin.ModelAdmin):
# ...
list_display = (’question’, ’pub_date’, ’was_published_recently’)
Now the poll change list page looks like this:
You can click on the column headers to sort by those values – except in the case of the was_published_recently
header, because sorting by the output of an arbitrary method is not supported. Also note that the column header for
was_published_recently is, by default, the name of the method (with underscores replaced with spaces), and
that each line contains the string representation of the output.
You can improve that by giving that method (in polls/models.py) a few attributes, as follows:
class Poll(models.Model):
# ...
def was_published_recently(self):
return self.pub_date >= timezone.now() - datetime.timedelta(days=1)
was_published_recently.admin_order_field = ’pub_date’
was_published_recently.boolean = True
was_published_recently.short_description = ’Published recently?’
Edit your polls/admin.py file again and add an improvement to the Poll change list page: Filters. Add the
following line to PollAdmin:
list_filter = [’pub_date’]
32 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-36-2048.jpg)
![Django Documentation, Release 1.5.1
That adds a “Filter” sidebar that lets people filter the change list by the pub_date field:
The type of filter displayed depends on the type of field you’re filtering on. Because pub_date is a
DateTimeField, Django knows to give appropriate filter options: “Any date,” “Today,” “Past 7 days,” “This
month,” “This year.”
This is shaping up well. Let’s add some search capability:
search_fields = [’question’]
That adds a search box at the top of the change list. When somebody enters search terms, Django will search the
question field. You can use as many fields as you’d like – although because it uses a LIKE query behind the
scenes, keep it reasonable, to keep your database happy.
Finally, because Poll objects have dates, it’d be convenient to be able to drill down by date. Add this line:
date_hierarchy = ’pub_date’
That adds hierarchical navigation, by date, to the top of the change list page. At top level, it displays all available
years. Then it drills down to months and, ultimately, days.
Now’s also a good time to note that change lists give you free pagination. The default is to display 100 items per page.
Change-list pagination, search boxes, filters, date-hierarchies and column-header-ordering all work together like you
think they should.
2.4.9 Customize the admin look and feel
Clearly, having “Django administration” at the top of each admin page is ridiculous. It’s just placeholder text.
That’s easy to change, though, using Django’s template system. The Django admin is powered by Django itself, and
its interfaces use Django’s own template system.
Customizing your project’s templates
Create a templates directory in your project directory. Templates can live anywhere on your filesystem that Django
can access. (Django runs as whatever user your server runs.) However, keeping your templates within the project is a
good convention to follow.
Open your settings file (mysite/settings.py, remember) and add a TEMPLATE_DIRS setting:
TEMPLATE_DIRS = (
’/path/to/mysite/templates’, # Change this to your own directory.
)
Now copy the template admin/base_site.html from within the default Django admin tem-plate
directory in the source code of Django itself (django/contrib/admin/templates)
2.4. Writing your first Django app, part 2 33](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-37-2048.jpg)
![Django Documentation, Release 1.5.1
into an admin subdirectory of whichever directory you’re using in TEMPLATE_DIRS.
For example, if your TEMPLATE_DIRS includes ’/path/to/mysite/templates’, as
above, then copy django/contrib/admin/templates/admin/base_site.html to
/path/to/mysite/templates/admin/base_site.html. Don’t forget that admin subdirectory.
Where are the Django source files?
If you have difficulty finding where the Django source files are located on your system, run the following command:
python -c "
import sys
sys.path = sys.path[1:]
import django
print(django.__path__)"
Then, just edit the file and replace the generic Django text with your own site’s name as you see fit.
This template file contains lots of text like {% block branding %} and {{ title }}. The {% and {{ tags
are part of Django’s template language. When Django renders admin/base_site.html, this template language
will be evaluated to produce the final HTML page. Don’t worry if you can’t make any sense of the template right now
– we’ll delve into Django’s templating language in Tutorial 3.
Note that any of Django’s default admin templates can be overridden. To override a template, just do the same thing
you did with base_site.html – copy it from the default directory into your custom directory, and make changes.
Customizing your application’s templates
Astute readers will ask: But if TEMPLATE_DIRS was empty by default, how was Django finding the default admin
templates? The answer is that, by default, Django automatically looks for a templates/ subdirectory within each
application package, for use as a fallback (don’t forget that django.contrib.admin is an application).
Our poll application is not very complex and doesn’t need custom admin templates. But if it grew more sophisticated
and required modification of Django’s standard admin templates for some of its functionality, it would be more sensible
to modify the application’s templates, rather than those in the project. That way, you could include the polls application
in any new project and be assured that it would find the custom templates it needed.
See the template loader documentation for more information about how Django finds its templates.
2.4.10 Customize the admin index page
On a similar note, you might want to customize the look and feel of the Django admin index page.
By default, it displays all the apps in INSTALLED_APPS that have been registered with the admin application, in
alphabetical order. You may want to make significant changes to the layout. After all, the index is probably the most
important page of the admin, and it should be easy to use.
The template to customize is admin/index.html. (Do the same as with admin/base_site.html in the
previous section – copy it from the default directory to your custom template directory.) Edit the file, and you’ll see it
uses a template variable called app_list. That variable contains every installed Django app. Instead of using that,
you can hard-code links to object-specific admin pages in whatever way you think is best. Again, don’t worry if you
can’t understand the template language – we’ll cover that in more detail in Tutorial 3.
When you’re comfortable with the admin site, read part 3 of this tutorial to start working on public poll views.
34 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-38-2048.jpg)
![Django Documentation, Release 1.5.1
Note that the regular expressions for the include() functions don’t have a $ (end-of-string match character) but
rather a trailing slash. Whenever Django encounters include(), it chops off whatever part of the URL matched up
to that point and sends the remaining string to the included URLconf for further processing.
The idea behind include() is to make it easy to plug-and-play URLs. Since polls are in their own URLconf
(polls/urls.py), they can be placed under “/polls/”, or under “/fun_polls/”, or under “/content/polls/”, or any
other path root, and the app will still work.
Here’s what happens if a user goes to “/polls/34/” in this system:
• Django will find the match at ’^polls/’
• Then, Django will strip off the matching text ("polls/") and send the remaining text – "34/" – to the
‘polls.urls’ URLconf for further processing which matches r’^(?P<poll_id>d+)/$’ resulting in a call
to the detail() view like so:
detail(request=<HttpRequest object>, poll_id=’34’)
The poll_id=’34’ part comes from (?P<poll_id>d+). Using parentheses around a pattern “captures” the
text matched by that pattern and sends it as an argument to the view function; ?P<poll_id> defines the name that
will be used to identify the matched pattern; and d+ is a regular expression to match a sequence of digits (i.e., a
number).
Because the URL patterns are regular expressions, there really is no limit on what you can do with them. And there’s
no need to add URL cruft such as .html – unless you want to, in which case you can do something like this:
(r’^polls/latest.html$’, ’polls.views.index’),
But, don’t do that. It’s silly.
2.5.4 Write views that actually do something
Each view is responsible for doing one of two things: returning an HttpResponse object containing the content for
the requested page, or raising an exception such as Http404. The rest is up to you.
Your view can read records from a database, or not. It can use a template system such as Django’s – or a third-party
Python template system – or not. It can generate a PDF file, output XML, create a ZIP file on the fly, anything you
want, using whatever Python libraries you want.
All Django wants is that HttpResponse. Or an exception.
Because it’s convenient, let’s use Django’s own database API, which we covered in Tutorial 1. Here’s one stab at
the index() view, which displays the latest 5 poll questions in the system, separated by commas, according to
publication date:
from django.http import HttpResponse
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.order_by(’-pub_date’)[:5]
output = ’, ’.join([p.question for p in latest_poll_list])
return HttpResponse(output)
There’s a problem here, though: the page’s design is hard-coded in the view. If you want to change the way the page
looks, you’ll have to edit this Python code. So let’s use Django’s template system to separate the design from Python
by creating a template that the view can use.
First, create a directory called templates in your polls directory. Django will look for templates in there.
38 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-42-2048.jpg)
![Django Documentation, Release 1.5.1
Django’s TEMPLATE_LOADERS setting contains a list of callables that know how to import templates from various
sources. One of the defaults is django.template.loaders.app_directories.Loader which looks for
a “templates” subdirectory in each of the INSTALLED_APPS - this is how Django knows to find the polls templates
even though we didn’t modify TEMPLATE_DIRS, as we did in Tutorial 2.
Organizing templates
We could have all our templates together, in one big templates directory, and it would work perfectly well. However,
this template belongs to the polls application, so unlike the admin template we created in the previous tutorial, we’ll
put this one in the application’s template directory (polls/templates) rather than the project’s (templates).
We’ll discuss in more detail in the reusable apps tutorial why we do this.
Within the templates directory you have just created, create another directory called polls,
and within that create a file called index.html. In other words, your template should be at
polls/templates/polls/index.html. Because of how the app_directories template loader
works as described above, you can refer to this template within Django simply as polls/index.html.
Template namespacing
Now we might be able to get away with putting our templates directly in polls/templates (rather than creating
another polls subdirectory), but it would actually be a bad idea. Django will choose the first template it finds whose
name matches, and if you had a template with the same name in a different application, Django would be unable to
distinguish between them. We need to be able to point Django at the right one, and the easiest way to ensure this is by
namespacing them. That is, by putting those templates inside another directory named for the application itself.
Put the following code in that template:
{% if latest_poll_list %}
<ul>
{% for poll in latest_poll_list %}
<li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li>
{% endfor %}
</ul>
{% else %}
<p>No polls are available.</p>
{% endif %}
Now let’s use that html template in our index view:
from django.http import HttpResponse
from django.template import Context, loader
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.order_by(’-pub_date’)[:5]
template = loader.get_template(’polls/index.html’)
context = Context({
’latest_poll_list’: latest_poll_list,
})
return HttpResponse(template.render(context))
That code loads the template called polls/index.html and passes it a context. The context is a dictionary
mapping template variable names to Python objects.
Load the page by pointing your browser at “/polls/”, and you should see a bulleted-list containing the “What’s up” poll
from Tutorial 1. The link points to the poll’s detail page.
2.5. Writing your first Django app, part 3 39](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-43-2048.jpg)
![Django Documentation, Release 1.5.1
A shortcut: render()
It’s a very common idiom to load a template, fill a context and return an HttpResponse object with the result of the
rendered template. Django provides a shortcut. Here’s the full index() view, rewritten:
from django.shortcuts import render
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.all().order_by(’-pub_date’)[:5]
context = {’latest_poll_list’: latest_poll_list}
return render(request, ’polls/index.html’, context)
Note that once we’ve done this in all these views, we no longer need to import loader, Context and
HttpResponse (you’ll want to keep HttpResponse if you still have the stub methods for detail, results,
and vote).
The render() function takes the request object as its first argument, a template name as its second argument and a
dictionary as its optional third argument. It returns an HttpResponse object of the given template rendered with
the given context.
2.5.5 Raising a 404 error
Now, let’s tackle the poll detail view – the page that displays the question for a given poll. Here’s the view:
from django.http import Http404
# ...
def detail(request, poll_id):
try:
poll = Poll.objects.get(pk=poll_id)
except Poll.DoesNotExist:
raise Http404
return render(request, ’polls/detail.html’, {’poll’: poll})
The new concept here: The view raises the Http404 exception if a poll with the requested ID doesn’t exist.
We’ll discuss what you could put in that polls/detail.html template a bit later, but if you’d like to quickly get
the above example working, a file containing just:
{{ poll }}
will get you started for now.
A shortcut: get_object_or_404()
It’s a very common idiom to use get() and raise Http404 if the object doesn’t exist. Django provides a shortcut.
Here’s the detail() view, rewritten:
from django.shortcuts import render, get_object_or_404
# ...
def detail(request, poll_id):
poll = get_object_or_404(Poll, pk=poll_id)
return render(request, ’polls/detail.html’, {’poll’: poll})
40 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-44-2048.jpg)
![Django Documentation, Release 1.5.1
form will alter data server-side. Whenever you create a form that alters data server-side, use method="post".
This tip isn’t specific to Django; it’s just good Web development practice.
• forloop.counter indicates how many times the for tag has gone through its loop
• Since we’re creating a POST form (which can have the effect of modifying data), we need to worry about Cross
Site Request Forgeries. Thankfully, you don’t have to worry too hard, because Django comes with a very easy-to-
use system for protecting against it. In short, all POST forms that are targeted at internal URLs should use
the {% csrf_token %} template tag.
Now, let’s create a Django view that handles the submitted data and does something with it. Remember, in Tutorial 3,
we created a URLconf for the polls application that includes this line:
url(r’^(?P<poll_id>d+)/vote/$’, views.vote, name=’vote’),
We also created a dummy implementation of the vote() function. Let’s create a real version. Add the following to
polls/views.py:
from django.shortcuts import get_object_or_404, render
from django.http import HttpResponseRedirect, HttpResponse
from django.core.urlresolvers import reverse
from polls.models import Choice, Poll
# ...
def vote(request, poll_id):
p = get_object_or_404(Poll, pk=poll_id)
try:
selected_choice = p.choice_set.get(pk=request.POST[’choice’])
except (KeyError, Choice.DoesNotExist):
# Redisplay the poll voting form.
return render(request, ’polls/detail.html’, {
’poll’: p,
’error_message’: "You didn’t select a choice.",
})
else:
selected_choice.votes += 1
selected_choice.save()
# Always return an HttpResponseRedirect after successfully dealing
# with POST data. This prevents data from being posted twice if a
# user hits the Back button.
return HttpResponseRedirect(reverse(’polls:results’, args=(p.id,)))
This code includes a few things we haven’t covered yet in this tutorial:
• request.POST is a dictionary-like object that lets you access submitted data by key name. In this case,
request.POST[’choice’] returns the ID of the selected choice, as a string. request.POST values are
always strings.
Note that Django also provides request.GET for accessing GET data in the same way – but we’re explicitly
using request.POST in our code, to ensure that data is only altered via a POST call.
• request.POST[’choice’] will raise KeyError if choice wasn’t provided in POST data. The above
code checks for KeyError and redisplays the poll form with an error message if choice isn’t given.
• After incrementing the choice count, the code returns an HttpResponseRedirect rather than a normal
HttpResponse. HttpResponseRedirect takes a single argument: the URL to which the user will be
redirected (see the following point for how we construct the URL in this case).
As the Python comment above points out, you should always return an HttpResponseRedirect after
successfully dealing with POST data. This tip isn’t specific to Django; it’s just goodWeb development practice.
44 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-48-2048.jpg)
![Django Documentation, Release 1.5.1
Generally, when writing a Django app, you’ll evaluate whether generic views are a good fit for your problem, and
you’ll use them from the beginning, rather than refactoring your code halfway through. But this tutorial intentionally
has focused on writing the views “the hard way” until now, to focus on core concepts.
You should know basic math before you start using a calculator.
Amend URLconf
First, open the polls/urls.py URLconf and change it like so:
from django.conf.urls import patterns, url
from django.views.generic import DetailView, ListView
from polls.models import Poll
urlpatterns = patterns(’’,
url(r’^$’,
ListView.as_view(
queryset=Poll.objects.order_by(’-pub_date’)[:5],
context_object_name=’latest_poll_list’,
template_name=’polls/index.html’),
name=’index’),
url(r’^(?P<pk>d+)/$’,
DetailView.as_view(
model=Poll,
template_name=’polls/detail.html’),
name=’detail’),
url(r’^(?P<pk>d+)/results/$’,
DetailView.as_view(
model=Poll,
template_name=’polls/results.html’),
name=’results’),
url(r’^(?P<poll_id>d+)/vote/$’, ’polls.views.vote’, name=’vote’),
)
Amend views
We’re using two generic views here: ListView and DetailView. Respectively, those two views abstract the
concepts of “display a list of objects” and “display a detail page for a particular type of object.”
• Each generic view needs to know what model it will be acting upon. This is provided using the model param-eter.
• The DetailView generic view expects the primary key value captured from the URL to be called "pk", so
we’ve changed poll_id to pk for the generic views.
By default, the DetailView generic view uses a template called <app name>/<model
name>_detail.html. In our case, it’ll use the template "polls/poll_detail.html". The
template_name argument is used to tell Django to use a specific template name instead of the autogener-ated
default template name. We also specify the template_name for the results list view – this ensures that the
results view and the detail view have a different appearance when rendered, even though they’re both a DetailView
behind the scenes.
Similarly, the ListView generic view uses a default template called <app name>/<model
name>_list.html; we use template_name to tell ListView to use our existing "polls/index.html"
template.
46 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-50-2048.jpg)
![Django Documentation, Release 1.5.1
We will start again with the shell, where we need to do a couple of things that won’t be necessary in tests.py. The
first is to set up the test environment in the shell:
>>> from django.test.utils import setup_test_environment
>>> setup_test_environment()
Next we need to import the test client class (later in tests.py we will use the django.test.TestCase class,
which comes with its own client, so this won’t be required):
>>> from django.test.client import Client
>>> # create an instance of the client for our use
>>> client = Client()
With that ready, we can ask the client to do some work for us:
>>> # get a response from ’/’
>>> response = client.get(’/’)
>>> # we should expect a 404 from that address
>>> response.status_code
404
>>> # on the other hand we should expect to find something at ’/polls/’
>>> # we’ll use ’reverse()’ rather than a harcoded URL
>>> from django.core.urlresolvers import reverse
>>> response = client.get(reverse(’polls:index’))
>>> response.status_code
200
>>> response.content
’nnn <p>No polls are available.</p>nn’
>>> # note - you might get unexpected results if your ‘‘TIME_ZONE‘‘
>>> # in ‘‘settings.py‘‘ is not correct. If you need to change it,
>>> # you will also need to restart your shell session
>>> from polls.models import Poll
>>> from django.utils import timezone
>>> # create a Poll and save it
>>> p = Poll(question="Who is your favorite Beatle?", pub_date=timezone.now())
>>> p.save()
>>> # check the response once again
>>> response = client.get(’/polls/’)
>>> response.content
’nnn <ul>n n <li><a href="/polls/1/">Who is your favorite Beatle?</a></li>n n >>> response.context[’latest_poll_list’]
[<Poll: Who is your favorite Beatle?>]
Improving our view
The list of polls shows polls that aren’t published yet (i.e. those that have a pub_date in the future). Let’s fix that.
In Tutorial 4 we deleted the view functions from views.py in favor of a ListView in urls.py:
url(r’^$’,
ListView.as_view(
queryset=Poll.objects.order_by(’-pub_date’)[:5],
context_object_name=’latest_poll_list’,
template_name=’polls/index.html’),
name=’index’),
response.context_data[’latest_poll_list’] extracts the data this view places into the context.
We need to amend the line that gives us the queryset:
52 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-56-2048.jpg)
![Django Documentation, Release 1.5.1
queryset=Poll.objects.order_by(’-pub_date’)[:5],
Let’s change the queryset so that it also checks the date by comparing it with timezone.now(). First we need to
add an import:
from django.utils import timezone
and then we must amend the existing url function to:
url(r’^$’,
ListView.as_view(
queryset=Poll.objects.filter(pub_date__lte=timezone.now)
.order_by(’-pub_date’)[:5],
context_object_name=’latest_poll_list’,
template_name=’polls/index.html’),
name=’index’),
Poll.objects.filter(pub_date__lte=timezone.now) returns a queryset containing Polls whose
pub_date is less than or equal to - that is, earlier than or equal to - timezone.now. Notice that we use a callable
queryset argument, timezone.now, which will be evaluated at request time. If we had included the parentheses,
timezone.now() would be evaluated just once when the web server is started.
Testing our new view
Now you can satisfy yourself that this behaves as expected by firing up the runserver, loading the site in your browser,
creating Polls with dates in the past and future, and checking that only those that have been published are listed.
You don’t want to have to do that every single time you make any change that might affect this - so let’s also create a
test, based on our shell session above.
Add the following to polls/tests.py:
from django.core.urlresolvers import reverse
and we’ll create a factory method to create polls as well as a new test class:
def create_poll(question, days):
"""
Creates a poll with the given ‘question‘ published the given number of
‘days‘ offset to now (negative for polls published in the past,
positive for polls that have yet to be published).
"""
return Poll.objects.create(question=question,
pub_date=timezone.now() + datetime.timedelta(days=days))
class PollViewTests(TestCase):
def test_index_view_with_no_polls(self):
"""
If no polls exist, an appropriate message should be displayed.
"""
response = self.client.get(reverse(’polls:index’))
self.assertEqual(response.status_code, 200)
self.assertContains(response, "No polls are available.")
self.assertQuerysetEqual(response.context[’latest_poll_list’], [])
def test_index_view_with_a_past_poll(self):
"""
Polls with a pub_date in the past should be displayed on the index page.
"""
2.7. Writing your first Django app, part 5 53](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-57-2048.jpg)
![Django Documentation, Release 1.5.1
create_poll(question="Past poll.", days=-30)
response = self.client.get(reverse(’polls:index’))
self.assertQuerysetEqual(
response.context[’latest_poll_list’],
[’<Poll: Past poll.>’]
)
def test_index_view_with_a_future_poll(self):
"""
Polls with a pub_date in the future should not be displayed on the
index page.
"""
create_poll(question="Future poll.", days=30)
response = self.client.get(reverse(’polls:index’))
self.assertContains(response, "No polls are available.", status_code=200)
self.assertQuerysetEqual(response.context[’latest_poll_list’], [])
def test_index_view_with_future_poll_and_past_poll(self):
"""
Even if both past and future polls exist, only past polls should be
displayed.
"""
create_poll(question="Past poll.", days=-30)
create_poll(question="Future poll.", days=30)
response = self.client.get(reverse(’polls:index’))
self.assertQuerysetEqual(
response.context[’latest_poll_list’],
[’<Poll: Past poll.>’]
)
def test_index_view_with_two_past_polls(self):
"""
The polls index page may display multiple polls.
"""
create_poll(question="Past poll 1.", days=-30)
create_poll(question="Past poll 2.", days=-5)
response = self.client.get(reverse(’polls:index’))
self.assertQuerysetEqual(
response.context[’latest_poll_list’],
[’<Poll: Past poll 2.>’, ’<Poll: Past poll 1.>’]
)
Let’s look at some of these more closely.
First is a poll factory method, create_poll, to take some repetition out of the process of creating polls.
test_index_view_with_no_polls doesn’t create any polls, but checks the message: “No polls are available.”
and verifies the latest_poll_list is empty. Note that the django.test.TestCase class provides some
additional assertion methods. In these examples, we use assertContains() and assertQuerysetEqual().
In test_index_view_with_a_past_poll, we create a poll and verify that it appears in the list.
In test_index_view_with_a_future_poll, we create a poll with a pub_date in the future. The database
is reset for each test method, so the first poll is no longer there, and so again the index shouldn’t have any polls in it.
And so on. In effect, we are using the tests to tell a story of admin input and user experience on the site, and checking
that at every state and for every new change in the state of the system, the expected results are published.
54 Chapter 2. Getting started](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-58-2048.jpg)
![Django Documentation, Release 1.5.1
README = open(os.path.join(os.path.dirname(__file__), ’README.rst’)).read()
# allow setup.py to be run from any path
os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir)))
setup(
name=’django-polls’,
version=’0.1’,
packages=[’polls’],
include_package_data=True,
license=’BSD License’, # example license
description=’A simple Django app to conduct Web-based polls.’,
long_description=README,
url=’http://www.example.com/’,
author=’Your Name’,
author_email=’yourname@example.com’,
classifiers=[
’Environment :: Web Environment’,
’Framework :: Django’,
’Intended Audience :: Developers’,
’License :: OSI Approved :: BSD License’, # example license
’Operating System :: OS Independent’,
’Programming Language :: Python’,
’Programming Language :: Python :: 2.6’,
’Programming Language :: Python :: 2.7’,
’Topic :: Internet :: WWW/HTTP’,
’Topic :: Internet :: WWW/HTTP :: Dynamic Content’,
],
)
I thought you said we were going to use distribute?
Distribute is a drop-in replacement for setuptools. Even though we appear to import from setuptools, since
we have distribute installed, it will override the import.
6. Only Python modules and packages are included in the package by default. To include additional files,
we’ll need to create a MANIFEST.in file. The distribute docs referred to in the previous step discuss
this file in more details. To include the templates, the README.rst and our LICENSE file, create a file
django-polls/MANIFEST.in with the following contents:
include LICENSE
include README.rst
recursive-include polls/templates *
7. It’s optional, but recommended, to include detailed documentation with your app. Create an
empty directory django-polls/docs for future documentation. Add an additional line to
django-polls/MANIFEST.in:
recursive-include docs *
Note that the docs directory won’t be included in your package unless you add some files to it. Many Django
apps also provide their documentation online through sites like readthedocs.org.
8. Try building your package with python setup.py sdist (run from inside django-polls). This cre-ates
a directory called dist and builds your new package, django-polls-0.1.tar.gz.
For more information on packaging, see The Hitchhiker’s Guide to Packaging.
2.9. Advanced tutorial: How to write reusable apps 61](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-65-2048.jpg)
![Django Documentation, Release 1.5.1
a database user with these permissions, you’ll specify the details in your project’s settings file, see DATABASES for
details.
If you’re using Django’s testing framework to test database queries, Django will need permission to create a test
database.
3.1.4 Remove any old versions of Django
If you are upgrading your installation of Django from a previous version, you will need to uninstall the old Django
version before installing the new version.
If you installed Django using pip or easy_install previously, installing with pip or easy_install again will
automatically take care of the old version, so you don’t need to do it yourself.
If you previously installed Django using python setup.py install, uninstalling is as simple as deleting the
django directory from your Python site-packages. To find the directory you need to remove, you can run the
following at your shell prompt (not the interactive Python prompt):
python -c "import sys; sys.path = sys.path[1:]; import django; print(django.__path__)"
3.1.5 Install the Django code
Installation instructions are slightly different depending on whether you’re installing a distribution-specific package,
downloading the latest official release, or fetching the latest development version.
It’s easy, no matter which way you choose.
Installing a distribution-specific package
Check the distribution specific notes to see if your platform/distribution provides official Django packages/installers.
Distribution-provided packages will typically allow for automatic installation of dependencies and easy upgrade paths.
Installing an official release with pip
This is the recommended way to install Django.
1. Install pip. The easiest is to use the standalone pip installer. If your distribution already has pip installed, you
might need to update it if it’s outdated. (If it’s outdated, you’ll know because installation won’t work.)
2. (optional) Take a look at virtualenv and virtualenvwrapper. These tools provide isolated Python environments,
which are more practical than installing packages systemwide. They also allow installing packages without
administrator privileges. It’s up to you to decide if you want to learn and use them.
3. If you’re using Linux, Mac OS X or some other flavor of Unix, enter the command sudo pip install
Django at the shell prompt. If you’re using Windows, start a command shell with administrator privi-leges
and run the command pip install Django. This will install Django in your Python installation’s
site-packages directory.
If you’re using a virtualenv, you don’t need sudo or administrator privileges, and this will install Django in the
virtualenv’s site-packages directory.
3.1. How to install Django 77](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-81-2048.jpg)
![Django Documentation, Release 1.5.1
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through=’Membership’)
def __unicode__(self):
return self.name
class Membership(models.Model):
person = models.ForeignKey(Person)
group = models.ForeignKey(Group)
date_joined = models.DateField()
invite_reason = models.CharField(max_length=64)
When you set up the intermediary model, you explicitly specify foreign keys to the models that are involved in the
ManyToMany relation. This explicit declaration defines how the two models are related.
There are a few restrictions on the intermediate model:
• Your intermediate model must contain one - and only one - foreign key to the target model (this would be
Person in our example). If you have more than one foreign key, a validation error will be raised.
• Your intermediate model must contain one - and only one - foreign key to the source model (this would be
Group in our example). If you have more than one foreign key, a validation error will be raised.
• The only exception to this is a model which has a many-to-many relationship to itself, through an intermediary
model. In this case, two foreign keys to the same model are permitted, but they will be treated as the two
(different) sides of the many-to-many relation.
• When defining a many-to-many relationship from a model to itself, using an intermediary model, you must use
symmetrical=False (see the model field reference).
Now that you have set up your ManyToManyField to use your intermediary model (Membership, in this case),
you’re ready to start creating some many-to-many relationships. You do this by creating instances of the intermediate
model:
>>> ringo = Person.objects.create(name="Ringo Starr")
>>> paul = Person.objects.create(name="Paul McCartney")
>>> beatles = Group.objects.create(name="The Beatles")
>>> m1 = Membership(person=ringo, group=beatles,
... date_joined=date(1962, 8, 16),
... invite_reason= "Needed a new drummer.")
>>> m1.save()
>>> beatles.members.all()
[<Person: Ringo Starr>]
>>> ringo.group_set.all()
[<Group: The Beatles>]
>>> m2 = Membership.objects.create(person=paul, group=beatles,
... date_joined=date(1960, 8, 1),
... invite_reason= "Wanted to form a band.")
>>> beatles.members.all()
[<Person: Ringo Starr>, <Person: Paul McCartney>]
Unlike normal many-to-many fields, you can’t use add, create, or assignment (i.e., beatles.members =
[...]) to create relationships:
# THIS WILL NOT WORK
>>> beatles.members.add(john)
# NEITHER WILL THIS
>>> beatles.members.create(name="George Harrison")
3.2. Models and databases 85](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-89-2048.jpg)
![Django Documentation, Release 1.5.1
# AND NEITHER WILL THIS
>>> beatles.members = [john, paul, ringo, george]
Why? You can’t just create a relationship between a Person and a Group - you need to specify all the detail for the
relationship required by the Membership model. The simple add, create and assignment calls don’t provide a
way to specify this extra detail. As a result, they are disabled for many-to-many relationships that use an intermediate
model. The only way to create this type of relationship is to create instances of the intermediate model.
The remove() method is disabled for similar reasons. However, the clear() method can be used to remove all
many-to-many relationships for an instance:
# Beatles have broken up
>>> beatles.members.clear()
Once you have established the many-to-many relationships by creating instances of your intermediate model, you can
issue queries. Just as with normal many-to-many relationships, you can query using the attributes of the many-to-many-
related model:
# Find all the groups with a member whose name starts with ’Paul’
>>> Group.objects.filter(members__name__startswith=’Paul’)
[<Group: The Beatles>]
As you are using an intermediate model, you can also query on its attributes:
# Find all the members of the Beatles that joined after 1 Jan 1961
>>> Person.objects.filter(
... group__name=’The Beatles’,
... membership__date_joined__gt=date(1961,1,1))
[<Person: Ringo Starr]
If you need to access a membership’s information you may do so by directly querying the Membership model:
>>> ringos_membership = Membership.objects.get(group=beatles, person=ringo)
>>> ringos_membership.date_joined
datetime.date(1962, 8, 16)
>>> ringos_membership.invite_reason
u’Needed a new drummer.’
Another way to access the same information is by querying the many-to-many reverse relationship from a Person
object:
>>> ringos_membership = ringo.membership_set.get(group=beatles)
>>> ringos_membership.date_joined
datetime.date(1962, 8, 16)
>>> ringos_membership.invite_reason
u’Needed a new drummer.’
One-to-one relationships To define a one-to-one relationship, use OneToOneField. You use it just like any other
Field type: by including it as a class attribute of your model.
This is most useful on the primary key of an object when that object “extends” another object in some way.
OneToOneField requires a positional argument: the class to which the model is related.
For example, if you were building a database of “places”, you would build pretty standard stuff such as address, phone
number, etc. in the database. Then, if you wanted to build a database of restaurants on top of the places, instead of
repeating yourself and replicating those fields in the Restaurant model, you could make Restaurant have a
OneToOneField to Place (because a restaurant “is a” place; in fact, to handle this you’d typically use inheritance,
which involves an implicit one-to-one relation).
86 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-90-2048.jpg)
![Django Documentation, Release 1.5.1
Meta options
Give your model metadata by using an inner class Meta, like so:
class Ox(models.Model):
horn_length = models.IntegerField()
class Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
Model metadata is “anything that’s not a field”, such as ordering options (ordering), database table name
(db_table), or human-readable singular and plural names (verbose_name and verbose_name_plural).
None are required, and adding class Meta to a model is completely optional.
A complete list of all possible Meta options can be found in the model option reference.
Model methods
Define custom methods on a model to add custom “row-level” functionality to your objects. Whereas Manager
methods are intended to do “table-wide” things, model methods should act on a particular model instance.
This is a valuable technique for keeping business logic in one place – the model.
For example, this model has a few custom methods:
from django.contrib.localflavor.us.models import USStateField
class Person(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
birth_date = models.DateField()
address = models.CharField(max_length=100)
city = models.CharField(max_length=50)
state = USStateField() # Yes, this is America-centric...
def baby_boomer_status(self):
"Returns the person’s baby-boomer status."
import datetime
if self.birth_date < datetime.date(1945, 8, 1):
return "Pre-boomer"
elif self.birth_date < datetime.date(1965, 1, 1):
return "Baby boomer"
else:
return "Post-boomer"
def is_midwestern(self):
"Returns True if this person is from the Midwest."
return self.state in (’IL’, ’WI’, ’MI’, ’IN’, ’OH’, ’IA’, ’MO’)
def _get_full_name(self):
"Returns the person’s full name."
return ’%s %s’ % (self.first_name, self.last_name)
full_name = property(_get_full_name)
The last method in this example is a property.
The model instance reference has a complete list of methods automatically given to each model. You can override
most of these – see overriding predefined model methods, below – but there are a couple that you’ll almost always
88 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-92-2048.jpg)
![Django Documentation, Release 1.5.1
Meta inheritance When an abstract base class is created, Django makes any Meta inner class you declared in the
base class available as an attribute. If a child class does not declare its own Meta class, it will inherit the parent’s Meta.
If the child wants to extend the parent’s Meta class, it can subclass it. For example:
class CommonInfo(models.Model):
...
class Meta:
abstract = True
ordering = [’name’]
class Student(CommonInfo):
...
class Meta(CommonInfo.Meta):
db_table = ’student_info’
Django does make one adjustment to the Meta class of an abstract base class: before installing the Meta attribute,
it sets abstract=False. This means that children of abstract base classes don’t automatically become abstract
classes themselves. Of course, you can make an abstract base class that inherits from another abstract base class. You
just need to remember to explicitly set abstract=True each time.
Some attributes won’t make sense to include in the Meta class of an abstract base class. For example, including
db_table would mean that all the child classes (the ones that don’t specify their own Meta) would use the same
database table, which is almost certainly not what you want.
Be careful with related_name If you are using the related_name attribute on a ForeignKey or
ManyToManyField, you must always specify a unique reverse name for the field. This would normally cause a
problem in abstract base classes, since the fields on this class are included into each of the child classes, with exactly
the same values for the attributes (including related_name) each time.
To work around this problem, when you are using related_name in an abstract base class (only), part of the name
should contain ’%(app_label)s’ and ’%(class)s’.
• ’%(class)s’ is replaced by the lower-cased name of the child class that the field is used in.
• ’%(app_label)s’ is replaced by the lower-cased name of the app the child class is contained within. Each
installed application name must be unique and the model class names within each app must also be unique,
therefore the resulting name will end up being different.
For example, given an app common/models.py:
class Base(models.Model):
m2m = models.ManyToManyField(OtherModel, related_name="%(app_label)s_%(class)s_related")
class Meta:
abstract = True
class ChildA(Base):
pass
class ChildB(Base):
pass
Along with another app rare/models.py:
from common.models import Base
class ChildB(Base):
pass
3.2. Models and databases 91](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-95-2048.jpg)
![Django Documentation, Release 1.5.1
# Remove parent’s ordering effect
ordering = []
Inheritance and reverse relations Because multi-table inheritance uses an implicit OneToOneField to link the
child and the parent, it’s possible to move from the parent down to the child, as in the above example. However, this
uses up the name that is the default related_name value for ForeignKey and ManyToManyField relations.
If you are putting those types of relations on a subclass of another model, you must specify the related_name
attribute on each such field. If you forget, Django will raise an error when you run validate or syncdb.
For example, using the above Place class again, let’s create another subclass with a ManyToManyField:
class Supplier(Place):
# Must specify related_name on all relations.
customers = models.ManyToManyField(Restaurant, related_name=’provider’)
Specifying the parent link field As mentioned, Django will automatically create a OneToOneField linking your
child class back any non-abstract parent models. If you want to control the name of the attribute linking back to the
parent, you can create your own OneToOneField and set parent_link=True to indicate that your field is the
link back to the parent class.
Proxy models
When using multi-table inheritance, a new database table is created for each subclass of a model. This is usually the
desired behavior, since the subclass needs a place to store any additional data fields that are not present on the base
class. Sometimes, however, you only want to change the Python behavior of a model – perhaps to change the default
manager, or add a new method.
This is what proxy model inheritance is for: creating a proxy for the original model. You can create, delete and update
instances of the proxy model and all the data will be saved as if you were using the original (non-proxied) model. The
difference is that you can change things like the default model ordering or the default manager in the proxy, without
having to alter the original.
Proxy models are declared like normal models. You tell Django that it’s a proxy model by setting the proxy attribute
of the Meta class to True.
For example, suppose you want to add a method to the Person model described above. You can do it like this:
class MyPerson(Person):
class Meta:
proxy = True
def do_something(self):
...
The MyPerson class operates on the same database table as its parent Person class. In particular, any new instances
of Person will also be accessible through MyPerson, and vice-versa:
>>> p = Person.objects.create(first_name="foobar")
>>> MyPerson.objects.get(first_name="foobar")
<MyPerson: foobar>
You could also use a proxy model to define a different default ordering on a model. You might not always want to
order the Person model, but regularly order by the last_name attribute when you use the proxy. This is easy:
3.2. Models and databases 93](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-97-2048.jpg)
![Django Documentation, Release 1.5.1
class OrderedPerson(Person):
class Meta:
ordering = ["last_name"]
proxy = True
Now normal Person queries will be unordered and OrderedPerson queries will be ordered by last_name.
QuerySets still return the model that was requested There is no way to have Django return, say, a MyPerson
object whenever you query for Person objects. A queryset for Person objects will return those types of objects.
The whole point of proxy objects is that code relying on the original Person will use those and your own code can
use the extensions you included (that no other code is relying on anyway). It is not a way to replace the Person (or
any other) model everywhere with something of your own creation.
Base class restrictions A proxy model must inherit from exactly one non-abstract model class. You can’t inherit
from multiple non-abstract models as the proxy model doesn’t provide any connection between the rows in the different
database tables. A proxy model can inherit from any number of abstract model classes, providing they do not define
any model fields.
Proxy models inherit any Meta options that they don’t define from their non-abstract model parent (the model they
are proxying for).
Proxy model managers If you don’t specify any model managers on a proxy model, it inherits the managers from
its model parents. If you define a manager on the proxy model, it will become the default, although any managers
defined on the parent classes will still be available.
Continuing our example from above, you could change the default manager used when you query the Person model
like this:
class NewManager(models.Manager):
...
class MyPerson(Person):
objects = NewManager()
class Meta:
proxy = True
If you wanted to add a new manager to the Proxy, without replacing the existing default, you can use the techniques
described in the custom manager documentation: create a base class containing the new managers and inherit that
after the primary base class:
# Create an abstract class for the new manager.
class ExtraManagers(models.Model):
secondary = NewManager()
class Meta:
abstract = True
class MyPerson(Person, ExtraManagers):
class Meta:
proxy = True
You probably won’t need to do this very often, but, when you do, it’s possible.
94 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-98-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Entry.objects.filter(
... headline__startswith=’What’
... ).exclude(
... pub_date__gte=datetime.date.today()
... ).filter(
... pub_date__gte=datetime(2005, 1, 30)
... )
This takes the initial QuerySet of all entries in the database, adds a filter, then an exclusion, then another filter. The
final result is a QuerySet containing all entries with a headline that starts with “What”, that were published between
January 30, 2005, and the current day.
Filtered QuerySets are unique Each time you refine a QuerySet, you get a brand-new QuerySet that is in no
way bound to the previous QuerySet. Each refinement creates a separate and distinct QuerySet that can be stored,
used and reused.
Example:
>> q1 = Entry.objects.filter(headline__startswith="What")
>> q2 = q1.exclude(pub_date__gte=datetime.date.today())
>> q3 = q1.filter(pub_date__gte=datetime.date.today())
These three QuerySets are separate. The first is a base QuerySet containing all entries that contain a headline
starting with “What”. The second is a subset of the first, with an additional criteria that excludes records whose
pub_date is greater than now. The third is a subset of the first, with an additional criteria that selects only the
records whose pub_date is greater than now. The initial QuerySet (q1) is unaffected by the refinement process.
QuerySets are lazy QuerySets are lazy – the act of creating a QuerySet doesn’t involve any database activity.
You can stack filters together all day long, and Django won’t actually run the query until the QuerySet is evaluated.
Take a look at this example:
>>> q = Entry.objects.filter(headline__startswith="What")
>>> q = q.filter(pub_date__lte=datetime.date.today())
>>> q = q.exclude(body_text__icontains="food")
>>> print(q)
Though this looks like three database hits, in fact it hits the database only once, at the last line (print(q)). In
general, the results of a QuerySet aren’t fetched from the database until you “ask” for them. When you do, the
QuerySet is evaluated by accessing the database. For more details on exactly when evaluation takes place, see
When QuerySets are evaluated.
Retrieving a single object with get
filter() will always give you a QuerySet, even if only a single object matches the query - in this case, it will be
a QuerySet containing a single element.
If you know there is only one object that matches your query, you can use the get() method on a Manager which
returns the object directly:
>>> one_entry = Entry.objects.get(pk=1)
You can use any query expression with get(), just like with filter() - again, see Field lookups below.
Note that there is a difference between using get(), and using filter() with a slice of [0]. If there are no results
that match the query, get() will raise a DoesNotExist exception. This exception is an attribute of the model
3.2. Models and databases 99](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-103-2048.jpg)
![Django Documentation, Release 1.5.1
class that the query is being performed on - so in the code above, if there is no Entry object with a primary key of 1,
Django will raise Entry.DoesNotExist.
Similarly, Django will complain if more than one item matches the get() query. In this case, it will raise
MultipleObjectsReturned, which again is an attribute of the model class itself.
Other QuerySet methods
Most of the time you’ll use all(), get(), filter() and exclude() when you need to look up objects from
the database. However, that’s far from all there is; see the QuerySet API Reference for a complete list of all the various
QuerySet methods.
Limiting QuerySets
Use a subset of Python’s array-slicing syntax to limit your QuerySet to a certain number of results. This is the
equivalent of SQL’s LIMIT and OFFSET clauses.
For example, this returns the first 5 objects (LIMIT 5):
>>> Entry.objects.all()[:5]
This returns the sixth through tenth objects (OFFSET 5 LIMIT 5):
>>> Entry.objects.all()[5:10]
Negative indexing (i.e. Entry.objects.all()[-1]) is not supported.
Generally, slicing a QuerySet returns a new QuerySet – it doesn’t evaluate the query. An exception is if you use
the “step” parameter of Python slice syntax. For example, this would actually execute the query in order to return a
list of every second object of the first 10:
>>> Entry.objects.all()[:10:2]
To retrieve a single object rather than a list (e.g. SELECT foo FROM bar LIMIT 1), use a simple index instead
of a slice. For example, this returns the first Entry in the database, after ordering entries alphabetically by headline:
>>> Entry.objects.order_by(’headline’)[0]
This is roughly equivalent to:
>>> Entry.objects.order_by(’headline’)[0:1].get()
Note, however, that the first of these will raise IndexError while the second will raise DoesNotExist if no
objects match the given criteria. See get() for more details.
Field lookups
Field lookups are how you specify the meat of an SQL WHERE clause. They’re specified as keyword arguments to the
QuerySet methods filter(), exclude() and get().
Basic lookups keyword arguments take the form field__lookuptype=value. (That’s a double-underscore).
For example:
>>> Entry.objects.filter(pub_date__lte=’2006-01-01’)
translates (roughly) into the following SQL:
100 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-104-2048.jpg)
![Django Documentation, Release 1.5.1
New in version 1.5: .bitand() and .bitor() The F() objects now support bitwise operations by .bitand()
and .bitor(), for example:
>>> F(’somefield’).bitand(16)
Changed in version 1.5: The previously undocumented operators & and | no longer produce bitwise operations, use
.bitand() and .bitor() instead.
The pk lookup shortcut
For convenience, Django provides a pk lookup shortcut, which stands for “primary key”.
In the example Blog model, the primary key is the id field, so these three statements are equivalent:
>>> Blog.objects.get(id__exact=14) # Explicit form
>>> Blog.objects.get(id=14) # __exact is implied
>>> Blog.objects.get(pk=14) # pk implies id__exact
The use of pk isn’t limited to __exact queries – any query term can be combined with pk to perform a query on the
primary key of a model:
# Get blogs entries with id 1, 4 and 7
>>> Blog.objects.filter(pk__in=[1,4,7])
# Get all blog entries with id > 14
>>> Blog.objects.filter(pk__gt=14)
pk lookups also work across joins. For example, these three statements are equivalent:
>>> Entry.objects.filter(blog__id__exact=3) # Explicit form
>>> Entry.objects.filter(blog__id=3) # __exact is implied
>>> Entry.objects.filter(blog__pk=3) # __pk implies __id__exact
Escaping percent signs and underscores in LIKE statements
The field lookups that equate to LIKE SQL statements (iexact, contains, icontains, startswith,
istartswith, endswith and iendswith) will automatically escape the two special characters used in LIKE
statements – the percent sign and the underscore. (In a LIKE statement, the percent sign signifies a multiple-character
wildcard and the underscore signifies a single-character wildcard.)
This means things should work intuitively, so the abstraction doesn’t leak. For example, to retrieve all the entries that
contain a percent sign, just use the percent sign as any other character:
>>> Entry.objects.filter(headline__contains=’%’)
Django takes care of the quoting for you; the resulting SQL will look something like this:
SELECT ... WHERE headline LIKE ’%%%’;
Same goes for underscores. Both percentage signs and underscores are handled for you transparently.
Caching and QuerySets
Each QuerySet contains a cache, to minimize database access. It’s important to understand how it works, in order
to write the most efficient code.
104 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-108-2048.jpg)
![Django Documentation, Release 1.5.1
In a newly created QuerySet, the cache is empty. The first time a QuerySet is evaluated – and, hence, a database
query happens – Django saves the query results in the QuerySet‘s cache and returns the results that have been
explicitly requested (e.g., the next element, if the QuerySet is being iterated over). Subsequent evaluations of the
QuerySet reuse the cached results.
Keep this caching behavior in mind, because it may bite you if you don’t use your QuerySets correctly. For example,
the following will create two QuerySets, evaluate them, and throw them away:
>>> print([e.headline for e in Entry.objects.all()])
>>> print([e.pub_date for e in Entry.objects.all()])
That means the same database query will be executed twice, effectively doubling your database load. Also, there’s
a possibility the two lists may not include the same database records, because an Entry may have been added or
deleted in the split second between the two requests.
To avoid this problem, simply save the QuerySet and reuse it:
>>> queryset = Entry.objects.all()
>>> print([p.headline for p in queryset]) # Evaluate the query set.
>>> print([p.pub_date for p in queryset]) # Re-use the cache from the evaluation.
Complex lookups with Q objects
class Q
Keyword argument queries – in filter(), etc. – are “AND”ed together. If you need to execute more complex
queries (for example, queries with OR statements), you can use Q objects.
A Q object (django.db.models.Q) is an object used to encapsulate a collection of keyword arguments. These
keyword arguments are specified as in “Field lookups” above.
For example, this Q object encapsulates a single LIKE query:
from django.db.models import Q
Q(question__startswith=’What’)
Q objects can be combined using the & and | operators. When an operator is used on two Q objects, it yields a new Q
object.
For example, this statement yields a single Q object that represents the “OR” of two "question__startswith"
queries:
Q(question__startswith=’Who’) | Q(question__startswith=’What’)
This is equivalent to the following SQL WHERE clause:
WHERE question LIKE ’Who%’ OR question LIKE ’What%’
You can compose statements of arbitrary complexity by combining Q objects with the & and | operators and use
parenthetical grouping. Also, Q objects can be negated using the ~ operator, allowing for combined lookups that
combine both a normal query and a negated (NOT) query:
Q(question__startswith=’Who’) | ~Q(pub_date__year=2005)
Each lookup function that takes keyword-arguments (e.g. filter(), exclude(), get()) can also be passed
one or more Q objects as positional (not-named) arguments. If you provide multiple Q object arguments to a lookup
function, the arguments will be “AND”ed together. For example:
3.2. Models and databases 105](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-109-2048.jpg)
![Django Documentation, Release 1.5.1
Entry.objects.filter(pub_date__year=2005).delete()
Keep in mind that this will, whenever possible, be executed purely in SQL, and so the delete() methods of in-dividual
object instances will not necessarily be called during the process. If you’ve provided a custom delete()
method on a model class and want to ensure that it is called, you will need to “manually” delete instances of that model
(e.g., by iterating over a QuerySet and calling delete() on each object individually) rather than using the bulk
delete() method of a QuerySet.
When Django deletes an object, by default it emulates the behavior of the SQL constraint ON DELETE CASCADE –
in other words, any objects which had foreign keys pointing at the object to be deleted will be deleted along with it.
For example:
b = Blog.objects.get(pk=1)
# This will delete the Blog and all of its Entry objects.
b.delete()
This cascade behavior is customizable via the on_delete argument to the ForeignKey.
Note that delete() is the only QuerySet method that is not exposed on a Manager itself. This is a safety
mechanism to prevent you from accidentally requesting Entry.objects.delete(), and deleting all the entries.
If you do want to delete all the objects, then you have to explicitly request a complete query set:
Entry.objects.all().delete()
Copying model instances
Although there is no built-in method for copying model instances, it is possible to easily create new instance with all
fields’ values copied. In the simplest case, you can just set pk to None. Using our blog example:
blog = Blog(name=’My blog’, tagline=’Blogging is easy’)
blog.save() # blog.pk == 1
blog.pk = None
blog.save() # blog.pk == 2
Things get more complicated if you use inheritance. Consider a subclass of Blog:
class ThemeBlog(Blog):
theme = models.CharField(max_length=200)
django_blog = ThemeBlog(name=’Django’, tagline=’Django is easy’, theme=’python’)
django_blog.save() # django_blog.pk == 3
Due to how inheritance works, you have to set both pk and id to None:
django_blog.pk = None
django_blog.id = None
django_blog.save() # django_blog.pk == 4
This process does not copy related objects. If you want to copy relations, you have to write a little bit more code. In
our example, Entry has a many to many field to Author:
entry = Entry.objects.all()[0] # some previous entry
old_authors = entry.authors.all()
entry.pk = None
entry.save()
entry.authors = old_authors # saves new many2many relations
3.2. Models and databases 107](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-111-2048.jpg)
![Django Documentation, Release 1.5.1
You can override the FOO_set name by setting the related_name parameter in the ForeignKey()
definition. For example, if the Entry model was altered to blog = ForeignKey(Blog,
related_name=’entries’), the above example code would look like this:
>>> b = Blog.objects.get(id=1)
>>> b.entries.all() # Returns all Entry objects related to Blog.
# b.entries is a Manager that returns QuerySets.
>>> b.entries.filter(headline__contains=’Lennon’)
>>> b.entries.count()
You cannot access a reverse ForeignKey Manager from the class; it must be accessed from an instance:
>>> Blog.entry_set
Traceback:
...
AttributeError: "Manager must be accessed via instance".
In addition to the QuerySet methods defined in “Retrieving objects” above, the ForeignKey Manager has ad-ditional
methods used to handle the set of related objects. A synopsis of each is below, and complete details can be
found in the related objects reference.
add(obj1, obj2, ...) Adds the specified model objects to the related object set.
create(**kwargs) Creates a new object, saves it and puts it in the related object set. Returns the newly created
object.
remove(obj1, obj2, ...) Removes the specified model objects from the related object set.
clear() Removes all objects from the related object set.
To assign the members of a related set in one fell swoop, just assign to it from any iterable object. The iterable can
contain object instances, or just a list of primary key values. For example:
b = Blog.objects.get(id=1)
b.entry_set = [e1, e2]
In this example, e1 and e2 can be full Entry instances, or integer primary key values.
If the clear() method is available, any pre-existing objects will be removed from the entry_set before all
objects in the iterable (in this case, a list) are added to the set. If the clear() method is not available, all objects in
the iterable will be added without removing any existing elements.
Each “reverse” operation described in this section has an immediate effect on the database. Every addition, creation
and deletion is immediately and automatically saved to the database.
Many-to-many relationships
Both ends of a many-to-many relationship get automatic API access to the other end. The API works just as a
“backward” one-to-many relationship, above.
The only difference is in the attribute naming: The model that defines the ManyToManyField uses the attribute
name of that field itself, whereas the “reverse” model uses the lowercased model name of the original model, plus
’_set’ (just like reverse one-to-many relationships).
An example makes this easier to understand:
e = Entry.objects.get(id=3)
e.authors.all() # Returns all Author objects for this Entry.
e.authors.count()
e.authors.filter(name__contains=’John’)
110 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-114-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Book.objects.all().aggregate(Avg(’price’))
{’price__avg’: 34.35}
# Max price across all books.
>>> from django.db.models import Max
>>> Book.objects.all().aggregate(Max(’price’))
{’price__max’: Decimal(’81.20’)}
# All the following queries involve traversing the Book<->Publisher
# many-to-many relationship backward
# Each publisher, each with a count of books as a "num_books" attribute.
>>> from django.db.models import Count
>>> pubs = Publisher.objects.annotate(num_books=Count(’book’))
>>> pubs
[<Publisher BaloneyPress>, <Publisher SalamiPress>, ...]
>>> pubs[0].num_books
73
# The top 5 publishers, in order by number of books.
>>> pubs = Publisher.objects.annotate(num_books=Count(’book’)).order_by(’-num_books’)[:5]
>>> pubs[0].num_books
1323
Generating aggregates over a QuerySet
Django provides two ways to generate aggregates. The first way is to generate summary values over an entire
QuerySet. For example, say you wanted to calculate the average price of all books available for sale. Django’s
query syntax provides a means for describing the set of all books:
>>> Book.objects.all()
What we need is a way to calculate summary values over the objects that belong to this QuerySet. This is done by
appending an aggregate() clause onto the QuerySet:
>>> from django.db.models import Avg
>>> Book.objects.all().aggregate(Avg(’price’))
{’price__avg’: 34.35}
The all() is redundant in this example, so this could be simplified to:
>>> Book.objects.aggregate(Avg(’price’))
{’price__avg’: 34.35}
The argument to the aggregate() clause describes the aggregate value that we want to compute - in this case, the
average of the price field on the Book model. A list of the aggregate functions that are available can be found in the
QuerySet reference.
aggregate() is a terminal clause for a QuerySet that, when invoked, returns a dictionary of name-value pairs.
The name is an identifier for the aggregate value; the value is the computed aggregate. The name is automatically
generated from the name of the field and the aggregate function. If you want to manually specify a name for the
aggregate value, you can do so by providing that name when you specify the aggregate clause:
>>> Book.objects.aggregate(average_price=Avg(’price’))
{’average_price’: 34.35}
If you want to generate more than one aggregate, you just add another argument to the aggregate() clause. So, if
we also wanted to know the maximum and minimum price of all books, we would issue the query:
3.2. Models and databases 113](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-117-2048.jpg)
![Django Documentation, Release 1.5.1
>>> from django.db.models import Avg, Max, Min, Count
>>> Book.objects.aggregate(Avg(’price’), Max(’price’), Min(’price’))
{’price__avg’: 34.35, ’price__max’: Decimal(’81.20’), ’price__min’: Decimal(’12.99’)}
Generating aggregates for each item in a QuerySet
The second way to generate summary values is to generate an independent summary for each object in a QuerySet.
For example, if you are retrieving a list of books, you may want to know how many authors contributed to each book.
Each Book has a many-to-many relationship with the Author; we want to summarize this relationship for each book
in the QuerySet.
Per-object summaries can be generated using the annotate() clause. When an annotate() clause is specified,
each object in the QuerySet will be annotated with the specified values.
The syntax for these annotations is identical to that used for the aggregate() clause. Each argument to
annotate() describes an aggregate that is to be calculated. For example, to annotate books with the number of
authors:
# Build an annotated queryset
>>> q = Book.objects.annotate(Count(’authors’))
# Interrogate the first object in the queryset
>>> q[0]
<Book: The Definitive Guide to Django>
>>> q[0].authors__count
2
# Interrogate the second object in the queryset
>>> q[1]
<Book: Practical Django Projects>
>>> q[1].authors__count
1
As with aggregate(), the name for the annotation is automatically derived from the name of the aggregate function
and the name of the field being aggregated. You can override this default name by providing an alias when you specify
the annotation:
>>> q = Book.objects.annotate(num_authors=Count(’authors’))
>>> q[0].num_authors
2
>>> q[1].num_authors
1
Unlike aggregate(), annotate() is not a terminal clause. The output of the annotate() clause is
a QuerySet; this QuerySet can be modified using any other QuerySet operation, including filter(),
order_by(), or even additional calls to annotate().
Joins and aggregates
So far, we have dealt with aggregates over fields that belong to the model being queried. However, sometimes the
value you want to aggregate will belong to a model that is related to the model you are querying.
When specifying the field to be aggregated in an aggregate function, Django will allow you to use the same double
underscore notation that is used when referring to related fields in filters. Django will then handle any table joins that
are required to retrieve and aggregate the related value.
For example, to find the price range of books offered in each store, you could use the annotation:
114 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-118-2048.jpg)
![Django Documentation, Release 1.5.1
returned in the result set, the method for evaluating annotations is slightly different. Instead of returning an annotated
result for each result in the original QuerySet, the original results are grouped according to the unique combinations
of the fields specified in the values() clause. An annotation is then provided for each unique group; the annotation
is computed over all members of the group.
For example, consider an author query that attempts to find out the average rating of books written by each author:
>>> Author.objects.annotate(average_rating=Avg(’book__rating’))
This will return one result for each author in the database, annotated with their average book rating.
However, the result will be slightly different if you use a values() clause:
>>> Author.objects.values(’name’).annotate(average_rating=Avg(’book__rating’))
In this example, the authors will be grouped by name, so you will only get an annotated result for each unique author
name. This means if you have two authors with the same name, their results will be merged into a single result in the
output of the query; the average will be computed as the average over the books written by both authors.
Order of annotate() and values() clauses As with the filter() clause, the order in which annotate()
and values() clauses are applied to a query is significant. If the values() clause precedes the annotate(),
the annotation will be computed using the grouping described by the values() clause.
However, if the annotate() clause precedes the values() clause, the annotations will be generated over the
entire query set. In this case, the values() clause only constrains the fields that are generated on output.
For example, if we reverse the order of the values() and annotate() clause from our previous example:
>>> Author.objects.annotate(average_rating=Avg(’book__rating’)).values(’name’, ’average_rating’)
This will now yield one unique result for each author; however, only the author’s name and the average_rating
annotation will be returned in the output data.
You should also note that average_rating has been explicitly included in the list of values to be returned. This is
required because of the ordering of the values() and annotate() clause.
If the values() clause precedes the annotate() clause, any annotations will be automatically added to the result
set. However, if the values() clause is applied after the annotate() clause, you need to explicitly include the
aggregate column.
Interaction with default ordering or order_by() Fields that are mentioned in the order_by() part of a
queryset (or which are used in the default ordering on a model) are used when selecting the output data, even if they
are not otherwise specified in the values() call. These extra fields are used to group “like” results together and they
can make otherwise identical result rows appear to be separate. This shows up, particularly, when counting things.
By way of example, suppose you have a model like this:
class Item(models.Model):
name = models.CharField(max_length=10)
data = models.IntegerField()
class Meta:
ordering = ["name"]
The important part here is the default ordering on the name field. If you want to count how many times each distinct
data value appears, you might try this:
# Warning: not quite correct!
Item.objects.values("data").annotate(Count("id"))
3.2. Models and databases 117](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-121-2048.jpg)
![Django Documentation, Release 1.5.1
Custom Managers
You can use a custom Manager in a particular model by extending the base Manager class and instantiating your
custom Manager in your model.
There are two reasons you might want to customize a Manager: to add extra Manager methods, and/or to modify
the initial QuerySet the Manager returns.
Adding extra Manager methods
Adding extra Manager methods is the preferred way to add “table-level” functionality to your models. (For “row-level”
functionality – i.e., functions that act on a single instance of a model object – use Model methods, not custom
Manager methods.)
A custom Manager method can return anything you want. It doesn’t have to return a QuerySet.
For example, this custom Manager offers a method with_counts(), which returns a list of all OpinionPoll
objects, each with an extra num_responses attribute that is the result of an aggregate query:
class PollManager(models.Manager):
def with_counts(self):
from django.db import connection
cursor = connection.cursor()
cursor.execute("""
SELECT p.id, p.question, p.poll_date, COUNT(*)
FROM polls_opinionpoll p, polls_response r
WHERE p.id = r.poll_id
GROUP BY p.id, p.question, p.poll_date
ORDER BY p.poll_date DESC""")
result_list = []
for row in cursor.fetchall():
p = self.model(id=row[0], question=row[1], poll_date=row[2])
p.num_responses = row[3]
result_list.append(p)
return result_list
class OpinionPoll(models.Model):
question = models.CharField(max_length=200)
poll_date = models.DateField()
objects = PollManager()
class Response(models.Model):
poll = models.ForeignKey(OpinionPoll)
person_name = models.CharField(max_length=50)
response = models.TextField()
With this example, you’d use OpinionPoll.objects.with_counts() to return that list of OpinionPoll
objects with num_responses attributes.
Another thing to note about this example is that Manager methods can access self.model to get the model class
to which they’re attached.
Modifying initial Manager QuerySets
A Manager‘s base QuerySet returns all objects in the system. For example, using this model:
3.2. Models and databases 119](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-123-2048.jpg)
![Django Documentation, Release 1.5.1
For more details check out the documentation for the db_table option, which also lets you manually set the database
table name.
Warning: No checking is done on the SQL statement that is passed in to .raw(). Django expects that the
statement will return a set of rows from the database, but does nothing to enforce that. If the query does not return
rows, a (possibly cryptic) error will result.
Mapping query fields to model fields
raw() automatically maps fields in the query to fields on the model.
The order of fields in your query doesn’t matter. In other words, both of the following queries work identically:
>>> Person.objects.raw(’SELECT id, first_name, last_name, birth_date FROM myapp_person’)
...
>>> Person.objects.raw(’SELECT last_name, birth_date, first_name, id FROM myapp_person’)
...
Matching is done by name. This means that you can use SQL’s AS clauses to map fields in the query to model fields.
So if you had some other table that had Person data in it, you could easily map it into Person instances:
>>> Person.objects.raw(’’’SELECT first AS first_name,
... last AS last_name,
... bd AS birth_date,
... pk as id,
... FROM some_other_table’’’)
As long as the names match, the model instances will be created correctly.
Alternatively, you can map fields in the query to model fields using the translations argument to raw(). This
is a dictionary mapping names of fields in the query to names of fields on the model. For example, the above query
could also be written:
>>> name_map = {’first’: ’first_name’, ’last’: ’last_name’, ’bd’: ’birth_date’, ’pk’: ’id’}
>>> Person.objects.raw(’SELECT * FROM some_other_table’, translations=name_map)
Index lookups
raw() supports indexing, so if you need only the first result you can write:
>>> first_person = Person.objects.raw(’SELECT * from myapp_person’)[0]
However, the indexing and slicing are not performed at the database level. If you have a big amount of Person
objects in your database, it is more efficient to limit the query at the SQL level:
>>> first_person = Person.objects.raw(’SELECT * from myapp_person LIMIT 1’)[0]
Deferring model fields
Fields may also be left out:
>>> people = Person.objects.raw(’SELECT id, first_name FROM myapp_person’)
3.2. Models and databases 125](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-129-2048.jpg)
![Django Documentation, Release 1.5.1
The Person objects returned by this query will be deferred model instances (see defer()). This means that the
fields that are omitted from the query will be loaded on demand. For example:
>>> for p in Person.objects.raw(’SELECT id, first_name FROM myapp_person’):
... print(p.first_name, # This will be retrieved by the original query
... p.last_name) # This will be retrieved on demand
...
John Smith
Jane Jones
From outward appearances, this looks like the query has retrieved both the first name and last name. However, this
example actually issued 3 queries. Only the first names were retrieved by the raw() query – the last names were both
retrieved on demand when they were printed.
There is only one field that you can’t leave out - the primary key field. Django uses the primary key to identify model
instances, so it must always be included in a raw query. An InvalidQuery exception will be raised if you forget to
include the primary key.
Adding annotations
You can also execute queries containing fields that aren’t defined on the model. For example, we could use Post-greSQL’s
age() function to get a list of people with their ages calculated by the database:
>>> people = Person.objects.raw(’SELECT *, age(birth_date) AS age FROM myapp_person’)
>>> for p in people:
... print("%s is %s." % (p.first_name, p.age))
John is 37.
Jane is 42.
...
Passing parameters into raw()
If you need to perform parameterized queries, you can use the params argument to raw():
>>> lname = ’Doe’
>>> Person.objects.raw(’SELECT * FROM myapp_person WHERE last_name = %s’, [lname])
params is a list of parameters. You’ll use %s placeholders in the query string (regardless of your database engine);
they’ll be replaced with parameters from the params list.
Warning: Do not use string formatting on raw queries!
It’s tempting to write the above query as:
>>> query = ’SELECT * FROM myapp_person WHERE last_name = %s’ % lname
>>> Person.objects.raw(query)
Don’t.
Using the params list completely protects you from SQL injection attacks, a common exploit where attackers
inject arbitrary SQL into your database. If you use string interpolation, sooner or later you’ll fall victim to SQL
injection. As long as you remember to always use the params list you’ll be protected.
126 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-130-2048.jpg)
![Django Documentation, Release 1.5.1
Executing custom SQL directly
Sometimes even Manager.raw() isn’t quite enough: you might need to perform queries that don’t map cleanly to
models, or directly execute UPDATE, INSERT, or DELETE queries.
In these cases, you can always access the database directly, routing around the model layer entirely.
The object django.db.connection represents the default database connection, and
django.db.transaction represents the default database transaction. To use the database connection,
call connection.cursor() to get a cursor object. Then, call cursor.execute(sql, [params]) to
execute the SQL and cursor.fetchone() or cursor.fetchall() to return the resulting rows. After
performing a data changing operation, you should then call transaction.commit_unless_managed() to
ensure your changes are committed to the database. If your query is purely a data retrieval operation, no commit is
required. For example:
def my_custom_sql():
from django.db import connection, transaction
cursor = connection.cursor()
# Data modifying operation - commit required
cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz])
transaction.commit_unless_managed()
# Data retrieval operation - no commit required
cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz])
row = cursor.fetchone()
return row
If you are using more than one database, you can use django.db.connections to obtain the connection (and
cursor) for a specific database. django.db.connections is a dictionary-like object that allows you to retrieve a
specific connection using its alias:
from django.db import connections
cursor = connections[’my_db_alias’].cursor()
# Your code here...
transaction.commit_unless_managed(using=’my_db_alias’)
By default, the Python DB API will return results without their field names, which means you end up with a list of
values, rather than a dict. At a small performance cost, you can return results as a dict by using something like
this:
def dictfetchall(cursor):
"Returns all rows from a cursor as a dict"
desc = cursor.description
return [
dict(zip([col[0] for col in desc], row))
for row in cursor.fetchall()
]
Here is an example of the difference between the two:
>>> cursor.execute("SELECT id, parent_id from test LIMIT 2");
>>> cursor.fetchall()
((54360982L, None), (54360880L, None))
>>> cursor.execute("SELECT id, parent_id from test LIMIT 2");
>>> dictfetchall(cursor)
[{’parent_id’: None, ’id’: 54360982L}, {’parent_id’: None, ’id’: 54360880L}]
3.2. Models and databases 127](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-131-2048.jpg)
![Django Documentation, Release 1.5.1
Now we’ll need to handle routing. First we want a router that knows to send queries for the auth app to auth_db:
class AuthRouter(object):
"""
A router to control all database operations on models in the
auth application.
"""
def db_for_read(self, model, **hints):
"""
Attempts to read auth models go to auth_db.
"""
if model._meta.app_label == ’auth’:
return ’auth_db’
return None
def db_for_write(self, model, **hints):
"""
Attempts to write auth models go to auth_db.
"""
if model._meta.app_label == ’auth’:
return ’auth_db’
return None
def allow_relation(self, obj1, obj2, **hints):
"""
Allow relations if a model in the auth app is involved.
"""
if obj1._meta.app_label == ’auth’ or
obj2._meta.app_label == ’auth’:
return True
return None
def allow_syncdb(self, db, model):
"""
Make sure the auth app only appears in the ’auth_db’
database.
"""
if db == ’auth_db’:
return model._meta.app_label == ’auth’
elif model._meta.app_label == ’auth’:
return False
return None
And we also want a router that sends all other apps to the master/slave configuration, and randomly chooses a slave to
read from:
import random
class MasterSlaveRouter(object):
def db_for_read(self, model, **hints):
"""
Reads go to a randomly-chosen slave.
"""
return random.choice([’slave1’, ’slave2’])
def db_for_write(self, model, **hints):
"""
Writes always go to master.
"""
3.2. Models and databases 137](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-141-2048.jpg)
![Django Documentation, Release 1.5.1
return ’master’
def allow_relation(self, obj1, obj2, **hints):
"""
Relations between objects are allowed if both objects are
in the master/slave pool.
"""
db_list = (’master’, ’slave1’, ’slave2’)
if obj1.state.db in db_list and obj2.state.db in db_list:
return True
return None
def allow_syncdb(self, db, model):
"""
All non-auth models end up in this pool.
"""
return True
Finally, in the settings file, we add the following (substituting path.to. with the actual python path to the module(s)
where the routers are defined):
DATABASE_ROUTERS = [’path.to.AuthRouter’, ’path.to.MasterSlaveRouter’]
The order in which routers are processed is significant. Routers will be queried in the order the are
listed in the DATABASE_ROUTERS setting . In this example, the AuthRouter is processed before the
MasterSlaveRouter, and as a result, decisions concerning the models in auth are processed before
any other decision is made. If the DATABASE_ROUTERS setting listed the two routers in the other order,
MasterSlaveRouter.allow_syncdb() would be processed first. The catch-all nature of the MasterSlaveR-outer
implementation would mean that all models would be available on all databases.
With this setup installed, lets run some Django code:
>>> # This retrieval will be performed on the ’auth_db’ database
>>> fred = User.objects.get(username=’fred’)
>>> fred.first_name = ’Frederick’
>>> # This save will also be directed to ’auth_db’
>>> fred.save()
>>> # These retrieval will be randomly allocated to a slave database
>>> dna = Person.objects.get(name=’Douglas Adams’)
>>> # A new object has no database allocation when created
>>> mh = Book(title=’Mostly Harmless’)
>>> # This assignment will consult the router, and set mh onto
>>> # the same database as the author object
>>> mh.author = dna
>>> # This save will force the ’mh’ instance onto the master database...
>>> mh.save()
>>> # ... but if we re-retrieve the object, it will come back on a slave
>>> mh = Book.objects.get(title=’Mostly Harmless’)
138 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-142-2048.jpg)
![Django Documentation, Release 1.5.1
# on the ’other’ database.
return super(MultiDBTabularInline, self).formfield_for_manytomany(db_field, request=request, Once you’ve written your model admin definitions, they can be registered with any Admin instance:
from django.contrib import admin
# Specialize the multi-db admin objects for use with specific models.
class BookInline(MultiDBTabularInline):
model = Book
class PublisherAdmin(MultiDBModelAdmin):
inlines = [BookInline]
admin.site.register(Author, MultiDBModelAdmin)
admin.site.register(Publisher, PublisherAdmin)
othersite = admin.AdminSite(’othersite’)
othersite.register(Publisher, MultiDBModelAdmin)
This example sets up two admin sites. On the first site, the Author and Publisher objects are exposed;
Publisher objects have an tabular inline showing books published by that publisher. The second site exposes
just publishers, without the inlines.
Using raw cursors with multiple databases
If you are using more than one database you can use django.db.connections to obtain the connection (and
cursor) for a specific database. django.db.connections is a dictionary-like object that allows you to retrieve a
specific connection using its alias:
from django.db import connections
cursor = connections[’my_db_alias’].cursor()
Limitations of multiple databases
Cross-database relations
Django doesn’t currently provide any support for foreign key or many-to-many relationships spanning multiple
databases. If you have used a router to partition models to different databases, any foreign key and many-to-many
relationships defined by those models must be internal to a single database.
This is because of referential integrity. In order to maintain a relationship between two objects, Django needs to know
that the primary key of the related object is valid. If the primary key is stored on a separate database, it’s not possible
to easily evaluate the validity of a primary key.
If you’re using Postgres, Oracle, or MySQL with InnoDB, this is enforced at the database integrity level – database
level key constraints prevent the creation of relations that can’t be validated.
However, if you’re using SQLite or MySQL with MyISAM tables, there is no enforced referential integrity; as a
result, you may be able to ‘fake’ cross database foreign keys. However, this configuration is not officially supported
by Django.
142 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-146-2048.jpg)
![Django Documentation, Release 1.5.1
{% endfor %}
{% else %}
<p>No messages today.</p>
{% endif %}
{% endwith %}
{% endif %}
It is optimal because:
1. Since QuerySets are lazy, this does no database queries if ‘display_inbox’ is False.
2. Use of with means that we store user.emails.all in a variable for later use, allowing its cache to be
re-used.
3. The line {% if emails %} causes QuerySet.__bool__() to be called, which causes the
user.emails.all() query to be run on the database, and at the least the first line to be turned into an
ORM object. If there aren’t any results, it will return False, otherwise True.
4. The use of {{ emails|length }} calls QuerySet.__len__(), filling out the rest of the cache without
doing another query.
5. The for loop iterates over the already filled cache.
In total, this code does either one or zero database queries. The only deliberate optimization performed is the use of the
with tag. Using QuerySet.exists() or QuerySet.count() at any point would cause additional queries.
Use QuerySet.update() and delete()
Rather than retrieve a load of objects, set some values, and save them individual, use a bulk SQL UPDATE statement,
via QuerySet.update(). Similarly, do bulk deletes where possible.
Note, however, that these bulk update methods cannot call the save() or delete() methods of individual instances,
which means that any custom behavior you have added for these methods will not be executed, including anything
driven from the normal database object signals.
Use foreign key values directly
If you only need a foreign key value, use the foreign key value that is already on the object you’ve got, rather than
getting the whole related object and taking its primary key. i.e. do:
entry.blog_id
instead of:
entry.blog.id
Insert in bulk
When creating objects, where possible, use the bulk_create() method to reduce the number of SQL queries. For
example:
Entry.objects.bulk_create([
Entry(headline="Python 3.0 Released"),
Entry(headline="Python 3.1 Planned")
])
148 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-152-2048.jpg)
![Django Documentation, Release 1.5.1
Create an Article:
>>> a1 = Article(headline=’Django lets you build Web apps easily’)
You can’t associate it with a Publication until it’s been saved:
>>> a1.publications.add(p1)
Traceback (most recent call last):
...
ValueError: ’Article’ instance needs to have a primary key value before a many-to-many relationship can Save it!
>>> a1.save()
Associate the Article with a Publication:
>>> a1.publications.add(p1)
Create another Article, and set it to appear in both Publications:
>>> a2 = Article(headline=’NASA uses Python’)
>>> a2.save()
>>> a2.publications.add(p1, p2)
>>> a2.publications.add(p3)
Adding a second time is OK:
>>> a2.publications.add(p3)
Adding an object of the wrong type raises TypeError:
>>> a2.publications.add(a1)
Traceback (most recent call last):
...
TypeError: ’Publication’ instance expected
Create and add a Publication to an Article in one step using create():
>>> new_publication = a2.publications.create(title=’Highlights for Children’)
Article objects have access to their related Publication objects:
>>> a1.publications.all()
[<Publication: The Python Journal>]
>>> a2.publications.all()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, Publication objects have access to their related Article objects:
>>> p2.article_set.all()
[<Article: NASA uses Python>]
>>> p1.article_set.all()
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Publication.objects.get(id=4).article_set.all()
[<Article: NASA uses Python>]
Many-to-many relationships can be queried using lookups across relationships:
>>> Article.objects.filter(publications__id__exact=1)
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__pk=1)
150 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-154-2048.jpg)
![Django Documentation, Release 1.5.1
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications=1)
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications=p1)
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__title__startswith="Science")
[<Article: NASA uses Python>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__title__startswith="Science").distinct()
[<Article: NASA uses Python>]
The count() function respects distinct() as well:
>>> Article.objects.filter(publications__title__startswith="Science").count()
2
>>> Article.objects.filter(publications__title__startswith="Science").distinct().count()
1
>>> Article.objects.filter(publications__in=[1,2]).distinct()
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__in=[p1,p2]).distinct()
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
Reverse m2m queries are supported (i.e., starting at the table that doesn’t have a ManyToManyField):
>>> Publication.objects.filter(id__exact=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(pk=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article__headline__startswith="NASA")
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, >>> Publication.objects.filter(article__id__exact=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article__pk=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article=a1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article__in=[1,2]).distinct()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, >>> Publication.objects.filter(article__in=[a1,a2]).distinct()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, Excluding a related item works as you would expect, too (although the SQL involved is a little complex):
>>> Article.objects.exclude(publications=p2)
[<Article: Django lets you build Web apps easily>]
If we delete a Publication, its Articles won’t be able to access it:
>>> p1.delete()
>>> Publication.objects.all()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>]
3.2. Models and databases 151](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-155-2048.jpg)
![Django Documentation, Release 1.5.1
>>> a1 = Article.objects.get(pk=1)
>>> a1.publications.all()
[]
If we delete an Article, its Publications won’t be able to access it:
>>> a2.delete()
>>> Article.objects.all()
[<Article: Django lets you build Web apps easily>]
>>> p2.article_set.all()
[]
Adding via the ‘other’ end of an m2m:
>>> a4 = Article(headline=’NASA finds intelligent life on Earth’)
>>> a4.save()
>>> p2.article_set.add(a4)
>>> p2.article_set.all()
[<Article: NASA finds intelligent life on Earth>]
>>> a4.publications.all()
[<Publication: Science News>]
Adding via the other end using keywords:
>>> new_article = p2.article_set.create(headline=’Oxygen-free diet works wonders’)
>>> p2.article_set.all()
[<Article: NASA finds intelligent life on Earth>, <Article: Oxygen-free diet works wonders>]
>>> a5 = p2.article_set.all()[1]
>>> a5.publications.all()
[<Publication: Science News>]
Removing Publication from an Article:
>>> a4.publications.remove(p2)
>>> p2.article_set.all()
[<Article: Oxygen-free diet works wonders>]
>>> a4.publications.all()
[]
And from the other end:
>>> p2.article_set.remove(a5)
>>> p2.article_set.all()
[]
>>> a5.publications.all()
[]
Relation sets can be assigned. Assignment clears any existing set members:
>>> a4.publications.all()
[<Publication: Science News>]
>>> a4.publications = [p3]
>>> a4.publications.all()
[<Publication: Science Weekly>]
Relation sets can be cleared:
>>> p2.article_set.clear()
>>> p2.article_set.all()
[]
152 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-156-2048.jpg)
![Django Documentation, Release 1.5.1
And you can clear from the other end:
>>> p2.article_set.add(a4, a5)
>>> p2.article_set.all()
[<Article: NASA finds intelligent life on Earth>, <Article: Oxygen-free diet works wonders>]
>>> a4.publications.all()
[<Publication: Science News>, <Publication: Science Weekly>]
>>> a4.publications.clear()
>>> a4.publications.all()
[]
>>> p2.article_set.all()
[<Article: Oxygen-free diet works wonders>]
Recreate the Article and Publication we have deleted:
>>> p1 = Publication(title=’The Python Journal’)
>>> p1.save()
>>> a2 = Article(headline=’NASA uses Python’)
>>> a2.save()
>>> a2.publications.add(p1, p2, p3)
Bulk delete some Publications - references to deleted publications should go:
>>> Publication.objects.filter(title__startswith=’Science’).delete()
>>> Publication.objects.all()
[<Publication: Highlights for Children>, <Publication: The Python Journal>]
>>> Article.objects.all()
[<Article: Django lets you build Web apps easily>, <Article: NASA finds intelligent life on Earth>, <>>> a2.publications.all()
[<Publication: The Python Journal>]
Bulk delete some articles - references to deleted objects should go:
>>> q = Article.objects.filter(headline__startswith=’Django’)
>>> print(q)
[<Article: Django lets you build Web apps easily>]
>>> q.delete()
After the delete(), the QuerySet cache needs to be cleared, and the referenced objects should be gone:
>>> print(q)
[]
>>> p1.article_set.all()
[<Article: NASA uses Python>]
An alternate to calling clear() is to assign the empty set:
>>> p1.article_set = []
>>> p1.article_set.all()
[]
>>> a2.publications = [p1, new_publication]
>>> a2.publications.all()
[<Publication: Highlights for Children>, <Publication: The Python Journal>]
>>> a2.publications = []
>>> a2.publications.all()
[]
3.2. Models and databases 153](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-157-2048.jpg)
![Django Documentation, Release 1.5.1
>>> new_article = r.article_set.create(headline="John’s second story", pub_date=date(2005, 7, 29))
>>> new_article
<Article: John’s second story>
>>> new_article.reporter
<Reporter: John Smith>
>>> new_article.reporter.id
1
Create a new article, and add it to the article set:
>>> new_article2 = Article(headline="Paul’s story", pub_date=date(2006, 1, 17))
>>> r.article_set.add(new_article2)
>>> new_article2.reporter
<Reporter: John Smith>
>>> new_article2.reporter.id
1
>>> r.article_set.all()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
Add the same article to a different article set - check that it moves:
>>> r2.article_set.add(new_article2)
>>> new_article2.reporter.id
2
>>> new_article2.reporter
<Reporter: Paul Jones>
Adding an object of the wrong type raises TypeError:
>>> r.article_set.add(r2)
Traceback (most recent call last):
...
TypeError: ’Article’ instance expected
>>> r.article_set.all()
[<Article: John’s second story>, <Article: This is a test>]
>>> r2.article_set.all()
[<Article: Paul’s story>]
>>> r.article_set.count()
2
>>> r2.article_set.count()
1
Note that in the last example the article has moved from John to Paul.
Related managers support field lookups as well. The API automatically follows relationships as far as you need. Use
double underscores to separate relationships. This works as many levels deep as you want. There’s no limit. For
example:
>>> r.article_set.filter(headline__startswith=’This’)
[<Article: This is a test>]
# Find all Articles for any Reporter whose first name is "John".
>>> Article.objects.filter(reporter__first_name__exact=’John’)
[<Article: John’s second story>, <Article: This is a test>]
Exact match is implied here:
3.2. Models and databases 155](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-159-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Article.objects.filter(reporter__first_name=’John’)
[<Article: John’s second story>, <Article: This is a test>]
Query twice over the related field. This translates to an AND condition in the WHERE clause:
>>> Article.objects.filter(reporter__first_name__exact=’John’, reporter__last_name__exact=’Smith’)
[<Article: John’s second story>, <Article: This is a test>]
For the related lookup you can supply a primary key value or pass the related object explicitly:
>>> Article.objects.filter(reporter__pk=1)
[<Article: John’s second story>, <Article: This is a test>]
>>> Article.objects.filter(reporter=1)
[<Article: John’s second story>, <Article: This is a test>]
>>> Article.objects.filter(reporter=r)
[<Article: John’s second story>, <Article: This is a test>]
>>> Article.objects.filter(reporter__in=[1,2]).distinct()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
>>> Article.objects.filter(reporter__in=[r,r2]).distinct()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
You can also use a queryset instead of a literal list of instances:
>>> Article.objects.filter(reporter__in=Reporter.objects.filter(first_name=’John’)).distinct()
[<Article: John’s second story>, <Article: This is a test>]
Querying in the opposite direction:
>>> Reporter.objects.filter(article__pk=1)
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article=1)
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article=a)
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article__headline__startswith=’This’)
[<Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>]
>>> Reporter.objects.filter(article__headline__startswith=’This’).distinct()
[<Reporter: John Smith>]
Counting in the opposite direction works in conjunction with distinct():
>>> Reporter.objects.filter(article__headline__startswith=’This’).count()
3
>>> Reporter.objects.filter(article__headline__startswith=’This’).distinct().count()
1
Queries can go round in circles:
>>> Reporter.objects.filter(article__reporter__first_name__startswith=’John’)
[<Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>]
>>> Reporter.objects.filter(article__reporter__first_name__startswith=’John’).distinct()
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article__reporter__exact=r).distinct()
[<Reporter: John Smith>]
If you delete a reporter, his articles will be deleted (assuming that the ForeignKey was defined with
django.db.models.ForeignKey.on_delete set to CASCADE, which is the default):
156 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-160-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Article.objects.all()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
>>> Reporter.objects.order_by(’first_name’)
[<Reporter: John Smith>, <Reporter: Paul Jones>]
>>> r2.delete()
>>> Article.objects.all()
[<Article: John’s second story>, <Article: This is a test>]
>>> Reporter.objects.order_by(’first_name’)
[<Reporter: John Smith>]
You can delete using a JOIN in the query:
>>> Reporter.objects.filter(article__headline__startswith=’This’).delete()
>>> Reporter.objects.all()
[]
>>> Article.objects.all()
[]
One-to-one relationships
To define a one-to-one relationship, use OneToOneField.
In this example, a Place optionally can be a Restaurant:
from django.db import models, transaction, IntegrityError
class Place(models.Model):
name = models.CharField(max_length=50)
address = models.CharField(max_length=80)
def __unicode__(self):
return u"%s the place" % self.name
class Restaurant(models.Model):
place = models.OneToOneField(Place, primary_key=True)
serves_hot_dogs = models.BooleanField()
serves_pizza = models.BooleanField()
def __unicode__(self):
return u"%s the restaurant" % self.place.name
class Waiter(models.Model):
restaurant = models.ForeignKey(Restaurant)
name = models.CharField(max_length=50)
def __unicode__(self):
return u"%s the waiter at %s" % (self.name, self.restaurant)
What follows are examples of operations that can be performed using the Python API facilities.
Create a couple of Places:
>>> p1 = Place(name=’Demon Dogs’, address=’944 W. Fullerton’)
>>> p1.save()
>>> p2 = Place(name=’Ace Hardware’, address=’1013 N. Ashland’)
>>> p2.save()
Create a Restaurant. Pass the ID of the “parent” object as this object’s ID:
3.2. Models and databases 157](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-161-2048.jpg)
![Django Documentation, Release 1.5.1
>>> r = Restaurant(place=p1, serves_hot_dogs=True, serves_pizza=False)
>>> r.save()
A Restaurant can access its place:
>>> r.place
<Place: Demon Dogs the place>
A Place can access its restaurant, if available:
>>> p1.restaurant
<Restaurant: Demon Dogs the restaurant>
p2 doesn’t have an associated restaurant:
>>> p2.restaurant
Traceback (most recent call last):
...
DoesNotExist: Restaurant matching query does not exist.
Set the place using assignment notation. Because place is the primary key on Restaurant, the save will create a new
restaurant:
>>> r.place = p2
>>> r.save()
>>> p2.restaurant
<Restaurant: Ace Hardware the restaurant>
>>> r.place
<Place: Ace Hardware the place>
Set the place back again, using assignment in the reverse direction:
>>> p1.restaurant = r
>>> p1.restaurant
<Restaurant: Demon Dogs the restaurant>
Restaurant.objects.all() just returns the Restaurants, not the Places. Note that there are two restaurants - Ace Hardware
the Restaurant was created in the call to r.place = p2:
>>> Restaurant.objects.all()
[<Restaurant: Demon Dogs the restaurant>, <Restaurant: Ace Hardware the restaurant>]
Place.objects.all() returns all Places, regardless of whether they have Restaurants:
>>> Place.objects.order_by(’name’)
[<Place: Ace Hardware the place>, <Place: Demon Dogs the place>]
You can query the models using lookups across relationships:
>>> Restaurant.objects.get(place=p1)
<Restaurant: Demon Dogs the restaurant>
>>> Restaurant.objects.get(place__pk=1)
<Restaurant: Demon Dogs the restaurant>
>>> Restaurant.objects.filter(place__name__startswith="Demon")
[<Restaurant: Demon Dogs the restaurant>]
>>> Restaurant.objects.exclude(place__address__contains="Ashland")
[<Restaurant: Demon Dogs the restaurant>]
This of course works in reverse:
158 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-162-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Place.objects.get(pk=1)
<Place: Demon Dogs the place>
>>> Place.objects.get(restaurant__place__exact=p1)
<Place: Demon Dogs the place>
>>> Place.objects.get(restaurant=r)
<Place: Demon Dogs the place>
>>> Place.objects.get(restaurant__place__name__startswith="Demon")
<Place: Demon Dogs the place>
Add a Waiter to the Restaurant:
>>> w = r.waiter_set.create(name=’Joe’)
>>> w.save()
>>> w
<Waiter: Joe the waiter at Demon Dogs the restaurant>
Query the waiters:
>>> Waiter.objects.filter(restaurant__place=p1)
[<Waiter: Joe the waiter at Demon Dogs the restaurant>]
>>> Waiter.objects.filter(restaurant__place__name__startswith="Demon")
[<Waiter: Joe the waiter at Demon Dogs the restaurant>]
3.3 Handling HTTP requests
Information on handling HTTP requests in Django:
3.3.1 URL dispatcher
A clean, elegant URL scheme is an important detail in a high-quality Web application. Django lets you design URLs
however you want, with no framework limitations.
There’s no .php or .cgi required, and certainly none of that 0,2097,1-1-1928,00 nonsense.
See Cool URIs don’t change, by World Wide Web creator Tim Berners-Lee, for excellent arguments on why URLs
should be clean and usable.
Overview
To design URLs for an app, you create a Python module informally called a URLconf (URL configuration). This
module is pure Python code and is a simple mapping between URL patterns (simple regular expressions) to Python
functions (your views).
This mapping can be as short or as long as needed. It can reference other mappings. And, because it’s pure Python
code, it can be constructed dynamically. New in version 1.4: Django also provides a way to translate URLs according
to the active language. See the internationalization documentation for more information.
How Django processes a request
When a user requests a page from your Django-powered site, this is the algorithm the system follows to determine
which Python code to execute:
3.3. Handling HTTP requests 159](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-163-2048.jpg)
![Django Documentation, Release 1.5.1
• handler403 – See django.conf.urls.handler403.
New in version 1.4: handler403 is new in Django 1.4.
The view prefix
You can specify a common prefix in your patterns() call, to cut down on code duplication.
Here’s the example URLconf from the Django overview:
from django.conf.urls import patterns, url
urlpatterns = patterns(’’,
url(r’^articles/(d{4})/$’, ’news.views.year_archive’),
url(r’^articles/(d{4})/(d{2})/$’, ’news.views.month_archive’),
url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’news.views.article_detail’),
)
In this example, each view has a common prefix – ’news.views’. Instead of typing that out for each entry in
urlpatterns, you can use the first argument to the patterns() function to specify a prefix to apply to each
view function.
With this in mind, the above example can be written more concisely as:
from django.conf.urls import patterns, url
urlpatterns = patterns(’news.views’,
url(r’^articles/(d{4})/$’, ’year_archive’),
url(r’^articles/(d{4})/(d{2})/$’, ’month_archive’),
url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’article_detail’),
)
Note that you don’t put a trailing dot (".") in the prefix. Django puts that in automatically.
Multiple view prefixes
In practice, you’ll probably end up mixing and matching views to the point where the views in your urlpatterns
won’t have a common prefix. However, you can still take advantage of the view prefix shortcut to remove duplication.
Just add multiple patterns() objects together, like this:
Old:
from django.conf.urls import patterns, url
urlpatterns = patterns(’’,
url(r’^$’, ’myapp.views.app_index’),
url(r’^(?P<year>d{4})/(?P<month>[a-z]{3})/$’, ’myapp.views.month_display’),
url(r’^tag/(?P<tag>w+)/$’, ’weblog.views.tag’),
)
New:
from django.conf.urls import patterns, url
urlpatterns = patterns(’myapp.views’,
url(r’^$’, ’app_index’),
url(r’^(?P<year>d{4})/(?P<month>[a-z]{3})/$’,’month_display’),
)
3.3. Handling HTTP requests 163](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-167-2048.jpg)
![Django Documentation, Release 1.5.1
from django.core.exceptions import PermissionDenied
def edit(request, pk):
if not request.user.is_staff:
raise PermissionDenied
# ...
It is possible to override django.views.defaults.permission_denied in the same way you can for the
404 and 500 views by specifying a handler403 in your URLconf:
handler403 = ’mysite.views.my_custom_permission_denied_view’
3.3.3 View decorators
Django provides several decorators that can be applied to views to support various HTTP features.
Allowed HTTP methods
The decorators in django.views.decorators.http can be used to restrict access to views based on the request
method. These decorators will return a django.http.HttpResponseNotAllowed if the conditions are not
met.
require_http_methods(request_method_list)
Decorator to require that a view only accept particular request methods. Usage:
from django.views.decorators.http import require_http_methods
@require_http_methods(["GET", "POST"])
def my_view(request):
# I can assume now that only GET or POST requests make it this far
# ...
pass
Note that request methods should be in uppercase.
require_GET()
Decorator to require that a view only accept the GET method.
require_POST()
Decorator to require that a view only accept the POST method.
require_safe()
New in version 1.4. Decorator to require that a view only accept the GET and HEAD methods. These methods
are commonly considered “safe” because they should not have the significance of taking an action other than
retrieving the requested resource.
Note: Django will automatically strip the content of responses to HEAD requests while leaving the headers un-changed,
so you may handle HEAD requests exactly like GET requests in your views. Since some software, such
as link checkers, rely on HEAD requests, you might prefer using require_safe instead of require_GET.
Conditional view processing
The following decorators in django.views.decorators.http can be used to control caching behavior on
particular views.
174 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-178-2048.jpg)
![Django Documentation, Release 1.5.1
condition(etag_func=None, last_modified_func=None)
etag(etag_func)
last_modified(last_modified_func)
These decorators can be used to generate ETag and Last-Modified headers; see conditional view process-ing.
GZip compression
The decorators in django.views.decorators.gzip control content compression on a per-view basis.
gzip_page()
This decorator compresses content if the browser allows gzip compression. It sets the Vary header accordingly,
so that caches will base their storage on the Accept-Encoding header.
Vary headers
The decorators in django.views.decorators.vary can be used to control caching based on specific request
headers.
vary_on_cookie(func)
vary_on_headers(*headers)
The Vary header defines which request headers a cache mechanism should take into account when building its
cache key.
See using vary headers.
3.3.4 File Uploads
When Django handles a file upload, the file data ends up placed in request.FILES (for more on the request
object see the documentation for request and response objects). This document explains how files are stored on disk
and in memory, and how to customize the default behavior.
Basic file uploads
Consider a simple form containing a FileField:
from django import forms
class UploadFileForm(forms.Form):
title = forms.CharField(max_length=50)
file = forms.FileField()
A view handling this form will receive the file data in request.FILES, which is a dictionary containing a key for
each FileField (or ImageField, or other FileField subclass) in the form. So the data from the above form
would be accessible as request.FILES[’file’].
Note that request.FILES will only contain data if the request method was POST and the <form> that posted the
request has the attribute enctype="multipart/form-data". Otherwise, request.FILES will be empty.
Most of the time, you’ll simply pass the file data from request into the form as described in Binding uploaded files
to a form. This would look something like:
3.3. Handling HTTP requests 175](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-179-2048.jpg)
![Django Documentation, Release 1.5.1
from django.http import HttpResponseRedirect
from django.shortcuts import render_to_response
# Imaginary function to handle an uploaded file.
from somewhere import handle_uploaded_file
def upload_file(request):
if request.method == ’POST’:
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
handle_uploaded_file(request.FILES[’file’])
return HttpResponseRedirect(’/success/url/’)
else:
form = UploadFileForm()
return render_to_response(’upload.html’, {’form’: form})
Notice that we have to pass request.FILES into the form’s constructor; this is how file data gets bound into a
form.
Handling uploaded files
class UploadedFile
The final piece of the puzzle is handling the actual file data from request.FILES. Each entry in this dictio-nary
is an UploadedFile object – a simple wrapper around an uploaded file. You’ll usually use one of these
methods to access the uploaded content:
read()
Read the entire uploaded data from the file. Be careful with this method: if the uploaded file is huge it can
overwhelm your system if you try to read it into memory. You’ll probably want to use chunks() instead;
see below.
multiple_chunks()
Returns True if the uploaded file is big enough to require reading in multiple chunks. By default this will
be any file larger than 2.5 megabytes, but that’s configurable; see below.
chunks()
A generator returning chunks of the file. If multiple_chunks() is True, you should use this method
in a loop instead of read().
In practice, it’s often easiest simply to use chunks() all the time; see the example below.
name
The name of the uploaded file (e.g. my_file.txt).
size
The size, in bytes, of the uploaded file.
There are a few other methods and attributes available on UploadedFile objects; see UploadedFile objects for a
complete reference.
Putting it all together, here’s a common way you might handle an uploaded file:
def handle_uploaded_file(f):
with open(’some/file/name.txt’, ’wb+’) as destination:
for chunk in f.chunks():
destination.write(chunk)
Looping over UploadedFile.chunks() instead of using read() ensures that large files don’t overwhelm your
system’s memory.
176 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-180-2048.jpg)
![Django Documentation, Release 1.5.1
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import ModelFormWithFileField
def upload_file(request):
if request.method == ’POST’:
form = ModelFormWithFileField(request.POST, request.FILES)
if form.is_valid():
# file is saved
form.save()
return HttpResponseRedirect(’/success/url/’)
else:
form = ModelFormWithFileField()
return render(request, ’upload.html’, {’form’: form})
If you are constructing an object manually, you can simply assign the file object from request.FILES to the file
field in the model:
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import UploadFileForm
from .models import ModelWithFileField
def upload_file(request):
if request.method == ’POST’:
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
instance = ModelWithFileField(file_field=request.FILES[’file’])
instance.save()
return HttpResponseRedirect(’/success/url/’)
else:
form = UploadFileForm()
return render(request, ’upload.html’, {’form’: form})
UploadedFile objects
In addition to those inherited from File, all UploadedFile objects define the following methods/attributes:
UploadedFile.content_type
The content-type header uploaded with the file (e.g. text/plain or application/pdf). Like any data
supplied by the user, you shouldn’t trust that the uploaded file is actually this type. You’ll still need to validate
that the file contains the content that the content-type header claims – “trust but verify.”
UploadedFile.charset
For text/* content-types, the character set (i.e. utf8) supplied by the browser. Again, “trust but verify” is
the best policy here.
UploadedFile.temporary_file_path
Only files uploaded onto disk will have this method; it returns the full path to the temporary uploaded file.
Note: Like regular Python files, you can read the file line-by-line simply by iterating over the uploaded file:
for line in uploadedfile:
do_something_with(line)
However, unlike standard Python files, UploadedFile only understands n (also known as “Unix-style”) line
endings. If you know that you need to handle uploaded files with different line endings, you’ll need to do so in your
178 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-182-2048.jpg)
![Django Documentation, Release 1.5.1
view.
Upload Handlers
When a user uploads a file, Django passes off the file data to an upload handler – a small class that handles file data as
it gets uploaded. Upload handlers are initially defined in the FILE_UPLOAD_HANDLERS setting, which defaults to:
("django.core.files.uploadhandler.MemoryFileUploadHandler",
"django.core.files.uploadhandler.TemporaryFileUploadHandler",)
Together the MemoryFileUploadHandler and TemporaryFileUploadHandler provide Django’s default
file upload behavior of reading small files into memory and large ones onto disk.
You can write custom handlers that customize how Django handles files. You could, for example, use custom handlers
to enforce user-level quotas, compress data on the fly, render progress bars, and even send data to another storage
location directly without storing it locally.
Modifying upload handlers on the fly
Sometimes particular views require different upload behavior. In these cases, you can override upload handlers on a
per-request basis by modifying request.upload_handlers. By default, this list will contain the upload handlers
given by FILE_UPLOAD_HANDLERS, but you can modify the list as you would any other list.
For instance, suppose you’ve written a ProgressBarUploadHandler that provides feedback on upload progress
to some sort of AJAX widget. You’d add this handler to your upload handlers like this:
request.upload_handlers.insert(0, ProgressBarUploadHandler())
You’d probably want to use list.insert() in this case (instead of append()) because a progress bar handler
would need to run before any other handlers. Remember, the upload handlers are processed in order.
If you want to replace the upload handlers completely, you can just assign a new list:
request.upload_handlers = [ProgressBarUploadHandler()]
Note: You can only modify upload handlers before accessing request.POST or request.FILES – it
doesn’t make sense to change upload handlers after upload handling has already started. If you try to modify
request.upload_handlers after reading from request.POST or request.FILES Django will throw an
error.
Thus, you should always modify uploading handlers as early in your view as possible.
Also, request.POST is accessed by CsrfViewMiddleware which is enabled by default. This means you will
need to use csrf_exempt() on your view to allow you to change the upload handlers. You will then need to use
csrf_protect() on the function that actually processes the request. Note that this means that the handlers may
start receiving the file upload before the CSRF checks have been done. Example code:
from django.views.decorators.csrf import csrf_exempt, csrf_protect
@csrf_exempt
def upload_file_view(request):
request.upload_handlers.insert(0, ProgressBarUploadHandler())
return _upload_file_view(request)
@csrf_protect
3.3. Handling HTTP requests 179](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-183-2048.jpg)
![Django Documentation, Release 1.5.1
This method may raise a StopFutureHandlers exception to prevent future handlers from handling this file.
FileUploadHandler.upload_complete(self) Callback signaling that the entire upload (all files) has
completed.
FileUploadHandler.handle_raw_input(self, input_data, META, content_length, boundary, encoding)
Allows the handler to completely override the parsing of the raw HTTP input.
input_data is a file-like object that supports read()-ing.
META is the same object as request.META.
content_length is the length of the data in input_data. Don’t read more than content_length
bytes from input_data.
boundary is the MIME boundary for this request.
encoding is the encoding of the request.
Return None if you want upload handling to continue, or a tuple of (POST, FILES) if you want to return
the new data structures suitable for the request directly.
3.3.5 Django shortcut functions
The package django.shortcuts collects helper functions and classes that “span” multiple levels of MVC. In
other words, these functions/classes introduce controlled coupling for convenience’s sake.
render
render(request, template_name[, dictionary][, context_instance][, content_type][, status][, current_app])
Combines a given template with a given context dictionary and returns an HttpResponse object with that
rendered text.
render() is the same as a call to render_to_response() with a context_instance argument that forces
the use of a RequestContext.
Required arguments
request The request object used to generate this response.
template_name The full name of a template to use or sequence of template names.
Optional arguments
dictionary A dictionary of values to add to the template context. By default, this is an empty dictionary. If a
value in the dictionary is callable, the view will call it just before rendering the template.
context_instance The context instance to render the template with. By default, the template will be rendered
with a RequestContext instance (filled with values from request and dictionary).
content_type The MIME type to use for the resulting document. Defaults to the value of the
DEFAULT_CONTENT_TYPE setting. Changed in version 1.5: This parameter used to be called mimetype.
status The status code for the response. Defaults to 200.
current_app A hint indicating which application contains the current view. See the namespaced URL resolution
strategy for more information.
3.3. Handling HTTP requests 181](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-185-2048.jpg)
![Django Documentation, Release 1.5.1
Example
The following example renders the template myapp/index.html with the MIME type
application/xhtml+xml:
from django.shortcuts import render
def my_view(request):
# View code here...
return render(request, ’myapp/index.html’, {"foo": "bar"},
content_type="application/xhtml+xml")
This example is equivalent to:
from django.http import HttpResponse
from django.template import RequestContext, loader
def my_view(request):
# View code here...
t = loader.get_template(’myapp/template.html’)
c = RequestContext(request, {’foo’: ’bar’})
return HttpResponse(t.render(c),
content_type="application/xhtml+xml")
render_to_response
render_to_response(template_name[, dictionary][, context_instance][, content_type])
Renders a given template with a given context dictionary and returns an HttpResponse object with that
rendered text.
Required arguments
template_name The full name of a template to use or sequence of template names. If a sequence is given, the first
template that exists will be used. See the template loader documentation for more information on how templates
are found.
Optional arguments
dictionary A dictionary of values to add to the template context. By default, this is an empty dictionary. If a
value in the dictionary is callable, the view will call it just before rendering the template.
context_instance The context instance to render the template with. By default, the template will be rendered
with a Context instance (filled with values from dictionary). If you need to use context processors, render
the template with a RequestContext instance instead. Your code might look something like this:
return render_to_response(’my_template.html’,
my_data_dictionary,
context_instance=RequestContext(request))
content_type The MIME type to use for the resulting document. Defaults to the value of the
DEFAULT_CONTENT_TYPE setting. Changed in version 1.5: This parameter used to be called mimetype.
182 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-186-2048.jpg)
![Django Documentation, Release 1.5.1
Example
The following example renders the template myapp/index.html with the MIME type
application/xhtml+xml:
from django.shortcuts import render_to_response
def my_view(request):
# View code here...
return render_to_response(’myapp/index.html’, {"foo": "bar"},
mimetype="application/xhtml+xml")
This example is equivalent to:
from django.http import HttpResponse
from django.template import Context, loader
def my_view(request):
# View code here...
t = loader.get_template(’myapp/template.html’)
c = Context({’foo’: ’bar’})
return HttpResponse(t.render(c),
content_type="application/xhtml+xml")
redirect
redirect(to[, permanent=False ], *args, **kwargs)
Returns an HttpResponseRedirect to the appropriate URL for the arguments passed.
The arguments could be:
•A model: the model’s get_absolute_url() function will be called.
•A view name, possibly with arguments: urlresolvers.reverse will be used to reverse-resolve the
name.
•A URL, which will be used as-is for the redirect location.
By default issues a temporary redirect; pass permanent=True to issue a permanent redirect
Examples
You can use the redirect() function in a number of ways.
1. By passing some object; that object’s get_absolute_url() method will be called to figure out the redirect
URL:
from django.shortcuts import redirect
def my_view(request):
...
object = MyModel.objects.get(...)
return redirect(object)
2. By passing the name of a view and optionally some positional or keyword arguments; the URL will be reverse
resolved using the reverse() method:
3.3. Handling HTTP requests 183](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-187-2048.jpg)
![Django Documentation, Release 1.5.1
Warning: The session data is signed but not encrypted
When using the cookies backend the session data can be read by the client.
A MAC (Message Authentication Code) is used to protect the data against changes by the client, so that the session
data will be invalidated when being tampered with. The same invalidation happens if the client storing the cookie
(e.g. your user’s browser) can’t store all of the session cookie and drops data. Even though Django compresses the
data, it’s still entirely possible to exceed the common limit of 4096 bytes per cookie.
No freshness guarantee
Note also that while the MAC can guarantee the authenticity of the data (that it was generated by your site, and
not someone else), and the integrity of the data (that it is all there and correct), it cannot guarantee freshness i.e.
that you are being sent back the last thing you sent to the client. This means that for some uses of session data, the
cookie backend might open you up to replay attacks. Cookies will only be detected as ‘stale’ if they are older than
your SESSION_COOKIE_AGE.
Performance
Finally, the size of a cookie can have an impact on the speed of your site.
Using sessions in views
When SessionMiddleware is activated, each HttpRequest object – the first argument to any Django view
function – will have a session attribute, which is a dictionary-like object.
You can read it and write to request.session at any point in your view. You can edit it multiple times.
class backends.base.SessionBase
This is the base class for all session objects. It has the following standard dictionary methods:
__getitem__(key)
Example: fav_color = request.session[’fav_color’]
__setitem__(key, value)
Example: request.session[’fav_color’] = ’blue’
__delitem__(key)
Example: del request.session[’fav_color’]. This raises KeyError if the given key isn’t
already in the session.
__contains__(key)
Example: ’fav_color’ in request.session
get(key, default=None)
Example: fav_color = request.session.get(’fav_color’, ’red’)
pop(key)
Example: fav_color = request.session.pop(’fav_color’)
keys()
items()
setdefault()
clear()
It also has these methods:
flush()
Delete the current session data from the session and regenerate the session key value that is sent back to
the user in the cookie. This is used if you want to ensure that the previous session data can’t be accessed
again from the user’s browser (for example, the django.contrib.auth.logout() function calls
it).
3.3. Handling HTTP requests 191](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-195-2048.jpg)
![Django Documentation, Release 1.5.1
• Don’t override request.session with a new object, and don’t access or set its attributes. Use it like a
Python dictionary.
Examples
This simplistic view sets a has_commented variable to True after a user posts a comment. It doesn’t let a user post
a comment more than once:
def post_comment(request, new_comment):
if request.session.get(’has_commented’, False):
return HttpResponse("You’ve already commented.")
c = comments.Comment(comment=new_comment)
c.save()
request.session[’has_commented’] = True
return HttpResponse(’Thanks for your comment!’)
This simplistic view logs in a “member” of the site:
def login(request):
m = Member.objects.get(username=request.POST[’username’])
if m.password == request.POST[’password’]:
request.session[’member_id’] = m.id
return HttpResponse("You’re logged in.")
else:
return HttpResponse("Your username and password didn’t match.")
...And this one logs a member out, according to login() above:
def logout(request):
try:
del request.session[’member_id’]
except KeyError:
pass
return HttpResponse("You’re logged out.")
The standard django.contrib.auth.logout() function actually does a bit more than this to prevent inadver-tent
data leakage. It calls the flush() method of request.session. We are using this example as a demonstra-tion
of how to work with session objects, not as a full logout() implementation.
Setting test cookies
As a convenience, Django provides an easy way to test whether the user’s browser accepts cookies. Just call the
set_test_cookie() method of request.session in a view, and call test_cookie_worked() in a sub-sequent
view – not in the same view call.
This awkward split between set_test_cookie() and test_cookie_worked() is necessary due to the way
cookies work. When you set a cookie, you can’t actually tell whether a browser accepted it until the browser’s next
request.
It’s good practice to use delete_test_cookie() to clean up after yourself. Do this after you’ve verified that the
test cookie worked.
Here’s a typical usage example:
def login(request):
if request.method == ’POST’:
if request.session.test_cookie_worked():
3.3. Handling HTTP requests 193](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-197-2048.jpg)
![Django Documentation, Release 1.5.1
request.session.delete_test_cookie()
return HttpResponse("You’re logged in.")
else:
return HttpResponse("Please enable cookies and try again.")
request.session.set_test_cookie()
return render_to_response(’foo/login_form.html’)
Using sessions out of views
An API is available to manipulate session data outside of a view:
>>> from django.contrib.sessions.backends.db import SessionStore
>>> import datetime
>>> s = SessionStore()
>>> s[’last_login’] = datetime.datetime(2005, 8, 20, 13, 35, 10)
>>> s.save()
>>> s.session_key
’2b1189a188b44ad18c35e113ac6ceead’
>>> s = SessionStore(session_key=’2b1189a188b44ad18c35e113ac6ceead’)
>>> s[’last_login’]
datetime.datetime(2005, 8, 20, 13, 35, 0)
In order to prevent session fixation attacks, sessions keys that don’t exist are regenerated:
>>> from django.contrib.sessions.backends.db import SessionStore
>>> s = SessionStore(session_key=’no-such-session-here’)
>>> s.save()
>>> s.session_key
’ff882814010ccbc3c870523934fee5a2’
If you’re using the django.contrib.sessions.backends.db backend, each session is just a normal Django
model. The Session model is defined in django/contrib/sessions/models.py. Because it’s a normal
model, you can access sessions using the normal Django database API:
>>> from django.contrib.sessions.models import Session
>>> s = Session.objects.get(pk=’2b1189a188b44ad18c35e113ac6ceead’)
>>> s.expire_date
datetime.datetime(2005, 8, 20, 13, 35, 12)
Note that you’ll need to call get_decoded() to get the session dictionary. This is necessary because the dictionary
is stored in an encoded format:
>>> s.session_data
’KGRwMQpTJ19hdXRoX3VzZXJfaWQnCnAyCkkxCnMuMTExY2ZjODI2Yj...’
>>> s.get_decoded()
{’user_id’: 42}
When sessions are saved
By default, Django only saves to the session database when the session has been modified – that is if any of its
dictionary values have been assigned or deleted:
# Session is modified.
request.session[’foo’] = ’bar’
194 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-198-2048.jpg)
![Django Documentation, Release 1.5.1
# Session is modified.
del request.session[’foo’]
# Session is modified.
request.session[’foo’] = {}
# Gotcha: Session is NOT modified, because this alters
# request.session[’foo’] instead of request.session.
request.session[’foo’][’bar’] = ’baz’
In the last case of the above example, we can tell the session object explicitly that it has been modified by setting the
modified attribute on the session object:
request.session.modified = True
To change this default behavior, set the SESSION_SAVE_EVERY_REQUEST setting to True. When set to True,
Django will save the session to the database on every single request.
Note that the session cookie is only sent when a session has been created or modified. If
SESSION_SAVE_EVERY_REQUEST is True, the session cookie will be sent on every request.
Similarly, the expires part of a session cookie is updated each time the session cookie is sent. Changed in version
1.5: The session is not saved if the response’s status code is 500.
Browser-length sessions vs. persistent sessions
You can control whether the session framework uses browser-length sessions vs. persistent sessions with the
SESSION_EXPIRE_AT_BROWSER_CLOSE setting.
By default, SESSION_EXPIRE_AT_BROWSER_CLOSE is set to False, which means session cookies will be stored
in users’ browsers for as long as SESSION_COOKIE_AGE. Use this if you don’t want people to have to log in every
time they open a browser.
If SESSION_EXPIRE_AT_BROWSER_CLOSE is set to True, Django will use browser-length cookies – cookies
that expire as soon as the user closes his or her browser. Use this if you want people to have to log in every time they
open a browser.
This setting is a global default and can be overwritten at a per-session level by explicitly calling the set_expiry()
method of request.session as described above in using sessions in views.
Note: Some browsers (Chrome, for example) provide settings that allow users to continue brows-ing
sessions after closing and re-opening the browser. In some cases, this can interfere with the
SESSION_EXPIRE_AT_BROWSER_CLOSE setting and prevent sessions from expiring on browser close. Please
be aware of this while testing Django applications which have the SESSION_EXPIRE_AT_BROWSER_CLOSE set-ting
enabled.
Clearing the session store
As users create new sessions on your website, session data can accumulate in your session store. If you’re using the
database backend, the django_session database table will grow. If you’re using the file backend, your temporary
directory will contain an increasing number of files.
To understand this problem, consider what happens with the database backend. When a user logs in, Django adds a
row to the django_session database table. Django updates this row each time the session data changes. If the
user logs out manually, Django deletes the row. But if the user does not log out, the row never gets deleted. A similar
process happens with the file backend.
3.3. Handling HTTP requests 195](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-199-2048.jpg)
![Django Documentation, Release 1.5.1
Note: You can still access the unvalidated data directly from request.POST at this point, but the validated data is
better.
In the above example, cc_myself will be a boolean value. Likewise, fields such as IntegerField and
FloatField convert values to a Python int and float respectively.
Read-only fields are not available in form.cleaned_data (and setting a value in a custom clean() method
won’t have any effect). These fields are displayed as text rather than as input elements, and thus are not posted back
to the server.
Extending the earlier example, here’s how the form data could be processed:
if form.is_valid():
subject = form.cleaned_data[’subject’]
message = form.cleaned_data[’message’]
sender = form.cleaned_data[’sender’]
cc_myself = form.cleaned_data[’cc_myself’]
recipients = [’info@example.com’]
if cc_myself:
recipients.append(sender)
from django.core.mail import send_mail
send_mail(subject, message, sender, recipients)
return HttpResponseRedirect(’/thanks/’) # Redirect after POST
Tip: For more on sending email from Django, see Sending email.
Displaying a form using a template
Forms are designed to work with the Django template language. In the above example, we passed our ContactForm
instance to the template using the context variable form. Here’s a simple example template:
<form action="/contact/" method="post">{% csrf_token %}
{{ form.as_p }}
<input type="submit" value="Submit" />
</form>
The form only outputs its own fields; it is up to you to provide the surrounding <form> tags and the submit button.
If your form includes uploaded files, be sure to include enctype="multipart/form-data" in the form el-ement.
If you wish to write a generic template that will work whether or not the form has files, you can use the
is_multipart() attribute on the form:
<form action="/contact/" method="post"
{% if form.is_multipart %}enctype="multipart/form-data"{% endif %}>
Forms and Cross Site Request Forgery protection
Django ships with an easy-to-use protection against Cross Site Request Forgeries. When submitting a form via POST
with CSRF protection enabled you must use the csrf_token template tag as in the preceding example. However,
since CSRF protection is not directly tied to forms in templates, this tag is omitted from the following examples in this
document.
form.as_p will output the form with each form field and accompanying label wrapped in a paragraph. Here’s the
output for our example template:
200 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-204-2048.jpg)
![Django Documentation, Release 1.5.1
3.4.3 Further topics
This covers the basics, but forms can do a whole lot more:
Formsets
class django.forms.formset.BaseFormSet
A formset is a layer of abstraction to work with multiple forms on the same page. It can be best compared to a data
grid. Let’s say you have the following form:
>>> from django import forms
>>> class ArticleForm(forms.Form):
... title = forms.CharField()
... pub_date = forms.DateField()
You might want to allow the user to create several articles at once. To create a formset out of an ArticleForm you
would do:
>>> from django.forms.formsets import formset_factory
>>> ArticleFormSet = formset_factory(ArticleForm)
You now have created a formset named ArticleFormSet. The formset gives you the ability to iterate over the
forms in the formset and display them as you would with a regular form:
>>> formset = ArticleFormSet()
>>> for form in formset:
... print(form.as_table())
<tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_As you can see it only displayed one empty form. The number of empty forms that is displayed is controlled by the
extra parameter. By default, formset_factory defines one extra form; the following example will display two
blank forms:
>>> ArticleFormSet = formset_factory(ArticleForm, extra=2)
Iterating over the formset will render the forms in the order they were created. You can change this order by
providing an alternate implementation for the __iter__() method.
Formsets can also be indexed into, which returns the corresponding form. If you override __iter__, you will need
to also override __getitem__ to have matching behavior.
Using initial data with a formset
Initial data is what drives the main usability of a formset. As shown above you can define the number of extra forms.
What this means is that you are telling the formset how many additional forms to show in addition to the number of
forms it generates from the initial data. Lets take a look at an example:
>>> ArticleFormSet = formset_factory(ArticleForm, extra=2)
>>> formset = ArticleFormSet(initial=[
... {’title’: u’Django is now open source’,
... ’pub_date’: datetime.date.today(),}
... ])
>>> for form in formset:
... print(form.as_table())
204 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-208-2048.jpg)
![Django Documentation, Release 1.5.1
... ’form-1-pub_date’: u’’, # <-- this date is missing but required
... }
>>> formset = ArticleFormSet(data)
>>> formset.is_valid()
False
>>> formset.errors
[{}, {’pub_date’: [u’This field is required.’]}]
As we can see, formset.errors is a list whose entries correspond to the forms in the formset. Validation was
performed for each of the two forms, and the expected error message appears for the second item. New in version 1.4.
We can also check if form data differs from the initial data (i.e. the form was sent without any data):
>>> data = {
... ’form-TOTAL_FORMS’: u’1’,
... ’form-INITIAL_FORMS’: u’0’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’’,
... ’form-0-pub_date’: u’’,
... }
>>> formset = ArticleFormSet(data)
>>> formset.has_changed()
False
Understanding the ManagementForm You may have noticed the additional data (form-TOTAL_FORMS,
form-INITIAL_FORMS and form-MAX_NUM_FORMS) that was required in the formset’s data above. This data is
required for the ManagementForm. This form is used by the formset to manage the collection of forms contained
in the formset. If you don’t provide this management data, an exception will be raised:
>>> data = {
... ’form-0-title’: u’Test’,
... ’form-0-pub_date’: u’’,
... }
>>> formset = ArticleFormSet(data)
Traceback (most recent call last):
...
django.forms.util.ValidationError: [u’ManagementForm data is missing or has been tampered with’]
It is used to keep track of how many form instances are being displayed. If you are adding new forms via JavaScript,
you should increment the count fields in this form as well. On the other hand, if you are using JavaScript to allow
deletion of existing objects, then you need to ensure the ones being removed are properly marked for deletion by
including form-#-DELETE in the POST data. It is expected that all forms are present in the POST data regardless.
The management form is available as an attribute of the formset itself. When rendering a formset in a template, you
can include all the management data by rendering {{ my_formset.management_form }} (substituting the
name of your formset as appropriate).
total_form_count and initial_form_count BaseFormSet has a couple of methods that are closely
related to the ManagementForm, total_form_count and initial_form_count.
total_form_count returns the total number of forms in this formset. initial_form_count returns the
number of forms in the formset that were pre-filled, and is also used to determine how many forms are required. You
will probably never need to override either of these methods, so please be sure you understand what they do before
doing so.
empty_form BaseFormSet provides an additional attribute empty_form which returns a form instance with
a prefix of __prefix__ for easier use in dynamic forms with JavaScript.
206 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-210-2048.jpg)
![Django Documentation, Release 1.5.1
Custom formset validation A formset has a clean method similar to the one on a Form class. This is where you
define your own validation that works at the formset level:
>>> from django.forms.formsets import BaseFormSet
>>> class BaseArticleFormSet(BaseFormSet):
... def clean(self):
... """Checks that no two articles have the same title."""
... if any(self.errors):
... # Don’t bother validating the formset unless each form is valid on its own
... return
... titles = []
... for i in range(0, self.total_form_count()):
... form = self.forms[i]
... title = form.cleaned_data[’title’]
... if title in titles:
... raise forms.ValidationError("Articles in a set must have distinct titles.")
... titles.append(title)
>>> ArticleFormSet = formset_factory(ArticleForm, formset=BaseArticleFormSet)
>>> data = {
... ’form-TOTAL_FORMS’: u’2’,
... ’form-INITIAL_FORMS’: u’0’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’Test’,
... ’form-0-pub_date’: u’1904-06-16’,
... ’form-1-title’: u’Test’,
... ’form-1-pub_date’: u’1912-06-23’,
... }
>>> formset = ArticleFormSet(data)
>>> formset.is_valid()
False
>>> formset.errors
[{}, {}]
>>> formset.non_form_errors()
[u’Articles in a set must have distinct titles.’]
The formset clean method is called after all the Form.clean methods have been called. The errors will be found
using the non_form_errors() method on the formset.
Dealing with ordering and deletion of forms
Common use cases with a formset is dealing with ordering and deletion of the form instances. This has been dealt
with for you. The formset_factory provides two optional parameters can_order and can_delete that will
do the extra work of adding the extra fields and providing simpler ways of getting to that data.
can_order Default: False
Lets you create a formset with the ability to order:
>>> ArticleFormSet = formset_factory(ArticleForm, can_order=True)
>>> formset = ArticleFormSet(initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> for form in formset:
... print(form.as_table())
3.4. Working with forms 207](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-211-2048.jpg)
![Django Documentation, Release 1.5.1
<tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-ORDER">Order:</label></th><td><input type="text" name="form-0-ORDER" value="<tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" value="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_<tr><th><label for="id_form-1-ORDER">Order:</label></th><td><input type="text" name="form-1-ORDER" value="<tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_<tr><th><label for="id_form-2-ORDER">Order:</label></th><td><input type="text" name="form-2-ORDER" id="This adds an additional field to each form. This new field is named ORDER and is an forms.IntegerField.
For the forms that came from the initial data it automatically assigned them a numeric value. Let’s look at what will
happen when the user changes these values:
>>> data = {
... ’form-TOTAL_FORMS’: u’3’,
... ’form-INITIAL_FORMS’: u’2’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’Article #1’,
... ’form-0-pub_date’: u’2008-05-10’,
... ’form-0-ORDER’: u’2’,
... ’form-1-title’: u’Article #2’,
... ’form-1-pub_date’: u’2008-05-11’,
... ’form-1-ORDER’: u’1’,
... ’form-2-title’: u’Article #3’,
... ’form-2-pub_date’: u’2008-05-01’,
... ’form-2-ORDER’: u’0’,
... }
>>> formset = ArticleFormSet(data, initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> formset.is_valid()
True
>>> for form in formset.ordered_forms:
... print(form.cleaned_data)
{’pub_date’: datetime.date(2008, 5, 1), ’ORDER’: 0, ’title’: u’Article #3’}
{’pub_date’: datetime.date(2008, 5, 11), ’ORDER’: 1, ’title’: u’Article #2’}
{’pub_date’: datetime.date(2008, 5, 10), ’ORDER’: 2, ’title’: u’Article #1’}
can_delete Default: False
Lets you create a formset with the ability to delete:
>>> ArticleFormSet = formset_factory(ArticleForm, can_delete=True)
>>> formset = ArticleFormSet(initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> for form in formset:
.... print(form.as_table())
<input type="hidden" name="form-TOTAL_FORMS" value="3" id="id_form-TOTAL_FORMS" /><input type="hidden" <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-DELETE">Delete:</label></th><td><input type="checkbox" name="form-0-DELETE" <tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" value="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_208 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-212-2048.jpg)
![Django Documentation, Release 1.5.1
<tr><th><label for="id_form-1-DELETE">Delete:</label></th><td><input type="checkbox" name="form-1-DELETE" <tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_<tr><th><label for="id_form-2-DELETE">Delete:</label></th><td><input type="checkbox" name="form-2-DELETE" Similar to can_order this adds a new field to each form named DELETE and is a forms.BooleanField. When
data comes through marking any of the delete fields you can access them with deleted_forms:
>>> data = {
... ’form-TOTAL_FORMS’: u’3’,
... ’form-INITIAL_FORMS’: u’2’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’Article #1’,
... ’form-0-pub_date’: u’2008-05-10’,
... ’form-0-DELETE’: u’on’,
... ’form-1-title’: u’Article #2’,
... ’form-1-pub_date’: u’2008-05-11’,
... ’form-1-DELETE’: u’’,
... ’form-2-title’: u’’,
... ’form-2-pub_date’: u’’,
... ’form-2-DELETE’: u’’,
... }
>>> formset = ArticleFormSet(data, initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> [form.cleaned_data for form in formset.deleted_forms]
[{’DELETE’: True, ’pub_date’: datetime.date(2008, 5, 10), ’title’: u’Article #1’}]
Adding additional fields to a formset
If you need to add additional fields to the formset this can be easily accomplished. The formset base class provides
an add_fields method. You can simply override this method to add your own fields or even redefine the default
fields/attributes of the order and deletion fields:
>>> class BaseArticleFormSet(BaseFormSet):
... def add_fields(self, form, index):
... super(BaseArticleFormSet, self).add_fields(form, index)
... form.fields["my_field"] = forms.CharField()
>>> ArticleFormSet = formset_factory(ArticleForm, formset=BaseArticleFormSet)
>>> formset = ArticleFormSet()
>>> for form in formset:
... print(form.as_table())
<tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-my_field">My field:</label></th><td><input type="text" name="form-0-my_Using a formset in views and templates
Using a formset inside a view is as easy as using a regular Form class. The only thing you will want to be aware of is
making sure to use the management form inside the template. Let’s look at a sample view:
3.4. Working with forms 209](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-213-2048.jpg)
![Django Documentation, Release 1.5.1
The save() method returns the instances that have been saved to the database. If a given instance’s data didn’t change
in the bound data, the instance won’t be saved to the database and won’t be included in the return value (instances,
in the above example).
When fields are missing from the form (for example because they have been excluded), these fields will not be
set by the save() method. You can find more information about this restriction, which also holds for regular
ModelForms, in Using a subset of fields on the form.
Pass commit=False to return the unsaved model instances:
# don’t save to the database
>>> instances = formset.save(commit=False)
>>> for instance in instances:
... # do something with instance
... instance.save()
This gives you the ability to attach data to the instances before saving them to the database. If your formset contains
a ManyToManyField, you’ll also need to call formset.save_m2m() to ensure the many-to-many relationships
are saved properly.
Limiting the number of editable objects As with regular formsets, you can use the max_num and extra param-eters
to modelformset_factory() to limit the number of extra forms displayed.
max_num does not prevent existing objects from being displayed:
>>> Author.objects.order_by(’name’)
[<Author: Charles Baudelaire>, <Author: Paul Verlaine>, <Author: Walt Whitman>]
>>> AuthorFormSet = modelformset_factory(Author, max_num=1)
>>> formset = AuthorFormSet(queryset=Author.objects.order_by(’name’))
>>> [x.name for x in formset.get_queryset()]
[u’Charles Baudelaire’, u’Paul Verlaine’, u’Walt Whitman’]
If the value of max_num is greater than the number of existing related objects, up to extra additional blank forms
will be added to the formset, so long as the total number of forms does not exceed max_num:
>>> AuthorFormSet = modelformset_factory(Author, max_num=4, extra=2)
>>> formset = AuthorFormSet(queryset=Author.objects.order_by(’name’))
>>> for form in formset:
... print(form.as_table())
<tr><th><label for="id_form-0-name">Name:</label></th><td><input id="id_form-0-name" type="text" name="<tr><th><label for="id_form-1-name">Name:</label></th><td><input id="id_form-1-name" type="text" name="<tr><th><label for="id_form-2-name">Name:</label></th><td><input id="id_form-2-name" type="text" name="<tr><th><label for="id_form-3-name">Name:</label></th><td><input id="id_form-3-name" type="text" name="A max_num value of None (the default) puts a high limit on the number of forms displayed (1000). In practice this
is equivalent to no limit.
Using a model formset in a view Model formsets are very similar to formsets. Let’s say we want to present a
formset to edit Author model instances:
def manage_authors(request):
AuthorFormSet = modelformset_factory(Author)
if request.method == ’POST’:
formset = AuthorFormSet(request.POST, request.FILES)
if formset.is_valid():
formset.save()
# do something.
220 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-224-2048.jpg)
![Django Documentation, Release 1.5.1
However, Media objects have some other interesting properties.
Media subsets If you only want media of a particular type, you can use the subscript operator to filter out a medium
of interest. For example:
>>> w = CalendarWidget()
>>> print(w.media)
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
<script type="text/javascript" src="http://static.example.com/animations.js"></script>
<script type="text/javascript" src="http://static.example.com/actions.js"></script>
>>> print(w.media)[’css’]
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
When you use the subscript operator, the value that is returned is a new Media object – but one that only contains the
media of interest.
Combining media objects Media objects can also be added together. When two media objects are added, the
resulting Media object contains the union of the media from both files:
>>> class CalendarWidget(forms.TextInput):
... class Media:
... css = {
... ’all’: (’pretty.css’,)
... }
... js = (’animations.js’, ’actions.js’)
>>> class OtherWidget(forms.TextInput):
... class Media:
... js = (’whizbang.js’,)
>>> w1 = CalendarWidget()
>>> w2 = OtherWidget()
>>> print(w1.media + w2.media)
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
<script type="text/javascript" src="http://static.example.com/animations.js"></script>
<script type="text/javascript" src="http://static.example.com/actions.js"></script>
<script type="text/javascript" src="http://static.example.com/whizbang.js"></script>
Media on Forms
Widgets aren’t the only objects that can have media definitions – forms can also define media. The rules for media
definitions on forms are the same as the rules for widgets: declarations can be static or dynamic; path and inheritance
rules for those declarations are exactly the same.
Regardless of whether you define a media declaration, all Form objects have a media property. The default value for
this property is the result of adding the media definitions for all widgets that are part of the form:
>>> class ContactForm(forms.Form):
... date = DateField(widget=CalendarWidget)
... name = CharField(max_length=40, widget=OtherWidget)
>>> f = ContactForm()
>>> f.media
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
<script type="text/javascript" src="http://static.example.com/animations.js"></script>
3.4. Working with forms 227](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-231-2048.jpg)
![Django Documentation, Release 1.5.1
Because dictionary lookup happens first, that behavior kicks in and provides a default value instead of using the
intended .iteritems() method. In this case, consider converting to a dictionary first.
In the above example, {{ section.title }} will be replaced with the title attribute of the section object.
If you use a variable that doesn’t exist, the template system will insert the value of the
TEMPLATE_STRING_IF_INVALID setting, which is set to ” (the empty string) by default.
Note that “bar” in a template expression like {{ foo.bar }} will be interpreted as a literal string and not using the
value of the variable “bar”, if one exists in the template context.
3.5.3 Filters
You can modify variables for display by using filters.
Filters look like this: {{ name|lower }}. This displays the value of the {{ name }} variable after being
filtered through the lower filter, which converts text to lowercase. Use a pipe (|) to apply a filter.
Filters can be “chained.” The output of one filter is applied to the next. {{ text|escape|linebreaks }} is a
common idiom for escaping text contents, then converting line breaks to <p> tags.
Some filters take arguments. A filter argument looks like this: {{ bio|truncatewords:30 }}. This will
display the first 30 words of the bio variable.
Filter arguments that contain spaces must be quoted; for example, to join a list with commas and spaced you’d use {{
list|join:", " }}.
Django provides about thirty built-in template filters. You can read all about them in the built-in filter reference. To
give you a taste of what’s available, here are some of the more commonly used template filters:
default If a variable is false or empty, use given default. Otherwise, use the value of the variable
For example:
{{ value|default:"nothing" }}
If value isn’t provided or is empty, the above will display “nothing”.
length Returns the length of the value. This works for both strings and lists; for example:
{{ value|length }}
If value is [’a’, ’b’, ’c’, ’d’], the output will be 4.
striptags Strips all [X]HTML tags. For example:
{{ value|striptags }}
If value is "<b>Joel</b> <button>is</button> a <span>slug</span>", the output will be
"Joel is a slug".
Again, these are just a few examples; see the built-in filter reference for the complete list.
You can also create your own custom template filters; see Custom template tags and filters.
See Also:
Django’s admin interface can include a complete reference of all template tags and filters available for a given site.
See The Django admin documentation generator.
230 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-234-2048.jpg)
![Django Documentation, Release 1.5.1
Let’s start by looking at some examples of showing a list of objects or an individual object.
We’ll be using these models:
# models.py
from django.db import models
class Publisher(models.Model):
name = models.CharField(max_length=30)
address = models.CharField(max_length=50)
city = models.CharField(max_length=60)
state_province = models.CharField(max_length=30)
country = models.CharField(max_length=50)
website = models.URLField()
class Meta:
ordering = ["-name"]
def __unicode__(self):
return self.name
class Book(models.Model):
title = models.CharField(max_length=100)
authors = models.ManyToManyField(’Author’)
publisher = models.ForeignKey(Publisher)
publication_date = models.DateField()
Now we need to define a view:
# views.py
from django.views.generic import ListView
from books.models import Publisher
class PublisherList(ListView):
model = Publisher
Finally hook that view into your urls:
# urls.py
from django.conf.urls import patterns, url
from books.views import PublisherList
urlpatterns = patterns(’’,
url(r’^publishers/$’, PublisherList.as_view()),
)
That’s all the Python code we need to write. We still need to write a template, however. We could explicitly
tell the view which template to use by adding a template_name attribute to the view, but in the absence of
an explicit template Django will infer one from the object’s name. In this case, the inferred template will be
"books/publisher_list.html" – the “books” part comes from the name of the app that defines the model,
while the “publisher” bit is just the lowercased version of the model’s name.
Note: Thus, when (for example) the django.template.loaders.app_directories.Loader
template loader is enabled in TEMPLATE_LOADERS, a template location could be:
/path/to/project/books/templates/books/publisher_list.html
This template will be rendered against a context containing a variable called object_list that contains all the
publisher objects. A very simple template might look like the following:
3.6. Class-based views 243](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-247-2048.jpg)
![Django Documentation, Release 1.5.1
def get_context_data(self, **kwargs):
# Call the base implementation first to get a context
context = super(PublisherDetail, self).get_context_data(**kwargs)
# Add in a QuerySet of all the books
context[’book_list’] = Book.objects.all()
return context
Note: Generally, get_context_data will merge the context data of all parent classes with those of the current class.
To preserve this behavior in your own classes where you want to alter the context, you should be sure to call
get_context_data on the super class. When no two classes try to define the same key, this will give the expected
results. However if any class attempts to override a key after parent classes have set it (after the call to super), any
children of that class will also need to explictly set it after super if they want to be sure to override all parents. If you’re
having trouble, review the method resolution order of your view.
Viewing subsets of objects
Now let’s take a closer look at the model argument we’ve been using all along. The model argument, which specifies
the database model that the view will operate upon, is available on all the generic views that operate on a single object
or a collection of objects. However, the model argument is not the only way to specify the objects that the view will
operate upon – you can also specify the list of objects using the queryset argument:
from django.views.generic import DetailView
from books.models import Publisher, Book
class PublisherDetail(DetailView):
context_object_name = ’publisher’
queryset = Publisher.objects.all()
Specifying model = Publisher is really just shorthand for saying queryset =
Publisher.objects.all(). However, by using queryset to define a filtered list of objects you can
be more specific about the objects that will be visible in the view (see Making queries for more information about
QuerySet objects, and see the class-based views reference for the complete details).
To pick a simple example, we might want to order a list of books by publication date, with the most recent first:
from django.views.generic import ListView
from books.models import Book
class BookList(ListView):
queryset = Book.objects.order_by(’-publication_date’)
context_object_name = ’book_list’
That’s a pretty simple example, but it illustrates the idea nicely. Of course, you’ll usually want to do more than just
reorder objects. If you want to present a list of books by a particular publisher, you can use the same technique:
from django.views.generic import ListView
from books.models import Book
class AcmeBookList(ListView):
context_object_name = ’book_list’
queryset = Book.objects.filter(publisher__name=’Acme Publishing’)
template_name = ’books/acme_list.html’
3.6. Class-based views 245](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-249-2048.jpg)
![Django Documentation, Release 1.5.1
Notice that along with a filtered queryset, we’re also using a custom template name. If we didn’t, the generic view
would use the same template as the “vanilla” object list, which might not be what we want.
Also notice that this isn’t a very elegant way of doing publisher-specific books. If we want to add another publisher
page, we’d need another handful of lines in the URLconf, and more than a few publishers would get unreasonable.
We’ll deal with this problem in the next section.
Note: If you get a 404 when requesting /books/acme/, check to ensure you actually have a Publisher with the
name ‘ACME Publishing’. Generic views have an allow_empty parameter for this case. See the class-based-views
reference for more details.
Dynamic filtering
Another common need is to filter down the objects given in a list page by some key in the URL. Earlier we hard-coded
the publisher’s name in the URLconf, but what if we wanted to write a view that displayed all the books by some
arbitrary publisher?
Handily, the ListView has a get_queryset() method we can override. Previously, it has just been returning the
value of the queryset attribute, but now we can add more logic.
The key part to making this work is that when class-based views are called, various useful things are stored on self; as
well as the request (self.request) this includes the positional (self.args) and name-based (self.kwargs)
arguments captured according to the URLconf.
Here, we have a URLconf with a single captured group:
# urls.py
from books.views import PublisherBookList
urlpatterns = patterns(’’,
(r’^books/([w-]+)/$’, PublisherBookList.as_view()),
)
Next, we’ll write the PublisherBookList view itself:
# views.py
from django.shortcuts import get_object_or_404
from django.views.generic import ListView
from books.models import Book, Publisher
class PublisherBookList(ListView):
template_name = ’books/books_by_publisher.html’
def get_queryset(self):
self.publisher = get_object_or_404(Publisher, name=self.args[0])
return Book.objects.filter(publisher=self.publisher)
As you can see, it’s quite easy to add more logic to the queryset selection; if we wanted, we could use
self.request.user to filter using the current user, or other more complex logic.
We can also add the publisher into the context at the same time, so we can use it in the template:
# ...
def get_context_data(self, **kwargs):
# Call the base implementation first to get a context
context = super(PublisherBookList, self).get_context_data(**kwargs)
246 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-250-2048.jpg)
![Django Documentation, Release 1.5.1
# Add in the publisher
context[’publisher’] = self.publisher
return context
Performing extra work
The last common pattern we’ll look at involves doing some extra work before or after calling the generic view.
Imagine we had a last_accessed field on our Author object that we were using to keep track of the last time
anybody looked at that author:
# models.py
class Author(models.Model):
salutation = models.CharField(max_length=10)
name = models.CharField(max_length=200)
email = models.EmailField()
headshot = models.ImageField(upload_to=’/tmp’)
last_accessed = models.DateTimeField()
The generic DetailView class, of course, wouldn’t know anything about this field, but once again we could easily
write a custom view to keep that field updated.
First, we’d need to add an author detail bit in the URLconf to point to a custom view:
from books.views import AuthorDetailView
urlpatterns = patterns(’’,
#...
url(r’^authors/(?P<pk>d+)/$’, AuthorDetailView.as_view(), name=’author-detail’),
)
Then we’d write our new view – get_object is the method that retrieves the object – so we simply override it and
wrap the call:
from django.views.generic import DetailView
from django.shortcuts import get_object_or_404
from django.utils import timezone
from books.models import Author
class AuthorDetailView(DetailView):
queryset = Author.objects.all()
def get_object(self):
# Call the superclass
object = super(AuthorDetailView, self).get_object()
# Record the last accessed date
object.last_accessed = timezone.now()
object.save()
# Return the object
return object
Note: The URLconf here uses the named group pk - this name is the default name that DetailView uses to find
the value of the primary key used to filter the queryset.
If you want to call the group something else, you can set pk_url_kwarg on the view. More details can be found in
the reference for DetailView
3.6. Class-based views 247](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-251-2048.jpg)
![Django Documentation, Release 1.5.1
Note that you’ll need to decorate this view using login_required(), or alternatively handle unauthorized users
in the form_valid().
AJAX example
Here is a simple example showing how you might go about implementing a form that works for AJAX requests as
well as ‘normal’ form POSTs:
import json
from django.http import HttpResponse
from django.views.generic.edit import CreateView
class AjaxableResponseMixin(object):
"""
Mixin to add AJAX support to a form.
Must be used with an object-based FormView (e.g. CreateView)
"""
def render_to_json_response(self, context, **response_kwargs):
data = json.dumps(context)
response_kwargs[’content_type’] = ’application/json’
return HttpResponse(data, **response_kwargs)
def form_invalid(self, form):
response = super(AjaxableResponseMixin, self).form_invalid(form)
if self.request.is_ajax():
return self.render_to_json_response(form.errors, status=400)
else:
return response
def form_valid(self, form):
# We make sure to call the parent’s form_valid() method because
# it might do some processing (in the case of CreateView, it will
# call form.save() for example).
response = super(AjaxableResponseMixin, self).form_valid(form)
if self.request.is_ajax():
data = {
’pk’: self.object.pk,
}
return self.render_to_json_response(data)
else:
return response
class AuthorCreate(AjaxableResponseMixin, CreateView):
model = Author
3.6.4 Using mixins with class-based views
Caution: This is an advanced topic. A working knowledge of Django’s class-based views is advised before
exploring these techniques.
Django’s built-in class-based views provide a lot of functionality, but some of it you may want to use separately.
For instance, you may want to write a view that renders a template to make the HTTP response, but you can’t use
TemplateView; perhaps you need to render a template only on POST, with GET doing something else entirely.
While you could use TemplateResponse directly, this will likely result in duplicate code.
3.6. Class-based views 251](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-255-2048.jpg)
![Django Documentation, Release 1.5.1
explictly ensure the Publisher is in the context data. ListView will add in the suitable page_obj and paginator
for us providing we remember to call super().
Now we can write a new PublisherDetail:
from django.views.generic import ListView
from django.views.generic.detail import SingleObjectMixin
from books.models import Publisher
class PublisherDetail(SingleObjectMixin, ListView):
paginate_by = 2
template_name = "books/publisher_detail.html"
def get_context_data(self, **kwargs):
kwargs[’publisher’] = self.object
return super(PublisherDetail, self).get_context_data(**kwargs)
def get_queryset(self):
self.object = self.get_object(Publisher.objects.all())
return self.object.book_set.all()
Notice how we set self.object within get_queryset() so we can use it again later in
get_context_data(). If you don’t set template_name, the template will default to the normal ListView
choice, which in this case would be "books/book_list.html" because it’s a list of books; ListView knows
nothing about SingleObjectMixin, so it doesn’t have any clue this view is anything to do with a Publisher.
The paginate_by is deliberately small in the example so you don’t have to create lots of books to see the pagination
working! Here’s the template you’d want to use:
{% extends "base.html" %}
{% block content %}
<h2>Publisher {{ publisher.name }}</h2>
<ol>
{% for book in page_obj %}
<li>{{ book.title }}</li>
{% endfor %}
</ol>
<div class="pagination">
<span class="step-links">
{% if page_obj.has_previous %}
<a href="?page={{ page_obj.previous_page_number }}">previous</a>
{% endif %}
<span class="current">
Page {{ page_obj.number }} of {{ paginator.num_pages }}.
</span>
{% if page_obj.has_next %}
<a href="?page={{ page_obj.next_page_number }}">next</a>
{% endif %}
</span>
</div>
{% endblock %}
3.6. Class-based views 255](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-259-2048.jpg)
![Django Documentation, Release 1.5.1
def get(self, request, *args, **kwargs):
view = AuthorDisplay.as_view()
return view(request, *args, **kwargs)
def post(self, request, *args, **kwargs):
view = AuthorInterest.as_view()
return view(request, *args, **kwargs)
This approach can also be used with any other generic class-based views or your own class-based views inheriting
directly from View or TemplateView, as it keeps the different views as separate as possible.
More than just HTML
Where class based views shine is when you want to do the same thing many times. Suppose you’re writing an API,
and every view should return JSON instead of rendered HTML.
We can create a mixin class to use in all of our views, handling the conversion to JSON once.
For example, a simple JSON mixin might look something like this:
import json
from django.http import HttpResponse
class JSONResponseMixin(object):
"""
A mixin that can be used to render a JSON response.
"""
response_class = HttpResponse
def render_to_response(self, context, **response_kwargs):
"""
Returns a JSON response, transforming ’context’ to make the payload.
"""
response_kwargs[’content_type’] = ’application/json’
return self.response_class(
self.convert_context_to_json(context),
**response_kwargs
)
def convert_context_to_json(self, context):
"Convert the context dictionary into a JSON object"
# Note: This is *EXTREMELY* naive; in reality, you’ll need
# to do much more complex handling to ensure that arbitrary
# objects -- such as Django model instances or querysets
# -- can be serialized as JSON.
return json.dumps(context)
Now we mix this into the base TemplateView:
from django.views.generic import TemplateView
class JSONView(JSONResponseMixin, TemplateView):
pass
Equally we could use our mixin with one of the generic views. We can make our own version of DetailView
by mixing JSONResponseMixin with the django.views.generic.detail.BaseDetailView – (the
DetailView before template rendering behavior has been mixed in):
3.6. Class-based views 259](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-263-2048.jpg)
![Django Documentation, Release 1.5.1
# some_app/views.py
from django.views.generic import TemplateView
class AboutView(TemplateView):
template_name = "about.html"
Then we just need to add this new view into our URLconf. ~django.views.generic.base.TemplateView is a class, not
a function, so we point the URL to the as_view() class method instead, which provides a function-like entry to
class-based views:
# urls.py
from django.conf.urls import patterns
from some_app.views import AboutView
urlpatterns = patterns(’’,
(r’^about/’, AboutView.as_view()),
)
For more information on how to use the built in generic views, consult the next topic on generic class based views.
Supporting other HTTP methods
Suppose somebody wants to access our book library over HTTP using the views as an API. The API client would
connect every now and then and download book data for the books published since last visit. But if no new books
appeared since then, it is a waste of CPU time and bandwidth to fetch the books from the database, render a full
response and send it to the client. It might be preferable to ask the API when the most recent book was published.
We map the URL to book list view in the URLconf:
from django.conf.urls import patterns
from books.views import BookListView
urlpatterns = patterns(’’,
(r’^books/$’, BookListView.as_view()),
)
And the view:
from django.http import HttpResponse
from django.views.generic import ListView
from books.models import Book
class BookListView(ListView):
model = Book
def head(self, *args, **kwargs):
last_book = self.get_queryset().latest(’publication_date’)
response = HttpResponse(’’)
# RFC 1123 date format
response[’Last-Modified’] = last_book.publication_date.strftime(’%a, %d %b %Y %H:%M:%S GMT’)
return response
If the view is accessed from a GET request, a plain-and-simple object list is returned in the response (using
book_list.html template). But if the client issues a HEAD request, the response has an empty body and the
Last-Modified header indicates when the most recent book was published. Based on this information, the client
may or may not download the full object list.
3.6. Class-based views 261](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-265-2048.jpg)
![Django Documentation, Release 1.5.1
Now you can use any of the documented attributes and methods of the File class.
Be aware that files created in this way are not automatically closed. The following approach may be used to close files
automatically:
>>> from django.core.files import File
# Create a Python file object using open() and the with statement
>>> with open(’/tmp/hello.world’, ’w’) as f:
>>> myfile = File(f)
>>> for line in myfile:
>>> print line
>>> myfile.closed
True
>>> f.closed
True
Closing files is especially important when accessing file fields in a loop over a large number of objects:: If files are
not manually closed after accessing them, the risk of running out of file descriptors may arise. This may lead to the
following error:
IOError: [Errno 24] Too many open files
3.7.3 File storage
Behind the scenes, Django delegates decisions about how and where to store files to a file storage system. This is the
object that actually understands things like file systems, opening and reading files, etc.
Django’s default file storage is given by the DEFAULT_FILE_STORAGE setting; if you don’t explicitly provide a
storage system, this is the one that will be used.
See below for details of the built-in default file storage system, and see Writing a custom storage system for information
on writing your own file storage system.
Storage objects
Though most of the time you’ll want to use a File object (which delegates to the proper storage for that file), you can
use file storage systems directly. You can create an instance of some custom file storage class, or – often more useful
– you can use the global default storage system:
>>> from django.core.files.storage import default_storage
>>> from django.core.files.base import ContentFile
>>> path = default_storage.save(’/path/to/file’, ContentFile(’new content’))
>>> path
u’/path/to/file’
>>> default_storage.size(path)
11
>>> default_storage.open(path).read()
’new content’
>>> default_storage.delete(path)
>>> default_storage.exists(path)
False
See File storage API for the file storage API.
3.7. Managing files 263](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-267-2048.jpg)
![Django Documentation, Release 1.5.1
The headers sent via **extra should follow CGI specification. For example, emulating a different
“Host” header as sent in the HTTP request from the browser to the server should be passed as HTTP_HOST.
If you already have the GET arguments in URL-encoded form, you can use that encoding instead of using
the data argument. For example, the previous GET request could also be posed as:
>>> c = Client()
>>> c.get(’/customers/details/?name=fred&age=7’)
If you provide a URL with both an encoded GET data and a data argument, the data argument will take
precedence.
If you set follow to True the client will follow any redirects and a redirect_chain attribute will
be set in the response object containing tuples of the intermediate urls and status codes.
If you had a URL /redirect_me/ that redirected to /next/, that redirected to /final/, this is what
you’d see:
>>> response = c.get(’/redirect_me/’, follow=True)
>>> response.redirect_chain
[(u’http://testserver/next/’, 302), (u’http://testserver/final/’, 302)]
post(path, data={}, content_type=MULTIPART_CONTENT, follow=False, **extra)
Makes a POST request on the provided path and returns a Response object, which is documented
below.
The key-value pairs in the data dictionary are used to submit POST data. For example:
>>> c = Client()
>>> c.post(’/login/’, {’name’: ’fred’, ’passwd’: ’secret’})
...will result in the evaluation of a POST request to this URL:
/login/
...with this POST data:
name=fred&passwd=secret
If you provide content_type (e.g. text/xml for an XML payload), the contents of data will be
sent as-is in the POST request, using content_type in the HTTP Content-Type header.
If you don’t provide a value for content_type, the values in data will be transmitted with a con-tent
type of multipart/form-data. In this case, the key-value pairs in data will be encoded as a
multipart message and used to create the POST data payload.
To submit multiple values for a given key – for example, to specify the selections for a <select
multiple> – provide the values as a list or tuple for the required key. For example, this value of data
would submit three selected values for the field named choices:
{’choices’: (’a’, ’b’, ’d’)}
Submitting files is a special case. To POST a file, you need only provide the file field name as a key, and a
file handle to the file you wish to upload as a value. For example:
>>> c = Client()
>>> with open(’wishlist.doc’) as fp:
... c.post(’/customers/wishes/’, {’name’: ’fred’, ’attachment’: fp})
(The name attachment here is not relevant; use whatever name your file-processing code expects.)
3.8. Testing in Django 271](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-275-2048.jpg)
![Django Documentation, Release 1.5.1
>>> c = Client()
>>> c.login(username=’fred’, password=’secret’)
# Now you can access a view that’s only available to logged-in users.
If you’re using a different authentication backend, this method may require different credentials. It requires
whichever credentials are required by your backend’s authenticate() method.
login() returns True if it the credentials were accepted and login was successful.
Finally, you’ll need to remember to create user accounts before you can use this method. As we explained
above, the test runner is executed using a test database, which contains no users by default. As a result,
user accounts that are valid on your production site will not work under test conditions. You’ll need to
create users as part of the test suite – either manually (using the Django model API) or with a test fixture.
Remember that if you want your test user to have a password, you can’t set the user’s password by setting
the password attribute directly – you must use the set_password() function to store a correctly hashed
password. Alternatively, you can use the create_user() helper method to create a new user with a
correctly hashed password.
logout()
If your site uses Django’s authentication system, the logout() method can be used to simulate the effect
of a user logging out of your site.
After you call this method, the test client will have all the cookies and session data cleared to defaults.
Subsequent requests will appear to come from an AnonymousUser.
Testing responses The get() and post() methods both return a Response object. This Response object is
not the same as the HttpResponse object returned Django views; the test response object has some additional data
useful for test code to verify.
Specifically, a Response object has the following attributes:
class Response
client
The test client that was used to make the request that resulted in the response.
content
The body of the response, as a string. This is the final page content as rendered by the view, or any error
message.
context
The template Context instance that was used to render the template that produced the response content.
If the rendered page used multiple templates, then context will be a list of Context objects, in the
order in which they were rendered.
Regardless of the number of templates used during rendering, you can retrieve context values using the []
operator. For example, the context variable name could be retrieved using:
>>> response = client.get(’/foo/’)
>>> response.context[’name’]
’Arthur’
request
The request data that stimulated the response.
status_code
The HTTP status of the response, as an integer. See RFC 2616 for a full list of HTTP status codes.
3.8. Testing in Django 273](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-277-2048.jpg)
![Django Documentation, Release 1.5.1
templates
A list of Template instances used to render the final content, in the order they were rendered. For each
template in the list, use template.name to get the template’s file name, if the template was loaded from
a file. (The name is a string such as ’admin/index.html’.)
You can also use dictionary syntax on the response object to query the value of any settings in the HTTP headers. For
example, you could determine the content type of a response using response[’Content-Type’].
Exceptions If you point the test client at a view that raises an exception, that exception will be visible in the test
case. You can then use a standard try ... except block or assertRaises() to test for exceptions.
The only exceptions that are not visible to the test client are Http404, PermissionDenied and SystemExit.
Django catches these exceptions internally and converts them into the appropriate HTTP response codes. In these
cases, you can check response.status_code in your test.
Persistent state The test client is stateful. If a response returns a cookie, then that cookie will be stored in the test
client and sent with all subsequent get() and post() requests.
Expiration policies for these cookies are not followed. If you want a cookie to expire, either delete it manually or
create a new Client instance (which will effectively delete all cookies).
A test client has two attributes that store persistent state information. You can access these properties as part of a test
condition.
Client.cookies
A Python SimpleCookie object, containing the current values of all the client cookies. See the documentation
of the Cookie module for more.
Client.session
A dictionary-like object containing session information. See the session documentation for full details.
To modify the session and then save it, it must be stored in a variable first (because a new SessionStore is
created every time this property is accessed):
def test_something(self):
session = self.client.session
session[’somekey’] = ’test’
session.save()
Example The following is a simple unit test using the test client:
from django.utils import unittest
from django.test.client import Client
class SimpleTest(unittest.TestCase):
def setUp(self):
# Every test needs a client.
self.client = Client()
def test_details(self):
# Issue a GET request.
response = self.client.get(’/customer/details/’)
# Check that the response is 200 OK.
self.assertEqual(response.status_code, 200)
# Check that the rendered context contains 5 customers.
self.assertEqual(len(response.context[’customers’]), 5)
274 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-278-2048.jpg)
![Django Documentation, Release 1.5.1
LiveServerTestCase New in version 1.4.
class LiveServerTestCase
LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches
a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test
clients other than the Django dummy client such as, for example, the Selenium client, to execute a series of functional
tests inside a browser and simulate a real user’s actions.
By default the live server’s address is ’localhost:8081’ and the full URL can be accessed during the tests with
self.live_server_url. If you’d like to change the default address (in the case, for example, where the 8081
port is already taken) then you may pass a different one to the test command via the --liveserver option, for
example:
./manage.py test --liveserver=localhost:8082
Another way of changing the default server address is by setting the DJANGO_LIVE_TEST_SERVER_ADDRESS
environment variable somewhere in your code (for example, in a custom test runner):
import os
os.environ[’DJANGO_LIVE_TEST_SERVER_ADDRESS’] = ’localhost:8082’
In the case where the tests are run by multiple processes in parallel (for example, in the context of several simulta-neous
continuous integration builds), the processes will compete for the same address, and therefore your tests might
randomly fail with an “Address already in use” error. To avoid this problem, you can pass a comma-separated list of
ports or ranges of ports (at least as many as the number of potential parallel processes). For example:
./manage.py test --liveserver=localhost:8082,8090-8100,9000-9200,7041
Then, during test execution, each new live test server will try every specified port until it finds one that is free and
takes it.
To demonstrate how to use LiveServerTestCase, let’s write a simple Selenium test. First of all, you need to
install the selenium package into your Python path:
pip install selenium
Then, add a LiveServerTestCase-based test to your app’s tests module (for example: myapp/tests.py).
The code for this test may look as follows:
from django.test import LiveServerTestCase
from selenium.webdriver.firefox.webdriver import WebDriver
class MySeleniumTests(LiveServerTestCase):
fixtures = [’user-data.json’]
@classmethod
def setUpClass(cls):
cls.selenium = WebDriver()
super(MySeleniumTests, cls).setUpClass()
@classmethod
def tearDownClass(cls):
cls.selenium.quit()
super(MySeleniumTests, cls).tearDownClass()
def test_login(self):
self.selenium.get(’%s%s’ % (self.live_server_url, ’/login/’))
username_input = self.selenium.find_element_by_name("username")
username_input.send_keys(’myuser’)
278 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-282-2048.jpg)
![Django Documentation, Release 1.5.1
password_input = self.selenium.find_element_by_name("password")
password_input.send_keys(’secret’)
self.selenium.find_element_by_xpath(’//input[@value="Log in"]’).click()
Finally, you may run the test as follows:
./manage.py test myapp.MySeleniumTests.test_login
This example will automatically open Firefox then go to the login page, enter the credentials and press the “Log in”
button. Selenium offers other drivers in case you do not have Firefox installed or wish to use another browser. The
example above is just a tiny fraction of what the Selenium client can do; check out the full reference for more details.
Note: LiveServerTestCase makes use of the staticfiles contrib app so you’ll need to have your project config-ured
accordingly (in particular by setting STATIC_URL).
Note: When using an in-memory SQLite database to run the tests, the same database connection will be shared
by two threads in parallel: the thread in which the live server is run and the thread in which the test case is run.
It’s important to prevent simultaneous database queries via this shared connection by the two threads, as that may
sometimes randomly cause the tests to fail. So you need to ensure that the two threads don’t access the database at the
same time. In particular, this means that in some cases (for example, just after clicking a link or submitting a form),
you might need to check that a response is received by Selenium and that the next page is loaded before proceeding
with further test execution. Do this, for example, by making Selenium wait until the <body> HTML tag is found in
the response (requires Selenium > 2.13):
def test_login(self):
from selenium.webdriver.support.wait import WebDriverWait
timeout = 2
...
self.selenium.find_element_by_xpath(’//input[@value="Log in"]’).click()
# Wait until the response is received
WebDriverWait(self.selenium, timeout).until(
lambda driver: driver.find_element_by_tag_name(’body’))
The tricky thing here is that there’s really no such thing as a “page load,” especially in modern Web apps that generate
HTML dynamically after the server generates the initial document. So, simply checking for the presence of <body>
in the response might not necessarily be appropriate for all use cases. Please refer to the Selenium FAQ and Selenium
documentation for more information.
Test cases features
Default test client
TestCase.client
Every test case in a django.test.TestCase instance has access to an instance of a Django test client. This client
can be accessed as self.client. This client is recreated for each test, so you don’t have to worry about state (such
as cookies) carrying over from one test to another.
This means, instead of instantiating a Client in each test:
from django.utils import unittest
from django.test.client import Client
class SimpleTest(unittest.TestCase):
def test_details(self):
client = Client()
3.8. Testing in Django 279](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-283-2048.jpg)
![Django Documentation, Release 1.5.1
Initial SQL data and testing
Django provides a second way to insert initial data into models – the custom SQL hook. However, this technique
cannot be used to provide initial data for testing purposes. Django’s test framework flushes the contents of the test
database after each test; as a result, any data added using the custom SQL hook will be lost.
Once you’ve created a fixture and placed it in a fixtures directory in one of your INSTALLED_APPS, you can
use it in your unit tests by specifying a fixtures class attribute on your django.test.TestCase subclass:
from django.test import TestCase
from myapp.models import Animal
class AnimalTestCase(TestCase):
fixtures = [’mammals.json’, ’birds’]
def setUp(self):
# Test definitions as before.
call_setup_methods()
def testFluffyAnimals(self):
# A test that uses the fixtures.
call_some_test_code()
Here’s specifically what will happen:
• At the start of each test case, before setUp() is run, Django will flush the database, returning the database to
the state it was in directly after syncdb was called.
• Then, all the named fixtures are installed. In this example, Django will install any JSON fixture named
mammals, followed by any fixture named birds. See the loaddata documentation for more details on
defining and installing fixtures.
This flush/load procedure is repeated for each test in the test case, so you can be certain that the outcome of a test will
not be affected by another test, or by the order of test execution.
URLconf configuration
TestCase.urls
If your application provides views, you may want to include tests that use the test client to exercise those views.
However, an end user is free to deploy the views in your application at any URL of their choosing. This means that
your tests can’t rely upon the fact that your views will be available at a particular URL.
In order to provide a reliable URL space for your test, django.test.TestCase provides the ability to customize
the URLconf configuration for the duration of the execution of a test suite. If your TestCase instance defines an
urls attribute, the TestCase will use the value of that attribute as the ROOT_URLCONF for the duration of that
test.
For example:
from django.test import TestCase
class TestMyViews(TestCase):
urls = ’myapp.test_urls’
def testIndexPageView(self):
# Here you’d test your view using ‘‘Client‘‘.
call_some_test_code()
This test case will use the contents of myapp.test_urls as the URLconf for the duration of the test case.
3.8. Testing in Django 281](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-285-2048.jpg)
![Django Documentation, Release 1.5.1
SimpleTestCase.assertFieldOutput(self, fieldclass, valid, invalid, field_args=None,
field_kwargs=None, empty_value=u’‘)
New in version 1.4. Asserts that a form field behaves correctly with various inputs.
Parameters
• fieldclass – the class of the field to be tested.
• valid – a dictionary mapping valid inputs to their expected cleaned values.
• invalid – a dictionary mapping invalid inputs to one or more raised error messages.
• field_args – the args passed to instantiate the field.
• field_kwargs – the kwargs passed to instantiate the field.
• empty_value – the expected clean output for inputs in EMPTY_VALUES.
For example, the following code tests that an EmailField accepts “a@a.com” as a valid email address, but
rejects “aaa” with a reasonable error message:
self.assertFieldOutput(EmailField, {’a@a.com’: ’a@a.com’}, {’aaa’: [u’Enter a valid email address.’]})
TestCase.assertContains(response, text, count=None, status_code=200, msg_prefix=’‘, html=False)
Asserts that a Response instance produced the given status_code and that text appears in the content
of the response. If count is provided, text must occur exactly count times in the response. New in version
1.4. Set html to True to handle text as HTML. The comparison with the response content will be based
on HTML semantics instead of character-by-character equality. Whitespace is ignored in most cases, attribute
ordering is not significant. See assertHTMLEqual() for more details.
TestCase.assertNotContains(response, text, status_code=200, msg_prefix=’‘, html=False)
Asserts that a Response instance produced the given status_code and that text does not appears in the
content of the response. New in version 1.4. Set html to True to handle text as HTML. The comparison with
the response content will be based on HTML semantics instead of character-by-character equality. Whitespace
is ignored in most cases, attribute ordering is not significant. See assertHTMLEqual() for more details.
TestCase.assertFormError(response, form, field, errors, msg_prefix=’‘)
Asserts that a field on a form raises the provided list of errors when rendered on the form.
form is the name the Form instance was given in the template context.
field is the name of the field on the form to check. If field has a value of None, non-field errors (errors
you can access via form.non_field_errors()) will be checked.
errors is an error string, or a list of error strings, that are expected as a result of form validation.
TestCase.assertTemplateUsed(response, template_name, msg_prefix=’‘)
Asserts that the template with the given name was used in rendering the response.
The name is a string such as ’admin/index.html’. New in version 1.4. You can use this as a context
manager, like this:
with self.assertTemplateUsed(’index.html’):
render_to_string(’index.html’)
with self.assertTemplateUsed(template_name=’index.html’):
render_to_string(’index.html’)
TestCase.assertTemplateNotUsed(response, template_name, msg_prefix=’‘)
Asserts that the template with the given name was not used in rendering the response. New in version 1.4. You
can use this as a context manager in the same way as assertTemplateUsed().
284 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-288-2048.jpg)
![Django Documentation, Release 1.5.1
SimpleTestCase.assertHTMLNotEqual(html1, html2, msg=None)
New in version 1.4. Asserts that the strings html1 and html2 are not equal. The comparison is based on
HTML semantics. See assertHTMLEqual() for details.
html1 and html2 must be valid HTML. An AssertionError will be raised if one of them cannot be
parsed.
SimpleTestCase.assertXMLEqual(xml1, xml2, msg=None)
New in version 1.5. Asserts that the strings xml1 and xml2 are equal. The comparison is based on XML
semantics. Similarily to assertHTMLEqual(), the comparison is made on parsed content, hence only se-mantic
differences are considered, not syntax differences. When unvalid XML is passed in any parameter, an
AssertionError is always raised, even if both string are identical.
SimpleTestCase.assertXMLNotEqual(xml1, xml2, msg=None)
New in version 1.5. Asserts that the strings xml1 and xml2 are not equal. The comparison is based on XML
semantics. See assertXMLEqual() for details.
Email services
If any of your Django views send email using Django’s email functionality, you probably don’t want to send email
each time you run a test using that view. For this reason, Django’s test runner automatically redirects all Django-sent
email to a dummy outbox. This lets you test every aspect of sending email – from the number of messages sent to the
contents of each message – without actually sending the messages.
The test runner accomplishes this by transparently replacing the normal email backend with a testing backend. (Don’t
worry – this has no effect on any other email senders outside of Django, such as your machine’s mail server, if you’re
running one.)
django.core.mail.outbox
During test running, each outgoing email is saved in django.core.mail.outbox. This is a simple list of all
EmailMessage instances that have been sent. The outbox attribute is a special attribute that is created only when
the locmem email backend is used. It doesn’t normally exist as part of the django.core.mail module and you
can’t import it directly. The code below shows how to access this attribute correctly.
Here’s an example test that examines django.core.mail.outbox for length and contents:
from django.core import mail
from django.test import TestCase
class EmailTest(TestCase):
def test_send_email(self):
# Send message.
mail.send_mail(’Subject here’, ’Here is the message.’,
’from@example.com’, [’to@example.com’],
fail_silently=False)
# Test that one message has been sent.
self.assertEqual(len(mail.outbox), 1)
# Verify that the subject of the first message is correct.
self.assertEqual(mail.outbox[0].subject, ’Subject here’)
As noted previously, the test outbox is emptied at the start of every test in a Django TestCase. To empty the outbox
manually, assign the empty list to mail.outbox:
from django.core import mail
286 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-290-2048.jpg)
![Django Documentation, Release 1.5.1
# Empty the test outbox
mail.outbox = []
Skipping tests
The unittest library provides the @skipIf and @skipUnless decorators to allow you to skip tests if you know
ahead of time that those tests are going to fail under certain conditions.
For example, if your test requires a particular optional library in order to succeed, you could decorate the test case
with @skipIf. Then, the test runner will report that the test wasn’t executed and why, instead of failing the test or
omitting the test altogether.
To supplement these test skipping behaviors, Django provides two additional skip decorators. Instead of testing a
generic boolean, these decorators check the capabilities of the database, and skip the test if the database doesn’t
support a specific named feature.
The decorators use a string identifier to describe database features. This string corresponds to attributes of the
database connection features class. See django.db.backends.BaseDatabaseFeatures class for a full list
of database features that can be used as a basis for skipping tests.
skipIfDBFeature(feature_name_string)
Skip the decorated test if the named database feature is supported.
For example, the following test will not be executed if the database supports transactions (e.g., it would not run under
PostgreSQL, but it would under MySQL with MyISAM tables):
class MyTests(TestCase):
@skipIfDBFeature(’supports_transactions’)
def test_transaction_behavior(self):
# ... conditional test code
skipUnlessDBFeature(feature_name_string)
Skip the decorated test if the named database feature is not supported.
For example, the following test will only be executed if the database supports transactions (e.g., it would run under
PostgreSQL, but not under MySQL with MyISAM tables):
class MyTests(TestCase):
@skipUnlessDBFeature(’supports_transactions’)
def test_transaction_behavior(self):
# ... conditional test code
3.8.2 Django and doctests
Doctests use Python’s standard doctest module, which searches your docstrings for statements that resemble a
session of the Python interactive interpreter. A full explanation of how doctest works is out of the scope of this
document; read Python’s official documentation for the details.
What’s a docstring?
A good explanation of docstrings (and some guidelines for using them effectively) can be found in PEP 257:
A docstring is a string literal that occurs as the first statement in a module, function, class, or method
definition. Such a docstring becomes the __doc__ special attribute of that object.
For example, this function has a docstring that describes what it does:
3.8. Testing in Django 287](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-291-2048.jpg)
![Django Documentation, Release 1.5.1
’TEST_MIRROR’: ’default’
# ... plus some other settings
}
}
In this setup, we have two database servers: dbmaster, described by the database alias default, and dbslave
described by the alias slave. As you might expect, dbslave has been configured by the database administrator as
a read slave of dbmaster, so in normal activity, any write to default will appear on slave.
If Django created two independent test databases, this would break any tests that expected replication to occur. How-ever,
the slave database has been configured as a test mirror (using the TEST_MIRROR setting), indicating that
under testing, slave should be treated as a mirror of default.
When the test environment is configured, a test version of slave will not be created. Instead the connection to slave
will be redirected to point at default. As a result, writes to default will appear on slave – but because they are
actually the same database, not because there is data replication between the two databases.
Controlling creation order for test databases
By default, Django will always create the default database first. However, no guarantees are made on the creation
order of any other databases in your test setup.
If your database configuration requires a specific creation order, you can specify the dependencies that exist using the
TEST_DEPENDENCIES setting. Consider the following (simplified) example database configuration:
DATABASES = {
’default’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’]
},
’diamonds’: {
# ... db settings
},
’clubs’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’]
},
’spades’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’,’hearts’]
},
’hearts’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’,’clubs’]
}
}
Under this configuration, the diamonds database will be created first, as it is the only database alias without de-pendencies.
The default and clubs alias will be created next (although the order of creation of this pair is not
guaranteed); then hearts; and finally spades.
If there are any circular dependencies in the TEST_DEPENDENCIES definition, an ImproperlyConfigured
exception will be raised.
290 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-294-2048.jpg)
![Django Documentation, Release 1.5.1
Returns the result produced by the running the test suite.
DjangoTestSuiteRunner.teardown_databases(old_config, **kwargs)
Destroys the test databases, restoring pre-test conditions.
old_config is a data structure defining the changes in the database configuration that need to be reversed. It
is the return value of the setup_databases() method.
DjangoTestSuiteRunner.teardown_test_environment(**kwargs)
Restores the pre-test environment.
DjangoTestSuiteRunner.suite_result(suite, result, **kwargs)
Computes and returns a return code based on a test suite, and the result from that test suite.
Testing utilities
To assist in the creation of your own test runner, Django provides a number of utility methods in the
django.test.utils module.
setup_test_environment()
Performs any global pre-test setup, such as the installing the instrumentation of the template rendering system
and setting up the dummy email outbox.
teardown_test_environment()
Performs any global post-test teardown, such as removing the black magic hooks into the template system and
restoring normal email services.
The creation module of the database backend (connection.creation) also provides some utilities that can be
useful during testing.
create_test_db([verbosity=1, autoclobber=False ])
Creates a new test database and runs syncdb against it.
verbosity has the same behavior as in run_tests().
autoclobber describes the behavior that will occur if a database with the same name as the test database is
discovered:
•If autoclobber is False, the user will be asked to approve destroying the existing database.
sys.exit is called if the user does not approve.
•If autoclobber is True, the database will be destroyed without consulting the user.
Returns the name of the test database that it created.
create_test_db() has the side effect of modifying the value of NAME in DATABASES to match the name
of the test database.
destroy_test_db(old_database_name[, verbosity=1 ])
Destroys the database whose name is the value of NAME in DATABASES, and sets NAME to the value of
old_database_name.
The verbosity argument has the same behavior as for DjangoTestSuiteRunner.
Integration with coverage.py
Code coverage describes how much source code has been tested. It shows which parts of your code are being exercised
by tests and which are not. It’s an important part of testing applications, so it’s strongly recommended to check the
coverage of your tests.
3.8. Testing in Django 293](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-297-2048.jpg)
![Django Documentation, Release 1.5.1
Django can be easily integrated with coverage.py, a tool for measuring code coverage of Python programs. First,
install coverage.py. Next, run the following from your project folder containing manage.py:
coverage run --source=’.’ manage.py test myapp
This runs your tests and collects coverage data of the executed files in your project. You can see a report of this data
by typing following command:
coverage report
Note that some Django code was executed while running tests, but it is not listed here because of the source flag
passed to the previous command.
For more options like annotated HTML listings detailing missed lines, see the coverage.py docs.
Automated testing is an extremely useful bug-killing tool for the modern Web developer. You can use a collection of
tests – a test suite – to solve, or avoid, a number of problems:
• When you’re writing new code, you can use tests to validate your code works as expected.
• When you’re refactoring or modifying old code, you can use tests to ensure your changes haven’t affected your
application’s behavior unexpectedly.
Testing a Web application is a complex task, because a Web application is made of several layers of logic – from
HTTP-level request handling, to form validation and processing, to template rendering. With Django’s test-execution
framework and assorted utilities, you can simulate requests, insert test data, inspect your application’s output and
generally verify your code is doing what it should be doing.
The best part is, it’s really easy.
3.8.4 Unit tests v. doctests
There are two primary ways to write tests with Django, corresponding to the two test frameworks that ship in the
Python standard library. The two frameworks are:
• Unit tests – tests that are expressed as methods on a Python class that subclasses unittest.TestCase or
Django’s customized TestCase. For example:
import unittest
class MyFuncTestCase(unittest.TestCase):
def testBasic(self):
a = [’larry’, ’curly’, ’moe’]
self.assertEqual(my_func(a, 0), ’larry’)
self.assertEqual(my_func(a, 1), ’curly’)
• Doctests – tests that are embedded in your functions’ docstrings and are written in a way that emulates a session
of the Python interactive interpreter. For example:
def my_func(a_list, idx):
"""
>>> a = [’larry’, ’curly’, ’moe’]
>>> my_func(a, 0)
’larry’
>>> my_func(a, 1)
’curly’
"""
return a_list[idx]
294 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-298-2048.jpg)
![Django Documentation, Release 1.5.1
If you have the Django admin installed, you can also change user’s passwords on the authentication system’s admin
pages.
Django also provides views and forms that may be used to allow users to change their own passwords.
Authenticating Users
authenticate(**credentials)
To authenticate a given username and password, use authenticate(). It takes credentials in the form of
keyword arguments, for the default configuration this is username and password, and it returns a User
object if the password is valid for the given username. If the password is invalid, authenticate() returns
None. Example:
from django.contrib.auth import authenticate
user = authenticate(username=’john’, password=’secret’)
if user is not None:
# the password verified for the user
if user.is_active:
print("User is valid, active and authenticated")
else:
print("The password is valid, but the account has been disabled!")
else:
# the authentication system was unable to verify the username and password
print("The username and password were incorrect.")
Permissions and Authorization
Django comes with a simple permissions system. It provides a way to assign permissions to specific users and groups
of users.
It’s used by the Django admin site, but you’re welcome to use it in your own code.
The Django admin site uses permissions as follows:
• Access to view the “add” form and add an object is limited to users with the “add” permission for that type of
object.
• Access to view the change list, view the “change” form and change an object is limited to users with the “change”
permission for that type of object.
• Access to delete an object is limited to users with the “delete” permission for that type of object.
Permissions can be set not only per type of object, but also per specific object instance. By using the
has_add_permission(), has_change_permission() and has_delete_permission() methods
provided by the ModelAdmin class, it is possible to customize permissions for different object instances of the
same type.
User objects have two many-to-many fields: groups and user_permissions. User objects can access their
related objects in the same way as any other Django model:
myuser.groups = [group_list]
myuser.groups.add(group, group, ...)
myuser.groups.remove(group, group, ...)
myuser.groups.clear()
myuser.user_permissions = [permission_list]
myuser.user_permissions.add(permission, permission, ...)
myuser.user_permissions.remove(permission, permission, ...)
myuser.user_permissions.clear()
3.9. User authentication in Django 297](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-301-2048.jpg)
![Django Documentation, Release 1.5.1
if request.user.is_authenticated():
# Do something for authenticated users.
else:
# Do something for anonymous users.
How to log a user in
If you have an authenticated user you want to attach to the current session - this is done with a login() function.
login()
To log a user in, from a view, use login(). It takes an HttpRequest object and a User object. login()
saves the user’s ID in the session, using Django’s session framework.
Note that any data set during the anonymous session is retained in the session after a user logs in.
This example shows how you might use both authenticate() and login():
from django.contrib.auth import authenticate, login
def my_view(request):
username = request.POST[’username’]
password = request.POST[’password’]
user = authenticate(username=username, password=password)
if user is not None:
if user.is_active:
login(request, user)
# Redirect to a success page.
else:
# Return a ’disabled account’ error message
else:
# Return an ’invalid login’ error message.
Calling authenticate() first
When you’re manually logging a user in, you must call authenticate() before you call login().
authenticate() sets an attribute on the User noting which authentication backend successfully authenticated
that user (see the backends documentation for details), and this information is needed later during the login process.
An error will be raise if you try to login a user object retrieved from the database directly.
How to log a user out
logout()
To log out a user who has been logged in via django.contrib.auth.login(), use
django.contrib.auth.logout() within your view. It takes an HttpRequest object and has
no return value. Example:
from django.contrib.auth import logout
def logout_view(request):
logout(request)
# Redirect to a success page.
Note that logout() doesn’t throw any errors if the user wasn’t logged in.
3.9. User authentication in Django 299](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-303-2048.jpg)
![Django Documentation, Release 1.5.1
When you call logout(), the session data for the current request is completely cleaned out. All existing data
is removed. This is to prevent another person from using the same Web browser to log in and have access to
the previous user’s session data. If you want to put anything into the session that will be available to the user
immediately after logging out, do that after calling django.contrib.auth.logout().
Limiting access to logged-in users
The raw way The simple, raw way to limit access to pages is to check
request.user.is_authenticated() and either redirect to a login page:
from django.shortcuts import redirect
def my_view(request):
if not request.user.is_authenticated():
return redirect(’/login/?next=%s’ % request.path)
# ...
...or display an error message:
from django.shortcuts import render
def my_view(request):
if not request.user.is_authenticated():
return render(request, ’myapp/login_error.html’)
# ...
The login_required decorator
login_required([redirect_field_name=REDIRECT_FIELD_NAME, login_url=None ])
As a shortcut, you can use the convenient login_required() decorator:
from django.contrib.auth.decorators import login_required
@login_required
def my_view(request):
...
login_required() does the following:
•If the user isn’t logged in, redirect to settings.LOGIN_URL, passing the current absolute path in the
query string. Example: /accounts/login/?next=/polls/3/.
•If the user is logged in, execute the view normally. The view code is free to assume the user is logged in.
By default, the path that the user should be redirected to upon successful authentication is stored in a
query string parameter called "next". If you would prefer to use a different name for this parameter,
login_required() takes an optional redirect_field_name parameter:
from django.contrib.auth.decorators import login_required
@login_required(redirect_field_name=’my_redirect_field’)
def my_view(request):
...
Note that if you provide a value to redirect_field_name, you will most likely need to customize your
login template as well, since the template context variable which stores the redirect path will use the value of
redirect_field_name as its key rather than "next" (the default).
login_required() also takes an optional login_url parameter. Example:
300 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-304-2048.jpg)
![Django Documentation, Release 1.5.1
from django.contrib.auth.decorators import login_required
@login_required(login_url=’/accounts/login/’)
def my_view(request):
...
Note that if you don’t specify the login_url parameter, you’ll need to ensure that the
settings.LOGIN_URL and your login view are properly associated. For example, using the defaults, add
the following line to your URLconf:
(r’^accounts/login/$’, ’django.contrib.auth.views.login’),
Changed in version 1.5. The settings.LOGIN_URL also accepts view function names and named URL
patterns. This allows you to freely remap your login view within your URLconf without having to update the
setting.
Note: The login_required decorator does NOT check the is_active flag on a user.
Limiting access to logged-in users that pass a test To limit access based on certain permissions or some other test,
you’d do essentially the same thing as described in the previous section.
The simple way is to run your test on request.user in the view directly. For example, this view checks to make
sure the user has an email in the desired domain:
def my_view(request):
if not ’@example.com’ in request.user.email:
return HttpResponse("You can’t vote in this poll.")
# ...
user_passes_test(func[, login_url=None ])
As a shortcut, you can use the convenient user_passes_test decorator:
from django.contrib.auth.decorators import user_passes_test
def email_check(user):
return ’@example.com’ in user.email
@user_passes_test(email_check)
def my_view(request):
...
user_passes_test() takes a required argument: a callable that takes a User object and returns True if
the user is allowed to view the page. Note that user_passes_test() does not automatically check that the
User is not anonymous.
user_passes_test() takes an optional login_url argument, which lets you specify the URL for your
login page (settings.LOGIN_URL by default).
For example:
@user_passes_test(email_check, login_url=’/login/’)
def my_view(request):
...
The permission_required decorator
3.9. User authentication in Django 301](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-305-2048.jpg)
![Django Documentation, Release 1.5.1
permission_required([login_url=None, raise_exception=False ])
It’s a relatively common task to check whether a user has a particular permission. For that reason, Django
provides a shortcut for that case: the permission_required() decorator.:
from django.contrib.auth.decorators import permission_required
@permission_required(’polls.can_vote’)
def my_view(request):
...
As for the has_perm() method, permission names take the form "<app label>.<permission
codename>" (i.e. polls.can_vote for a permission on a model in the polls application).
Note that permission_required() also takes an optional login_url parameter. Example:
from django.contrib.auth.decorators import permission_required
@permission_required(’polls.can_vote’, login_url=’/loginpage/’)
def my_view(request):
...
As in the login_required() decorator, login_url defaults to settings.LOGIN_URL. Changed in
version 1.4. Added raise_exception parameter. If given, the decorator will raise PermissionDenied,
prompting the 403 (HTTP Forbidden) view instead of redirecting to the login page.
Applying permissions to generic views To apply a permission to a class-based generic view, decorate the
View.dispatch method on the class. See Decorating the class for details.
Authentication Views
Django provides several views that you can use for handling login, logout, and password management. These make
use of the stock auth forms but you can pass in your own forms as well.
Django provides no default template for the authentication views - however the template context is documented for
each view below. New in version 1.4. The built-in views all return a TemplateResponse instance, which allows
you to easily customize the response data before rendering. For more details, see the TemplateResponse documenta-tion.
Most built-in authentication views provide a URL name for easier reference. See the URL documentation for details
on using named URL patterns.
login(request[, template_name, redirect_field_name, authentication_form ])
URL name: login
See the URL documentation for details on using named URL patterns.
Here’s what django.contrib.auth.views.login does:
•If called via GET, it displays a login form that POSTs to the same URL. More on this in a bit.
•If called via POST with user submitted credentials, it tries to log the user in. If login is suc-cessful,
the view redirects to the URL specified in next. If next isn’t provided, it redirects to
settings.LOGIN_REDIRECT_URL (which defaults to /accounts/profile/). If login isn’t suc-cessful,
it redisplays the login form.
It’s your responsibility to provide the html for the login template , called registration/login.html by
default. This template gets passed four template context variables:
•form: A Form object representing the AuthenticationForm.
302 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-306-2048.jpg)
![Django Documentation, Release 1.5.1
•next: The URL to redirect to after successful login. This may contain a query string, too.
•site: The current Site, according to the SITE_ID setting. If you don’t have the site framework
installed, this will be set to an instance of RequestSite, which derives the site name and domain from
the current HttpRequest.
•site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to
the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework.
If you’d prefer not to call the template registration/login.html, you can pass the template_name
parameter via the extra arguments to the view in your URLconf. For example, this URLconf line would use
myapp/login.html instead:
(r’^accounts/login/$’, ’django.contrib.auth.views.login’, {’template_name’: ’myapp/login.html’}),
You can also specify the name of the GET field which contains the URL to redirect to after login by passing
redirect_field_name to the view. By default, the field is called next.
Here’s a sample registration/login.html template you can use as a starting point. It assumes you
have a base.html template that defines a content block:
{% extends "base.html" %}
{% block content %}
{% if form.errors %}
<p>Your username and password didn’t match. Please try again.</p>
{% endif %}
<form method="post" action="{% url ’django.contrib.auth.views.login’ %}">
{% csrf_token %}
<table>
<tr>
<td>{{ form.username.label_tag }}</td>
<td>{{ form.username }}</td>
</tr>
<tr>
<td>{{ form.password.label_tag }}</td>
<td>{{ form.password }}</td>
</tr>
</table>
<input type="submit" value="login" />
<input type="hidden" name="next" value="{{ next }}" />
</form>
{% endblock %}
If you have customized authentication (see Customizing Authentication) you can pass a custom authentication
form to the login view via the authentication_form parameter. This form must accept a request
keyword argument in its __init__ method, and provide a get_user method which returns the authenticated
user object (this method is only ever called after successful form validation).
logout(request[, next_page, template_name, redirect_field_name ])
Logs a user out.
URL name: logout
Optional arguments:
•next_page: The URL to redirect to after logout.
3.9. User authentication in Django 303](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-307-2048.jpg)
![Django Documentation, Release 1.5.1
•template_name: The full name of a template to display after logging the user out. Defaults to
registration/logged_out.html if no argument is supplied.
•redirect_field_name: The name of a GET field containing the URL to redirect to after log out.
Overrides next_page if the given GET parameter is passed.
Template context:
•title: The string “Logged out”, localized.
•site: The current Site, according to the SITE_ID setting. If you don’t have the site framework
installed, this will be set to an instance of RequestSite, which derives the site name and domain from
the current HttpRequest.
•site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to
the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework.
logout_then_login(request[, login_url ])
Logs a user out, then redirects to the login page.
URL name: No default URL provided
Optional arguments:
•login_url: The URL of the login page to redirect to. Defaults to settings.LOGIN_URL if not
supplied.
password_change(request[, template_name, post_change_redirect, password_change_form ])
Allows a user to change their password.
URL name: password_change
Optional arguments:
•template_name: The full name of a template to use for displaying the password change form. Defaults
to registration/password_change_form.html if not supplied.
•post_change_redirect: The URL to redirect to after a successful password change.
•password_change_form: A custom “change password” form which must accept a user key-word
argument. The form is responsible for actually changing the user’s password. Defaults to
PasswordChangeForm.
Template context:
•form: The password change form (see password_change_form above).
password_change_done(request[, template_name ])
The page shown after a user has changed their password.
URL name: password_change_done
Optional arguments:
•template_name: The full name of a template to use. Defaults to
registration/password_change_done.html if not supplied.
password_reset(request[, is_admin_site, template_name, email_template_name, password_reset_form,
token_generator, post_reset_redirect, from_email ])
Allows a user to reset their password by generating a one-time use link that can be used to reset the password, and
sending that link to the user’s registered email address. Changed in version 1.4: Users flagged with an unusable
password (see set_unusable_password() will not be able to request a password reset to prevent misuse
when using an external authentication source like LDAP. URL name: password_reset
Optional arguments:
304 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-308-2048.jpg)
![Django Documentation, Release 1.5.1
•template_name: The full name of a template to use for displaying the password reset form. Defaults
to registration/password_reset_form.html if not supplied.
•email_template_name: The full name of a template to use for generating the email with the reset
password link. Defaults to registration/password_reset_email.html if not supplied.
•subject_template_name: The full name of a template to use for the subject of the email with the
reset password link. Defaults to registration/password_reset_subject.txt if not supplied.
New in version 1.4.
•password_reset_form: Form that will be used to get the email of the user to reset the password for.
Defaults to PasswordResetForm.
•token_generator: Instance of the class to check the one time link.
This will default to default_token_generator, it’s an instance of
django.contrib.auth.tokens.PasswordResetTokenGenerator.
•post_reset_redirect: The URL to redirect to after a successful password reset request.
•from_email: A valid email address. By default Django uses the DEFAULT_FROM_EMAIL.
Template context:
•form: The form (see password_reset_form above) for resetting the user’s password.
Email template context:
•email: An alias for user.email
•user: The current User, according to the email form field. Only active users are able to reset their
passwords (User.is_active is True).
•site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to
the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework.
•domain: An alias for site.domain. If you don’t have the site framework installed, this will be set to
the value of request.get_host().
•protocol: http or https
•uid: The user’s id encoded in base 36.
•token: Token to check that the reset link is valid.
Sample registration/password_reset_email.html (email body template):
Someone asked for password reset for email {{ email }}. Follow the link below:
{{ protocol}}://{{ domain }}{% url ’password_reset_confirm’ uidb36=uid token=token %}
The same template context is used for subject template. Subject must be single line plain text string.
password_reset_done(request[, template_name ])
The page shown after a user has been emailed a link to reset their password. This view is called by default if the
password_reset() view doesn’t have an explicit post_reset_redirect URL set.
URL name: password_reset_done
Optional arguments:
•template_name: The full name of a template to use. Defaults to
registration/password_reset_done.html if not supplied.
password_reset_confirm(request[, uidb36, token, template_name, token_generator,
set_password_form, post_reset_redirect ])
Presents a form for entering a new password.
3.9. User authentication in Django 305](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-309-2048.jpg)
![Django Documentation, Release 1.5.1
URL name: password_reset_confirm
Optional arguments:
•uidb36: The user’s id encoded in base 36. Defaults to None.
•token: Token to check that the password is valid. Defaults to None.
•template_name: The full name of a template to display the confirm password view. Default value is
registration/password_reset_confirm.html.
•token_generator: Instance of the class to check the password.
This will default to default_token_generator, it’s an instance of
django.contrib.auth.tokens.PasswordResetTokenGenerator.
•set_password_form: Form that will be used to set the password. Defaults to SetPasswordForm
•post_reset_redirect: URL to redirect after the password reset done. Defaults to None.
Template context:
•form: The form (see set_password_form above) for setting the new user’s password.
•validlink: Boolean, True if the link (combination of uidb36 and token) is valid or unused yet.
password_reset_complete(request[, template_name ])
Presents a view which informs the user that the password has been successfully changed.
URL name: password_reset_complete
Optional arguments:
•template_name: The full name of a template to display the view. Defaults to
registration/password_reset_complete.html.
Helper functions
redirect_to_login(next[, login_url, redirect_field_name ])
Redirects to the login page, and then back to another URL after a successful login.
Required arguments:
•next: The URL to redirect to after a successful login.
Optional arguments:
•login_url: The URL of the login page to redirect to. Defaults to settings.LOGIN_URL if not
supplied.
•redirect_field_name: The name of a GET field containing the URL to redirect to after log out.
Overrides next if the given GET parameter is passed.
Built-in forms
If you don’t want to use the built-in views, but want the convenience of not having to write forms for this functionality,
the authentication system provides several built-in forms located in django.contrib.auth.forms:
Note: The built-in authentication forms make certain assumptions about the user model that they are working with.
If you’re using a custom User model, it may be necessary to define your own forms for the authentication system. For
more information, refer to the documentation about using the built-in authentication forms with custom user models.
306 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-310-2048.jpg)
![Django Documentation, Release 1.5.1
3.9.2 Password management in Django
Password management is something that should generally not be reinvented unnecessarily, and Django endeavors to
provide a secure and flexible set of tools for managing user passwords. This document describes how Django stores
passwords, how the storage hashing can be configured, and some utilities to work with hashed passwords.
How Django stores passwords
New in version 1.4: Django 1.4 introduces a new flexible password storage system and uses PBKDF2 by default.
Previous versions of Django used SHA1, and other algorithms couldn’t be chosen. The password attribute of a
User object is a string in this format:
<algorithm>$<iterations>$<salt>$<hash>
Those are the components used for storing a User’s password, separated by the dollar-sign character and consist of: the
hashing algorithm, the number of algorithm iterations (work factor), the random salt, and the resulting password hash.
The algorithm is one of a number of one-way hashing or password storage algorithms Django can use; see below.
Iterations describe the number of times the algorithm is run over the hash. Salt is the random seed used and the hash
is the result of the one-way function.
By default, Django uses the PBKDF2 algorithm with a SHA256 hash, a password stretching mechanism recommended
by NIST. This should be sufficient for most users: it’s quite secure, requiring massive amounts of computing time to
break.
However, depending on your requirements, you may choose a different algorithm, or even use a custom algorithm to
match your specific security situation. Again, most users shouldn’t need to do this – if you’re not sure, you probably
don’t. If you do, please read on:
Django chooses the algorithm to use by consulting the PASSWORD_HASHERS setting. This is a list
of hashing algorithm classes that this Django installation supports. The first entry in this list (that is,
settings.PASSWORD_HASHERS[0]) will be used to store passwords, and all the other entries are valid hash-ers
that can be used to check existing passwords. This means that if you want to use a different algorithm, you’ll need
to modify PASSWORD_HASHERS to list your preferred algorithm first in the list.
The default for PASSWORD_HASHERS is:
PASSWORD_HASHERS = (
’django.contrib.auth.hashers.PBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’,
’django.contrib.auth.hashers.BCryptPasswordHasher’,
’django.contrib.auth.hashers.SHA1PasswordHasher’,
’django.contrib.auth.hashers.MD5PasswordHasher’,
’django.contrib.auth.hashers.CryptPasswordHasher’,
)
This means that Django will use PBKDF2 to store all passwords, but will support checking passwords stored with
PBKDF2SHA1, bcrypt, SHA1, etc. The next few sections describe a couple of common ways advanced users may
want to modify this setting.
Using bcrypt with Django
Bcrypt is a popular password storage algorithm that’s specifically designed for long-term password storage. It’s not
the default used by Django since it requires the use of third-party libraries, but since many people may want to use it
Django supports bcrypt with minimal effort.
To use Bcrypt as your default storage algorithm, do the following:
3.9. User authentication in Django 309](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-313-2048.jpg)
![Django Documentation, Release 1.5.1
1. Install the py-bcrypt library (probably by running sudo pip install py-bcrypt, or downloading the
library and installing it with python setup.py install).
2. Modify PASSWORD_HASHERS to list BCryptPasswordHasher first. That is, in your settings file, you’d
put:
PASSWORD_HASHERS = (
’django.contrib.auth.hashers.BCryptPasswordHasher’,
’django.contrib.auth.hashers.PBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’,
’django.contrib.auth.hashers.SHA1PasswordHasher’,
’django.contrib.auth.hashers.MD5PasswordHasher’,
’django.contrib.auth.hashers.CryptPasswordHasher’,
)
(You need to keep the other entries in this list, or else Django won’t be able to upgrade passwords; see below).
That’s it – now your Django install will use Bcrypt as the default storage algorithm.
Password truncation with BCryptPasswordHasher
The designers of bcrypt truncate all passwords at 72 characters which means that
bcrypt(password_with_100_chars) == bcrypt(password_with_100_chars[:72]).
BCryptPasswordHasher does not have any special handling and thus is also subject to this hidden pass-word
length limit. The practical ramification of this truncation is pretty marginal as the average user does not have a
password greater than 72 characters in length and even being truncated at 72 the compute powered required to brute
force bcrypt in any useful amount of time is still astronomical.
Other bcrypt implementations
There are several other implementations that allow bcrypt to be used with Django. Django’s
bcrypt support is NOT directly compatible with these. To upgrade, you will need to modify the
hashes in your database to be in the form bcrypt$(raw bcrypt output). For example:
bcrypt$$2a$12$NT0I31Sa7ihGEWpka9ASYrEFkhuTNeBQ2xfZskIiiJeyFXhRgS.Sy.
Increasing the work factor
The PBKDF2 and bcrypt algorithms use a number of iterations or rounds of hashing. This deliberately slows down
attackers, making attacks against hashed passwords harder. However, as computing power increases, the number of
iterations needs to be increased. We’ve chosen a reasonable default (and will increase it with each release of Django),
but you may wish to tune it up or down, depending on your security needs and available processing power. To do so,
you’ll subclass the appropriate algorithm and override the iterations parameters. For example, to increase the
number of iterations used by the default PBKDF2 algorithm:
1. Create a subclass of django.contrib.auth.hashers.PBKDF2PasswordHasher:
from django.contrib.auth.hashers import PBKDF2PasswordHasher
class MyPBKDF2PasswordHasher(PBKDF2PasswordHasher):
"""
A subclass of PBKDF2PasswordHasher that uses 100 times more iterations.
"""
iterations = PBKDF2PasswordHasher.iterations * 100
Save this somewhere in your project. For example, you might put this in a file like myproject/hashers.py.
310 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-314-2048.jpg)
![Django Documentation, Release 1.5.1
2. Add your new hasher as the first entry in PASSWORD_HASHERS:
PASSWORD_HASHERS = (
’myproject.hashers.MyPBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’,
’django.contrib.auth.hashers.BCryptPasswordHasher’,
’django.contrib.auth.hashers.SHA1PasswordHasher’,
’django.contrib.auth.hashers.MD5PasswordHasher’,
’django.contrib.auth.hashers.CryptPasswordHasher’,
)
That’s it – now your Django install will use more iterations when it stores passwords using PBKDF2.
Password upgrading
When users log in, if their passwords are stored with anything other than the preferred algorithm, Django will auto-matically
upgrade the algorithm to the preferred one. This means that old installs of Django will get automatically
more secure as users log in, and it also means that you can switch to new (and better) storage algorithms as they get
invented.
However, Django can only upgrade passwords that use algorithms mentioned in PASSWORD_HASHERS, so as you
upgrade to new systems you should make sure never to remove entries from this list. If you do, users using un-mentioned
algorithms won’t be able to upgrade.
Manually managing a user’s password
New in version 1.4: The django.contrib.auth.hashers module provides a set of functions to create and
validate hashed password. You can use them independently from the User model.
check_password(password, encoded)
New in version 1.4. If you’d like to manually authenticate a user by comparing a plain-text password to the
hashed password in the database, use the convenience function check_password(). It takes two arguments:
the plain-text password to check, and the full value of a user’s password field in the database to check against,
and returns True if they match, False otherwise.
make_password(password[, salt, hashers ])
New in version 1.4. Creates a hashed password in the format used by this application. It takes one mandatory
argument: the password in plain-text. Optionally, you can provide a salt and a hashing algorithm to use, if you
don’t want to use the defaults (first entry of PASSWORD_HASHERS setting). Currently supported algorithms are:
’pbkdf2_sha256’, ’pbkdf2_sha1’, ’bcrypt’ (see Using bcrypt with Django), ’sha1’, ’md5’,
’unsalted_md5’ (only for backward compatibility) and ’crypt’ if you have the crypt library installed.
If the password argument is None, an unusable password is returned (a one that will be never accepted by
check_password()).
is_password_usable(encoded_password)
New in version 1.4. Checks if the given string is a hashed password that has a chance of being verified against
check_password().
3.9.3 Customizing authentication in Django
The authentication that comes with Django is good enough for most common cases, but you may have needs not met
by the out-of-the-box defaults. To customize authentication to your projects needs involves understanding what points
of the provided system are extendible or replaceable. This document provides details about how the auth system can
be customized.
3.9. User authentication in Django 311](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-315-2048.jpg)
![Django Documentation, Release 1.5.1
The easiest way to construct a compliant custom User model is to inherit from AbstractBaseUser.
AbstractBaseUser provides the core implementation of a User model, including hashed passwords and tok-enized
password resets. You must then provide some key implementation details:
class models.CustomUser
USERNAME_FIELD
A string describing the name of the field on the User model that is used as the unique identifier. This will
usually be a username of some kind, but it can also be an email address, or any other unique identifier. The
field must be unique (i.e., have unique=True set in its definition).
In the following example, the field identifier is used as the identifying field:
class MyUser(AbstractBaseUser):
identifier = models.CharField(max_length=40, unique=True, db_index=True)
...
USERNAME_FIELD = ’identifier’
REQUIRED_FIELDS
A list of the field names that must be provided when creating a user via the createsuperuser man-agement
command. The user will be prompted to supply a value for each of these fields. It should include
any field for which blank is False or undefined, but may include additional fields you want prompted
for when a user is created interactively. However, it will not work for ForeignKey fields.
For example, here is the partial definition for a User model that defines two required fields - a date of
birth and height:
class MyUser(AbstractBaseUser):
...
date_of_birth = models.DateField()
height = models.FloatField()
...
REQUIRED_FIELDS = [’date_of_birth’, ’height’]
Note: REQUIRED_FIELDS must contain all required fields on your User model, but should not contain
the USERNAME_FIELD.
is_active
A boolean attribute that indicates whether the user is considered “active”. This attribute is provided as
an attribute on AbstractBaseUser defaulting to True. How you choose to implement it will de-pend
on the details of your chosen auth backends. See the documentation of the attribute on the
builtin user model for details.
get_full_name()
A longer formal identifier for the user. A common interpretation would be the full name name of the user,
but it can be any string that identifies the user.
get_short_name()
A short, informal identifier for the user. A common interpretation would be the first name of the user,
but it can be any string that identifies the user in an informal way. It may also return the same value as
django.contrib.auth.models.User.get_full_name().
The following methods are available on any subclass of AbstractBaseUser:
class models.AbstractBaseUser
318 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-322-2048.jpg)
![Django Documentation, Release 1.5.1
Creates and saves a superuser with the given email, date of
birth and password.
"""
user = self.create_user(email,
password=password,
date_of_birth=date_of_birth
)
user.is_admin = True
user.save(using=self._db)
return user
class MyUser(AbstractBaseUser):
email = models.EmailField(
verbose_name=’email address’,
max_length=255,
unique=True,
db_index=True,
)
date_of_birth = models.DateField()
is_active = models.BooleanField(default=True)
is_admin = models.BooleanField(default=False)
objects = MyUserManager()
USERNAME_FIELD = ’email’
REQUIRED_FIELDS = [’date_of_birth’]
def get_full_name(self):
# The user is identified by their email address
return self.email
def get_short_name(self):
# The user is identified by their email address
return self.email
def __unicode__(self):
return self.email
def has_perm(self, perm, obj=None):
"Does the user have a specific permission?"
# Simplest possible answer: Yes, always
return True
def has_module_perms(self, app_label):
"Does the user have permissions to view the app ‘app_label‘?"
# Simplest possible answer: Yes, always
return True
@property
def is_staff(self):
"Is the user a member of staff?"
# Simplest possible answer: All admins are staff
return self.is_admin
Then, to register this custom User model with Django’s admin, the following code would be required in the app’s
admin.py file:
324 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-328-2048.jpg)
![Django Documentation, Release 1.5.1
from django import forms
from django.contrib import admin
from django.contrib.auth.models import Group
from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.forms import ReadOnlyPasswordHashField
from customauth.models import MyUser
class UserCreationForm(forms.ModelForm):
"""A form for creating new users. Includes all the required
fields, plus a repeated password."""
password1 = forms.CharField(label=’Password’, widget=forms.PasswordInput)
password2 = forms.CharField(label=’Password confirmation’, widget=forms.PasswordInput)
class Meta:
model = MyUser
fields = (’email’, ’date_of_birth’)
def clean_password2(self):
# Check that the two password entries match
password1 = self.cleaned_data.get("password1")
password2 = self.cleaned_data.get("password2")
if password1 and password2 and password1 != password2:
raise forms.ValidationError("Passwords don’t match")
return password2
def save(self, commit=True):
# Save the provided password in hashed format
user = super(UserCreationForm, self).save(commit=False)
user.set_password(self.cleaned_data["password1"])
if commit:
user.save()
return user
class UserChangeForm(forms.ModelForm):
"""A form for updating users. Includes all the fields on
the user, but replaces the password field with admin’s
password hash display field.
"""
password = ReadOnlyPasswordHashField()
class Meta:
model = MyUser
def clean_password(self):
# Regardless of what the user provides, return the initial value.
# This is done here, rather than on the field, because the
# field does not have access to the initial value
return self.initial["password"]
class MyUserAdmin(UserAdmin):
# The forms to add and change user instances
form = UserChangeForm
add_form = UserCreationForm
3.9. User authentication in Django 325](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-329-2048.jpg)
![Django Documentation, Release 1.5.1
CACHES = {
’default’: {
’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’,
’LOCATION’: ’unix:/tmp/memcached.sock’,
}
}
One excellent feature of Memcached is its ability to share cache over multiple servers. This means you can run
Memcached daemons on multiple machines, and the program will treat the group of machines as a single cache,
without the need to duplicate cache values on each machine. To take advantage of this feature, include all server
addresses in LOCATION, either separated by semicolons or as a list.
In this example, the cache is shared over Memcached instances running on IP address 172.19.26.240 and
172.19.26.242, both on port 11211:
CACHES = {
’default’: {
’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’,
’LOCATION’: [
’172.19.26.240:11211’,
’172.19.26.242:11211’,
]
}
}
In the following example, the cache is shared over Memcached instances running on the IP addresses 172.19.26.240
(port 11211), 172.19.26.242 (port 11212), and 172.19.26.244 (port 11213):
CACHES = {
’default’: {
’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’,
’LOCATION’: [
’172.19.26.240:11211’,
’172.19.26.242:11212’,
’172.19.26.244:11213’,
]
}
}
A final point about Memcached is that memory-based caching has one disadvantage: Because the cached data is stored
in memory, the data will be lost if your server crashes. Clearly, memory isn’t intended for permanent data storage, so
don’t rely on memory-based caching as your only data storage. Without a doubt, none of the Django caching backends
should be used for permanent storage – they’re all intended to be solutions for caching, not storage – but we point this
out here because memory-based caching is particularly temporary.
Database caching
To use a database table as your cache backend, first create a cache table in your database by running this command:
python manage.py createcachetable [cache_table_name]
...where [cache_table_name] is the name of the database table to create. (This name can be whatever you want,
as long as it’s a valid table name that’s not already being used in your database.) This command creates a single table
in your database that is in the proper format that Django’s database-cache system expects.
Once you’ve created that database table, set your BACKEND setting to
"django.core.cache.backends.db.DatabaseCache", and LOCATION to tablename – the name of
the database table. In this example, the cache table’s name is my_cache_table:
3.10. Django’s cache framework 329](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-333-2048.jpg)
![Django Documentation, Release 1.5.1
>>> cache.set(’my_key’, ’hello, world!’, 30)
>>> cache.get(’my_key’)
’hello, world!’
The timeout argument is optional and defaults to the timeout argument of the appropriate backend in the CACHES
setting (explained above). It’s the number of seconds the value should be stored in the cache.
If the object doesn’t exist in the cache, cache.get() returns None:
# Wait 30 seconds for ’my_key’ to expire...
>>> cache.get(’my_key’)
None
We advise against storing the literal value None in the cache, because you won’t be able to distinguish between your
stored None value and a cache miss signified by a return value of None.
cache.get() can take a default argument. This specifies which value to return if the object doesn’t exist in the
cache:
>>> cache.get(’my_key’, ’has expired’)
’has expired’
To add a key only if it doesn’t already exist, use the add() method. It takes the same parameters as set(), but it
will not attempt to update the cache if the key specified is already present:
>>> cache.set(’add_key’, ’Initial value’)
>>> cache.add(’add_key’, ’New value’)
>>> cache.get(’add_key’)
’Initial value’
If you need to know whether add() stored a value in the cache, you can check the return value. It will return True
if the value was stored, False otherwise.
There’s also a get_many() interface that only hits the cache once. get_many() returns a dictionary with all the
keys you asked for that actually exist in the cache (and haven’t expired):
>>> cache.set(’a’, 1)
>>> cache.set(’b’, 2)
>>> cache.set(’c’, 3)
>>> cache.get_many([’a’, ’b’, ’c’])
{’a’: 1, ’b’: 2, ’c’: 3}
To set multiple values more efficiently, use set_many() to pass a dictionary of key-value pairs:
>>> cache.set_many({’a’: 1, ’b’: 2, ’c’: 3})
>>> cache.get_many([’a’, ’b’, ’c’])
{’a’: 1, ’b’: 2, ’c’: 3}
Like cache.set(), set_many() takes an optional timeout parameter.
You can delete keys explicitly with delete(). This is an easy way of clearing the cache for a particular object:
>>> cache.delete(’a’)
If you want to clear a bunch of keys at once, delete_many() can take a list of keys to be cleared:
>>> cache.delete_many([’a’, ’b’, ’c’])
Finally, if you want to delete all the keys in the cache, use cache.clear(). Be careful with this; clear() will
remove everything from the cache, not just the keys set by your application.
3.10. Django’s cache framework 337](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-341-2048.jpg)
![Django Documentation, Release 1.5.1
>>> cache.get(’my_key’, version=2)
’hello world!’
The version of a specific key can be incremented and decremented using the incr_version() and
decr_version() methods. This enables specific keys to be bumped to a new version, leaving other keys unaf-fected.
Continuing our previous example:
# Increment the version of ’my_key’
>>> cache.incr_version(’my_key’)
# The default version still isn’t available
>>> cache.get(’my_key’)
None
# Version 2 isn’t available, either
>>> cache.get(’my_key’, version=2)
None
# But version 3 *is* available
>>> cache.get(’my_key’, version=3)
’hello world!’
Cache key transformation
As described in the previous two sections, the cache key provided by a user is not used verbatim – it is combined with
the cache prefix and key version to provide a final cache key. By default, the three parts are joined using colons to
produce a final string:
def make_key(key, key_prefix, version):
return ’:’.join([key_prefix, str(version), key])
If you want to combine the parts in different ways, or apply other processing to the final key (e.g., taking a hash digest
of the key parts), you can provide a custom key function.
The KEY_FUNCTION cache setting specifies a dotted-path to a function matching the prototype of make_key()
above. If provided, this custom key function will be used instead of the default key combining function.
Cache key warnings
Memcached, the most commonly-used production cache backend, does not allow cache keys longer than 250 char-acters
or containing whitespace or control characters, and using such keys will cause an exception. To encour-age
cache-portable code and minimize unpleasant surprises, the other built-in cache backends issue a warning
(django.core.cache.backends.base.CacheKeyWarning) if a key is used that would cause an error on
memcached.
If you are using a production backend that can accept a wider range of keys (a custom backend, or one of
the non-memcached built-in backends), and want to use this wider range without warnings, you can silence
CacheKeyWarning with this code in the management module of one of your INSTALLED_APPS:
import warnings
from django.core.cache import CacheKeyWarning
warnings.simplefilter("ignore", CacheKeyWarning)
If you want to instead provide custom key validation logic for one of the built-in backends, you can subclass it, override
just the validate_key method, and follow the instructions for using a custom cache backend. For instance, to do
this for the locmem backend, put this code in a module:
3.10. Django’s cache framework 339](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-343-2048.jpg)
![Django Documentation, Release 1.5.1
from django.views.decorators.vary import vary_on_headers
@vary_on_headers(’User-Agent’)
def my_view(request):
# ...
In this case, a caching mechanism (such as Django’s own cache middleware) will cache a separate version of the page
for each unique user-agent.
The advantage to using the vary_on_headers decorator rather than manually setting the Vary header (using
something like response[’Vary’] = ’user-agent’) is that the decorator adds to the Vary header (which
may already exist), rather than setting it from scratch and potentially overriding anything that was already in there.
You can pass multiple headers to vary_on_headers():
@vary_on_headers(’User-Agent’, ’Cookie’)
def my_view(request):
# ...
This tells upstream caches to vary on both, which means each combination of user-agent and cookie will get its own
cache value. For example, a request with the user-agent Mozilla and the cookie value foo=bar will be considered
different from a request with the user-agent Mozilla and the cookie value foo=ham.
Because varying on cookie is so common, there’s a vary_on_cookie decorator. These two views are equivalent:
@vary_on_cookie
def my_view(request):
# ...
@vary_on_headers(’Cookie’)
def my_view(request):
# ...
The headers you pass to vary_on_headers are not case sensitive; "User-Agent" is the same thing as
"user-agent".
You can also use a helper function, django.utils.cache.patch_vary_headers, directly. This function
sets, or adds to, the Vary header. For example:
from django.utils.cache import patch_vary_headers
def my_view(request):
# ...
response = render_to_response(’template_name’, context)
patch_vary_headers(response, [’Cookie’])
return response
patch_vary_headers takes an HttpResponse instance as its first argument and a list/tuple of case-insensitive
header names as its second argument.
For more on Vary headers, see the official Vary spec.
3.10.8 Controlling cache: Using other headers
Other problems with caching are the privacy of data and the question of where data should be stored in a cascade of
caches.
A user usually faces two kinds of caches: his or her own browser cache (a private cache) and his or her provider’s
cache (a public cache). A public cache is used by multiple users and controlled by someone else. This poses problems
3.10. Django’s cache framework 341](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-345-2048.jpg)
![Django Documentation, Release 1.5.1
The code lives in the django.core.mail module.
3.13.1 Quick example
In two lines:
from django.core.mail import send_mail
send_mail(’Subject here’, ’Here is the message.’, ’from@example.com’,
[’to@example.com’], fail_silently=False)
Mail is sent using the SMTP host and port specified in the EMAIL_HOST and EMAIL_PORT settings. The
EMAIL_HOST_USER and EMAIL_HOST_PASSWORD settings, if set, are used to authenticate to the SMTP server,
and the EMAIL_USE_TLS setting controls whether a secure connection is used.
Note: The character set of email sent with django.core.mail will be set to the value of your
DEFAULT_CHARSET setting.
3.13.2 send_mail()
send_mail(subject, message, from_email, recipient_list, fail_silently=False, auth_user=None,
auth_password=None, connection=None)
The simplest way to send email is using django.core.mail.send_mail().
The subject, message, from_email and recipient_list parameters are required.
• subject: A string.
• message: A string.
• from_email: A string.
• recipient_list: A list of strings, each an email address. Each member of recipient_list will see
the other recipients in the “To:” field of the email message.
• fail_silently: A boolean. If it’s False, send_mail will raise an smtplib.SMTPException. See
the smtplib docs for a list of possible exceptions, all of which are subclasses of SMTPException.
• auth_user: The optional username to use to authenticate to the SMTP server. If this isn’t provided, Django
will use the value of the EMAIL_HOST_USER setting.
• auth_password: The optional password to use to authenticate to the SMTP server. If this isn’t provided,
Django will use the value of the EMAIL_HOST_PASSWORD setting.
• connection: The optional email backend to use to send the mail. If unspecified, an instance of the default
backend will be used. See the documentation on Email backends for more details.
3.13.3 send_mass_mail()
send_mass_mail(datatuple, fail_silently=False, auth_user=None, auth_password=None, connec-tion=
None)
django.core.mail.send_mass_mail() is intended to handle mass emailing.
datatuple is a tuple in which each element is in this format:
3.13. Sending email 349](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-353-2048.jpg)
![Django Documentation, Release 1.5.1
(subject, message, from_email, recipient_list)
fail_silently, auth_user and auth_password have the same functions as in send_mail().
Each separate element of datatuple results in a separate email message. As in send_mail(), recipients in the
same recipient_list will all see the other addresses in the email messages’ “To:” field.
For example, the following code would send two different messages to two different sets of recipients; however, only
one connection to the mail server would be opened:
message1 = (’Subject here’, ’Here is the message’, ’from@example.com’, [’first@example.com’, ’other@example.message2 = (’Another Subject’, ’Here is another message’, ’from@example.com’, [’second@test.com’])
send_mass_mail((message1, message2), fail_silently=False)
send_mass_mail() vs. send_mail()
The main difference between send_mass_mail() and send_mail() is that send_mail() opens a connec-tion
to the mail server each time it’s executed, while send_mass_mail() uses a single connection for all of its
messages. This makes send_mass_mail() slightly more efficient.
3.13.4 mail_admins()
mail_admins(subject, message, fail_silently=False, connection=None, html_message=None)
django.core.mail.mail_admins() is a shortcut for sending an email to the site admins, as defined in the
ADMINS setting.
mail_admins() prefixes the subject with the value of the EMAIL_SUBJECT_PREFIX setting, which is
"[Django] " by default.
The “From:” header of the email will be the value of the SERVER_EMAIL setting.
This method exists for convenience and readability.
If html_message is provided, the resulting email will be a multipart/alternative email with message
as the text/plain content type and html_message as the text/html content type.
3.13.5 mail_managers()
mail_managers(subject, message, fail_silently=False, connection=None, html_message=None)
django.core.mail.mail_managers() is just like mail_admins(), except it sends an email to the site
managers, as defined in the MANAGERS setting.
3.13.6 Examples
This sends a single email to john@example.com and jane@example.com, with them both appearing in the “To:”:
send_mail(’Subject’, ’Message.’, ’from@example.com’,
[’john@example.com’, ’jane@example.com’])
This sends a message to john@example.com and jane@example.com, with them both receiving a separate email:
350 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-354-2048.jpg)
![Django Documentation, Release 1.5.1
datatuple = (
(’Subject’, ’Message.’, ’from@example.com’, [’john@example.com’]),
(’Subject’, ’Message.’, ’from@example.com’, [’jane@example.com’]),
)
send_mass_mail(datatuple)
3.13.7 Preventing header injection
Header injection is a security exploit in which an attacker inserts extra email headers to control the “To:” and “From:”
in email messages that your scripts generate.
The Django email functions outlined above all protect against header injection by forbidding newlines in header
values. If any subject, from_email or recipient_list contains a newline (in either Unix,Windows or Mac
style), the email function (e.g. send_mail()) will raise django.core.mail.BadHeaderError (a subclass
of ValueError) and, hence, will not send the email. It’s your responsibility to validate all data before passing it to
the email functions.
If a message contains headers at the start of the string, the headers will simply be printed as the first bit of the email
message.
Here’s an example view that takes a subject, message and from_email from the request’s POST data, sends
that to admin@example.com and redirects to “/contact/thanks/” when it’s done:
from django.core.mail import send_mail, BadHeaderError
def send_email(request):
subject = request.POST.get(’subject’, ’’)
message = request.POST.get(’message’, ’’)
from_email = request.POST.get(’from_email’, ’’)
if subject and message and from_email:
try:
send_mail(subject, message, from_email, [’admin@example.com’])
except BadHeaderError:
return HttpResponse(’Invalid header found.’)
return HttpResponseRedirect(’/contact/thanks/’)
else:
# In reality we’d use a form class
# to get proper validation errors.
return HttpResponse(’Make sure all fields are entered and valid.’)
3.13.8 The EmailMessage class
Django’s send_mail() and send_mass_mail() functions are actually thin wrappers that make use of the
EmailMessage class.
Not all features of the EmailMessage class are available through the send_mail() and related wrapper functions.
If you wish to use advanced features, such as BCC’ed recipients, file attachments, or multi-part email, you’ll need to
create EmailMessage instances directly.
Note: This is a design feature. send_mail() and related functions were originally the only interface Django
provided. However, the list of parameters they accepted was slowly growing over time. It made sense to move to a
more object-oriented design for email messages and retain the original functions only for backwards compatibility.
EmailMessage is responsible for creating the email message itself. The email backend is then responsible for
sending the email.
3.13. Sending email 351](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-355-2048.jpg)
![Django Documentation, Release 1.5.1
For convenience, EmailMessage provides a simple send() method for sending a single email. If you need to send
multiple messages, the email backend API provides an alternative.
EmailMessage Objects
class EmailMessage
The EmailMessage class is initialized with the following parameters (in the given order, if positional arguments are
used). All parameters are optional and can be set at any time prior to calling the send() method.
• subject: The subject line of the email.
• body: The body text. This should be a plain text message.
• from_email: The sender’s address. Both fred@example.com and Fred <fred@example.com>
forms are legal. If omitted, the DEFAULT_FROM_EMAIL setting is used.
• to: A list or tuple of recipient addresses.
• bcc: A list or tuple of addresses used in the “Bcc” header when sending the email.
• connection: An email backend instance. Use this parameter if you want to use the same connection for
multiple messages. If omitted, a new connection is created when send() is called.
• attachments: A list of attachments to put on the message. These can be either
email.MIMEBase.MIMEBase instances, or (filename, content, mimetype) triples.
• headers: A dictionary of extra headers to put on the message. The keys are the header name, values are the
header values. It’s up to the caller to ensure header names and values are in the correct format for an email
message.
• cc: A list or tuple of recipient addresses used in the “Cc” header when sending the email.
For example:
email = EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to1@example.com’, ’to2@example.com’], [’bcc@example.com’],
headers = {’Reply-To’: ’another@example.com’})
The class has the following methods:
• send(fail_silently=False) sends the message. If a connection was specified when the email was
constructed, that connection will be used. Otherwise, an instance of the default backend will be instantiated and
used. If the keyword argument fail_silently is True, exceptions raised while sending the message will
be quashed.
• message() constructs a django.core.mail.SafeMIMEText object (a subclass of Python’s
email.MIMEText.MIMEText class) or a django.core.mail.SafeMIMEMultipart object hold-ing
the message to be sent. If you ever need to extend the EmailMessage class, you’ll probably want to
override this method to put the content you want into the MIME object.
• recipients() returns a list of all the recipients of the message, whether they’re recorded in the to, cc or
bcc attributes. This is another method you might need to override when subclassing, because the SMTP server
needs to be told the full list of recipients when the message is sent. If you add another way to specify recipients
in your class, they need to be returned from this method as well.
• attach() creates a new file attachment and adds it to the message. There are two ways to call attach():
– You can pass it a single argument that is an email.MIMEBase.MIMEBase instance. This will be
inserted directly into the resulting message.
352 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-356-2048.jpg)
![Django Documentation, Release 1.5.1
– Alternatively, you can pass attach() three arguments: filename, content and mimetype.
filename is the name of the file attachment as it will appear in the email, content is the data that
will be contained inside the attachment and mimetype is the optional MIME type for the attachment. If
you omit mimetype, the MIME content type will be guessed from the filename of the attachment.
For example:
message.attach(’design.png’, img_data, ’image/png’)
• attach_file() creates a new attachment using a file from your filesystem. Call it with the path of the file to
attach and, optionally, the MIME type to use for the attachment. If the MIME type is omitted, it will be guessed
from the filename. The simplest use would be:
message.attach_file(’/images/weather_map.png’)
Sending alternative content types
It can be useful to include multiple versions of the content in an email; the classic example is to send both text and
HTML versions of a message. With Django’s email library, you can do this using the EmailMultiAlternatives
class. This subclass of EmailMessage has an attach_alternative() method for including extra versions of
the message body in the email. All the other methods (including the class initialization) are inherited directly from
EmailMessage.
To send a text and HTML combination, you could write:
from django.core.mail import EmailMultiAlternatives
subject, from_email, to = ’hello’, ’from@example.com’, ’to@example.com’
text_content = ’This is an important message.’
html_content = ’<p>This is an <strong>important</strong> message.</p>’
msg = EmailMultiAlternatives(subject, text_content, from_email, [to])
msg.attach_alternative(html_content, "text/html")
msg.send()
By default, the MIME type of the body parameter in an EmailMessage is "text/plain". It is good practice
to leave this alone, because it guarantees that any recipient will be able to read the email, regardless of their mail
client. However, if you are confident that your recipients can handle an alternative content type, you can use the
content_subtype attribute on the EmailMessage class to change the main content type. The major type will
always be "text", but you can change the subtype. For example:
msg = EmailMessage(subject, html_content, from_email, [to])
msg.content_subtype = "html" # Main content is now text/html
msg.send()
3.13.9 Email backends
The actual sending of an email is handled by the email backend.
The email backend class has the following methods:
• open() instantiates an long-lived email-sending connection.
• close() closes the current email-sending connection.
• send_messages(email_messages) sends a list of EmailMessage objects. If the connection is not
open, this call will implicitly open the connection, and close the connection afterwards. If the connection is
already open, it will be left open after mail has been sent.
3.13. Sending email 353](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-357-2048.jpg)
![Django Documentation, Release 1.5.1
In this example, the call to send_messages() opens a connection on the backend, sends the list of messages, and
then closes the connection again.
The second approach is to use the open() and close() methods on the email backend to manually control the
connection. send_messages() will not manually open or close the connection if it is already open, so if you
manually open the connection, you can control when it is closed. For example:
from django.core import mail
connection = mail.get_connection()
# Manually open the connection
connection.open()
# Construct an email message that uses the connection
email1 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to1@example.com’], connection=connection)
email1.send() # Send the email
# Construct two more messages
email2 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to2@example.com’])
email3 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to3@example.com’])
# Send the two emails in a single call -
connection.send_messages([email2, email3])
# The connection was already open so send_messages() doesn’t close it.
# We need to manually close the connection.
connection.close()
3.13.10 Testing email sending
There are times when you do not want Django to send emails at all. For example, while developing a Web site, you
probably don’t want to send out thousands of emails – but you may want to validate that emails will be sent to the
right people under the right conditions, and that those emails will contain the correct content.
The easiest way to test your project’s use of email is to use the console email backend. This backend redirects all
email to stdout, allowing you to inspect the content of mail.
The file email backend can also be useful during development – this backend dumps the contents of every SMTP
connection to a file that can be inspected at your leisure.
Another approach is to use a “dumb” SMTP server that receives the emails locally and displays them to the terminal,
but does not actually send anything. Python has a built-in way to accomplish this with a single command:
python -m smtpd -n -c DebuggingServer localhost:1025
This command will start a simple SMTP server listening on port 1025 of localhost. This server simply prints to
standard output all email headers and the email body. You then only need to set the EMAIL_HOST and EMAIL_PORT
accordingly, and you are set.
For a more detailed discussion of testing and processing of emails locally, see the Python documentation for the
smtpd module.
356 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-360-2048.jpg)
![Django Documentation, Release 1.5.1
from django.utils.translation import ugettext as _
def my_view(request):
output = _("Welcome to my site.")
return HttpResponse(output)
Obviously, you could code this without using the alias. This example is identical to the previous one:
from django.utils.translation import ugettext
def my_view(request):
output = ugettext("Welcome to my site.")
return HttpResponse(output)
Translation works on computed values. This example is identical to the previous two:
def my_view(request):
words = [’Welcome’, ’to’, ’my’, ’site.’]
output = _(’ ’.join(words))
return HttpResponse(output)
Translation works on variables. Again, here’s an identical example:
def my_view(request):
sentence = ’Welcome to my site.’
output = _(sentence)
return HttpResponse(output)
(The caveat with using variables or computed values, as in the previous two examples, is that Django’s translation-string-
detecting utility, django-admin.py makemessages, won’t be able to find these strings. More on
makemessages later.)
The strings you pass to _() or ugettext() can take placeholders, specified with Python’s standard named-string
interpolation syntax. Example:
def my_view(request, m, d):
output = _(’Today is %(month)s %(day)s.’) % {’month’: m, ’day’: d}
return HttpResponse(output)
This technique lets language-specific translations reorder the placeholder text. For example, an English translation
may be "Today is November 26.", while a Spanish translation may be "Hoy es 26 de Noviembre."
– with the the month and the day placeholders swapped.
For this reason, you should use named-string interpolation (e.g., %(day)s) instead of positional interpolation (e.g.,
%s or %d) whenever you have more than a single parameter. If you used positional interpolation, translations wouldn’t
be able to reorder placeholder text.
Comments for translators
If you would like to give translators hints about a translatable string, you can add a comment prefixed with the
Translators keyword on the line preceding the string, e.g.:
def my_view(request):
# Translators: This message appears on the home page only
output = ugettext("Welcome to my site.")
The comment will then appear in the resulting .po file associated with the translatable contruct located below it and
should also be displayed by most translation tools.
358 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-362-2048.jpg)
![Django Documentation, Release 1.5.1
’There are %(count)d %(name)s available.’,
count
) % {
’count’: count,
’name’: name
}
Here we reuse localizable, hopefully already translated literals (contained in the verbose_name and
verbose_name_plural model Meta options) for other parts of the sentence so all of it is consistently based
on the cardinality of the elements at play.
Note: When using this technique, make sure you use a single name for every extrapolated variable included in the
literal. In the example above note how we used the name Python variable in both translation strings. This example
would fail:
from django.utils.translation import ungettext
from myapp.models import Report
count = Report.objects.count()
d = {
’count’: count,
’name’: Report._meta.verbose_name,
’plural_name’: Report._meta.verbose_name_plural
}
text = ungettext(
’There is %(count)d %(name)s available.’,
’There are %(count)d %(plural_name)s available.’,
count
) % d
You would get an error when running django-admin.py compilemessages:
a format specification for argument ’name’, as in ’msgstr[0]’, doesn’t exist in ’msgid’
Contextual markers
Sometimes words have several meanings, such as "May" in English, which refers to a month
name and to a verb. To enable translators to translate these words correctly in different
contexts, you can use the django.utils.translation.pgettext() function, or the
django.utils.translation.npgettext() function if the string needs pluralization. Both take a
context string as the first variable.
In the resulting .po file, the string will then appear as often as there are different contextual markers for the same
string (the context will appear on the msgctxt line), allowing the translator to give a different translation for each of
them.
For example:
from django.utils.translation import pgettext
month = pgettext("month name", "May")
or:
from django.utils.translation import pgettext_lazy
360 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-364-2048.jpg)
![Django Documentation, Release 1.5.1
verbose_name = _(’my thing’)
verbose_name_plural = _(’my things’)
Model methods short_description attribute values For model methods, you can provide translations to
Django and the admin site with the short_description attribute:
from django.utils.translation import ugettext_lazy as _
class MyThing(models.Model):
kind = models.ForeignKey(ThingKind, related_name=’kinds’,
verbose_name=_(’kind’))
def is_mouse(self):
return self.kind.type == MOUSE_TYPE
is_mouse.short_description = _(’Is it a mouse?’)
Working with lazy translation objects
The result of a ugettext_lazy() call can be used wherever you would use a unicode string (an object with type
unicode) in Python. If you try to use it where a bytestring (a str object) is expected, things will not work as
expected, since a ugettext_lazy() object doesn’t know how to convert itself to a bytestring. You can’t use a
unicode string inside a bytestring, either, so this is consistent with normal Python behavior. For example:
# This is fine: putting a unicode proxy into a unicode string.
u"Hello %s" % ugettext_lazy("people")
# This will not work, since you cannot insert a unicode object
# into a bytestring (nor can you insert our unicode proxy there)
"Hello %s" % ugettext_lazy("people")
If you ever see output that looks like "hello <django.utils.functional...>", you have tried to insert
the result of ugettext_lazy() into a bytestring. That’s a bug in your code.
If you don’t like the long ugettext_lazy name, you can just alias it as _ (underscore), like so:
from django.utils.translation import ugettext_lazy as _
class MyThing(models.Model):
name = models.CharField(help_text=_(’This is the help text’))
Using ugettext_lazy() and ungettext_lazy() to mark strings in models and utility functions is a common
operation. When you’re working with these objects elsewhere in your code, you should ensure that you don’t acci-dentally
convert them to strings, because they should be converted as late as possible (so that the correct locale is in
effect). This necessitates the use of the helper function described next.
Joining strings: string_concat() Standard Python string joins (”.join([...])) will not work on lists contain-ing
lazy translation objects. Instead, you can use django.utils.translation.string_concat(), which
creates a lazy object that concatenates its contents and converts them to strings only when the result is included in a
string. For example:
from django.utils.translation import string_concat
...
name = ugettext_lazy(’John Lennon’)
instrument = ugettext_lazy(’guitar’)
result = string_concat(name, ’: ’, instrument)
362 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-366-2048.jpg)
![Django Documentation, Release 1.5.1
In this case, the lazy translations in result will only be converted to strings when result itself is used in a string
(usually at template rendering time).
Other uses of lazy in delayed translations For any other case where you would like to delay the translation, but
have to pass the translatable string as argument to another function, you can wrap this function inside a lazy call
yourself. For example:
from django.utils import six # Python 3 compatibility
from django.utils.functional import lazy
from django.utils.safestring import mark_safe
from django.utils.translation import ugettext_lazy as _
mark_safe_lazy = lazy(mark_safe, six.text_type)
And then later:
lazy_string = mark_safe_lazy(_("<p>My <strong>string!</strong></p>"))
Localized names of languages
get_language_info()
The get_language_info() function provides detailed information about languages:
>>> from django.utils.translation import get_language_info
>>> li = get_language_info(’de’)
>>> print(li[’name’], li[’name_local’], li[’bidi’])
German Deutsch False
The name and name_local attributes of the dictionary contain the name of the language in English and in the
language itself, respectively. The bidi attribute is True only for bi-directional languages.
The source of the language information is the django.conf.locale module. Similar access to this information
is available for template code. See below.
Internationalization: in template code
Translations in Django templates uses two template tags and a slightly different syntax than in Python code. To give
your template access to these tags, put {% load i18n %} toward the top of your template. As with all template
tags, this tag needs to be loaded in all templates which use translations, even those templates that extend from other
templates which have already loaded the i18n tag.
trans template tag
The {% trans %} template tag translates either a constant string (enclosed in single or double quotes) or variable
content:
<title>{% trans "This is the title." %}</title>
<title>{% trans myvar %}</title>
If the noop option is present, variable lookup still takes place but the translation is skipped. This is useful when
“stubbing out” content that will require translation in the future:
<title>{% trans "myvar" noop %}</title>
3.14. Internationalization and localization 363](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-367-2048.jpg)
![Django Documentation, Release 1.5.1
{% some_special_tag _("Page not found") value|yesno:_("yes,no") %}
In this case, both the tag and the filter will see the already-translated string, so they don’t need to be aware of transla-tions.
Note: In this example, the translation infrastructure will be passed the string "yes,no", not the individual strings
"yes" and "no". The translated string will need to contain the comma so that the filter parsing code knows how
to split up the arguments. For example, a German translator might translate the string "yes,no" as "ja,nein"
(keeping the comma intact).
You can also retrieve information about any of the available languages using provided template tags and filters. To get
information about a single language, use the {% get_language_info %} tag:
{% get_language_info for LANGUAGE_CODE as lang %}
{% get_language_info for "pl" as lang %}
You can then access the information:
Language code: {{ lang.code }}<br />
Name of language: {{ lang.name_local }}<br />
Name in English: {{ lang.name }}<br />
Bi-directional: {{ lang.bidi }}
You can also use the {% get_language_info_list %} template tag to retrieve information for a list of lan-guages
(e.g. active languages as specified in LANGUAGES). See the section about the set_language redirect view for
an example of how to display a language selector using {% get_language_info_list %}.
In addition to LANGUAGES style nested tuples, {% get_language_info_list %} supports simple lists of
language codes. If you do this in your view:
return render_to_response(’mytemplate.html’, {
’available_languages’: [’en’, ’es’, ’fr’],
}, RequestContext(request))
you can iterate over those languages in the template:
{% get_language_info_list for available_languages as langs %}
{% for lang in langs %} ... {% endfor %}
There are also simple filters available for convenience:
• {{ LANGUAGE_CODE|language_name }} (“German”)
• {{ LANGUAGE_CODE|language_name_local }} (“Deutsch”)
• {{ LANGUAGE_CODE|bidi }} (False)
Internationalization: in JavaScript code
Adding translations to JavaScript poses some problems:
• JavaScript code doesn’t have access to a gettext implementation.
• JavaScript code doesn’t have access to .po or .mo files; they need to be delivered by the server.
• The translation catalogs for JavaScript should be kept as small as possible.
Django provides an integrated solution for these problems: It passes the translations into JavaScript, so you can call
gettext, etc., from within JavaScript.
3.14. Internationalization and localization 367](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-371-2048.jpg)
![Django Documentation, Release 1.5.1
var object_cnt = 1 // or 0, or 2, or 3, ...
s = ngettext(’literal for the singular case’,
’literal for the plural case’, object_cnt);
and even a string interpolation function:
function interpolate(fmt, obj, named);
The interpolation syntax is borrowed from Python, so the interpolate function supports both positional and named
interpolation:
• Positional interpolation: obj contains a JavaScript Array object whose elements values are then sequentially
interpolated in their corresponding fmt placeholders in the same order they appear. For example:
fmts = ngettext(’There is %s object. Remaining: %s’,
’There are %s objects. Remaining: %s’, 11);
s = interpolate(fmts, [11, 20]);
// s is ’There are 11 objects. Remaining: 20’
• Named interpolation: This mode is selected by passing the optional boolean named parameter as true. obj
contains a JavaScript object or associative array. For example:
d = {
count: 10,
total: 50
};
fmts = ngettext(’Total: %(total)s, there is %(count)s object’,
’there are %(count)s of a total of %(total)s objects’, d.count);
s = interpolate(fmts, d, true);
You shouldn’t go over the top with string interpolation, though: this is still JavaScript, so the code has to make repeated
regular-expression substitutions. This isn’t as fast as string interpolation in Python, so keep it to those cases where you
really need it (for example, in conjunction with ngettext to produce proper pluralizations).
Note on performance
The javascript_catalog() view generates the catalog from .mo files on every request. Since its output is
constant— at least for a given version of a site— it’s a good candidate for caching.
Server-side caching will reduce CPU load. It’s easily implemented with the cache_page() decorator. To trigger
cache invalidation when your translations change, provide a version-dependant key prefix, as shown in the example
below, or map the view at a version-dependant URL.
from django.views.decorators.cache import cache_page
from django.views.i18n import javascript_catalog
# The value returned by get_version() must change when translations change.
@cache_page(86400, key_prefix=’js18n-%s’ % get_version())
def cached_javascript_catalog(request, domain=’djangojs’, packages=None):
return javascript_catalog(request, domain, packages)
Client-side caching will save bandwidth and make your site load faster. If you’re using ETags (USE_ETAGS =
True), you’re already covered. Otherwise, you can apply conditional decorators. In the following example, the cache
is invalidated whenever your restart your application server.
from django.utils import timezone
from django.views.decorators.http import last_modified
3.14. Internationalization and localization 369](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-373-2048.jpg)
![Django Documentation, Release 1.5.1
from django.views.i18n import javascript_catalog
last_modified_date = timezone.now()
@last_modified(lambda req, **kw: last_modified_date)
def cached_javascript_catalog(request, domain=’djangojs’, packages=None):
return javascript_catalog(request, domain, packages)
You can even pre-generate the javascript catalog as part of your deployment procedure and serve it as a static file. This
radical technique is implemented in django-statici18n.
Internationalization: in URL patterns
New in version 1.4. Django provides two mechanisms to internationalize URL patterns:
• Adding the language prefix to the root of the URL patterns to make it possible for LocaleMiddleware to
detect the language to activate from the requested URL.
• Making URL patterns themselves translatable via the django.utils.translation.ugettext_lazy()
function.
Warning: Using either one of these features requires that an active language be set for each re-quest;
in other words, you need to have django.middleware.locale.LocaleMiddleware in your
MIDDLEWARE_CLASSES setting.
Language prefix in URL patterns
i18n_patterns(prefix, pattern_description, ...)
This function can be used in your root URLconf as a replacement for the normal
django.conf.urls.patterns() function. Django will automatically prepend the current active language
code to all url patterns defined within i18n_patterns(). Example URL patterns:
from django.conf.urls import patterns, include, url
from django.conf.urls.i18n import i18n_patterns
urlpatterns = patterns(’’,
url(r’^sitemap.xml$’, ’sitemap.view’, name=’sitemap_xml’),
)
news_patterns = patterns(’’,
url(r’^$’, ’news.views.index’, name=’index’),
url(r’^category/(?P<slug>[w-]+)/$’, ’news.views.category’, name=’category’),
url(r’^(?P<slug>[w-]+)/$’, ’news.views.details’, name=’detail’),
)
urlpatterns += i18n_patterns(’’,
url(r’^about/$’, ’about.view’, name=’about’),
url(r’^news/’, include(news_patterns, namespace=’news’)),
)
After defining these URL patterns, Django will automatically add the language prefix to the URL patterns that were
added by the i18n_patterns function. Example:
from django.core.urlresolvers import reverse
from django.utils.translation import activate
370 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-374-2048.jpg)
![Django Documentation, Release 1.5.1
>>> activate(’en’)
>>> reverse(’sitemap_xml’)
’/sitemap.xml’
>>> reverse(’news:index’)
’/en/news/’
>>> activate(’nl’)
>>> reverse(’news:detail’, kwargs={’slug’: ’news-slug’})
’/nl/news/news-slug/’
Warning: i18n_patterns() is only allowed in your root URLconf. Using it within an included URLconf
will throw an ImproperlyConfigured exception.
Warning: Ensure that you don’t have non-prefixed URL patterns that might collide with an automatically-added
language prefix.
Translating URL patterns
URL patterns can also be marked translatable using the ugettext_lazy() function. Example:
from django.conf.urls import patterns, include, url
from django.conf.urls.i18n import i18n_patterns
from django.utils.translation import ugettext_lazy as _
urlpatterns = patterns(’’
url(r’^sitemap.xml$’, ’sitemap.view’, name=’sitemap_xml’),
)
news_patterns = patterns(’’
url(r’^$’, ’news.views.index’, name=’index’),
url(_(r’^category/(?P<slug>[w-]+)/$’), ’news.views.category’, name=’category’),
url(r’^(?P<slug>[w-]+)/$’, ’news.views.details’, name=’detail’),
)
urlpatterns += i18n_patterns(’’,
url(_(r’^about/$’), ’about.view’, name=’about’),
url(_(r’^news/’), include(news_patterns, namespace=’news’)),
)
After you’ve created the translations, the reverse() function will return the URL in the active language. Example:
from django.core.urlresolvers import reverse
from django.utils.translation import activate
>>> activate(’en’)
>>> reverse(’news:category’, kwargs={’slug’: ’recent’})
’/en/news/category/recent/’
>>> activate(’nl’)
>>> reverse(’news:category’, kwargs={’slug’: ’recent’})
’/nl/nieuws/categorie/recent/’
Warning: In most cases, it’s best to use translated URLs only within a language-code-prefixed block of patterns
(using i18n_patterns()), to avoid the possibility that a carelessly translated URL causes a collision with a
non-translated URL pattern.
3.14. Internationalization and localization 371](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-375-2048.jpg)
![Django Documentation, Release 1.5.1
Create a view that can set the current timezone:
import pytz
from django.shortcuts import redirect, render
def set_timezone(request):
if request.method == ’POST’:
request.session[’django_timezone’] = pytz.timezone(request.POST[’timezone’])
return redirect(’/’)
else:
return render(request, ’template.html’, {’timezones’: pytz.common_timezones})
Include a form in template.html that will POST to this view:
{% load tz %}
<form action="{% url ’set_timezone’ %}" method="POST">
{% csrf_token %}
<label for="timezone">Time zone:</label>
<select name="timezone">
{% for tz in timezones %}
<option value="{{ tz }}"{% if tz == TIME_ZONE %} selected="selected"{% endif %}>{{ tz }}</option>
{% endfor %}
</select>
<input type="submit" value="Set" />
</form>
Time zone aware input in forms
When you enable time zone support, Django interprets datetimes entered in forms in the current time zone and returns
aware datetime objects in cleaned_data.
If the current time zone raises an exception for datetimes that don’t exist or are ambiguous because they fall in a DST
transition (the timezones provided by pytz do this), such datetimes will be reported as invalid values.
Time zone aware output in templates
When you enable time zone support, Django converts aware datetime objects to the current time zone when they’re
rendered in templates. This behaves very much like format localization.
Warning: Django doesn’t convert naive datetime objects, because they could be ambiguous, and because your
code should never produce naive datetimes when time zone support is enabled. However, you can force conversion
with the template filters described below.
Conversion to local time isn’t always appropriate – you may be generating output for computers rather than for humans.
The following filters and tags, provided by the tz template tag library, allow you to control the time zone conversions.
Template tags
localtime Enables or disables conversion of aware datetime objects to the current time zone in the contained block.
This tag has exactly the same effects as the USE_TZ setting as far as the template engine is concerned. It allows a
more fine grained control of conversion.
To activate or deactivate conversion for a template block, use:
3.14. Internationalization and localization 385](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-389-2048.jpg)
![Django Documentation, Release 1.5.1
’class’: ’django.utils.log.NullHandler’,
},
’console’:{
’level’: ’DEBUG’,
’class’: ’logging.StreamHandler’,
’formatter’: ’simple’
},
’mail_admins’: {
’level’: ’ERROR’,
’class’: ’django.utils.log.AdminEmailHandler’,
’filters’: [’special’]
}
},
’loggers’: {
’django’: {
’handlers’: [’null’],
’propagate’: True,
’level’: ’INFO’,
},
’django.request’: {
’handlers’: [’mail_admins’],
’level’: ’ERROR’,
’propagate’: False,
},
’myproject.custom’: {
’handlers’: [’console’, ’mail_admins’],
’level’: ’INFO’,
’filters’: [’special’]
}
}
}
This logging configuration does the following things:
• Identifies the configuration as being in ‘dictConfig version 1’ format. At present, this is the only dictConfig
format version.
• Disables all existing logging configurations.
• Defines two formatters:
– simple, that just outputs the log level name (e.g., DEBUG) and the log message.
The format string is a normal Python formatting string describing the details that are to be output on
each logging line. The full list of detail that can be output can be found in the formatter documentation.
– verbose, that outputs the log level name, the log message, plus the time, process, thread and module that
generate the log message.
• Defines one filter – project.logging.SpecialFilter, using the alias special. If this filter re-quired
additional arguments at time of construction, they can be provided as additional keys in the filter con-figuration
dictionary. In this case, the argument foo will be given a value of bar when instantiating the
SpecialFilter.
• Defines three handlers:
– null, a NullHandler, which will pass any DEBUG (or higher) message to /dev/null.
– console, a StreamHandler, which will print any DEBUG (or higher) message to stderr. This handler uses
the simple output format.
396 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-400-2048.jpg)
![Django Documentation, Release 1.5.1
django.request
Log messages related to the handling of requests. 5XX responses are raised as ERROR messages; 4XX responses are
raised as WARNING messages.
Messages to this logger have the following extra context:
• status_code: The HTTP response code associated with the request.
• request: The request object that generated the logging message.
django.db.backends
Messages relating to the interaction of code with the database. For example, every SQL statement executed by a
request is logged at the DEBUG level to this logger.
Messages to this logger have the following extra context:
• duration: The time taken to execute the SQL statement.
• sql: The SQL statement that was executed.
• params: The parameters that were used in the SQL call.
For performance reasons, SQL logging is only enabled when settings.DEBUG is set to True, regardless of the
logging level or handlers that are installed.
Handlers
Django provides one log handler in addition to those provided by the Python logging module.
class AdminEmailHandler([include_html=False ])
This handler sends an email to the site admins for each log message it receives.
If the log record contains a request attribute, the full details of the request will be included in the email.
If the log record contains stack trace information, that stack trace will be included in the email.
The include_html argument of AdminEmailHandler is used to control whether the traceback email
includes an HTML attachment containing the full content of the debug Web page that would have been pro-duced
if DEBUG were True. To set this value in your configuration, include it in the handler definition for
django.utils.log.AdminEmailHandler, like this:
’handlers’: {
’mail_admins’: {
’level’: ’ERROR’,
’class’: ’django.utils.log.AdminEmailHandler’,
’include_html’: True,
}
},
Note that this HTML version of the email contains a full traceback, with names and values of local variables at
each level of the stack, plus the values of your Django settings. This information is potentially very sensitive,
and you may not want to send it over email. Consider using something such as Sentry to get the best of both
worlds – the rich information of full tracebacks plus the security of not sending the information over email. You
may also explicitly designate certain sensitive information to be filtered out of error reports – learn more on
Filtering error reports.
398 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-402-2048.jpg)
![Django Documentation, Release 1.5.1
Filters
Django provides two log filters in addition to those provided by the Python logging module.
class CallbackFilter(callback)
New in version 1.4. This filter accepts a callback function (which should accept a single argument, the record to
be logged), and calls it for each record that passes through the filter. Handling of that record will not proceed if
the callback returns False.
For instance, to filter out UnreadablePostError (raised when a user cancels an upload) from the admin
emails, you would create a filter function:
from django.http import UnreadablePostError
def skip_unreadable_post(record):
if record.exc_info:
exc_type, exc_value = record.exc_info[:2]
if isinstance(exc_value, UnreadablePostError):
return False
return True
and then add it to your logging config:
’filters’: {
’skip_unreadable_posts’: {
’()’: ’django.utils.log.CallbackFilter’,
’callback’: skip_unreadable_post,
}
},
’handlers’: {
’mail_admins’: {
’level’: ’ERROR’,
’filters’: [’skip_unreadable_posts’],
’class’: ’django.utils.log.AdminEmailHandler’
}
},
class RequireDebugFalse
New in version 1.4. This filter will only pass on records when settings.DEBUG is False.
This filter is used as follows in the default LOGGING configuration to ensure that the AdminEmailHandler
only sends error emails to admins when DEBUG is False:
’filters’: {
’require_debug_false’: {
’()’: ’django.utils.log.RequireDebugFalse’,
}
},
’handlers’: {
’mail_admins’: {
’level’: ’ERROR’,
’filters’: [’require_debug_false’],
’class’: ’django.utils.log.AdminEmailHandler’
}
},
class RequireDebugTrue
New in version 1.5. This filter is similar to RequireDebugFalse, except that records are passed only when
DEBUG is True.
3.15. Logging 399](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-403-2048.jpg)
![Django Documentation, Release 1.5.1
3.15.5 Django’s default logging configuration
By default, Django configures the django.request logger so that all messages with ERROR or CRITICAL level
are sent to AdminEmailHandler, as long as the DEBUG setting is set to False.
All messages reaching the django catch-all logger when DEBUG is True are sent to the console. They are simply
discarded (sent to NullHandler) when DEBUG is False. Changed in version 1.5. See also Configuring logging to
learn how you can complement or replace this default logging configuration.
3.16 Pagination
Django provides a few classes that help you manage paginated data – that is, data that’s split across several pages, with
“Previous/Next” links. These classes live in django/core/paginator.py.
3.16.1 Example
Give Paginator a list of objects, plus the number of items you’d like to have on each page, and it gives you methods
for accessing the items for each page:
>>> from django.core.paginator import Paginator
>>> objects = [’john’, ’paul’, ’george’, ’ringo’]
>>> p = Paginator(objects, 2)
>>> p.count
4
>>> p.num_pages
2
>>> p.page_range
[1, 2]
>>> page1 = p.page(1)
>>> page1
<Page 1 of 2>
>>> page1.object_list
[’john’, ’paul’]
>>> page2 = p.page(2)
>>> page2.object_list
[’george’, ’ringo’]
>>> page2.has_next()
False
>>> page2.has_previous()
True
>>> page2.has_other_pages()
True
>>> page2.next_page_number()
Traceback (most recent call last):
...
EmptyPage: That page contains no results
>>> page2.previous_page_number()
1
>>> page2.start_index() # The 1-based index of the first item on this page
3
>>> page2.end_index() # The 1-based index of the last item on this page
4
400 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-404-2048.jpg)
![Django Documentation, Release 1.5.1
Paginator.num_pages
The total number of pages.
Paginator.page_range
A 1-based range of page numbers, e.g., [1, 2, 3, 4].
3.16.4 InvalidPage exceptions
exception InvalidPage
A base class for exceptions raised when a paginator is passed an invalid page number.
The Paginator.page() method raises an exception if the requested page is invalid (i.e., not an integer) or contains
no objects. Generally, it’s enough to trap the InvalidPage exception, but if you’d like more granularity, you can
trap either of the following exceptions:
exception PageNotAnInteger
Raised when page() is given a value that isn’t an integer.
exception EmptyPage
Raised when page() is given a valid value but no objects exist on that page.
Both of the exceptions are subclasses of InvalidPage, so you can handle them both with a simple except
InvalidPage.
3.16.5 Page objects
You usually won’t construct Page objects by hand – you’ll get them using Paginator.page().
class Page(object_list, number, paginator)
New in version 1.4: A page acts like a sequence of Page.object_list when using len() or iterating it directly.
Methods
Page.has_next()
Returns True if there’s a next page.
Page.has_previous()
Returns True if there’s a previous page.
Page.has_other_pages()
Returns True if there’s a next or previous page.
Page.next_page_number()
Returns the next page number. Changed in version 1.5. Raises InvalidPage if next page doesn’t exist.
Page.previous_page_number()
Returns the previous page number. Changed in version 1.5. Raises InvalidPage if previous page doesn’t
exist.
Page.start_index()
Returns the 1-based index of the first object on the page, relative to all of the objects in the paginator’s list.
For example, when paginating a list of 5 objects with 2 objects per page, the second page’s start_index()
would return 3.
3.16. Pagination 403](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-407-2048.jpg)
![Django Documentation, Release 1.5.1
name = models.CharField(max_length=100)
author = models.ForeignKey(Person)
Ordinarily, serialized data for Book would use an integer to refer to the author. For example, in JSON, a Book might
be serialized as:
...
{
pk: 1,
model: store.book,
fields: {
name: Mostly Harmless,
author: 42
}
}
...
This isn’t a particularly natural way to refer to an author. It requires that you know the primary key value for the
author; it also requires that this primary key value is stable and predictable.
However, if we add natural key handling to Person, the fixture becomes much more humane. To add natural key
handling, you define a default Manager for Person with a get_by_natural_key() method. In the case of a
Person, a good natural key might be the pair of first and last name:
from django.db import models
class PersonManager(models.Manager):
def get_by_natural_key(self, first_name, last_name):
return self.get(first_name=first_name, last_name=last_name)
class Person(models.Model):
objects = PersonManager()
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
birthdate = models.DateField()
class Meta:
unique_together = ((’first_name’, ’last_name’),)
Now books can use that natural key to refer to Person objects:
...
{
pk: 1,
model: store.book,
fields: {
name: Mostly Harmless,
author: [Douglas, Adams]
}
}
...
When you try to load this serialized data, Django will use the get_by_natural_key() method to resolve
[Douglas, Adams] into the primary key of an actual Person object.
Note: Whatever fields you use for a natural key must be able to uniquely identify an object. This will usually mean
that your model will have a uniqueness clause (either unique=True on a single field, or unique_together over
multiple fields) for the field or fields in your natural key. However, uniqueness doesn’t need to be enforced at the
416 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-420-2048.jpg)
![Django Documentation, Release 1.5.1
database level. If you are certain that a set of fields will be effectively unique, you can still use those fields as a natural
key.
Serialization of natural keys
So how do you get Django to emit a natural key when serializing an object? Firstly, you need to add another method
– this time to the model itself:
class Person(models.Model):
objects = PersonManager()
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
birthdate = models.DateField()
def natural_key(self):
return (self.first_name, self.last_name)
class Meta:
unique_together = ((’first_name’, ’last_name’),)
That method should always return a natural key tuple – in this example, (first name, last name). Then,
when you call serializers.serialize(), you provide a use_natural_keys=True argument:
serializers.serialize(’json’, [book1, book2], indent=2, use_natural_keys=True)
When use_natural_keys=True is specified, Django will use the natural_key() method to serialize any
reference to objects of the type that defines the method.
If you are using dumpdata to generate serialized data, you use the --natural command line flag to generate
natural keys.
Note: You don’t need to define both natural_key() and get_by_natural_key(). If you don’t want Django
to output natural keys during serialization, but you want to retain the ability to load natural keys, then you can opt to
not implement the natural_key() method.
Conversely, if (for some strange reason) you want Django to output natural keys during serialization, but not be able
to load those key values, just don’t define the get_by_natural_key() method.
Dependencies during serialization
Since natural keys rely on database lookups to resolve references, it is important that the data exists before it is
referenced. You can’t make a “forward reference” with natural keys – the data you’re referencing must exist before
you include a natural key reference to that data.
To accommodate this limitation, calls to dumpdata that use the --natural option will serialize any model with a
natural_key() method before serializing standard primary key objects.
However, this may not always be enough. If your natural key refers to another object (by using a foreign key or natural
key to another object as part of a natural key), then you need to be able to ensure that the objects on which a natural
key depends occur in the serialized data before the natural key requires them.
To control this ordering, you can define dependencies on your natural_key() methods. You do this by setting a
dependencies attribute on the natural_key() method itself.
3.19. Serializing Django objects 417](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-421-2048.jpg)
![Django Documentation, Release 1.5.1
For example, let’s add a natural key to the Book model from the example above:
class Book(models.Model):
name = models.CharField(max_length=100)
author = models.ForeignKey(Person)
def natural_key(self):
return (self.name,) + self.author.natural_key()
The natural key for a Book is a combination of its name and its author. This means that Person must be serialized
before Book. To define this dependency, we add one extra line:
def natural_key(self):
return (self.name,) + self.author.natural_key()
natural_key.dependencies = [’example_app.person’]
This definition ensures that all Person objects are serialized before any Book objects. In turn, any object referencing
Book will be serialized after both Person and Book have been serialized.
3.20 Django settings
A Django settings file contains all the configuration of your Django installation. This document explains how settings
work and which settings are available.
3.20.1 The basics
A settings file is just a Python module with module-level variables.
Here are a couple of example settings:
DEBUG = False
DEFAULT_FROM_EMAIL = ’webmaster@example.com’
TEMPLATE_DIRS = (’/home/templates/mike’, ’/home/templates/john’)
Because a settings file is a Python module, the following apply:
• It doesn’t allow for Python syntax errors.
• It can assign settings dynamically using normal Python syntax. For example:
MY_SETTING = [str(i) for i in range(30)]
• It can import values from other settings files.
3.20.2 Designating the settings
DJANGO_SETTINGS_MODULE
When you use Django, you have to tell it which settings you’re using. Do this by using an environment variable,
DJANGO_SETTINGS_MODULE.
The value of DJANGO_SETTINGS_MODULE should be in Python path syntax, e.g. mysite.settings. Note that
the settings module should be on the Python import search path.
418 Chapter 3. Using Django](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-422-2048.jpg)
![Django Documentation, Release 1.5.1
The django-admin.py utility
When using django-admin.py, you can either set the environment variable once, or explicitly pass in the settings
module each time you run the utility.
Example (Unix Bash shell):
export DJANGO_SETTINGS_MODULE=mysite.settings
django-admin.py runserver
Example (Windows shell):
set DJANGO_SETTINGS_MODULE=mysite.settings
django-admin.py runserver
Use the --settings command-line argument to specify the settings manually:
django-admin.py runserver --settings=mysite.settings
On the server (mod_wsgi)
In your live server environment, you’ll need to tell your WSGI application what settings file to use. Do that with
os.environ:
import os
os.environ[’DJANGO_SETTINGS_MODULE’] = ’mysite.settings’
Read the Django mod_wsgi documentation for more information and other common elements to a Django WSGI
application.
3.20.3 Default settings
A Django settings file doesn’t have to define any settings if it doesn’t need to. Each setting has a sensible default value.
These defaults live in the module django/conf/global_settings.py.
Here’s the algorithm Django uses in compiling settings:
• Load settings from global_settings.py.
• Load settings from the specified settings file, overriding the global settings as necessary.
Note that a settings file should not import from global_settings, because that’s redundant.
Seeing which settings you’ve changed
There’s an easy way to view which of your settings deviate from the default settings. The command python
manage.py diffsettings displays differences between the current settings file and Django’s default settings.
For more, see the diffsettings documentation.
3.20.4 Using settings in Python code
In your Django apps, use settings by importing the object django.conf.settings. Example:
3.20. Django settings 419](https://image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-423-2048.jpg)
![Django Documentation, Release 1.5.1
manage.py syncdb
The syncdb command looks at all your available models and creates tables in your database for whichever tables
don’t already exist.
2.1.3 Enjoy the free API
With that, you’ve got a free, and rich, Python API to access your data. The API is created on the fly, no code generation
necessary:
# Import the models we created from our "news" app
>>> from news.models import Reporter, Article
# No reporters are in the system yet.
>>> Reporter.objects.all()
[]
# Create a new Reporter.
>>> r = Reporter(full_name=’John Smith’)
# Save the object into the database. You have to call save() explicitly.
>>> r.save()
# Now it has an ID.
>>> r.id
1
# Now the new reporter is in the database.
>>> Reporter.objects.all()
[<Reporter: John Smith>]
# Fields are represented as attributes on the Python object.
>>> r.full_name
’John Smith’
# Django provides a rich database lookup API.
>>> Reporter.objects.get(id=1)
<Reporter: John Smith>
>>> Reporter.objects.get(full_name__startswith=’John’)
<Reporter: John Smith>
>>> Reporter.objects.get(full_name__contains=’mith’)
<Reporter: John Smith>
>>> Reporter.objects.get(id=2)
Traceback (most recent call last):
...
DoesNotExist: Reporter matching query does not exist. Lookup parameters were {’id’: 2}
# Create an article.
>>> from datetime import date
>>> a = Article(pub_date=date.today(), headline=’Django is cool’,
... content=’Yeah.’, reporter=r)
>>> a.save()
# Now the article is in the database.
>>> Article.objects.all()
[<Article: Django is cool>]
8 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-12-2048.jpg)
![Django Documentation, Release 1.5.1
# Article objects get API access to related Reporter objects.
>>> r = a.reporter
>>> r.full_name
’John Smith’
# And vice versa: Reporter objects get API access to Article objects.
>>> r.article_set.all()
[<Article: Django is cool>]
# The API follows relationships as far as you need, performing efficient
# JOINs for you behind the scenes.
# This finds all articles by a reporter whose name starts with "John".
>>> Article.objects.filter(reporter__full_name__startswith="John")
[<Article: Django is cool>]
# Change an object by altering its attributes and calling save().
>>> r.full_name = ’Billy Goat’
>>> r.save()
# Delete an object with delete().
>>> r.delete()
2.1.4 A dynamic admin interface: it’s not just scaffolding – it’s the whole house
Once your models are defined, Django can automatically create a professional, production ready administrative inter-face
– a Web site that lets authenticated users add, change and delete objects. It’s as easy as registering your model in
the admin site:
# In models.py...
from django.db import models
class Article(models.Model):
pub_date = models.DateField()
headline = models.CharField(max_length=200)
content = models.TextField()
reporter = models.ForeignKey(Reporter)
# In admin.py in the same directory...
import models
from django.contrib import admin
admin.site.register(models.Article)
The philosophy here is that your site is edited by a staff, or a client, or maybe just you – and you don’t want to have to
deal with creating backend interfaces just to manage content.
One typical workflow in creating Django apps is to create models and get the admin sites up and running as fast as
possible, so your staff (or clients) can start populating data. Then, develop the way data is presented to the public.
2.1.5 Design your URLs
A clean, elegant URL scheme is an important detail in a high-quality Web application. Django encourages beautiful
URL design and doesn’t put any cruft in URLs, like .php or .asp.
2.1. Django at a glance 9](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-13-2048.jpg)
![Django Documentation, Release 1.5.1
The next obvious steps are for you to download Django, read the tutorial and join the community. Thanks for your
interest!
2.2 Quick install guide
Before you can use Django, you’ll need to get it installed. We have a complete installation guide that covers all
the possibilities; this guide will guide you to a simple, minimal installation that’ll work while you walk through the
introduction.
2.2.1 Install Python
Being a Python Web framework, Django requires Python. It works with any Python version from 2.6.5 to 2.7. It also
features experimental support for versions 3.2 and 3.3. All these versions of Python include a lightweight database
called SQLite so you won’t need to set up a database just yet.
Get Python at http://www.python.org. If you’re running Linux or Mac OS X, you probably already have it installed.
Django on Jython
If you use Jython (a Python implementation for the Java platform), you’ll need to follow a few additional steps. See
Running Django on Jython for details.
You can verify that Python is installed by typing python from your shell; you should see something like:
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
2.2.2 Set up a database
This step is only necessary if you’d like to work with a “large” database engine like PostgreSQL, MySQL, or Oracle.
To install such a database, consult the database installation information.
2.2.3 Remove any old versions of Django
If you are upgrading your installation of Django from a previous version, you will need to uninstall the old Django
version before installing the new version.
2.2.4 Install Django
You’ve got three easy options to install Django:
• Install a version of Django provided by your operating system distribution. This is the quickest option for those
who have operating systems that distribute Django.
• Install an official release. This is the best approach for users who want a stable version number and aren’t
concerned about running a slightly older version of Django.
• Install the latest development version. This is best for users who want the latest-and-greatest features and aren’t
afraid of running brand-new code.
12 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-16-2048.jpg)
![Django Documentation, Release 1.5.1
project since the last time you ran syncdb. syncdb can be called as often as you like, and it will only ever create the
tables that don’t exist.
Read the django-admin.py documentation for full information on what the manage.py utility can do.
2.3.4 Playing with the API
Now, let’s hop into the interactive Python shell and play around with the free API Django gives you. To invoke the
Python shell, use this command:
python manage.py shell
We’re using this instead of simply typing “python”, because manage.py sets the DJANGO_SETTINGS_MODULE
environment variable, which gives Django the Python import path to your mysite/settings.py file.
Bypassing manage.py
If you’d rather not use manage.py, no problem. Just set the DJANGO_SETTINGS_MODULE environment variable
to mysite.settings and run python from the same directory manage.py is in (or ensure that directory is on
the Python path, so that import mysite works).
For more information on all of this, see the django-admin.py documentation.
Once you’re in the shell, explore the database API:
>>> from polls.models import Poll, Choice # Import the model classes we just wrote.
# No polls are in the system yet.
>>> Poll.objects.all()
[]
# Create a new Poll.
# Support for time zones is enabled in the default settings file, so
# Django expects a datetime with tzinfo for pub_date. Use timezone.now()
# instead of datetime.datetime.now() and it will do the right thing.
>>> from django.utils import timezone
>>> p = Poll(question="What’s new?", pub_date=timezone.now())
# Save the object into the database. You have to call save() explicitly.
>>> p.save()
# Now it has an ID. Note that this might say "1L" instead of "1", depending
# on which database you’re using. That’s no biggie; it just means your
# database backend prefers to return integers as Python long integer
# objects.
>>> p.id
1
# Access database columns via Python attributes.
>>> p.question
"What’s new?"
>>> p.pub_date
datetime.datetime(2012, 2, 26, 13, 0, 0, 775217, tzinfo=<UTC>)
# Change values by changing the attributes, then calling save().
>>> p.question = "What’s up?"
>>> p.save()
20 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-24-2048.jpg)
![Django Documentation, Release 1.5.1
# objects.all() displays all the polls in the database.
>>> Poll.objects.all()
[<Poll: Poll object>]
Wait a minute. <Poll: Poll object> is, utterly, an unhelpful representation of this object. Let’s fix that by
editing the polls model (in the polls/models.py file) and adding a __unicode__() method to both Poll and
Choice:
class Poll(models.Model):
# ...
def __unicode__(self):
return self.question
class Choice(models.Model):
# ...
def __unicode__(self):
return self.choice_text
It’s important to add __unicode__() methods to your models, not only for your own sanity when dealing with the
interactive prompt, but also because objects’ representations are used throughout Django’s automatically-generated
admin.
Why __unicode__() and not __str__()?
If you’re familiar with Python, you might be in the habit of adding __str__() methods to your classes, not
__unicode__() methods. We use __unicode__() here because Django models deal with Unicode by default.
All data stored in your database is converted to Unicode when it’s returned.
Django models have a default __str__() method that calls __unicode__() and converts the result to a UTF-8
bytestring. This means that unicode(p) will return a Unicode string, and str(p) will return a normal string, with
characters encoded as UTF-8.
If all of this is gibberish to you, just remember to add __unicode__() methods to your models. With any luck,
things should Just Work for you.
Note these are normal Python methods. Let’s add a custom method, just for demonstration:
import datetime
from django.utils import timezone
# ...
class Poll(models.Model):
# ...
def was_published_recently(self):
return self.pub_date >= timezone.now() - datetime.timedelta(days=1)
Note the addition of import datetime and from django.utils import timezone, to reference
Python’s standard datetime module and Django’s time-zone-related utilities in django.utils.timezone,
respectively. If you aren’t familiar with time zone handling in Python, you can learn more in the time zone support
docs.
Save these changes and start a new Python interactive shell by running python manage.py shell again:
>>> from polls.models import Poll, Choice
# Make sure our __unicode__() addition worked.
>>> Poll.objects.all()
[<Poll: What’s up?>]
2.3. Writing your first Django app, part 1 21](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-25-2048.jpg)
![Django Documentation, Release 1.5.1
# Django provides a rich database lookup API that’s entirely driven by
# keyword arguments.
>>> Poll.objects.filter(id=1)
[<Poll: What’s up?>]
>>> Poll.objects.filter(question__startswith=’What’)
[<Poll: What’s up?>]
# Get the poll that was published this year.
>>> from django.utils import timezone
>>> current_year = timezone.now().year
>>> Poll.objects.get(pub_date__year=current_year)
<Poll: What’s up?>
# Request an ID that doesn’t exist, this will raise an exception.
>>> Poll.objects.get(id=2)
Traceback (most recent call last):
...
DoesNotExist: Poll matching query does not exist. Lookup parameters were {’id’: 2}
# Lookup by a primary key is the most common case, so Django provides a
# shortcut for primary-key exact lookups.
# The following is identical to Poll.objects.get(id=1).
>>> Poll.objects.get(pk=1)
<Poll: What’s up?>
# Make sure our custom method worked.
>>> p = Poll.objects.get(pk=1)
>>> p.was_published_recently()
True
# Give the Poll a couple of Choices. The create call constructs a new
# Choice object, does the INSERT statement, adds the choice to the set
# of available choices and returns the new Choice object. Django creates
# a set to hold the "other side" of a ForeignKey relation
# (e.g. a poll’s choices) which can be accessed via the API.
>>> p = Poll.objects.get(pk=1)
# Display any choices from the related object set -- none so far.
>>> p.choice_set.all()
[]
# Create three choices.
>>> p.choice_set.create(choice_text=’Not much’, votes=0)
<Choice: Not much>
>>> p.choice_set.create(choice_text=’The sky’, votes=0)
<Choice: The sky>
>>> c = p.choice_set.create(choice_text=’Just hacking again’, votes=0)
# Choice objects have API access to their related Poll objects.
>>> c.poll
<Poll: What’s up?>
# And vice versa: Poll objects get access to Choice objects.
>>> p.choice_set.all()
[<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>]
>>> p.choice_set.count()
3
22 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-26-2048.jpg)
![Django Documentation, Release 1.5.1
# The API automatically follows relationships as far as you need.
# Use double underscores to separate relationships.
# This works as many levels deep as you want; there’s no limit.
# Find all Choices for any poll whose pub_date is in this year
# (reusing the ’current_year’ variable we created above).
>>> Choice.objects.filter(poll__pub_date__year=current_year)
[<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>]
# Let’s delete one of the choices. Use delete() for that.
>>> c = p.choice_set.filter(choice_text__startswith=’Just hacking’)
>>> c.delete()
For more information on model relations, see Accessing related objects. For more on how to use double underscores
to perform field lookups via the API, see Field lookups. For full details on the database API, see our Database API
reference.
When you’re comfortable with the API, read part 2 of this tutorial to get Django’s automatic admin working.
2.4 Writing your first Django app, part 2
This tutorial begins where Tutorial 1 left off. We’re continuing the Web-poll application and will focus on Django’s
automatically-generated admin site.
Philosophy
Generating admin sites for your staff or clients to add, change and delete content is tedious work that doesn’t require
much creativity. For that reason, Django entirely automates creation of admin interfaces for models.
Django was written in a newsroom environment, with a very clear separation between “content publishers” and the
“public” site. Site managers use the system to add news stories, events, sports scores, etc., and that content is displayed
on the public site. Django solves the problem of creating a unified interface for site administrators to edit content.
The admin isn’t intended to be used by site visitors. It’s for site managers.
2.4.1 Activate the admin site
The Django admin site is not activated by default – it’s an opt-in thing. To activate the admin site for your installation,
do these three things:
• Uncomment "django.contrib.admin" in the INSTALLED_APPS setting.
• Run python manage.py syncdb. Since you have added a new application to INSTALLED_APPS, the
database tables need to be updated.
• Edit your mysite/urls.py file and uncomment the lines that reference the admin – there are three lines in
total to uncomment. This file is a URLconf; we’ll dig into URLconfs in the next tutorial. For now, all you need
to know is that it maps URL roots to applications. In the end, you should have a urls.py file that looks like
this:
from django.conf.urls import patterns, include, url
# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()
2.4. Writing your first Django app, part 2 23](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-27-2048.jpg)
![Django Documentation, Release 1.5.1
2.4.6 Customize the admin form
Take a few minutes to marvel at all the code you didn’t have to write. By registering the Poll model with
admin.site.register(Poll), Django was able to construct a default form representation. Often, you’ll want
to customize how the admin form looks and works. You’ll do this by telling Django the options you want when you
register the object.
Let’s see how this works by re-ordering the fields on the edit form. Replace the admin.site.register(Poll)
line with:
class PollAdmin(admin.ModelAdmin):
fields = [’pub_date’, ’question’]
admin.site.register(Poll, PollAdmin)
You’ll follow this pattern – create a model admin object, then pass it as the second argument to
admin.site.register() – any time you need to change the admin options for an object.
This particular change above makes the “Publication date” come before the “Question” field:
This isn’t impressive with only two fields, but for admin forms with dozens of fields, choosing an intuitive order is an
important usability detail.
And speaking of forms with dozens of fields, you might want to split the form up into fieldsets:
2.4. Writing your first Django app, part 2 27](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-31-2048.jpg)
![Django Documentation, Release 1.5.1
class PollAdmin(admin.ModelAdmin):
fieldsets = [
(None, {’fields’: [’question’]}),
(’Date information’, {’fields’: [’pub_date’]}),
]
admin.site.register(Poll, PollAdmin)
The first element of each tuple in fieldsets is the title of the fieldset. Here’s what our form looks like now:
You can assign arbitrary HTML classes to each fieldset. Django provides a "collapse" class that displays a
particular fieldset initially collapsed. This is useful when you have a long form that contains a number of fields that
aren’t commonly used:
class PollAdmin(admin.ModelAdmin):
fieldsets = [
(None, {’fields’: [’question’]}),
(’Date information’, {’fields’: [’pub_date’], ’classes’: [’collapse’]}),
]
28 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-32-2048.jpg)
![Django Documentation, Release 1.5.1
class PollAdmin(admin.ModelAdmin):
fieldsets = [
(None, {’fields’: [’question’]}),
(’Date information’, {’fields’: [’pub_date’], ’classes’: [’collapse’]}),
]
inlines = [ChoiceInline]
admin.site.register(Poll, PollAdmin)
This tells Django: “Choice objects are edited on the Poll admin page. By default, provide enough fields for 3
choices.”
Load the “Add poll” page to see how that looks, you may need to restart your development server:
It works like this: There are three slots for related Choices – as specified by extra – and each time you come back
to the “Change” page for an already-created object, you get another three extra slots.
At the end of the three current slots you will find an “Add another Choice” link. If you click on it, a new slot will be
added. If you want to remove the added slot, you can click on the X to the top right of the added slot. Note that you
can’t remove the original three slots. This image shows an added slot:
30 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-34-2048.jpg)
![Django Documentation, Release 1.5.1
By default, Django displays the str() of each object. But sometimes it’d be more helpful if we could display
individual fields. To do that, use the list_display admin option, which is a tuple of field names to display, as
columns, on the change list page for the object:
class PollAdmin(admin.ModelAdmin):
# ...
list_display = (’question’, ’pub_date’)
Just for good measure, let’s also include the was_published_recently custom method from Tutorial 1:
class PollAdmin(admin.ModelAdmin):
# ...
list_display = (’question’, ’pub_date’, ’was_published_recently’)
Now the poll change list page looks like this:
You can click on the column headers to sort by those values – except in the case of the was_published_recently
header, because sorting by the output of an arbitrary method is not supported. Also note that the column header for
was_published_recently is, by default, the name of the method (with underscores replaced with spaces), and
that each line contains the string representation of the output.
You can improve that by giving that method (in polls/models.py) a few attributes, as follows:
class Poll(models.Model):
# ...
def was_published_recently(self):
return self.pub_date >= timezone.now() - datetime.timedelta(days=1)
was_published_recently.admin_order_field = ’pub_date’
was_published_recently.boolean = True
was_published_recently.short_description = ’Published recently?’
Edit your polls/admin.py file again and add an improvement to the Poll change list page: Filters. Add the
following line to PollAdmin:
list_filter = [’pub_date’]
32 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-36-2048.jpg)
![Django Documentation, Release 1.5.1
That adds a “Filter” sidebar that lets people filter the change list by the pub_date field:
The type of filter displayed depends on the type of field you’re filtering on. Because pub_date is a
DateTimeField, Django knows to give appropriate filter options: “Any date,” “Today,” “Past 7 days,” “This
month,” “This year.”
This is shaping up well. Let’s add some search capability:
search_fields = [’question’]
That adds a search box at the top of the change list. When somebody enters search terms, Django will search the
question field. You can use as many fields as you’d like – although because it uses a LIKE query behind the
scenes, keep it reasonable, to keep your database happy.
Finally, because Poll objects have dates, it’d be convenient to be able to drill down by date. Add this line:
date_hierarchy = ’pub_date’
That adds hierarchical navigation, by date, to the top of the change list page. At top level, it displays all available
years. Then it drills down to months and, ultimately, days.
Now’s also a good time to note that change lists give you free pagination. The default is to display 100 items per page.
Change-list pagination, search boxes, filters, date-hierarchies and column-header-ordering all work together like you
think they should.
2.4.9 Customize the admin look and feel
Clearly, having “Django administration” at the top of each admin page is ridiculous. It’s just placeholder text.
That’s easy to change, though, using Django’s template system. The Django admin is powered by Django itself, and
its interfaces use Django’s own template system.
Customizing your project’s templates
Create a templates directory in your project directory. Templates can live anywhere on your filesystem that Django
can access. (Django runs as whatever user your server runs.) However, keeping your templates within the project is a
good convention to follow.
Open your settings file (mysite/settings.py, remember) and add a TEMPLATE_DIRS setting:
TEMPLATE_DIRS = (
’/path/to/mysite/templates’, # Change this to your own directory.
)
Now copy the template admin/base_site.html from within the default Django admin tem-plate
directory in the source code of Django itself (django/contrib/admin/templates)
2.4. Writing your first Django app, part 2 33](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-37-2048.jpg)
![Django Documentation, Release 1.5.1
into an admin subdirectory of whichever directory you’re using in TEMPLATE_DIRS.
For example, if your TEMPLATE_DIRS includes ’/path/to/mysite/templates’, as
above, then copy django/contrib/admin/templates/admin/base_site.html to
/path/to/mysite/templates/admin/base_site.html. Don’t forget that admin subdirectory.
Where are the Django source files?
If you have difficulty finding where the Django source files are located on your system, run the following command:
python -c "
import sys
sys.path = sys.path[1:]
import django
print(django.__path__)"
Then, just edit the file and replace the generic Django text with your own site’s name as you see fit.
This template file contains lots of text like {% block branding %} and {{ title }}. The {% and {{ tags
are part of Django’s template language. When Django renders admin/base_site.html, this template language
will be evaluated to produce the final HTML page. Don’t worry if you can’t make any sense of the template right now
– we’ll delve into Django’s templating language in Tutorial 3.
Note that any of Django’s default admin templates can be overridden. To override a template, just do the same thing
you did with base_site.html – copy it from the default directory into your custom directory, and make changes.
Customizing your application’s templates
Astute readers will ask: But if TEMPLATE_DIRS was empty by default, how was Django finding the default admin
templates? The answer is that, by default, Django automatically looks for a templates/ subdirectory within each
application package, for use as a fallback (don’t forget that django.contrib.admin is an application).
Our poll application is not very complex and doesn’t need custom admin templates. But if it grew more sophisticated
and required modification of Django’s standard admin templates for some of its functionality, it would be more sensible
to modify the application’s templates, rather than those in the project. That way, you could include the polls application
in any new project and be assured that it would find the custom templates it needed.
See the template loader documentation for more information about how Django finds its templates.
2.4.10 Customize the admin index page
On a similar note, you might want to customize the look and feel of the Django admin index page.
By default, it displays all the apps in INSTALLED_APPS that have been registered with the admin application, in
alphabetical order. You may want to make significant changes to the layout. After all, the index is probably the most
important page of the admin, and it should be easy to use.
The template to customize is admin/index.html. (Do the same as with admin/base_site.html in the
previous section – copy it from the default directory to your custom template directory.) Edit the file, and you’ll see it
uses a template variable called app_list. That variable contains every installed Django app. Instead of using that,
you can hard-code links to object-specific admin pages in whatever way you think is best. Again, don’t worry if you
can’t understand the template language – we’ll cover that in more detail in Tutorial 3.
When you’re comfortable with the admin site, read part 3 of this tutorial to start working on public poll views.
34 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-38-2048.jpg)
![Django Documentation, Release 1.5.1
Note that the regular expressions for the include() functions don’t have a $ (end-of-string match character) but
rather a trailing slash. Whenever Django encounters include(), it chops off whatever part of the URL matched up
to that point and sends the remaining string to the included URLconf for further processing.
The idea behind include() is to make it easy to plug-and-play URLs. Since polls are in their own URLconf
(polls/urls.py), they can be placed under “/polls/”, or under “/fun_polls/”, or under “/content/polls/”, or any
other path root, and the app will still work.
Here’s what happens if a user goes to “/polls/34/” in this system:
• Django will find the match at ’^polls/’
• Then, Django will strip off the matching text ("polls/") and send the remaining text – "34/" – to the
‘polls.urls’ URLconf for further processing which matches r’^(?P<poll_id>d+)/$’ resulting in a call
to the detail() view like so:
detail(request=<HttpRequest object>, poll_id=’34’)
The poll_id=’34’ part comes from (?P<poll_id>d+). Using parentheses around a pattern “captures” the
text matched by that pattern and sends it as an argument to the view function; ?P<poll_id> defines the name that
will be used to identify the matched pattern; and d+ is a regular expression to match a sequence of digits (i.e., a
number).
Because the URL patterns are regular expressions, there really is no limit on what you can do with them. And there’s
no need to add URL cruft such as .html – unless you want to, in which case you can do something like this:
(r’^polls/latest.html$’, ’polls.views.index’),
But, don’t do that. It’s silly.
2.5.4 Write views that actually do something
Each view is responsible for doing one of two things: returning an HttpResponse object containing the content for
the requested page, or raising an exception such as Http404. The rest is up to you.
Your view can read records from a database, or not. It can use a template system such as Django’s – or a third-party
Python template system – or not. It can generate a PDF file, output XML, create a ZIP file on the fly, anything you
want, using whatever Python libraries you want.
All Django wants is that HttpResponse. Or an exception.
Because it’s convenient, let’s use Django’s own database API, which we covered in Tutorial 1. Here’s one stab at
the index() view, which displays the latest 5 poll questions in the system, separated by commas, according to
publication date:
from django.http import HttpResponse
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.order_by(’-pub_date’)[:5]
output = ’, ’.join([p.question for p in latest_poll_list])
return HttpResponse(output)
There’s a problem here, though: the page’s design is hard-coded in the view. If you want to change the way the page
looks, you’ll have to edit this Python code. So let’s use Django’s template system to separate the design from Python
by creating a template that the view can use.
First, create a directory called templates in your polls directory. Django will look for templates in there.
38 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-42-2048.jpg)
![Django Documentation, Release 1.5.1
Django’s TEMPLATE_LOADERS setting contains a list of callables that know how to import templates from various
sources. One of the defaults is django.template.loaders.app_directories.Loader which looks for
a “templates” subdirectory in each of the INSTALLED_APPS - this is how Django knows to find the polls templates
even though we didn’t modify TEMPLATE_DIRS, as we did in Tutorial 2.
Organizing templates
We could have all our templates together, in one big templates directory, and it would work perfectly well. However,
this template belongs to the polls application, so unlike the admin template we created in the previous tutorial, we’ll
put this one in the application’s template directory (polls/templates) rather than the project’s (templates).
We’ll discuss in more detail in the reusable apps tutorial why we do this.
Within the templates directory you have just created, create another directory called polls,
and within that create a file called index.html. In other words, your template should be at
polls/templates/polls/index.html. Because of how the app_directories template loader
works as described above, you can refer to this template within Django simply as polls/index.html.
Template namespacing
Now we might be able to get away with putting our templates directly in polls/templates (rather than creating
another polls subdirectory), but it would actually be a bad idea. Django will choose the first template it finds whose
name matches, and if you had a template with the same name in a different application, Django would be unable to
distinguish between them. We need to be able to point Django at the right one, and the easiest way to ensure this is by
namespacing them. That is, by putting those templates inside another directory named for the application itself.
Put the following code in that template:
{% if latest_poll_list %}
<ul>
{% for poll in latest_poll_list %}
<li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li>
{% endfor %}
</ul>
{% else %}
<p>No polls are available.</p>
{% endif %}
Now let’s use that html template in our index view:
from django.http import HttpResponse
from django.template import Context, loader
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.order_by(’-pub_date’)[:5]
template = loader.get_template(’polls/index.html’)
context = Context({
’latest_poll_list’: latest_poll_list,
})
return HttpResponse(template.render(context))
That code loads the template called polls/index.html and passes it a context. The context is a dictionary
mapping template variable names to Python objects.
Load the page by pointing your browser at “/polls/”, and you should see a bulleted-list containing the “What’s up” poll
from Tutorial 1. The link points to the poll’s detail page.
2.5. Writing your first Django app, part 3 39](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-43-2048.jpg)
![Django Documentation, Release 1.5.1
A shortcut: render()
It’s a very common idiom to load a template, fill a context and return an HttpResponse object with the result of the
rendered template. Django provides a shortcut. Here’s the full index() view, rewritten:
from django.shortcuts import render
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.all().order_by(’-pub_date’)[:5]
context = {’latest_poll_list’: latest_poll_list}
return render(request, ’polls/index.html’, context)
Note that once we’ve done this in all these views, we no longer need to import loader, Context and
HttpResponse (you’ll want to keep HttpResponse if you still have the stub methods for detail, results,
and vote).
The render() function takes the request object as its first argument, a template name as its second argument and a
dictionary as its optional third argument. It returns an HttpResponse object of the given template rendered with
the given context.
2.5.5 Raising a 404 error
Now, let’s tackle the poll detail view – the page that displays the question for a given poll. Here’s the view:
from django.http import Http404
# ...
def detail(request, poll_id):
try:
poll = Poll.objects.get(pk=poll_id)
except Poll.DoesNotExist:
raise Http404
return render(request, ’polls/detail.html’, {’poll’: poll})
The new concept here: The view raises the Http404 exception if a poll with the requested ID doesn’t exist.
We’ll discuss what you could put in that polls/detail.html template a bit later, but if you’d like to quickly get
the above example working, a file containing just:
{{ poll }}
will get you started for now.
A shortcut: get_object_or_404()
It’s a very common idiom to use get() and raise Http404 if the object doesn’t exist. Django provides a shortcut.
Here’s the detail() view, rewritten:
from django.shortcuts import render, get_object_or_404
# ...
def detail(request, poll_id):
poll = get_object_or_404(Poll, pk=poll_id)
return render(request, ’polls/detail.html’, {’poll’: poll})
40 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-44-2048.jpg)
![Django Documentation, Release 1.5.1
form will alter data server-side. Whenever you create a form that alters data server-side, use method="post".
This tip isn’t specific to Django; it’s just good Web development practice.
• forloop.counter indicates how many times the for tag has gone through its loop
• Since we’re creating a POST form (which can have the effect of modifying data), we need to worry about Cross
Site Request Forgeries. Thankfully, you don’t have to worry too hard, because Django comes with a very easy-to-
use system for protecting against it. In short, all POST forms that are targeted at internal URLs should use
the {% csrf_token %} template tag.
Now, let’s create a Django view that handles the submitted data and does something with it. Remember, in Tutorial 3,
we created a URLconf for the polls application that includes this line:
url(r’^(?P<poll_id>d+)/vote/$’, views.vote, name=’vote’),
We also created a dummy implementation of the vote() function. Let’s create a real version. Add the following to
polls/views.py:
from django.shortcuts import get_object_or_404, render
from django.http import HttpResponseRedirect, HttpResponse
from django.core.urlresolvers import reverse
from polls.models import Choice, Poll
# ...
def vote(request, poll_id):
p = get_object_or_404(Poll, pk=poll_id)
try:
selected_choice = p.choice_set.get(pk=request.POST[’choice’])
except (KeyError, Choice.DoesNotExist):
# Redisplay the poll voting form.
return render(request, ’polls/detail.html’, {
’poll’: p,
’error_message’: "You didn’t select a choice.",
})
else:
selected_choice.votes += 1
selected_choice.save()
# Always return an HttpResponseRedirect after successfully dealing
# with POST data. This prevents data from being posted twice if a
# user hits the Back button.
return HttpResponseRedirect(reverse(’polls:results’, args=(p.id,)))
This code includes a few things we haven’t covered yet in this tutorial:
• request.POST is a dictionary-like object that lets you access submitted data by key name. In this case,
request.POST[’choice’] returns the ID of the selected choice, as a string. request.POST values are
always strings.
Note that Django also provides request.GET for accessing GET data in the same way – but we’re explicitly
using request.POST in our code, to ensure that data is only altered via a POST call.
• request.POST[’choice’] will raise KeyError if choice wasn’t provided in POST data. The above
code checks for KeyError and redisplays the poll form with an error message if choice isn’t given.
• After incrementing the choice count, the code returns an HttpResponseRedirect rather than a normal
HttpResponse. HttpResponseRedirect takes a single argument: the URL to which the user will be
redirected (see the following point for how we construct the URL in this case).
As the Python comment above points out, you should always return an HttpResponseRedirect after
successfully dealing with POST data. This tip isn’t specific to Django; it’s just goodWeb development practice.
44 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-48-2048.jpg)
![Django Documentation, Release 1.5.1
Generally, when writing a Django app, you’ll evaluate whether generic views are a good fit for your problem, and
you’ll use them from the beginning, rather than refactoring your code halfway through. But this tutorial intentionally
has focused on writing the views “the hard way” until now, to focus on core concepts.
You should know basic math before you start using a calculator.
Amend URLconf
First, open the polls/urls.py URLconf and change it like so:
from django.conf.urls import patterns, url
from django.views.generic import DetailView, ListView
from polls.models import Poll
urlpatterns = patterns(’’,
url(r’^$’,
ListView.as_view(
queryset=Poll.objects.order_by(’-pub_date’)[:5],
context_object_name=’latest_poll_list’,
template_name=’polls/index.html’),
name=’index’),
url(r’^(?P<pk>d+)/$’,
DetailView.as_view(
model=Poll,
template_name=’polls/detail.html’),
name=’detail’),
url(r’^(?P<pk>d+)/results/$’,
DetailView.as_view(
model=Poll,
template_name=’polls/results.html’),
name=’results’),
url(r’^(?P<poll_id>d+)/vote/$’, ’polls.views.vote’, name=’vote’),
)
Amend views
We’re using two generic views here: ListView and DetailView. Respectively, those two views abstract the
concepts of “display a list of objects” and “display a detail page for a particular type of object.”
• Each generic view needs to know what model it will be acting upon. This is provided using the model param-eter.
• The DetailView generic view expects the primary key value captured from the URL to be called "pk", so
we’ve changed poll_id to pk for the generic views.
By default, the DetailView generic view uses a template called <app name>/<model
name>_detail.html. In our case, it’ll use the template "polls/poll_detail.html". The
template_name argument is used to tell Django to use a specific template name instead of the autogener-ated
default template name. We also specify the template_name for the results list view – this ensures that the
results view and the detail view have a different appearance when rendered, even though they’re both a DetailView
behind the scenes.
Similarly, the ListView generic view uses a default template called <app name>/<model
name>_list.html; we use template_name to tell ListView to use our existing "polls/index.html"
template.
46 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-50-2048.jpg)
![Django Documentation, Release 1.5.1
We will start again with the shell, where we need to do a couple of things that won’t be necessary in tests.py. The
first is to set up the test environment in the shell:
>>> from django.test.utils import setup_test_environment
>>> setup_test_environment()
Next we need to import the test client class (later in tests.py we will use the django.test.TestCase class,
which comes with its own client, so this won’t be required):
>>> from django.test.client import Client
>>> # create an instance of the client for our use
>>> client = Client()
With that ready, we can ask the client to do some work for us:
>>> # get a response from ’/’
>>> response = client.get(’/’)
>>> # we should expect a 404 from that address
>>> response.status_code
404
>>> # on the other hand we should expect to find something at ’/polls/’
>>> # we’ll use ’reverse()’ rather than a harcoded URL
>>> from django.core.urlresolvers import reverse
>>> response = client.get(reverse(’polls:index’))
>>> response.status_code
200
>>> response.content
’nnn <p>No polls are available.</p>nn’
>>> # note - you might get unexpected results if your ‘‘TIME_ZONE‘‘
>>> # in ‘‘settings.py‘‘ is not correct. If you need to change it,
>>> # you will also need to restart your shell session
>>> from polls.models import Poll
>>> from django.utils import timezone
>>> # create a Poll and save it
>>> p = Poll(question="Who is your favorite Beatle?", pub_date=timezone.now())
>>> p.save()
>>> # check the response once again
>>> response = client.get(’/polls/’)
>>> response.content
’nnn <ul>n n <li><a href="/polls/1/">Who is your favorite Beatle?</a></li>n n >>> response.context[’latest_poll_list’]
[<Poll: Who is your favorite Beatle?>]
Improving our view
The list of polls shows polls that aren’t published yet (i.e. those that have a pub_date in the future). Let’s fix that.
In Tutorial 4 we deleted the view functions from views.py in favor of a ListView in urls.py:
url(r’^$’,
ListView.as_view(
queryset=Poll.objects.order_by(’-pub_date’)[:5],
context_object_name=’latest_poll_list’,
template_name=’polls/index.html’),
name=’index’),
response.context_data[’latest_poll_list’] extracts the data this view places into the context.
We need to amend the line that gives us the queryset:
52 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-56-2048.jpg)
![Django Documentation, Release 1.5.1
queryset=Poll.objects.order_by(’-pub_date’)[:5],
Let’s change the queryset so that it also checks the date by comparing it with timezone.now(). First we need to
add an import:
from django.utils import timezone
and then we must amend the existing url function to:
url(r’^$’,
ListView.as_view(
queryset=Poll.objects.filter(pub_date__lte=timezone.now)
.order_by(’-pub_date’)[:5],
context_object_name=’latest_poll_list’,
template_name=’polls/index.html’),
name=’index’),
Poll.objects.filter(pub_date__lte=timezone.now) returns a queryset containing Polls whose
pub_date is less than or equal to - that is, earlier than or equal to - timezone.now. Notice that we use a callable
queryset argument, timezone.now, which will be evaluated at request time. If we had included the parentheses,
timezone.now() would be evaluated just once when the web server is started.
Testing our new view
Now you can satisfy yourself that this behaves as expected by firing up the runserver, loading the site in your browser,
creating Polls with dates in the past and future, and checking that only those that have been published are listed.
You don’t want to have to do that every single time you make any change that might affect this - so let’s also create a
test, based on our shell session above.
Add the following to polls/tests.py:
from django.core.urlresolvers import reverse
and we’ll create a factory method to create polls as well as a new test class:
def create_poll(question, days):
"""
Creates a poll with the given ‘question‘ published the given number of
‘days‘ offset to now (negative for polls published in the past,
positive for polls that have yet to be published).
"""
return Poll.objects.create(question=question,
pub_date=timezone.now() + datetime.timedelta(days=days))
class PollViewTests(TestCase):
def test_index_view_with_no_polls(self):
"""
If no polls exist, an appropriate message should be displayed.
"""
response = self.client.get(reverse(’polls:index’))
self.assertEqual(response.status_code, 200)
self.assertContains(response, "No polls are available.")
self.assertQuerysetEqual(response.context[’latest_poll_list’], [])
def test_index_view_with_a_past_poll(self):
"""
Polls with a pub_date in the past should be displayed on the index page.
"""
2.7. Writing your first Django app, part 5 53](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-57-2048.jpg)
![Django Documentation, Release 1.5.1
create_poll(question="Past poll.", days=-30)
response = self.client.get(reverse(’polls:index’))
self.assertQuerysetEqual(
response.context[’latest_poll_list’],
[’<Poll: Past poll.>’]
)
def test_index_view_with_a_future_poll(self):
"""
Polls with a pub_date in the future should not be displayed on the
index page.
"""
create_poll(question="Future poll.", days=30)
response = self.client.get(reverse(’polls:index’))
self.assertContains(response, "No polls are available.", status_code=200)
self.assertQuerysetEqual(response.context[’latest_poll_list’], [])
def test_index_view_with_future_poll_and_past_poll(self):
"""
Even if both past and future polls exist, only past polls should be
displayed.
"""
create_poll(question="Past poll.", days=-30)
create_poll(question="Future poll.", days=30)
response = self.client.get(reverse(’polls:index’))
self.assertQuerysetEqual(
response.context[’latest_poll_list’],
[’<Poll: Past poll.>’]
)
def test_index_view_with_two_past_polls(self):
"""
The polls index page may display multiple polls.
"""
create_poll(question="Past poll 1.", days=-30)
create_poll(question="Past poll 2.", days=-5)
response = self.client.get(reverse(’polls:index’))
self.assertQuerysetEqual(
response.context[’latest_poll_list’],
[’<Poll: Past poll 2.>’, ’<Poll: Past poll 1.>’]
)
Let’s look at some of these more closely.
First is a poll factory method, create_poll, to take some repetition out of the process of creating polls.
test_index_view_with_no_polls doesn’t create any polls, but checks the message: “No polls are available.”
and verifies the latest_poll_list is empty. Note that the django.test.TestCase class provides some
additional assertion methods. In these examples, we use assertContains() and assertQuerysetEqual().
In test_index_view_with_a_past_poll, we create a poll and verify that it appears in the list.
In test_index_view_with_a_future_poll, we create a poll with a pub_date in the future. The database
is reset for each test method, so the first poll is no longer there, and so again the index shouldn’t have any polls in it.
And so on. In effect, we are using the tests to tell a story of admin input and user experience on the site, and checking
that at every state and for every new change in the state of the system, the expected results are published.
54 Chapter 2. Getting started](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-58-2048.jpg)
![Django Documentation, Release 1.5.1
README = open(os.path.join(os.path.dirname(__file__), ’README.rst’)).read()
# allow setup.py to be run from any path
os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir)))
setup(
name=’django-polls’,
version=’0.1’,
packages=[’polls’],
include_package_data=True,
license=’BSD License’, # example license
description=’A simple Django app to conduct Web-based polls.’,
long_description=README,
url=’http://www.example.com/’,
author=’Your Name’,
author_email=’yourname@example.com’,
classifiers=[
’Environment :: Web Environment’,
’Framework :: Django’,
’Intended Audience :: Developers’,
’License :: OSI Approved :: BSD License’, # example license
’Operating System :: OS Independent’,
’Programming Language :: Python’,
’Programming Language :: Python :: 2.6’,
’Programming Language :: Python :: 2.7’,
’Topic :: Internet :: WWW/HTTP’,
’Topic :: Internet :: WWW/HTTP :: Dynamic Content’,
],
)
I thought you said we were going to use distribute?
Distribute is a drop-in replacement for setuptools. Even though we appear to import from setuptools, since
we have distribute installed, it will override the import.
6. Only Python modules and packages are included in the package by default. To include additional files,
we’ll need to create a MANIFEST.in file. The distribute docs referred to in the previous step discuss
this file in more details. To include the templates, the README.rst and our LICENSE file, create a file
django-polls/MANIFEST.in with the following contents:
include LICENSE
include README.rst
recursive-include polls/templates *
7. It’s optional, but recommended, to include detailed documentation with your app. Create an
empty directory django-polls/docs for future documentation. Add an additional line to
django-polls/MANIFEST.in:
recursive-include docs *
Note that the docs directory won’t be included in your package unless you add some files to it. Many Django
apps also provide their documentation online through sites like readthedocs.org.
8. Try building your package with python setup.py sdist (run from inside django-polls). This cre-ates
a directory called dist and builds your new package, django-polls-0.1.tar.gz.
For more information on packaging, see The Hitchhiker’s Guide to Packaging.
2.9. Advanced tutorial: How to write reusable apps 61](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-65-2048.jpg)
![Django Documentation, Release 1.5.1
a database user with these permissions, you’ll specify the details in your project’s settings file, see DATABASES for
details.
If you’re using Django’s testing framework to test database queries, Django will need permission to create a test
database.
3.1.4 Remove any old versions of Django
If you are upgrading your installation of Django from a previous version, you will need to uninstall the old Django
version before installing the new version.
If you installed Django using pip or easy_install previously, installing with pip or easy_install again will
automatically take care of the old version, so you don’t need to do it yourself.
If you previously installed Django using python setup.py install, uninstalling is as simple as deleting the
django directory from your Python site-packages. To find the directory you need to remove, you can run the
following at your shell prompt (not the interactive Python prompt):
python -c "import sys; sys.path = sys.path[1:]; import django; print(django.__path__)"
3.1.5 Install the Django code
Installation instructions are slightly different depending on whether you’re installing a distribution-specific package,
downloading the latest official release, or fetching the latest development version.
It’s easy, no matter which way you choose.
Installing a distribution-specific package
Check the distribution specific notes to see if your platform/distribution provides official Django packages/installers.
Distribution-provided packages will typically allow for automatic installation of dependencies and easy upgrade paths.
Installing an official release with pip
This is the recommended way to install Django.
1. Install pip. The easiest is to use the standalone pip installer. If your distribution already has pip installed, you
might need to update it if it’s outdated. (If it’s outdated, you’ll know because installation won’t work.)
2. (optional) Take a look at virtualenv and virtualenvwrapper. These tools provide isolated Python environments,
which are more practical than installing packages systemwide. They also allow installing packages without
administrator privileges. It’s up to you to decide if you want to learn and use them.
3. If you’re using Linux, Mac OS X or some other flavor of Unix, enter the command sudo pip install
Django at the shell prompt. If you’re using Windows, start a command shell with administrator privi-leges
and run the command pip install Django. This will install Django in your Python installation’s
site-packages directory.
If you’re using a virtualenv, you don’t need sudo or administrator privileges, and this will install Django in the
virtualenv’s site-packages directory.
3.1. How to install Django 77](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-81-2048.jpg)
![Django Documentation, Release 1.5.1
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through=’Membership’)
def __unicode__(self):
return self.name
class Membership(models.Model):
person = models.ForeignKey(Person)
group = models.ForeignKey(Group)
date_joined = models.DateField()
invite_reason = models.CharField(max_length=64)
When you set up the intermediary model, you explicitly specify foreign keys to the models that are involved in the
ManyToMany relation. This explicit declaration defines how the two models are related.
There are a few restrictions on the intermediate model:
• Your intermediate model must contain one - and only one - foreign key to the target model (this would be
Person in our example). If you have more than one foreign key, a validation error will be raised.
• Your intermediate model must contain one - and only one - foreign key to the source model (this would be
Group in our example). If you have more than one foreign key, a validation error will be raised.
• The only exception to this is a model which has a many-to-many relationship to itself, through an intermediary
model. In this case, two foreign keys to the same model are permitted, but they will be treated as the two
(different) sides of the many-to-many relation.
• When defining a many-to-many relationship from a model to itself, using an intermediary model, you must use
symmetrical=False (see the model field reference).
Now that you have set up your ManyToManyField to use your intermediary model (Membership, in this case),
you’re ready to start creating some many-to-many relationships. You do this by creating instances of the intermediate
model:
>>> ringo = Person.objects.create(name="Ringo Starr")
>>> paul = Person.objects.create(name="Paul McCartney")
>>> beatles = Group.objects.create(name="The Beatles")
>>> m1 = Membership(person=ringo, group=beatles,
... date_joined=date(1962, 8, 16),
... invite_reason= "Needed a new drummer.")
>>> m1.save()
>>> beatles.members.all()
[<Person: Ringo Starr>]
>>> ringo.group_set.all()
[<Group: The Beatles>]
>>> m2 = Membership.objects.create(person=paul, group=beatles,
... date_joined=date(1960, 8, 1),
... invite_reason= "Wanted to form a band.")
>>> beatles.members.all()
[<Person: Ringo Starr>, <Person: Paul McCartney>]
Unlike normal many-to-many fields, you can’t use add, create, or assignment (i.e., beatles.members =
[...]) to create relationships:
# THIS WILL NOT WORK
>>> beatles.members.add(john)
# NEITHER WILL THIS
>>> beatles.members.create(name="George Harrison")
3.2. Models and databases 85](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-89-2048.jpg)
![Django Documentation, Release 1.5.1
# AND NEITHER WILL THIS
>>> beatles.members = [john, paul, ringo, george]
Why? You can’t just create a relationship between a Person and a Group - you need to specify all the detail for the
relationship required by the Membership model. The simple add, create and assignment calls don’t provide a
way to specify this extra detail. As a result, they are disabled for many-to-many relationships that use an intermediate
model. The only way to create this type of relationship is to create instances of the intermediate model.
The remove() method is disabled for similar reasons. However, the clear() method can be used to remove all
many-to-many relationships for an instance:
# Beatles have broken up
>>> beatles.members.clear()
Once you have established the many-to-many relationships by creating instances of your intermediate model, you can
issue queries. Just as with normal many-to-many relationships, you can query using the attributes of the many-to-many-
related model:
# Find all the groups with a member whose name starts with ’Paul’
>>> Group.objects.filter(members__name__startswith=’Paul’)
[<Group: The Beatles>]
As you are using an intermediate model, you can also query on its attributes:
# Find all the members of the Beatles that joined after 1 Jan 1961
>>> Person.objects.filter(
... group__name=’The Beatles’,
... membership__date_joined__gt=date(1961,1,1))
[<Person: Ringo Starr]
If you need to access a membership’s information you may do so by directly querying the Membership model:
>>> ringos_membership = Membership.objects.get(group=beatles, person=ringo)
>>> ringos_membership.date_joined
datetime.date(1962, 8, 16)
>>> ringos_membership.invite_reason
u’Needed a new drummer.’
Another way to access the same information is by querying the many-to-many reverse relationship from a Person
object:
>>> ringos_membership = ringo.membership_set.get(group=beatles)
>>> ringos_membership.date_joined
datetime.date(1962, 8, 16)
>>> ringos_membership.invite_reason
u’Needed a new drummer.’
One-to-one relationships To define a one-to-one relationship, use OneToOneField. You use it just like any other
Field type: by including it as a class attribute of your model.
This is most useful on the primary key of an object when that object “extends” another object in some way.
OneToOneField requires a positional argument: the class to which the model is related.
For example, if you were building a database of “places”, you would build pretty standard stuff such as address, phone
number, etc. in the database. Then, if you wanted to build a database of restaurants on top of the places, instead of
repeating yourself and replicating those fields in the Restaurant model, you could make Restaurant have a
OneToOneField to Place (because a restaurant “is a” place; in fact, to handle this you’d typically use inheritance,
which involves an implicit one-to-one relation).
86 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-90-2048.jpg)
![Django Documentation, Release 1.5.1
Meta options
Give your model metadata by using an inner class Meta, like so:
class Ox(models.Model):
horn_length = models.IntegerField()
class Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
Model metadata is “anything that’s not a field”, such as ordering options (ordering), database table name
(db_table), or human-readable singular and plural names (verbose_name and verbose_name_plural).
None are required, and adding class Meta to a model is completely optional.
A complete list of all possible Meta options can be found in the model option reference.
Model methods
Define custom methods on a model to add custom “row-level” functionality to your objects. Whereas Manager
methods are intended to do “table-wide” things, model methods should act on a particular model instance.
This is a valuable technique for keeping business logic in one place – the model.
For example, this model has a few custom methods:
from django.contrib.localflavor.us.models import USStateField
class Person(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
birth_date = models.DateField()
address = models.CharField(max_length=100)
city = models.CharField(max_length=50)
state = USStateField() # Yes, this is America-centric...
def baby_boomer_status(self):
"Returns the person’s baby-boomer status."
import datetime
if self.birth_date < datetime.date(1945, 8, 1):
return "Pre-boomer"
elif self.birth_date < datetime.date(1965, 1, 1):
return "Baby boomer"
else:
return "Post-boomer"
def is_midwestern(self):
"Returns True if this person is from the Midwest."
return self.state in (’IL’, ’WI’, ’MI’, ’IN’, ’OH’, ’IA’, ’MO’)
def _get_full_name(self):
"Returns the person’s full name."
return ’%s %s’ % (self.first_name, self.last_name)
full_name = property(_get_full_name)
The last method in this example is a property.
The model instance reference has a complete list of methods automatically given to each model. You can override
most of these – see overriding predefined model methods, below – but there are a couple that you’ll almost always
88 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-92-2048.jpg)
![Django Documentation, Release 1.5.1
Meta inheritance When an abstract base class is created, Django makes any Meta inner class you declared in the
base class available as an attribute. If a child class does not declare its own Meta class, it will inherit the parent’s Meta.
If the child wants to extend the parent’s Meta class, it can subclass it. For example:
class CommonInfo(models.Model):
...
class Meta:
abstract = True
ordering = [’name’]
class Student(CommonInfo):
...
class Meta(CommonInfo.Meta):
db_table = ’student_info’
Django does make one adjustment to the Meta class of an abstract base class: before installing the Meta attribute,
it sets abstract=False. This means that children of abstract base classes don’t automatically become abstract
classes themselves. Of course, you can make an abstract base class that inherits from another abstract base class. You
just need to remember to explicitly set abstract=True each time.
Some attributes won’t make sense to include in the Meta class of an abstract base class. For example, including
db_table would mean that all the child classes (the ones that don’t specify their own Meta) would use the same
database table, which is almost certainly not what you want.
Be careful with related_name If you are using the related_name attribute on a ForeignKey or
ManyToManyField, you must always specify a unique reverse name for the field. This would normally cause a
problem in abstract base classes, since the fields on this class are included into each of the child classes, with exactly
the same values for the attributes (including related_name) each time.
To work around this problem, when you are using related_name in an abstract base class (only), part of the name
should contain ’%(app_label)s’ and ’%(class)s’.
• ’%(class)s’ is replaced by the lower-cased name of the child class that the field is used in.
• ’%(app_label)s’ is replaced by the lower-cased name of the app the child class is contained within. Each
installed application name must be unique and the model class names within each app must also be unique,
therefore the resulting name will end up being different.
For example, given an app common/models.py:
class Base(models.Model):
m2m = models.ManyToManyField(OtherModel, related_name="%(app_label)s_%(class)s_related")
class Meta:
abstract = True
class ChildA(Base):
pass
class ChildB(Base):
pass
Along with another app rare/models.py:
from common.models import Base
class ChildB(Base):
pass
3.2. Models and databases 91](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-95-2048.jpg)
![Django Documentation, Release 1.5.1
# Remove parent’s ordering effect
ordering = []
Inheritance and reverse relations Because multi-table inheritance uses an implicit OneToOneField to link the
child and the parent, it’s possible to move from the parent down to the child, as in the above example. However, this
uses up the name that is the default related_name value for ForeignKey and ManyToManyField relations.
If you are putting those types of relations on a subclass of another model, you must specify the related_name
attribute on each such field. If you forget, Django will raise an error when you run validate or syncdb.
For example, using the above Place class again, let’s create another subclass with a ManyToManyField:
class Supplier(Place):
# Must specify related_name on all relations.
customers = models.ManyToManyField(Restaurant, related_name=’provider’)
Specifying the parent link field As mentioned, Django will automatically create a OneToOneField linking your
child class back any non-abstract parent models. If you want to control the name of the attribute linking back to the
parent, you can create your own OneToOneField and set parent_link=True to indicate that your field is the
link back to the parent class.
Proxy models
When using multi-table inheritance, a new database table is created for each subclass of a model. This is usually the
desired behavior, since the subclass needs a place to store any additional data fields that are not present on the base
class. Sometimes, however, you only want to change the Python behavior of a model – perhaps to change the default
manager, or add a new method.
This is what proxy model inheritance is for: creating a proxy for the original model. You can create, delete and update
instances of the proxy model and all the data will be saved as if you were using the original (non-proxied) model. The
difference is that you can change things like the default model ordering or the default manager in the proxy, without
having to alter the original.
Proxy models are declared like normal models. You tell Django that it’s a proxy model by setting the proxy attribute
of the Meta class to True.
For example, suppose you want to add a method to the Person model described above. You can do it like this:
class MyPerson(Person):
class Meta:
proxy = True
def do_something(self):
...
The MyPerson class operates on the same database table as its parent Person class. In particular, any new instances
of Person will also be accessible through MyPerson, and vice-versa:
>>> p = Person.objects.create(first_name="foobar")
>>> MyPerson.objects.get(first_name="foobar")
<MyPerson: foobar>
You could also use a proxy model to define a different default ordering on a model. You might not always want to
order the Person model, but regularly order by the last_name attribute when you use the proxy. This is easy:
3.2. Models and databases 93](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-97-2048.jpg)
![Django Documentation, Release 1.5.1
class OrderedPerson(Person):
class Meta:
ordering = ["last_name"]
proxy = True
Now normal Person queries will be unordered and OrderedPerson queries will be ordered by last_name.
QuerySets still return the model that was requested There is no way to have Django return, say, a MyPerson
object whenever you query for Person objects. A queryset for Person objects will return those types of objects.
The whole point of proxy objects is that code relying on the original Person will use those and your own code can
use the extensions you included (that no other code is relying on anyway). It is not a way to replace the Person (or
any other) model everywhere with something of your own creation.
Base class restrictions A proxy model must inherit from exactly one non-abstract model class. You can’t inherit
from multiple non-abstract models as the proxy model doesn’t provide any connection between the rows in the different
database tables. A proxy model can inherit from any number of abstract model classes, providing they do not define
any model fields.
Proxy models inherit any Meta options that they don’t define from their non-abstract model parent (the model they
are proxying for).
Proxy model managers If you don’t specify any model managers on a proxy model, it inherits the managers from
its model parents. If you define a manager on the proxy model, it will become the default, although any managers
defined on the parent classes will still be available.
Continuing our example from above, you could change the default manager used when you query the Person model
like this:
class NewManager(models.Manager):
...
class MyPerson(Person):
objects = NewManager()
class Meta:
proxy = True
If you wanted to add a new manager to the Proxy, without replacing the existing default, you can use the techniques
described in the custom manager documentation: create a base class containing the new managers and inherit that
after the primary base class:
# Create an abstract class for the new manager.
class ExtraManagers(models.Model):
secondary = NewManager()
class Meta:
abstract = True
class MyPerson(Person, ExtraManagers):
class Meta:
proxy = True
You probably won’t need to do this very often, but, when you do, it’s possible.
94 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-98-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Entry.objects.filter(
... headline__startswith=’What’
... ).exclude(
... pub_date__gte=datetime.date.today()
... ).filter(
... pub_date__gte=datetime(2005, 1, 30)
... )
This takes the initial QuerySet of all entries in the database, adds a filter, then an exclusion, then another filter. The
final result is a QuerySet containing all entries with a headline that starts with “What”, that were published between
January 30, 2005, and the current day.
Filtered QuerySets are unique Each time you refine a QuerySet, you get a brand-new QuerySet that is in no
way bound to the previous QuerySet. Each refinement creates a separate and distinct QuerySet that can be stored,
used and reused.
Example:
>> q1 = Entry.objects.filter(headline__startswith="What")
>> q2 = q1.exclude(pub_date__gte=datetime.date.today())
>> q3 = q1.filter(pub_date__gte=datetime.date.today())
These three QuerySets are separate. The first is a base QuerySet containing all entries that contain a headline
starting with “What”. The second is a subset of the first, with an additional criteria that excludes records whose
pub_date is greater than now. The third is a subset of the first, with an additional criteria that selects only the
records whose pub_date is greater than now. The initial QuerySet (q1) is unaffected by the refinement process.
QuerySets are lazy QuerySets are lazy – the act of creating a QuerySet doesn’t involve any database activity.
You can stack filters together all day long, and Django won’t actually run the query until the QuerySet is evaluated.
Take a look at this example:
>>> q = Entry.objects.filter(headline__startswith="What")
>>> q = q.filter(pub_date__lte=datetime.date.today())
>>> q = q.exclude(body_text__icontains="food")
>>> print(q)
Though this looks like three database hits, in fact it hits the database only once, at the last line (print(q)). In
general, the results of a QuerySet aren’t fetched from the database until you “ask” for them. When you do, the
QuerySet is evaluated by accessing the database. For more details on exactly when evaluation takes place, see
When QuerySets are evaluated.
Retrieving a single object with get
filter() will always give you a QuerySet, even if only a single object matches the query - in this case, it will be
a QuerySet containing a single element.
If you know there is only one object that matches your query, you can use the get() method on a Manager which
returns the object directly:
>>> one_entry = Entry.objects.get(pk=1)
You can use any query expression with get(), just like with filter() - again, see Field lookups below.
Note that there is a difference between using get(), and using filter() with a slice of [0]. If there are no results
that match the query, get() will raise a DoesNotExist exception. This exception is an attribute of the model
3.2. Models and databases 99](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-103-2048.jpg)
![Django Documentation, Release 1.5.1
class that the query is being performed on - so in the code above, if there is no Entry object with a primary key of 1,
Django will raise Entry.DoesNotExist.
Similarly, Django will complain if more than one item matches the get() query. In this case, it will raise
MultipleObjectsReturned, which again is an attribute of the model class itself.
Other QuerySet methods
Most of the time you’ll use all(), get(), filter() and exclude() when you need to look up objects from
the database. However, that’s far from all there is; see the QuerySet API Reference for a complete list of all the various
QuerySet methods.
Limiting QuerySets
Use a subset of Python’s array-slicing syntax to limit your QuerySet to a certain number of results. This is the
equivalent of SQL’s LIMIT and OFFSET clauses.
For example, this returns the first 5 objects (LIMIT 5):
>>> Entry.objects.all()[:5]
This returns the sixth through tenth objects (OFFSET 5 LIMIT 5):
>>> Entry.objects.all()[5:10]
Negative indexing (i.e. Entry.objects.all()[-1]) is not supported.
Generally, slicing a QuerySet returns a new QuerySet – it doesn’t evaluate the query. An exception is if you use
the “step” parameter of Python slice syntax. For example, this would actually execute the query in order to return a
list of every second object of the first 10:
>>> Entry.objects.all()[:10:2]
To retrieve a single object rather than a list (e.g. SELECT foo FROM bar LIMIT 1), use a simple index instead
of a slice. For example, this returns the first Entry in the database, after ordering entries alphabetically by headline:
>>> Entry.objects.order_by(’headline’)[0]
This is roughly equivalent to:
>>> Entry.objects.order_by(’headline’)[0:1].get()
Note, however, that the first of these will raise IndexError while the second will raise DoesNotExist if no
objects match the given criteria. See get() for more details.
Field lookups
Field lookups are how you specify the meat of an SQL WHERE clause. They’re specified as keyword arguments to the
QuerySet methods filter(), exclude() and get().
Basic lookups keyword arguments take the form field__lookuptype=value. (That’s a double-underscore).
For example:
>>> Entry.objects.filter(pub_date__lte=’2006-01-01’)
translates (roughly) into the following SQL:
100 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-104-2048.jpg)
![Django Documentation, Release 1.5.1
New in version 1.5: .bitand() and .bitor() The F() objects now support bitwise operations by .bitand()
and .bitor(), for example:
>>> F(’somefield’).bitand(16)
Changed in version 1.5: The previously undocumented operators & and | no longer produce bitwise operations, use
.bitand() and .bitor() instead.
The pk lookup shortcut
For convenience, Django provides a pk lookup shortcut, which stands for “primary key”.
In the example Blog model, the primary key is the id field, so these three statements are equivalent:
>>> Blog.objects.get(id__exact=14) # Explicit form
>>> Blog.objects.get(id=14) # __exact is implied
>>> Blog.objects.get(pk=14) # pk implies id__exact
The use of pk isn’t limited to __exact queries – any query term can be combined with pk to perform a query on the
primary key of a model:
# Get blogs entries with id 1, 4 and 7
>>> Blog.objects.filter(pk__in=[1,4,7])
# Get all blog entries with id > 14
>>> Blog.objects.filter(pk__gt=14)
pk lookups also work across joins. For example, these three statements are equivalent:
>>> Entry.objects.filter(blog__id__exact=3) # Explicit form
>>> Entry.objects.filter(blog__id=3) # __exact is implied
>>> Entry.objects.filter(blog__pk=3) # __pk implies __id__exact
Escaping percent signs and underscores in LIKE statements
The field lookups that equate to LIKE SQL statements (iexact, contains, icontains, startswith,
istartswith, endswith and iendswith) will automatically escape the two special characters used in LIKE
statements – the percent sign and the underscore. (In a LIKE statement, the percent sign signifies a multiple-character
wildcard and the underscore signifies a single-character wildcard.)
This means things should work intuitively, so the abstraction doesn’t leak. For example, to retrieve all the entries that
contain a percent sign, just use the percent sign as any other character:
>>> Entry.objects.filter(headline__contains=’%’)
Django takes care of the quoting for you; the resulting SQL will look something like this:
SELECT ... WHERE headline LIKE ’%%%’;
Same goes for underscores. Both percentage signs and underscores are handled for you transparently.
Caching and QuerySets
Each QuerySet contains a cache, to minimize database access. It’s important to understand how it works, in order
to write the most efficient code.
104 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-108-2048.jpg)
![Django Documentation, Release 1.5.1
In a newly created QuerySet, the cache is empty. The first time a QuerySet is evaluated – and, hence, a database
query happens – Django saves the query results in the QuerySet‘s cache and returns the results that have been
explicitly requested (e.g., the next element, if the QuerySet is being iterated over). Subsequent evaluations of the
QuerySet reuse the cached results.
Keep this caching behavior in mind, because it may bite you if you don’t use your QuerySets correctly. For example,
the following will create two QuerySets, evaluate them, and throw them away:
>>> print([e.headline for e in Entry.objects.all()])
>>> print([e.pub_date for e in Entry.objects.all()])
That means the same database query will be executed twice, effectively doubling your database load. Also, there’s
a possibility the two lists may not include the same database records, because an Entry may have been added or
deleted in the split second between the two requests.
To avoid this problem, simply save the QuerySet and reuse it:
>>> queryset = Entry.objects.all()
>>> print([p.headline for p in queryset]) # Evaluate the query set.
>>> print([p.pub_date for p in queryset]) # Re-use the cache from the evaluation.
Complex lookups with Q objects
class Q
Keyword argument queries – in filter(), etc. – are “AND”ed together. If you need to execute more complex
queries (for example, queries with OR statements), you can use Q objects.
A Q object (django.db.models.Q) is an object used to encapsulate a collection of keyword arguments. These
keyword arguments are specified as in “Field lookups” above.
For example, this Q object encapsulates a single LIKE query:
from django.db.models import Q
Q(question__startswith=’What’)
Q objects can be combined using the & and | operators. When an operator is used on two Q objects, it yields a new Q
object.
For example, this statement yields a single Q object that represents the “OR” of two "question__startswith"
queries:
Q(question__startswith=’Who’) | Q(question__startswith=’What’)
This is equivalent to the following SQL WHERE clause:
WHERE question LIKE ’Who%’ OR question LIKE ’What%’
You can compose statements of arbitrary complexity by combining Q objects with the & and | operators and use
parenthetical grouping. Also, Q objects can be negated using the ~ operator, allowing for combined lookups that
combine both a normal query and a negated (NOT) query:
Q(question__startswith=’Who’) | ~Q(pub_date__year=2005)
Each lookup function that takes keyword-arguments (e.g. filter(), exclude(), get()) can also be passed
one or more Q objects as positional (not-named) arguments. If you provide multiple Q object arguments to a lookup
function, the arguments will be “AND”ed together. For example:
3.2. Models and databases 105](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-109-2048.jpg)
![Django Documentation, Release 1.5.1
Entry.objects.filter(pub_date__year=2005).delete()
Keep in mind that this will, whenever possible, be executed purely in SQL, and so the delete() methods of in-dividual
object instances will not necessarily be called during the process. If you’ve provided a custom delete()
method on a model class and want to ensure that it is called, you will need to “manually” delete instances of that model
(e.g., by iterating over a QuerySet and calling delete() on each object individually) rather than using the bulk
delete() method of a QuerySet.
When Django deletes an object, by default it emulates the behavior of the SQL constraint ON DELETE CASCADE –
in other words, any objects which had foreign keys pointing at the object to be deleted will be deleted along with it.
For example:
b = Blog.objects.get(pk=1)
# This will delete the Blog and all of its Entry objects.
b.delete()
This cascade behavior is customizable via the on_delete argument to the ForeignKey.
Note that delete() is the only QuerySet method that is not exposed on a Manager itself. This is a safety
mechanism to prevent you from accidentally requesting Entry.objects.delete(), and deleting all the entries.
If you do want to delete all the objects, then you have to explicitly request a complete query set:
Entry.objects.all().delete()
Copying model instances
Although there is no built-in method for copying model instances, it is possible to easily create new instance with all
fields’ values copied. In the simplest case, you can just set pk to None. Using our blog example:
blog = Blog(name=’My blog’, tagline=’Blogging is easy’)
blog.save() # blog.pk == 1
blog.pk = None
blog.save() # blog.pk == 2
Things get more complicated if you use inheritance. Consider a subclass of Blog:
class ThemeBlog(Blog):
theme = models.CharField(max_length=200)
django_blog = ThemeBlog(name=’Django’, tagline=’Django is easy’, theme=’python’)
django_blog.save() # django_blog.pk == 3
Due to how inheritance works, you have to set both pk and id to None:
django_blog.pk = None
django_blog.id = None
django_blog.save() # django_blog.pk == 4
This process does not copy related objects. If you want to copy relations, you have to write a little bit more code. In
our example, Entry has a many to many field to Author:
entry = Entry.objects.all()[0] # some previous entry
old_authors = entry.authors.all()
entry.pk = None
entry.save()
entry.authors = old_authors # saves new many2many relations
3.2. Models and databases 107](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-111-2048.jpg)
![Django Documentation, Release 1.5.1
You can override the FOO_set name by setting the related_name parameter in the ForeignKey()
definition. For example, if the Entry model was altered to blog = ForeignKey(Blog,
related_name=’entries’), the above example code would look like this:
>>> b = Blog.objects.get(id=1)
>>> b.entries.all() # Returns all Entry objects related to Blog.
# b.entries is a Manager that returns QuerySets.
>>> b.entries.filter(headline__contains=’Lennon’)
>>> b.entries.count()
You cannot access a reverse ForeignKey Manager from the class; it must be accessed from an instance:
>>> Blog.entry_set
Traceback:
...
AttributeError: "Manager must be accessed via instance".
In addition to the QuerySet methods defined in “Retrieving objects” above, the ForeignKey Manager has ad-ditional
methods used to handle the set of related objects. A synopsis of each is below, and complete details can be
found in the related objects reference.
add(obj1, obj2, ...) Adds the specified model objects to the related object set.
create(**kwargs) Creates a new object, saves it and puts it in the related object set. Returns the newly created
object.
remove(obj1, obj2, ...) Removes the specified model objects from the related object set.
clear() Removes all objects from the related object set.
To assign the members of a related set in one fell swoop, just assign to it from any iterable object. The iterable can
contain object instances, or just a list of primary key values. For example:
b = Blog.objects.get(id=1)
b.entry_set = [e1, e2]
In this example, e1 and e2 can be full Entry instances, or integer primary key values.
If the clear() method is available, any pre-existing objects will be removed from the entry_set before all
objects in the iterable (in this case, a list) are added to the set. If the clear() method is not available, all objects in
the iterable will be added without removing any existing elements.
Each “reverse” operation described in this section has an immediate effect on the database. Every addition, creation
and deletion is immediately and automatically saved to the database.
Many-to-many relationships
Both ends of a many-to-many relationship get automatic API access to the other end. The API works just as a
“backward” one-to-many relationship, above.
The only difference is in the attribute naming: The model that defines the ManyToManyField uses the attribute
name of that field itself, whereas the “reverse” model uses the lowercased model name of the original model, plus
’_set’ (just like reverse one-to-many relationships).
An example makes this easier to understand:
e = Entry.objects.get(id=3)
e.authors.all() # Returns all Author objects for this Entry.
e.authors.count()
e.authors.filter(name__contains=’John’)
110 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-114-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Book.objects.all().aggregate(Avg(’price’))
{’price__avg’: 34.35}
# Max price across all books.
>>> from django.db.models import Max
>>> Book.objects.all().aggregate(Max(’price’))
{’price__max’: Decimal(’81.20’)}
# All the following queries involve traversing the Book<->Publisher
# many-to-many relationship backward
# Each publisher, each with a count of books as a "num_books" attribute.
>>> from django.db.models import Count
>>> pubs = Publisher.objects.annotate(num_books=Count(’book’))
>>> pubs
[<Publisher BaloneyPress>, <Publisher SalamiPress>, ...]
>>> pubs[0].num_books
73
# The top 5 publishers, in order by number of books.
>>> pubs = Publisher.objects.annotate(num_books=Count(’book’)).order_by(’-num_books’)[:5]
>>> pubs[0].num_books
1323
Generating aggregates over a QuerySet
Django provides two ways to generate aggregates. The first way is to generate summary values over an entire
QuerySet. For example, say you wanted to calculate the average price of all books available for sale. Django’s
query syntax provides a means for describing the set of all books:
>>> Book.objects.all()
What we need is a way to calculate summary values over the objects that belong to this QuerySet. This is done by
appending an aggregate() clause onto the QuerySet:
>>> from django.db.models import Avg
>>> Book.objects.all().aggregate(Avg(’price’))
{’price__avg’: 34.35}
The all() is redundant in this example, so this could be simplified to:
>>> Book.objects.aggregate(Avg(’price’))
{’price__avg’: 34.35}
The argument to the aggregate() clause describes the aggregate value that we want to compute - in this case, the
average of the price field on the Book model. A list of the aggregate functions that are available can be found in the
QuerySet reference.
aggregate() is a terminal clause for a QuerySet that, when invoked, returns a dictionary of name-value pairs.
The name is an identifier for the aggregate value; the value is the computed aggregate. The name is automatically
generated from the name of the field and the aggregate function. If you want to manually specify a name for the
aggregate value, you can do so by providing that name when you specify the aggregate clause:
>>> Book.objects.aggregate(average_price=Avg(’price’))
{’average_price’: 34.35}
If you want to generate more than one aggregate, you just add another argument to the aggregate() clause. So, if
we also wanted to know the maximum and minimum price of all books, we would issue the query:
3.2. Models and databases 113](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-117-2048.jpg)
![Django Documentation, Release 1.5.1
>>> from django.db.models import Avg, Max, Min, Count
>>> Book.objects.aggregate(Avg(’price’), Max(’price’), Min(’price’))
{’price__avg’: 34.35, ’price__max’: Decimal(’81.20’), ’price__min’: Decimal(’12.99’)}
Generating aggregates for each item in a QuerySet
The second way to generate summary values is to generate an independent summary for each object in a QuerySet.
For example, if you are retrieving a list of books, you may want to know how many authors contributed to each book.
Each Book has a many-to-many relationship with the Author; we want to summarize this relationship for each book
in the QuerySet.
Per-object summaries can be generated using the annotate() clause. When an annotate() clause is specified,
each object in the QuerySet will be annotated with the specified values.
The syntax for these annotations is identical to that used for the aggregate() clause. Each argument to
annotate() describes an aggregate that is to be calculated. For example, to annotate books with the number of
authors:
# Build an annotated queryset
>>> q = Book.objects.annotate(Count(’authors’))
# Interrogate the first object in the queryset
>>> q[0]
<Book: The Definitive Guide to Django>
>>> q[0].authors__count
2
# Interrogate the second object in the queryset
>>> q[1]
<Book: Practical Django Projects>
>>> q[1].authors__count
1
As with aggregate(), the name for the annotation is automatically derived from the name of the aggregate function
and the name of the field being aggregated. You can override this default name by providing an alias when you specify
the annotation:
>>> q = Book.objects.annotate(num_authors=Count(’authors’))
>>> q[0].num_authors
2
>>> q[1].num_authors
1
Unlike aggregate(), annotate() is not a terminal clause. The output of the annotate() clause is
a QuerySet; this QuerySet can be modified using any other QuerySet operation, including filter(),
order_by(), or even additional calls to annotate().
Joins and aggregates
So far, we have dealt with aggregates over fields that belong to the model being queried. However, sometimes the
value you want to aggregate will belong to a model that is related to the model you are querying.
When specifying the field to be aggregated in an aggregate function, Django will allow you to use the same double
underscore notation that is used when referring to related fields in filters. Django will then handle any table joins that
are required to retrieve and aggregate the related value.
For example, to find the price range of books offered in each store, you could use the annotation:
114 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-118-2048.jpg)
![Django Documentation, Release 1.5.1
returned in the result set, the method for evaluating annotations is slightly different. Instead of returning an annotated
result for each result in the original QuerySet, the original results are grouped according to the unique combinations
of the fields specified in the values() clause. An annotation is then provided for each unique group; the annotation
is computed over all members of the group.
For example, consider an author query that attempts to find out the average rating of books written by each author:
>>> Author.objects.annotate(average_rating=Avg(’book__rating’))
This will return one result for each author in the database, annotated with their average book rating.
However, the result will be slightly different if you use a values() clause:
>>> Author.objects.values(’name’).annotate(average_rating=Avg(’book__rating’))
In this example, the authors will be grouped by name, so you will only get an annotated result for each unique author
name. This means if you have two authors with the same name, their results will be merged into a single result in the
output of the query; the average will be computed as the average over the books written by both authors.
Order of annotate() and values() clauses As with the filter() clause, the order in which annotate()
and values() clauses are applied to a query is significant. If the values() clause precedes the annotate(),
the annotation will be computed using the grouping described by the values() clause.
However, if the annotate() clause precedes the values() clause, the annotations will be generated over the
entire query set. In this case, the values() clause only constrains the fields that are generated on output.
For example, if we reverse the order of the values() and annotate() clause from our previous example:
>>> Author.objects.annotate(average_rating=Avg(’book__rating’)).values(’name’, ’average_rating’)
This will now yield one unique result for each author; however, only the author’s name and the average_rating
annotation will be returned in the output data.
You should also note that average_rating has been explicitly included in the list of values to be returned. This is
required because of the ordering of the values() and annotate() clause.
If the values() clause precedes the annotate() clause, any annotations will be automatically added to the result
set. However, if the values() clause is applied after the annotate() clause, you need to explicitly include the
aggregate column.
Interaction with default ordering or order_by() Fields that are mentioned in the order_by() part of a
queryset (or which are used in the default ordering on a model) are used when selecting the output data, even if they
are not otherwise specified in the values() call. These extra fields are used to group “like” results together and they
can make otherwise identical result rows appear to be separate. This shows up, particularly, when counting things.
By way of example, suppose you have a model like this:
class Item(models.Model):
name = models.CharField(max_length=10)
data = models.IntegerField()
class Meta:
ordering = ["name"]
The important part here is the default ordering on the name field. If you want to count how many times each distinct
data value appears, you might try this:
# Warning: not quite correct!
Item.objects.values("data").annotate(Count("id"))
3.2. Models and databases 117](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-121-2048.jpg)
![Django Documentation, Release 1.5.1
Custom Managers
You can use a custom Manager in a particular model by extending the base Manager class and instantiating your
custom Manager in your model.
There are two reasons you might want to customize a Manager: to add extra Manager methods, and/or to modify
the initial QuerySet the Manager returns.
Adding extra Manager methods
Adding extra Manager methods is the preferred way to add “table-level” functionality to your models. (For “row-level”
functionality – i.e., functions that act on a single instance of a model object – use Model methods, not custom
Manager methods.)
A custom Manager method can return anything you want. It doesn’t have to return a QuerySet.
For example, this custom Manager offers a method with_counts(), which returns a list of all OpinionPoll
objects, each with an extra num_responses attribute that is the result of an aggregate query:
class PollManager(models.Manager):
def with_counts(self):
from django.db import connection
cursor = connection.cursor()
cursor.execute("""
SELECT p.id, p.question, p.poll_date, COUNT(*)
FROM polls_opinionpoll p, polls_response r
WHERE p.id = r.poll_id
GROUP BY p.id, p.question, p.poll_date
ORDER BY p.poll_date DESC""")
result_list = []
for row in cursor.fetchall():
p = self.model(id=row[0], question=row[1], poll_date=row[2])
p.num_responses = row[3]
result_list.append(p)
return result_list
class OpinionPoll(models.Model):
question = models.CharField(max_length=200)
poll_date = models.DateField()
objects = PollManager()
class Response(models.Model):
poll = models.ForeignKey(OpinionPoll)
person_name = models.CharField(max_length=50)
response = models.TextField()
With this example, you’d use OpinionPoll.objects.with_counts() to return that list of OpinionPoll
objects with num_responses attributes.
Another thing to note about this example is that Manager methods can access self.model to get the model class
to which they’re attached.
Modifying initial Manager QuerySets
A Manager‘s base QuerySet returns all objects in the system. For example, using this model:
3.2. Models and databases 119](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-123-2048.jpg)
![Django Documentation, Release 1.5.1
For more details check out the documentation for the db_table option, which also lets you manually set the database
table name.
Warning: No checking is done on the SQL statement that is passed in to .raw(). Django expects that the
statement will return a set of rows from the database, but does nothing to enforce that. If the query does not return
rows, a (possibly cryptic) error will result.
Mapping query fields to model fields
raw() automatically maps fields in the query to fields on the model.
The order of fields in your query doesn’t matter. In other words, both of the following queries work identically:
>>> Person.objects.raw(’SELECT id, first_name, last_name, birth_date FROM myapp_person’)
...
>>> Person.objects.raw(’SELECT last_name, birth_date, first_name, id FROM myapp_person’)
...
Matching is done by name. This means that you can use SQL’s AS clauses to map fields in the query to model fields.
So if you had some other table that had Person data in it, you could easily map it into Person instances:
>>> Person.objects.raw(’’’SELECT first AS first_name,
... last AS last_name,
... bd AS birth_date,
... pk as id,
... FROM some_other_table’’’)
As long as the names match, the model instances will be created correctly.
Alternatively, you can map fields in the query to model fields using the translations argument to raw(). This
is a dictionary mapping names of fields in the query to names of fields on the model. For example, the above query
could also be written:
>>> name_map = {’first’: ’first_name’, ’last’: ’last_name’, ’bd’: ’birth_date’, ’pk’: ’id’}
>>> Person.objects.raw(’SELECT * FROM some_other_table’, translations=name_map)
Index lookups
raw() supports indexing, so if you need only the first result you can write:
>>> first_person = Person.objects.raw(’SELECT * from myapp_person’)[0]
However, the indexing and slicing are not performed at the database level. If you have a big amount of Person
objects in your database, it is more efficient to limit the query at the SQL level:
>>> first_person = Person.objects.raw(’SELECT * from myapp_person LIMIT 1’)[0]
Deferring model fields
Fields may also be left out:
>>> people = Person.objects.raw(’SELECT id, first_name FROM myapp_person’)
3.2. Models and databases 125](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-129-2048.jpg)
![Django Documentation, Release 1.5.1
The Person objects returned by this query will be deferred model instances (see defer()). This means that the
fields that are omitted from the query will be loaded on demand. For example:
>>> for p in Person.objects.raw(’SELECT id, first_name FROM myapp_person’):
... print(p.first_name, # This will be retrieved by the original query
... p.last_name) # This will be retrieved on demand
...
John Smith
Jane Jones
From outward appearances, this looks like the query has retrieved both the first name and last name. However, this
example actually issued 3 queries. Only the first names were retrieved by the raw() query – the last names were both
retrieved on demand when they were printed.
There is only one field that you can’t leave out - the primary key field. Django uses the primary key to identify model
instances, so it must always be included in a raw query. An InvalidQuery exception will be raised if you forget to
include the primary key.
Adding annotations
You can also execute queries containing fields that aren’t defined on the model. For example, we could use Post-greSQL’s
age() function to get a list of people with their ages calculated by the database:
>>> people = Person.objects.raw(’SELECT *, age(birth_date) AS age FROM myapp_person’)
>>> for p in people:
... print("%s is %s." % (p.first_name, p.age))
John is 37.
Jane is 42.
...
Passing parameters into raw()
If you need to perform parameterized queries, you can use the params argument to raw():
>>> lname = ’Doe’
>>> Person.objects.raw(’SELECT * FROM myapp_person WHERE last_name = %s’, [lname])
params is a list of parameters. You’ll use %s placeholders in the query string (regardless of your database engine);
they’ll be replaced with parameters from the params list.
Warning: Do not use string formatting on raw queries!
It’s tempting to write the above query as:
>>> query = ’SELECT * FROM myapp_person WHERE last_name = %s’ % lname
>>> Person.objects.raw(query)
Don’t.
Using the params list completely protects you from SQL injection attacks, a common exploit where attackers
inject arbitrary SQL into your database. If you use string interpolation, sooner or later you’ll fall victim to SQL
injection. As long as you remember to always use the params list you’ll be protected.
126 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-130-2048.jpg)
![Django Documentation, Release 1.5.1
Executing custom SQL directly
Sometimes even Manager.raw() isn’t quite enough: you might need to perform queries that don’t map cleanly to
models, or directly execute UPDATE, INSERT, or DELETE queries.
In these cases, you can always access the database directly, routing around the model layer entirely.
The object django.db.connection represents the default database connection, and
django.db.transaction represents the default database transaction. To use the database connection,
call connection.cursor() to get a cursor object. Then, call cursor.execute(sql, [params]) to
execute the SQL and cursor.fetchone() or cursor.fetchall() to return the resulting rows. After
performing a data changing operation, you should then call transaction.commit_unless_managed() to
ensure your changes are committed to the database. If your query is purely a data retrieval operation, no commit is
required. For example:
def my_custom_sql():
from django.db import connection, transaction
cursor = connection.cursor()
# Data modifying operation - commit required
cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz])
transaction.commit_unless_managed()
# Data retrieval operation - no commit required
cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz])
row = cursor.fetchone()
return row
If you are using more than one database, you can use django.db.connections to obtain the connection (and
cursor) for a specific database. django.db.connections is a dictionary-like object that allows you to retrieve a
specific connection using its alias:
from django.db import connections
cursor = connections[’my_db_alias’].cursor()
# Your code here...
transaction.commit_unless_managed(using=’my_db_alias’)
By default, the Python DB API will return results without their field names, which means you end up with a list of
values, rather than a dict. At a small performance cost, you can return results as a dict by using something like
this:
def dictfetchall(cursor):
"Returns all rows from a cursor as a dict"
desc = cursor.description
return [
dict(zip([col[0] for col in desc], row))
for row in cursor.fetchall()
]
Here is an example of the difference between the two:
>>> cursor.execute("SELECT id, parent_id from test LIMIT 2");
>>> cursor.fetchall()
((54360982L, None), (54360880L, None))
>>> cursor.execute("SELECT id, parent_id from test LIMIT 2");
>>> dictfetchall(cursor)
[{’parent_id’: None, ’id’: 54360982L}, {’parent_id’: None, ’id’: 54360880L}]
3.2. Models and databases 127](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-131-2048.jpg)
![Django Documentation, Release 1.5.1
Now we’ll need to handle routing. First we want a router that knows to send queries for the auth app to auth_db:
class AuthRouter(object):
"""
A router to control all database operations on models in the
auth application.
"""
def db_for_read(self, model, **hints):
"""
Attempts to read auth models go to auth_db.
"""
if model._meta.app_label == ’auth’:
return ’auth_db’
return None
def db_for_write(self, model, **hints):
"""
Attempts to write auth models go to auth_db.
"""
if model._meta.app_label == ’auth’:
return ’auth_db’
return None
def allow_relation(self, obj1, obj2, **hints):
"""
Allow relations if a model in the auth app is involved.
"""
if obj1._meta.app_label == ’auth’ or
obj2._meta.app_label == ’auth’:
return True
return None
def allow_syncdb(self, db, model):
"""
Make sure the auth app only appears in the ’auth_db’
database.
"""
if db == ’auth_db’:
return model._meta.app_label == ’auth’
elif model._meta.app_label == ’auth’:
return False
return None
And we also want a router that sends all other apps to the master/slave configuration, and randomly chooses a slave to
read from:
import random
class MasterSlaveRouter(object):
def db_for_read(self, model, **hints):
"""
Reads go to a randomly-chosen slave.
"""
return random.choice([’slave1’, ’slave2’])
def db_for_write(self, model, **hints):
"""
Writes always go to master.
"""
3.2. Models and databases 137](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-141-2048.jpg)
![Django Documentation, Release 1.5.1
return ’master’
def allow_relation(self, obj1, obj2, **hints):
"""
Relations between objects are allowed if both objects are
in the master/slave pool.
"""
db_list = (’master’, ’slave1’, ’slave2’)
if obj1.state.db in db_list and obj2.state.db in db_list:
return True
return None
def allow_syncdb(self, db, model):
"""
All non-auth models end up in this pool.
"""
return True
Finally, in the settings file, we add the following (substituting path.to. with the actual python path to the module(s)
where the routers are defined):
DATABASE_ROUTERS = [’path.to.AuthRouter’, ’path.to.MasterSlaveRouter’]
The order in which routers are processed is significant. Routers will be queried in the order the are
listed in the DATABASE_ROUTERS setting . In this example, the AuthRouter is processed before the
MasterSlaveRouter, and as a result, decisions concerning the models in auth are processed before
any other decision is made. If the DATABASE_ROUTERS setting listed the two routers in the other order,
MasterSlaveRouter.allow_syncdb() would be processed first. The catch-all nature of the MasterSlaveR-outer
implementation would mean that all models would be available on all databases.
With this setup installed, lets run some Django code:
>>> # This retrieval will be performed on the ’auth_db’ database
>>> fred = User.objects.get(username=’fred’)
>>> fred.first_name = ’Frederick’
>>> # This save will also be directed to ’auth_db’
>>> fred.save()
>>> # These retrieval will be randomly allocated to a slave database
>>> dna = Person.objects.get(name=’Douglas Adams’)
>>> # A new object has no database allocation when created
>>> mh = Book(title=’Mostly Harmless’)
>>> # This assignment will consult the router, and set mh onto
>>> # the same database as the author object
>>> mh.author = dna
>>> # This save will force the ’mh’ instance onto the master database...
>>> mh.save()
>>> # ... but if we re-retrieve the object, it will come back on a slave
>>> mh = Book.objects.get(title=’Mostly Harmless’)
138 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-142-2048.jpg)
![Django Documentation, Release 1.5.1
# on the ’other’ database.
return super(MultiDBTabularInline, self).formfield_for_manytomany(db_field, request=request, Once you’ve written your model admin definitions, they can be registered with any Admin instance:
from django.contrib import admin
# Specialize the multi-db admin objects for use with specific models.
class BookInline(MultiDBTabularInline):
model = Book
class PublisherAdmin(MultiDBModelAdmin):
inlines = [BookInline]
admin.site.register(Author, MultiDBModelAdmin)
admin.site.register(Publisher, PublisherAdmin)
othersite = admin.AdminSite(’othersite’)
othersite.register(Publisher, MultiDBModelAdmin)
This example sets up two admin sites. On the first site, the Author and Publisher objects are exposed;
Publisher objects have an tabular inline showing books published by that publisher. The second site exposes
just publishers, without the inlines.
Using raw cursors with multiple databases
If you are using more than one database you can use django.db.connections to obtain the connection (and
cursor) for a specific database. django.db.connections is a dictionary-like object that allows you to retrieve a
specific connection using its alias:
from django.db import connections
cursor = connections[’my_db_alias’].cursor()
Limitations of multiple databases
Cross-database relations
Django doesn’t currently provide any support for foreign key or many-to-many relationships spanning multiple
databases. If you have used a router to partition models to different databases, any foreign key and many-to-many
relationships defined by those models must be internal to a single database.
This is because of referential integrity. In order to maintain a relationship between two objects, Django needs to know
that the primary key of the related object is valid. If the primary key is stored on a separate database, it’s not possible
to easily evaluate the validity of a primary key.
If you’re using Postgres, Oracle, or MySQL with InnoDB, this is enforced at the database integrity level – database
level key constraints prevent the creation of relations that can’t be validated.
However, if you’re using SQLite or MySQL with MyISAM tables, there is no enforced referential integrity; as a
result, you may be able to ‘fake’ cross database foreign keys. However, this configuration is not officially supported
by Django.
142 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-146-2048.jpg)
![Django Documentation, Release 1.5.1
{% endfor %}
{% else %}
<p>No messages today.</p>
{% endif %}
{% endwith %}
{% endif %}
It is optimal because:
1. Since QuerySets are lazy, this does no database queries if ‘display_inbox’ is False.
2. Use of with means that we store user.emails.all in a variable for later use, allowing its cache to be
re-used.
3. The line {% if emails %} causes QuerySet.__bool__() to be called, which causes the
user.emails.all() query to be run on the database, and at the least the first line to be turned into an
ORM object. If there aren’t any results, it will return False, otherwise True.
4. The use of {{ emails|length }} calls QuerySet.__len__(), filling out the rest of the cache without
doing another query.
5. The for loop iterates over the already filled cache.
In total, this code does either one or zero database queries. The only deliberate optimization performed is the use of the
with tag. Using QuerySet.exists() or QuerySet.count() at any point would cause additional queries.
Use QuerySet.update() and delete()
Rather than retrieve a load of objects, set some values, and save them individual, use a bulk SQL UPDATE statement,
via QuerySet.update(). Similarly, do bulk deletes where possible.
Note, however, that these bulk update methods cannot call the save() or delete() methods of individual instances,
which means that any custom behavior you have added for these methods will not be executed, including anything
driven from the normal database object signals.
Use foreign key values directly
If you only need a foreign key value, use the foreign key value that is already on the object you’ve got, rather than
getting the whole related object and taking its primary key. i.e. do:
entry.blog_id
instead of:
entry.blog.id
Insert in bulk
When creating objects, where possible, use the bulk_create() method to reduce the number of SQL queries. For
example:
Entry.objects.bulk_create([
Entry(headline="Python 3.0 Released"),
Entry(headline="Python 3.1 Planned")
])
148 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-152-2048.jpg)
![Django Documentation, Release 1.5.1
Create an Article:
>>> a1 = Article(headline=’Django lets you build Web apps easily’)
You can’t associate it with a Publication until it’s been saved:
>>> a1.publications.add(p1)
Traceback (most recent call last):
...
ValueError: ’Article’ instance needs to have a primary key value before a many-to-many relationship can Save it!
>>> a1.save()
Associate the Article with a Publication:
>>> a1.publications.add(p1)
Create another Article, and set it to appear in both Publications:
>>> a2 = Article(headline=’NASA uses Python’)
>>> a2.save()
>>> a2.publications.add(p1, p2)
>>> a2.publications.add(p3)
Adding a second time is OK:
>>> a2.publications.add(p3)
Adding an object of the wrong type raises TypeError:
>>> a2.publications.add(a1)
Traceback (most recent call last):
...
TypeError: ’Publication’ instance expected
Create and add a Publication to an Article in one step using create():
>>> new_publication = a2.publications.create(title=’Highlights for Children’)
Article objects have access to their related Publication objects:
>>> a1.publications.all()
[<Publication: The Python Journal>]
>>> a2.publications.all()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, Publication objects have access to their related Article objects:
>>> p2.article_set.all()
[<Article: NASA uses Python>]
>>> p1.article_set.all()
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Publication.objects.get(id=4).article_set.all()
[<Article: NASA uses Python>]
Many-to-many relationships can be queried using lookups across relationships:
>>> Article.objects.filter(publications__id__exact=1)
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__pk=1)
150 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-154-2048.jpg)
![Django Documentation, Release 1.5.1
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications=1)
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications=p1)
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__title__startswith="Science")
[<Article: NASA uses Python>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__title__startswith="Science").distinct()
[<Article: NASA uses Python>]
The count() function respects distinct() as well:
>>> Article.objects.filter(publications__title__startswith="Science").count()
2
>>> Article.objects.filter(publications__title__startswith="Science").distinct().count()
1
>>> Article.objects.filter(publications__in=[1,2]).distinct()
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
>>> Article.objects.filter(publications__in=[p1,p2]).distinct()
[<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>]
Reverse m2m queries are supported (i.e., starting at the table that doesn’t have a ManyToManyField):
>>> Publication.objects.filter(id__exact=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(pk=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article__headline__startswith="NASA")
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, >>> Publication.objects.filter(article__id__exact=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article__pk=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article=1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article=a1)
[<Publication: The Python Journal>]
>>> Publication.objects.filter(article__in=[1,2]).distinct()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, >>> Publication.objects.filter(article__in=[a1,a2]).distinct()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, Excluding a related item works as you would expect, too (although the SQL involved is a little complex):
>>> Article.objects.exclude(publications=p2)
[<Article: Django lets you build Web apps easily>]
If we delete a Publication, its Articles won’t be able to access it:
>>> p1.delete()
>>> Publication.objects.all()
[<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>]
3.2. Models and databases 151](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-155-2048.jpg)
![Django Documentation, Release 1.5.1
>>> a1 = Article.objects.get(pk=1)
>>> a1.publications.all()
[]
If we delete an Article, its Publications won’t be able to access it:
>>> a2.delete()
>>> Article.objects.all()
[<Article: Django lets you build Web apps easily>]
>>> p2.article_set.all()
[]
Adding via the ‘other’ end of an m2m:
>>> a4 = Article(headline=’NASA finds intelligent life on Earth’)
>>> a4.save()
>>> p2.article_set.add(a4)
>>> p2.article_set.all()
[<Article: NASA finds intelligent life on Earth>]
>>> a4.publications.all()
[<Publication: Science News>]
Adding via the other end using keywords:
>>> new_article = p2.article_set.create(headline=’Oxygen-free diet works wonders’)
>>> p2.article_set.all()
[<Article: NASA finds intelligent life on Earth>, <Article: Oxygen-free diet works wonders>]
>>> a5 = p2.article_set.all()[1]
>>> a5.publications.all()
[<Publication: Science News>]
Removing Publication from an Article:
>>> a4.publications.remove(p2)
>>> p2.article_set.all()
[<Article: Oxygen-free diet works wonders>]
>>> a4.publications.all()
[]
And from the other end:
>>> p2.article_set.remove(a5)
>>> p2.article_set.all()
[]
>>> a5.publications.all()
[]
Relation sets can be assigned. Assignment clears any existing set members:
>>> a4.publications.all()
[<Publication: Science News>]
>>> a4.publications = [p3]
>>> a4.publications.all()
[<Publication: Science Weekly>]
Relation sets can be cleared:
>>> p2.article_set.clear()
>>> p2.article_set.all()
[]
152 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-156-2048.jpg)
![Django Documentation, Release 1.5.1
And you can clear from the other end:
>>> p2.article_set.add(a4, a5)
>>> p2.article_set.all()
[<Article: NASA finds intelligent life on Earth>, <Article: Oxygen-free diet works wonders>]
>>> a4.publications.all()
[<Publication: Science News>, <Publication: Science Weekly>]
>>> a4.publications.clear()
>>> a4.publications.all()
[]
>>> p2.article_set.all()
[<Article: Oxygen-free diet works wonders>]
Recreate the Article and Publication we have deleted:
>>> p1 = Publication(title=’The Python Journal’)
>>> p1.save()
>>> a2 = Article(headline=’NASA uses Python’)
>>> a2.save()
>>> a2.publications.add(p1, p2, p3)
Bulk delete some Publications - references to deleted publications should go:
>>> Publication.objects.filter(title__startswith=’Science’).delete()
>>> Publication.objects.all()
[<Publication: Highlights for Children>, <Publication: The Python Journal>]
>>> Article.objects.all()
[<Article: Django lets you build Web apps easily>, <Article: NASA finds intelligent life on Earth>, <>>> a2.publications.all()
[<Publication: The Python Journal>]
Bulk delete some articles - references to deleted objects should go:
>>> q = Article.objects.filter(headline__startswith=’Django’)
>>> print(q)
[<Article: Django lets you build Web apps easily>]
>>> q.delete()
After the delete(), the QuerySet cache needs to be cleared, and the referenced objects should be gone:
>>> print(q)
[]
>>> p1.article_set.all()
[<Article: NASA uses Python>]
An alternate to calling clear() is to assign the empty set:
>>> p1.article_set = []
>>> p1.article_set.all()
[]
>>> a2.publications = [p1, new_publication]
>>> a2.publications.all()
[<Publication: Highlights for Children>, <Publication: The Python Journal>]
>>> a2.publications = []
>>> a2.publications.all()
[]
3.2. Models and databases 153](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-157-2048.jpg)
![Django Documentation, Release 1.5.1
>>> new_article = r.article_set.create(headline="John’s second story", pub_date=date(2005, 7, 29))
>>> new_article
<Article: John’s second story>
>>> new_article.reporter
<Reporter: John Smith>
>>> new_article.reporter.id
1
Create a new article, and add it to the article set:
>>> new_article2 = Article(headline="Paul’s story", pub_date=date(2006, 1, 17))
>>> r.article_set.add(new_article2)
>>> new_article2.reporter
<Reporter: John Smith>
>>> new_article2.reporter.id
1
>>> r.article_set.all()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
Add the same article to a different article set - check that it moves:
>>> r2.article_set.add(new_article2)
>>> new_article2.reporter.id
2
>>> new_article2.reporter
<Reporter: Paul Jones>
Adding an object of the wrong type raises TypeError:
>>> r.article_set.add(r2)
Traceback (most recent call last):
...
TypeError: ’Article’ instance expected
>>> r.article_set.all()
[<Article: John’s second story>, <Article: This is a test>]
>>> r2.article_set.all()
[<Article: Paul’s story>]
>>> r.article_set.count()
2
>>> r2.article_set.count()
1
Note that in the last example the article has moved from John to Paul.
Related managers support field lookups as well. The API automatically follows relationships as far as you need. Use
double underscores to separate relationships. This works as many levels deep as you want. There’s no limit. For
example:
>>> r.article_set.filter(headline__startswith=’This’)
[<Article: This is a test>]
# Find all Articles for any Reporter whose first name is "John".
>>> Article.objects.filter(reporter__first_name__exact=’John’)
[<Article: John’s second story>, <Article: This is a test>]
Exact match is implied here:
3.2. Models and databases 155](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-159-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Article.objects.filter(reporter__first_name=’John’)
[<Article: John’s second story>, <Article: This is a test>]
Query twice over the related field. This translates to an AND condition in the WHERE clause:
>>> Article.objects.filter(reporter__first_name__exact=’John’, reporter__last_name__exact=’Smith’)
[<Article: John’s second story>, <Article: This is a test>]
For the related lookup you can supply a primary key value or pass the related object explicitly:
>>> Article.objects.filter(reporter__pk=1)
[<Article: John’s second story>, <Article: This is a test>]
>>> Article.objects.filter(reporter=1)
[<Article: John’s second story>, <Article: This is a test>]
>>> Article.objects.filter(reporter=r)
[<Article: John’s second story>, <Article: This is a test>]
>>> Article.objects.filter(reporter__in=[1,2]).distinct()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
>>> Article.objects.filter(reporter__in=[r,r2]).distinct()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
You can also use a queryset instead of a literal list of instances:
>>> Article.objects.filter(reporter__in=Reporter.objects.filter(first_name=’John’)).distinct()
[<Article: John’s second story>, <Article: This is a test>]
Querying in the opposite direction:
>>> Reporter.objects.filter(article__pk=1)
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article=1)
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article=a)
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article__headline__startswith=’This’)
[<Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>]
>>> Reporter.objects.filter(article__headline__startswith=’This’).distinct()
[<Reporter: John Smith>]
Counting in the opposite direction works in conjunction with distinct():
>>> Reporter.objects.filter(article__headline__startswith=’This’).count()
3
>>> Reporter.objects.filter(article__headline__startswith=’This’).distinct().count()
1
Queries can go round in circles:
>>> Reporter.objects.filter(article__reporter__first_name__startswith=’John’)
[<Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>]
>>> Reporter.objects.filter(article__reporter__first_name__startswith=’John’).distinct()
[<Reporter: John Smith>]
>>> Reporter.objects.filter(article__reporter__exact=r).distinct()
[<Reporter: John Smith>]
If you delete a reporter, his articles will be deleted (assuming that the ForeignKey was defined with
django.db.models.ForeignKey.on_delete set to CASCADE, which is the default):
156 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-160-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Article.objects.all()
[<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>]
>>> Reporter.objects.order_by(’first_name’)
[<Reporter: John Smith>, <Reporter: Paul Jones>]
>>> r2.delete()
>>> Article.objects.all()
[<Article: John’s second story>, <Article: This is a test>]
>>> Reporter.objects.order_by(’first_name’)
[<Reporter: John Smith>]
You can delete using a JOIN in the query:
>>> Reporter.objects.filter(article__headline__startswith=’This’).delete()
>>> Reporter.objects.all()
[]
>>> Article.objects.all()
[]
One-to-one relationships
To define a one-to-one relationship, use OneToOneField.
In this example, a Place optionally can be a Restaurant:
from django.db import models, transaction, IntegrityError
class Place(models.Model):
name = models.CharField(max_length=50)
address = models.CharField(max_length=80)
def __unicode__(self):
return u"%s the place" % self.name
class Restaurant(models.Model):
place = models.OneToOneField(Place, primary_key=True)
serves_hot_dogs = models.BooleanField()
serves_pizza = models.BooleanField()
def __unicode__(self):
return u"%s the restaurant" % self.place.name
class Waiter(models.Model):
restaurant = models.ForeignKey(Restaurant)
name = models.CharField(max_length=50)
def __unicode__(self):
return u"%s the waiter at %s" % (self.name, self.restaurant)
What follows are examples of operations that can be performed using the Python API facilities.
Create a couple of Places:
>>> p1 = Place(name=’Demon Dogs’, address=’944 W. Fullerton’)
>>> p1.save()
>>> p2 = Place(name=’Ace Hardware’, address=’1013 N. Ashland’)
>>> p2.save()
Create a Restaurant. Pass the ID of the “parent” object as this object’s ID:
3.2. Models and databases 157](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-161-2048.jpg)
![Django Documentation, Release 1.5.1
>>> r = Restaurant(place=p1, serves_hot_dogs=True, serves_pizza=False)
>>> r.save()
A Restaurant can access its place:
>>> r.place
<Place: Demon Dogs the place>
A Place can access its restaurant, if available:
>>> p1.restaurant
<Restaurant: Demon Dogs the restaurant>
p2 doesn’t have an associated restaurant:
>>> p2.restaurant
Traceback (most recent call last):
...
DoesNotExist: Restaurant matching query does not exist.
Set the place using assignment notation. Because place is the primary key on Restaurant, the save will create a new
restaurant:
>>> r.place = p2
>>> r.save()
>>> p2.restaurant
<Restaurant: Ace Hardware the restaurant>
>>> r.place
<Place: Ace Hardware the place>
Set the place back again, using assignment in the reverse direction:
>>> p1.restaurant = r
>>> p1.restaurant
<Restaurant: Demon Dogs the restaurant>
Restaurant.objects.all() just returns the Restaurants, not the Places. Note that there are two restaurants - Ace Hardware
the Restaurant was created in the call to r.place = p2:
>>> Restaurant.objects.all()
[<Restaurant: Demon Dogs the restaurant>, <Restaurant: Ace Hardware the restaurant>]
Place.objects.all() returns all Places, regardless of whether they have Restaurants:
>>> Place.objects.order_by(’name’)
[<Place: Ace Hardware the place>, <Place: Demon Dogs the place>]
You can query the models using lookups across relationships:
>>> Restaurant.objects.get(place=p1)
<Restaurant: Demon Dogs the restaurant>
>>> Restaurant.objects.get(place__pk=1)
<Restaurant: Demon Dogs the restaurant>
>>> Restaurant.objects.filter(place__name__startswith="Demon")
[<Restaurant: Demon Dogs the restaurant>]
>>> Restaurant.objects.exclude(place__address__contains="Ashland")
[<Restaurant: Demon Dogs the restaurant>]
This of course works in reverse:
158 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-162-2048.jpg)
![Django Documentation, Release 1.5.1
>>> Place.objects.get(pk=1)
<Place: Demon Dogs the place>
>>> Place.objects.get(restaurant__place__exact=p1)
<Place: Demon Dogs the place>
>>> Place.objects.get(restaurant=r)
<Place: Demon Dogs the place>
>>> Place.objects.get(restaurant__place__name__startswith="Demon")
<Place: Demon Dogs the place>
Add a Waiter to the Restaurant:
>>> w = r.waiter_set.create(name=’Joe’)
>>> w.save()
>>> w
<Waiter: Joe the waiter at Demon Dogs the restaurant>
Query the waiters:
>>> Waiter.objects.filter(restaurant__place=p1)
[<Waiter: Joe the waiter at Demon Dogs the restaurant>]
>>> Waiter.objects.filter(restaurant__place__name__startswith="Demon")
[<Waiter: Joe the waiter at Demon Dogs the restaurant>]
3.3 Handling HTTP requests
Information on handling HTTP requests in Django:
3.3.1 URL dispatcher
A clean, elegant URL scheme is an important detail in a high-quality Web application. Django lets you design URLs
however you want, with no framework limitations.
There’s no .php or .cgi required, and certainly none of that 0,2097,1-1-1928,00 nonsense.
See Cool URIs don’t change, by World Wide Web creator Tim Berners-Lee, for excellent arguments on why URLs
should be clean and usable.
Overview
To design URLs for an app, you create a Python module informally called a URLconf (URL configuration). This
module is pure Python code and is a simple mapping between URL patterns (simple regular expressions) to Python
functions (your views).
This mapping can be as short or as long as needed. It can reference other mappings. And, because it’s pure Python
code, it can be constructed dynamically. New in version 1.4: Django also provides a way to translate URLs according
to the active language. See the internationalization documentation for more information.
How Django processes a request
When a user requests a page from your Django-powered site, this is the algorithm the system follows to determine
which Python code to execute:
3.3. Handling HTTP requests 159](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-163-2048.jpg)
![Django Documentation, Release 1.5.1
• handler403 – See django.conf.urls.handler403.
New in version 1.4: handler403 is new in Django 1.4.
The view prefix
You can specify a common prefix in your patterns() call, to cut down on code duplication.
Here’s the example URLconf from the Django overview:
from django.conf.urls import patterns, url
urlpatterns = patterns(’’,
url(r’^articles/(d{4})/$’, ’news.views.year_archive’),
url(r’^articles/(d{4})/(d{2})/$’, ’news.views.month_archive’),
url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’news.views.article_detail’),
)
In this example, each view has a common prefix – ’news.views’. Instead of typing that out for each entry in
urlpatterns, you can use the first argument to the patterns() function to specify a prefix to apply to each
view function.
With this in mind, the above example can be written more concisely as:
from django.conf.urls import patterns, url
urlpatterns = patterns(’news.views’,
url(r’^articles/(d{4})/$’, ’year_archive’),
url(r’^articles/(d{4})/(d{2})/$’, ’month_archive’),
url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’article_detail’),
)
Note that you don’t put a trailing dot (".") in the prefix. Django puts that in automatically.
Multiple view prefixes
In practice, you’ll probably end up mixing and matching views to the point where the views in your urlpatterns
won’t have a common prefix. However, you can still take advantage of the view prefix shortcut to remove duplication.
Just add multiple patterns() objects together, like this:
Old:
from django.conf.urls import patterns, url
urlpatterns = patterns(’’,
url(r’^$’, ’myapp.views.app_index’),
url(r’^(?P<year>d{4})/(?P<month>[a-z]{3})/$’, ’myapp.views.month_display’),
url(r’^tag/(?P<tag>w+)/$’, ’weblog.views.tag’),
)
New:
from django.conf.urls import patterns, url
urlpatterns = patterns(’myapp.views’,
url(r’^$’, ’app_index’),
url(r’^(?P<year>d{4})/(?P<month>[a-z]{3})/$’,’month_display’),
)
3.3. Handling HTTP requests 163](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-167-2048.jpg)
![Django Documentation, Release 1.5.1
from django.core.exceptions import PermissionDenied
def edit(request, pk):
if not request.user.is_staff:
raise PermissionDenied
# ...
It is possible to override django.views.defaults.permission_denied in the same way you can for the
404 and 500 views by specifying a handler403 in your URLconf:
handler403 = ’mysite.views.my_custom_permission_denied_view’
3.3.3 View decorators
Django provides several decorators that can be applied to views to support various HTTP features.
Allowed HTTP methods
The decorators in django.views.decorators.http can be used to restrict access to views based on the request
method. These decorators will return a django.http.HttpResponseNotAllowed if the conditions are not
met.
require_http_methods(request_method_list)
Decorator to require that a view only accept particular request methods. Usage:
from django.views.decorators.http import require_http_methods
@require_http_methods(["GET", "POST"])
def my_view(request):
# I can assume now that only GET or POST requests make it this far
# ...
pass
Note that request methods should be in uppercase.
require_GET()
Decorator to require that a view only accept the GET method.
require_POST()
Decorator to require that a view only accept the POST method.
require_safe()
New in version 1.4. Decorator to require that a view only accept the GET and HEAD methods. These methods
are commonly considered “safe” because they should not have the significance of taking an action other than
retrieving the requested resource.
Note: Django will automatically strip the content of responses to HEAD requests while leaving the headers un-changed,
so you may handle HEAD requests exactly like GET requests in your views. Since some software, such
as link checkers, rely on HEAD requests, you might prefer using require_safe instead of require_GET.
Conditional view processing
The following decorators in django.views.decorators.http can be used to control caching behavior on
particular views.
174 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-178-2048.jpg)
![Django Documentation, Release 1.5.1
condition(etag_func=None, last_modified_func=None)
etag(etag_func)
last_modified(last_modified_func)
These decorators can be used to generate ETag and Last-Modified headers; see conditional view process-ing.
GZip compression
The decorators in django.views.decorators.gzip control content compression on a per-view basis.
gzip_page()
This decorator compresses content if the browser allows gzip compression. It sets the Vary header accordingly,
so that caches will base their storage on the Accept-Encoding header.
Vary headers
The decorators in django.views.decorators.vary can be used to control caching based on specific request
headers.
vary_on_cookie(func)
vary_on_headers(*headers)
The Vary header defines which request headers a cache mechanism should take into account when building its
cache key.
See using vary headers.
3.3.4 File Uploads
When Django handles a file upload, the file data ends up placed in request.FILES (for more on the request
object see the documentation for request and response objects). This document explains how files are stored on disk
and in memory, and how to customize the default behavior.
Basic file uploads
Consider a simple form containing a FileField:
from django import forms
class UploadFileForm(forms.Form):
title = forms.CharField(max_length=50)
file = forms.FileField()
A view handling this form will receive the file data in request.FILES, which is a dictionary containing a key for
each FileField (or ImageField, or other FileField subclass) in the form. So the data from the above form
would be accessible as request.FILES[’file’].
Note that request.FILES will only contain data if the request method was POST and the <form> that posted the
request has the attribute enctype="multipart/form-data". Otherwise, request.FILES will be empty.
Most of the time, you’ll simply pass the file data from request into the form as described in Binding uploaded files
to a form. This would look something like:
3.3. Handling HTTP requests 175](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-179-2048.jpg)
![Django Documentation, Release 1.5.1
from django.http import HttpResponseRedirect
from django.shortcuts import render_to_response
# Imaginary function to handle an uploaded file.
from somewhere import handle_uploaded_file
def upload_file(request):
if request.method == ’POST’:
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
handle_uploaded_file(request.FILES[’file’])
return HttpResponseRedirect(’/success/url/’)
else:
form = UploadFileForm()
return render_to_response(’upload.html’, {’form’: form})
Notice that we have to pass request.FILES into the form’s constructor; this is how file data gets bound into a
form.
Handling uploaded files
class UploadedFile
The final piece of the puzzle is handling the actual file data from request.FILES. Each entry in this dictio-nary
is an UploadedFile object – a simple wrapper around an uploaded file. You’ll usually use one of these
methods to access the uploaded content:
read()
Read the entire uploaded data from the file. Be careful with this method: if the uploaded file is huge it can
overwhelm your system if you try to read it into memory. You’ll probably want to use chunks() instead;
see below.
multiple_chunks()
Returns True if the uploaded file is big enough to require reading in multiple chunks. By default this will
be any file larger than 2.5 megabytes, but that’s configurable; see below.
chunks()
A generator returning chunks of the file. If multiple_chunks() is True, you should use this method
in a loop instead of read().
In practice, it’s often easiest simply to use chunks() all the time; see the example below.
name
The name of the uploaded file (e.g. my_file.txt).
size
The size, in bytes, of the uploaded file.
There are a few other methods and attributes available on UploadedFile objects; see UploadedFile objects for a
complete reference.
Putting it all together, here’s a common way you might handle an uploaded file:
def handle_uploaded_file(f):
with open(’some/file/name.txt’, ’wb+’) as destination:
for chunk in f.chunks():
destination.write(chunk)
Looping over UploadedFile.chunks() instead of using read() ensures that large files don’t overwhelm your
system’s memory.
176 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-180-2048.jpg)
![Django Documentation, Release 1.5.1
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import ModelFormWithFileField
def upload_file(request):
if request.method == ’POST’:
form = ModelFormWithFileField(request.POST, request.FILES)
if form.is_valid():
# file is saved
form.save()
return HttpResponseRedirect(’/success/url/’)
else:
form = ModelFormWithFileField()
return render(request, ’upload.html’, {’form’: form})
If you are constructing an object manually, you can simply assign the file object from request.FILES to the file
field in the model:
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import UploadFileForm
from .models import ModelWithFileField
def upload_file(request):
if request.method == ’POST’:
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
instance = ModelWithFileField(file_field=request.FILES[’file’])
instance.save()
return HttpResponseRedirect(’/success/url/’)
else:
form = UploadFileForm()
return render(request, ’upload.html’, {’form’: form})
UploadedFile objects
In addition to those inherited from File, all UploadedFile objects define the following methods/attributes:
UploadedFile.content_type
The content-type header uploaded with the file (e.g. text/plain or application/pdf). Like any data
supplied by the user, you shouldn’t trust that the uploaded file is actually this type. You’ll still need to validate
that the file contains the content that the content-type header claims – “trust but verify.”
UploadedFile.charset
For text/* content-types, the character set (i.e. utf8) supplied by the browser. Again, “trust but verify” is
the best policy here.
UploadedFile.temporary_file_path
Only files uploaded onto disk will have this method; it returns the full path to the temporary uploaded file.
Note: Like regular Python files, you can read the file line-by-line simply by iterating over the uploaded file:
for line in uploadedfile:
do_something_with(line)
However, unlike standard Python files, UploadedFile only understands n (also known as “Unix-style”) line
endings. If you know that you need to handle uploaded files with different line endings, you’ll need to do so in your
178 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-182-2048.jpg)
![Django Documentation, Release 1.5.1
view.
Upload Handlers
When a user uploads a file, Django passes off the file data to an upload handler – a small class that handles file data as
it gets uploaded. Upload handlers are initially defined in the FILE_UPLOAD_HANDLERS setting, which defaults to:
("django.core.files.uploadhandler.MemoryFileUploadHandler",
"django.core.files.uploadhandler.TemporaryFileUploadHandler",)
Together the MemoryFileUploadHandler and TemporaryFileUploadHandler provide Django’s default
file upload behavior of reading small files into memory and large ones onto disk.
You can write custom handlers that customize how Django handles files. You could, for example, use custom handlers
to enforce user-level quotas, compress data on the fly, render progress bars, and even send data to another storage
location directly without storing it locally.
Modifying upload handlers on the fly
Sometimes particular views require different upload behavior. In these cases, you can override upload handlers on a
per-request basis by modifying request.upload_handlers. By default, this list will contain the upload handlers
given by FILE_UPLOAD_HANDLERS, but you can modify the list as you would any other list.
For instance, suppose you’ve written a ProgressBarUploadHandler that provides feedback on upload progress
to some sort of AJAX widget. You’d add this handler to your upload handlers like this:
request.upload_handlers.insert(0, ProgressBarUploadHandler())
You’d probably want to use list.insert() in this case (instead of append()) because a progress bar handler
would need to run before any other handlers. Remember, the upload handlers are processed in order.
If you want to replace the upload handlers completely, you can just assign a new list:
request.upload_handlers = [ProgressBarUploadHandler()]
Note: You can only modify upload handlers before accessing request.POST or request.FILES – it
doesn’t make sense to change upload handlers after upload handling has already started. If you try to modify
request.upload_handlers after reading from request.POST or request.FILES Django will throw an
error.
Thus, you should always modify uploading handlers as early in your view as possible.
Also, request.POST is accessed by CsrfViewMiddleware which is enabled by default. This means you will
need to use csrf_exempt() on your view to allow you to change the upload handlers. You will then need to use
csrf_protect() on the function that actually processes the request. Note that this means that the handlers may
start receiving the file upload before the CSRF checks have been done. Example code:
from django.views.decorators.csrf import csrf_exempt, csrf_protect
@csrf_exempt
def upload_file_view(request):
request.upload_handlers.insert(0, ProgressBarUploadHandler())
return _upload_file_view(request)
@csrf_protect
3.3. Handling HTTP requests 179](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-183-2048.jpg)
![Django Documentation, Release 1.5.1
This method may raise a StopFutureHandlers exception to prevent future handlers from handling this file.
FileUploadHandler.upload_complete(self) Callback signaling that the entire upload (all files) has
completed.
FileUploadHandler.handle_raw_input(self, input_data, META, content_length, boundary, encoding)
Allows the handler to completely override the parsing of the raw HTTP input.
input_data is a file-like object that supports read()-ing.
META is the same object as request.META.
content_length is the length of the data in input_data. Don’t read more than content_length
bytes from input_data.
boundary is the MIME boundary for this request.
encoding is the encoding of the request.
Return None if you want upload handling to continue, or a tuple of (POST, FILES) if you want to return
the new data structures suitable for the request directly.
3.3.5 Django shortcut functions
The package django.shortcuts collects helper functions and classes that “span” multiple levels of MVC. In
other words, these functions/classes introduce controlled coupling for convenience’s sake.
render
render(request, template_name[, dictionary][, context_instance][, content_type][, status][, current_app])
Combines a given template with a given context dictionary and returns an HttpResponse object with that
rendered text.
render() is the same as a call to render_to_response() with a context_instance argument that forces
the use of a RequestContext.
Required arguments
request The request object used to generate this response.
template_name The full name of a template to use or sequence of template names.
Optional arguments
dictionary A dictionary of values to add to the template context. By default, this is an empty dictionary. If a
value in the dictionary is callable, the view will call it just before rendering the template.
context_instance The context instance to render the template with. By default, the template will be rendered
with a RequestContext instance (filled with values from request and dictionary).
content_type The MIME type to use for the resulting document. Defaults to the value of the
DEFAULT_CONTENT_TYPE setting. Changed in version 1.5: This parameter used to be called mimetype.
status The status code for the response. Defaults to 200.
current_app A hint indicating which application contains the current view. See the namespaced URL resolution
strategy for more information.
3.3. Handling HTTP requests 181](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-185-2048.jpg)
![Django Documentation, Release 1.5.1
Example
The following example renders the template myapp/index.html with the MIME type
application/xhtml+xml:
from django.shortcuts import render
def my_view(request):
# View code here...
return render(request, ’myapp/index.html’, {"foo": "bar"},
content_type="application/xhtml+xml")
This example is equivalent to:
from django.http import HttpResponse
from django.template import RequestContext, loader
def my_view(request):
# View code here...
t = loader.get_template(’myapp/template.html’)
c = RequestContext(request, {’foo’: ’bar’})
return HttpResponse(t.render(c),
content_type="application/xhtml+xml")
render_to_response
render_to_response(template_name[, dictionary][, context_instance][, content_type])
Renders a given template with a given context dictionary and returns an HttpResponse object with that
rendered text.
Required arguments
template_name The full name of a template to use or sequence of template names. If a sequence is given, the first
template that exists will be used. See the template loader documentation for more information on how templates
are found.
Optional arguments
dictionary A dictionary of values to add to the template context. By default, this is an empty dictionary. If a
value in the dictionary is callable, the view will call it just before rendering the template.
context_instance The context instance to render the template with. By default, the template will be rendered
with a Context instance (filled with values from dictionary). If you need to use context processors, render
the template with a RequestContext instance instead. Your code might look something like this:
return render_to_response(’my_template.html’,
my_data_dictionary,
context_instance=RequestContext(request))
content_type The MIME type to use for the resulting document. Defaults to the value of the
DEFAULT_CONTENT_TYPE setting. Changed in version 1.5: This parameter used to be called mimetype.
182 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-186-2048.jpg)
![Django Documentation, Release 1.5.1
Example
The following example renders the template myapp/index.html with the MIME type
application/xhtml+xml:
from django.shortcuts import render_to_response
def my_view(request):
# View code here...
return render_to_response(’myapp/index.html’, {"foo": "bar"},
mimetype="application/xhtml+xml")
This example is equivalent to:
from django.http import HttpResponse
from django.template import Context, loader
def my_view(request):
# View code here...
t = loader.get_template(’myapp/template.html’)
c = Context({’foo’: ’bar’})
return HttpResponse(t.render(c),
content_type="application/xhtml+xml")
redirect
redirect(to[, permanent=False ], *args, **kwargs)
Returns an HttpResponseRedirect to the appropriate URL for the arguments passed.
The arguments could be:
•A model: the model’s get_absolute_url() function will be called.
•A view name, possibly with arguments: urlresolvers.reverse will be used to reverse-resolve the
name.
•A URL, which will be used as-is for the redirect location.
By default issues a temporary redirect; pass permanent=True to issue a permanent redirect
Examples
You can use the redirect() function in a number of ways.
1. By passing some object; that object’s get_absolute_url() method will be called to figure out the redirect
URL:
from django.shortcuts import redirect
def my_view(request):
...
object = MyModel.objects.get(...)
return redirect(object)
2. By passing the name of a view and optionally some positional or keyword arguments; the URL will be reverse
resolved using the reverse() method:
3.3. Handling HTTP requests 183](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-187-2048.jpg)
![Django Documentation, Release 1.5.1
Warning: The session data is signed but not encrypted
When using the cookies backend the session data can be read by the client.
A MAC (Message Authentication Code) is used to protect the data against changes by the client, so that the session
data will be invalidated when being tampered with. The same invalidation happens if the client storing the cookie
(e.g. your user’s browser) can’t store all of the session cookie and drops data. Even though Django compresses the
data, it’s still entirely possible to exceed the common limit of 4096 bytes per cookie.
No freshness guarantee
Note also that while the MAC can guarantee the authenticity of the data (that it was generated by your site, and
not someone else), and the integrity of the data (that it is all there and correct), it cannot guarantee freshness i.e.
that you are being sent back the last thing you sent to the client. This means that for some uses of session data, the
cookie backend might open you up to replay attacks. Cookies will only be detected as ‘stale’ if they are older than
your SESSION_COOKIE_AGE.
Performance
Finally, the size of a cookie can have an impact on the speed of your site.
Using sessions in views
When SessionMiddleware is activated, each HttpRequest object – the first argument to any Django view
function – will have a session attribute, which is a dictionary-like object.
You can read it and write to request.session at any point in your view. You can edit it multiple times.
class backends.base.SessionBase
This is the base class for all session objects. It has the following standard dictionary methods:
__getitem__(key)
Example: fav_color = request.session[’fav_color’]
__setitem__(key, value)
Example: request.session[’fav_color’] = ’blue’
__delitem__(key)
Example: del request.session[’fav_color’]. This raises KeyError if the given key isn’t
already in the session.
__contains__(key)
Example: ’fav_color’ in request.session
get(key, default=None)
Example: fav_color = request.session.get(’fav_color’, ’red’)
pop(key)
Example: fav_color = request.session.pop(’fav_color’)
keys()
items()
setdefault()
clear()
It also has these methods:
flush()
Delete the current session data from the session and regenerate the session key value that is sent back to
the user in the cookie. This is used if you want to ensure that the previous session data can’t be accessed
again from the user’s browser (for example, the django.contrib.auth.logout() function calls
it).
3.3. Handling HTTP requests 191](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-195-2048.jpg)
![Django Documentation, Release 1.5.1
• Don’t override request.session with a new object, and don’t access or set its attributes. Use it like a
Python dictionary.
Examples
This simplistic view sets a has_commented variable to True after a user posts a comment. It doesn’t let a user post
a comment more than once:
def post_comment(request, new_comment):
if request.session.get(’has_commented’, False):
return HttpResponse("You’ve already commented.")
c = comments.Comment(comment=new_comment)
c.save()
request.session[’has_commented’] = True
return HttpResponse(’Thanks for your comment!’)
This simplistic view logs in a “member” of the site:
def login(request):
m = Member.objects.get(username=request.POST[’username’])
if m.password == request.POST[’password’]:
request.session[’member_id’] = m.id
return HttpResponse("You’re logged in.")
else:
return HttpResponse("Your username and password didn’t match.")
...And this one logs a member out, according to login() above:
def logout(request):
try:
del request.session[’member_id’]
except KeyError:
pass
return HttpResponse("You’re logged out.")
The standard django.contrib.auth.logout() function actually does a bit more than this to prevent inadver-tent
data leakage. It calls the flush() method of request.session. We are using this example as a demonstra-tion
of how to work with session objects, not as a full logout() implementation.
Setting test cookies
As a convenience, Django provides an easy way to test whether the user’s browser accepts cookies. Just call the
set_test_cookie() method of request.session in a view, and call test_cookie_worked() in a sub-sequent
view – not in the same view call.
This awkward split between set_test_cookie() and test_cookie_worked() is necessary due to the way
cookies work. When you set a cookie, you can’t actually tell whether a browser accepted it until the browser’s next
request.
It’s good practice to use delete_test_cookie() to clean up after yourself. Do this after you’ve verified that the
test cookie worked.
Here’s a typical usage example:
def login(request):
if request.method == ’POST’:
if request.session.test_cookie_worked():
3.3. Handling HTTP requests 193](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-197-2048.jpg)
![Django Documentation, Release 1.5.1
request.session.delete_test_cookie()
return HttpResponse("You’re logged in.")
else:
return HttpResponse("Please enable cookies and try again.")
request.session.set_test_cookie()
return render_to_response(’foo/login_form.html’)
Using sessions out of views
An API is available to manipulate session data outside of a view:
>>> from django.contrib.sessions.backends.db import SessionStore
>>> import datetime
>>> s = SessionStore()
>>> s[’last_login’] = datetime.datetime(2005, 8, 20, 13, 35, 10)
>>> s.save()
>>> s.session_key
’2b1189a188b44ad18c35e113ac6ceead’
>>> s = SessionStore(session_key=’2b1189a188b44ad18c35e113ac6ceead’)
>>> s[’last_login’]
datetime.datetime(2005, 8, 20, 13, 35, 0)
In order to prevent session fixation attacks, sessions keys that don’t exist are regenerated:
>>> from django.contrib.sessions.backends.db import SessionStore
>>> s = SessionStore(session_key=’no-such-session-here’)
>>> s.save()
>>> s.session_key
’ff882814010ccbc3c870523934fee5a2’
If you’re using the django.contrib.sessions.backends.db backend, each session is just a normal Django
model. The Session model is defined in django/contrib/sessions/models.py. Because it’s a normal
model, you can access sessions using the normal Django database API:
>>> from django.contrib.sessions.models import Session
>>> s = Session.objects.get(pk=’2b1189a188b44ad18c35e113ac6ceead’)
>>> s.expire_date
datetime.datetime(2005, 8, 20, 13, 35, 12)
Note that you’ll need to call get_decoded() to get the session dictionary. This is necessary because the dictionary
is stored in an encoded format:
>>> s.session_data
’KGRwMQpTJ19hdXRoX3VzZXJfaWQnCnAyCkkxCnMuMTExY2ZjODI2Yj...’
>>> s.get_decoded()
{’user_id’: 42}
When sessions are saved
By default, Django only saves to the session database when the session has been modified – that is if any of its
dictionary values have been assigned or deleted:
# Session is modified.
request.session[’foo’] = ’bar’
194 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-198-2048.jpg)
![Django Documentation, Release 1.5.1
# Session is modified.
del request.session[’foo’]
# Session is modified.
request.session[’foo’] = {}
# Gotcha: Session is NOT modified, because this alters
# request.session[’foo’] instead of request.session.
request.session[’foo’][’bar’] = ’baz’
In the last case of the above example, we can tell the session object explicitly that it has been modified by setting the
modified attribute on the session object:
request.session.modified = True
To change this default behavior, set the SESSION_SAVE_EVERY_REQUEST setting to True. When set to True,
Django will save the session to the database on every single request.
Note that the session cookie is only sent when a session has been created or modified. If
SESSION_SAVE_EVERY_REQUEST is True, the session cookie will be sent on every request.
Similarly, the expires part of a session cookie is updated each time the session cookie is sent. Changed in version
1.5: The session is not saved if the response’s status code is 500.
Browser-length sessions vs. persistent sessions
You can control whether the session framework uses browser-length sessions vs. persistent sessions with the
SESSION_EXPIRE_AT_BROWSER_CLOSE setting.
By default, SESSION_EXPIRE_AT_BROWSER_CLOSE is set to False, which means session cookies will be stored
in users’ browsers for as long as SESSION_COOKIE_AGE. Use this if you don’t want people to have to log in every
time they open a browser.
If SESSION_EXPIRE_AT_BROWSER_CLOSE is set to True, Django will use browser-length cookies – cookies
that expire as soon as the user closes his or her browser. Use this if you want people to have to log in every time they
open a browser.
This setting is a global default and can be overwritten at a per-session level by explicitly calling the set_expiry()
method of request.session as described above in using sessions in views.
Note: Some browsers (Chrome, for example) provide settings that allow users to continue brows-ing
sessions after closing and re-opening the browser. In some cases, this can interfere with the
SESSION_EXPIRE_AT_BROWSER_CLOSE setting and prevent sessions from expiring on browser close. Please
be aware of this while testing Django applications which have the SESSION_EXPIRE_AT_BROWSER_CLOSE set-ting
enabled.
Clearing the session store
As users create new sessions on your website, session data can accumulate in your session store. If you’re using the
database backend, the django_session database table will grow. If you’re using the file backend, your temporary
directory will contain an increasing number of files.
To understand this problem, consider what happens with the database backend. When a user logs in, Django adds a
row to the django_session database table. Django updates this row each time the session data changes. If the
user logs out manually, Django deletes the row. But if the user does not log out, the row never gets deleted. A similar
process happens with the file backend.
3.3. Handling HTTP requests 195](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-199-2048.jpg)
![Django Documentation, Release 1.5.1
Note: You can still access the unvalidated data directly from request.POST at this point, but the validated data is
better.
In the above example, cc_myself will be a boolean value. Likewise, fields such as IntegerField and
FloatField convert values to a Python int and float respectively.
Read-only fields are not available in form.cleaned_data (and setting a value in a custom clean() method
won’t have any effect). These fields are displayed as text rather than as input elements, and thus are not posted back
to the server.
Extending the earlier example, here’s how the form data could be processed:
if form.is_valid():
subject = form.cleaned_data[’subject’]
message = form.cleaned_data[’message’]
sender = form.cleaned_data[’sender’]
cc_myself = form.cleaned_data[’cc_myself’]
recipients = [’info@example.com’]
if cc_myself:
recipients.append(sender)
from django.core.mail import send_mail
send_mail(subject, message, sender, recipients)
return HttpResponseRedirect(’/thanks/’) # Redirect after POST
Tip: For more on sending email from Django, see Sending email.
Displaying a form using a template
Forms are designed to work with the Django template language. In the above example, we passed our ContactForm
instance to the template using the context variable form. Here’s a simple example template:
<form action="/contact/" method="post">{% csrf_token %}
{{ form.as_p }}
<input type="submit" value="Submit" />
</form>
The form only outputs its own fields; it is up to you to provide the surrounding <form> tags and the submit button.
If your form includes uploaded files, be sure to include enctype="multipart/form-data" in the form el-ement.
If you wish to write a generic template that will work whether or not the form has files, you can use the
is_multipart() attribute on the form:
<form action="/contact/" method="post"
{% if form.is_multipart %}enctype="multipart/form-data"{% endif %}>
Forms and Cross Site Request Forgery protection
Django ships with an easy-to-use protection against Cross Site Request Forgeries. When submitting a form via POST
with CSRF protection enabled you must use the csrf_token template tag as in the preceding example. However,
since CSRF protection is not directly tied to forms in templates, this tag is omitted from the following examples in this
document.
form.as_p will output the form with each form field and accompanying label wrapped in a paragraph. Here’s the
output for our example template:
200 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-204-2048.jpg)
![Django Documentation, Release 1.5.1
3.4.3 Further topics
This covers the basics, but forms can do a whole lot more:
Formsets
class django.forms.formset.BaseFormSet
A formset is a layer of abstraction to work with multiple forms on the same page. It can be best compared to a data
grid. Let’s say you have the following form:
>>> from django import forms
>>> class ArticleForm(forms.Form):
... title = forms.CharField()
... pub_date = forms.DateField()
You might want to allow the user to create several articles at once. To create a formset out of an ArticleForm you
would do:
>>> from django.forms.formsets import formset_factory
>>> ArticleFormSet = formset_factory(ArticleForm)
You now have created a formset named ArticleFormSet. The formset gives you the ability to iterate over the
forms in the formset and display them as you would with a regular form:
>>> formset = ArticleFormSet()
>>> for form in formset:
... print(form.as_table())
<tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_As you can see it only displayed one empty form. The number of empty forms that is displayed is controlled by the
extra parameter. By default, formset_factory defines one extra form; the following example will display two
blank forms:
>>> ArticleFormSet = formset_factory(ArticleForm, extra=2)
Iterating over the formset will render the forms in the order they were created. You can change this order by
providing an alternate implementation for the __iter__() method.
Formsets can also be indexed into, which returns the corresponding form. If you override __iter__, you will need
to also override __getitem__ to have matching behavior.
Using initial data with a formset
Initial data is what drives the main usability of a formset. As shown above you can define the number of extra forms.
What this means is that you are telling the formset how many additional forms to show in addition to the number of
forms it generates from the initial data. Lets take a look at an example:
>>> ArticleFormSet = formset_factory(ArticleForm, extra=2)
>>> formset = ArticleFormSet(initial=[
... {’title’: u’Django is now open source’,
... ’pub_date’: datetime.date.today(),}
... ])
>>> for form in formset:
... print(form.as_table())
204 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-208-2048.jpg)
![Django Documentation, Release 1.5.1
... ’form-1-pub_date’: u’’, # <-- this date is missing but required
... }
>>> formset = ArticleFormSet(data)
>>> formset.is_valid()
False
>>> formset.errors
[{}, {’pub_date’: [u’This field is required.’]}]
As we can see, formset.errors is a list whose entries correspond to the forms in the formset. Validation was
performed for each of the two forms, and the expected error message appears for the second item. New in version 1.4.
We can also check if form data differs from the initial data (i.e. the form was sent without any data):
>>> data = {
... ’form-TOTAL_FORMS’: u’1’,
... ’form-INITIAL_FORMS’: u’0’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’’,
... ’form-0-pub_date’: u’’,
... }
>>> formset = ArticleFormSet(data)
>>> formset.has_changed()
False
Understanding the ManagementForm You may have noticed the additional data (form-TOTAL_FORMS,
form-INITIAL_FORMS and form-MAX_NUM_FORMS) that was required in the formset’s data above. This data is
required for the ManagementForm. This form is used by the formset to manage the collection of forms contained
in the formset. If you don’t provide this management data, an exception will be raised:
>>> data = {
... ’form-0-title’: u’Test’,
... ’form-0-pub_date’: u’’,
... }
>>> formset = ArticleFormSet(data)
Traceback (most recent call last):
...
django.forms.util.ValidationError: [u’ManagementForm data is missing or has been tampered with’]
It is used to keep track of how many form instances are being displayed. If you are adding new forms via JavaScript,
you should increment the count fields in this form as well. On the other hand, if you are using JavaScript to allow
deletion of existing objects, then you need to ensure the ones being removed are properly marked for deletion by
including form-#-DELETE in the POST data. It is expected that all forms are present in the POST data regardless.
The management form is available as an attribute of the formset itself. When rendering a formset in a template, you
can include all the management data by rendering {{ my_formset.management_form }} (substituting the
name of your formset as appropriate).
total_form_count and initial_form_count BaseFormSet has a couple of methods that are closely
related to the ManagementForm, total_form_count and initial_form_count.
total_form_count returns the total number of forms in this formset. initial_form_count returns the
number of forms in the formset that were pre-filled, and is also used to determine how many forms are required. You
will probably never need to override either of these methods, so please be sure you understand what they do before
doing so.
empty_form BaseFormSet provides an additional attribute empty_form which returns a form instance with
a prefix of __prefix__ for easier use in dynamic forms with JavaScript.
206 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-210-2048.jpg)
![Django Documentation, Release 1.5.1
Custom formset validation A formset has a clean method similar to the one on a Form class. This is where you
define your own validation that works at the formset level:
>>> from django.forms.formsets import BaseFormSet
>>> class BaseArticleFormSet(BaseFormSet):
... def clean(self):
... """Checks that no two articles have the same title."""
... if any(self.errors):
... # Don’t bother validating the formset unless each form is valid on its own
... return
... titles = []
... for i in range(0, self.total_form_count()):
... form = self.forms[i]
... title = form.cleaned_data[’title’]
... if title in titles:
... raise forms.ValidationError("Articles in a set must have distinct titles.")
... titles.append(title)
>>> ArticleFormSet = formset_factory(ArticleForm, formset=BaseArticleFormSet)
>>> data = {
... ’form-TOTAL_FORMS’: u’2’,
... ’form-INITIAL_FORMS’: u’0’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’Test’,
... ’form-0-pub_date’: u’1904-06-16’,
... ’form-1-title’: u’Test’,
... ’form-1-pub_date’: u’1912-06-23’,
... }
>>> formset = ArticleFormSet(data)
>>> formset.is_valid()
False
>>> formset.errors
[{}, {}]
>>> formset.non_form_errors()
[u’Articles in a set must have distinct titles.’]
The formset clean method is called after all the Form.clean methods have been called. The errors will be found
using the non_form_errors() method on the formset.
Dealing with ordering and deletion of forms
Common use cases with a formset is dealing with ordering and deletion of the form instances. This has been dealt
with for you. The formset_factory provides two optional parameters can_order and can_delete that will
do the extra work of adding the extra fields and providing simpler ways of getting to that data.
can_order Default: False
Lets you create a formset with the ability to order:
>>> ArticleFormSet = formset_factory(ArticleForm, can_order=True)
>>> formset = ArticleFormSet(initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> for form in formset:
... print(form.as_table())
3.4. Working with forms 207](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-211-2048.jpg)
![Django Documentation, Release 1.5.1
<tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-ORDER">Order:</label></th><td><input type="text" name="form-0-ORDER" value="<tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" value="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_<tr><th><label for="id_form-1-ORDER">Order:</label></th><td><input type="text" name="form-1-ORDER" value="<tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_<tr><th><label for="id_form-2-ORDER">Order:</label></th><td><input type="text" name="form-2-ORDER" id="This adds an additional field to each form. This new field is named ORDER and is an forms.IntegerField.
For the forms that came from the initial data it automatically assigned them a numeric value. Let’s look at what will
happen when the user changes these values:
>>> data = {
... ’form-TOTAL_FORMS’: u’3’,
... ’form-INITIAL_FORMS’: u’2’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’Article #1’,
... ’form-0-pub_date’: u’2008-05-10’,
... ’form-0-ORDER’: u’2’,
... ’form-1-title’: u’Article #2’,
... ’form-1-pub_date’: u’2008-05-11’,
... ’form-1-ORDER’: u’1’,
... ’form-2-title’: u’Article #3’,
... ’form-2-pub_date’: u’2008-05-01’,
... ’form-2-ORDER’: u’0’,
... }
>>> formset = ArticleFormSet(data, initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> formset.is_valid()
True
>>> for form in formset.ordered_forms:
... print(form.cleaned_data)
{’pub_date’: datetime.date(2008, 5, 1), ’ORDER’: 0, ’title’: u’Article #3’}
{’pub_date’: datetime.date(2008, 5, 11), ’ORDER’: 1, ’title’: u’Article #2’}
{’pub_date’: datetime.date(2008, 5, 10), ’ORDER’: 2, ’title’: u’Article #1’}
can_delete Default: False
Lets you create a formset with the ability to delete:
>>> ArticleFormSet = formset_factory(ArticleForm, can_delete=True)
>>> formset = ArticleFormSet(initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> for form in formset:
.... print(form.as_table())
<input type="hidden" name="form-TOTAL_FORMS" value="3" id="id_form-TOTAL_FORMS" /><input type="hidden" <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-DELETE">Delete:</label></th><td><input type="checkbox" name="form-0-DELETE" <tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" value="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_208 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-212-2048.jpg)
![Django Documentation, Release 1.5.1
<tr><th><label for="id_form-1-DELETE">Delete:</label></th><td><input type="checkbox" name="form-1-DELETE" <tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_<tr><th><label for="id_form-2-DELETE">Delete:</label></th><td><input type="checkbox" name="form-2-DELETE" Similar to can_order this adds a new field to each form named DELETE and is a forms.BooleanField. When
data comes through marking any of the delete fields you can access them with deleted_forms:
>>> data = {
... ’form-TOTAL_FORMS’: u’3’,
... ’form-INITIAL_FORMS’: u’2’,
... ’form-MAX_NUM_FORMS’: u’’,
... ’form-0-title’: u’Article #1’,
... ’form-0-pub_date’: u’2008-05-10’,
... ’form-0-DELETE’: u’on’,
... ’form-1-title’: u’Article #2’,
... ’form-1-pub_date’: u’2008-05-11’,
... ’form-1-DELETE’: u’’,
... ’form-2-title’: u’’,
... ’form-2-pub_date’: u’’,
... ’form-2-DELETE’: u’’,
... }
>>> formset = ArticleFormSet(data, initial=[
... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)},
... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)},
... ])
>>> [form.cleaned_data for form in formset.deleted_forms]
[{’DELETE’: True, ’pub_date’: datetime.date(2008, 5, 10), ’title’: u’Article #1’}]
Adding additional fields to a formset
If you need to add additional fields to the formset this can be easily accomplished. The formset base class provides
an add_fields method. You can simply override this method to add your own fields or even redefine the default
fields/attributes of the order and deletion fields:
>>> class BaseArticleFormSet(BaseFormSet):
... def add_fields(self, form, index):
... super(BaseArticleFormSet, self).add_fields(form, index)
... form.fields["my_field"] = forms.CharField()
>>> ArticleFormSet = formset_factory(ArticleForm, formset=BaseArticleFormSet)
>>> formset = ArticleFormSet()
>>> for form in formset:
... print(form.as_table())
<tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-my_field">My field:</label></th><td><input type="text" name="form-0-my_Using a formset in views and templates
Using a formset inside a view is as easy as using a regular Form class. The only thing you will want to be aware of is
making sure to use the management form inside the template. Let’s look at a sample view:
3.4. Working with forms 209](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-213-2048.jpg)
![Django Documentation, Release 1.5.1
The save() method returns the instances that have been saved to the database. If a given instance’s data didn’t change
in the bound data, the instance won’t be saved to the database and won’t be included in the return value (instances,
in the above example).
When fields are missing from the form (for example because they have been excluded), these fields will not be
set by the save() method. You can find more information about this restriction, which also holds for regular
ModelForms, in Using a subset of fields on the form.
Pass commit=False to return the unsaved model instances:
# don’t save to the database
>>> instances = formset.save(commit=False)
>>> for instance in instances:
... # do something with instance
... instance.save()
This gives you the ability to attach data to the instances before saving them to the database. If your formset contains
a ManyToManyField, you’ll also need to call formset.save_m2m() to ensure the many-to-many relationships
are saved properly.
Limiting the number of editable objects As with regular formsets, you can use the max_num and extra param-eters
to modelformset_factory() to limit the number of extra forms displayed.
max_num does not prevent existing objects from being displayed:
>>> Author.objects.order_by(’name’)
[<Author: Charles Baudelaire>, <Author: Paul Verlaine>, <Author: Walt Whitman>]
>>> AuthorFormSet = modelformset_factory(Author, max_num=1)
>>> formset = AuthorFormSet(queryset=Author.objects.order_by(’name’))
>>> [x.name for x in formset.get_queryset()]
[u’Charles Baudelaire’, u’Paul Verlaine’, u’Walt Whitman’]
If the value of max_num is greater than the number of existing related objects, up to extra additional blank forms
will be added to the formset, so long as the total number of forms does not exceed max_num:
>>> AuthorFormSet = modelformset_factory(Author, max_num=4, extra=2)
>>> formset = AuthorFormSet(queryset=Author.objects.order_by(’name’))
>>> for form in formset:
... print(form.as_table())
<tr><th><label for="id_form-0-name">Name:</label></th><td><input id="id_form-0-name" type="text" name="<tr><th><label for="id_form-1-name">Name:</label></th><td><input id="id_form-1-name" type="text" name="<tr><th><label for="id_form-2-name">Name:</label></th><td><input id="id_form-2-name" type="text" name="<tr><th><label for="id_form-3-name">Name:</label></th><td><input id="id_form-3-name" type="text" name="A max_num value of None (the default) puts a high limit on the number of forms displayed (1000). In practice this
is equivalent to no limit.
Using a model formset in a view Model formsets are very similar to formsets. Let’s say we want to present a
formset to edit Author model instances:
def manage_authors(request):
AuthorFormSet = modelformset_factory(Author)
if request.method == ’POST’:
formset = AuthorFormSet(request.POST, request.FILES)
if formset.is_valid():
formset.save()
# do something.
220 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-224-2048.jpg)
![Django Documentation, Release 1.5.1
However, Media objects have some other interesting properties.
Media subsets If you only want media of a particular type, you can use the subscript operator to filter out a medium
of interest. For example:
>>> w = CalendarWidget()
>>> print(w.media)
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
<script type="text/javascript" src="http://static.example.com/animations.js"></script>
<script type="text/javascript" src="http://static.example.com/actions.js"></script>
>>> print(w.media)[’css’]
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
When you use the subscript operator, the value that is returned is a new Media object – but one that only contains the
media of interest.
Combining media objects Media objects can also be added together. When two media objects are added, the
resulting Media object contains the union of the media from both files:
>>> class CalendarWidget(forms.TextInput):
... class Media:
... css = {
... ’all’: (’pretty.css’,)
... }
... js = (’animations.js’, ’actions.js’)
>>> class OtherWidget(forms.TextInput):
... class Media:
... js = (’whizbang.js’,)
>>> w1 = CalendarWidget()
>>> w2 = OtherWidget()
>>> print(w1.media + w2.media)
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
<script type="text/javascript" src="http://static.example.com/animations.js"></script>
<script type="text/javascript" src="http://static.example.com/actions.js"></script>
<script type="text/javascript" src="http://static.example.com/whizbang.js"></script>
Media on Forms
Widgets aren’t the only objects that can have media definitions – forms can also define media. The rules for media
definitions on forms are the same as the rules for widgets: declarations can be static or dynamic; path and inheritance
rules for those declarations are exactly the same.
Regardless of whether you define a media declaration, all Form objects have a media property. The default value for
this property is the result of adding the media definitions for all widgets that are part of the form:
>>> class ContactForm(forms.Form):
... date = DateField(widget=CalendarWidget)
... name = CharField(max_length=40, widget=OtherWidget)
>>> f = ContactForm()
>>> f.media
<link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" />
<script type="text/javascript" src="http://static.example.com/animations.js"></script>
3.4. Working with forms 227](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-231-2048.jpg)
![Django Documentation, Release 1.5.1
Because dictionary lookup happens first, that behavior kicks in and provides a default value instead of using the
intended .iteritems() method. In this case, consider converting to a dictionary first.
In the above example, {{ section.title }} will be replaced with the title attribute of the section object.
If you use a variable that doesn’t exist, the template system will insert the value of the
TEMPLATE_STRING_IF_INVALID setting, which is set to ” (the empty string) by default.
Note that “bar” in a template expression like {{ foo.bar }} will be interpreted as a literal string and not using the
value of the variable “bar”, if one exists in the template context.
3.5.3 Filters
You can modify variables for display by using filters.
Filters look like this: {{ name|lower }}. This displays the value of the {{ name }} variable after being
filtered through the lower filter, which converts text to lowercase. Use a pipe (|) to apply a filter.
Filters can be “chained.” The output of one filter is applied to the next. {{ text|escape|linebreaks }} is a
common idiom for escaping text contents, then converting line breaks to <p> tags.
Some filters take arguments. A filter argument looks like this: {{ bio|truncatewords:30 }}. This will
display the first 30 words of the bio variable.
Filter arguments that contain spaces must be quoted; for example, to join a list with commas and spaced you’d use {{
list|join:", " }}.
Django provides about thirty built-in template filters. You can read all about them in the built-in filter reference. To
give you a taste of what’s available, here are some of the more commonly used template filters:
default If a variable is false or empty, use given default. Otherwise, use the value of the variable
For example:
{{ value|default:"nothing" }}
If value isn’t provided or is empty, the above will display “nothing”.
length Returns the length of the value. This works for both strings and lists; for example:
{{ value|length }}
If value is [’a’, ’b’, ’c’, ’d’], the output will be 4.
striptags Strips all [X]HTML tags. For example:
{{ value|striptags }}
If value is "<b>Joel</b> <button>is</button> a <span>slug</span>", the output will be
"Joel is a slug".
Again, these are just a few examples; see the built-in filter reference for the complete list.
You can also create your own custom template filters; see Custom template tags and filters.
See Also:
Django’s admin interface can include a complete reference of all template tags and filters available for a given site.
See The Django admin documentation generator.
230 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-234-2048.jpg)
![Django Documentation, Release 1.5.1
Let’s start by looking at some examples of showing a list of objects or an individual object.
We’ll be using these models:
# models.py
from django.db import models
class Publisher(models.Model):
name = models.CharField(max_length=30)
address = models.CharField(max_length=50)
city = models.CharField(max_length=60)
state_province = models.CharField(max_length=30)
country = models.CharField(max_length=50)
website = models.URLField()
class Meta:
ordering = ["-name"]
def __unicode__(self):
return self.name
class Book(models.Model):
title = models.CharField(max_length=100)
authors = models.ManyToManyField(’Author’)
publisher = models.ForeignKey(Publisher)
publication_date = models.DateField()
Now we need to define a view:
# views.py
from django.views.generic import ListView
from books.models import Publisher
class PublisherList(ListView):
model = Publisher
Finally hook that view into your urls:
# urls.py
from django.conf.urls import patterns, url
from books.views import PublisherList
urlpatterns = patterns(’’,
url(r’^publishers/$’, PublisherList.as_view()),
)
That’s all the Python code we need to write. We still need to write a template, however. We could explicitly
tell the view which template to use by adding a template_name attribute to the view, but in the absence of
an explicit template Django will infer one from the object’s name. In this case, the inferred template will be
"books/publisher_list.html" – the “books” part comes from the name of the app that defines the model,
while the “publisher” bit is just the lowercased version of the model’s name.
Note: Thus, when (for example) the django.template.loaders.app_directories.Loader
template loader is enabled in TEMPLATE_LOADERS, a template location could be:
/path/to/project/books/templates/books/publisher_list.html
This template will be rendered against a context containing a variable called object_list that contains all the
publisher objects. A very simple template might look like the following:
3.6. Class-based views 243](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-247-2048.jpg)
![Django Documentation, Release 1.5.1
def get_context_data(self, **kwargs):
# Call the base implementation first to get a context
context = super(PublisherDetail, self).get_context_data(**kwargs)
# Add in a QuerySet of all the books
context[’book_list’] = Book.objects.all()
return context
Note: Generally, get_context_data will merge the context data of all parent classes with those of the current class.
To preserve this behavior in your own classes where you want to alter the context, you should be sure to call
get_context_data on the super class. When no two classes try to define the same key, this will give the expected
results. However if any class attempts to override a key after parent classes have set it (after the call to super), any
children of that class will also need to explictly set it after super if they want to be sure to override all parents. If you’re
having trouble, review the method resolution order of your view.
Viewing subsets of objects
Now let’s take a closer look at the model argument we’ve been using all along. The model argument, which specifies
the database model that the view will operate upon, is available on all the generic views that operate on a single object
or a collection of objects. However, the model argument is not the only way to specify the objects that the view will
operate upon – you can also specify the list of objects using the queryset argument:
from django.views.generic import DetailView
from books.models import Publisher, Book
class PublisherDetail(DetailView):
context_object_name = ’publisher’
queryset = Publisher.objects.all()
Specifying model = Publisher is really just shorthand for saying queryset =
Publisher.objects.all(). However, by using queryset to define a filtered list of objects you can
be more specific about the objects that will be visible in the view (see Making queries for more information about
QuerySet objects, and see the class-based views reference for the complete details).
To pick a simple example, we might want to order a list of books by publication date, with the most recent first:
from django.views.generic import ListView
from books.models import Book
class BookList(ListView):
queryset = Book.objects.order_by(’-publication_date’)
context_object_name = ’book_list’
That’s a pretty simple example, but it illustrates the idea nicely. Of course, you’ll usually want to do more than just
reorder objects. If you want to present a list of books by a particular publisher, you can use the same technique:
from django.views.generic import ListView
from books.models import Book
class AcmeBookList(ListView):
context_object_name = ’book_list’
queryset = Book.objects.filter(publisher__name=’Acme Publishing’)
template_name = ’books/acme_list.html’
3.6. Class-based views 245](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-249-2048.jpg)
![Django Documentation, Release 1.5.1
Notice that along with a filtered queryset, we’re also using a custom template name. If we didn’t, the generic view
would use the same template as the “vanilla” object list, which might not be what we want.
Also notice that this isn’t a very elegant way of doing publisher-specific books. If we want to add another publisher
page, we’d need another handful of lines in the URLconf, and more than a few publishers would get unreasonable.
We’ll deal with this problem in the next section.
Note: If you get a 404 when requesting /books/acme/, check to ensure you actually have a Publisher with the
name ‘ACME Publishing’. Generic views have an allow_empty parameter for this case. See the class-based-views
reference for more details.
Dynamic filtering
Another common need is to filter down the objects given in a list page by some key in the URL. Earlier we hard-coded
the publisher’s name in the URLconf, but what if we wanted to write a view that displayed all the books by some
arbitrary publisher?
Handily, the ListView has a get_queryset() method we can override. Previously, it has just been returning the
value of the queryset attribute, but now we can add more logic.
The key part to making this work is that when class-based views are called, various useful things are stored on self; as
well as the request (self.request) this includes the positional (self.args) and name-based (self.kwargs)
arguments captured according to the URLconf.
Here, we have a URLconf with a single captured group:
# urls.py
from books.views import PublisherBookList
urlpatterns = patterns(’’,
(r’^books/([w-]+)/$’, PublisherBookList.as_view()),
)
Next, we’ll write the PublisherBookList view itself:
# views.py
from django.shortcuts import get_object_or_404
from django.views.generic import ListView
from books.models import Book, Publisher
class PublisherBookList(ListView):
template_name = ’books/books_by_publisher.html’
def get_queryset(self):
self.publisher = get_object_or_404(Publisher, name=self.args[0])
return Book.objects.filter(publisher=self.publisher)
As you can see, it’s quite easy to add more logic to the queryset selection; if we wanted, we could use
self.request.user to filter using the current user, or other more complex logic.
We can also add the publisher into the context at the same time, so we can use it in the template:
# ...
def get_context_data(self, **kwargs):
# Call the base implementation first to get a context
context = super(PublisherBookList, self).get_context_data(**kwargs)
246 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-250-2048.jpg)
![Django Documentation, Release 1.5.1
# Add in the publisher
context[’publisher’] = self.publisher
return context
Performing extra work
The last common pattern we’ll look at involves doing some extra work before or after calling the generic view.
Imagine we had a last_accessed field on our Author object that we were using to keep track of the last time
anybody looked at that author:
# models.py
class Author(models.Model):
salutation = models.CharField(max_length=10)
name = models.CharField(max_length=200)
email = models.EmailField()
headshot = models.ImageField(upload_to=’/tmp’)
last_accessed = models.DateTimeField()
The generic DetailView class, of course, wouldn’t know anything about this field, but once again we could easily
write a custom view to keep that field updated.
First, we’d need to add an author detail bit in the URLconf to point to a custom view:
from books.views import AuthorDetailView
urlpatterns = patterns(’’,
#...
url(r’^authors/(?P<pk>d+)/$’, AuthorDetailView.as_view(), name=’author-detail’),
)
Then we’d write our new view – get_object is the method that retrieves the object – so we simply override it and
wrap the call:
from django.views.generic import DetailView
from django.shortcuts import get_object_or_404
from django.utils import timezone
from books.models import Author
class AuthorDetailView(DetailView):
queryset = Author.objects.all()
def get_object(self):
# Call the superclass
object = super(AuthorDetailView, self).get_object()
# Record the last accessed date
object.last_accessed = timezone.now()
object.save()
# Return the object
return object
Note: The URLconf here uses the named group pk - this name is the default name that DetailView uses to find
the value of the primary key used to filter the queryset.
If you want to call the group something else, you can set pk_url_kwarg on the view. More details can be found in
the reference for DetailView
3.6. Class-based views 247](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-251-2048.jpg)
![Django Documentation, Release 1.5.1
Note that you’ll need to decorate this view using login_required(), or alternatively handle unauthorized users
in the form_valid().
AJAX example
Here is a simple example showing how you might go about implementing a form that works for AJAX requests as
well as ‘normal’ form POSTs:
import json
from django.http import HttpResponse
from django.views.generic.edit import CreateView
class AjaxableResponseMixin(object):
"""
Mixin to add AJAX support to a form.
Must be used with an object-based FormView (e.g. CreateView)
"""
def render_to_json_response(self, context, **response_kwargs):
data = json.dumps(context)
response_kwargs[’content_type’] = ’application/json’
return HttpResponse(data, **response_kwargs)
def form_invalid(self, form):
response = super(AjaxableResponseMixin, self).form_invalid(form)
if self.request.is_ajax():
return self.render_to_json_response(form.errors, status=400)
else:
return response
def form_valid(self, form):
# We make sure to call the parent’s form_valid() method because
# it might do some processing (in the case of CreateView, it will
# call form.save() for example).
response = super(AjaxableResponseMixin, self).form_valid(form)
if self.request.is_ajax():
data = {
’pk’: self.object.pk,
}
return self.render_to_json_response(data)
else:
return response
class AuthorCreate(AjaxableResponseMixin, CreateView):
model = Author
3.6.4 Using mixins with class-based views
Caution: This is an advanced topic. A working knowledge of Django’s class-based views is advised before
exploring these techniques.
Django’s built-in class-based views provide a lot of functionality, but some of it you may want to use separately.
For instance, you may want to write a view that renders a template to make the HTTP response, but you can’t use
TemplateView; perhaps you need to render a template only on POST, with GET doing something else entirely.
While you could use TemplateResponse directly, this will likely result in duplicate code.
3.6. Class-based views 251](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-255-2048.jpg)
![Django Documentation, Release 1.5.1
explictly ensure the Publisher is in the context data. ListView will add in the suitable page_obj and paginator
for us providing we remember to call super().
Now we can write a new PublisherDetail:
from django.views.generic import ListView
from django.views.generic.detail import SingleObjectMixin
from books.models import Publisher
class PublisherDetail(SingleObjectMixin, ListView):
paginate_by = 2
template_name = "books/publisher_detail.html"
def get_context_data(self, **kwargs):
kwargs[’publisher’] = self.object
return super(PublisherDetail, self).get_context_data(**kwargs)
def get_queryset(self):
self.object = self.get_object(Publisher.objects.all())
return self.object.book_set.all()
Notice how we set self.object within get_queryset() so we can use it again later in
get_context_data(). If you don’t set template_name, the template will default to the normal ListView
choice, which in this case would be "books/book_list.html" because it’s a list of books; ListView knows
nothing about SingleObjectMixin, so it doesn’t have any clue this view is anything to do with a Publisher.
The paginate_by is deliberately small in the example so you don’t have to create lots of books to see the pagination
working! Here’s the template you’d want to use:
{% extends "base.html" %}
{% block content %}
<h2>Publisher {{ publisher.name }}</h2>
<ol>
{% for book in page_obj %}
<li>{{ book.title }}</li>
{% endfor %}
</ol>
<div class="pagination">
<span class="step-links">
{% if page_obj.has_previous %}
<a href="?page={{ page_obj.previous_page_number }}">previous</a>
{% endif %}
<span class="current">
Page {{ page_obj.number }} of {{ paginator.num_pages }}.
</span>
{% if page_obj.has_next %}
<a href="?page={{ page_obj.next_page_number }}">next</a>
{% endif %}
</span>
</div>
{% endblock %}
3.6. Class-based views 255](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-259-2048.jpg)
![Django Documentation, Release 1.5.1
def get(self, request, *args, **kwargs):
view = AuthorDisplay.as_view()
return view(request, *args, **kwargs)
def post(self, request, *args, **kwargs):
view = AuthorInterest.as_view()
return view(request, *args, **kwargs)
This approach can also be used with any other generic class-based views or your own class-based views inheriting
directly from View or TemplateView, as it keeps the different views as separate as possible.
More than just HTML
Where class based views shine is when you want to do the same thing many times. Suppose you’re writing an API,
and every view should return JSON instead of rendered HTML.
We can create a mixin class to use in all of our views, handling the conversion to JSON once.
For example, a simple JSON mixin might look something like this:
import json
from django.http import HttpResponse
class JSONResponseMixin(object):
"""
A mixin that can be used to render a JSON response.
"""
response_class = HttpResponse
def render_to_response(self, context, **response_kwargs):
"""
Returns a JSON response, transforming ’context’ to make the payload.
"""
response_kwargs[’content_type’] = ’application/json’
return self.response_class(
self.convert_context_to_json(context),
**response_kwargs
)
def convert_context_to_json(self, context):
"Convert the context dictionary into a JSON object"
# Note: This is *EXTREMELY* naive; in reality, you’ll need
# to do much more complex handling to ensure that arbitrary
# objects -- such as Django model instances or querysets
# -- can be serialized as JSON.
return json.dumps(context)
Now we mix this into the base TemplateView:
from django.views.generic import TemplateView
class JSONView(JSONResponseMixin, TemplateView):
pass
Equally we could use our mixin with one of the generic views. We can make our own version of DetailView
by mixing JSONResponseMixin with the django.views.generic.detail.BaseDetailView – (the
DetailView before template rendering behavior has been mixed in):
3.6. Class-based views 259](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-263-2048.jpg)
![Django Documentation, Release 1.5.1
# some_app/views.py
from django.views.generic import TemplateView
class AboutView(TemplateView):
template_name = "about.html"
Then we just need to add this new view into our URLconf. ~django.views.generic.base.TemplateView is a class, not
a function, so we point the URL to the as_view() class method instead, which provides a function-like entry to
class-based views:
# urls.py
from django.conf.urls import patterns
from some_app.views import AboutView
urlpatterns = patterns(’’,
(r’^about/’, AboutView.as_view()),
)
For more information on how to use the built in generic views, consult the next topic on generic class based views.
Supporting other HTTP methods
Suppose somebody wants to access our book library over HTTP using the views as an API. The API client would
connect every now and then and download book data for the books published since last visit. But if no new books
appeared since then, it is a waste of CPU time and bandwidth to fetch the books from the database, render a full
response and send it to the client. It might be preferable to ask the API when the most recent book was published.
We map the URL to book list view in the URLconf:
from django.conf.urls import patterns
from books.views import BookListView
urlpatterns = patterns(’’,
(r’^books/$’, BookListView.as_view()),
)
And the view:
from django.http import HttpResponse
from django.views.generic import ListView
from books.models import Book
class BookListView(ListView):
model = Book
def head(self, *args, **kwargs):
last_book = self.get_queryset().latest(’publication_date’)
response = HttpResponse(’’)
# RFC 1123 date format
response[’Last-Modified’] = last_book.publication_date.strftime(’%a, %d %b %Y %H:%M:%S GMT’)
return response
If the view is accessed from a GET request, a plain-and-simple object list is returned in the response (using
book_list.html template). But if the client issues a HEAD request, the response has an empty body and the
Last-Modified header indicates when the most recent book was published. Based on this information, the client
may or may not download the full object list.
3.6. Class-based views 261](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-265-2048.jpg)
![Django Documentation, Release 1.5.1
Now you can use any of the documented attributes and methods of the File class.
Be aware that files created in this way are not automatically closed. The following approach may be used to close files
automatically:
>>> from django.core.files import File
# Create a Python file object using open() and the with statement
>>> with open(’/tmp/hello.world’, ’w’) as f:
>>> myfile = File(f)
>>> for line in myfile:
>>> print line
>>> myfile.closed
True
>>> f.closed
True
Closing files is especially important when accessing file fields in a loop over a large number of objects:: If files are
not manually closed after accessing them, the risk of running out of file descriptors may arise. This may lead to the
following error:
IOError: [Errno 24] Too many open files
3.7.3 File storage
Behind the scenes, Django delegates decisions about how and where to store files to a file storage system. This is the
object that actually understands things like file systems, opening and reading files, etc.
Django’s default file storage is given by the DEFAULT_FILE_STORAGE setting; if you don’t explicitly provide a
storage system, this is the one that will be used.
See below for details of the built-in default file storage system, and see Writing a custom storage system for information
on writing your own file storage system.
Storage objects
Though most of the time you’ll want to use a File object (which delegates to the proper storage for that file), you can
use file storage systems directly. You can create an instance of some custom file storage class, or – often more useful
– you can use the global default storage system:
>>> from django.core.files.storage import default_storage
>>> from django.core.files.base import ContentFile
>>> path = default_storage.save(’/path/to/file’, ContentFile(’new content’))
>>> path
u’/path/to/file’
>>> default_storage.size(path)
11
>>> default_storage.open(path).read()
’new content’
>>> default_storage.delete(path)
>>> default_storage.exists(path)
False
See File storage API for the file storage API.
3.7. Managing files 263](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-267-2048.jpg)
![Django Documentation, Release 1.5.1
The headers sent via **extra should follow CGI specification. For example, emulating a different
“Host” header as sent in the HTTP request from the browser to the server should be passed as HTTP_HOST.
If you already have the GET arguments in URL-encoded form, you can use that encoding instead of using
the data argument. For example, the previous GET request could also be posed as:
>>> c = Client()
>>> c.get(’/customers/details/?name=fred&age=7’)
If you provide a URL with both an encoded GET data and a data argument, the data argument will take
precedence.
If you set follow to True the client will follow any redirects and a redirect_chain attribute will
be set in the response object containing tuples of the intermediate urls and status codes.
If you had a URL /redirect_me/ that redirected to /next/, that redirected to /final/, this is what
you’d see:
>>> response = c.get(’/redirect_me/’, follow=True)
>>> response.redirect_chain
[(u’http://testserver/next/’, 302), (u’http://testserver/final/’, 302)]
post(path, data={}, content_type=MULTIPART_CONTENT, follow=False, **extra)
Makes a POST request on the provided path and returns a Response object, which is documented
below.
The key-value pairs in the data dictionary are used to submit POST data. For example:
>>> c = Client()
>>> c.post(’/login/’, {’name’: ’fred’, ’passwd’: ’secret’})
...will result in the evaluation of a POST request to this URL:
/login/
...with this POST data:
name=fred&passwd=secret
If you provide content_type (e.g. text/xml for an XML payload), the contents of data will be
sent as-is in the POST request, using content_type in the HTTP Content-Type header.
If you don’t provide a value for content_type, the values in data will be transmitted with a con-tent
type of multipart/form-data. In this case, the key-value pairs in data will be encoded as a
multipart message and used to create the POST data payload.
To submit multiple values for a given key – for example, to specify the selections for a <select
multiple> – provide the values as a list or tuple for the required key. For example, this value of data
would submit three selected values for the field named choices:
{’choices’: (’a’, ’b’, ’d’)}
Submitting files is a special case. To POST a file, you need only provide the file field name as a key, and a
file handle to the file you wish to upload as a value. For example:
>>> c = Client()
>>> with open(’wishlist.doc’) as fp:
... c.post(’/customers/wishes/’, {’name’: ’fred’, ’attachment’: fp})
(The name attachment here is not relevant; use whatever name your file-processing code expects.)
3.8. Testing in Django 271](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-275-2048.jpg)
![Django Documentation, Release 1.5.1
>>> c = Client()
>>> c.login(username=’fred’, password=’secret’)
# Now you can access a view that’s only available to logged-in users.
If you’re using a different authentication backend, this method may require different credentials. It requires
whichever credentials are required by your backend’s authenticate() method.
login() returns True if it the credentials were accepted and login was successful.
Finally, you’ll need to remember to create user accounts before you can use this method. As we explained
above, the test runner is executed using a test database, which contains no users by default. As a result,
user accounts that are valid on your production site will not work under test conditions. You’ll need to
create users as part of the test suite – either manually (using the Django model API) or with a test fixture.
Remember that if you want your test user to have a password, you can’t set the user’s password by setting
the password attribute directly – you must use the set_password() function to store a correctly hashed
password. Alternatively, you can use the create_user() helper method to create a new user with a
correctly hashed password.
logout()
If your site uses Django’s authentication system, the logout() method can be used to simulate the effect
of a user logging out of your site.
After you call this method, the test client will have all the cookies and session data cleared to defaults.
Subsequent requests will appear to come from an AnonymousUser.
Testing responses The get() and post() methods both return a Response object. This Response object is
not the same as the HttpResponse object returned Django views; the test response object has some additional data
useful for test code to verify.
Specifically, a Response object has the following attributes:
class Response
client
The test client that was used to make the request that resulted in the response.
content
The body of the response, as a string. This is the final page content as rendered by the view, or any error
message.
context
The template Context instance that was used to render the template that produced the response content.
If the rendered page used multiple templates, then context will be a list of Context objects, in the
order in which they were rendered.
Regardless of the number of templates used during rendering, you can retrieve context values using the []
operator. For example, the context variable name could be retrieved using:
>>> response = client.get(’/foo/’)
>>> response.context[’name’]
’Arthur’
request
The request data that stimulated the response.
status_code
The HTTP status of the response, as an integer. See RFC 2616 for a full list of HTTP status codes.
3.8. Testing in Django 273](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-277-2048.jpg)
![Django Documentation, Release 1.5.1
templates
A list of Template instances used to render the final content, in the order they were rendered. For each
template in the list, use template.name to get the template’s file name, if the template was loaded from
a file. (The name is a string such as ’admin/index.html’.)
You can also use dictionary syntax on the response object to query the value of any settings in the HTTP headers. For
example, you could determine the content type of a response using response[’Content-Type’].
Exceptions If you point the test client at a view that raises an exception, that exception will be visible in the test
case. You can then use a standard try ... except block or assertRaises() to test for exceptions.
The only exceptions that are not visible to the test client are Http404, PermissionDenied and SystemExit.
Django catches these exceptions internally and converts them into the appropriate HTTP response codes. In these
cases, you can check response.status_code in your test.
Persistent state The test client is stateful. If a response returns a cookie, then that cookie will be stored in the test
client and sent with all subsequent get() and post() requests.
Expiration policies for these cookies are not followed. If you want a cookie to expire, either delete it manually or
create a new Client instance (which will effectively delete all cookies).
A test client has two attributes that store persistent state information. You can access these properties as part of a test
condition.
Client.cookies
A Python SimpleCookie object, containing the current values of all the client cookies. See the documentation
of the Cookie module for more.
Client.session
A dictionary-like object containing session information. See the session documentation for full details.
To modify the session and then save it, it must be stored in a variable first (because a new SessionStore is
created every time this property is accessed):
def test_something(self):
session = self.client.session
session[’somekey’] = ’test’
session.save()
Example The following is a simple unit test using the test client:
from django.utils import unittest
from django.test.client import Client
class SimpleTest(unittest.TestCase):
def setUp(self):
# Every test needs a client.
self.client = Client()
def test_details(self):
# Issue a GET request.
response = self.client.get(’/customer/details/’)
# Check that the response is 200 OK.
self.assertEqual(response.status_code, 200)
# Check that the rendered context contains 5 customers.
self.assertEqual(len(response.context[’customers’]), 5)
274 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-278-2048.jpg)
![Django Documentation, Release 1.5.1
LiveServerTestCase New in version 1.4.
class LiveServerTestCase
LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches
a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test
clients other than the Django dummy client such as, for example, the Selenium client, to execute a series of functional
tests inside a browser and simulate a real user’s actions.
By default the live server’s address is ’localhost:8081’ and the full URL can be accessed during the tests with
self.live_server_url. If you’d like to change the default address (in the case, for example, where the 8081
port is already taken) then you may pass a different one to the test command via the --liveserver option, for
example:
./manage.py test --liveserver=localhost:8082
Another way of changing the default server address is by setting the DJANGO_LIVE_TEST_SERVER_ADDRESS
environment variable somewhere in your code (for example, in a custom test runner):
import os
os.environ[’DJANGO_LIVE_TEST_SERVER_ADDRESS’] = ’localhost:8082’
In the case where the tests are run by multiple processes in parallel (for example, in the context of several simulta-neous
continuous integration builds), the processes will compete for the same address, and therefore your tests might
randomly fail with an “Address already in use” error. To avoid this problem, you can pass a comma-separated list of
ports or ranges of ports (at least as many as the number of potential parallel processes). For example:
./manage.py test --liveserver=localhost:8082,8090-8100,9000-9200,7041
Then, during test execution, each new live test server will try every specified port until it finds one that is free and
takes it.
To demonstrate how to use LiveServerTestCase, let’s write a simple Selenium test. First of all, you need to
install the selenium package into your Python path:
pip install selenium
Then, add a LiveServerTestCase-based test to your app’s tests module (for example: myapp/tests.py).
The code for this test may look as follows:
from django.test import LiveServerTestCase
from selenium.webdriver.firefox.webdriver import WebDriver
class MySeleniumTests(LiveServerTestCase):
fixtures = [’user-data.json’]
@classmethod
def setUpClass(cls):
cls.selenium = WebDriver()
super(MySeleniumTests, cls).setUpClass()
@classmethod
def tearDownClass(cls):
cls.selenium.quit()
super(MySeleniumTests, cls).tearDownClass()
def test_login(self):
self.selenium.get(’%s%s’ % (self.live_server_url, ’/login/’))
username_input = self.selenium.find_element_by_name("username")
username_input.send_keys(’myuser’)
278 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-282-2048.jpg)
![Django Documentation, Release 1.5.1
password_input = self.selenium.find_element_by_name("password")
password_input.send_keys(’secret’)
self.selenium.find_element_by_xpath(’//input[@value="Log in"]’).click()
Finally, you may run the test as follows:
./manage.py test myapp.MySeleniumTests.test_login
This example will automatically open Firefox then go to the login page, enter the credentials and press the “Log in”
button. Selenium offers other drivers in case you do not have Firefox installed or wish to use another browser. The
example above is just a tiny fraction of what the Selenium client can do; check out the full reference for more details.
Note: LiveServerTestCase makes use of the staticfiles contrib app so you’ll need to have your project config-ured
accordingly (in particular by setting STATIC_URL).
Note: When using an in-memory SQLite database to run the tests, the same database connection will be shared
by two threads in parallel: the thread in which the live server is run and the thread in which the test case is run.
It’s important to prevent simultaneous database queries via this shared connection by the two threads, as that may
sometimes randomly cause the tests to fail. So you need to ensure that the two threads don’t access the database at the
same time. In particular, this means that in some cases (for example, just after clicking a link or submitting a form),
you might need to check that a response is received by Selenium and that the next page is loaded before proceeding
with further test execution. Do this, for example, by making Selenium wait until the <body> HTML tag is found in
the response (requires Selenium > 2.13):
def test_login(self):
from selenium.webdriver.support.wait import WebDriverWait
timeout = 2
...
self.selenium.find_element_by_xpath(’//input[@value="Log in"]’).click()
# Wait until the response is received
WebDriverWait(self.selenium, timeout).until(
lambda driver: driver.find_element_by_tag_name(’body’))
The tricky thing here is that there’s really no such thing as a “page load,” especially in modern Web apps that generate
HTML dynamically after the server generates the initial document. So, simply checking for the presence of <body>
in the response might not necessarily be appropriate for all use cases. Please refer to the Selenium FAQ and Selenium
documentation for more information.
Test cases features
Default test client
TestCase.client
Every test case in a django.test.TestCase instance has access to an instance of a Django test client. This client
can be accessed as self.client. This client is recreated for each test, so you don’t have to worry about state (such
as cookies) carrying over from one test to another.
This means, instead of instantiating a Client in each test:
from django.utils import unittest
from django.test.client import Client
class SimpleTest(unittest.TestCase):
def test_details(self):
client = Client()
3.8. Testing in Django 279](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-283-2048.jpg)
![Django Documentation, Release 1.5.1
Initial SQL data and testing
Django provides a second way to insert initial data into models – the custom SQL hook. However, this technique
cannot be used to provide initial data for testing purposes. Django’s test framework flushes the contents of the test
database after each test; as a result, any data added using the custom SQL hook will be lost.
Once you’ve created a fixture and placed it in a fixtures directory in one of your INSTALLED_APPS, you can
use it in your unit tests by specifying a fixtures class attribute on your django.test.TestCase subclass:
from django.test import TestCase
from myapp.models import Animal
class AnimalTestCase(TestCase):
fixtures = [’mammals.json’, ’birds’]
def setUp(self):
# Test definitions as before.
call_setup_methods()
def testFluffyAnimals(self):
# A test that uses the fixtures.
call_some_test_code()
Here’s specifically what will happen:
• At the start of each test case, before setUp() is run, Django will flush the database, returning the database to
the state it was in directly after syncdb was called.
• Then, all the named fixtures are installed. In this example, Django will install any JSON fixture named
mammals, followed by any fixture named birds. See the loaddata documentation for more details on
defining and installing fixtures.
This flush/load procedure is repeated for each test in the test case, so you can be certain that the outcome of a test will
not be affected by another test, or by the order of test execution.
URLconf configuration
TestCase.urls
If your application provides views, you may want to include tests that use the test client to exercise those views.
However, an end user is free to deploy the views in your application at any URL of their choosing. This means that
your tests can’t rely upon the fact that your views will be available at a particular URL.
In order to provide a reliable URL space for your test, django.test.TestCase provides the ability to customize
the URLconf configuration for the duration of the execution of a test suite. If your TestCase instance defines an
urls attribute, the TestCase will use the value of that attribute as the ROOT_URLCONF for the duration of that
test.
For example:
from django.test import TestCase
class TestMyViews(TestCase):
urls = ’myapp.test_urls’
def testIndexPageView(self):
# Here you’d test your view using ‘‘Client‘‘.
call_some_test_code()
This test case will use the contents of myapp.test_urls as the URLconf for the duration of the test case.
3.8. Testing in Django 281](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-285-2048.jpg)
![Django Documentation, Release 1.5.1
SimpleTestCase.assertFieldOutput(self, fieldclass, valid, invalid, field_args=None,
field_kwargs=None, empty_value=u’‘)
New in version 1.4. Asserts that a form field behaves correctly with various inputs.
Parameters
• fieldclass – the class of the field to be tested.
• valid – a dictionary mapping valid inputs to their expected cleaned values.
• invalid – a dictionary mapping invalid inputs to one or more raised error messages.
• field_args – the args passed to instantiate the field.
• field_kwargs – the kwargs passed to instantiate the field.
• empty_value – the expected clean output for inputs in EMPTY_VALUES.
For example, the following code tests that an EmailField accepts “a@a.com” as a valid email address, but
rejects “aaa” with a reasonable error message:
self.assertFieldOutput(EmailField, {’a@a.com’: ’a@a.com’}, {’aaa’: [u’Enter a valid email address.’]})
TestCase.assertContains(response, text, count=None, status_code=200, msg_prefix=’‘, html=False)
Asserts that a Response instance produced the given status_code and that text appears in the content
of the response. If count is provided, text must occur exactly count times in the response. New in version
1.4. Set html to True to handle text as HTML. The comparison with the response content will be based
on HTML semantics instead of character-by-character equality. Whitespace is ignored in most cases, attribute
ordering is not significant. See assertHTMLEqual() for more details.
TestCase.assertNotContains(response, text, status_code=200, msg_prefix=’‘, html=False)
Asserts that a Response instance produced the given status_code and that text does not appears in the
content of the response. New in version 1.4. Set html to True to handle text as HTML. The comparison with
the response content will be based on HTML semantics instead of character-by-character equality. Whitespace
is ignored in most cases, attribute ordering is not significant. See assertHTMLEqual() for more details.
TestCase.assertFormError(response, form, field, errors, msg_prefix=’‘)
Asserts that a field on a form raises the provided list of errors when rendered on the form.
form is the name the Form instance was given in the template context.
field is the name of the field on the form to check. If field has a value of None, non-field errors (errors
you can access via form.non_field_errors()) will be checked.
errors is an error string, or a list of error strings, that are expected as a result of form validation.
TestCase.assertTemplateUsed(response, template_name, msg_prefix=’‘)
Asserts that the template with the given name was used in rendering the response.
The name is a string such as ’admin/index.html’. New in version 1.4. You can use this as a context
manager, like this:
with self.assertTemplateUsed(’index.html’):
render_to_string(’index.html’)
with self.assertTemplateUsed(template_name=’index.html’):
render_to_string(’index.html’)
TestCase.assertTemplateNotUsed(response, template_name, msg_prefix=’‘)
Asserts that the template with the given name was not used in rendering the response. New in version 1.4. You
can use this as a context manager in the same way as assertTemplateUsed().
284 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-288-2048.jpg)
![Django Documentation, Release 1.5.1
SimpleTestCase.assertHTMLNotEqual(html1, html2, msg=None)
New in version 1.4. Asserts that the strings html1 and html2 are not equal. The comparison is based on
HTML semantics. See assertHTMLEqual() for details.
html1 and html2 must be valid HTML. An AssertionError will be raised if one of them cannot be
parsed.
SimpleTestCase.assertXMLEqual(xml1, xml2, msg=None)
New in version 1.5. Asserts that the strings xml1 and xml2 are equal. The comparison is based on XML
semantics. Similarily to assertHTMLEqual(), the comparison is made on parsed content, hence only se-mantic
differences are considered, not syntax differences. When unvalid XML is passed in any parameter, an
AssertionError is always raised, even if both string are identical.
SimpleTestCase.assertXMLNotEqual(xml1, xml2, msg=None)
New in version 1.5. Asserts that the strings xml1 and xml2 are not equal. The comparison is based on XML
semantics. See assertXMLEqual() for details.
Email services
If any of your Django views send email using Django’s email functionality, you probably don’t want to send email
each time you run a test using that view. For this reason, Django’s test runner automatically redirects all Django-sent
email to a dummy outbox. This lets you test every aspect of sending email – from the number of messages sent to the
contents of each message – without actually sending the messages.
The test runner accomplishes this by transparently replacing the normal email backend with a testing backend. (Don’t
worry – this has no effect on any other email senders outside of Django, such as your machine’s mail server, if you’re
running one.)
django.core.mail.outbox
During test running, each outgoing email is saved in django.core.mail.outbox. This is a simple list of all
EmailMessage instances that have been sent. The outbox attribute is a special attribute that is created only when
the locmem email backend is used. It doesn’t normally exist as part of the django.core.mail module and you
can’t import it directly. The code below shows how to access this attribute correctly.
Here’s an example test that examines django.core.mail.outbox for length and contents:
from django.core import mail
from django.test import TestCase
class EmailTest(TestCase):
def test_send_email(self):
# Send message.
mail.send_mail(’Subject here’, ’Here is the message.’,
’from@example.com’, [’to@example.com’],
fail_silently=False)
# Test that one message has been sent.
self.assertEqual(len(mail.outbox), 1)
# Verify that the subject of the first message is correct.
self.assertEqual(mail.outbox[0].subject, ’Subject here’)
As noted previously, the test outbox is emptied at the start of every test in a Django TestCase. To empty the outbox
manually, assign the empty list to mail.outbox:
from django.core import mail
286 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-290-2048.jpg)
![Django Documentation, Release 1.5.1
# Empty the test outbox
mail.outbox = []
Skipping tests
The unittest library provides the @skipIf and @skipUnless decorators to allow you to skip tests if you know
ahead of time that those tests are going to fail under certain conditions.
For example, if your test requires a particular optional library in order to succeed, you could decorate the test case
with @skipIf. Then, the test runner will report that the test wasn’t executed and why, instead of failing the test or
omitting the test altogether.
To supplement these test skipping behaviors, Django provides two additional skip decorators. Instead of testing a
generic boolean, these decorators check the capabilities of the database, and skip the test if the database doesn’t
support a specific named feature.
The decorators use a string identifier to describe database features. This string corresponds to attributes of the
database connection features class. See django.db.backends.BaseDatabaseFeatures class for a full list
of database features that can be used as a basis for skipping tests.
skipIfDBFeature(feature_name_string)
Skip the decorated test if the named database feature is supported.
For example, the following test will not be executed if the database supports transactions (e.g., it would not run under
PostgreSQL, but it would under MySQL with MyISAM tables):
class MyTests(TestCase):
@skipIfDBFeature(’supports_transactions’)
def test_transaction_behavior(self):
# ... conditional test code
skipUnlessDBFeature(feature_name_string)
Skip the decorated test if the named database feature is not supported.
For example, the following test will only be executed if the database supports transactions (e.g., it would run under
PostgreSQL, but not under MySQL with MyISAM tables):
class MyTests(TestCase):
@skipUnlessDBFeature(’supports_transactions’)
def test_transaction_behavior(self):
# ... conditional test code
3.8.2 Django and doctests
Doctests use Python’s standard doctest module, which searches your docstrings for statements that resemble a
session of the Python interactive interpreter. A full explanation of how doctest works is out of the scope of this
document; read Python’s official documentation for the details.
What’s a docstring?
A good explanation of docstrings (and some guidelines for using them effectively) can be found in PEP 257:
A docstring is a string literal that occurs as the first statement in a module, function, class, or method
definition. Such a docstring becomes the __doc__ special attribute of that object.
For example, this function has a docstring that describes what it does:
3.8. Testing in Django 287](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-291-2048.jpg)
![Django Documentation, Release 1.5.1
’TEST_MIRROR’: ’default’
# ... plus some other settings
}
}
In this setup, we have two database servers: dbmaster, described by the database alias default, and dbslave
described by the alias slave. As you might expect, dbslave has been configured by the database administrator as
a read slave of dbmaster, so in normal activity, any write to default will appear on slave.
If Django created two independent test databases, this would break any tests that expected replication to occur. How-ever,
the slave database has been configured as a test mirror (using the TEST_MIRROR setting), indicating that
under testing, slave should be treated as a mirror of default.
When the test environment is configured, a test version of slave will not be created. Instead the connection to slave
will be redirected to point at default. As a result, writes to default will appear on slave – but because they are
actually the same database, not because there is data replication between the two databases.
Controlling creation order for test databases
By default, Django will always create the default database first. However, no guarantees are made on the creation
order of any other databases in your test setup.
If your database configuration requires a specific creation order, you can specify the dependencies that exist using the
TEST_DEPENDENCIES setting. Consider the following (simplified) example database configuration:
DATABASES = {
’default’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’]
},
’diamonds’: {
# ... db settings
},
’clubs’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’]
},
’spades’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’,’hearts’]
},
’hearts’: {
# ... db settings
’TEST_DEPENDENCIES’: [’diamonds’,’clubs’]
}
}
Under this configuration, the diamonds database will be created first, as it is the only database alias without de-pendencies.
The default and clubs alias will be created next (although the order of creation of this pair is not
guaranteed); then hearts; and finally spades.
If there are any circular dependencies in the TEST_DEPENDENCIES definition, an ImproperlyConfigured
exception will be raised.
290 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-294-2048.jpg)
![Django Documentation, Release 1.5.1
Returns the result produced by the running the test suite.
DjangoTestSuiteRunner.teardown_databases(old_config, **kwargs)
Destroys the test databases, restoring pre-test conditions.
old_config is a data structure defining the changes in the database configuration that need to be reversed. It
is the return value of the setup_databases() method.
DjangoTestSuiteRunner.teardown_test_environment(**kwargs)
Restores the pre-test environment.
DjangoTestSuiteRunner.suite_result(suite, result, **kwargs)
Computes and returns a return code based on a test suite, and the result from that test suite.
Testing utilities
To assist in the creation of your own test runner, Django provides a number of utility methods in the
django.test.utils module.
setup_test_environment()
Performs any global pre-test setup, such as the installing the instrumentation of the template rendering system
and setting up the dummy email outbox.
teardown_test_environment()
Performs any global post-test teardown, such as removing the black magic hooks into the template system and
restoring normal email services.
The creation module of the database backend (connection.creation) also provides some utilities that can be
useful during testing.
create_test_db([verbosity=1, autoclobber=False ])
Creates a new test database and runs syncdb against it.
verbosity has the same behavior as in run_tests().
autoclobber describes the behavior that will occur if a database with the same name as the test database is
discovered:
•If autoclobber is False, the user will be asked to approve destroying the existing database.
sys.exit is called if the user does not approve.
•If autoclobber is True, the database will be destroyed without consulting the user.
Returns the name of the test database that it created.
create_test_db() has the side effect of modifying the value of NAME in DATABASES to match the name
of the test database.
destroy_test_db(old_database_name[, verbosity=1 ])
Destroys the database whose name is the value of NAME in DATABASES, and sets NAME to the value of
old_database_name.
The verbosity argument has the same behavior as for DjangoTestSuiteRunner.
Integration with coverage.py
Code coverage describes how much source code has been tested. It shows which parts of your code are being exercised
by tests and which are not. It’s an important part of testing applications, so it’s strongly recommended to check the
coverage of your tests.
3.8. Testing in Django 293](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-297-2048.jpg)
![Django Documentation, Release 1.5.1
Django can be easily integrated with coverage.py, a tool for measuring code coverage of Python programs. First,
install coverage.py. Next, run the following from your project folder containing manage.py:
coverage run --source=’.’ manage.py test myapp
This runs your tests and collects coverage data of the executed files in your project. You can see a report of this data
by typing following command:
coverage report
Note that some Django code was executed while running tests, but it is not listed here because of the source flag
passed to the previous command.
For more options like annotated HTML listings detailing missed lines, see the coverage.py docs.
Automated testing is an extremely useful bug-killing tool for the modern Web developer. You can use a collection of
tests – a test suite – to solve, or avoid, a number of problems:
• When you’re writing new code, you can use tests to validate your code works as expected.
• When you’re refactoring or modifying old code, you can use tests to ensure your changes haven’t affected your
application’s behavior unexpectedly.
Testing a Web application is a complex task, because a Web application is made of several layers of logic – from
HTTP-level request handling, to form validation and processing, to template rendering. With Django’s test-execution
framework and assorted utilities, you can simulate requests, insert test data, inspect your application’s output and
generally verify your code is doing what it should be doing.
The best part is, it’s really easy.
3.8.4 Unit tests v. doctests
There are two primary ways to write tests with Django, corresponding to the two test frameworks that ship in the
Python standard library. The two frameworks are:
• Unit tests – tests that are expressed as methods on a Python class that subclasses unittest.TestCase or
Django’s customized TestCase. For example:
import unittest
class MyFuncTestCase(unittest.TestCase):
def testBasic(self):
a = [’larry’, ’curly’, ’moe’]
self.assertEqual(my_func(a, 0), ’larry’)
self.assertEqual(my_func(a, 1), ’curly’)
• Doctests – tests that are embedded in your functions’ docstrings and are written in a way that emulates a session
of the Python interactive interpreter. For example:
def my_func(a_list, idx):
"""
>>> a = [’larry’, ’curly’, ’moe’]
>>> my_func(a, 0)
’larry’
>>> my_func(a, 1)
’curly’
"""
return a_list[idx]
294 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-298-2048.jpg)
![Django Documentation, Release 1.5.1
If you have the Django admin installed, you can also change user’s passwords on the authentication system’s admin
pages.
Django also provides views and forms that may be used to allow users to change their own passwords.
Authenticating Users
authenticate(**credentials)
To authenticate a given username and password, use authenticate(). It takes credentials in the form of
keyword arguments, for the default configuration this is username and password, and it returns a User
object if the password is valid for the given username. If the password is invalid, authenticate() returns
None. Example:
from django.contrib.auth import authenticate
user = authenticate(username=’john’, password=’secret’)
if user is not None:
# the password verified for the user
if user.is_active:
print("User is valid, active and authenticated")
else:
print("The password is valid, but the account has been disabled!")
else:
# the authentication system was unable to verify the username and password
print("The username and password were incorrect.")
Permissions and Authorization
Django comes with a simple permissions system. It provides a way to assign permissions to specific users and groups
of users.
It’s used by the Django admin site, but you’re welcome to use it in your own code.
The Django admin site uses permissions as follows:
• Access to view the “add” form and add an object is limited to users with the “add” permission for that type of
object.
• Access to view the change list, view the “change” form and change an object is limited to users with the “change”
permission for that type of object.
• Access to delete an object is limited to users with the “delete” permission for that type of object.
Permissions can be set not only per type of object, but also per specific object instance. By using the
has_add_permission(), has_change_permission() and has_delete_permission() methods
provided by the ModelAdmin class, it is possible to customize permissions for different object instances of the
same type.
User objects have two many-to-many fields: groups and user_permissions. User objects can access their
related objects in the same way as any other Django model:
myuser.groups = [group_list]
myuser.groups.add(group, group, ...)
myuser.groups.remove(group, group, ...)
myuser.groups.clear()
myuser.user_permissions = [permission_list]
myuser.user_permissions.add(permission, permission, ...)
myuser.user_permissions.remove(permission, permission, ...)
myuser.user_permissions.clear()
3.9. User authentication in Django 297](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-301-2048.jpg)
![Django Documentation, Release 1.5.1
if request.user.is_authenticated():
# Do something for authenticated users.
else:
# Do something for anonymous users.
How to log a user in
If you have an authenticated user you want to attach to the current session - this is done with a login() function.
login()
To log a user in, from a view, use login(). It takes an HttpRequest object and a User object. login()
saves the user’s ID in the session, using Django’s session framework.
Note that any data set during the anonymous session is retained in the session after a user logs in.
This example shows how you might use both authenticate() and login():
from django.contrib.auth import authenticate, login
def my_view(request):
username = request.POST[’username’]
password = request.POST[’password’]
user = authenticate(username=username, password=password)
if user is not None:
if user.is_active:
login(request, user)
# Redirect to a success page.
else:
# Return a ’disabled account’ error message
else:
# Return an ’invalid login’ error message.
Calling authenticate() first
When you’re manually logging a user in, you must call authenticate() before you call login().
authenticate() sets an attribute on the User noting which authentication backend successfully authenticated
that user (see the backends documentation for details), and this information is needed later during the login process.
An error will be raise if you try to login a user object retrieved from the database directly.
How to log a user out
logout()
To log out a user who has been logged in via django.contrib.auth.login(), use
django.contrib.auth.logout() within your view. It takes an HttpRequest object and has
no return value. Example:
from django.contrib.auth import logout
def logout_view(request):
logout(request)
# Redirect to a success page.
Note that logout() doesn’t throw any errors if the user wasn’t logged in.
3.9. User authentication in Django 299](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-303-2048.jpg)
![Django Documentation, Release 1.5.1
When you call logout(), the session data for the current request is completely cleaned out. All existing data
is removed. This is to prevent another person from using the same Web browser to log in and have access to
the previous user’s session data. If you want to put anything into the session that will be available to the user
immediately after logging out, do that after calling django.contrib.auth.logout().
Limiting access to logged-in users
The raw way The simple, raw way to limit access to pages is to check
request.user.is_authenticated() and either redirect to a login page:
from django.shortcuts import redirect
def my_view(request):
if not request.user.is_authenticated():
return redirect(’/login/?next=%s’ % request.path)
# ...
...or display an error message:
from django.shortcuts import render
def my_view(request):
if not request.user.is_authenticated():
return render(request, ’myapp/login_error.html’)
# ...
The login_required decorator
login_required([redirect_field_name=REDIRECT_FIELD_NAME, login_url=None ])
As a shortcut, you can use the convenient login_required() decorator:
from django.contrib.auth.decorators import login_required
@login_required
def my_view(request):
...
login_required() does the following:
•If the user isn’t logged in, redirect to settings.LOGIN_URL, passing the current absolute path in the
query string. Example: /accounts/login/?next=/polls/3/.
•If the user is logged in, execute the view normally. The view code is free to assume the user is logged in.
By default, the path that the user should be redirected to upon successful authentication is stored in a
query string parameter called "next". If you would prefer to use a different name for this parameter,
login_required() takes an optional redirect_field_name parameter:
from django.contrib.auth.decorators import login_required
@login_required(redirect_field_name=’my_redirect_field’)
def my_view(request):
...
Note that if you provide a value to redirect_field_name, you will most likely need to customize your
login template as well, since the template context variable which stores the redirect path will use the value of
redirect_field_name as its key rather than "next" (the default).
login_required() also takes an optional login_url parameter. Example:
300 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-304-2048.jpg)
![Django Documentation, Release 1.5.1
from django.contrib.auth.decorators import login_required
@login_required(login_url=’/accounts/login/’)
def my_view(request):
...
Note that if you don’t specify the login_url parameter, you’ll need to ensure that the
settings.LOGIN_URL and your login view are properly associated. For example, using the defaults, add
the following line to your URLconf:
(r’^accounts/login/$’, ’django.contrib.auth.views.login’),
Changed in version 1.5. The settings.LOGIN_URL also accepts view function names and named URL
patterns. This allows you to freely remap your login view within your URLconf without having to update the
setting.
Note: The login_required decorator does NOT check the is_active flag on a user.
Limiting access to logged-in users that pass a test To limit access based on certain permissions or some other test,
you’d do essentially the same thing as described in the previous section.
The simple way is to run your test on request.user in the view directly. For example, this view checks to make
sure the user has an email in the desired domain:
def my_view(request):
if not ’@example.com’ in request.user.email:
return HttpResponse("You can’t vote in this poll.")
# ...
user_passes_test(func[, login_url=None ])
As a shortcut, you can use the convenient user_passes_test decorator:
from django.contrib.auth.decorators import user_passes_test
def email_check(user):
return ’@example.com’ in user.email
@user_passes_test(email_check)
def my_view(request):
...
user_passes_test() takes a required argument: a callable that takes a User object and returns True if
the user is allowed to view the page. Note that user_passes_test() does not automatically check that the
User is not anonymous.
user_passes_test() takes an optional login_url argument, which lets you specify the URL for your
login page (settings.LOGIN_URL by default).
For example:
@user_passes_test(email_check, login_url=’/login/’)
def my_view(request):
...
The permission_required decorator
3.9. User authentication in Django 301](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-305-2048.jpg)
![Django Documentation, Release 1.5.1
permission_required([login_url=None, raise_exception=False ])
It’s a relatively common task to check whether a user has a particular permission. For that reason, Django
provides a shortcut for that case: the permission_required() decorator.:
from django.contrib.auth.decorators import permission_required
@permission_required(’polls.can_vote’)
def my_view(request):
...
As for the has_perm() method, permission names take the form "<app label>.<permission
codename>" (i.e. polls.can_vote for a permission on a model in the polls application).
Note that permission_required() also takes an optional login_url parameter. Example:
from django.contrib.auth.decorators import permission_required
@permission_required(’polls.can_vote’, login_url=’/loginpage/’)
def my_view(request):
...
As in the login_required() decorator, login_url defaults to settings.LOGIN_URL. Changed in
version 1.4. Added raise_exception parameter. If given, the decorator will raise PermissionDenied,
prompting the 403 (HTTP Forbidden) view instead of redirecting to the login page.
Applying permissions to generic views To apply a permission to a class-based generic view, decorate the
View.dispatch method on the class. See Decorating the class for details.
Authentication Views
Django provides several views that you can use for handling login, logout, and password management. These make
use of the stock auth forms but you can pass in your own forms as well.
Django provides no default template for the authentication views - however the template context is documented for
each view below. New in version 1.4. The built-in views all return a TemplateResponse instance, which allows
you to easily customize the response data before rendering. For more details, see the TemplateResponse documenta-tion.
Most built-in authentication views provide a URL name for easier reference. See the URL documentation for details
on using named URL patterns.
login(request[, template_name, redirect_field_name, authentication_form ])
URL name: login
See the URL documentation for details on using named URL patterns.
Here’s what django.contrib.auth.views.login does:
•If called via GET, it displays a login form that POSTs to the same URL. More on this in a bit.
•If called via POST with user submitted credentials, it tries to log the user in. If login is suc-cessful,
the view redirects to the URL specified in next. If next isn’t provided, it redirects to
settings.LOGIN_REDIRECT_URL (which defaults to /accounts/profile/). If login isn’t suc-cessful,
it redisplays the login form.
It’s your responsibility to provide the html for the login template , called registration/login.html by
default. This template gets passed four template context variables:
•form: A Form object representing the AuthenticationForm.
302 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-306-2048.jpg)
![Django Documentation, Release 1.5.1
•next: The URL to redirect to after successful login. This may contain a query string, too.
•site: The current Site, according to the SITE_ID setting. If you don’t have the site framework
installed, this will be set to an instance of RequestSite, which derives the site name and domain from
the current HttpRequest.
•site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to
the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework.
If you’d prefer not to call the template registration/login.html, you can pass the template_name
parameter via the extra arguments to the view in your URLconf. For example, this URLconf line would use
myapp/login.html instead:
(r’^accounts/login/$’, ’django.contrib.auth.views.login’, {’template_name’: ’myapp/login.html’}),
You can also specify the name of the GET field which contains the URL to redirect to after login by passing
redirect_field_name to the view. By default, the field is called next.
Here’s a sample registration/login.html template you can use as a starting point. It assumes you
have a base.html template that defines a content block:
{% extends "base.html" %}
{% block content %}
{% if form.errors %}
<p>Your username and password didn’t match. Please try again.</p>
{% endif %}
<form method="post" action="{% url ’django.contrib.auth.views.login’ %}">
{% csrf_token %}
<table>
<tr>
<td>{{ form.username.label_tag }}</td>
<td>{{ form.username }}</td>
</tr>
<tr>
<td>{{ form.password.label_tag }}</td>
<td>{{ form.password }}</td>
</tr>
</table>
<input type="submit" value="login" />
<input type="hidden" name="next" value="{{ next }}" />
</form>
{% endblock %}
If you have customized authentication (see Customizing Authentication) you can pass a custom authentication
form to the login view via the authentication_form parameter. This form must accept a request
keyword argument in its __init__ method, and provide a get_user method which returns the authenticated
user object (this method is only ever called after successful form validation).
logout(request[, next_page, template_name, redirect_field_name ])
Logs a user out.
URL name: logout
Optional arguments:
•next_page: The URL to redirect to after logout.
3.9. User authentication in Django 303](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-307-2048.jpg)
![Django Documentation, Release 1.5.1
•template_name: The full name of a template to display after logging the user out. Defaults to
registration/logged_out.html if no argument is supplied.
•redirect_field_name: The name of a GET field containing the URL to redirect to after log out.
Overrides next_page if the given GET parameter is passed.
Template context:
•title: The string “Logged out”, localized.
•site: The current Site, according to the SITE_ID setting. If you don’t have the site framework
installed, this will be set to an instance of RequestSite, which derives the site name and domain from
the current HttpRequest.
•site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to
the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework.
logout_then_login(request[, login_url ])
Logs a user out, then redirects to the login page.
URL name: No default URL provided
Optional arguments:
•login_url: The URL of the login page to redirect to. Defaults to settings.LOGIN_URL if not
supplied.
password_change(request[, template_name, post_change_redirect, password_change_form ])
Allows a user to change their password.
URL name: password_change
Optional arguments:
•template_name: The full name of a template to use for displaying the password change form. Defaults
to registration/password_change_form.html if not supplied.
•post_change_redirect: The URL to redirect to after a successful password change.
•password_change_form: A custom “change password” form which must accept a user key-word
argument. The form is responsible for actually changing the user’s password. Defaults to
PasswordChangeForm.
Template context:
•form: The password change form (see password_change_form above).
password_change_done(request[, template_name ])
The page shown after a user has changed their password.
URL name: password_change_done
Optional arguments:
•template_name: The full name of a template to use. Defaults to
registration/password_change_done.html if not supplied.
password_reset(request[, is_admin_site, template_name, email_template_name, password_reset_form,
token_generator, post_reset_redirect, from_email ])
Allows a user to reset their password by generating a one-time use link that can be used to reset the password, and
sending that link to the user’s registered email address. Changed in version 1.4: Users flagged with an unusable
password (see set_unusable_password() will not be able to request a password reset to prevent misuse
when using an external authentication source like LDAP. URL name: password_reset
Optional arguments:
304 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-308-2048.jpg)
![Django Documentation, Release 1.5.1
•template_name: The full name of a template to use for displaying the password reset form. Defaults
to registration/password_reset_form.html if not supplied.
•email_template_name: The full name of a template to use for generating the email with the reset
password link. Defaults to registration/password_reset_email.html if not supplied.
•subject_template_name: The full name of a template to use for the subject of the email with the
reset password link. Defaults to registration/password_reset_subject.txt if not supplied.
New in version 1.4.
•password_reset_form: Form that will be used to get the email of the user to reset the password for.
Defaults to PasswordResetForm.
•token_generator: Instance of the class to check the one time link.
This will default to default_token_generator, it’s an instance of
django.contrib.auth.tokens.PasswordResetTokenGenerator.
•post_reset_redirect: The URL to redirect to after a successful password reset request.
•from_email: A valid email address. By default Django uses the DEFAULT_FROM_EMAIL.
Template context:
•form: The form (see password_reset_form above) for resetting the user’s password.
Email template context:
•email: An alias for user.email
•user: The current User, according to the email form field. Only active users are able to reset their
passwords (User.is_active is True).
•site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to
the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework.
•domain: An alias for site.domain. If you don’t have the site framework installed, this will be set to
the value of request.get_host().
•protocol: http or https
•uid: The user’s id encoded in base 36.
•token: Token to check that the reset link is valid.
Sample registration/password_reset_email.html (email body template):
Someone asked for password reset for email {{ email }}. Follow the link below:
{{ protocol}}://{{ domain }}{% url ’password_reset_confirm’ uidb36=uid token=token %}
The same template context is used for subject template. Subject must be single line plain text string.
password_reset_done(request[, template_name ])
The page shown after a user has been emailed a link to reset their password. This view is called by default if the
password_reset() view doesn’t have an explicit post_reset_redirect URL set.
URL name: password_reset_done
Optional arguments:
•template_name: The full name of a template to use. Defaults to
registration/password_reset_done.html if not supplied.
password_reset_confirm(request[, uidb36, token, template_name, token_generator,
set_password_form, post_reset_redirect ])
Presents a form for entering a new password.
3.9. User authentication in Django 305](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-309-2048.jpg)
![Django Documentation, Release 1.5.1
URL name: password_reset_confirm
Optional arguments:
•uidb36: The user’s id encoded in base 36. Defaults to None.
•token: Token to check that the password is valid. Defaults to None.
•template_name: The full name of a template to display the confirm password view. Default value is
registration/password_reset_confirm.html.
•token_generator: Instance of the class to check the password.
This will default to default_token_generator, it’s an instance of
django.contrib.auth.tokens.PasswordResetTokenGenerator.
•set_password_form: Form that will be used to set the password. Defaults to SetPasswordForm
•post_reset_redirect: URL to redirect after the password reset done. Defaults to None.
Template context:
•form: The form (see set_password_form above) for setting the new user’s password.
•validlink: Boolean, True if the link (combination of uidb36 and token) is valid or unused yet.
password_reset_complete(request[, template_name ])
Presents a view which informs the user that the password has been successfully changed.
URL name: password_reset_complete
Optional arguments:
•template_name: The full name of a template to display the view. Defaults to
registration/password_reset_complete.html.
Helper functions
redirect_to_login(next[, login_url, redirect_field_name ])
Redirects to the login page, and then back to another URL after a successful login.
Required arguments:
•next: The URL to redirect to after a successful login.
Optional arguments:
•login_url: The URL of the login page to redirect to. Defaults to settings.LOGIN_URL if not
supplied.
•redirect_field_name: The name of a GET field containing the URL to redirect to after log out.
Overrides next if the given GET parameter is passed.
Built-in forms
If you don’t want to use the built-in views, but want the convenience of not having to write forms for this functionality,
the authentication system provides several built-in forms located in django.contrib.auth.forms:
Note: The built-in authentication forms make certain assumptions about the user model that they are working with.
If you’re using a custom User model, it may be necessary to define your own forms for the authentication system. For
more information, refer to the documentation about using the built-in authentication forms with custom user models.
306 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-310-2048.jpg)
![Django Documentation, Release 1.5.1
3.9.2 Password management in Django
Password management is something that should generally not be reinvented unnecessarily, and Django endeavors to
provide a secure and flexible set of tools for managing user passwords. This document describes how Django stores
passwords, how the storage hashing can be configured, and some utilities to work with hashed passwords.
How Django stores passwords
New in version 1.4: Django 1.4 introduces a new flexible password storage system and uses PBKDF2 by default.
Previous versions of Django used SHA1, and other algorithms couldn’t be chosen. The password attribute of a
User object is a string in this format:
<algorithm>$<iterations>$<salt>$<hash>
Those are the components used for storing a User’s password, separated by the dollar-sign character and consist of: the
hashing algorithm, the number of algorithm iterations (work factor), the random salt, and the resulting password hash.
The algorithm is one of a number of one-way hashing or password storage algorithms Django can use; see below.
Iterations describe the number of times the algorithm is run over the hash. Salt is the random seed used and the hash
is the result of the one-way function.
By default, Django uses the PBKDF2 algorithm with a SHA256 hash, a password stretching mechanism recommended
by NIST. This should be sufficient for most users: it’s quite secure, requiring massive amounts of computing time to
break.
However, depending on your requirements, you may choose a different algorithm, or even use a custom algorithm to
match your specific security situation. Again, most users shouldn’t need to do this – if you’re not sure, you probably
don’t. If you do, please read on:
Django chooses the algorithm to use by consulting the PASSWORD_HASHERS setting. This is a list
of hashing algorithm classes that this Django installation supports. The first entry in this list (that is,
settings.PASSWORD_HASHERS[0]) will be used to store passwords, and all the other entries are valid hash-ers
that can be used to check existing passwords. This means that if you want to use a different algorithm, you’ll need
to modify PASSWORD_HASHERS to list your preferred algorithm first in the list.
The default for PASSWORD_HASHERS is:
PASSWORD_HASHERS = (
’django.contrib.auth.hashers.PBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’,
’django.contrib.auth.hashers.BCryptPasswordHasher’,
’django.contrib.auth.hashers.SHA1PasswordHasher’,
’django.contrib.auth.hashers.MD5PasswordHasher’,
’django.contrib.auth.hashers.CryptPasswordHasher’,
)
This means that Django will use PBKDF2 to store all passwords, but will support checking passwords stored with
PBKDF2SHA1, bcrypt, SHA1, etc. The next few sections describe a couple of common ways advanced users may
want to modify this setting.
Using bcrypt with Django
Bcrypt is a popular password storage algorithm that’s specifically designed for long-term password storage. It’s not
the default used by Django since it requires the use of third-party libraries, but since many people may want to use it
Django supports bcrypt with minimal effort.
To use Bcrypt as your default storage algorithm, do the following:
3.9. User authentication in Django 309](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-313-2048.jpg)
![Django Documentation, Release 1.5.1
1. Install the py-bcrypt library (probably by running sudo pip install py-bcrypt, or downloading the
library and installing it with python setup.py install).
2. Modify PASSWORD_HASHERS to list BCryptPasswordHasher first. That is, in your settings file, you’d
put:
PASSWORD_HASHERS = (
’django.contrib.auth.hashers.BCryptPasswordHasher’,
’django.contrib.auth.hashers.PBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’,
’django.contrib.auth.hashers.SHA1PasswordHasher’,
’django.contrib.auth.hashers.MD5PasswordHasher’,
’django.contrib.auth.hashers.CryptPasswordHasher’,
)
(You need to keep the other entries in this list, or else Django won’t be able to upgrade passwords; see below).
That’s it – now your Django install will use Bcrypt as the default storage algorithm.
Password truncation with BCryptPasswordHasher
The designers of bcrypt truncate all passwords at 72 characters which means that
bcrypt(password_with_100_chars) == bcrypt(password_with_100_chars[:72]).
BCryptPasswordHasher does not have any special handling and thus is also subject to this hidden pass-word
length limit. The practical ramification of this truncation is pretty marginal as the average user does not have a
password greater than 72 characters in length and even being truncated at 72 the compute powered required to brute
force bcrypt in any useful amount of time is still astronomical.
Other bcrypt implementations
There are several other implementations that allow bcrypt to be used with Django. Django’s
bcrypt support is NOT directly compatible with these. To upgrade, you will need to modify the
hashes in your database to be in the form bcrypt$(raw bcrypt output). For example:
bcrypt$$2a$12$NT0I31Sa7ihGEWpka9ASYrEFkhuTNeBQ2xfZskIiiJeyFXhRgS.Sy.
Increasing the work factor
The PBKDF2 and bcrypt algorithms use a number of iterations or rounds of hashing. This deliberately slows down
attackers, making attacks against hashed passwords harder. However, as computing power increases, the number of
iterations needs to be increased. We’ve chosen a reasonable default (and will increase it with each release of Django),
but you may wish to tune it up or down, depending on your security needs and available processing power. To do so,
you’ll subclass the appropriate algorithm and override the iterations parameters. For example, to increase the
number of iterations used by the default PBKDF2 algorithm:
1. Create a subclass of django.contrib.auth.hashers.PBKDF2PasswordHasher:
from django.contrib.auth.hashers import PBKDF2PasswordHasher
class MyPBKDF2PasswordHasher(PBKDF2PasswordHasher):
"""
A subclass of PBKDF2PasswordHasher that uses 100 times more iterations.
"""
iterations = PBKDF2PasswordHasher.iterations * 100
Save this somewhere in your project. For example, you might put this in a file like myproject/hashers.py.
310 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-314-2048.jpg)
![Django Documentation, Release 1.5.1
2. Add your new hasher as the first entry in PASSWORD_HASHERS:
PASSWORD_HASHERS = (
’myproject.hashers.MyPBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2PasswordHasher’,
’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’,
’django.contrib.auth.hashers.BCryptPasswordHasher’,
’django.contrib.auth.hashers.SHA1PasswordHasher’,
’django.contrib.auth.hashers.MD5PasswordHasher’,
’django.contrib.auth.hashers.CryptPasswordHasher’,
)
That’s it – now your Django install will use more iterations when it stores passwords using PBKDF2.
Password upgrading
When users log in, if their passwords are stored with anything other than the preferred algorithm, Django will auto-matically
upgrade the algorithm to the preferred one. This means that old installs of Django will get automatically
more secure as users log in, and it also means that you can switch to new (and better) storage algorithms as they get
invented.
However, Django can only upgrade passwords that use algorithms mentioned in PASSWORD_HASHERS, so as you
upgrade to new systems you should make sure never to remove entries from this list. If you do, users using un-mentioned
algorithms won’t be able to upgrade.
Manually managing a user’s password
New in version 1.4: The django.contrib.auth.hashers module provides a set of functions to create and
validate hashed password. You can use them independently from the User model.
check_password(password, encoded)
New in version 1.4. If you’d like to manually authenticate a user by comparing a plain-text password to the
hashed password in the database, use the convenience function check_password(). It takes two arguments:
the plain-text password to check, and the full value of a user’s password field in the database to check against,
and returns True if they match, False otherwise.
make_password(password[, salt, hashers ])
New in version 1.4. Creates a hashed password in the format used by this application. It takes one mandatory
argument: the password in plain-text. Optionally, you can provide a salt and a hashing algorithm to use, if you
don’t want to use the defaults (first entry of PASSWORD_HASHERS setting). Currently supported algorithms are:
’pbkdf2_sha256’, ’pbkdf2_sha1’, ’bcrypt’ (see Using bcrypt with Django), ’sha1’, ’md5’,
’unsalted_md5’ (only for backward compatibility) and ’crypt’ if you have the crypt library installed.
If the password argument is None, an unusable password is returned (a one that will be never accepted by
check_password()).
is_password_usable(encoded_password)
New in version 1.4. Checks if the given string is a hashed password that has a chance of being verified against
check_password().
3.9.3 Customizing authentication in Django
The authentication that comes with Django is good enough for most common cases, but you may have needs not met
by the out-of-the-box defaults. To customize authentication to your projects needs involves understanding what points
of the provided system are extendible or replaceable. This document provides details about how the auth system can
be customized.
3.9. User authentication in Django 311](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-315-2048.jpg)
![Django Documentation, Release 1.5.1
The easiest way to construct a compliant custom User model is to inherit from AbstractBaseUser.
AbstractBaseUser provides the core implementation of a User model, including hashed passwords and tok-enized
password resets. You must then provide some key implementation details:
class models.CustomUser
USERNAME_FIELD
A string describing the name of the field on the User model that is used as the unique identifier. This will
usually be a username of some kind, but it can also be an email address, or any other unique identifier. The
field must be unique (i.e., have unique=True set in its definition).
In the following example, the field identifier is used as the identifying field:
class MyUser(AbstractBaseUser):
identifier = models.CharField(max_length=40, unique=True, db_index=True)
...
USERNAME_FIELD = ’identifier’
REQUIRED_FIELDS
A list of the field names that must be provided when creating a user via the createsuperuser man-agement
command. The user will be prompted to supply a value for each of these fields. It should include
any field for which blank is False or undefined, but may include additional fields you want prompted
for when a user is created interactively. However, it will not work for ForeignKey fields.
For example, here is the partial definition for a User model that defines two required fields - a date of
birth and height:
class MyUser(AbstractBaseUser):
...
date_of_birth = models.DateField()
height = models.FloatField()
...
REQUIRED_FIELDS = [’date_of_birth’, ’height’]
Note: REQUIRED_FIELDS must contain all required fields on your User model, but should not contain
the USERNAME_FIELD.
is_active
A boolean attribute that indicates whether the user is considered “active”. This attribute is provided as
an attribute on AbstractBaseUser defaulting to True. How you choose to implement it will de-pend
on the details of your chosen auth backends. See the documentation of the attribute on the
builtin user model for details.
get_full_name()
A longer formal identifier for the user. A common interpretation would be the full name name of the user,
but it can be any string that identifies the user.
get_short_name()
A short, informal identifier for the user. A common interpretation would be the first name of the user,
but it can be any string that identifies the user in an informal way. It may also return the same value as
django.contrib.auth.models.User.get_full_name().
The following methods are available on any subclass of AbstractBaseUser:
class models.AbstractBaseUser
318 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-322-2048.jpg)
![Django Documentation, Release 1.5.1
Creates and saves a superuser with the given email, date of
birth and password.
"""
user = self.create_user(email,
password=password,
date_of_birth=date_of_birth
)
user.is_admin = True
user.save(using=self._db)
return user
class MyUser(AbstractBaseUser):
email = models.EmailField(
verbose_name=’email address’,
max_length=255,
unique=True,
db_index=True,
)
date_of_birth = models.DateField()
is_active = models.BooleanField(default=True)
is_admin = models.BooleanField(default=False)
objects = MyUserManager()
USERNAME_FIELD = ’email’
REQUIRED_FIELDS = [’date_of_birth’]
def get_full_name(self):
# The user is identified by their email address
return self.email
def get_short_name(self):
# The user is identified by their email address
return self.email
def __unicode__(self):
return self.email
def has_perm(self, perm, obj=None):
"Does the user have a specific permission?"
# Simplest possible answer: Yes, always
return True
def has_module_perms(self, app_label):
"Does the user have permissions to view the app ‘app_label‘?"
# Simplest possible answer: Yes, always
return True
@property
def is_staff(self):
"Is the user a member of staff?"
# Simplest possible answer: All admins are staff
return self.is_admin
Then, to register this custom User model with Django’s admin, the following code would be required in the app’s
admin.py file:
324 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-328-2048.jpg)
![Django Documentation, Release 1.5.1
from django import forms
from django.contrib import admin
from django.contrib.auth.models import Group
from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.forms import ReadOnlyPasswordHashField
from customauth.models import MyUser
class UserCreationForm(forms.ModelForm):
"""A form for creating new users. Includes all the required
fields, plus a repeated password."""
password1 = forms.CharField(label=’Password’, widget=forms.PasswordInput)
password2 = forms.CharField(label=’Password confirmation’, widget=forms.PasswordInput)
class Meta:
model = MyUser
fields = (’email’, ’date_of_birth’)
def clean_password2(self):
# Check that the two password entries match
password1 = self.cleaned_data.get("password1")
password2 = self.cleaned_data.get("password2")
if password1 and password2 and password1 != password2:
raise forms.ValidationError("Passwords don’t match")
return password2
def save(self, commit=True):
# Save the provided password in hashed format
user = super(UserCreationForm, self).save(commit=False)
user.set_password(self.cleaned_data["password1"])
if commit:
user.save()
return user
class UserChangeForm(forms.ModelForm):
"""A form for updating users. Includes all the fields on
the user, but replaces the password field with admin’s
password hash display field.
"""
password = ReadOnlyPasswordHashField()
class Meta:
model = MyUser
def clean_password(self):
# Regardless of what the user provides, return the initial value.
# This is done here, rather than on the field, because the
# field does not have access to the initial value
return self.initial["password"]
class MyUserAdmin(UserAdmin):
# The forms to add and change user instances
form = UserChangeForm
add_form = UserCreationForm
3.9. User authentication in Django 325](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-329-2048.jpg)
![Django Documentation, Release 1.5.1
CACHES = {
’default’: {
’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’,
’LOCATION’: ’unix:/tmp/memcached.sock’,
}
}
One excellent feature of Memcached is its ability to share cache over multiple servers. This means you can run
Memcached daemons on multiple machines, and the program will treat the group of machines as a single cache,
without the need to duplicate cache values on each machine. To take advantage of this feature, include all server
addresses in LOCATION, either separated by semicolons or as a list.
In this example, the cache is shared over Memcached instances running on IP address 172.19.26.240 and
172.19.26.242, both on port 11211:
CACHES = {
’default’: {
’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’,
’LOCATION’: [
’172.19.26.240:11211’,
’172.19.26.242:11211’,
]
}
}
In the following example, the cache is shared over Memcached instances running on the IP addresses 172.19.26.240
(port 11211), 172.19.26.242 (port 11212), and 172.19.26.244 (port 11213):
CACHES = {
’default’: {
’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’,
’LOCATION’: [
’172.19.26.240:11211’,
’172.19.26.242:11212’,
’172.19.26.244:11213’,
]
}
}
A final point about Memcached is that memory-based caching has one disadvantage: Because the cached data is stored
in memory, the data will be lost if your server crashes. Clearly, memory isn’t intended for permanent data storage, so
don’t rely on memory-based caching as your only data storage. Without a doubt, none of the Django caching backends
should be used for permanent storage – they’re all intended to be solutions for caching, not storage – but we point this
out here because memory-based caching is particularly temporary.
Database caching
To use a database table as your cache backend, first create a cache table in your database by running this command:
python manage.py createcachetable [cache_table_name]
...where [cache_table_name] is the name of the database table to create. (This name can be whatever you want,
as long as it’s a valid table name that’s not already being used in your database.) This command creates a single table
in your database that is in the proper format that Django’s database-cache system expects.
Once you’ve created that database table, set your BACKEND setting to
"django.core.cache.backends.db.DatabaseCache", and LOCATION to tablename – the name of
the database table. In this example, the cache table’s name is my_cache_table:
3.10. Django’s cache framework 329](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-333-2048.jpg)
![Django Documentation, Release 1.5.1
>>> cache.set(’my_key’, ’hello, world!’, 30)
>>> cache.get(’my_key’)
’hello, world!’
The timeout argument is optional and defaults to the timeout argument of the appropriate backend in the CACHES
setting (explained above). It’s the number of seconds the value should be stored in the cache.
If the object doesn’t exist in the cache, cache.get() returns None:
# Wait 30 seconds for ’my_key’ to expire...
>>> cache.get(’my_key’)
None
We advise against storing the literal value None in the cache, because you won’t be able to distinguish between your
stored None value and a cache miss signified by a return value of None.
cache.get() can take a default argument. This specifies which value to return if the object doesn’t exist in the
cache:
>>> cache.get(’my_key’, ’has expired’)
’has expired’
To add a key only if it doesn’t already exist, use the add() method. It takes the same parameters as set(), but it
will not attempt to update the cache if the key specified is already present:
>>> cache.set(’add_key’, ’Initial value’)
>>> cache.add(’add_key’, ’New value’)
>>> cache.get(’add_key’)
’Initial value’
If you need to know whether add() stored a value in the cache, you can check the return value. It will return True
if the value was stored, False otherwise.
There’s also a get_many() interface that only hits the cache once. get_many() returns a dictionary with all the
keys you asked for that actually exist in the cache (and haven’t expired):
>>> cache.set(’a’, 1)
>>> cache.set(’b’, 2)
>>> cache.set(’c’, 3)
>>> cache.get_many([’a’, ’b’, ’c’])
{’a’: 1, ’b’: 2, ’c’: 3}
To set multiple values more efficiently, use set_many() to pass a dictionary of key-value pairs:
>>> cache.set_many({’a’: 1, ’b’: 2, ’c’: 3})
>>> cache.get_many([’a’, ’b’, ’c’])
{’a’: 1, ’b’: 2, ’c’: 3}
Like cache.set(), set_many() takes an optional timeout parameter.
You can delete keys explicitly with delete(). This is an easy way of clearing the cache for a particular object:
>>> cache.delete(’a’)
If you want to clear a bunch of keys at once, delete_many() can take a list of keys to be cleared:
>>> cache.delete_many([’a’, ’b’, ’c’])
Finally, if you want to delete all the keys in the cache, use cache.clear(). Be careful with this; clear() will
remove everything from the cache, not just the keys set by your application.
3.10. Django’s cache framework 337](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-341-2048.jpg)
![Django Documentation, Release 1.5.1
>>> cache.get(’my_key’, version=2)
’hello world!’
The version of a specific key can be incremented and decremented using the incr_version() and
decr_version() methods. This enables specific keys to be bumped to a new version, leaving other keys unaf-fected.
Continuing our previous example:
# Increment the version of ’my_key’
>>> cache.incr_version(’my_key’)
# The default version still isn’t available
>>> cache.get(’my_key’)
None
# Version 2 isn’t available, either
>>> cache.get(’my_key’, version=2)
None
# But version 3 *is* available
>>> cache.get(’my_key’, version=3)
’hello world!’
Cache key transformation
As described in the previous two sections, the cache key provided by a user is not used verbatim – it is combined with
the cache prefix and key version to provide a final cache key. By default, the three parts are joined using colons to
produce a final string:
def make_key(key, key_prefix, version):
return ’:’.join([key_prefix, str(version), key])
If you want to combine the parts in different ways, or apply other processing to the final key (e.g., taking a hash digest
of the key parts), you can provide a custom key function.
The KEY_FUNCTION cache setting specifies a dotted-path to a function matching the prototype of make_key()
above. If provided, this custom key function will be used instead of the default key combining function.
Cache key warnings
Memcached, the most commonly-used production cache backend, does not allow cache keys longer than 250 char-acters
or containing whitespace or control characters, and using such keys will cause an exception. To encour-age
cache-portable code and minimize unpleasant surprises, the other built-in cache backends issue a warning
(django.core.cache.backends.base.CacheKeyWarning) if a key is used that would cause an error on
memcached.
If you are using a production backend that can accept a wider range of keys (a custom backend, or one of
the non-memcached built-in backends), and want to use this wider range without warnings, you can silence
CacheKeyWarning with this code in the management module of one of your INSTALLED_APPS:
import warnings
from django.core.cache import CacheKeyWarning
warnings.simplefilter("ignore", CacheKeyWarning)
If you want to instead provide custom key validation logic for one of the built-in backends, you can subclass it, override
just the validate_key method, and follow the instructions for using a custom cache backend. For instance, to do
this for the locmem backend, put this code in a module:
3.10. Django’s cache framework 339](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-343-2048.jpg)
![Django Documentation, Release 1.5.1
from django.views.decorators.vary import vary_on_headers
@vary_on_headers(’User-Agent’)
def my_view(request):
# ...
In this case, a caching mechanism (such as Django’s own cache middleware) will cache a separate version of the page
for each unique user-agent.
The advantage to using the vary_on_headers decorator rather than manually setting the Vary header (using
something like response[’Vary’] = ’user-agent’) is that the decorator adds to the Vary header (which
may already exist), rather than setting it from scratch and potentially overriding anything that was already in there.
You can pass multiple headers to vary_on_headers():
@vary_on_headers(’User-Agent’, ’Cookie’)
def my_view(request):
# ...
This tells upstream caches to vary on both, which means each combination of user-agent and cookie will get its own
cache value. For example, a request with the user-agent Mozilla and the cookie value foo=bar will be considered
different from a request with the user-agent Mozilla and the cookie value foo=ham.
Because varying on cookie is so common, there’s a vary_on_cookie decorator. These two views are equivalent:
@vary_on_cookie
def my_view(request):
# ...
@vary_on_headers(’Cookie’)
def my_view(request):
# ...
The headers you pass to vary_on_headers are not case sensitive; "User-Agent" is the same thing as
"user-agent".
You can also use a helper function, django.utils.cache.patch_vary_headers, directly. This function
sets, or adds to, the Vary header. For example:
from django.utils.cache import patch_vary_headers
def my_view(request):
# ...
response = render_to_response(’template_name’, context)
patch_vary_headers(response, [’Cookie’])
return response
patch_vary_headers takes an HttpResponse instance as its first argument and a list/tuple of case-insensitive
header names as its second argument.
For more on Vary headers, see the official Vary spec.
3.10.8 Controlling cache: Using other headers
Other problems with caching are the privacy of data and the question of where data should be stored in a cascade of
caches.
A user usually faces two kinds of caches: his or her own browser cache (a private cache) and his or her provider’s
cache (a public cache). A public cache is used by multiple users and controlled by someone else. This poses problems
3.10. Django’s cache framework 341](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-345-2048.jpg)
![Django Documentation, Release 1.5.1
The code lives in the django.core.mail module.
3.13.1 Quick example
In two lines:
from django.core.mail import send_mail
send_mail(’Subject here’, ’Here is the message.’, ’from@example.com’,
[’to@example.com’], fail_silently=False)
Mail is sent using the SMTP host and port specified in the EMAIL_HOST and EMAIL_PORT settings. The
EMAIL_HOST_USER and EMAIL_HOST_PASSWORD settings, if set, are used to authenticate to the SMTP server,
and the EMAIL_USE_TLS setting controls whether a secure connection is used.
Note: The character set of email sent with django.core.mail will be set to the value of your
DEFAULT_CHARSET setting.
3.13.2 send_mail()
send_mail(subject, message, from_email, recipient_list, fail_silently=False, auth_user=None,
auth_password=None, connection=None)
The simplest way to send email is using django.core.mail.send_mail().
The subject, message, from_email and recipient_list parameters are required.
• subject: A string.
• message: A string.
• from_email: A string.
• recipient_list: A list of strings, each an email address. Each member of recipient_list will see
the other recipients in the “To:” field of the email message.
• fail_silently: A boolean. If it’s False, send_mail will raise an smtplib.SMTPException. See
the smtplib docs for a list of possible exceptions, all of which are subclasses of SMTPException.
• auth_user: The optional username to use to authenticate to the SMTP server. If this isn’t provided, Django
will use the value of the EMAIL_HOST_USER setting.
• auth_password: The optional password to use to authenticate to the SMTP server. If this isn’t provided,
Django will use the value of the EMAIL_HOST_PASSWORD setting.
• connection: The optional email backend to use to send the mail. If unspecified, an instance of the default
backend will be used. See the documentation on Email backends for more details.
3.13.3 send_mass_mail()
send_mass_mail(datatuple, fail_silently=False, auth_user=None, auth_password=None, connec-tion=
None)
django.core.mail.send_mass_mail() is intended to handle mass emailing.
datatuple is a tuple in which each element is in this format:
3.13. Sending email 349](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-353-2048.jpg)
![Django Documentation, Release 1.5.1
(subject, message, from_email, recipient_list)
fail_silently, auth_user and auth_password have the same functions as in send_mail().
Each separate element of datatuple results in a separate email message. As in send_mail(), recipients in the
same recipient_list will all see the other addresses in the email messages’ “To:” field.
For example, the following code would send two different messages to two different sets of recipients; however, only
one connection to the mail server would be opened:
message1 = (’Subject here’, ’Here is the message’, ’from@example.com’, [’first@example.com’, ’other@example.message2 = (’Another Subject’, ’Here is another message’, ’from@example.com’, [’second@test.com’])
send_mass_mail((message1, message2), fail_silently=False)
send_mass_mail() vs. send_mail()
The main difference between send_mass_mail() and send_mail() is that send_mail() opens a connec-tion
to the mail server each time it’s executed, while send_mass_mail() uses a single connection for all of its
messages. This makes send_mass_mail() slightly more efficient.
3.13.4 mail_admins()
mail_admins(subject, message, fail_silently=False, connection=None, html_message=None)
django.core.mail.mail_admins() is a shortcut for sending an email to the site admins, as defined in the
ADMINS setting.
mail_admins() prefixes the subject with the value of the EMAIL_SUBJECT_PREFIX setting, which is
"[Django] " by default.
The “From:” header of the email will be the value of the SERVER_EMAIL setting.
This method exists for convenience and readability.
If html_message is provided, the resulting email will be a multipart/alternative email with message
as the text/plain content type and html_message as the text/html content type.
3.13.5 mail_managers()
mail_managers(subject, message, fail_silently=False, connection=None, html_message=None)
django.core.mail.mail_managers() is just like mail_admins(), except it sends an email to the site
managers, as defined in the MANAGERS setting.
3.13.6 Examples
This sends a single email to john@example.com and jane@example.com, with them both appearing in the “To:”:
send_mail(’Subject’, ’Message.’, ’from@example.com’,
[’john@example.com’, ’jane@example.com’])
This sends a message to john@example.com and jane@example.com, with them both receiving a separate email:
350 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-354-2048.jpg)
![Django Documentation, Release 1.5.1
datatuple = (
(’Subject’, ’Message.’, ’from@example.com’, [’john@example.com’]),
(’Subject’, ’Message.’, ’from@example.com’, [’jane@example.com’]),
)
send_mass_mail(datatuple)
3.13.7 Preventing header injection
Header injection is a security exploit in which an attacker inserts extra email headers to control the “To:” and “From:”
in email messages that your scripts generate.
The Django email functions outlined above all protect against header injection by forbidding newlines in header
values. If any subject, from_email or recipient_list contains a newline (in either Unix,Windows or Mac
style), the email function (e.g. send_mail()) will raise django.core.mail.BadHeaderError (a subclass
of ValueError) and, hence, will not send the email. It’s your responsibility to validate all data before passing it to
the email functions.
If a message contains headers at the start of the string, the headers will simply be printed as the first bit of the email
message.
Here’s an example view that takes a subject, message and from_email from the request’s POST data, sends
that to admin@example.com and redirects to “/contact/thanks/” when it’s done:
from django.core.mail import send_mail, BadHeaderError
def send_email(request):
subject = request.POST.get(’subject’, ’’)
message = request.POST.get(’message’, ’’)
from_email = request.POST.get(’from_email’, ’’)
if subject and message and from_email:
try:
send_mail(subject, message, from_email, [’admin@example.com’])
except BadHeaderError:
return HttpResponse(’Invalid header found.’)
return HttpResponseRedirect(’/contact/thanks/’)
else:
# In reality we’d use a form class
# to get proper validation errors.
return HttpResponse(’Make sure all fields are entered and valid.’)
3.13.8 The EmailMessage class
Django’s send_mail() and send_mass_mail() functions are actually thin wrappers that make use of the
EmailMessage class.
Not all features of the EmailMessage class are available through the send_mail() and related wrapper functions.
If you wish to use advanced features, such as BCC’ed recipients, file attachments, or multi-part email, you’ll need to
create EmailMessage instances directly.
Note: This is a design feature. send_mail() and related functions were originally the only interface Django
provided. However, the list of parameters they accepted was slowly growing over time. It made sense to move to a
more object-oriented design for email messages and retain the original functions only for backwards compatibility.
EmailMessage is responsible for creating the email message itself. The email backend is then responsible for
sending the email.
3.13. Sending email 351](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-355-2048.jpg)
![Django Documentation, Release 1.5.1
For convenience, EmailMessage provides a simple send() method for sending a single email. If you need to send
multiple messages, the email backend API provides an alternative.
EmailMessage Objects
class EmailMessage
The EmailMessage class is initialized with the following parameters (in the given order, if positional arguments are
used). All parameters are optional and can be set at any time prior to calling the send() method.
• subject: The subject line of the email.
• body: The body text. This should be a plain text message.
• from_email: The sender’s address. Both fred@example.com and Fred <fred@example.com>
forms are legal. If omitted, the DEFAULT_FROM_EMAIL setting is used.
• to: A list or tuple of recipient addresses.
• bcc: A list or tuple of addresses used in the “Bcc” header when sending the email.
• connection: An email backend instance. Use this parameter if you want to use the same connection for
multiple messages. If omitted, a new connection is created when send() is called.
• attachments: A list of attachments to put on the message. These can be either
email.MIMEBase.MIMEBase instances, or (filename, content, mimetype) triples.
• headers: A dictionary of extra headers to put on the message. The keys are the header name, values are the
header values. It’s up to the caller to ensure header names and values are in the correct format for an email
message.
• cc: A list or tuple of recipient addresses used in the “Cc” header when sending the email.
For example:
email = EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to1@example.com’, ’to2@example.com’], [’bcc@example.com’],
headers = {’Reply-To’: ’another@example.com’})
The class has the following methods:
• send(fail_silently=False) sends the message. If a connection was specified when the email was
constructed, that connection will be used. Otherwise, an instance of the default backend will be instantiated and
used. If the keyword argument fail_silently is True, exceptions raised while sending the message will
be quashed.
• message() constructs a django.core.mail.SafeMIMEText object (a subclass of Python’s
email.MIMEText.MIMEText class) or a django.core.mail.SafeMIMEMultipart object hold-ing
the message to be sent. If you ever need to extend the EmailMessage class, you’ll probably want to
override this method to put the content you want into the MIME object.
• recipients() returns a list of all the recipients of the message, whether they’re recorded in the to, cc or
bcc attributes. This is another method you might need to override when subclassing, because the SMTP server
needs to be told the full list of recipients when the message is sent. If you add another way to specify recipients
in your class, they need to be returned from this method as well.
• attach() creates a new file attachment and adds it to the message. There are two ways to call attach():
– You can pass it a single argument that is an email.MIMEBase.MIMEBase instance. This will be
inserted directly into the resulting message.
352 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-356-2048.jpg)
![Django Documentation, Release 1.5.1
– Alternatively, you can pass attach() three arguments: filename, content and mimetype.
filename is the name of the file attachment as it will appear in the email, content is the data that
will be contained inside the attachment and mimetype is the optional MIME type for the attachment. If
you omit mimetype, the MIME content type will be guessed from the filename of the attachment.
For example:
message.attach(’design.png’, img_data, ’image/png’)
• attach_file() creates a new attachment using a file from your filesystem. Call it with the path of the file to
attach and, optionally, the MIME type to use for the attachment. If the MIME type is omitted, it will be guessed
from the filename. The simplest use would be:
message.attach_file(’/images/weather_map.png’)
Sending alternative content types
It can be useful to include multiple versions of the content in an email; the classic example is to send both text and
HTML versions of a message. With Django’s email library, you can do this using the EmailMultiAlternatives
class. This subclass of EmailMessage has an attach_alternative() method for including extra versions of
the message body in the email. All the other methods (including the class initialization) are inherited directly from
EmailMessage.
To send a text and HTML combination, you could write:
from django.core.mail import EmailMultiAlternatives
subject, from_email, to = ’hello’, ’from@example.com’, ’to@example.com’
text_content = ’This is an important message.’
html_content = ’<p>This is an <strong>important</strong> message.</p>’
msg = EmailMultiAlternatives(subject, text_content, from_email, [to])
msg.attach_alternative(html_content, "text/html")
msg.send()
By default, the MIME type of the body parameter in an EmailMessage is "text/plain". It is good practice
to leave this alone, because it guarantees that any recipient will be able to read the email, regardless of their mail
client. However, if you are confident that your recipients can handle an alternative content type, you can use the
content_subtype attribute on the EmailMessage class to change the main content type. The major type will
always be "text", but you can change the subtype. For example:
msg = EmailMessage(subject, html_content, from_email, [to])
msg.content_subtype = "html" # Main content is now text/html
msg.send()
3.13.9 Email backends
The actual sending of an email is handled by the email backend.
The email backend class has the following methods:
• open() instantiates an long-lived email-sending connection.
• close() closes the current email-sending connection.
• send_messages(email_messages) sends a list of EmailMessage objects. If the connection is not
open, this call will implicitly open the connection, and close the connection afterwards. If the connection is
already open, it will be left open after mail has been sent.
3.13. Sending email 353](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-357-2048.jpg)
![Django Documentation, Release 1.5.1
In this example, the call to send_messages() opens a connection on the backend, sends the list of messages, and
then closes the connection again.
The second approach is to use the open() and close() methods on the email backend to manually control the
connection. send_messages() will not manually open or close the connection if it is already open, so if you
manually open the connection, you can control when it is closed. For example:
from django.core import mail
connection = mail.get_connection()
# Manually open the connection
connection.open()
# Construct an email message that uses the connection
email1 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to1@example.com’], connection=connection)
email1.send() # Send the email
# Construct two more messages
email2 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to2@example.com’])
email3 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’,
[’to3@example.com’])
# Send the two emails in a single call -
connection.send_messages([email2, email3])
# The connection was already open so send_messages() doesn’t close it.
# We need to manually close the connection.
connection.close()
3.13.10 Testing email sending
There are times when you do not want Django to send emails at all. For example, while developing a Web site, you
probably don’t want to send out thousands of emails – but you may want to validate that emails will be sent to the
right people under the right conditions, and that those emails will contain the correct content.
The easiest way to test your project’s use of email is to use the console email backend. This backend redirects all
email to stdout, allowing you to inspect the content of mail.
The file email backend can also be useful during development – this backend dumps the contents of every SMTP
connection to a file that can be inspected at your leisure.
Another approach is to use a “dumb” SMTP server that receives the emails locally and displays them to the terminal,
but does not actually send anything. Python has a built-in way to accomplish this with a single command:
python -m smtpd -n -c DebuggingServer localhost:1025
This command will start a simple SMTP server listening on port 1025 of localhost. This server simply prints to
standard output all email headers and the email body. You then only need to set the EMAIL_HOST and EMAIL_PORT
accordingly, and you are set.
For a more detailed discussion of testing and processing of emails locally, see the Python documentation for the
smtpd module.
356 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-360-2048.jpg)
![Django Documentation, Release 1.5.1
from django.utils.translation import ugettext as _
def my_view(request):
output = _("Welcome to my site.")
return HttpResponse(output)
Obviously, you could code this without using the alias. This example is identical to the previous one:
from django.utils.translation import ugettext
def my_view(request):
output = ugettext("Welcome to my site.")
return HttpResponse(output)
Translation works on computed values. This example is identical to the previous two:
def my_view(request):
words = [’Welcome’, ’to’, ’my’, ’site.’]
output = _(’ ’.join(words))
return HttpResponse(output)
Translation works on variables. Again, here’s an identical example:
def my_view(request):
sentence = ’Welcome to my site.’
output = _(sentence)
return HttpResponse(output)
(The caveat with using variables or computed values, as in the previous two examples, is that Django’s translation-string-
detecting utility, django-admin.py makemessages, won’t be able to find these strings. More on
makemessages later.)
The strings you pass to _() or ugettext() can take placeholders, specified with Python’s standard named-string
interpolation syntax. Example:
def my_view(request, m, d):
output = _(’Today is %(month)s %(day)s.’) % {’month’: m, ’day’: d}
return HttpResponse(output)
This technique lets language-specific translations reorder the placeholder text. For example, an English translation
may be "Today is November 26.", while a Spanish translation may be "Hoy es 26 de Noviembre."
– with the the month and the day placeholders swapped.
For this reason, you should use named-string interpolation (e.g., %(day)s) instead of positional interpolation (e.g.,
%s or %d) whenever you have more than a single parameter. If you used positional interpolation, translations wouldn’t
be able to reorder placeholder text.
Comments for translators
If you would like to give translators hints about a translatable string, you can add a comment prefixed with the
Translators keyword on the line preceding the string, e.g.:
def my_view(request):
# Translators: This message appears on the home page only
output = ugettext("Welcome to my site.")
The comment will then appear in the resulting .po file associated with the translatable contruct located below it and
should also be displayed by most translation tools.
358 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-362-2048.jpg)
![Django Documentation, Release 1.5.1
’There are %(count)d %(name)s available.’,
count
) % {
’count’: count,
’name’: name
}
Here we reuse localizable, hopefully already translated literals (contained in the verbose_name and
verbose_name_plural model Meta options) for other parts of the sentence so all of it is consistently based
on the cardinality of the elements at play.
Note: When using this technique, make sure you use a single name for every extrapolated variable included in the
literal. In the example above note how we used the name Python variable in both translation strings. This example
would fail:
from django.utils.translation import ungettext
from myapp.models import Report
count = Report.objects.count()
d = {
’count’: count,
’name’: Report._meta.verbose_name,
’plural_name’: Report._meta.verbose_name_plural
}
text = ungettext(
’There is %(count)d %(name)s available.’,
’There are %(count)d %(plural_name)s available.’,
count
) % d
You would get an error when running django-admin.py compilemessages:
a format specification for argument ’name’, as in ’msgstr[0]’, doesn’t exist in ’msgid’
Contextual markers
Sometimes words have several meanings, such as "May" in English, which refers to a month
name and to a verb. To enable translators to translate these words correctly in different
contexts, you can use the django.utils.translation.pgettext() function, or the
django.utils.translation.npgettext() function if the string needs pluralization. Both take a
context string as the first variable.
In the resulting .po file, the string will then appear as often as there are different contextual markers for the same
string (the context will appear on the msgctxt line), allowing the translator to give a different translation for each of
them.
For example:
from django.utils.translation import pgettext
month = pgettext("month name", "May")
or:
from django.utils.translation import pgettext_lazy
360 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-364-2048.jpg)
![Django Documentation, Release 1.5.1
verbose_name = _(’my thing’)
verbose_name_plural = _(’my things’)
Model methods short_description attribute values For model methods, you can provide translations to
Django and the admin site with the short_description attribute:
from django.utils.translation import ugettext_lazy as _
class MyThing(models.Model):
kind = models.ForeignKey(ThingKind, related_name=’kinds’,
verbose_name=_(’kind’))
def is_mouse(self):
return self.kind.type == MOUSE_TYPE
is_mouse.short_description = _(’Is it a mouse?’)
Working with lazy translation objects
The result of a ugettext_lazy() call can be used wherever you would use a unicode string (an object with type
unicode) in Python. If you try to use it where a bytestring (a str object) is expected, things will not work as
expected, since a ugettext_lazy() object doesn’t know how to convert itself to a bytestring. You can’t use a
unicode string inside a bytestring, either, so this is consistent with normal Python behavior. For example:
# This is fine: putting a unicode proxy into a unicode string.
u"Hello %s" % ugettext_lazy("people")
# This will not work, since you cannot insert a unicode object
# into a bytestring (nor can you insert our unicode proxy there)
"Hello %s" % ugettext_lazy("people")
If you ever see output that looks like "hello <django.utils.functional...>", you have tried to insert
the result of ugettext_lazy() into a bytestring. That’s a bug in your code.
If you don’t like the long ugettext_lazy name, you can just alias it as _ (underscore), like so:
from django.utils.translation import ugettext_lazy as _
class MyThing(models.Model):
name = models.CharField(help_text=_(’This is the help text’))
Using ugettext_lazy() and ungettext_lazy() to mark strings in models and utility functions is a common
operation. When you’re working with these objects elsewhere in your code, you should ensure that you don’t acci-dentally
convert them to strings, because they should be converted as late as possible (so that the correct locale is in
effect). This necessitates the use of the helper function described next.
Joining strings: string_concat() Standard Python string joins (”.join([...])) will not work on lists contain-ing
lazy translation objects. Instead, you can use django.utils.translation.string_concat(), which
creates a lazy object that concatenates its contents and converts them to strings only when the result is included in a
string. For example:
from django.utils.translation import string_concat
...
name = ugettext_lazy(’John Lennon’)
instrument = ugettext_lazy(’guitar’)
result = string_concat(name, ’: ’, instrument)
362 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-366-2048.jpg)
![Django Documentation, Release 1.5.1
In this case, the lazy translations in result will only be converted to strings when result itself is used in a string
(usually at template rendering time).
Other uses of lazy in delayed translations For any other case where you would like to delay the translation, but
have to pass the translatable string as argument to another function, you can wrap this function inside a lazy call
yourself. For example:
from django.utils import six # Python 3 compatibility
from django.utils.functional import lazy
from django.utils.safestring import mark_safe
from django.utils.translation import ugettext_lazy as _
mark_safe_lazy = lazy(mark_safe, six.text_type)
And then later:
lazy_string = mark_safe_lazy(_("<p>My <strong>string!</strong></p>"))
Localized names of languages
get_language_info()
The get_language_info() function provides detailed information about languages:
>>> from django.utils.translation import get_language_info
>>> li = get_language_info(’de’)
>>> print(li[’name’], li[’name_local’], li[’bidi’])
German Deutsch False
The name and name_local attributes of the dictionary contain the name of the language in English and in the
language itself, respectively. The bidi attribute is True only for bi-directional languages.
The source of the language information is the django.conf.locale module. Similar access to this information
is available for template code. See below.
Internationalization: in template code
Translations in Django templates uses two template tags and a slightly different syntax than in Python code. To give
your template access to these tags, put {% load i18n %} toward the top of your template. As with all template
tags, this tag needs to be loaded in all templates which use translations, even those templates that extend from other
templates which have already loaded the i18n tag.
trans template tag
The {% trans %} template tag translates either a constant string (enclosed in single or double quotes) or variable
content:
<title>{% trans "This is the title." %}</title>
<title>{% trans myvar %}</title>
If the noop option is present, variable lookup still takes place but the translation is skipped. This is useful when
“stubbing out” content that will require translation in the future:
<title>{% trans "myvar" noop %}</title>
3.14. Internationalization and localization 363](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-367-2048.jpg)
![Django Documentation, Release 1.5.1
{% some_special_tag _("Page not found") value|yesno:_("yes,no") %}
In this case, both the tag and the filter will see the already-translated string, so they don’t need to be aware of transla-tions.
Note: In this example, the translation infrastructure will be passed the string "yes,no", not the individual strings
"yes" and "no". The translated string will need to contain the comma so that the filter parsing code knows how
to split up the arguments. For example, a German translator might translate the string "yes,no" as "ja,nein"
(keeping the comma intact).
You can also retrieve information about any of the available languages using provided template tags and filters. To get
information about a single language, use the {% get_language_info %} tag:
{% get_language_info for LANGUAGE_CODE as lang %}
{% get_language_info for "pl" as lang %}
You can then access the information:
Language code: {{ lang.code }}<br />
Name of language: {{ lang.name_local }}<br />
Name in English: {{ lang.name }}<br />
Bi-directional: {{ lang.bidi }}
You can also use the {% get_language_info_list %} template tag to retrieve information for a list of lan-guages
(e.g. active languages as specified in LANGUAGES). See the section about the set_language redirect view for
an example of how to display a language selector using {% get_language_info_list %}.
In addition to LANGUAGES style nested tuples, {% get_language_info_list %} supports simple lists of
language codes. If you do this in your view:
return render_to_response(’mytemplate.html’, {
’available_languages’: [’en’, ’es’, ’fr’],
}, RequestContext(request))
you can iterate over those languages in the template:
{% get_language_info_list for available_languages as langs %}
{% for lang in langs %} ... {% endfor %}
There are also simple filters available for convenience:
• {{ LANGUAGE_CODE|language_name }} (“German”)
• {{ LANGUAGE_CODE|language_name_local }} (“Deutsch”)
• {{ LANGUAGE_CODE|bidi }} (False)
Internationalization: in JavaScript code
Adding translations to JavaScript poses some problems:
• JavaScript code doesn’t have access to a gettext implementation.
• JavaScript code doesn’t have access to .po or .mo files; they need to be delivered by the server.
• The translation catalogs for JavaScript should be kept as small as possible.
Django provides an integrated solution for these problems: It passes the translations into JavaScript, so you can call
gettext, etc., from within JavaScript.
3.14. Internationalization and localization 367](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-371-2048.jpg)
![Django Documentation, Release 1.5.1
var object_cnt = 1 // or 0, or 2, or 3, ...
s = ngettext(’literal for the singular case’,
’literal for the plural case’, object_cnt);
and even a string interpolation function:
function interpolate(fmt, obj, named);
The interpolation syntax is borrowed from Python, so the interpolate function supports both positional and named
interpolation:
• Positional interpolation: obj contains a JavaScript Array object whose elements values are then sequentially
interpolated in their corresponding fmt placeholders in the same order they appear. For example:
fmts = ngettext(’There is %s object. Remaining: %s’,
’There are %s objects. Remaining: %s’, 11);
s = interpolate(fmts, [11, 20]);
// s is ’There are 11 objects. Remaining: 20’
• Named interpolation: This mode is selected by passing the optional boolean named parameter as true. obj
contains a JavaScript object or associative array. For example:
d = {
count: 10,
total: 50
};
fmts = ngettext(’Total: %(total)s, there is %(count)s object’,
’there are %(count)s of a total of %(total)s objects’, d.count);
s = interpolate(fmts, d, true);
You shouldn’t go over the top with string interpolation, though: this is still JavaScript, so the code has to make repeated
regular-expression substitutions. This isn’t as fast as string interpolation in Python, so keep it to those cases where you
really need it (for example, in conjunction with ngettext to produce proper pluralizations).
Note on performance
The javascript_catalog() view generates the catalog from .mo files on every request. Since its output is
constant— at least for a given version of a site— it’s a good candidate for caching.
Server-side caching will reduce CPU load. It’s easily implemented with the cache_page() decorator. To trigger
cache invalidation when your translations change, provide a version-dependant key prefix, as shown in the example
below, or map the view at a version-dependant URL.
from django.views.decorators.cache import cache_page
from django.views.i18n import javascript_catalog
# The value returned by get_version() must change when translations change.
@cache_page(86400, key_prefix=’js18n-%s’ % get_version())
def cached_javascript_catalog(request, domain=’djangojs’, packages=None):
return javascript_catalog(request, domain, packages)
Client-side caching will save bandwidth and make your site load faster. If you’re using ETags (USE_ETAGS =
True), you’re already covered. Otherwise, you can apply conditional decorators. In the following example, the cache
is invalidated whenever your restart your application server.
from django.utils import timezone
from django.views.decorators.http import last_modified
3.14. Internationalization and localization 369](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-373-2048.jpg)
![Django Documentation, Release 1.5.1
from django.views.i18n import javascript_catalog
last_modified_date = timezone.now()
@last_modified(lambda req, **kw: last_modified_date)
def cached_javascript_catalog(request, domain=’djangojs’, packages=None):
return javascript_catalog(request, domain, packages)
You can even pre-generate the javascript catalog as part of your deployment procedure and serve it as a static file. This
radical technique is implemented in django-statici18n.
Internationalization: in URL patterns
New in version 1.4. Django provides two mechanisms to internationalize URL patterns:
• Adding the language prefix to the root of the URL patterns to make it possible for LocaleMiddleware to
detect the language to activate from the requested URL.
• Making URL patterns themselves translatable via the django.utils.translation.ugettext_lazy()
function.
Warning: Using either one of these features requires that an active language be set for each re-quest;
in other words, you need to have django.middleware.locale.LocaleMiddleware in your
MIDDLEWARE_CLASSES setting.
Language prefix in URL patterns
i18n_patterns(prefix, pattern_description, ...)
This function can be used in your root URLconf as a replacement for the normal
django.conf.urls.patterns() function. Django will automatically prepend the current active language
code to all url patterns defined within i18n_patterns(). Example URL patterns:
from django.conf.urls import patterns, include, url
from django.conf.urls.i18n import i18n_patterns
urlpatterns = patterns(’’,
url(r’^sitemap.xml$’, ’sitemap.view’, name=’sitemap_xml’),
)
news_patterns = patterns(’’,
url(r’^$’, ’news.views.index’, name=’index’),
url(r’^category/(?P<slug>[w-]+)/$’, ’news.views.category’, name=’category’),
url(r’^(?P<slug>[w-]+)/$’, ’news.views.details’, name=’detail’),
)
urlpatterns += i18n_patterns(’’,
url(r’^about/$’, ’about.view’, name=’about’),
url(r’^news/’, include(news_patterns, namespace=’news’)),
)
After defining these URL patterns, Django will automatically add the language prefix to the URL patterns that were
added by the i18n_patterns function. Example:
from django.core.urlresolvers import reverse
from django.utils.translation import activate
370 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-374-2048.jpg)
![Django Documentation, Release 1.5.1
>>> activate(’en’)
>>> reverse(’sitemap_xml’)
’/sitemap.xml’
>>> reverse(’news:index’)
’/en/news/’
>>> activate(’nl’)
>>> reverse(’news:detail’, kwargs={’slug’: ’news-slug’})
’/nl/news/news-slug/’
Warning: i18n_patterns() is only allowed in your root URLconf. Using it within an included URLconf
will throw an ImproperlyConfigured exception.
Warning: Ensure that you don’t have non-prefixed URL patterns that might collide with an automatically-added
language prefix.
Translating URL patterns
URL patterns can also be marked translatable using the ugettext_lazy() function. Example:
from django.conf.urls import patterns, include, url
from django.conf.urls.i18n import i18n_patterns
from django.utils.translation import ugettext_lazy as _
urlpatterns = patterns(’’
url(r’^sitemap.xml$’, ’sitemap.view’, name=’sitemap_xml’),
)
news_patterns = patterns(’’
url(r’^$’, ’news.views.index’, name=’index’),
url(_(r’^category/(?P<slug>[w-]+)/$’), ’news.views.category’, name=’category’),
url(r’^(?P<slug>[w-]+)/$’, ’news.views.details’, name=’detail’),
)
urlpatterns += i18n_patterns(’’,
url(_(r’^about/$’), ’about.view’, name=’about’),
url(_(r’^news/’), include(news_patterns, namespace=’news’)),
)
After you’ve created the translations, the reverse() function will return the URL in the active language. Example:
from django.core.urlresolvers import reverse
from django.utils.translation import activate
>>> activate(’en’)
>>> reverse(’news:category’, kwargs={’slug’: ’recent’})
’/en/news/category/recent/’
>>> activate(’nl’)
>>> reverse(’news:category’, kwargs={’slug’: ’recent’})
’/nl/nieuws/categorie/recent/’
Warning: In most cases, it’s best to use translated URLs only within a language-code-prefixed block of patterns
(using i18n_patterns()), to avoid the possibility that a carelessly translated URL causes a collision with a
non-translated URL pattern.
3.14. Internationalization and localization 371](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-375-2048.jpg)
![Django Documentation, Release 1.5.1
Create a view that can set the current timezone:
import pytz
from django.shortcuts import redirect, render
def set_timezone(request):
if request.method == ’POST’:
request.session[’django_timezone’] = pytz.timezone(request.POST[’timezone’])
return redirect(’/’)
else:
return render(request, ’template.html’, {’timezones’: pytz.common_timezones})
Include a form in template.html that will POST to this view:
{% load tz %}
<form action="{% url ’set_timezone’ %}" method="POST">
{% csrf_token %}
<label for="timezone">Time zone:</label>
<select name="timezone">
{% for tz in timezones %}
<option value="{{ tz }}"{% if tz == TIME_ZONE %} selected="selected"{% endif %}>{{ tz }}</option>
{% endfor %}
</select>
<input type="submit" value="Set" />
</form>
Time zone aware input in forms
When you enable time zone support, Django interprets datetimes entered in forms in the current time zone and returns
aware datetime objects in cleaned_data.
If the current time zone raises an exception for datetimes that don’t exist or are ambiguous because they fall in a DST
transition (the timezones provided by pytz do this), such datetimes will be reported as invalid values.
Time zone aware output in templates
When you enable time zone support, Django converts aware datetime objects to the current time zone when they’re
rendered in templates. This behaves very much like format localization.
Warning: Django doesn’t convert naive datetime objects, because they could be ambiguous, and because your
code should never produce naive datetimes when time zone support is enabled. However, you can force conversion
with the template filters described below.
Conversion to local time isn’t always appropriate – you may be generating output for computers rather than for humans.
The following filters and tags, provided by the tz template tag library, allow you to control the time zone conversions.
Template tags
localtime Enables or disables conversion of aware datetime objects to the current time zone in the contained block.
This tag has exactly the same effects as the USE_TZ setting as far as the template engine is concerned. It allows a
more fine grained control of conversion.
To activate or deactivate conversion for a template block, use:
3.14. Internationalization and localization 385](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-389-2048.jpg)
![Django Documentation, Release 1.5.1
’class’: ’django.utils.log.NullHandler’,
},
’console’:{
’level’: ’DEBUG’,
’class’: ’logging.StreamHandler’,
’formatter’: ’simple’
},
’mail_admins’: {
’level’: ’ERROR’,
’class’: ’django.utils.log.AdminEmailHandler’,
’filters’: [’special’]
}
},
’loggers’: {
’django’: {
’handlers’: [’null’],
’propagate’: True,
’level’: ’INFO’,
},
’django.request’: {
’handlers’: [’mail_admins’],
’level’: ’ERROR’,
’propagate’: False,
},
’myproject.custom’: {
’handlers’: [’console’, ’mail_admins’],
’level’: ’INFO’,
’filters’: [’special’]
}
}
}
This logging configuration does the following things:
• Identifies the configuration as being in ‘dictConfig version 1’ format. At present, this is the only dictConfig
format version.
• Disables all existing logging configurations.
• Defines two formatters:
– simple, that just outputs the log level name (e.g., DEBUG) and the log message.
The format string is a normal Python formatting string describing the details that are to be output on
each logging line. The full list of detail that can be output can be found in the formatter documentation.
– verbose, that outputs the log level name, the log message, plus the time, process, thread and module that
generate the log message.
• Defines one filter – project.logging.SpecialFilter, using the alias special. If this filter re-quired
additional arguments at time of construction, they can be provided as additional keys in the filter con-figuration
dictionary. In this case, the argument foo will be given a value of bar when instantiating the
SpecialFilter.
• Defines three handlers:
– null, a NullHandler, which will pass any DEBUG (or higher) message to /dev/null.
– console, a StreamHandler, which will print any DEBUG (or higher) message to stderr. This handler uses
the simple output format.
396 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-400-2048.jpg)
![Django Documentation, Release 1.5.1
django.request
Log messages related to the handling of requests. 5XX responses are raised as ERROR messages; 4XX responses are
raised as WARNING messages.
Messages to this logger have the following extra context:
• status_code: The HTTP response code associated with the request.
• request: The request object that generated the logging message.
django.db.backends
Messages relating to the interaction of code with the database. For example, every SQL statement executed by a
request is logged at the DEBUG level to this logger.
Messages to this logger have the following extra context:
• duration: The time taken to execute the SQL statement.
• sql: The SQL statement that was executed.
• params: The parameters that were used in the SQL call.
For performance reasons, SQL logging is only enabled when settings.DEBUG is set to True, regardless of the
logging level or handlers that are installed.
Handlers
Django provides one log handler in addition to those provided by the Python logging module.
class AdminEmailHandler([include_html=False ])
This handler sends an email to the site admins for each log message it receives.
If the log record contains a request attribute, the full details of the request will be included in the email.
If the log record contains stack trace information, that stack trace will be included in the email.
The include_html argument of AdminEmailHandler is used to control whether the traceback email
includes an HTML attachment containing the full content of the debug Web page that would have been pro-duced
if DEBUG were True. To set this value in your configuration, include it in the handler definition for
django.utils.log.AdminEmailHandler, like this:
’handlers’: {
’mail_admins’: {
’level’: ’ERROR’,
’class’: ’django.utils.log.AdminEmailHandler’,
’include_html’: True,
}
},
Note that this HTML version of the email contains a full traceback, with names and values of local variables at
each level of the stack, plus the values of your Django settings. This information is potentially very sensitive,
and you may not want to send it over email. Consider using something such as Sentry to get the best of both
worlds – the rich information of full tracebacks plus the security of not sending the information over email. You
may also explicitly designate certain sensitive information to be filtered out of error reports – learn more on
Filtering error reports.
398 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-402-2048.jpg)
![Django Documentation, Release 1.5.1
Filters
Django provides two log filters in addition to those provided by the Python logging module.
class CallbackFilter(callback)
New in version 1.4. This filter accepts a callback function (which should accept a single argument, the record to
be logged), and calls it for each record that passes through the filter. Handling of that record will not proceed if
the callback returns False.
For instance, to filter out UnreadablePostError (raised when a user cancels an upload) from the admin
emails, you would create a filter function:
from django.http import UnreadablePostError
def skip_unreadable_post(record):
if record.exc_info:
exc_type, exc_value = record.exc_info[:2]
if isinstance(exc_value, UnreadablePostError):
return False
return True
and then add it to your logging config:
’filters’: {
’skip_unreadable_posts’: {
’()’: ’django.utils.log.CallbackFilter’,
’callback’: skip_unreadable_post,
}
},
’handlers’: {
’mail_admins’: {
’level’: ’ERROR’,
’filters’: [’skip_unreadable_posts’],
’class’: ’django.utils.log.AdminEmailHandler’
}
},
class RequireDebugFalse
New in version 1.4. This filter will only pass on records when settings.DEBUG is False.
This filter is used as follows in the default LOGGING configuration to ensure that the AdminEmailHandler
only sends error emails to admins when DEBUG is False:
’filters’: {
’require_debug_false’: {
’()’: ’django.utils.log.RequireDebugFalse’,
}
},
’handlers’: {
’mail_admins’: {
’level’: ’ERROR’,
’filters’: [’require_debug_false’],
’class’: ’django.utils.log.AdminEmailHandler’
}
},
class RequireDebugTrue
New in version 1.5. This filter is similar to RequireDebugFalse, except that records are passed only when
DEBUG is True.
3.15. Logging 399](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-403-2048.jpg)
![Django Documentation, Release 1.5.1
3.15.5 Django’s default logging configuration
By default, Django configures the django.request logger so that all messages with ERROR or CRITICAL level
are sent to AdminEmailHandler, as long as the DEBUG setting is set to False.
All messages reaching the django catch-all logger when DEBUG is True are sent to the console. They are simply
discarded (sent to NullHandler) when DEBUG is False. Changed in version 1.5. See also Configuring logging to
learn how you can complement or replace this default logging configuration.
3.16 Pagination
Django provides a few classes that help you manage paginated data – that is, data that’s split across several pages, with
“Previous/Next” links. These classes live in django/core/paginator.py.
3.16.1 Example
Give Paginator a list of objects, plus the number of items you’d like to have on each page, and it gives you methods
for accessing the items for each page:
>>> from django.core.paginator import Paginator
>>> objects = [’john’, ’paul’, ’george’, ’ringo’]
>>> p = Paginator(objects, 2)
>>> p.count
4
>>> p.num_pages
2
>>> p.page_range
[1, 2]
>>> page1 = p.page(1)
>>> page1
<Page 1 of 2>
>>> page1.object_list
[’john’, ’paul’]
>>> page2 = p.page(2)
>>> page2.object_list
[’george’, ’ringo’]
>>> page2.has_next()
False
>>> page2.has_previous()
True
>>> page2.has_other_pages()
True
>>> page2.next_page_number()
Traceback (most recent call last):
...
EmptyPage: That page contains no results
>>> page2.previous_page_number()
1
>>> page2.start_index() # The 1-based index of the first item on this page
3
>>> page2.end_index() # The 1-based index of the last item on this page
4
400 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-404-2048.jpg)
![Django Documentation, Release 1.5.1
Paginator.num_pages
The total number of pages.
Paginator.page_range
A 1-based range of page numbers, e.g., [1, 2, 3, 4].
3.16.4 InvalidPage exceptions
exception InvalidPage
A base class for exceptions raised when a paginator is passed an invalid page number.
The Paginator.page() method raises an exception if the requested page is invalid (i.e., not an integer) or contains
no objects. Generally, it’s enough to trap the InvalidPage exception, but if you’d like more granularity, you can
trap either of the following exceptions:
exception PageNotAnInteger
Raised when page() is given a value that isn’t an integer.
exception EmptyPage
Raised when page() is given a valid value but no objects exist on that page.
Both of the exceptions are subclasses of InvalidPage, so you can handle them both with a simple except
InvalidPage.
3.16.5 Page objects
You usually won’t construct Page objects by hand – you’ll get them using Paginator.page().
class Page(object_list, number, paginator)
New in version 1.4: A page acts like a sequence of Page.object_list when using len() or iterating it directly.
Methods
Page.has_next()
Returns True if there’s a next page.
Page.has_previous()
Returns True if there’s a previous page.
Page.has_other_pages()
Returns True if there’s a next or previous page.
Page.next_page_number()
Returns the next page number. Changed in version 1.5. Raises InvalidPage if next page doesn’t exist.
Page.previous_page_number()
Returns the previous page number. Changed in version 1.5. Raises InvalidPage if previous page doesn’t
exist.
Page.start_index()
Returns the 1-based index of the first object on the page, relative to all of the objects in the paginator’s list.
For example, when paginating a list of 5 objects with 2 objects per page, the second page’s start_index()
would return 3.
3.16. Pagination 403](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-407-2048.jpg)
![Django Documentation, Release 1.5.1
name = models.CharField(max_length=100)
author = models.ForeignKey(Person)
Ordinarily, serialized data for Book would use an integer to refer to the author. For example, in JSON, a Book might
be serialized as:
...
{
pk: 1,
model: store.book,
fields: {
name: Mostly Harmless,
author: 42
}
}
...
This isn’t a particularly natural way to refer to an author. It requires that you know the primary key value for the
author; it also requires that this primary key value is stable and predictable.
However, if we add natural key handling to Person, the fixture becomes much more humane. To add natural key
handling, you define a default Manager for Person with a get_by_natural_key() method. In the case of a
Person, a good natural key might be the pair of first and last name:
from django.db import models
class PersonManager(models.Manager):
def get_by_natural_key(self, first_name, last_name):
return self.get(first_name=first_name, last_name=last_name)
class Person(models.Model):
objects = PersonManager()
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
birthdate = models.DateField()
class Meta:
unique_together = ((’first_name’, ’last_name’),)
Now books can use that natural key to refer to Person objects:
...
{
pk: 1,
model: store.book,
fields: {
name: Mostly Harmless,
author: [Douglas, Adams]
}
}
...
When you try to load this serialized data, Django will use the get_by_natural_key() method to resolve
[Douglas, Adams] into the primary key of an actual Person object.
Note: Whatever fields you use for a natural key must be able to uniquely identify an object. This will usually mean
that your model will have a uniqueness clause (either unique=True on a single field, or unique_together over
multiple fields) for the field or fields in your natural key. However, uniqueness doesn’t need to be enforced at the
416 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-420-2048.jpg)
![Django Documentation, Release 1.5.1
database level. If you are certain that a set of fields will be effectively unique, you can still use those fields as a natural
key.
Serialization of natural keys
So how do you get Django to emit a natural key when serializing an object? Firstly, you need to add another method
– this time to the model itself:
class Person(models.Model):
objects = PersonManager()
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
birthdate = models.DateField()
def natural_key(self):
return (self.first_name, self.last_name)
class Meta:
unique_together = ((’first_name’, ’last_name’),)
That method should always return a natural key tuple – in this example, (first name, last name). Then,
when you call serializers.serialize(), you provide a use_natural_keys=True argument:
serializers.serialize(’json’, [book1, book2], indent=2, use_natural_keys=True)
When use_natural_keys=True is specified, Django will use the natural_key() method to serialize any
reference to objects of the type that defines the method.
If you are using dumpdata to generate serialized data, you use the --natural command line flag to generate
natural keys.
Note: You don’t need to define both natural_key() and get_by_natural_key(). If you don’t want Django
to output natural keys during serialization, but you want to retain the ability to load natural keys, then you can opt to
not implement the natural_key() method.
Conversely, if (for some strange reason) you want Django to output natural keys during serialization, but not be able
to load those key values, just don’t define the get_by_natural_key() method.
Dependencies during serialization
Since natural keys rely on database lookups to resolve references, it is important that the data exists before it is
referenced. You can’t make a “forward reference” with natural keys – the data you’re referencing must exist before
you include a natural key reference to that data.
To accommodate this limitation, calls to dumpdata that use the --natural option will serialize any model with a
natural_key() method before serializing standard primary key objects.
However, this may not always be enough. If your natural key refers to another object (by using a foreign key or natural
key to another object as part of a natural key), then you need to be able to ensure that the objects on which a natural
key depends occur in the serialized data before the natural key requires them.
To control this ordering, you can define dependencies on your natural_key() methods. You do this by setting a
dependencies attribute on the natural_key() method itself.
3.19. Serializing Django objects 417](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-421-2048.jpg)
![Django Documentation, Release 1.5.1
For example, let’s add a natural key to the Book model from the example above:
class Book(models.Model):
name = models.CharField(max_length=100)
author = models.ForeignKey(Person)
def natural_key(self):
return (self.name,) + self.author.natural_key()
The natural key for a Book is a combination of its name and its author. This means that Person must be serialized
before Book. To define this dependency, we add one extra line:
def natural_key(self):
return (self.name,) + self.author.natural_key()
natural_key.dependencies = [’example_app.person’]
This definition ensures that all Person objects are serialized before any Book objects. In turn, any object referencing
Book will be serialized after both Person and Book have been serialized.
3.20 Django settings
A Django settings file contains all the configuration of your Django installation. This document explains how settings
work and which settings are available.
3.20.1 The basics
A settings file is just a Python module with module-level variables.
Here are a couple of example settings:
DEBUG = False
DEFAULT_FROM_EMAIL = ’webmaster@example.com’
TEMPLATE_DIRS = (’/home/templates/mike’, ’/home/templates/john’)
Because a settings file is a Python module, the following apply:
• It doesn’t allow for Python syntax errors.
• It can assign settings dynamically using normal Python syntax. For example:
MY_SETTING = [str(i) for i in range(30)]
• It can import values from other settings files.
3.20.2 Designating the settings
DJANGO_SETTINGS_MODULE
When you use Django, you have to tell it which settings you’re using. Do this by using an environment variable,
DJANGO_SETTINGS_MODULE.
The value of DJANGO_SETTINGS_MODULE should be in Python path syntax, e.g. mysite.settings. Note that
the settings module should be on the Python import search path.
418 Chapter 3. Using Django](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-422-2048.jpg)
![Django Documentation, Release 1.5.1
The django-admin.py utility
When using django-admin.py, you can either set the environment variable once, or explicitly pass in the settings
module each time you run the utility.
Example (Unix Bash shell):
export DJANGO_SETTINGS_MODULE=mysite.settings
django-admin.py runserver
Example (Windows shell):
set DJANGO_SETTINGS_MODULE=mysite.settings
django-admin.py runserver
Use the --settings command-line argument to specify the settings manually:
django-admin.py runserver --settings=mysite.settings
On the server (mod_wsgi)
In your live server environment, you’ll need to tell your WSGI application what settings file to use. Do that with
os.environ:
import os
os.environ[’DJANGO_SETTINGS_MODULE’] = ’mysite.settings’
Read the Django mod_wsgi documentation for more information and other common elements to a Django WSGI
application.
3.20.3 Default settings
A Django settings file doesn’t have to define any settings if it doesn’t need to. Each setting has a sensible default value.
These defaults live in the module django/conf/global_settings.py.
Here’s the algorithm Django uses in compiling settings:
• Load settings from global_settings.py.
• Load settings from the specified settings file, overriding the global settings as necessary.
Note that a settings file should not import from global_settings, because that’s redundant.
Seeing which settings you’ve changed
There’s an easy way to view which of your settings deviate from the default settings. The command python
manage.py diffsettings displays differences between the current settings file and Django’s default settings.
For more, see the diffsettings documentation.
3.20.4 Using settings in Python code
In your Django apps, use settings by importing the object django.conf.settings. Example:
3.20. Django settings 419](https://crownmelresort.com/image.slidesharecdn.com/django-141201081021-conversion-gate02/75/Django-423-2048.jpg)
Uploaded byNarcisse Siewe
Django
This document provides an overview of the key components of Django, including designing models, installing Django, accessing the model API, creating an admin interface, designing URLs and views, writing templates, and more. It explains these concepts at a high level and provides examples to illustrate how each component works and fits together. The goal is to help new users understand the basics of how to build a database-driven web application with Django.
Django
- 1. Django Documentation Release1.5.1 Django Software Foundation May 01, 2013
- 3.
- 4.
- 5. CHAPTER ONE DJANGODOCUMENTATION Everything you need to know about Django. 1.1 Getting help Having trouble? We’d like to help! • Try the FAQ – it’s got answers to many common questions. • Looking for specific information? Try the genindex, modindex or the detailed table of contents. • Search for information in the archives of the django-users mailing list, or post a question. • Ask a question in the #django IRC channel, or search the IRC logs to see if it’s been asked before. • Report bugs with Django in our ticket tracker. 1.2 First steps Are you new to Django or to programming? This is the place to start! • From scratch: Overview | Installation • Tutorial: Part 1 | Part 2 | Part 3 | Part 4 | Part 5 | Part 6 • Advanced Tutorials: How to write reusable apps | Writing your first patch for Django 1.3 The model layer Django provides an abstraction layer (the “models”) for structuring and manipulating the data of yourWeb application. Learn more about it below: • Models: Model syntax | Field types | Meta options • QuerySets: Executing queries | QuerySet method reference • Model instances: Instance methods | Accessing related objects • Advanced: Managers | Raw SQL | Transactions | Aggregation | Custom fields | Multiple databases • Other: Supported databases | Legacy databases | Providing initial data | Optimize database access 1
- 6. Django Documentation, Release1.5.1 1.4 The view layer Django has the concept of “views” to encapsulate the logic responsible for processing a user’s request and for returning the response. Find all you need to know about views via the links below: • The basics: URLconfs | View functions | Shortcuts | Decorators • Reference: Built-in Views | Request/response objects | TemplateResponse objects • File uploads: Overview | File objects | Storage API | Managing files | Custom storage • Class-based views: Overview | Built-in display views | Built-in editing views | Using mixins | API reference | Flattened index • Advanced: Generating CSV | Generating PDF • Middleware: Overview | Built-in middleware classes 1.5 The template layer The template layer provides a designer-friendly syntax for rendering the information to be presented to the user. Learn how this syntax can be used by designers and how it can be extended by programmers: • For designers: Syntax overview | Built-in tags and filters | Web design helpers | Humanization • For programmers: Template API | Custom tags and filters 1.6 Forms Django provides a rich framework to facilitate the creation of forms and the manipulation of form data. • The basics: Overview | Form API | Built-in fields | Built-in widgets • Advanced: Forms for models | Integrating media | Formsets | Customizing validation • Extras: Form preview | Form wizard 1.7 The development process Learn about the various components and tools to help you in the development and testing of Django applications: • Settings: Overview | Full list of settings • Exceptions: Overview • django-admin.py and manage.py: Overview | Adding custom commands • Testing: Introduction | Writing and running tests | Advanced topics | Doctests • Deployment: Overview | WSGI servers | FastCGI/SCGI/AJP | Deploying static files | Tracking code errors by email 2 Chapter 1. Django documentation
- 7. Django Documentation, Release1.5.1 1.8 The admin Find all you need to know about the automated admin interface, one of Django’s most popular features: • Admin site • Admin actions • Admin documentation generator 1.9 Security Security is a topic of paramount importance in the development of Web applications and Django provides multiple protection tools and mechanisms: • Security overview • Clickjacking protection • Cross Site Request Forgery protection • Cryptographic signing 1.10 Internationalization and localization Django offers a robust internationalization and localization framework to assist you in the development of applications for multiple languages and world regions: • Overview | Internationalization | Localization • “Local flavor” • Time zones 1.11 Python compatibility Django aims to be compatible with multiple different flavors and versions of Python: • Jython support • Python 3 compatibility 1.12 Geographic framework GeoDjango intends to be a world-class geographic Web framework. Its goal is to make it as easy as possible to build GIS Web applications and harness the power of spatially enabled data. 1.8. The admin 3
- 8. Django Documentation, Release1.5.1 1.13 Common Web application tools Django offers multiple tools commonly needed in the development of Web applications: • Authentication • Caching • Logging • Sending emails • Syndication feeds (RSS/Atom) • Comments, comment moderation and custom comments • Pagination • Messages framework • Serialization • Sessions • Sitemaps • Static files management • Data validation 1.14 Other core functionalities Learn about some other core functionalities of the Django framework: • Conditional content processing • Content types and generic relations • Databrowse • Flatpages • Redirects • Signals • The sites framework • Unicode in Django 1.15 The Django open-source project Learn about the development process for the Django project itself and about how you can contribute: • Community: How to get involved | The release process | Team of committers | The Django source code reposi-tory | Security policies • Design philosophies: Overview • Documentation: About this documentation • Third-party distributions: Overview 4 Chapter 1. Django documentation
- 9. Django Documentation, Release1.5.1 • Django over time: API stability | Release notes and upgrading instructions | Deprecation Timeline 1.15. The Django open-source project 5
- 10. Django Documentation, Release1.5.1 6 Chapter 1. Django documentation
- 11. CHAPTER TWO GETTINGSTARTED New to Django? Or to Web development in general? Well, you came to the right place: read this material to quickly get up and running. 2.1 Django at a glance Because Django was developed in a fast-paced newsroom environment, it was designed to make common Web-development tasks fast and easy. Here’s an informal overview of how to write a database-drivenWeb app with Django. The goal of this document is to give you enough technical specifics to understand how Django works, but this isn’t intended to be a tutorial or reference – but we’ve got both! When you’re ready to start a project, you can start with the tutorial or dive right into more detailed documentation. 2.1.1 Design your model Although you can use Django without a database, it comes with an object-relational mapper in which you describe your database layout in Python code. The data-model syntax offers many rich ways of representing your models – so far, it’s been solving two years’ worth of database-schema problems. Here’s a quick example, which might be saved in the file mysite/news/models.py: class Reporter(models.Model): full_name = models.CharField(max_length=70) def __unicode__(self): return self.full_name class Article(models.Model): pub_date = models.DateField() headline = models.CharField(max_length=200) content = models.TextField() reporter = models.ForeignKey(Reporter) def __unicode__(self): return self.headline 2.1.2 Install it Next, run the Django command-line utility to create the database tables automatically: 7
- 12. Django Documentation, Release1.5.1 manage.py syncdb The syncdb command looks at all your available models and creates tables in your database for whichever tables don’t already exist. 2.1.3 Enjoy the free API With that, you’ve got a free, and rich, Python API to access your data. The API is created on the fly, no code generation necessary: # Import the models we created from our "news" app >>> from news.models import Reporter, Article # No reporters are in the system yet. >>> Reporter.objects.all() [] # Create a new Reporter. >>> r = Reporter(full_name=’John Smith’) # Save the object into the database. You have to call save() explicitly. >>> r.save() # Now it has an ID. >>> r.id 1 # Now the new reporter is in the database. >>> Reporter.objects.all() [<Reporter: John Smith>] # Fields are represented as attributes on the Python object. >>> r.full_name ’John Smith’ # Django provides a rich database lookup API. >>> Reporter.objects.get(id=1) <Reporter: John Smith> >>> Reporter.objects.get(full_name__startswith=’John’) <Reporter: John Smith> >>> Reporter.objects.get(full_name__contains=’mith’) <Reporter: John Smith> >>> Reporter.objects.get(id=2) Traceback (most recent call last): ... DoesNotExist: Reporter matching query does not exist. Lookup parameters were {’id’: 2} # Create an article. >>> from datetime import date >>> a = Article(pub_date=date.today(), headline=’Django is cool’, ... content=’Yeah.’, reporter=r) >>> a.save() # Now the article is in the database. >>> Article.objects.all() [<Article: Django is cool>] 8 Chapter 2. Getting started
- 13. Django Documentation, Release1.5.1 # Article objects get API access to related Reporter objects. >>> r = a.reporter >>> r.full_name ’John Smith’ # And vice versa: Reporter objects get API access to Article objects. >>> r.article_set.all() [<Article: Django is cool>] # The API follows relationships as far as you need, performing efficient # JOINs for you behind the scenes. # This finds all articles by a reporter whose name starts with "John". >>> Article.objects.filter(reporter__full_name__startswith="John") [<Article: Django is cool>] # Change an object by altering its attributes and calling save(). >>> r.full_name = ’Billy Goat’ >>> r.save() # Delete an object with delete(). >>> r.delete() 2.1.4 A dynamic admin interface: it’s not just scaffolding – it’s the whole house Once your models are defined, Django can automatically create a professional, production ready administrative inter-face – a Web site that lets authenticated users add, change and delete objects. It’s as easy as registering your model in the admin site: # In models.py... from django.db import models class Article(models.Model): pub_date = models.DateField() headline = models.CharField(max_length=200) content = models.TextField() reporter = models.ForeignKey(Reporter) # In admin.py in the same directory... import models from django.contrib import admin admin.site.register(models.Article) The philosophy here is that your site is edited by a staff, or a client, or maybe just you – and you don’t want to have to deal with creating backend interfaces just to manage content. One typical workflow in creating Django apps is to create models and get the admin sites up and running as fast as possible, so your staff (or clients) can start populating data. Then, develop the way data is presented to the public. 2.1.5 Design your URLs A clean, elegant URL scheme is an important detail in a high-quality Web application. Django encourages beautiful URL design and doesn’t put any cruft in URLs, like .php or .asp. 2.1. Django at a glance 9
- 14. Django Documentation, Release1.5.1 To design URLs for an app, you create a Python module called a URLconf . A table of contents for your app, it contains a simple mapping between URL patterns and Python callback functions. URLconfs also serve to decouple URLs from Python code. Here’s what a URLconf might look like for the Reporter/Article example above: from django.conf.urls import patterns urlpatterns = patterns(’’, (r’^articles/(d{4})/$’, ’news.views.year_archive’), (r’^articles/(d{4})/(d{2})/$’, ’news.views.month_archive’), (r’^articles/(d{4})/(d{2})/(d+)/$’, ’news.views.article_detail’), ) The code above maps URLs, as simple regular expressions, to the location of Python callback functions (“views”). The regular expressions use parenthesis to “capture” values from the URLs. When a user requests a page, Django runs through each pattern, in order, and stops at the first one that matches the requested URL. (If none of them matches, Django calls a special-case 404 view.) This is blazingly fast, because the regular expressions are compiled at load time. Once one of the regexes matches, Django imports and calls the given view, which is a simple Python function. Each view gets passed a request object – which contains request metadata – and the values captured in the regex. For example, if a user requested the URL “/articles/2005/05/39323/”, Django would call the function news.views.article_detail(request, ’2005’, ’05’, ’39323’). 2.1.6 Write your views Each view is responsible for doing one of two things: Returning an HttpResponse object containing the content for the requested page, or raising an exception such as Http404. The rest is up to you. Generally, a view retrieves data according to the parameters, loads a template and renders the template with the retrieved data. Here’s an example view for year_archive from above: def year_archive(request, year): a_list = Article.objects.filter(pub_date__year=year) return render_to_response(’news/year_archive.html’, {’year’: year, ’article_list’: a_list}) This example uses Django’s template system, which has several powerful features but strives to stay simple enough for non-programmers to use. 2.1.7 Design your templates The code above loads the news/year_archive.html template. Django has a template search path, which allows you to minimize redundancy among templates. In your Django settings, you specify a list of directories to check for templates. If a template doesn’t exist in the first directory, it checks the second, and so on. Let’s say the news/year_archive.html template was found. Here’s what that might look like: {% extends "base.html" %} {% block title %}Articles for {{ year }}{% endblock %} {% block content %} <h1>Articles for {{ year }}</h1> 10 Chapter 2. Getting started
- 15. Django Documentation, Release1.5.1 {% for article in article_list %} <p>{{ article.headline }}</p> <p>By {{ article.reporter.full_name }}</p> <p>Published {{ article.pub_date|date:"F j, Y" }}</p> {% endfor %} {% endblock %} Variables are surrounded by double-curly braces. {{ article.headline }} means “Output the value of the article’s headline attribute.” But dots aren’t used only for attribute lookup: They also can do dictionary-key lookup, index lookup and function calls. Note {{ article.pub_date|date:"F j, Y" }} uses a Unix-style “pipe” (the “|” character). This is called a template filter, and it’s a way to filter the value of a variable. In this case, the date filter formats a Python datetime object in the given format (as found in PHP’s date function). You can chain together as many filters as you’d like. You can write custom filters. You can write custom template tags, which run custom Python code behind the scenes. Finally, Django uses the concept of “template inheritance”: That’s what the {% extends "base.html" %} does. It means “First load the template called ‘base’, which has defined a bunch of blocks, and fill the blocks with the following blocks.” In short, that lets you dramatically cut down on redundancy in templates: each template has to define only what’s unique to that template. Here’s what the “base.html” template, including the use of static files, might look like: {% load staticfiles %} <html> <head> <title>{% block title %}{% endblock %}</title> </head> <body> <img src="{% static "images/sitelogo.png" %}" alt="Logo" /> {% block content %}{% endblock %} </body> </html> Simplistically, it defines the look-and-feel of the site (with the site’s logo), and provides “holes” for child templates to fill. This makes a site redesign as easy as changing a single file – the base template. It also lets you create multiple versions of a site, with different base templates, while reusing child templates. Django’s creators have used this technique to create strikingly different cell-phone editions of sites – simply by creating a new base template. Note that you don’t have to use Django’s template system if you prefer another system. While Django’s template system is particularly well-integrated with Django’s model layer, nothing forces you to use it. For that matter, you don’t have to use Django’s database API, either. You can use another database abstraction layer, you can read XML files, you can read files off disk, or anything you want. Each piece of Django – models, views, templates – is decoupled from the next. 2.1.8 This is just the surface This has been only a quick overview of Django’s functionality. Some more useful features: • A caching framework that integrates with memcached or other backends. • A syndication framework that makes creating RSS and Atom feeds as easy as writing a small Python class. • More sexy automatically-generated admin features – this overview barely scratched the surface. 2.1. Django at a glance 11
- 16. Django Documentation, Release1.5.1 The next obvious steps are for you to download Django, read the tutorial and join the community. Thanks for your interest! 2.2 Quick install guide Before you can use Django, you’ll need to get it installed. We have a complete installation guide that covers all the possibilities; this guide will guide you to a simple, minimal installation that’ll work while you walk through the introduction. 2.2.1 Install Python Being a Python Web framework, Django requires Python. It works with any Python version from 2.6.5 to 2.7. It also features experimental support for versions 3.2 and 3.3. All these versions of Python include a lightweight database called SQLite so you won’t need to set up a database just yet. Get Python at http://www.python.org. If you’re running Linux or Mac OS X, you probably already have it installed. Django on Jython If you use Jython (a Python implementation for the Java platform), you’ll need to follow a few additional steps. See Running Django on Jython for details. You can verify that Python is installed by typing python from your shell; you should see something like: Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 2.2.2 Set up a database This step is only necessary if you’d like to work with a “large” database engine like PostgreSQL, MySQL, or Oracle. To install such a database, consult the database installation information. 2.2.3 Remove any old versions of Django If you are upgrading your installation of Django from a previous version, you will need to uninstall the old Django version before installing the new version. 2.2.4 Install Django You’ve got three easy options to install Django: • Install a version of Django provided by your operating system distribution. This is the quickest option for those who have operating systems that distribute Django. • Install an official release. This is the best approach for users who want a stable version number and aren’t concerned about running a slightly older version of Django. • Install the latest development version. This is best for users who want the latest-and-greatest features and aren’t afraid of running brand-new code. 12 Chapter 2. Getting started
- 17. Django Documentation, Release1.5.1 Always refer to the documentation that corresponds to the version of Django you’re using! If you do either of the first two steps, keep an eye out for parts of the documentation marked new in development version. That phrase flags features that are only available in development versions of Django, and they likely won’t work with an official release. 2.2.5 Verifying To verify that Django can be seen by Python, type python from your shell. Then at the Python prompt, try to import Django: >>> import django >>> print(django.get_version()) 1.5 You may have another version of Django installed. 2.2.6 That’s it! That’s it – you can now move onto the tutorial. 2.3 Writing your first Django app, part 1 Let’s learn by example. Throughout this tutorial, we’ll walk you through the creation of a basic poll application. It’ll consist of two parts: • A public site that lets people view polls and vote in them. • An admin site that lets you add, change and delete polls. We’ll assume you have Django installed already. You can tell Django is installed and which version by running the following command: python -c "import django; print(django.get_version())" If Django is installed, you should see the version of your installation. If it isn’t, you’ll get an error telling “No module named django”. This tutorial is written for Django 1.5 and Python 2.x. If the Django version doesn’t match, you can refer to the tutorial for your version of Django or update Django to the newest version. If you are using Python 3.x, be aware that your code may need to differ from what is in the tutorial and you should continue using the tutorial only if you know what you are doing with Python 3.x. See How to install Django for advice on how to remove older versions of Django and install a newer one. Where to get help: If you’re having trouble going through this tutorial, please post a message to django-users or drop by #django on irc.freenode.net to chat with other Django users who might be able to help. 2.3. Writing your first Django app, part 1 13
- 18. Django Documentation, Release1.5.1 2.3.1 Creating a project If this is your first time using Django, you’ll have to take care of some initial setup. Namely, you’ll need to auto-generate some code that establishes a Django project – a collection of settings for an instance of Django, including database configuration, Django-specific options and application-specific settings. From the command line, cd into a directory where you’d like to store your code, then run the following command: django-admin.py startproject mysite This will create a mysite directory in your current directory. If it didn’t work, see Problems running django-admin. py. Note: You’ll need to avoid naming projects after built-in Python or Django components. In particular, this means you should avoid using names like django (which will conflict with Django itself) or test (which conflicts with a built-in Python package). Where should this code live? If your background is in plain old PHP (with no use of modern frameworks), you’re probably used to putting code under the Web server’s document root (in a place such as /var/www). With Django, you don’t do that. It’s not a good idea to put any of this Python code within your Web server’s document root, because it risks the possibility that people may be able to view your code over the Web. That’s not good for security. Put your code in some directory outside of the document root, such as /home/mycode. Let’s look at what startproject created: mysite/ manage.py mysite/ __init__.py settings.py urls.py wsgi.py Doesn’t match what you see? The default project layout recently changed. If you’re seeing a “flat” layout (with no inner mysite/ directory), you’re probably using a version of Django that doesn’t match this tutorial version. You’ll want to either switch to the older tutorial or the newer Django version. These files are: • The outer mysite/ directory is just a container for your project. Its name doesn’t matter to Django; you can rename it to anything you like. • manage.py: A command-line utility that lets you interact with this Django project in various ways. You can read all the details about manage.py in django-admin.py and manage.py. • The inner mysite/ directory is the actual Python package for your project. Its name is the Python package name you’ll need to use to import anything inside it (e.g. import mysite.settings). • mysite/__init__.py: An empty file that tells Python that this directory should be considered a Python package. (Read more about packages in the official Python docs if you’re a Python beginner.) 14 Chapter 2. Getting started
- 19. Django Documentation, Release1.5.1 • mysite/settings.py: Settings/configuration for this Django project. Django settings will tell you all about how settings work. • mysite/urls.py: The URL declarations for this Django project; a “table of contents” of your Django-powered site. You can read more about URLs in URL dispatcher. • mysite/wsgi.py: An entry-point for WSGI-compatible webservers to serve your project. See How to deploy with WSGI for more details. The development server Let’s verify this worked. Change into the outer mysite directory, if you haven’t already, and run the command python manage.py runserver. You’ll see the following output on the command line: Validating models... 0 errors found May 01, 2013 - 15:50:53 Django version 1.5, using settings ’mysite.settings’ Development server is running at http://127.0.0.1:8000/ Quit the server with CONTROL-C. You’ve started the Django development server, a lightweightWeb server written purely in Python. We’ve included this with Django so you can develop things rapidly, without having to deal with configuring a production server – such as Apache – until you’re ready for production. Now’s a good time to note: Don’t use this server in anything resembling a production environment. It’s intended only for use while developing. (We’re in the business of making Web frameworks, not Web servers.) Now that the server’s running, visit http://127.0.0.1:8000/ with your Web browser. You’ll see a “Welcome to Django” page, in pleasant, light-blue pastel. It worked! Changing the port By default, the runserver command starts the development server on the internal IP at port 8000. If you want to change the server’s port, pass it as a command-line argument. For instance, this command starts the server on port 8080: python manage.py runserver 8080 If you want to change the server’s IP, pass it along with the port. So to listen on all public IPs (useful if you want to show off your work on other computers), use: python manage.py runserver 0.0.0.0:8000 Full docs for the development server can be found in the runserver reference. Database setup Now, edit mysite/settings.py. It’s a normal Python module with module-level variables representing Django settings. Change the following keys in the DATABASES ’default’ item to match your database connection settings. • ENGINE – Either ’django.db.backends.postgresql_psycopg2’, ’django.db.backends.mysql’, ’django.db.backends.sqlite3’ or ’django.db.backends.oracle’. Other backends are also available. 2.3. Writing your first Django app, part 1 15
- 20. Django Documentation, Release1.5.1 • NAME – The name of your database. If you’re using SQLite, the database will be a file on your computer; in that case, NAME should be the full absolute path, including filename, of that file. If the file doesn’t exist, it will automatically be created when you synchronize the database for the first time (see below). When specifying the path, always use forward slashes, even on Windows (e.g. C:/homes/user/mysite/sqlite3.db). • USER – Your database username (not used for SQLite). • PASSWORD – Your database password (not used for SQLite). • HOST – The host your database is on. Leave this as an empty string (or possibly 127.0.0.1) if your database server is on the same physical machine (not used for SQLite). See HOST for details. If you’re new to databases, we recommend simply using SQLite by setting ENGINE to ’django.db.backends.sqlite3’ and NAME to the place where you’d like to store the database. SQLite is included in Python, so you won’t need to install anything else to support your database. Note: If you’re using PostgreSQL or MySQL, make sure you’ve created a database by this point. Do that with “CREATE DATABASE database_name;” within your database’s interactive prompt. If you’re using SQLite, you don’t need to create anything beforehand - the database file will be created automatically when it is needed. While you’re editing settings.py, set TIME_ZONE to your time zone. The default value is the Central time zone in the U.S. (Chicago). Also, note the INSTALLED_APPS setting toward the bottom of the file. That holds the names of all Django appli-cations that are activated in this Django instance. Apps can be used in multiple projects, and you can package and distribute them for use by others in their projects. By default, INSTALLED_APPS contains the following apps, all of which come with Django: • django.contrib.auth – An authentication system. • django.contrib.contenttypes – A framework for content types. • django.contrib.sessions – A session framework. • django.contrib.sites – A framework for managing multiple sites with one Django installation. • django.contrib.messages – A messaging framework. • django.contrib.staticfiles – A framework for managing static files. These applications are included by default as a convenience for the common case. Each of these applications makes use of at least one database table, though, so we need to create the tables in the database before we can use them. To do that, run the following command: python manage.py syncdb The syncdb command looks at the INSTALLED_APPS setting and creates any necessary database tables according to the database settings in your mysite/settings.py file. You’ll see a message for each database table it creates, and you’ll get a prompt asking you if you’d like to create a superuser account for the authentication system. Go ahead and do that. If you’re interested, run the command-line client for your database and type dt (PostgreSQL), SHOW TABLES; (MySQL), or .schema (SQLite) to display the tables Django created. For the minimalists 16 Chapter 2. Getting started
- 21. Django Documentation, Release1.5.1 Like we said above, the default applications are included for the common case, but not everybody needs them. If you don’t need any or all of them, feel free to comment-out or delete the appropriate line(s) from INSTALLED_APPS before running syncdb. The syncdb command will only create tables for apps in INSTALLED_APPS. 2.3.2 Creating models Now that your environment – a “project” – is set up, you’re set to start doing work. Each application you write in Django consists of a Python package, somewhere on your Python path, that follows a certain convention. Django comes with a utility that automatically generates the basic directory structure of an app, so you can focus on writing code rather than creating directories. Projects vs. apps What’s the difference between a project and an app? An app is aWeb application that does something – e.g., aWeblog system, a database of public records or a simple poll app. A project is a collection of configuration and apps for a particular Web site. A project can contain multiple apps. An app can be in multiple projects. Your apps can live anywhere on your Python path. In this tutorial, we’ll create our poll app right next to your manage.py file so that it can be imported as its own top-level module, rather than a submodule of mysite. To create your app, make sure you’re in the same directory as manage.py and type this command: python manage.py startapp polls That’ll create a directory polls, which is laid out like this: polls/ __init__.py models.py tests.py views.py This directory structure will house the poll application. The first step in writing a database Web app in Django is to define your models – essentially, your database layout, with additional metadata. Philosophy A model is the single, definitive source of data about your data. It contains the essential fields and behaviors of the data you’re storing. Django follows the DRY Principle. The goal is to define your data model in one place and automatically derive things from it. In our simple poll app, we’ll create two models: Poll and Choice. A Poll has a question and a publication date. A Choice has two fields: the text of the choice and a vote tally. Each Choice is associated with a Poll. These concepts are represented by simple Python classes. Edit the polls/models.py file so it looks like this: from django.db import models class Poll(models.Model): question = models.CharField(max_length=200) pub_date = models.DateTimeField(’date published’) class Choice(models.Model): 2.3. Writing your first Django app, part 1 17
- 22. Django Documentation, Release1.5.1 poll = models.ForeignKey(Poll) choice_text = models.CharField(max_length=200) votes = models.IntegerField(default=0) The code is straightforward. Each model is represented by a class that subclasses django.db.models.Model. Each model has a number of class variables, each of which represents a database field in the model. Each field is represented by an instance of a Field class – e.g., CharField for character fields and DateTimeField for datetimes. This tells Django what type of data each field holds. The name of each Field instance (e.g. question or pub_date ) is the field’s name, in machine-friendly format. You’ll use this value in your Python code, and your database will use it as the column name. You can use an optional first positional argument to a Field to designate a human-readable name. That’s used in a couple of introspective parts of Django, and it doubles as documentation. If this field isn’t provided, Django will use the machine-readable name. In this example, we’ve only defined a human-readable name for Poll.pub_date. For all other fields in this model, the field’s machine-readable name will suffice as its human-readable name. Some Field classes have required arguments. CharField, for example, requires that you give it a max_length. That’s used not only in the database schema, but in validation, as we’ll soon see. A Field can also have various optional arguments; in this case, we’ve set the default value of votes to 0. Finally, note a relationship is defined, using ForeignKey. That tells Django each Choice is related to a single Poll. Django supports all the common database relationships: many-to-ones, many-to-manys and one-to-ones. 2.3.3 Activating models That small bit of model code gives Django a lot of information. With it, Django is able to: • Create a database schema (CREATE TABLE statements) for this app. • Create a Python database-access API for accessing Poll and Choice objects. But first we need to tell our project that the polls app is installed. Philosophy Django apps are “pluggable”: You can use an app in multiple projects, and you can distribute apps, because they don’t have to be tied to a given Django installation. Edit the settings.py file again, and change the INSTALLED_APPS setting to include the string ’polls’. So it’ll look like this: INSTALLED_APPS = ( ’django.contrib.auth’, ’django.contrib.contenttypes’, ’django.contrib.sessions’, ’django.contrib.sites’, ’django.contrib.messages’, ’django.contrib.staticfiles’, # Uncomment the next line to enable the admin: # ’django.contrib.admin’, # Uncomment the next line to enable admin documentation: # ’django.contrib.admindocs’, ’polls’, ) Now Django knows to include the polls app. Let’s run another command: 18 Chapter 2. Getting started
- 23. Django Documentation, Release1.5.1 python manage.py sql polls You should see something similar to the following (the CREATE TABLE SQL statements for the polls app): BEGIN; CREATE TABLE "polls_poll" ( "id" serial NOT NULL PRIMARY KEY, "question" varchar(200) NOT NULL, "pub_date" timestamp with time zone NOT NULL ); CREATE TABLE "polls_choice" ( "id" serial NOT NULL PRIMARY KEY, "poll_id" integer NOT NULL REFERENCES "polls_poll" ("id") DEFERRABLE INITIALLY DEFERRED, "choice_text" varchar(200) NOT NULL, "votes" integer NOT NULL ); COMMIT; Note the following: • The exact output will vary depending on the database you are using. • Table names are automatically generated by combining the name of the app (polls) and the lowercase name of the model – poll and choice. (You can override this behavior.) • Primary keys (IDs) are added automatically. (You can override this, too.) • By convention, Django appends "_id" to the foreign key field name. (Yes, you can override this, as well.) • The foreign key relationship is made explicit by a REFERENCES statement. • It’s tailored to the database you’re using, so database-specific field types such as auto_increment (MySQL), serial (PostgreSQL), or integer primary key (SQLite) are handled for you automatically. Same goes for quoting of field names – e.g., using double quotes or single quotes. The author of this tutorial runs Post-greSQL, so the example output is in PostgreSQL syntax. • The sql command doesn’t actually run the SQL in your database - it just prints it to the screen so that you can see what SQL Django thinks is required. If you wanted to, you could copy and paste this SQL into your database prompt. However, as we will see shortly, Django provides an easier way of committing the SQL to the database. If you’re interested, also run the following commands: • python manage.py validate – Checks for any errors in the construction of your models. • python manage.py sqlcustom polls – Outputs any custom SQL statements (such as table modifica-tions or constraints) that are defined for the application. • python manage.py sqlclear polls – Outputs the necessary DROP TABLE statements for this app, according to which tables already exist in your database (if any). • python manage.py sqlindexes polls – Outputs the CREATE INDEX statements for this app. • python manage.py sqlall polls – A combination of all the SQL from the sql, sqlcustom, and sqlindexes commands. Looking at the output of those commands can help you understand what’s actually happening under the hood. Now, run syncdb again to create those model tables in your database: python manage.py syncdb The syncdb command runs the SQL from sqlall on your database for all apps in INSTALLED_APPS that don’t already exist in your database. This creates all the tables, initial data and indexes for any apps you’ve added to your 2.3. Writing your first Django app, part 1 19
- 24. Django Documentation, Release1.5.1 project since the last time you ran syncdb. syncdb can be called as often as you like, and it will only ever create the tables that don’t exist. Read the django-admin.py documentation for full information on what the manage.py utility can do. 2.3.4 Playing with the API Now, let’s hop into the interactive Python shell and play around with the free API Django gives you. To invoke the Python shell, use this command: python manage.py shell We’re using this instead of simply typing “python”, because manage.py sets the DJANGO_SETTINGS_MODULE environment variable, which gives Django the Python import path to your mysite/settings.py file. Bypassing manage.py If you’d rather not use manage.py, no problem. Just set the DJANGO_SETTINGS_MODULE environment variable to mysite.settings and run python from the same directory manage.py is in (or ensure that directory is on the Python path, so that import mysite works). For more information on all of this, see the django-admin.py documentation. Once you’re in the shell, explore the database API: >>> from polls.models import Poll, Choice # Import the model classes we just wrote. # No polls are in the system yet. >>> Poll.objects.all() [] # Create a new Poll. # Support for time zones is enabled in the default settings file, so # Django expects a datetime with tzinfo for pub_date. Use timezone.now() # instead of datetime.datetime.now() and it will do the right thing. >>> from django.utils import timezone >>> p = Poll(question="What’s new?", pub_date=timezone.now()) # Save the object into the database. You have to call save() explicitly. >>> p.save() # Now it has an ID. Note that this might say "1L" instead of "1", depending # on which database you’re using. That’s no biggie; it just means your # database backend prefers to return integers as Python long integer # objects. >>> p.id 1 # Access database columns via Python attributes. >>> p.question "What’s new?" >>> p.pub_date datetime.datetime(2012, 2, 26, 13, 0, 0, 775217, tzinfo=<UTC>) # Change values by changing the attributes, then calling save(). >>> p.question = "What’s up?" >>> p.save() 20 Chapter 2. Getting started
- 25. Django Documentation, Release1.5.1 # objects.all() displays all the polls in the database. >>> Poll.objects.all() [<Poll: Poll object>] Wait a minute. <Poll: Poll object> is, utterly, an unhelpful representation of this object. Let’s fix that by editing the polls model (in the polls/models.py file) and adding a __unicode__() method to both Poll and Choice: class Poll(models.Model): # ... def __unicode__(self): return self.question class Choice(models.Model): # ... def __unicode__(self): return self.choice_text It’s important to add __unicode__() methods to your models, not only for your own sanity when dealing with the interactive prompt, but also because objects’ representations are used throughout Django’s automatically-generated admin. Why __unicode__() and not __str__()? If you’re familiar with Python, you might be in the habit of adding __str__() methods to your classes, not __unicode__() methods. We use __unicode__() here because Django models deal with Unicode by default. All data stored in your database is converted to Unicode when it’s returned. Django models have a default __str__() method that calls __unicode__() and converts the result to a UTF-8 bytestring. This means that unicode(p) will return a Unicode string, and str(p) will return a normal string, with characters encoded as UTF-8. If all of this is gibberish to you, just remember to add __unicode__() methods to your models. With any luck, things should Just Work for you. Note these are normal Python methods. Let’s add a custom method, just for demonstration: import datetime from django.utils import timezone # ... class Poll(models.Model): # ... def was_published_recently(self): return self.pub_date >= timezone.now() - datetime.timedelta(days=1) Note the addition of import datetime and from django.utils import timezone, to reference Python’s standard datetime module and Django’s time-zone-related utilities in django.utils.timezone, respectively. If you aren’t familiar with time zone handling in Python, you can learn more in the time zone support docs. Save these changes and start a new Python interactive shell by running python manage.py shell again: >>> from polls.models import Poll, Choice # Make sure our __unicode__() addition worked. >>> Poll.objects.all() [<Poll: What’s up?>] 2.3. Writing your first Django app, part 1 21
- 26. Django Documentation, Release1.5.1 # Django provides a rich database lookup API that’s entirely driven by # keyword arguments. >>> Poll.objects.filter(id=1) [<Poll: What’s up?>] >>> Poll.objects.filter(question__startswith=’What’) [<Poll: What’s up?>] # Get the poll that was published this year. >>> from django.utils import timezone >>> current_year = timezone.now().year >>> Poll.objects.get(pub_date__year=current_year) <Poll: What’s up?> # Request an ID that doesn’t exist, this will raise an exception. >>> Poll.objects.get(id=2) Traceback (most recent call last): ... DoesNotExist: Poll matching query does not exist. Lookup parameters were {’id’: 2} # Lookup by a primary key is the most common case, so Django provides a # shortcut for primary-key exact lookups. # The following is identical to Poll.objects.get(id=1). >>> Poll.objects.get(pk=1) <Poll: What’s up?> # Make sure our custom method worked. >>> p = Poll.objects.get(pk=1) >>> p.was_published_recently() True # Give the Poll a couple of Choices. The create call constructs a new # Choice object, does the INSERT statement, adds the choice to the set # of available choices and returns the new Choice object. Django creates # a set to hold the "other side" of a ForeignKey relation # (e.g. a poll’s choices) which can be accessed via the API. >>> p = Poll.objects.get(pk=1) # Display any choices from the related object set -- none so far. >>> p.choice_set.all() [] # Create three choices. >>> p.choice_set.create(choice_text=’Not much’, votes=0) <Choice: Not much> >>> p.choice_set.create(choice_text=’The sky’, votes=0) <Choice: The sky> >>> c = p.choice_set.create(choice_text=’Just hacking again’, votes=0) # Choice objects have API access to their related Poll objects. >>> c.poll <Poll: What’s up?> # And vice versa: Poll objects get access to Choice objects. >>> p.choice_set.all() [<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>] >>> p.choice_set.count() 3 22 Chapter 2. Getting started
- 27. Django Documentation, Release1.5.1 # The API automatically follows relationships as far as you need. # Use double underscores to separate relationships. # This works as many levels deep as you want; there’s no limit. # Find all Choices for any poll whose pub_date is in this year # (reusing the ’current_year’ variable we created above). >>> Choice.objects.filter(poll__pub_date__year=current_year) [<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>] # Let’s delete one of the choices. Use delete() for that. >>> c = p.choice_set.filter(choice_text__startswith=’Just hacking’) >>> c.delete() For more information on model relations, see Accessing related objects. For more on how to use double underscores to perform field lookups via the API, see Field lookups. For full details on the database API, see our Database API reference. When you’re comfortable with the API, read part 2 of this tutorial to get Django’s automatic admin working. 2.4 Writing your first Django app, part 2 This tutorial begins where Tutorial 1 left off. We’re continuing the Web-poll application and will focus on Django’s automatically-generated admin site. Philosophy Generating admin sites for your staff or clients to add, change and delete content is tedious work that doesn’t require much creativity. For that reason, Django entirely automates creation of admin interfaces for models. Django was written in a newsroom environment, with a very clear separation between “content publishers” and the “public” site. Site managers use the system to add news stories, events, sports scores, etc., and that content is displayed on the public site. Django solves the problem of creating a unified interface for site administrators to edit content. The admin isn’t intended to be used by site visitors. It’s for site managers. 2.4.1 Activate the admin site The Django admin site is not activated by default – it’s an opt-in thing. To activate the admin site for your installation, do these three things: • Uncomment "django.contrib.admin" in the INSTALLED_APPS setting. • Run python manage.py syncdb. Since you have added a new application to INSTALLED_APPS, the database tables need to be updated. • Edit your mysite/urls.py file and uncomment the lines that reference the admin – there are three lines in total to uncomment. This file is a URLconf; we’ll dig into URLconfs in the next tutorial. For now, all you need to know is that it maps URL roots to applications. In the end, you should have a urls.py file that looks like this: from django.conf.urls import patterns, include, url # Uncomment the next two lines to enable the admin: from django.contrib import admin admin.autodiscover() 2.4. Writing your first Django app, part 2 23
- 28. Django Documentation, Release1.5.1 urlpatterns = patterns(’’, # Examples: # url(r’^$’, ’{{ project_name }}.views.home’, name=’home’), # url(r’^{{ project_name }}/’, include(’{{ project_name }}.foo.urls’)), # Uncomment the admin/doc line below to enable admin documentation: # url(r’^admin/doc/’, include(’django.contrib.admindocs.urls’)), # Uncomment the next line to enable the admin: url(r’^admin/’, include(admin.site.urls)), ) (The bold lines are the ones that needed to be uncommented.) 2.4.2 Start the development server Let’s start the development server and explore the admin site. Recall from Tutorial 1 that you start the development server like so: python manage.py runserver Now, open a Web browser and go to “/admin/” on your local domain – e.g., http://127.0.0.1:8000/admin/. You should see the admin’s login screen: Doesn’t match what you see? If at this point, instead of the above login page, you get an error page reporting something like: ImportError at /admin/ cannot import name patterns ... then you’re probably using a version of Django that doesn’t match this tutorial version. You’ll want to either switch to the older tutorial or the newer Django version. 24 Chapter 2. Getting started
- 29. Django Documentation, Release1.5.1 2.4.3 Enter the admin site Now, try logging in. (You created a superuser account in the first part of this tutorial, remember? If you didn’t create one or forgot the password you can create another one.) You should see the Django admin index page: You should see a few types of editable content, including groups, users and sites. These are core features Django ships with by default. 2.4.4 Make the poll app modifiable in the admin But where’s our poll app? It’s not displayed on the admin index page. Just one thing to do: we need to tell the admin that Poll objects have an admin interface. To do this, open the polls/admin.py file, and edit it to look like this: from django.contrib import admin from polls.models import Poll admin.site.register(Poll) You’ll need to restart the development server to see your changes. Normally, the server auto-reloads code every time you modify a file, but the action of creating a new file doesn’t trigger the auto-reloading logic. 2.4.5 Explore the free admin functionality Now that we’ve registered Poll, Django knows that it should be displayed on the admin index page: Click “Polls.” Now you’re at the “change list” page for polls. This page displays all the polls in the database and lets you choose one to change it. There’s the “What’s up?” poll we created in the first tutorial: 2.4. Writing your first Django app, part 2 25
- 30. Django Documentation, Release1.5.1 Click the “What’s up?” poll to edit it: Things to note here: • The form is automatically generated from the Poll model. • The different model field types (DateTimeField, CharField) correspond to the appropriate HTML input widget. Each type of field knows how to display itself in the Django admin. • Each DateTimeField gets free JavaScript shortcuts. Dates get a “Today” shortcut and calendar popup, and times get a “Now” shortcut and a convenient popup that lists commonly entered times. The bottom part of the page gives you a couple of options: • Save – Saves changes and returns to the change-list page for this type of object. • Save and continue editing – Saves changes and reloads the admin page for this object. • Save and add another – Saves changes and loads a new, blank form for this type of object. • Delete – Displays a delete confirmation page. If the value of “Date published” doesn’t match the time when you created the poll in Tutorial 1, it probably means you forgot to set the correct value for the TIME_ZONE setting. Change it, reload the page and check that the correct value appears. Change the “Date published” by clicking the “Today” and “Now” shortcuts. Then click “Save and continue editing.” Then click “History” in the upper right. You’ll see a page listing all changes made to this object via the Django admin, with the timestamp and username of the person who made the change: 26 Chapter 2. Getting started
- 31. Django Documentation, Release1.5.1 2.4.6 Customize the admin form Take a few minutes to marvel at all the code you didn’t have to write. By registering the Poll model with admin.site.register(Poll), Django was able to construct a default form representation. Often, you’ll want to customize how the admin form looks and works. You’ll do this by telling Django the options you want when you register the object. Let’s see how this works by re-ordering the fields on the edit form. Replace the admin.site.register(Poll) line with: class PollAdmin(admin.ModelAdmin): fields = [’pub_date’, ’question’] admin.site.register(Poll, PollAdmin) You’ll follow this pattern – create a model admin object, then pass it as the second argument to admin.site.register() – any time you need to change the admin options for an object. This particular change above makes the “Publication date” come before the “Question” field: This isn’t impressive with only two fields, but for admin forms with dozens of fields, choosing an intuitive order is an important usability detail. And speaking of forms with dozens of fields, you might want to split the form up into fieldsets: 2.4. Writing your first Django app, part 2 27
- 32. Django Documentation, Release1.5.1 class PollAdmin(admin.ModelAdmin): fieldsets = [ (None, {’fields’: [’question’]}), (’Date information’, {’fields’: [’pub_date’]}), ] admin.site.register(Poll, PollAdmin) The first element of each tuple in fieldsets is the title of the fieldset. Here’s what our form looks like now: You can assign arbitrary HTML classes to each fieldset. Django provides a "collapse" class that displays a particular fieldset initially collapsed. This is useful when you have a long form that contains a number of fields that aren’t commonly used: class PollAdmin(admin.ModelAdmin): fieldsets = [ (None, {’fields’: [’question’]}), (’Date information’, {’fields’: [’pub_date’], ’classes’: [’collapse’]}), ] 28 Chapter 2. Getting started
- 33. Django Documentation, Release1.5.1 2.4.7 Adding related objects OK, we have our Poll admin page. But a Poll has multiple Choices, and the admin page doesn’t display choices. Yet. There are two ways to solve this problem. The first is to register Choice with the admin just as we did with Poll. That’s easy: from polls.models import Choice admin.site.register(Choice) Now “Choices” is an available option in the Django admin. The “Add choice” form looks like this: In that form, the “Poll” field is a select box containing every poll in the database. Django knows that a ForeignKey should be represented in the admin as a <select> box. In our case, only one poll exists at this point. Also note the “Add Another” link next to “Poll.” Every object with a ForeignKey relationship to another gets this for free. When you click “Add Another,” you’ll get a popup window with the “Add poll” form. If you add a poll in that window and click “Save,” Django will save the poll to the database and dynamically add it as the selected choice on the “Add choice” form you’re looking at. But, really, this is an inefficient way of adding Choice objects to the system. It’d be better if you could add a bunch of Choices directly when you create the Poll object. Let’s make that happen. Remove the register() call for the Choice model. Then, edit the Poll registration code to read: from django.contrib import admin from polls.models import Choice, Poll class ChoiceInline(admin.StackedInline): model = Choice extra = 3 2.4. Writing your first Django app, part 2 29
- 34. Django Documentation, Release1.5.1 class PollAdmin(admin.ModelAdmin): fieldsets = [ (None, {’fields’: [’question’]}), (’Date information’, {’fields’: [’pub_date’], ’classes’: [’collapse’]}), ] inlines = [ChoiceInline] admin.site.register(Poll, PollAdmin) This tells Django: “Choice objects are edited on the Poll admin page. By default, provide enough fields for 3 choices.” Load the “Add poll” page to see how that looks, you may need to restart your development server: It works like this: There are three slots for related Choices – as specified by extra – and each time you come back to the “Change” page for an already-created object, you get another three extra slots. At the end of the three current slots you will find an “Add another Choice” link. If you click on it, a new slot will be added. If you want to remove the added slot, you can click on the X to the top right of the added slot. Note that you can’t remove the original three slots. This image shows an added slot: 30 Chapter 2. Getting started
- 35. Django Documentation, Release1.5.1 One small problem, though. It takes a lot of screen space to display all the fields for entering related Choice objects. For that reason, Django offers a tabular way of displaying inline related objects; you just need to change the ChoiceInline declaration to read: class ChoiceInline(admin.TabularInline): #... With that TabularInline (instead of StackedInline), the related objects are displayed in a more compact, table-based format: Note that there is an extra “Delete?” column that allows removing rows added using the “Add Another Choice” button and rows that have already been saved. 2.4.8 Customize the admin change list Now that the Poll admin page is looking good, let’s make some tweaks to the “change list” page – the one that displays all the polls in the system. Here’s what it looks like at this point: 2.4. Writing your first Django app, part 2 31
- 36. Django Documentation, Release1.5.1 By default, Django displays the str() of each object. But sometimes it’d be more helpful if we could display individual fields. To do that, use the list_display admin option, which is a tuple of field names to display, as columns, on the change list page for the object: class PollAdmin(admin.ModelAdmin): # ... list_display = (’question’, ’pub_date’) Just for good measure, let’s also include the was_published_recently custom method from Tutorial 1: class PollAdmin(admin.ModelAdmin): # ... list_display = (’question’, ’pub_date’, ’was_published_recently’) Now the poll change list page looks like this: You can click on the column headers to sort by those values – except in the case of the was_published_recently header, because sorting by the output of an arbitrary method is not supported. Also note that the column header for was_published_recently is, by default, the name of the method (with underscores replaced with spaces), and that each line contains the string representation of the output. You can improve that by giving that method (in polls/models.py) a few attributes, as follows: class Poll(models.Model): # ... def was_published_recently(self): return self.pub_date >= timezone.now() - datetime.timedelta(days=1) was_published_recently.admin_order_field = ’pub_date’ was_published_recently.boolean = True was_published_recently.short_description = ’Published recently?’ Edit your polls/admin.py file again and add an improvement to the Poll change list page: Filters. Add the following line to PollAdmin: list_filter = [’pub_date’] 32 Chapter 2. Getting started
- 37. Django Documentation, Release1.5.1 That adds a “Filter” sidebar that lets people filter the change list by the pub_date field: The type of filter displayed depends on the type of field you’re filtering on. Because pub_date is a DateTimeField, Django knows to give appropriate filter options: “Any date,” “Today,” “Past 7 days,” “This month,” “This year.” This is shaping up well. Let’s add some search capability: search_fields = [’question’] That adds a search box at the top of the change list. When somebody enters search terms, Django will search the question field. You can use as many fields as you’d like – although because it uses a LIKE query behind the scenes, keep it reasonable, to keep your database happy. Finally, because Poll objects have dates, it’d be convenient to be able to drill down by date. Add this line: date_hierarchy = ’pub_date’ That adds hierarchical navigation, by date, to the top of the change list page. At top level, it displays all available years. Then it drills down to months and, ultimately, days. Now’s also a good time to note that change lists give you free pagination. The default is to display 100 items per page. Change-list pagination, search boxes, filters, date-hierarchies and column-header-ordering all work together like you think they should. 2.4.9 Customize the admin look and feel Clearly, having “Django administration” at the top of each admin page is ridiculous. It’s just placeholder text. That’s easy to change, though, using Django’s template system. The Django admin is powered by Django itself, and its interfaces use Django’s own template system. Customizing your project’s templates Create a templates directory in your project directory. Templates can live anywhere on your filesystem that Django can access. (Django runs as whatever user your server runs.) However, keeping your templates within the project is a good convention to follow. Open your settings file (mysite/settings.py, remember) and add a TEMPLATE_DIRS setting: TEMPLATE_DIRS = ( ’/path/to/mysite/templates’, # Change this to your own directory. ) Now copy the template admin/base_site.html from within the default Django admin tem-plate directory in the source code of Django itself (django/contrib/admin/templates) 2.4. Writing your first Django app, part 2 33
- 38. Django Documentation, Release1.5.1 into an admin subdirectory of whichever directory you’re using in TEMPLATE_DIRS. For example, if your TEMPLATE_DIRS includes ’/path/to/mysite/templates’, as above, then copy django/contrib/admin/templates/admin/base_site.html to /path/to/mysite/templates/admin/base_site.html. Don’t forget that admin subdirectory. Where are the Django source files? If you have difficulty finding where the Django source files are located on your system, run the following command: python -c " import sys sys.path = sys.path[1:] import django print(django.__path__)" Then, just edit the file and replace the generic Django text with your own site’s name as you see fit. This template file contains lots of text like {% block branding %} and {{ title }}. The {% and {{ tags are part of Django’s template language. When Django renders admin/base_site.html, this template language will be evaluated to produce the final HTML page. Don’t worry if you can’t make any sense of the template right now – we’ll delve into Django’s templating language in Tutorial 3. Note that any of Django’s default admin templates can be overridden. To override a template, just do the same thing you did with base_site.html – copy it from the default directory into your custom directory, and make changes. Customizing your application’s templates Astute readers will ask: But if TEMPLATE_DIRS was empty by default, how was Django finding the default admin templates? The answer is that, by default, Django automatically looks for a templates/ subdirectory within each application package, for use as a fallback (don’t forget that django.contrib.admin is an application). Our poll application is not very complex and doesn’t need custom admin templates. But if it grew more sophisticated and required modification of Django’s standard admin templates for some of its functionality, it would be more sensible to modify the application’s templates, rather than those in the project. That way, you could include the polls application in any new project and be assured that it would find the custom templates it needed. See the template loader documentation for more information about how Django finds its templates. 2.4.10 Customize the admin index page On a similar note, you might want to customize the look and feel of the Django admin index page. By default, it displays all the apps in INSTALLED_APPS that have been registered with the admin application, in alphabetical order. You may want to make significant changes to the layout. After all, the index is probably the most important page of the admin, and it should be easy to use. The template to customize is admin/index.html. (Do the same as with admin/base_site.html in the previous section – copy it from the default directory to your custom template directory.) Edit the file, and you’ll see it uses a template variable called app_list. That variable contains every installed Django app. Instead of using that, you can hard-code links to object-specific admin pages in whatever way you think is best. Again, don’t worry if you can’t understand the template language – we’ll cover that in more detail in Tutorial 3. When you’re comfortable with the admin site, read part 3 of this tutorial to start working on public poll views. 34 Chapter 2. Getting started
- 39. Django Documentation, Release1.5.1 2.5 Writing your first Django app, part 3 This tutorial begins where Tutorial 2 left off. We’re continuing theWeb-poll application and will focus on creating the public interface – “views.” 2.5.1 Philosophy A view is a “type” of Web page in your Django application that generally serves a specific function and has a specific template. For example, in a blog application, you might have the following views: • Blog homepage – displays the latest few entries. • Entry “detail” page – permalink page for a single entry. • Year-based archive page – displays all months with entries in the given year. • Month-based archive page – displays all days with entries in the given month. • Day-based archive page – displays all entries in the given day. • Comment action – handles posting comments to a given entry. In our poll application, we’ll have the following four views: • Poll “index” page – displays the latest few polls. • Poll “detail” page – displays a poll question, with no results but with a form to vote. • Poll “results” page – displays results for a particular poll. • Vote action – handles voting for a particular choice in a particular poll. In Django, web pages and other content are delivered by views. Each view is represented by a simple Python function (or method, in the case of class-based views). Django will choose a view by examining the URL that’s requested (to be precise, the part of the URL after the domain name). Now in your time on the web you may have come across such beauties as “ME2/Sites/dirmod.asp?sid=&type=gen&mod=Core+Pages&gid=A6CD4967199A42D9B65B1B”. You will be pleased to know that Django allows us much more elegant URL patterns than that. A URL pattern is simply the general form of a URL - for example: /newsarchive/<year>/<month>/. To get from a URL to a view, Django uses what are known as ‘URLconfs’. A URLconf maps URL patterns (described as regular expressions) to views. This tutorial provides basic instruction in the use of URLconfs, and you can refer to django.core.urlresolvers for more information. 2.5.2 Write your first view Let’s write the first view. Open the file polls/views.py and put the following Python code in it: from django.http import HttpResponse def index(request): return HttpResponse("Hello, world. You’re at the poll index.") This is the simplest view possible in Django. To call the view, we need to map it to a URL - and for this we need a URLconf. To create a URLconf in the polls directory, create a file called urls.py. Your app directory should now look like: 2.5. Writing your first Django app, part 3 35
- 40. Django Documentation, Release1.5.1 polls/ __init__.py admin.py models.py tests.py urls.py views.py In the polls/urls.py file include the following code: from django.conf.urls import patterns, url from polls import views urlpatterns = patterns(’’, url(r’^$’, views.index, name=’index’) ) The next step is to point the root URLconf at the polls.urls module. In mysite/urls.py insert an include(), leaving you with: from django.conf.urls import patterns, include, url from django.contrib import admin admin.autodiscover() urlpatterns = patterns(’’, url(r’^polls/’, include(’polls.urls’)), url(r’^admin/’, include(admin.site.urls)), ) You have now wired an index view into the URLconf. Go to http://localhost:8000/polls/ in your browser, and you should see the text “Hello, world. You’re at the poll index.”, which you defined in the index view. The url() function is passed four arguments, two required: regex and view, and two optional: kwargs, and name. At this point, it’s worth reviewing what these arguments are for. url() argument: regex The term “regex” is a commonly used short form meaning “regular expression”, which is a syntax for matching patterns in strings, or in this case, url patterns. Django starts at the first regular expression and makes its way down the list, comparing the requested URL against each regular expression until it finds one that matches. Note that these regular expressions do not search GET and POST parameters, or the domain name. For example, in a request to http://www.example.com/myapp/, the URLconf will look for myapp/. In a request to http://www.example.com/myapp/?page=3, the URLconf will also look for myapp/. If you need help with regular expressions, see Wikipedia’s entry and the documentation of the re module. Also, the O’Reilly book “Mastering Regular Expressions” by Jeffrey Friedl is fantastic. In practice, however, you don’t need to be an expert on regular expressions, as you really only need to know how to capture simple patterns. In fact, complex regexes can have poor lookup performance, so you probably shouldn’t rely on the full power of regexes. Finally, a performance note: these regular expressions are compiled the first time the URLconf module is loaded. They’re super fast (as long as the lookups aren’t too complex as noted above). 36 Chapter 2. Getting started
- 41. Django Documentation, Release1.5.1 url() argument: view When Django finds a regular expression match, Django calls the specified view function, with an HttpRequest object as the first argument and any “captured” values from the regular expression as other arguments. If the regex uses simple captures, values are passed as positional arguments; if it uses named captures, values are passed as keyword arguments. We’ll give an example of this in a bit. url() argument: kwargs Arbitrary keyword arguments can be passed in a dictionary to the target view. We aren’t going to use this feature of Django in the tutorial. url() argument: name Naming your URL lets you refer to it unambiguously from elsewhere in Django especially templates. This powerful feature allows you to make global changes to the url patterns of your project while only touching a single file. 2.5.3 Writing more views Now let’s add a few more views to polls/views.py. These views are slightly different, because they take an argument: def detail(request, poll_id): return HttpResponse("You’re looking at poll %s." % poll_id) def results(request, poll_id): return HttpResponse("You’re looking at the results of poll %s." % poll_id) def vote(request, poll_id): return HttpResponse("You’re voting on poll %s." % poll_id) Wire these news views into the polls.urls module by adding the following url() calls: from django.conf.urls import patterns, url from polls import views urlpatterns = patterns(’’, # ex: /polls/ url(r’^$’, views.index, name=’index’), # ex: /polls/5/ url(r’^(?P<poll_id>d+)/$’, views.detail, name=’detail’), # ex: /polls/5/results/ url(r’^(?P<poll_id>d+)/results/$’, views.results, name=’results’), # ex: /polls/5/vote/ url(r’^(?P<poll_id>d+)/vote/$’, views.vote, name=’vote’), ) Take a look in your browser, at “/polls/34/”. It’ll run the detail() method and display whatever ID you provide in the URL. Try “/polls/34/results/” and “/polls/34/vote/” too – these will display the placeholder results and voting pages. When somebody requests a page from your Web site – say, “/polls/34/”, Django will load the mysite.urls Python module because it’s pointed to by the ROOT_URLCONF setting. It finds the variable named urlpatterns and traverses the regular expressions in order. The include() functions we are using simply reference other URLconfs. 2.5. Writing your first Django app, part 3 37
- 42. Django Documentation, Release1.5.1 Note that the regular expressions for the include() functions don’t have a $ (end-of-string match character) but rather a trailing slash. Whenever Django encounters include(), it chops off whatever part of the URL matched up to that point and sends the remaining string to the included URLconf for further processing. The idea behind include() is to make it easy to plug-and-play URLs. Since polls are in their own URLconf (polls/urls.py), they can be placed under “/polls/”, or under “/fun_polls/”, or under “/content/polls/”, or any other path root, and the app will still work. Here’s what happens if a user goes to “/polls/34/” in this system: • Django will find the match at ’^polls/’ • Then, Django will strip off the matching text ("polls/") and send the remaining text – "34/" – to the ‘polls.urls’ URLconf for further processing which matches r’^(?P<poll_id>d+)/$’ resulting in a call to the detail() view like so: detail(request=<HttpRequest object>, poll_id=’34’) The poll_id=’34’ part comes from (?P<poll_id>d+). Using parentheses around a pattern “captures” the text matched by that pattern and sends it as an argument to the view function; ?P<poll_id> defines the name that will be used to identify the matched pattern; and d+ is a regular expression to match a sequence of digits (i.e., a number). Because the URL patterns are regular expressions, there really is no limit on what you can do with them. And there’s no need to add URL cruft such as .html – unless you want to, in which case you can do something like this: (r’^polls/latest.html$’, ’polls.views.index’), But, don’t do that. It’s silly. 2.5.4 Write views that actually do something Each view is responsible for doing one of two things: returning an HttpResponse object containing the content for the requested page, or raising an exception such as Http404. The rest is up to you. Your view can read records from a database, or not. It can use a template system such as Django’s – or a third-party Python template system – or not. It can generate a PDF file, output XML, create a ZIP file on the fly, anything you want, using whatever Python libraries you want. All Django wants is that HttpResponse. Or an exception. Because it’s convenient, let’s use Django’s own database API, which we covered in Tutorial 1. Here’s one stab at the index() view, which displays the latest 5 poll questions in the system, separated by commas, according to publication date: from django.http import HttpResponse from polls.models import Poll def index(request): latest_poll_list = Poll.objects.order_by(’-pub_date’)[:5] output = ’, ’.join([p.question for p in latest_poll_list]) return HttpResponse(output) There’s a problem here, though: the page’s design is hard-coded in the view. If you want to change the way the page looks, you’ll have to edit this Python code. So let’s use Django’s template system to separate the design from Python by creating a template that the view can use. First, create a directory called templates in your polls directory. Django will look for templates in there. 38 Chapter 2. Getting started
- 43. Django Documentation, Release1.5.1 Django’s TEMPLATE_LOADERS setting contains a list of callables that know how to import templates from various sources. One of the defaults is django.template.loaders.app_directories.Loader which looks for a “templates” subdirectory in each of the INSTALLED_APPS - this is how Django knows to find the polls templates even though we didn’t modify TEMPLATE_DIRS, as we did in Tutorial 2. Organizing templates We could have all our templates together, in one big templates directory, and it would work perfectly well. However, this template belongs to the polls application, so unlike the admin template we created in the previous tutorial, we’ll put this one in the application’s template directory (polls/templates) rather than the project’s (templates). We’ll discuss in more detail in the reusable apps tutorial why we do this. Within the templates directory you have just created, create another directory called polls, and within that create a file called index.html. In other words, your template should be at polls/templates/polls/index.html. Because of how the app_directories template loader works as described above, you can refer to this template within Django simply as polls/index.html. Template namespacing Now we might be able to get away with putting our templates directly in polls/templates (rather than creating another polls subdirectory), but it would actually be a bad idea. Django will choose the first template it finds whose name matches, and if you had a template with the same name in a different application, Django would be unable to distinguish between them. We need to be able to point Django at the right one, and the easiest way to ensure this is by namespacing them. That is, by putting those templates inside another directory named for the application itself. Put the following code in that template: {% if latest_poll_list %} <ul> {% for poll in latest_poll_list %} <li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li> {% endfor %} </ul> {% else %} <p>No polls are available.</p> {% endif %} Now let’s use that html template in our index view: from django.http import HttpResponse from django.template import Context, loader from polls.models import Poll def index(request): latest_poll_list = Poll.objects.order_by(’-pub_date’)[:5] template = loader.get_template(’polls/index.html’) context = Context({ ’latest_poll_list’: latest_poll_list, }) return HttpResponse(template.render(context)) That code loads the template called polls/index.html and passes it a context. The context is a dictionary mapping template variable names to Python objects. Load the page by pointing your browser at “/polls/”, and you should see a bulleted-list containing the “What’s up” poll from Tutorial 1. The link points to the poll’s detail page. 2.5. Writing your first Django app, part 3 39
- 44. Django Documentation, Release1.5.1 A shortcut: render() It’s a very common idiom to load a template, fill a context and return an HttpResponse object with the result of the rendered template. Django provides a shortcut. Here’s the full index() view, rewritten: from django.shortcuts import render from polls.models import Poll def index(request): latest_poll_list = Poll.objects.all().order_by(’-pub_date’)[:5] context = {’latest_poll_list’: latest_poll_list} return render(request, ’polls/index.html’, context) Note that once we’ve done this in all these views, we no longer need to import loader, Context and HttpResponse (you’ll want to keep HttpResponse if you still have the stub methods for detail, results, and vote). The render() function takes the request object as its first argument, a template name as its second argument and a dictionary as its optional third argument. It returns an HttpResponse object of the given template rendered with the given context. 2.5.5 Raising a 404 error Now, let’s tackle the poll detail view – the page that displays the question for a given poll. Here’s the view: from django.http import Http404 # ... def detail(request, poll_id): try: poll = Poll.objects.get(pk=poll_id) except Poll.DoesNotExist: raise Http404 return render(request, ’polls/detail.html’, {’poll’: poll}) The new concept here: The view raises the Http404 exception if a poll with the requested ID doesn’t exist. We’ll discuss what you could put in that polls/detail.html template a bit later, but if you’d like to quickly get the above example working, a file containing just: {{ poll }} will get you started for now. A shortcut: get_object_or_404() It’s a very common idiom to use get() and raise Http404 if the object doesn’t exist. Django provides a shortcut. Here’s the detail() view, rewritten: from django.shortcuts import render, get_object_or_404 # ... def detail(request, poll_id): poll = get_object_or_404(Poll, pk=poll_id) return render(request, ’polls/detail.html’, {’poll’: poll}) 40 Chapter 2. Getting started
- 45. Django Documentation, Release1.5.1 The get_object_or_404() function takes a Django model as its first argument and an arbitrary number of keyword arguments, which it passes to the get() function of the model’s manager. It raises Http404 if the object doesn’t exist. Philosophy Why do we use a helper function get_object_or_404() instead of automatically catching the ObjectDoesNotExist exceptions at a higher level, or having the model API raise Http404 instead of ObjectDoesNotExist? Because that would couple the model layer to the view layer. One of the foremost design goals of Django is to maintain loose coupling. Some controlled coupling is introduced in the django.shortcuts module. There’s also a get_list_or_404() function, which works just as get_object_or_404() – except using filter() instead of get(). It raises Http404 if the list is empty. 2.5.6 Write a 404 (page not found) view When you raise Http404 from within a view, Django will load a special view devoted to handling 404 errors. It finds it by looking for the variable handler404 in your root URLconf (and only in your root URLconf; setting handler404 anywhere else will have no effect), which is a string in Python dotted syntax – the same format the normal URLconf callbacks use. A 404 view itself has nothing special: It’s just a normal view. You normally won’t have to bother with writing 404 views. If you don’t set handler404, the built-in view django.views.defaults.page_not_found() is used by default. Optionally, you can create a 404.html template in the root of your template directory. The default 404 view will then use that template for all 404 errors when DEBUG is set to False (in your settings module). If you do create the template, add at least some dummy content like “Page not found”. A couple more things to note about 404 views: • If DEBUG is set to True (in your settings module) then your 404 view will never be used (and thus the 404.html template will never be rendered) because the traceback will be displayed instead. • The 404 view is also called if Django doesn’t find a match after checking every regular expression in the URLconf. 2.5.7 Write a 500 (server error) view Similarly, your root URLconf may define a handler500, which points to a view to call in case of server errors. Server errors happen when you have runtime errors in view code. Likewise, you should create a 500.html template at the root of your template directory and add some content like “Something went wrong”. 2.5.8 Use the template system Back to the detail() view for our poll application. Given the context variable poll, here’s what the polls/detail.html template might look like: <h1>{{ poll.question }}</h1> <ul> {% for choice in poll.choice_set.all %} <li>{{ choice.choice_text }}</li> 2.5. Writing your first Django app, part 3 41
- 46. Django Documentation, Release1.5.1 {% endfor %} </ul> The template system uses dot-lookup syntax to access variable attributes. In the example of {{ poll.question }}, first Django does a dictionary lookup on the object poll. Failing that, it tries an attribute lookup – which works, in this case. If attribute lookup had failed, it would’ve tried a list-index lookup. Method-calling happens in the {% for %} loop: poll.choice_set.all is interpreted as the Python code poll.choice_set.all(), which returns an iterable of Choice objects and is suitable for use in the {% for %} tag. See the template guide for more about templates. 2.5.9 Removing hardcoded URLs in templates Remember, when we wrote the link to a poll in the polls/index.html template, the link was partially hardcoded like this: <li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li> The problem with this hardcoded, tightly-coupled approach is that it becomes challenging to change URLs on projects with a lot of templates. However, since you defined the name argument in the url() functions in the polls.urls module, you can remove a reliance on specific URL paths defined in your url configurations by using the {% url %} template tag: <li><a href="{% url ’detail’ poll.id %}">{{ poll.question }}</a></li> Note: If {% url ’detail’ poll.id %} (with quotes) doesn’t work, but {% url detail poll.id %} (without quotes) does, that means you’re using a version of Django < 1.5. In this case, add the following declaration at the top of your template: {% load url from future %} The way this works is by looking up the URL definition as specified in the polls.urls module. You can see exactly where the URL name of ‘detail’ is defined below: ... # the ’name’ value as called by the {% url %} template tag url(r’^(?P<poll_id>d+)/$’, views.detail, name=’detail’), ... If you want to change the URL of the polls detail view to something else, perhaps to something like polls/specifics/12/ instead of doing it in the template (or templates) you would change it in polls/urls.py: ... # added the word ’specifics’ url(r’^specifics/(?P<poll_id>d+)/$’, views.detail, name=’detail’), ... 2.5.10 Namespacing URL names The tutorial project has just one app, polls. In real Django projects, there might be five, ten, twenty apps or more. How does Django differentiate the URL names between them? For example, the polls app has a detail view, and so might an app on the same project that is for a blog. How does one make it so that Django knows which app view to create for a url when using the {% url %} template tag? 42 Chapter 2. Getting started
- 47. Django Documentation, Release1.5.1 The answer is to add namespaces to your root URLconf. In the mysite/urls.py file (the project’s urls.py, not the application’s), go ahead and change it to include namespacing: from django.conf.urls import patterns, include, url from django.contrib import admin admin.autodiscover() urlpatterns = patterns(’’, url(r’^polls/’, include(’polls.urls’, namespace="polls")), url(r’^admin/’, include(admin.site.urls)), ) Now change your polls/index.html template from: <li><a href="{% url ’detail’ poll.id %}">{{ poll.question }}</a></li> to point at the namespaced detail view: <li><a href="{% url ’polls:detail’ poll.id %}">{{ poll.question }}</a></li> When you’re comfortable with writing views, read part 4 of this tutorial to learn about simple form processing and generic views. 2.6 Writing your first Django app, part 4 This tutorial begins where Tutorial 3 left off. We’re continuing theWeb-poll application and will focus on simple form processing and cutting down our code. 2.6.1 Write a simple form Let’s update our poll detail template (“polls/detail.html”) from the last tutorial, so that the template contains an HTML <form> element: <h1>{{ poll.question }}</h1> {% if error_message %}<p><strong>{{ error_message }}</strong></p>{% endif %} <form action="{% url ’polls:vote’ poll.id %}" method="post"> {% csrf_token %} {% for choice in poll.choice_set.all %} <input type="radio" name="choice" id="choice{{ forloop.counter }}" value="{{ choice.id }}" /> <label for="choice{{ forloop.counter }}">{{ choice.choice_text }}</label><br /> {% endfor %} <input type="submit" value="Vote" /> </form> A quick rundown: • The above template displays a radio button for each poll choice. The value of each radio button is the associ-ated poll choice’s ID. The name of each radio button is "choice". That means, when somebody selects one of the radio buttons and submits the form, it’ll send the POST data choice=3. This is the basic concept of HTML forms. • We set the form’s action to {% url ’polls:vote’ poll.id %}, and we set method="post". Using method="post" (as opposed to method="get") is very important, because the act of submitting this 2.6. Writing your first Django app, part 4 43
- 48. Django Documentation, Release1.5.1 form will alter data server-side. Whenever you create a form that alters data server-side, use method="post". This tip isn’t specific to Django; it’s just good Web development practice. • forloop.counter indicates how many times the for tag has gone through its loop • Since we’re creating a POST form (which can have the effect of modifying data), we need to worry about Cross Site Request Forgeries. Thankfully, you don’t have to worry too hard, because Django comes with a very easy-to- use system for protecting against it. In short, all POST forms that are targeted at internal URLs should use the {% csrf_token %} template tag. Now, let’s create a Django view that handles the submitted data and does something with it. Remember, in Tutorial 3, we created a URLconf for the polls application that includes this line: url(r’^(?P<poll_id>d+)/vote/$’, views.vote, name=’vote’), We also created a dummy implementation of the vote() function. Let’s create a real version. Add the following to polls/views.py: from django.shortcuts import get_object_or_404, render from django.http import HttpResponseRedirect, HttpResponse from django.core.urlresolvers import reverse from polls.models import Choice, Poll # ... def vote(request, poll_id): p = get_object_or_404(Poll, pk=poll_id) try: selected_choice = p.choice_set.get(pk=request.POST[’choice’]) except (KeyError, Choice.DoesNotExist): # Redisplay the poll voting form. return render(request, ’polls/detail.html’, { ’poll’: p, ’error_message’: "You didn’t select a choice.", }) else: selected_choice.votes += 1 selected_choice.save() # Always return an HttpResponseRedirect after successfully dealing # with POST data. This prevents data from being posted twice if a # user hits the Back button. return HttpResponseRedirect(reverse(’polls:results’, args=(p.id,))) This code includes a few things we haven’t covered yet in this tutorial: • request.POST is a dictionary-like object that lets you access submitted data by key name. In this case, request.POST[’choice’] returns the ID of the selected choice, as a string. request.POST values are always strings. Note that Django also provides request.GET for accessing GET data in the same way – but we’re explicitly using request.POST in our code, to ensure that data is only altered via a POST call. • request.POST[’choice’] will raise KeyError if choice wasn’t provided in POST data. The above code checks for KeyError and redisplays the poll form with an error message if choice isn’t given. • After incrementing the choice count, the code returns an HttpResponseRedirect rather than a normal HttpResponse. HttpResponseRedirect takes a single argument: the URL to which the user will be redirected (see the following point for how we construct the URL in this case). As the Python comment above points out, you should always return an HttpResponseRedirect after successfully dealing with POST data. This tip isn’t specific to Django; it’s just goodWeb development practice. 44 Chapter 2. Getting started
- 49. Django Documentation, Release1.5.1 • We are using the reverse() function in the HttpResponseRedirect constructor in this example. This function helps avoid having to hardcode a URL in the view function. It is given the name of the view that we want to pass control to and the variable portion of the URL pattern that points to that view. In this case, using the URLconf we set up in Tutorial 3, this reverse() call will return a string like ’/polls/3/results/’ ... where the 3 is the value of p.id. This redirected URL will then call the ’results’ view to display the final page. As mentioned in Tutorial 3, request is a HttpRequest object. For more on HttpRequest objects, see the request and response documentation. After somebody votes in a poll, the vote() view redirects to the results page for the poll. Let’s write that view: def results(request, poll_id): poll = get_object_or_404(Poll, pk=poll_id) return render(request, ’polls/results.html’, {’poll’: poll}) This is almost exactly the same as the detail() view from Tutorial 3. The only difference is the template name. We’ll fix this redundancy later. Now, create a polls/results.html template: <h1>{{ poll.question }}</h1> <ul> {% for choice in poll.choice_set.all %} <li>{{ choice.choice_text }} -- {{ choice.votes }} vote{{ choice.votes|pluralize }}</li> {% endfor %} </ul> <a href="{% url ’polls:detail’ poll.id %}">Vote again?</a> Now, go to /polls/1/ in your browser and vote in the poll. You should see a results page that gets updated each time you vote. If you submit the form without having chosen a choice, you should see the error message. 2.6.2 Use generic views: Less code is better The detail() (from Tutorial 3) and results() views are stupidly simple – and, as mentioned above, redundant. The index() view (also from Tutorial 3), which displays a list of polls, is similar. These views represent a common case of basic Web development: getting data from the database according to a parameter passed in the URL, loading a template and returning the rendered template. Because this is so common, Django provides a shortcut, called the “generic views” system. Generic views abstract common patterns to the point where you don’t even need to write Python code to write an app. Let’s convert our poll app to use the generic views system, so we can delete a bunch of our own code. We’ll just have to take a few steps to make the conversion. We will: 1. Convert the URLconf. 2. Delete some of the old, unneeded views. 3. Fix up URL handling for the new views. Read on for details. Why the code-shuffle? 2.6. Writing your first Django app, part 4 45
- 50. Django Documentation, Release1.5.1 Generally, when writing a Django app, you’ll evaluate whether generic views are a good fit for your problem, and you’ll use them from the beginning, rather than refactoring your code halfway through. But this tutorial intentionally has focused on writing the views “the hard way” until now, to focus on core concepts. You should know basic math before you start using a calculator. Amend URLconf First, open the polls/urls.py URLconf and change it like so: from django.conf.urls import patterns, url from django.views.generic import DetailView, ListView from polls.models import Poll urlpatterns = patterns(’’, url(r’^$’, ListView.as_view( queryset=Poll.objects.order_by(’-pub_date’)[:5], context_object_name=’latest_poll_list’, template_name=’polls/index.html’), name=’index’), url(r’^(?P<pk>d+)/$’, DetailView.as_view( model=Poll, template_name=’polls/detail.html’), name=’detail’), url(r’^(?P<pk>d+)/results/$’, DetailView.as_view( model=Poll, template_name=’polls/results.html’), name=’results’), url(r’^(?P<poll_id>d+)/vote/$’, ’polls.views.vote’, name=’vote’), ) Amend views We’re using two generic views here: ListView and DetailView. Respectively, those two views abstract the concepts of “display a list of objects” and “display a detail page for a particular type of object.” • Each generic view needs to know what model it will be acting upon. This is provided using the model param-eter. • The DetailView generic view expects the primary key value captured from the URL to be called "pk", so we’ve changed poll_id to pk for the generic views. By default, the DetailView generic view uses a template called <app name>/<model name>_detail.html. In our case, it’ll use the template "polls/poll_detail.html". The template_name argument is used to tell Django to use a specific template name instead of the autogener-ated default template name. We also specify the template_name for the results list view – this ensures that the results view and the detail view have a different appearance when rendered, even though they’re both a DetailView behind the scenes. Similarly, the ListView generic view uses a default template called <app name>/<model name>_list.html; we use template_name to tell ListView to use our existing "polls/index.html" template. 46 Chapter 2. Getting started
- 51. Django Documentation, Release1.5.1 In previous parts of the tutorial, the templates have been provided with a context that contains the poll and latest_poll_list context variables. For DetailView the poll variable is provided automatically – since we’re using a Django model (Poll), Django is able to determine an appropriate name for the context variable. How-ever, for ListView, the automatically generated context variable is poll_list. To override this we provide the context_object_name option, specifying that we want to use latest_poll_list instead. As an alternative approach, you could change your templates to match the new default context variables – but it’s a lot easier to just tell Django to use the variable you want. You can now delete the index(), detail() and results() views from polls/views.py. We don’t need them anymore – they have been replaced by generic views. You can also delete the import for HttpResponse, which is no longer required. Run the server, and use your new polling app based on generic views. For full details on generic views, see the generic views documentation. When you’re comfortable with forms and generic views, read part 5 of this tutorial to learn about testing our polls app. 2.7 Writing your first Django app, part 5 This tutorial begins where Tutorial 4 left off. We’ve built aWeb-poll application, and we’ll now create some automated tests for it. 2.7.1 Introducing automated testing What are automated tests? Tests are simple routines that check the operation of your code. Testing operates at different levels. Some tests might apply to a tiny detail - does a particular model method return values as expected?, while others examine the overall operation of the software - does a sequence of user inputs on the site produce the desired result? That’s no different from the kind of testing you did earlier in Tutorial 1, using the shell to examine the behavior of a method, or running the application and entering data to check how it behaves. What’s different in automated tests is that the testing work is done for you by the system. You create a set of tests once, and then as you make changes to your app, you can check that your code still works as you originally intended, without having to perform time consuming manual testing. Why you need to create tests So why create tests, and why now? You may feel that you have quite enough on your plate just learning Python/Django, and having yet another thing to learn and do may seem overwhelming and perhaps unnecessary. After all, our polls application is working quite happily now; going through the trouble of creating automated tests is not going to make it work any better. If creating the polls application is the last bit of Django programming you will ever do, then true, you don’t need to know how to create automated tests. But, if that’s not the case, now is an excellent time to learn. Tests will save you time Up to a certain point, ‘checking that it seems to work’ will be a satisfactory test. In a more sophisticated application, you might have dozens of complex interactions between components. 2.7. Writing your first Django app, part 5 47
- 52. Django Documentation, Release1.5.1 A change in any of those components could have unexpected consequences on the application’s behavior. Checking that it still ‘seems to work’ could mean running through your code’s functionality with twenty different variations of your test data just to make sure you haven’t broken something - not a good use of your time. That’s especially true when automated tests could do this for you in seconds. If something’s gone wrong, tests will also assist in identifying the code that’s causing the unexpected behavior. Sometimes it may seem a chore to tear yourself away from your productive, creative programming work to face the unglamorous and unexciting business of writing tests, particularly when you know your code is working properly. However, the task of writing tests is a lot more fulfilling than spending hours testing your application manually or trying to identify the cause of a newly-introduced problem. Tests don’t just identify problems, they prevent them It’s a mistake to think of tests merely as a negative aspect of development. Without tests, the purpose or intended behavior of an application might be rather opaque. Even when it’s your own code, you will sometimes find yourself poking around in it trying to find out what exactly it’s doing. Tests change that; they light up your code from the inside, and when something goes wrong, they focus light on the part that has gone wrong - even if you hadn’t even realized it had gone wrong. Tests make your code more attractive You might have created a brilliant piece of software, but you will find that many other developers will simply refuse to look at it because it lacks tests; without tests, they won’t trust it. Jacob Kaplan-Moss, one of Django’s original developers, says “Code without tests is broken by design.” That other developers want to see tests in your software before they take it seriously is yet another reason for you to start writing tests. Tests help teams work together The previous points are written from the point of view of a single developer maintaining an application. Complex applications will be maintained by teams. Tests guarantee that colleagues don’t inadvertently break your code (and that you don’t break theirs without knowing). If you want to make a living as a Django programmer, you must be good at writing tests! 2.7.2 Basic testing strategies There are many ways to approach writing tests. Some programmers follow a discipline called “test-driven development”; they actually write their tests before they write their code. This might seem counter-intuitive, but in fact it’s similar to what most people will often do anyway: they describe a problem, then create some code to solve it. Test-driven development simply formalizes the problem in a Python test case. More often, a newcomer to testing will create some code and later decide that it should have some tests. Perhaps it would have been better to write some tests earlier, but it’s never too late to get started. Sometimes it’s difficult to figure out where to get started with writing tests. If you have written several thousand lines of Python, choosing something to test might not be easy. In such a case, it’s fruitful to write your first test the next time you make a change, either when you add a new feature or fix a bug. So let’s do that right away. 48 Chapter 2. Getting started
- 53. Django Documentation, Release1.5.1 2.7.3 Writing our first test We identify a bug Fortunately, there’s a little bug in the polls application for us to fix right away: the Poll.was_published_recently() method returns True if the Poll was published within the last day (which is correct) but also if the Poll‘s pub_date field is in the future (which certainly isn’t). You can see this in the Admin; create a poll whose date lies in the future; you’ll see that the Poll change list claims it was published recently. You can also see this using the shell: >>> import datetime >>> from django.utils import timezone >>> from polls.models import Poll >>> # create a Poll instance with pub_date 30 days in the future >>> future_poll = Poll(pub_date=timezone.now() + datetime.timedelta(days=30)) >>> # was it published recently? >>> future_poll.was_published_recently() True Since things in the future are not ‘recent’, this is clearly wrong. Create a test to expose the bug What we’ve just done in the shell to test for the problem is exactly what we can do in an automated test, so let’s turn that into an automated test. The best place for an application’s tests is in the application’s tests.py file - the testing system will look there for tests automatically. Put the following in the tests.py file in the polls application (you’ll notice tests.py contains some dummy tests, you can remove those): import datetime from django.utils import timezone from django.test import TestCase from polls.models import Poll class PollMethodTests(TestCase): def test_was_published_recently_with_future_poll(self): """ was_published_recently() should return False for polls whose pub_date is in the future """ future_poll = Poll(pub_date=timezone.now() + datetime.timedelta(days=30)) self.assertEqual(future_poll.was_published_recently(), False) What we have done here is created a django.test.TestCase subclass with a method that creates a Poll in-stance with a pub_date in the future. We then check the output of was_published_recently() - which ought to be False. 2.7. Writing your first Django app, part 5 49
- 54. Django Documentation, Release1.5.1 Running tests In the terminal, we can run our test: python manage.py test polls and you’ll see something like: Creating test database for alias ’default’... F ====================================================================== FAIL: test_was_published_recently_with_future_poll (polls.tests.PollMethodTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/path/to/mysite/polls/tests.py", line 16, in test_was_published_recently_with_future_poll self.assertEqual(future_poll.was_published_recently(), False) AssertionError: True != False ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (failures=1) Destroying test database for alias ’default’... What happened is this: • python manage.py test polls looked for tests in the polls application • it found a subclass of the django.test.TestCase class • it created a special database for the purpose of testing • it looked for test methods - ones whose names begin with test • in test_was_published_recently_with_future_poll it created a Poll instance whose pub_date field is 30 days in the future • ... and using the assertEqual() method, it discovered that its was_published_recently() returns True, though we wanted it to return False The test informs us which test failed and even the line on which the failure occurred. Fixing the bug We already know what the problem is: Poll.was_published_recently() should return False if its pub_date is in the future. Amend the method in models.py, so that it will only return True if the date is also in the past: def was_published_recently(self): now = timezone.now() return now - datetime.timedelta(days=1) <= self.pub_date < now and run the test again: Creating test database for alias ’default’... . ---------------------------------------------------------------------- Ran 1 test in 0.001s OK Destroying test database for alias ’default’... 50 Chapter 2. Getting started
- 55. Django Documentation, Release1.5.1 After identifying a bug, we wrote a test that exposes it and corrected the bug in the code so our test passes. Many other things might go wrong with our application in the future, but we can be sure that we won’t inadvertently reintroduce this bug, because simply running the test will warn us immediately. We can consider this little portion of the application pinned down safely forever. More comprehensive tests While we’re here, we can further pin down the was_published_recently() method; in fact, it would be posi-tively embarrassing if in fixing one bug we had introduced another. Add two more test methods to the same class, to test the behavior of the method more comprehensively: def test_was_published_recently_with_old_poll(self): """ was_published_recently() should return False for polls whose pub_date is older than 1 day """ old_poll = Poll(pub_date=timezone.now() - datetime.timedelta(days=30)) self.assertEqual(old_poll.was_published_recently(), False) def test_was_published_recently_with_recent_poll(self): """ was_published_recently() should return True for polls whose pub_date is within the last day """ recent_poll = Poll(pub_date=timezone.now() - datetime.timedelta(hours=1)) self.assertEqual(recent_poll.was_published_recently(), True) And now we have three tests that confirm that Poll.was_published_recently() returns sensible values for past, recent, and future polls. Again, polls is a simple application, but however complex it grows in the future and whatever other code it interacts with, we now have some guarantee that the method we have written tests for will behave in expected ways. 2.7.4 Test a view The polls application is fairly undiscriminating: it will publish any poll, including ones whose pub_date field lies in the future. We should improve this. Setting a pub_date in the future should mean that the Poll is published at that moment, but invisible until then. A test for a view When we fixed the bug above, we wrote the test first and then the code to fix it. In fact that was a simple example of test-driven development, but it doesn’t really matter in which order we do the work. In our first test, we focused closely on the internal behavior of the code. For this test, we want to check its behavior as it would be experienced by a user through a web browser. Before we try to fix anything, let’s have a look at the tools at our disposal. The Django test client Django provides a test Client to simulate a user interacting with the code at the view level. We can use it in tests.py or even in the shell. 2.7. Writing your first Django app, part 5 51
- 56. Django Documentation, Release1.5.1 We will start again with the shell, where we need to do a couple of things that won’t be necessary in tests.py. The first is to set up the test environment in the shell: >>> from django.test.utils import setup_test_environment >>> setup_test_environment() Next we need to import the test client class (later in tests.py we will use the django.test.TestCase class, which comes with its own client, so this won’t be required): >>> from django.test.client import Client >>> # create an instance of the client for our use >>> client = Client() With that ready, we can ask the client to do some work for us: >>> # get a response from ’/’ >>> response = client.get(’/’) >>> # we should expect a 404 from that address >>> response.status_code 404 >>> # on the other hand we should expect to find something at ’/polls/’ >>> # we’ll use ’reverse()’ rather than a harcoded URL >>> from django.core.urlresolvers import reverse >>> response = client.get(reverse(’polls:index’)) >>> response.status_code 200 >>> response.content ’nnn <p>No polls are available.</p>nn’ >>> # note - you might get unexpected results if your ‘‘TIME_ZONE‘‘ >>> # in ‘‘settings.py‘‘ is not correct. If you need to change it, >>> # you will also need to restart your shell session >>> from polls.models import Poll >>> from django.utils import timezone >>> # create a Poll and save it >>> p = Poll(question="Who is your favorite Beatle?", pub_date=timezone.now()) >>> p.save() >>> # check the response once again >>> response = client.get(’/polls/’) >>> response.content ’nnn <ul>n n <li><a href="/polls/1/">Who is your favorite Beatle?</a></li>n n >>> response.context[’latest_poll_list’] [<Poll: Who is your favorite Beatle?>] Improving our view The list of polls shows polls that aren’t published yet (i.e. those that have a pub_date in the future). Let’s fix that. In Tutorial 4 we deleted the view functions from views.py in favor of a ListView in urls.py: url(r’^$’, ListView.as_view( queryset=Poll.objects.order_by(’-pub_date’)[:5], context_object_name=’latest_poll_list’, template_name=’polls/index.html’), name=’index’), response.context_data[’latest_poll_list’] extracts the data this view places into the context. We need to amend the line that gives us the queryset: 52 Chapter 2. Getting started
- 57. Django Documentation, Release1.5.1 queryset=Poll.objects.order_by(’-pub_date’)[:5], Let’s change the queryset so that it also checks the date by comparing it with timezone.now(). First we need to add an import: from django.utils import timezone and then we must amend the existing url function to: url(r’^$’, ListView.as_view( queryset=Poll.objects.filter(pub_date__lte=timezone.now) .order_by(’-pub_date’)[:5], context_object_name=’latest_poll_list’, template_name=’polls/index.html’), name=’index’), Poll.objects.filter(pub_date__lte=timezone.now) returns a queryset containing Polls whose pub_date is less than or equal to - that is, earlier than or equal to - timezone.now. Notice that we use a callable queryset argument, timezone.now, which will be evaluated at request time. If we had included the parentheses, timezone.now() would be evaluated just once when the web server is started. Testing our new view Now you can satisfy yourself that this behaves as expected by firing up the runserver, loading the site in your browser, creating Polls with dates in the past and future, and checking that only those that have been published are listed. You don’t want to have to do that every single time you make any change that might affect this - so let’s also create a test, based on our shell session above. Add the following to polls/tests.py: from django.core.urlresolvers import reverse and we’ll create a factory method to create polls as well as a new test class: def create_poll(question, days): """ Creates a poll with the given ‘question‘ published the given number of ‘days‘ offset to now (negative for polls published in the past, positive for polls that have yet to be published). """ return Poll.objects.create(question=question, pub_date=timezone.now() + datetime.timedelta(days=days)) class PollViewTests(TestCase): def test_index_view_with_no_polls(self): """ If no polls exist, an appropriate message should be displayed. """ response = self.client.get(reverse(’polls:index’)) self.assertEqual(response.status_code, 200) self.assertContains(response, "No polls are available.") self.assertQuerysetEqual(response.context[’latest_poll_list’], []) def test_index_view_with_a_past_poll(self): """ Polls with a pub_date in the past should be displayed on the index page. """ 2.7. Writing your first Django app, part 5 53
- 58. Django Documentation, Release1.5.1 create_poll(question="Past poll.", days=-30) response = self.client.get(reverse(’polls:index’)) self.assertQuerysetEqual( response.context[’latest_poll_list’], [’<Poll: Past poll.>’] ) def test_index_view_with_a_future_poll(self): """ Polls with a pub_date in the future should not be displayed on the index page. """ create_poll(question="Future poll.", days=30) response = self.client.get(reverse(’polls:index’)) self.assertContains(response, "No polls are available.", status_code=200) self.assertQuerysetEqual(response.context[’latest_poll_list’], []) def test_index_view_with_future_poll_and_past_poll(self): """ Even if both past and future polls exist, only past polls should be displayed. """ create_poll(question="Past poll.", days=-30) create_poll(question="Future poll.", days=30) response = self.client.get(reverse(’polls:index’)) self.assertQuerysetEqual( response.context[’latest_poll_list’], [’<Poll: Past poll.>’] ) def test_index_view_with_two_past_polls(self): """ The polls index page may display multiple polls. """ create_poll(question="Past poll 1.", days=-30) create_poll(question="Past poll 2.", days=-5) response = self.client.get(reverse(’polls:index’)) self.assertQuerysetEqual( response.context[’latest_poll_list’], [’<Poll: Past poll 2.>’, ’<Poll: Past poll 1.>’] ) Let’s look at some of these more closely. First is a poll factory method, create_poll, to take some repetition out of the process of creating polls. test_index_view_with_no_polls doesn’t create any polls, but checks the message: “No polls are available.” and verifies the latest_poll_list is empty. Note that the django.test.TestCase class provides some additional assertion methods. In these examples, we use assertContains() and assertQuerysetEqual(). In test_index_view_with_a_past_poll, we create a poll and verify that it appears in the list. In test_index_view_with_a_future_poll, we create a poll with a pub_date in the future. The database is reset for each test method, so the first poll is no longer there, and so again the index shouldn’t have any polls in it. And so on. In effect, we are using the tests to tell a story of admin input and user experience on the site, and checking that at every state and for every new change in the state of the system, the expected results are published. 54 Chapter 2. Getting started
- 59. Django Documentation, Release1.5.1 Testing the DetailView What we have works well; however, even though future polls don’t appear in the index, users can still reach them if they know or guess the right URL. So we need similar constraints in the DetailViews, by adding: queryset=Poll.objects.filter(pub_date__lte=timezone.now) to them - for example: url(r’^(?P<pk>d+)/$’, DetailView.as_view( queryset=Poll.objects.filter(pub_date__lte=timezone.now), model=Poll, template_name=’polls/detail.html’), name=’detail’), and of course, we will add some tests, to check that a Poll whose pub_date is in the past can be displayed, and that one with a pub_date in the future is not: class PollIndexDetailTests(TestCase): def test_detail_view_with_a_future_poll(self): """ The detail view of a poll with a pub_date in the future should return a 404 not found. """ future_poll = create_poll(question=’Future poll.’, days=5) response = self.client.get(reverse(’polls:detail’, args=(future_poll.id,))) self.assertEqual(response.status_code, 404) def test_detail_view_with_a_past_poll(self): """ The detail view of a poll with a pub_date in the past should display the poll’s question. """ past_poll = create_poll(question=’Past Poll.’, days=-5) response = self.client.get(reverse(’polls:detail’, args=(past_poll.id,))) self.assertContains(response, past_poll.question, status_code=200) Ideas for more tests We ought to add similar queryset arguments to the other DetailView URLs, and create a new test class for each view. They’ll be very similar to what we have just created; in fact there will be a lot of repetition. We could also improve our application in other ways, adding tests along the way. For example, it’s silly that Polls can be published on the site that have no Choices. So, our views could check for this, and exclude such Polls. Our tests would create a Poll without Choices and then test that it’s not published, as well as create a similar Poll with Choices, and test that it is published. Perhaps logged-in admin users should be allowed to see unpublished Polls, but not ordinary visitors. Again: what-ever needs to be added to the software to accomplish this should be accompanied by a test, whether you write the test first and then make the code pass the test, or work out the logic in your code first and then write a test to prove it. At a certain point you are bound to look at your tests and wonder whether your code is suffering from test bloat, which brings us to: 2.7. Writing your first Django app, part 5 55
- 60. Django Documentation, Release1.5.1 2.7.5 When testing, more is better It might seem that our tests are growing out of control. At this rate there will soon be more code in our tests than in our application, and the repetition is unaesthetic, compared to the elegant conciseness of the rest of our code. It doesn’t matter. Let them grow. For the most part, you can write a test once and then forget about it. It will continue performing its useful function as you continue to develop your program. Sometimes tests will need to be updated. Suppose that we amend our views so that only Polls with Choices are published. In that case, many of our existing tests will fail - telling us exactly which tests need to be amended to bring them up to date, so to that extent tests help look after themselves. At worst, as you continue developing, you might find that you have some tests that are now redundant. Even that’s not a problem; in testing redundancy is a good thing. As long as your tests are sensibly arranged, they won’t become unmanageable. Good rules-of-thumb include having: • a separate TestClass for each model or view • a separate test method for each set of conditions you want to test • test method names that describe their function 2.7.6 Further testing This tutorial only introduces some of the basics of testing. There’s a great deal more you can do, and a number of very useful tools at your disposal to achieve some very clever things. For example, while our tests here have covered some of the internal logic of a model and the way our views publish information, you can use an “in-browser” framework such as Selenium to test the way your HTML actually renders in a browser. These tools allow you to check not just the behavior of your Django code, but also, for example, of your JavaScript. It’s quite something to see the tests launch a browser, and start interacting with your site, as if a human being were driving it! Django includes LiveServerTestCase to facilitate integration with tools like Selenium. If you have a complex application, you may want to run tests automatically with every commit for the purposes of continuous integration, so that quality control is itself - at least partially - automated. A good way to spot untested parts of your application is to check code coverage. This also helps identify fragile or even dead code. If you can’t test a piece of code, it usually means that code should be refactored or removed. Coverage will help to identify dead code. See Integration with coverage.py for details. Testing Django applications has comprehensive information about testing. 2.7.7 What’s next? For full details on testing, see Testing in Django. When you’re comfortable with testing Django views, read part 6 of this tutorial to learn about static files management. 2.8 Writing your first Django app, part 6 This tutorial begins where Tutorial 5 left off. We’ve built a testedWeb-poll application, and we’ll now add a stylesheet and an image. Aside from the HTML generated by the server, web applications generally need to serve additional files — such as images, JavaScript, or CSS—necessary to render the complete web page. In Django, we refer to these files as “static files”. 56 Chapter 2. Getting started
- 61. Django Documentation, Release1.5.1 For small projects, this isn’t a big deal, because you can just keep the static files somewhere your web server can find it. However, in bigger projects – especially those comprised of multiple apps – dealing with the multiple sets of static files provided by each application starts to get tricky. That’s what django.contrib.staticfiles is for: it collects static files from each of your applications (and any other places you specify) into a single location that can easily be served in production. 2.8.1 Customize your app’s look and feel First, create a directory called static in your polls directory. Django will look for static files there, similarly to how Django finds templates inside polls/templates/. Django’s STATICFILES_FINDERS setting contains a list of finders that know how to discover static files from various sources. One of the defaults is AppDirectoriesFinder which looks for a “static” subdirectory in each of the INSTALLED_APPS, like the one in polls we just created. The admin site uses the same directory structure for its static files. Within the static directory you have just created, create another directory called polls and within that create a file called style.css. In other words, your stylesheet should be at polls/static/polls/style.css. Because of how the AppDirectoriesFinder staticfile finder works, you can refer to this static file in Django simply as polls/style.css, similar to how you reference the path for templates. Static file namespacing Just like templates, we might be able to get away with putting our static files directly in polls/static (rather than creating another polls subdirectory), but it would actually be a bad idea. Django will choose the first static file it finds whose name matches, and if you had a static file with the same name in a different application, Django would be unable to distinguish between them. We need to be able to point Django at the right one, and the easiest way to ensure this is by namespacing them. That is, by putting those static files inside another directory named for the application itself. Put the following code in that stylesheet (polls/static/polls/style.css): li a { color: green; } Next, add the following at the top of polls/templates/polls/index.html: {% load staticfiles %} <link rel="stylesheet" type="text/css" href="{% static ’polls/style.css’ %}" /> {% load staticfiles %} loads the {% static %} template tag from the staticfiles template library. The {% static %} template tag generates the absolute URL of the static file. That’s all you need to do for development. Reload http://localhost:8000/polls/ and you should see that the poll links are green (Django style!) which means that your stylesheet was properly loaded. 2.8.2 Adding a background-image Next, we’ll create a subdirectory for images. Create an images subdirectory in the polls/static/polls/ directory. Inside this directory, put an image called background.gif. In other words, put your image in polls/static/polls/images/background.gif. Then, add to your stylesheet (polls/static/polls/style.css): 2.8. Writing your first Django app, part 6 57
- 62. Django Documentation, Release1.5.1 body { background: white url("images/background.gif") no-repeat right bottom; } Reload http://localhost:8000/polls/ and you should see the background loaded in the bottom right of the screen. Warning: Of course the {% static %} template tag is not available for use in static files like your stylesheet which aren’t generated by Django. You should always use relative paths to link your static files between each other, because then you can change STATIC_URL (used by the static template tag to generate its URLs) without having to modify a bunch of paths in your static files as well. These are the basics. For more details on settings and other bits included with the framework see the static files howto and the staticfiles reference. Deploying static files discusses how to use static files on a real server. 2.8.3 What’s next? The beginner tutorial ends here for the time being. In the meantime, you might want to check out some pointers on where to go from here. If you are familiar with Python packaging and interested in learning how to turn polls into a “reusable app”, check out Advanced tutorial: How to write reusable apps. 2.9 Advanced tutorial: How to write reusable apps This advanced tutorial begins where Tutorial 6 left off. We’ll be turning our Web-poll into a standalone Python package you can reuse in new projects and share with other people. If you haven’t recently completed Tutorials 1–6, we encourage you to review these so that your example project matches the one described below. 2.9.1 Reusability matters It’s a lot of work to design, build, test and maintain a web application. Many Python and Django projects share common problems. Wouldn’t it be great if we could save some of this repeated work? Reusability is the way of life in Python. The Python Package Index (PyPI) has a vast range of packages you can use in your own Python programs. Check out Django Packages for existing reusable apps you could incorporate in your project. Django itself is also just a Python package. This means that you can take existing Python packages or Django apps and compose them into your own web project. You only need to write the parts that make your project unique. Let’s say you were starting a new project that needed a polls app like the one we’ve been working on. How do you make this app reusable? Luckily, you’re well on the way already. In Tutorial 3, we saw how we could decouple polls from the project-level URLconf using an include. In this tutorial, we’ll take further steps to make the app easy to use in new projects and ready to publish for others to install and use. Package? App? A Python package provides a way of grouping related Python code for easy reuse. A package contains one or more files of Python code (also known as “modules”). A package can be imported with import foo.bar or from foo import bar. For a directory (like polls) to form a package, it must contain a special file __init__.py, even if this file is empty. 58 Chapter 2. Getting started
- 63. Django Documentation, Release1.5.1 A Django app is just a Python package that is specifically intended for use in a Django project. An app may also use common Django conventions, such as having a models.py file. Later on we use the term packaging to describe the process of making a Python package easy for others to install. It can be a little confusing, we know. 2.9.2 Your project and your reusable app After the previous tutorials, our project should look like this: mysite/ manage.py mysite/ __init__.py settings.py urls.py wsgi.py polls/ admin.py __init__.py models.py tests.py static/ style.css images/ background.gif templates/ polls/ detail.html index.html results.html urls.py views.py templates/ admin/ base_site.html You created mysite/templates in Tutorial 2, and polls/templates in Tutorial 3. Now perhaps it is clearer why we chose to have separate template directories for the project and application: everything that is part of the polls application is in polls. It makes the application self-contained and easier to drop into a new project. The polls directory could now be copied into a new Django project and immediately reused. It’s not quite ready to be published though. For that, we need to package the app to make it easy for others to install. 2.9.3 Installing some prerequisites The current state of Python packaging is a bit muddled with various tools. For this tutorial, we’re going to use distribute to build our package. It’s a community-maintained fork of the older setuptools project. We’ll also be using pip to uninstall it after we’re finished. You should install these two packages now. If you need help, you can refer to how to install Django with pip. You can install distribute the same way. 2.9. Advanced tutorial: How to write reusable apps 59
- 64. Django Documentation, Release1.5.1 2.9.4 Packaging your app Python packaging refers to preparing your app in a specific format that can be easily installed and used. Django itself is packaged very much like this. For a small app like polls, this process isn’t too difficult. 1. First, create a parent directory for polls, outside of your Django project. Call this directory django-polls. Choosing a name for your app When choosing a name for your package, check resources like PyPI to avoid naming conflicts with existing packages. It’s often useful to prepend django- to your module name when creating a package to distribute. This helps others looking for Django apps identify your app as Django specific. 2. Move the polls directory into the django-polls directory. 3. Create a file django-polls/README.rst with the following contents: ===== Polls ===== Polls is a simple Django app to conduct Web-based polls. For each question, visitors can choose between a fixed number of answers. Detailed documentation is in the "docs" directory. Quick start ----------- 1. Add "polls" to your INSTALLED_APPS setting like this:: INSTALLED_APPS = ( ... ’polls’, ) 2. Include the polls URLconf in your project urls.py like this:: url(r’^polls/’, include(’polls.urls’)), 3. Run ‘python manage.py syncdb‘ to create the polls models. 4. Start the development server and visit http://127.0.0.1:8000/admin/ to create a poll (you’ll need the Admin app enabled). 5. Visit http://127.0.0.1:8000/polls/ to participate in the poll. 4. Create a django-polls/LICENSE file. Choosing a license is beyond the scope of this tutorial, but suffice it to say that code released publicly without a license is useless. Django and many Django-compatible apps are distributed under the BSD license; however, you’re free to pick your own license. Just be aware that your licensing choice will affect who is able to use your code. 5. Next we’ll create a setup.py file which provides details about how to build and install the app. A full expla-nation of this file is beyond the scope of this tutorial, but the distribute docs have a good explanation. Create a file django-polls/setup.py with the following contents: import os from setuptools import setup 60 Chapter 2. Getting started
- 65. Django Documentation, Release1.5.1 README = open(os.path.join(os.path.dirname(__file__), ’README.rst’)).read() # allow setup.py to be run from any path os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))) setup( name=’django-polls’, version=’0.1’, packages=[’polls’], include_package_data=True, license=’BSD License’, # example license description=’A simple Django app to conduct Web-based polls.’, long_description=README, url=’http://www.example.com/’, author=’Your Name’, author_email=’yourname@example.com’, classifiers=[ ’Environment :: Web Environment’, ’Framework :: Django’, ’Intended Audience :: Developers’, ’License :: OSI Approved :: BSD License’, # example license ’Operating System :: OS Independent’, ’Programming Language :: Python’, ’Programming Language :: Python :: 2.6’, ’Programming Language :: Python :: 2.7’, ’Topic :: Internet :: WWW/HTTP’, ’Topic :: Internet :: WWW/HTTP :: Dynamic Content’, ], ) I thought you said we were going to use distribute? Distribute is a drop-in replacement for setuptools. Even though we appear to import from setuptools, since we have distribute installed, it will override the import. 6. Only Python modules and packages are included in the package by default. To include additional files, we’ll need to create a MANIFEST.in file. The distribute docs referred to in the previous step discuss this file in more details. To include the templates, the README.rst and our LICENSE file, create a file django-polls/MANIFEST.in with the following contents: include LICENSE include README.rst recursive-include polls/templates * 7. It’s optional, but recommended, to include detailed documentation with your app. Create an empty directory django-polls/docs for future documentation. Add an additional line to django-polls/MANIFEST.in: recursive-include docs * Note that the docs directory won’t be included in your package unless you add some files to it. Many Django apps also provide their documentation online through sites like readthedocs.org. 8. Try building your package with python setup.py sdist (run from inside django-polls). This cre-ates a directory called dist and builds your new package, django-polls-0.1.tar.gz. For more information on packaging, see The Hitchhiker’s Guide to Packaging. 2.9. Advanced tutorial: How to write reusable apps 61
- 66. Django Documentation, Release1.5.1 2.9.5 Using your own package Since we moved the polls directory out of the project, it’s no longer working. We’ll now fix this by installing our new django-polls package. Installing as a user library The following steps install django-polls as a user library. Per-user installs have a lot of advantages over installing the package system-wide, such as being usable on systems where you don’t have administrator access as well as preventing the package from affecting system services and other users of the machine. Python 2.6 added support for user libraries, so if you are using an older version this won’t work, but Django 1.5 requires Python 2.6 or newer anyway. Note that per-user installations can still affect the behavior of system tools that run as that user, so virtualenv is a more robust solution (see below). 1. Inside django-polls/dist, untar the new package django-polls-0.1.tar.gz (e.g. tar xzvf django-polls-0.1.tar.gz). If you’re using Windows, you can download the command-line tool bsdtar to do this, or you can use a GUI-based tool such as 7-zip. 2. Change into the directory created in step 1 (e.g. cd django-polls-0.1). 3. If you’re using GNU/Linux, Mac OS X or some other flavor of Unix, enter the command python setup.py install --user at the shell prompt. If you’re using Windows, start up a command shell and run the command setup.py install --user. With luck, your Django project should now work correctly again. Run the server again to confirm this. 4. To uninstall the package, use pip (you already installed it, right?): pip uninstall django-polls 2.9.6 Publishing your app Now that we’ve packaged and tested django-polls, it’s ready to share with the world! If this wasn’t just an example, you could now: • Email the package to a friend. • Upload the package on your Web site. • Post the package on a public repository, such as The Python Package Index (PyPI). For more information on PyPI, see the Quickstart section of The Hitchhiker’s Guide to Packaging. One detail this guide mentions is choosing the license under which your code is distributed. 2.9.7 Installing Python packages with virtualenv Earlier, we installed the polls app as a user library. This has some disadvantages: • Modifying the user libraries can affect other Python software on your system. • You won’t be able to run multiple versions of this package (or others with the same name). Typically, these situations only arise once you’re maintaining several Django projects. When they do, the best solution is to use virtualenv. This tool allows you to maintain multiple isolated Python environments, each with its own copy of the libraries and package namespace. 62 Chapter 2. Getting started
- 67. Django Documentation, Release1.5.1 2.10 What to read next So you’ve read all the introductory material and have decided you’d like to keep using Django. We’ve only just scratched the surface with this intro (in fact, if you’ve read every single word you’ve still read less than 10% of the overall documentation). So what’s next? Well, we’ve always been big fans of learning by doing. At this point you should know enough to start a project of your own and start fooling around. As you need to learn new tricks, come back to the documentation. We’ve put a lot of effort into making Django’s documentation useful, easy to read and as complete as possible. The rest of this document explains more about how the documentation works so that you can get the most out of it. (Yes, this is documentation about documentation. Rest assured we have no plans to write a document about how to read the document about documentation.) 2.10.1 Finding documentation Django’s got a lot of documentation – almost 200,000 words – so finding what you need can sometimes be tricky. A few good places to start are the search and the genindex. Or you can just browse around! 2.10.2 How the documentation is organized Django’s main documentation is broken up into “chunks” designed to fill different needs: • The introductory material is designed for people new to Django – or to Web development in general. It doesn’t cover anything in depth, but instead gives a high-level overview of how developing in Django “feels”. • The topic guides, on the other hand, dive deep into individual parts of Django. There are complete guides to Django’s model system, template engine, forms framework, and much more. This is probably where you’ll want to spend most of your time; if you work your way through these guides you should come out knowing pretty much everything there is to know about Django. • Web development is often broad, not deep – problems span many domains. We’ve written a set of how-to guides that answer common “How do I ...?” questions. Here you’ll find information about generating PDFs with Django, writing custom template tags, and more. Answers to really common questions can also be found in the FAQ. • The guides and how-to’s don’t cover every single class, function, and method available in Django – that would be overwhelming when you’re trying to learn. Instead, details about individual classes, functions, methods, and modules are kept in the reference. This is where you’ll turn to find the details of a particular function or whathaveyou. • Finally, there’s some “specialized” documentation not usually relevant to most developers. This includes the release notes and internals documentation for those who want to add code to Django itself, and a few other things that simply don’t fit elsewhere. 2.10.3 How documentation is updated Just as the Django code base is developed and improved on a daily basis, our documentation is consistently improving. We improve documentation for several reasons: 2.10. What to read next 63
- 68. Django Documentation, Release1.5.1 • To make content fixes, such as grammar/typo corrections. • To add information and/or examples to existing sections that need to be expanded. • To document Django features that aren’t yet documented. (The list of such features is shrinking but exists nonetheless.) • To add documentation for new features as new features get added, or as Django APIs or behaviors change. Django’s documentation is kept in the same source control system as its code. It lives in the docs directory of our Git repository. Each document online is a separate text file in the repository. 2.10.4 Where to get it You can read Django documentation in several ways. They are, in order of preference: On the Web The most recent version of the Django documentation lives at https://docs.djangoproject.com/en/dev/. These HTML pages are generated automatically from the text files in source control. That means they reflect the “latest and greatest” in Django – they include the very latest corrections and additions, and they discuss the latest Django features, which may only be available to users of the Django development version. (See “Differences between versions” below.) We encourage you to help improve the docs by submitting changes, corrections and suggestions in the ticket system. The Django developers actively monitor the ticket system and use your feedback to improve the documentation for everybody. Note, however, that tickets should explicitly relate to the documentation, rather than asking broad tech-support ques-tions. If you need help with your particular Django setup, try the django-users mailing list or the #django IRC channel instead. In plain text For offline reading, or just for convenience, you can read the Django documentation in plain text. If you’re using an official release of Django, note that the zipped package (tarball) of the code includes a docs/ directory, which contains all the documentation for that release. If you’re using the development version of Django (aka “trunk”), note that the docs/ directory contains all of the documentation. You can update your Git checkout to get the latest changes. One low-tech way of taking advantage of the text documentation is by using the Unix grep utility to search for a phrase in all of the documentation. For example, this will show you each mention of the phrase “max_length” in any Django document: $ grep -r max_length /path/to/django/docs/ As HTML, locally You can get a local copy of the HTML documentation following a few easy steps: • Django’s documentation uses a system called Sphinx to convert from plain text to HTML. You’ll need to install Sphinx by either downloading and installing the package from the Sphinx Web site, or with pip: $ sudo pip install Sphinx 64 Chapter 2. Getting started
- 69. Django Documentation, Release1.5.1 • Then, just use the included Makefile to turn the documentation into HTML: $ cd path/to/django/docs $ make html You’ll need GNU Make installed for this. If you’re on Windows you can alternatively use the included batch file: cd pathtodjangodocs make.bat html • The HTML documentation will be placed in docs/_build/html. Note: Generation of the Django documentation will work with Sphinx version 0.6 or newer, but we recommend going straight to Sphinx 1.0.2 or newer. 2.10.5 Differences between versions As previously mentioned, the text documentation in our Git repository contains the “latest and greatest” changes and additions. These changes often include documentation of new features added in the Django development version – the Git (“trunk”) version of Django. For that reason, it’s worth pointing out our policy on keeping straight the documentation for various versions of the framework. We follow this policy: • The primary documentation on djangoproject.com is an HTML version of the latest docs in Git. These docs always correspond to the latest official Django release, plus whatever features we’ve added/changed in the framework since the latest release. • As we add features to Django’s development version, we try to update the documentation in the same Git commit transaction. • To distinguish feature changes/additions in the docs, we use the phrase: “New in version X.Y”, being X.Y the next release version (hence, the one being developed). • Documentation fixes and improvements may be backported to the last release branch, at the discretion of the committer, however, once a version of Django is no longer supported, that version of the docs won’t get any further updates. • The main documentation Web page includes links to documentation for all previous versions. Be sure you are using the version of the docs corresponding to the version of Django you are using! 2.11 Writing your first patch for Django 2.11.1 Introduction Interested in giving back to the community a little? Maybe you’ve found a bug in Django that you’d like to see fixed, or maybe there’s a small feature you want added. Contributing back to Django itself is the best way to see your own concerns addressed. This may seem daunting at first, but it’s really pretty simple. We’ll walk you through the entire process, so you can learn by example. 2.11. Writing your first patch for Django 65
- 70. Django Documentation, Release1.5.1 Who’s this tutorial for? For this tutorial, we expect that you have at least a basic understanding of how Django works. This means you should be comfortable going through the existing tutorials on writing your first Django app. In addition, you should have a good understanding of Python itself. But if you don’t, Dive Into Python is a fantastic (and free) online book for beginning Python programmers. Those of you who are unfamiliar with version control systems and Trac will find that this tutorial and its links include just enough information to get started. However, you’ll probably want to read some more about these different tools if you plan on contributing to Django regularly. For the most part though, this tutorial tries to explain as much as possible, so that it can be of use to the widest audience. Where to get help: If you’re having trouble going through this tutorial, please post a message to django-developers or drop by #django-dev on irc.freenode.net to chat with other Django users who might be able to help. What does this tutorial cover? We’ll be walking you through contributing a patch to Django for the first time. By the end of this tutorial, you should have a basic understanding of both the tools and the processes involved. Specifically, we’ll be covering the following: • Installing Git. • How to download a development copy of Django. • Running Django’s test suite. • Writing a test for your patch. • Writing the code for your patch. • Testing your patch. • Generating a patch file for your changes. • Where to look for more information. Once you’re done with the tutorial, you can look through the rest of Django’s documentation on contributing. It contains lots of great information and is a must read for anyone who’d like to become a regular contributor to Django. If you’ve got questions, it’s probably got the answers. 2.11.2 Installing Git For this tutorial, you’ll need Git installed to download the current development version of Django and to generate patch files for the changes you make. To check whether or not you have Git installed, enter git into the command line. If you get messages saying that this command could be found, you’ll have to download and install it, see Git’s download page. If you’re not that familiar with Git, you can always find out more about its commands (once it’s installed) by typing git help into the command line. 66 Chapter 2. Getting started
- 71. Django Documentation, Release1.5.1 2.11.3 Getting a copy of Django’s development version The first step to contributing to Django is to get a copy of the source code. From the command line, use the cd command to navigate to the directory where you’ll want your local copy of Django to live. Download the Django source code repository using the following command: git clone https://github.com/django/django.git Note: For users who wish to use virtualenv, you can use: pip install -e /path/to/your/local/clone/django/ (where django is the directory of your clone that contains setup.py) to link your cloned checkout into a virtual environment. This is a great option to isolate your development copy of Django from the rest of your system and avoids potential package conflicts. 2.11.4 Rolling back to a previous revision of Django For this tutorial, we’ll be using ticket #17549 as a case study, so we’ll rewind Django’s version history in git to before that ticket’s patch was applied. This will allow us to go through all of the steps involved in writing that patch from scratch, including running Django’s test suite. Keep in mind that while we’ll be using an older revision of Django’s trunk for the purposes of the tutorial below, you should always use the current development revision of Django when working on your own patch for a ticket! Note: The patch for this ticket was written by Ulrich Petri, and it was applied to Django as commit ac2052ebc84c45709ab5f0f25e685bf656ce79bc. Consequently, we’ll be using the revision of Django just prior to that, commit 39f5bc7fc3a4bb43ed8a1358b17fe0521a1a63ac. Navigate into Django’s root directory (that’s the one that contains django, docs, tests, AUTHORS, etc.). You can then check out the older revision of Django that we’ll be using in the tutorial below: git checkout 39f5bc7fc3a4bb43ed8a1358b17fe0521a1a63ac 2.11.5 Running Django’s test suite for the first time When contributing to Django it’s very important that your code changes don’t introduce bugs into other areas of Django. One way to check that Django still works after you make your changes is by running Django’s test suite. If all the tests still pass, then you can be reasonably sure that your changes haven’t completely broken Django. If you’ve never run Django’s test suite before, it’s a good idea to run it once beforehand just to get familiar with what its output is supposed to look like. We can run the test suite by simply cd-ing into the Django tests/ directory and, if you’re using GNU/Linux, Mac OS X or some other flavor of Unix, run: PYTHONPATH=.. python runtests.py --settings=test_sqlite If you’re on Windows, the above should work provided that you are using “Git Bash” provided by the default Git install. GitHub has a nice tutorial. 2.11. Writing your first patch for Django 67
- 72. Django Documentation, Release1.5.1 Note: If you’re using virtualenv, you can omit PYTHONPATH=.. when running the tests. This instructs Python to look for Django in the parent directory of tests. virtualenv puts your copy of Django on the PYTHONPATH automatically. Now sit back and relax. Django’s entire test suite has over 4800 different tests, so it can take anywhere from 5 to 15 minutes to run, depending on the speed of your computer. While Django’s test suite is running, you’ll see a stream of characters representing the status of each test as it’s run. E indicates that an error was raised during a test, and F indicates that a test’s assertions failed. Both of these are considered to be test failures. Meanwhile, x and s indicated expected failures and skipped tests, respectively. Dots indicate passing tests. Skipped tests are typically due to missing external libraries required to run the test; see Running all the tests for a list of dependencies and be sure to install any for tests related to the changes you are making (we won’t need any for this tutorial). Once the tests complete, you should be greeted with a message informing you whether the test suite passed or failed. Since you haven’t yet made any changes to Django’s code, the entire test suite should pass. If you get failures or errors make sure you’ve followed all of the previous steps properly. See Running the unit tests for more information. Note that the latest Django trunk may not always be stable. When developing against trunk, you can check Django’s continuous integration builds to determine if the failures are specific to your machine or if they are also present in Django’s official builds. If you click to view a particular build, you can view the “Configuration Matrix” which shows failures broken down by Python version and database backend. Note: For this tutorial and the ticket we’re working on, testing against SQLite is sufficient, however, it’s possible (and sometimes necessary) to run the tests using a different database. 2.11.6 Writing some tests for your ticket In most cases, for a patch to be accepted into Django it has to include tests. For bug fix patches, this means writing a regression test to ensure that the bug is never reintroduced into Django later on. A regression test should be written in such a way that it will fail while the bug still exists and pass once the bug has been fixed. For patches containing new features, you’ll need to include tests which ensure that the new features are working correctly. They too should fail when the new feature is not present, and then pass once it has been implemented. A good way to do this is to write your new tests first, before making any changes to the code. This style of development is called test-driven development and can be applied to both entire projects and single patches. After writing your tests, you then run them to make sure that they do indeed fail (since you haven’t fixed that bug or added that feature yet). If your new tests don’t fail, you’ll need to fix them so that they do. After all, a regression test that passes regardless of whether a bug is present is not very helpful at preventing that bug from reoccurring down the road. Now for our hands-on example. Writing some tests for ticket #17549 Ticket #17549 describes the following, small feature addition: It’s useful for URLField to give you a way to open the URL; otherwise you might as well use a CharField. In order to resolve this ticket, we’ll add a render method to the AdminURLFieldWidget in order to display a clickable link above the input widget. Before we make those changes though, we’re going to write a couple tests to verify that our modification functions correctly and continues to function correctly in the future. 68 Chapter 2. Getting started
- 73. Django Documentation, Release1.5.1 Navigate to Django’s tests/regressiontests/admin_widgets/ folder and open the tests.py file. Add the following code on line 269 right before the AdminFileWidgetTest class: class AdminURLWidgetTest(DjangoTestCase): def test_render(self): w = widgets.AdminURLFieldWidget() self.assertHTMLEqual( conditional_escape(w.render(’test’, ’’)), ’<input class="vURLField" name="test" type="text" />’ ) self.assertHTMLEqual( conditional_escape(w.render(’test’, ’http://example.com’)), ’<p class="url">Currently:<a href="http://example.com">http://example.com</a><br />Change:<) def test_render_idn(self): w = widgets.AdminURLFieldWidget() self.assertHTMLEqual( conditional_escape(w.render(’test’, ’http://example-äüö.com’)), ’<p class="url">Currently:<a href="http://xn--example--7za4pnc.com">http://example-äüö.com</) def test_render_quoting(self): w = widgets.AdminURLFieldWidget() self.assertHTMLEqual( conditional_escape(w.render(’test’, ’http://example.com/<sometag>some text</sometag>’)), ’<p class="url">Currently:<a href="http://example.com/%3Csometag%3Esome%20text%3C/sometag%) self.assertHTMLEqual( conditional_escape(w.render(’test’, ’http://example-äüö.com/<sometag>some text</sometag>’)), ’<p class="url">Currently:<a href="http://xn--example--7za4pnc.com/%3Csometag%3Esome%20text%) The new tests check to see that the render method we’ll be adding works correctly in a couple different situations. But this testing thing looks kinda hard... If you’ve never had to deal with tests before, they can look a little hard to write at first glance. Fortunately, testing is a very big subject in computer programming, so there’s lots of information out there: • A good first look at writing tests for Django can be found in the documentation on Testing Django applications. • Dive Into Python (a free online book for beginning Python developers) includes a great introduction to Unit Testing. • After reading those, if you want something a little meatier to sink your teeth into, there’s always the Python unittest documentation. Running your new test Remember that we haven’t actually made any modifications to AdminURLFieldWidget yet, so our tests are going to fail. Let’s run all the tests in the model_forms_regress folder to make sure that’s really what happens. From the command line, cd into the Django tests/ directory and run: PYTHONPATH=.. python runtests.py --settings=test_sqlite admin_widgets If the tests ran correctly, you should see three failures corresponding to each of the test methods we added. If all of the tests passed, then you’ll want to make sure that you added the new test shown above to the appropriate folder and 2.11. Writing your first patch for Django 69
- 74. Django Documentation, Release1.5.1 class. 2.11.7 Writing the code for your ticket Next we’ll be adding the functionality described in ticket #17549 to Django. Writing the code for ticket #17549 Navigate to the django/django/contrib/admin/ folder and open the widgets.py file. Find the AdminURLFieldWidget class on line 302 and add the following render method after the existing __init__ method: def render(self, name, value, attrs=None): html = super(AdminURLFieldWidget, self).render(name, value, attrs) if value: value = force_text(self._format_value(value)) final_attrs = {’href’: mark_safe(smart_urlquote(value))} html = format_html( ’<p class="url">{0} <a {1}>{2}</a><br />{3} {4}</p>’, _(’Currently:’), flatatt(final_attrs), value, _(’Change:’), html ) return html Verifying your test now passes Once you’re done modifying Django, we need to make sure that the tests we wrote earlier pass, so we can see whether the code we wrote above is working correctly. To run the tests in the admin_widgets folder, cd into the Django tests/ directory and run: PYTHONPATH=.. python runtests.py --settings=test_sqlite admin_widgets Oops, good thing we wrote those tests! You should still see 3 failures with the following exception: NameError: global name ’smart_urlquote’ is not defined We forgot to add the import for that method. Go ahead and add the smart_urlquote import at the end of line 13 of django/contrib/admin/widgets.py so it looks as follows: from django.utils.html import escape, format_html, format_html_join, smart_urlquote Re-run the tests and everything should pass. If it doesn’t, make sure you correctly modified the AdminURLFieldWidget class as shown above and copied the new tests correctly. 2.11.8 Running Django’s test suite for the second time Once you’ve verified that your patch and your test are working correctly, it’s a good idea to run the entire Django test suite just to verify that your change hasn’t introduced any bugs into other areas of Django. While successfully passing the entire test suite doesn’t guarantee your code is bug free, it does help identify many bugs and regressions that might otherwise go unnoticed. To run the entire Django test suite, cd into the Django tests/ directory and run: 70 Chapter 2. Getting started
- 75. Django Documentation, Release1.5.1 PYTHONPATH=.. python runtests.py --settings=test_sqlite As long as you don’t see any failures, you’re good to go. Note that this fix also made a small CSS change to format the new widget. You can make the change if you’d like, but we’ll skip it for now in the interest of brevity. 2.11.9 Writing Documentation This is a new feature, so it should be documented. Add the following on line 925 of django/docs/ref/models/fields.txt beneath the existing docs for URLField: .. versionadded:: 1.5 The current value of the field will be displayed as a clickable link above the input widget. For more information on writing documentation, including an explanation of what the versionadded bit is all about, see Writing documentation. That page also includes an explanation of how to build a copy of the documentation locally, so you can preview the HTML that will be generated. 2.11.10 Generating a patch for your changes Now it’s time to generate a patch file that can be uploaded to Trac or applied to another copy of Django. To get a look at the content of your patch, run the following command: git diff This will display the differences between your current copy of Django (with your changes) and the revision that you initially checked out earlier in the tutorial. Once you’re done looking at the patch, hit the q key to exit back to the command line. If the patch’s content looked okay, you can run the following command to save the patch file to your current working directory: git diff > 17549.diff You should now have a file in the root Django directory called 17549.diff. This patch file contains all your changes and should look this: diff --git a/django/contrib/admin/widgets.py b/django/contrib/admin/widgets.py index 1e0bc2d..9e43a10 100644 --- a/django/contrib/admin/widgets.py +++ b/django/contrib/admin/widgets.py @@ -10,7 +10,7 @@ from django.contrib.admin.templatetags.admin_static import static from django.core.urlresolvers import reverse from django.forms.widgets import RadioFieldRenderer from django.forms.util import flatatt -from django.utils.html import escape, format_html, format_html_join +from django.utils.html import escape, format_html, format_html_join, smart_urlquote from django.utils.text import Truncator from django.utils.translation import ugettext as _ from django.utils.safestring import mark_safe @@ -306,6 +306,18 @@ class AdminURLFieldWidget(forms.TextInput): final_attrs.update(attrs) super(AdminURLFieldWidget, self).__init__(attrs=final_attrs) + def render(self, name, value, attrs=None): + html = super(AdminURLFieldWidget, self).render(name, value, attrs) 2.11. Writing your first patch for Django 71
- 76. Django Documentation, Release1.5.1 + if value: + value = force_text(self._format_value(value)) + final_attrs = {’href’: mark_safe(smart_urlquote(value))} + html = format_html( + ’<p class="url">{0} <a {1}>{2}</a><br />{3} {4}</p>’, + _(’Currently:’), flatatt(final_attrs), value, + _(’Change:’), html + ) + return html + class AdminIntegerFieldWidget(forms.TextInput): class_name = ’vIntegerField’ diff --git a/docs/ref/models/fields.txt b/docs/ref/models/fields.txt index 809d56e..d44f85f 100644 --- a/docs/ref/models/fields.txt +++ b/docs/ref/models/fields.txt @@ -922,6 +922,10 @@ Like all :class:‘CharField‘ subclasses, :class:‘URLField‘ takes the optional :attr:‘~CharField.max_length‘argument. If you don’t specify :attr:‘~CharField.max_length‘, a default of 200 is used. +.. versionadded:: 1.5 + +The current value of the field will be displayed as a clickable link above the +input widget. Relationship fields =================== diff --git a/tests/regressiontests/admin_widgets/tests.py b/tests/regressiontests/admin_widgets/tests.index 4b11543..94acc6d 100644 --- a/tests/regressiontests/admin_widgets/tests.py +++ b/tests/regressiontests/admin_widgets/tests.py @@ -265,6 +265,35 @@ class AdminSplitDateTimeWidgetTest(DjangoTestCase): ’<p class="datetime">Datum: <input value="01.12.2007" type="text" class="vDateField" ) +class AdminURLWidgetTest(DjangoTestCase): + def test_render(self): + w = widgets.AdminURLFieldWidget() + self.assertHTMLEqual( + conditional_escape(w.render(’test’, ’’)), + ’<input class="vURLField" name="test" type="text" />’ + ) + self.assertHTMLEqual( + conditional_escape(w.render(’test’, ’http://example.com’)), + ’<p class="url">Currently:<a href="http://example.com">http://example.com</a><br />Change:<+ ) + + def test_render_idn(self): + w = widgets.AdminURLFieldWidget() + self.assertHTMLEqual( + conditional_escape(w.render(’test’, ’http://example-äüö.com’)), + ’<p class="url">Currently:<a href="http://xn--example--7za4pnc.com">http://example-äüö.com</+ ) + + def test_render_quoting(self): + w = widgets.AdminURLFieldWidget() + self.assertHTMLEqual( 72 Chapter 2. Getting started
- 77. Django Documentation, Release1.5.1 + conditional_escape(w.render(’test’, ’http://example.com/<sometag>some text</sometag>’)), + ’<p class="url">Currently:<a href="http://example.com/%3Csometag%3Esome%20text%3C/sometag%+ ) + self.assertHTMLEqual( + conditional_escape(w.render(’test’, ’http://example-äüö.com/<sometag>some text</sometag>’)), + ’<p class="url">Currently:<a href="http://xn--example--7za4pnc.com/%3Csometag%3Esome%20text%+ ) class AdminFileWidgetTest(DjangoTestCase): def test_render(self): 2.11.11 So what do I do next? Congratulations, you’ve generated your very first Django patch! Now that you’ve got that under your belt, you can put those skills to good use by helping to improve Django’s codebase. Generating patches and attaching them to Trac tickets is useful, however, since we are using git - adopting a more git oriented workflow is recommended. Since we never committed our changes locally, perform the following to get your git branch back to a good starting point: git reset --hard HEAD git checkout master More information for new contributors Before you get too into writing patches for Django, there’s a little more information on contributing that you should probably take a look at: • You should make sure to read Django’s documentation on claiming tickets and submitting patches. It covers Trac etiquette, how to claim tickets for yourself, expected coding style for patches, and many other important details. • First time contributors should also read Django’s documentation for first time contributors. It has lots of good advice for those of us who are new to helping out with Django. • After those, if you’re still hungry for more information about contributing, you can always browse through the rest of Django’s documentation on contributing. It contains a ton of useful information and should be your first source for answering any questions you might have. Finding your first real ticket Once you’ve looked through some of that information, you’ll be ready to go out and find a ticket of your own to write a patch for. Pay special attention to tickets with the “easy pickings” criterion. These tickets are often much simpler in nature and are great for first time contributors. Once you’re familiar with contributing to Django, you can move on to writing patches for more difficult and complicated tickets. If you just want to get started already (and nobody would blame you!), try taking a look at the list of easy tickets that need patches and the easy tickets that have patches which need improvement. If you’re familiar with writing tests, you can also look at the list of easy tickets that need tests. Just remember to follow the guidelines about claiming tickets that were mentioned in the link to Django’s documentation on claiming tickets and submitting patches. 2.11. Writing your first patch for Django 73
- 78. Django Documentation, Release1.5.1 What’s next? After a ticket has a patch, it needs to be reviewed by a second set of eyes. After uploading a patch or submitting a pull request, be sure to update the ticket metadata by setting the flags on the ticket to say “has patch”, “doesn’t need tests”, etc, so others can find it for review. Contributing doesn’t necessarily always mean writing a patch from scratch. Reviewing existing patches is also a very helpful contribution. See Triaging tickets for details. See Also: If you’re new to Python, you might want to start by getting an idea of what the language is like. Django is 100% Python, so if you’ve got minimal comfort with Python you’ll probably get a lot more out of Django. If you’re new to programming entirely, you might want to start with this list of Python resources for non-programmers If you already know a few other languages and want to get up to speed with Python quickly, we recommend Dive Into Python (also available in a dead-tree version). If that’s not quite your style, there are quite a few other books about Python. 74 Chapter 2. Getting started
- 79. CHAPTER THREE USINGDJANGO Introductions to all the key parts of Django you’ll need to know: 3.1 How to install Django This document will get you up and running with Django. 3.1.1 Install Python Being a Python Web framework, Django requires Python. It works with any Python version from 2.6.5 to 2.7. It also features experimental support for versions from 3.2.3 to 3.3. Get Python at http://www.python.org. If you’re running Linux or Mac OS X, you probably already have it installed. Django on Jython If you use Jython (a Python implementation for the Java platform), you’ll need to follow a few additional steps. See Running Django on Jython for details. Python onWindows On Windows, you might need to adjust your PATH environment variable to include paths to Python executable and additional scripts. For example, if your Python is installed in C:Python27, the following paths need to be added to PATH: C:Python27;C:Python27Scripts; 3.1.2 Install Apache and mod_wsgi If you just want to experiment with Django, skip ahead to the next section; Django includes a lightweight web server you can use for testing, so you won’t need to set up Apache until you’re ready to deploy Django in production. If you want to use Django on a production site, use Apache with mod_wsgi. mod_wsgi can operate in one of two modes: an embedded mode and a daemon mode. In embedded mode, mod_wsgi is similar to mod_perl – it embeds Python within Apache and loads Python code into memory when the server starts. Code stays in memory throughout the life of an Apache process, which leads to significant performance gains over other server arrangements. In daemon 75
- 80. Django Documentation, Release1.5.1 mode, mod_wsgi spawns an independent daemon process that handles requests. The daemon process can run as a different user than the Web server, possibly leading to improved security, and the daemon process can be restarted without restarting the entire Apache Web server, possibly making refreshing your codebase more seamless. Consult the mod_wsgi documentation to determine which mode is right for your setup. Make sure you have Apache installed, with the mod_wsgi module activated. Django will work with any version of Apache that supports mod_wsgi. See How to use Django with mod_wsgi for information on how to configure mod_wsgi once you have it installed. If you can’t use mod_wsgi for some reason, fear not: Django supports many other deployment options. One is uWSGI; it works very well with nginx. Another is FastCGI, perfect for using Django with servers other than Apache. Additionally, Django follows the WSGI spec (PEP 3333), which allows it to run on a variety of server platforms. See the server-arrangements wiki page for specific installation instructions for each platform. 3.1.3 Get your database running If you plan to use Django’s database API functionality, you’ll need to make sure a database server is running. Django supports many different database servers and is officially supported with PostgreSQL, MySQL, Oracle and SQLite. If you are developing a simple project or something you don’t plan to deploy in a production environment, SQLite is generally the simplest option as it doesn’t require running a separate server. However, SQLite has many differences from other databases, so if you are working on something substantial, it’s recommended to develop with the same database as you plan on using in production. In addition to the officially supported databases, there are backends provided by 3rd parties that allow you to use other databases with Django: • Sybase SQL Anywhere • IBM DB2 • Microsoft SQL Server 2005 • Firebird • ODBC The Django versions and ORM features supported by these unofficial backends vary considerably. Queries regarding the specific capabilities of these unofficial backends, along with any support queries, should be directed to the support channels provided by each 3rd party project. In addition to a database backend, you’ll need to make sure your Python database bindings are installed. • If you’re using PostgreSQL, you’ll need the postgresql_psycopg2 package. You might want to refer to our PostgreSQL notes for further technical details specific to this database. If you’re on Windows, check out the unofficial compiled Windows version. • If you’re using MySQL, you’ll need the MySQL-python package, version 1.2.1p2 or higher. You will also want to read the database-specific notes for the MySQL backend. • If you’re using Oracle, you’ll need a copy of cx_Oracle, but please read the database-specific notes for the Oracle backend for important information regarding supported versions of both Oracle and cx_Oracle. • If you’re using an unofficial 3rd party backend, please consult the documentation provided for any additional requirements. If you plan to use Django’s manage.py syncdb command to automatically create database tables for your models (after first installing Django and creating a project), you’ll need to ensure that Django has permission to create and alter tables in the database you’re using; if you plan to manually create the tables, you can simply grant Django SELECT, INSERT, UPDATE and DELETE permissions. On some databases, Django will need ALTER TABLE privileges during syncdb but won’t issue ALTER TABLE statements on a table once syncdb has created it. After creating 76 Chapter 3. Using Django
- 81. Django Documentation, Release1.5.1 a database user with these permissions, you’ll specify the details in your project’s settings file, see DATABASES for details. If you’re using Django’s testing framework to test database queries, Django will need permission to create a test database. 3.1.4 Remove any old versions of Django If you are upgrading your installation of Django from a previous version, you will need to uninstall the old Django version before installing the new version. If you installed Django using pip or easy_install previously, installing with pip or easy_install again will automatically take care of the old version, so you don’t need to do it yourself. If you previously installed Django using python setup.py install, uninstalling is as simple as deleting the django directory from your Python site-packages. To find the directory you need to remove, you can run the following at your shell prompt (not the interactive Python prompt): python -c "import sys; sys.path = sys.path[1:]; import django; print(django.__path__)" 3.1.5 Install the Django code Installation instructions are slightly different depending on whether you’re installing a distribution-specific package, downloading the latest official release, or fetching the latest development version. It’s easy, no matter which way you choose. Installing a distribution-specific package Check the distribution specific notes to see if your platform/distribution provides official Django packages/installers. Distribution-provided packages will typically allow for automatic installation of dependencies and easy upgrade paths. Installing an official release with pip This is the recommended way to install Django. 1. Install pip. The easiest is to use the standalone pip installer. If your distribution already has pip installed, you might need to update it if it’s outdated. (If it’s outdated, you’ll know because installation won’t work.) 2. (optional) Take a look at virtualenv and virtualenvwrapper. These tools provide isolated Python environments, which are more practical than installing packages systemwide. They also allow installing packages without administrator privileges. It’s up to you to decide if you want to learn and use them. 3. If you’re using Linux, Mac OS X or some other flavor of Unix, enter the command sudo pip install Django at the shell prompt. If you’re using Windows, start a command shell with administrator privi-leges and run the command pip install Django. This will install Django in your Python installation’s site-packages directory. If you’re using a virtualenv, you don’t need sudo or administrator privileges, and this will install Django in the virtualenv’s site-packages directory. 3.1. How to install Django 77
- 82. Django Documentation, Release1.5.1 Installing an official release manually 1. Download the latest release from our download page. 2. Untar the downloaded file (e.g. tar xzvf Django-X.Y.tar.gz, where X.Y is the version number of the latest release). If you’re using Windows, you can download the command-line tool bsdtar to do this, or you can use a GUI-based tool such as 7-zip. 3. Change into the directory created in step 2 (e.g. cd Django-X.Y). 4. If you’re using Linux, Mac OS X or some other flavor of Unix, enter the command sudo python setup.py install at the shell prompt. If you’re using Windows, start a command shell with administrator privileges and run the command python setup.py install. This will install Django in your Python installation’s site-packages directory. Removing an old version If you use this installation technique, it is particularly important that you remove any existing installations of Django first. Otherwise, you can end up with a broken installation that includes files from previous versions that have since been removed from Django. Installing the development version Tracking Django development If you decide to use the latest development version of Django, you’ll want to pay close attention to the development timeline, and you’ll want to keep an eye on the release notes for the upcoming release. This will help you stay on top of any new features you might want to use, as well as any changes you’ll need to make to your code when updating your copy of Django. (For stable releases, any necessary changes are documented in the release notes.) If you’d like to be able to update your Django code occasionally with the latest bug fixes and improvements, follow these instructions: 1. Make sure that you have Git installed and that you can run its commands from a shell. (Enter git help at a shell prompt to test this.) 2. Check out Django’s main development branch (the ‘trunk’ or ‘master’) like so: git clone git://github.com/django/django.git django-trunk This will create a directory django-trunk in your current directory. 3. Make sure that the Python interpreter can load Django’s code. The most convenient way to do this is via pip. Run the following command: sudo pip install -e django-trunk/ (If using a virtualenv you can omit sudo.) This will make Django’s code importable, and will also make the django-admin.py utility command avail-able. In other words, you’re all set! If you don’t have pip available, see the alternative instructions for installing the development version without pip. 78 Chapter 3. Using Django
- 83. Django Documentation, Release1.5.1 Warning: Don’t run sudo python setup.py install, because you’ve already carried out the equivalent actions in step 3. When you want to update your copy of the Django source code, just run the command git pull from within the django-trunk directory. When you do this, Git will automatically download any changes. Installing the development version without pip If you don’t have pip, you can instead manually modify Python’s search path. First follow steps 1 and 2 above, so that you have a django-trunk directory with a checkout of Django’s lat-est code in it. Then add a .pth file containing the full path to the django-trunk directory to your system’s site-packages directory. For example, on a Unix-like system: echo WORKING-DIR/django-trunk > SITE-PACKAGES-DIR/django.pth In the above line, change WORKING-DIR/django-trunk to match the full path to your new django-trunk directory, and change SITE-PACKAGES-DIR to match the location of your system’s site-packages directory. The location of the site-packages directory depends on the operating system, and the location in which Python was installed. To find your system’s site-packages location, execute the following: python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())" (Note that this should be run from a shell prompt, not a Python interactive prompt.) Some Debian-based Linux distributions have separate site-packages directories for user-installed packages, such as when installing Django from a downloaded tarball. The command listed above will give you the system’s site-packages, the user’s directory can be found in /usr/local/lib/ instead of /usr/lib/. Next you need to make the django-admin.py utility available in your shell PATH. On Unix-like systems, create a symbolic link to the file django-trunk/django/bin/django-admin.py in a directory on your system path, such as /usr/local/bin. For example: ln -s WORKING-DIR/django-trunk/django/bin/django-admin.py /usr/local/bin/ (In the above line, change WORKING-DIR to match the full path to your new django-trunk directory.) This simply lets you type django-admin.py from within any directory, rather than having to qualify the command with the full path to the file. On Windows systems, the same result can be achieved by copying the file django-trunk/django/bin/django-admin.py to somewhere on your system path, for example C:Python27Scripts. 3.2 Models and databases A model is the single, definitive source of data about your data. It contains the essential fields and behaviors of the data you’re storing. Generally, each model maps to a single database table. 3.2.1 Models A model is the single, definitive source of data about your data. It contains the essential fields and behaviors of the data you’re storing. Generally, each model maps to a single database table. 3.2. Models and databases 79
- 84. Django Documentation, Release1.5.1 The basics: • Each model is a Python class that subclasses django.db.models.Model. • Each attribute of the model represents a database field. • With all of this, Django gives you an automatically-generated database-access API; see Making queries. Quick example This example model defines a Person, which has a first_name and last_name: from django.db import models class Person(models.Model): first_name = models.CharField(max_length=30) last_name = models.CharField(max_length=30) first_name and last_name are fields of the model. Each field is specified as a class attribute, and each attribute maps to a database column. The above Person model would create a database table like this: CREATE TABLE myapp_person ( "id" serial NOT NULL PRIMARY KEY, "first_name" varchar(30) NOT NULL, "last_name" varchar(30) NOT NULL ); Some technical notes: • The name of the table, myapp_person, is automatically derived from some model metadata but can be over-ridden. See Table names for more details.. • An id field is added automatically, but this behavior can be overridden. See Automatic primary key fields. • The CREATE TABLE SQL in this example is formatted using PostgreSQL syntax, but it’s worth noting Django uses SQL tailored to the database backend specified in your settings file. Using models Once you have defined your models, you need to tell Django you’re going to use those models. Do this by editing your settings file and changing the INSTALLED_APPS setting to add the name of the module that contains your models.py. For example, if the models for your application live in the module myapp.models (the package structure that is created for an application by the manage.py startapp script), INSTALLED_APPS should read, in part: INSTALLED_APPS = ( #... ’myapp’, #... ) When you add new apps to INSTALLED_APPS, be sure to run manage.py syncdb. 80 Chapter 3. Using Django
- 85. Django Documentation, Release1.5.1 Fields The most important part of a model – and the only required part of a model – is the list of database fields it defines. Fields are specified by class attributes. Be careful not to choose field names that conflict with the models API like clean, save, or delete. Example: class Musician(models.Model): first_name = models.CharField(max_length=50) last_name = models.CharField(max_length=50) instrument = models.CharField(max_length=100) class Album(models.Model): artist = models.ForeignKey(Musician) name = models.CharField(max_length=100) release_date = models.DateField() num_stars = models.IntegerField() Field types Each field in your model should be an instance of the appropriate Field class. Django uses the field class types to determine a few things: • The database column type (e.g. INTEGER, VARCHAR). • The default widget to use when rendering a form field (e.g. <input type="text">, <select>). • The minimal validation requirements, used in Django’s admin and in automatically-generated forms. Django ships with dozens of built-in field types; you can find the complete list in the model field reference. You can easily write your own fields if Django’s built-in ones don’t do the trick; see Writing custom model fields. Field options Each field takes a certain set of field-specific arguments (documented in the model field reference). For example, CharField (and its subclasses) require a max_length argument which specifies the size of the VARCHAR database field used to store the data. There’s also a set of common arguments available to all field types. All are optional. They’re fully explained in the reference, but here’s a quick summary of the most often-used ones: null If True, Django will store empty values as NULL in the database. Default is False. blank If True, the field is allowed to be blank. Default is False. Note that this is different than null. null is purely database-related, whereas blank is validation-related. If a field has blank=True, form validation will allow entry of an empty value. If a field has blank=False, the field will be required. choices An iterable (e.g., a list or tuple) of 2-tuples to use as choices for this field. If this is given, the default form widget will be a select box instead of the standard text field and will limit choices to the choices given. A choices list looks like this: YEAR_IN_SCHOOL_CHOICES = ( (’FR’, ’Freshman’), (’SO’, ’Sophomore’), (’JR’, ’Junior’), 3.2. Models and databases 81
- 86. Django Documentation, Release1.5.1 (’SR’, ’Senior’), (’GR’, ’Graduate’), ) The first element in each tuple is the value that will be stored in the database, the second element will be displayed by the default form widget or in a ModelChoiceField. Given an instance of a model object, the display value for a choices field can be accessed using the get_FOO_display method. For example: from django.db import models class Person(models.Model): SHIRT_SIZES = ( (’S’, ’Small’), (’M’, ’Medium’), (’L’, ’Large’), ) name = models.CharField(max_length=60) shirt_size = models.CharField(max_length=2, choices=SHIRT_SIZES) >>> p = Person(name="Fred Flintstone", shirt_size="L") >>> p.save() >>> p.shirt_size u’L’ >>> p.get_shirt_size_display() u’Large’ default The default value for the field. This can be a value or a callable object. If callable it will be called every time a new object is created. help_text Extra “help” text to be displayed with the form widget. It’s useful for documentation even if your field isn’t used on a form. primary_key If True, this field is the primary key for the model. If you don’t specify primary_key=True for any fields in your model, Django will automatically add an IntegerField to hold the primary key, so you don’t need to set primary_key=True on any of your fields unless you want to override the default primary-key behavior. For more, see Automatic primary key fields. unique If True, this field must be unique throughout the table. Again, these are just short descriptions of the most common field options. Full details can be found in the common model field option reference. Automatic primary key fields By default, Django gives each model the following field: id = models.AutoField(primary_key=True) This is an auto-incrementing primary key. If you’d like to specify a custom primary key, just specify primary_key=True on one of your fields. If Django sees you’ve explicitly set Field.primary_key, it won’t add the automatic id column. Each model requires exactly one field to have primary_key=True. 82 Chapter 3. Using Django
- 87. Django Documentation, Release1.5.1 Verbose field names Each field type, except for ForeignKey, ManyToManyField and OneToOneField, takes an optional first positional argument – a verbose name. If the verbose name isn’t given, Django will automatically create it using the field’s attribute name, converting underscores to spaces. In this example, the verbose name is "person’s first name": first_name = models.CharField("person’s first name", max_length=30) In this example, the verbose name is "first name": first_name = models.CharField(max_length=30) ForeignKey, ManyToManyField and OneToOneField require the first argument to be a model class, so use the verbose_name keyword argument: poll = models.ForeignKey(Poll, verbose_name="the related poll") sites = models.ManyToManyField(Site, verbose_name="list of sites") place = models.OneToOneField(Place, verbose_name="related place") The convention is not to capitalize the first letter of the verbose_name. Django will automatically capitalize the first letter where it needs to. Relationships Clearly, the power of relational databases lies in relating tables to each other. Django offers ways to define the three most common types of database relationships: many-to-one, many-to-many and one-to-one. Many-to-one relationships To define a many-to-one relationship, use django.db.models.ForeignKey. You use it just like any other Field type: by including it as a class attribute of your model. ForeignKey requires a positional argument: the class to which the model is related. For example, if a Car model has a Manufacturer – that is, a Manufacturer makes multiple cars but each Car only has one Manufacturer – use the following definitions: class Manufacturer(models.Model): # ... class Car(models.Model): manufacturer = models.ForeignKey(Manufacturer) # ... You can also create recursive relationships (an object with a many-to-one relationship to itself) and relationships to models not yet defined; see the model field reference for details. It’s suggested, but not required, that the name of a ForeignKey field (manufacturer in the example above) be the name of the model, lowercase. You can, of course, call the field whatever you want. For example: class Car(models.Model): company_that_makes_it = models.ForeignKey(Manufacturer) # ... See Also: ForeignKey fields accept a number of extra arguments which are explained in the model field reference. These options help define how the relationship should work; all are optional. 3.2. Models and databases 83
- 88. Django Documentation, Release1.5.1 For details on accessing backwards-related objects, see the Following relationships backward example. For sample code, see the Many-to-one relationship model example. Many-to-many relationships To define a many-to-many relationship, use ManyToManyField. You use it just like any other Field type: by including it as a class attribute of your model. ManyToManyField requires a positional argument: the class to which the model is related. For example, if a Pizza has multiple Topping objects – that is, a Topping can be on multiple pizzas and each Pizza has multiple toppings – here’s how you’d represent that: class Topping(models.Model): # ... class Pizza(models.Model): # ... toppings = models.ManyToManyField(Topping) As with ForeignKey, you can also create recursive relationships (an object with a many-to-many relationship to itself) and relationships to models not yet defined; see the model field reference for details. It’s suggested, but not required, that the name of a ManyToManyField (toppings in the example above) be a plural describing the set of related model objects. It doesn’t matter which model has the ManyToManyField, but you should only put it in one of the models – not both. Generally, ManyToManyField instances should go in the object that’s going to be edited on a form. In the above example, toppings is in Pizza (rather than Topping having a pizzas ManyToManyField ) because it’s more natural to think about a pizza having toppings than a topping being on multiple pizzas. The way it’s set up above, the Pizza form would let users select the toppings. See Also: See the Many-to-many relationship model example for a full example. ManyToManyField fields also accept a number of extra arguments which are explained in the model field reference. These options help define how the relationship should work; all are optional. Extra fields on many-to-many relationships When you’re only dealing with simple many-to-many relationships such as mixing and matching pizzas and toppings, a standard ManyToManyField is all you need. However, some-times you may need to associate data with the relationship between two models. For example, consider the case of an application tracking the musical groups which musicians belong to. There is a many-to-many relationship between a person and the groups of which they are a member, so you could use a ManyToManyField to represent this relationship. However, there is a lot of detail about the membership that you might want to collect, such as the date at which the person joined the group. For these situations, Django allows you to specify the model that will be used to govern the many-to-many rela-tionship. You can then put extra fields on the intermediate model. The intermediate model is associated with the ManyToManyField using the through argument to point to the model that will act as an intermediary. For our musician example, the code would look something like this: class Person(models.Model): name = models.CharField(max_length=128) def __unicode__(self): return self.name 84 Chapter 3. Using Django
- 89. Django Documentation, Release1.5.1 class Group(models.Model): name = models.CharField(max_length=128) members = models.ManyToManyField(Person, through=’Membership’) def __unicode__(self): return self.name class Membership(models.Model): person = models.ForeignKey(Person) group = models.ForeignKey(Group) date_joined = models.DateField() invite_reason = models.CharField(max_length=64) When you set up the intermediary model, you explicitly specify foreign keys to the models that are involved in the ManyToMany relation. This explicit declaration defines how the two models are related. There are a few restrictions on the intermediate model: • Your intermediate model must contain one - and only one - foreign key to the target model (this would be Person in our example). If you have more than one foreign key, a validation error will be raised. • Your intermediate model must contain one - and only one - foreign key to the source model (this would be Group in our example). If you have more than one foreign key, a validation error will be raised. • The only exception to this is a model which has a many-to-many relationship to itself, through an intermediary model. In this case, two foreign keys to the same model are permitted, but they will be treated as the two (different) sides of the many-to-many relation. • When defining a many-to-many relationship from a model to itself, using an intermediary model, you must use symmetrical=False (see the model field reference). Now that you have set up your ManyToManyField to use your intermediary model (Membership, in this case), you’re ready to start creating some many-to-many relationships. You do this by creating instances of the intermediate model: >>> ringo = Person.objects.create(name="Ringo Starr") >>> paul = Person.objects.create(name="Paul McCartney") >>> beatles = Group.objects.create(name="The Beatles") >>> m1 = Membership(person=ringo, group=beatles, ... date_joined=date(1962, 8, 16), ... invite_reason= "Needed a new drummer.") >>> m1.save() >>> beatles.members.all() [<Person: Ringo Starr>] >>> ringo.group_set.all() [<Group: The Beatles>] >>> m2 = Membership.objects.create(person=paul, group=beatles, ... date_joined=date(1960, 8, 1), ... invite_reason= "Wanted to form a band.") >>> beatles.members.all() [<Person: Ringo Starr>, <Person: Paul McCartney>] Unlike normal many-to-many fields, you can’t use add, create, or assignment (i.e., beatles.members = [...]) to create relationships: # THIS WILL NOT WORK >>> beatles.members.add(john) # NEITHER WILL THIS >>> beatles.members.create(name="George Harrison") 3.2. Models and databases 85
- 90. Django Documentation, Release1.5.1 # AND NEITHER WILL THIS >>> beatles.members = [john, paul, ringo, george] Why? You can’t just create a relationship between a Person and a Group - you need to specify all the detail for the relationship required by the Membership model. The simple add, create and assignment calls don’t provide a way to specify this extra detail. As a result, they are disabled for many-to-many relationships that use an intermediate model. The only way to create this type of relationship is to create instances of the intermediate model. The remove() method is disabled for similar reasons. However, the clear() method can be used to remove all many-to-many relationships for an instance: # Beatles have broken up >>> beatles.members.clear() Once you have established the many-to-many relationships by creating instances of your intermediate model, you can issue queries. Just as with normal many-to-many relationships, you can query using the attributes of the many-to-many- related model: # Find all the groups with a member whose name starts with ’Paul’ >>> Group.objects.filter(members__name__startswith=’Paul’) [<Group: The Beatles>] As you are using an intermediate model, you can also query on its attributes: # Find all the members of the Beatles that joined after 1 Jan 1961 >>> Person.objects.filter( ... group__name=’The Beatles’, ... membership__date_joined__gt=date(1961,1,1)) [<Person: Ringo Starr] If you need to access a membership’s information you may do so by directly querying the Membership model: >>> ringos_membership = Membership.objects.get(group=beatles, person=ringo) >>> ringos_membership.date_joined datetime.date(1962, 8, 16) >>> ringos_membership.invite_reason u’Needed a new drummer.’ Another way to access the same information is by querying the many-to-many reverse relationship from a Person object: >>> ringos_membership = ringo.membership_set.get(group=beatles) >>> ringos_membership.date_joined datetime.date(1962, 8, 16) >>> ringos_membership.invite_reason u’Needed a new drummer.’ One-to-one relationships To define a one-to-one relationship, use OneToOneField. You use it just like any other Field type: by including it as a class attribute of your model. This is most useful on the primary key of an object when that object “extends” another object in some way. OneToOneField requires a positional argument: the class to which the model is related. For example, if you were building a database of “places”, you would build pretty standard stuff such as address, phone number, etc. in the database. Then, if you wanted to build a database of restaurants on top of the places, instead of repeating yourself and replicating those fields in the Restaurant model, you could make Restaurant have a OneToOneField to Place (because a restaurant “is a” place; in fact, to handle this you’d typically use inheritance, which involves an implicit one-to-one relation). 86 Chapter 3. Using Django
- 91. Django Documentation, Release1.5.1 As with ForeignKey, a recursive relationship can be defined and references to as-yet undefined models can be made; see the model field reference for details. See Also: See the One-to-one relationship model example for a full example. OneToOneField fields also accept one specific, optional parent_link argument described in the model field reference. OneToOneField classes used to automatically become the primary key on a model. This is no longer true (although you can manually pass in the primary_key argument if you like). Thus, it’s now possible to have multiple fields of type OneToOneField on a single model. Models across files It’s perfectly OK to relate a model to one from another app. To do this, import the related model at the top of the file where your model is defined. Then, just refer to the other model class wherever needed. For example: from geography.models import ZipCode class Restaurant(models.Model): # ... zip_code = models.ForeignKey(ZipCode) Field name restrictions Django places only two restrictions on model field names: 1. A field name cannot be a Python reserved word, because that would result in a Python syntax error. For example: class Example(models.Model): pass = models.IntegerField() # ’pass’ is a reserved word! 2. A field name cannot contain more than one underscore in a row, due to the way Django’s query lookup syntax works. For example: class Example(models.Model): foo__bar = models.IntegerField() # ’foo__bar’ has two underscores! These limitations can be worked around, though, because your field name doesn’t necessarily have to match your database column name. See the db_column option. SQL reserved words, such as join, where or select, are allowed as model field names, because Django escapes all database table names and column names in every underlying SQL query. It uses the quoting syntax of your particular database engine. Custom field types If one of the existing model fields cannot be used to fit your purposes, or if you wish to take advantage of some less common database column types, you can create your own field class. Full coverage of creating your own fields is provided in Writing custom model fields. 3.2. Models and databases 87
- 92. Django Documentation, Release1.5.1 Meta options Give your model metadata by using an inner class Meta, like so: class Ox(models.Model): horn_length = models.IntegerField() class Meta: ordering = ["horn_length"] verbose_name_plural = "oxen" Model metadata is “anything that’s not a field”, such as ordering options (ordering), database table name (db_table), or human-readable singular and plural names (verbose_name and verbose_name_plural). None are required, and adding class Meta to a model is completely optional. A complete list of all possible Meta options can be found in the model option reference. Model methods Define custom methods on a model to add custom “row-level” functionality to your objects. Whereas Manager methods are intended to do “table-wide” things, model methods should act on a particular model instance. This is a valuable technique for keeping business logic in one place – the model. For example, this model has a few custom methods: from django.contrib.localflavor.us.models import USStateField class Person(models.Model): first_name = models.CharField(max_length=50) last_name = models.CharField(max_length=50) birth_date = models.DateField() address = models.CharField(max_length=100) city = models.CharField(max_length=50) state = USStateField() # Yes, this is America-centric... def baby_boomer_status(self): "Returns the person’s baby-boomer status." import datetime if self.birth_date < datetime.date(1945, 8, 1): return "Pre-boomer" elif self.birth_date < datetime.date(1965, 1, 1): return "Baby boomer" else: return "Post-boomer" def is_midwestern(self): "Returns True if this person is from the Midwest." return self.state in (’IL’, ’WI’, ’MI’, ’IN’, ’OH’, ’IA’, ’MO’) def _get_full_name(self): "Returns the person’s full name." return ’%s %s’ % (self.first_name, self.last_name) full_name = property(_get_full_name) The last method in this example is a property. The model instance reference has a complete list of methods automatically given to each model. You can override most of these – see overriding predefined model methods, below – but there are a couple that you’ll almost always 88 Chapter 3. Using Django
- 93. Django Documentation, Release1.5.1 want to define: __unicode__() A Python “magic method” that returns a unicode “representation” of any object. This is what Python and Django will use whenever a model instance needs to be coerced and displayed as a plain string. Most notably, this happens when you display an object in an interactive console or in the admin. You’ll always want to define this method; the default isn’t very helpful at all. get_absolute_url() This tells Django how to calculate the URL for an object. Django uses this in its admin interface, and any time it needs to figure out a URL for an object. Any object that has a URL that uniquely identifies it should define this method. Overriding predefined model methods There’s another set of model methods that encapsulate a bunch of database behavior that you’ll want to customize. In particular you’ll often want to change the way save() and delete() work. You’re free to override these methods (and any other model method) to alter behavior. A classic use-case for overriding the built-in methods is if you want something to happen whenever you save an object. For example (see save() for documentation of the parameters it accepts): class Blog(models.Model): name = models.CharField(max_length=100) tagline = models.TextField() def save(self, *args, **kwargs): do_something() super(Blog, self).save(*args, **kwargs) # Call the "real" save() method. do_something_else() You can also prevent saving: class Blog(models.Model): name = models.CharField(max_length=100) tagline = models.TextField() def save(self, *args, **kwargs): if self.name == "Yoko Ono’s blog": return # Yoko shall never have her own blog! else: super(Blog, self).save(*args, **kwargs) # Call the "real" save() method. It’s important to remember to call the superclass method – that’s that super(Blog, self).save(*args, **kwargs) business – to ensure that the object still gets saved into the database. If you forget to call the super-class method, the default behavior won’t happen and the database won’t get touched. It’s also important that you pass through the arguments that can be passed to the model method – that’s what the *args, **kwargs bit does. Django will, from time to time, extend the capabilities of built-in model methods, adding new arguments. If you use *args, **kwargs in your method definitions, you are guaranteed that your code will automatically support those arguments when they are added. Overridden model methods are not called on bulk operations Note that the delete() method for an object is not necessarily called when deleting objects in bulk using a QuerySet. To ensure customized delete logic gets executed, you can use pre_delete and/or post_delete signals. Unfortunately, there isn’t a workaround when creating or updating objects in bulk, since none of save(), pre_save, and post_save are called. 3.2. Models and databases 89
- 94. Django Documentation, Release1.5.1 Executing custom SQL Another common pattern is writing custom SQL statements in model methods and module-level methods. For more details on using raw SQL, see the documentation on using raw SQL. Model inheritance Model inheritance in Django works almost identically to the way normal class inheritance works in Python. The only decision you have to make is whether you want the parent models to be models in their own right (with their own database tables), or if the parents are just holders of common information that will only be visible through the child models. There are three styles of inheritance that are possible in Django. 1. Often, you will just want to use the parent class to hold information that you don’t want to have to type out for each child model. This class isn’t going to ever be used in isolation, so Abstract base classes are what you’re after. 2. If you’re subclassing an existing model (perhaps something from another application entirely) and want each model to have its own database table, Multi-table inheritance is the way to go. 3. Finally, if you only want to modify the Python-level behavior of a model, without changing the models fields in any way, you can use Proxy models. Abstract base classes Abstract base classes are useful when you want to put some common information into a number of other models. You write your base class and put abstract=True in the Meta class. This model will then not be used to create any database table. Instead, when it is used as a base class for other models, its fields will be added to those of the child class. It is an error to have fields in the abstract base class with the same name as those in the child (and Django will raise an exception). An example: class CommonInfo(models.Model): name = models.CharField(max_length=100) age = models.PositiveIntegerField() class Meta: abstract = True class Student(CommonInfo): home_group = models.CharField(max_length=5) The Student model will have three fields: name, age and home_group. The CommonInfo model cannot be used as a normal Django model, since it is an abstract base class. It does not generate a database table or have a manager, and cannot be instantiated or saved directly. For many uses, this type of model inheritance will be exactly what you want. It provides a way to factor out common information at the Python level, whilst still only creating one database table per child model at the database level. 90 Chapter 3. Using Django
- 95. Django Documentation, Release1.5.1 Meta inheritance When an abstract base class is created, Django makes any Meta inner class you declared in the base class available as an attribute. If a child class does not declare its own Meta class, it will inherit the parent’s Meta. If the child wants to extend the parent’s Meta class, it can subclass it. For example: class CommonInfo(models.Model): ... class Meta: abstract = True ordering = [’name’] class Student(CommonInfo): ... class Meta(CommonInfo.Meta): db_table = ’student_info’ Django does make one adjustment to the Meta class of an abstract base class: before installing the Meta attribute, it sets abstract=False. This means that children of abstract base classes don’t automatically become abstract classes themselves. Of course, you can make an abstract base class that inherits from another abstract base class. You just need to remember to explicitly set abstract=True each time. Some attributes won’t make sense to include in the Meta class of an abstract base class. For example, including db_table would mean that all the child classes (the ones that don’t specify their own Meta) would use the same database table, which is almost certainly not what you want. Be careful with related_name If you are using the related_name attribute on a ForeignKey or ManyToManyField, you must always specify a unique reverse name for the field. This would normally cause a problem in abstract base classes, since the fields on this class are included into each of the child classes, with exactly the same values for the attributes (including related_name) each time. To work around this problem, when you are using related_name in an abstract base class (only), part of the name should contain ’%(app_label)s’ and ’%(class)s’. • ’%(class)s’ is replaced by the lower-cased name of the child class that the field is used in. • ’%(app_label)s’ is replaced by the lower-cased name of the app the child class is contained within. Each installed application name must be unique and the model class names within each app must also be unique, therefore the resulting name will end up being different. For example, given an app common/models.py: class Base(models.Model): m2m = models.ManyToManyField(OtherModel, related_name="%(app_label)s_%(class)s_related") class Meta: abstract = True class ChildA(Base): pass class ChildB(Base): pass Along with another app rare/models.py: from common.models import Base class ChildB(Base): pass 3.2. Models and databases 91
- 96. Django Documentation, Release1.5.1 The reverse name of the common.ChildA.m2m field will be common_childa_related, whilst the reverse name of the common.ChildB.m2m field will be common_childb_related, and finally the reverse name of the rare.ChildB.m2m field will be rare_childb_related. It is up to you how you use the ’%(class)s’ and ’%(app_label)s portion to construct your related name, but if you forget to use it, Django will raise errors when you validate your models (or run syncdb). If you don’t specify a related_name attribute for a field in an abstract base class, the default reverse name will be the name of the child class followed by ’_set’, just as it normally would be if you’d declared the field directly on the child class. For example, in the above code, if the related_name attribute was omitted, the reverse name for the m2m field would be childa_set in the ChildA case and childb_set for the ChildB field. Multi-table inheritance The second type of model inheritance supported by Django is when each model in the hierarchy is a model all by itself. Each model corresponds to its own database table and can be queried and created individually. The inher-itance relationship introduces links between the child model and each of its parents (via an automatically-created OneToOneField). For example: class Place(models.Model): name = models.CharField(max_length=50) address = models.CharField(max_length=80) class Restaurant(Place): serves_hot_dogs = models.BooleanField() serves_pizza = models.BooleanField() All of the fields of Place will also be available in Restaurant, although the data will reside in a different database table. So these are both possible: >>> Place.objects.filter(name="Bob’s Cafe") >>> Restaurant.objects.filter(name="Bob’s Cafe") If you have a Place that is also a Restaurant, you can get from the Place object to the Restaurant object by using the lower-case version of the model name: >>> p = Place.objects.get(id=12) # If p is a Restaurant object, this will give the child class: >>> p.restaurant <Restaurant: ...> However, if p in the above example was not a Restaurant (it had been created directly as a Place object or was the parent of some other class), referring to p.restaurant would raise a Restaurant.DoesNotExist exception. Meta and multi-table inheritance In the multi-table inheritance situation, it doesn’t make sense for a child class to inherit from its parent’s Meta class. All the Meta options have already been applied to the parent class and applying them again would normally only lead to contradictory behavior (this is in contrast with the abstract base class case, where the base class doesn’t exist in its own right). So a child model does not have access to its parent’s Meta class. However, there are a few limited cases where the child inherits behavior from the parent: if the child does not specify an ordering attribute or a get_latest_by attribute, it will inherit these from its parent. If the parent has an ordering and you don’t want the child to have any natural ordering, you can explicitly disable it: class ChildModel(ParentModel): ... class Meta: 92 Chapter 3. Using Django
- 97. Django Documentation, Release1.5.1 # Remove parent’s ordering effect ordering = [] Inheritance and reverse relations Because multi-table inheritance uses an implicit OneToOneField to link the child and the parent, it’s possible to move from the parent down to the child, as in the above example. However, this uses up the name that is the default related_name value for ForeignKey and ManyToManyField relations. If you are putting those types of relations on a subclass of another model, you must specify the related_name attribute on each such field. If you forget, Django will raise an error when you run validate or syncdb. For example, using the above Place class again, let’s create another subclass with a ManyToManyField: class Supplier(Place): # Must specify related_name on all relations. customers = models.ManyToManyField(Restaurant, related_name=’provider’) Specifying the parent link field As mentioned, Django will automatically create a OneToOneField linking your child class back any non-abstract parent models. If you want to control the name of the attribute linking back to the parent, you can create your own OneToOneField and set parent_link=True to indicate that your field is the link back to the parent class. Proxy models When using multi-table inheritance, a new database table is created for each subclass of a model. This is usually the desired behavior, since the subclass needs a place to store any additional data fields that are not present on the base class. Sometimes, however, you only want to change the Python behavior of a model – perhaps to change the default manager, or add a new method. This is what proxy model inheritance is for: creating a proxy for the original model. You can create, delete and update instances of the proxy model and all the data will be saved as if you were using the original (non-proxied) model. The difference is that you can change things like the default model ordering or the default manager in the proxy, without having to alter the original. Proxy models are declared like normal models. You tell Django that it’s a proxy model by setting the proxy attribute of the Meta class to True. For example, suppose you want to add a method to the Person model described above. You can do it like this: class MyPerson(Person): class Meta: proxy = True def do_something(self): ... The MyPerson class operates on the same database table as its parent Person class. In particular, any new instances of Person will also be accessible through MyPerson, and vice-versa: >>> p = Person.objects.create(first_name="foobar") >>> MyPerson.objects.get(first_name="foobar") <MyPerson: foobar> You could also use a proxy model to define a different default ordering on a model. You might not always want to order the Person model, but regularly order by the last_name attribute when you use the proxy. This is easy: 3.2. Models and databases 93
- 98. Django Documentation, Release1.5.1 class OrderedPerson(Person): class Meta: ordering = ["last_name"] proxy = True Now normal Person queries will be unordered and OrderedPerson queries will be ordered by last_name. QuerySets still return the model that was requested There is no way to have Django return, say, a MyPerson object whenever you query for Person objects. A queryset for Person objects will return those types of objects. The whole point of proxy objects is that code relying on the original Person will use those and your own code can use the extensions you included (that no other code is relying on anyway). It is not a way to replace the Person (or any other) model everywhere with something of your own creation. Base class restrictions A proxy model must inherit from exactly one non-abstract model class. You can’t inherit from multiple non-abstract models as the proxy model doesn’t provide any connection between the rows in the different database tables. A proxy model can inherit from any number of abstract model classes, providing they do not define any model fields. Proxy models inherit any Meta options that they don’t define from their non-abstract model parent (the model they are proxying for). Proxy model managers If you don’t specify any model managers on a proxy model, it inherits the managers from its model parents. If you define a manager on the proxy model, it will become the default, although any managers defined on the parent classes will still be available. Continuing our example from above, you could change the default manager used when you query the Person model like this: class NewManager(models.Manager): ... class MyPerson(Person): objects = NewManager() class Meta: proxy = True If you wanted to add a new manager to the Proxy, without replacing the existing default, you can use the techniques described in the custom manager documentation: create a base class containing the new managers and inherit that after the primary base class: # Create an abstract class for the new manager. class ExtraManagers(models.Model): secondary = NewManager() class Meta: abstract = True class MyPerson(Person, ExtraManagers): class Meta: proxy = True You probably won’t need to do this very often, but, when you do, it’s possible. 94 Chapter 3. Using Django
- 99. Django Documentation, Release1.5.1 Differences between proxy inheritance and unmanaged models Proxy model inheritance might look fairly similar to creating an unmanaged model, using the managed attribute on a model’s Meta class. The two alternatives are not quite the same and it’s worth considering which one you should use. One difference is that you can (and, in fact, must unless you want an empty model) specify model fields on models with Meta.managed=False. You could, with careful setting of Meta.db_table create an unmanaged model that shadowed an existing model and add Python methods to it. However, that would be very repetitive and fragile as you need to keep both copies synchronized if you make any changes. The other difference that is more important for proxy models, is how model managers are handled. Proxy models are intended to behave exactly like the model they are proxying for. So they inherit the parent model’s managers, including the default manager. In the normal multi-table model inheritance case, children do not inherit managers from their parents as the custom managers aren’t always appropriate when extra fields are involved. The manager documentation has more details about this latter case. When these two features were implemented, attempts were made to squash them into a single option. It turned out that interactions with inheritance, in general, and managers, in particular, made the API very complicated and potentially difficult to understand and use. It turned out that two options were needed in any case, so the current separation arose. So, the general rules are: 1. If you are mirroring an existing model or database table and don’t want all the original database table columns, use Meta.managed=False. That option is normally useful for modeling database views and tables not under the control of Django. 2. If you are wanting to change the Python-only behavior of a model, but keep all the same fields as in the original, use Meta.proxy=True. This sets things up so that the proxy model is an exact copy of the storage structure of the original model when data is saved. Multiple inheritance Just as with Python’s subclassing, it’s possible for a Django model to inherit from multiple parent models. Keep in mind that normal Python name resolution rules apply. The first base class that a particular name (e.g. Meta) appears in will be the one that is used; for example, this means that if multiple parents contain a Meta class, only the first one is going to be used, and all others will be ignored. Generally, you won’t need to inherit from multiple parents. The main use-case where this is useful is for “mix-in” classes: adding a particular extra field or method to every class that inherits the mix-in. Try to keep your inheritance hierarchies as simple and straightforward as possible so that you won’t have to struggle to work out where a particular piece of information is coming from. Field name “hiding” is not permitted In normal Python class inheritance, it is permissible for a child class to override any attribute from the parent class. In Django, this is not permitted for attributes that are Field instances (at least, not at the moment). If a base class has a field called author, you cannot create another model field called author in any class that inherits from that base class. Overriding fields in a parent model leads to difficulties in areas such as initializing new instances (specifying which field is being initialized in Model.__init__) and serialization. These are features which normal Python class inheritance doesn’t have to deal with in quite the same way, so the difference between Django model inheritance and Python class inheritance isn’t arbitrary. This restriction only applies to attributes which are Field instances. Normal Python attributes can be overridden if you wish. It also only applies to the name of the attribute as Python sees it: if you are manually specifying the database column name, you can have the same column name appearing in both a child and an ancestor model for multi-table inheritance (they are columns in two different database tables). 3.2. Models and databases 95
- 100. Django Documentation, Release1.5.1 Django will raise a FieldError if you override any model field in any ancestor model. 3.2.2 Making queries Once you’ve created your data models, Django automatically gives you a database-abstraction API that lets you create, retrieve, update and delete objects. This document explains how to use this API. Refer to the data model reference for full details of all the various model lookup options. Throughout this guide (and in the reference), we’ll refer to the following models, which comprise a Weblog applica-tion: class Blog(models.Model): name = models.CharField(max_length=100) tagline = models.TextField() def __unicode__(self): return self.name class Author(models.Model): name = models.CharField(max_length=50) email = models.EmailField() def __unicode__(self): return self.name class Entry(models.Model): blog = models.ForeignKey(Blog) headline = models.CharField(max_length=255) body_text = models.TextField() pub_date = models.DateField() mod_date = models.DateField() authors = models.ManyToManyField(Author) n_comments = models.IntegerField() n_pingbacks = models.IntegerField() rating = models.IntegerField() def __unicode__(self): return self.headline Creating objects To represent database-table data in Python objects, Django uses an intuitive system: A model class represents a database table, and an instance of that class represents a particular record in the database table. To create an object, instantiate it using keyword arguments to the model class, then call save() to save it to the database. You import the model class from wherever it lives on the Python path, as you may expect. (We point this out here because previous Django versions required funky model importing.) Assuming models live in a file mysite/blog/models.py, here’s an example: >>> from blog.models import Blog >>> b = Blog(name=’Beatles Blog’, tagline=’All the latest Beatles news.’) >>> b.save() This performs an INSERT SQL statement behind the scenes. Django doesn’t hit the database until you explicitly call save(). 96 Chapter 3. Using Django
- 101. Django Documentation, Release1.5.1 The save() method has no return value. See Also: save() takes a number of advanced options not described here. See the documentation for save() for complete details. To create and save an object in a single step, use the create() method. Saving changes to objects To save changes to an object that’s already in the database, use save(). Given a Blog instance b5 that has already been saved to the database, this example changes its name and updates its record in the database: >> b5.name = ’New name’ >> b5.save() This performs an UPDATE SQL statement behind the scenes. Django doesn’t hit the database until you explicitly call save(). Saving ForeignKey and ManyToManyField fields Updating a ForeignKey field works exactly the same way as saving a normal field – simply assign an object of the right type to the field in question. This example updates the blog attribute of an Entry instance entry: >>> from blog.models import Entry >>> entry = Entry.objects.get(pk=1) >>> cheese_blog = Blog.objects.get(name="Cheddar Talk") >>> entry.blog = cheese_blog >>> entry.save() Updating a ManyToManyField works a little differently – use the add() method on the field to add a record to the relation. This example adds the Author instance joe to the entry object: >>> from blog.models import Author >>> joe = Author.objects.create(name="Joe") >>> entry.authors.add(joe) To add multiple records to a ManyToManyField in one go, include multiple arguments in the call to add(), like this: >>> john = Author.objects.create(name="John") >>> paul = Author.objects.create(name="Paul") >>> george = Author.objects.create(name="George") >>> ringo = Author.objects.create(name="Ringo") >>> entry.authors.add(john, paul, george, ringo) Django will complain if you try to assign or add an object of the wrong type. Retrieving objects To retrieve objects from your database, construct a QuerySet via a Manager on your model class. A QuerySet represents a collection of objects from your database. It can have zero, one or many filters – criteria that narrow down the collection based on given parameters. In SQL terms, a QuerySet equates to a SELECT statement, and a filter is a limiting clause such as WHERE or LIMIT. 3.2. Models and databases 97
- 102. Django Documentation, Release1.5.1 You get a QuerySet by using your model’s Manager. Each model has at least one Manager, and it’s called objects by default. Access it directly via the model class, like so: >>> Blog.objects <django.db.models.manager.Manager object at ...> >>> b = Blog(name=’Foo’, tagline=’Bar’) >>> b.objects Traceback: ... AttributeError: "Manager isn’t accessible via Blog instances." Note: Managers are accessible only via model classes, rather than from model instances, to enforce a separation between “table-level” operations and “record-level” operations. The Manager is the main source of QuerySets for a model. For example, Blog.objects.all() returns a QuerySet that contains all Blog objects in the database. Retrieving all objects The simplest way to retrieve objects from a table is to get all of them. To do this, use the all() method on a Manager: >>> all_entries = Entry.objects.all() The all() method returns a QuerySet of all the objects in the database. Retrieving specific objects with filters The QuerySet returned by all() describes all objects in the database table. Usually, though, you’ll need to select only a subset of the complete set of objects. To create such a subset, you refine the initial QuerySet, adding filter conditions. The two most common ways to refine a QuerySet are: filter(**kwargs) Returns a new QuerySet containing objects that match the given lookup parameters. exclude(**kwargs) Returns a new QuerySet containing objects that do not match the given lookup parame-ters. The lookup parameters (**kwargs in the above function definitions) should be in the format described in Field lookups below. For example, to get a QuerySet of blog entries from the year 2006, use filter() like so: Entry.objects.filter(pub_date__year=2006) With the default manager class, it is the same as: Entry.objects.all().filter(pub_date__year=2006) Chaining filters The result of refining a QuerySet is itself a QuerySet, so it’s possible to chain refinements together. For example: 98 Chapter 3. Using Django
- 103. Django Documentation, Release1.5.1 >>> Entry.objects.filter( ... headline__startswith=’What’ ... ).exclude( ... pub_date__gte=datetime.date.today() ... ).filter( ... pub_date__gte=datetime(2005, 1, 30) ... ) This takes the initial QuerySet of all entries in the database, adds a filter, then an exclusion, then another filter. The final result is a QuerySet containing all entries with a headline that starts with “What”, that were published between January 30, 2005, and the current day. Filtered QuerySets are unique Each time you refine a QuerySet, you get a brand-new QuerySet that is in no way bound to the previous QuerySet. Each refinement creates a separate and distinct QuerySet that can be stored, used and reused. Example: >> q1 = Entry.objects.filter(headline__startswith="What") >> q2 = q1.exclude(pub_date__gte=datetime.date.today()) >> q3 = q1.filter(pub_date__gte=datetime.date.today()) These three QuerySets are separate. The first is a base QuerySet containing all entries that contain a headline starting with “What”. The second is a subset of the first, with an additional criteria that excludes records whose pub_date is greater than now. The third is a subset of the first, with an additional criteria that selects only the records whose pub_date is greater than now. The initial QuerySet (q1) is unaffected by the refinement process. QuerySets are lazy QuerySets are lazy – the act of creating a QuerySet doesn’t involve any database activity. You can stack filters together all day long, and Django won’t actually run the query until the QuerySet is evaluated. Take a look at this example: >>> q = Entry.objects.filter(headline__startswith="What") >>> q = q.filter(pub_date__lte=datetime.date.today()) >>> q = q.exclude(body_text__icontains="food") >>> print(q) Though this looks like three database hits, in fact it hits the database only once, at the last line (print(q)). In general, the results of a QuerySet aren’t fetched from the database until you “ask” for them. When you do, the QuerySet is evaluated by accessing the database. For more details on exactly when evaluation takes place, see When QuerySets are evaluated. Retrieving a single object with get filter() will always give you a QuerySet, even if only a single object matches the query - in this case, it will be a QuerySet containing a single element. If you know there is only one object that matches your query, you can use the get() method on a Manager which returns the object directly: >>> one_entry = Entry.objects.get(pk=1) You can use any query expression with get(), just like with filter() - again, see Field lookups below. Note that there is a difference between using get(), and using filter() with a slice of [0]. If there are no results that match the query, get() will raise a DoesNotExist exception. This exception is an attribute of the model 3.2. Models and databases 99
- 104. Django Documentation, Release1.5.1 class that the query is being performed on - so in the code above, if there is no Entry object with a primary key of 1, Django will raise Entry.DoesNotExist. Similarly, Django will complain if more than one item matches the get() query. In this case, it will raise MultipleObjectsReturned, which again is an attribute of the model class itself. Other QuerySet methods Most of the time you’ll use all(), get(), filter() and exclude() when you need to look up objects from the database. However, that’s far from all there is; see the QuerySet API Reference for a complete list of all the various QuerySet methods. Limiting QuerySets Use a subset of Python’s array-slicing syntax to limit your QuerySet to a certain number of results. This is the equivalent of SQL’s LIMIT and OFFSET clauses. For example, this returns the first 5 objects (LIMIT 5): >>> Entry.objects.all()[:5] This returns the sixth through tenth objects (OFFSET 5 LIMIT 5): >>> Entry.objects.all()[5:10] Negative indexing (i.e. Entry.objects.all()[-1]) is not supported. Generally, slicing a QuerySet returns a new QuerySet – it doesn’t evaluate the query. An exception is if you use the “step” parameter of Python slice syntax. For example, this would actually execute the query in order to return a list of every second object of the first 10: >>> Entry.objects.all()[:10:2] To retrieve a single object rather than a list (e.g. SELECT foo FROM bar LIMIT 1), use a simple index instead of a slice. For example, this returns the first Entry in the database, after ordering entries alphabetically by headline: >>> Entry.objects.order_by(’headline’)[0] This is roughly equivalent to: >>> Entry.objects.order_by(’headline’)[0:1].get() Note, however, that the first of these will raise IndexError while the second will raise DoesNotExist if no objects match the given criteria. See get() for more details. Field lookups Field lookups are how you specify the meat of an SQL WHERE clause. They’re specified as keyword arguments to the QuerySet methods filter(), exclude() and get(). Basic lookups keyword arguments take the form field__lookuptype=value. (That’s a double-underscore). For example: >>> Entry.objects.filter(pub_date__lte=’2006-01-01’) translates (roughly) into the following SQL: 100 Chapter 3. Using Django
- 105. Django Documentation, Release1.5.1 SELECT * FROM blog_entry WHERE pub_date <= ’2006-01-01’; How this is possible Python has the ability to define functions that accept arbitrary name-value arguments whose names and values are evaluated at runtime. For more information, see Keyword Arguments in the official Python tutorial. Changed in version 1.4: The field specified in a lookup has to be the name of a model field. There’s one exception though, in case of a ForeignKey you can specify the field name suffixed with _id. In this case, the value parameter is expected to contain the raw value of the foreign model’s primary key. For example: >>> Entry.objects.filter(blog_id__exact=4) If you pass an invalid keyword argument, a lookup function will raise TypeError. The database API supports about two dozen lookup types; a complete reference can be found in the field lookup reference. To give you a taste of what’s available, here’s some of the more common lookups you’ll probably use: exact An “exact” match. For example: >>> Entry.objects.get(headline__exact="Man bites dog") Would generate SQL along these lines: SELECT ... WHERE headline = ’Man bites dog’; If you don’t provide a lookup type – that is, if your keyword argument doesn’t contain a double underscore – the lookup type is assumed to be exact. For example, the following two statements are equivalent: >>> Blog.objects.get(id__exact=14) # Explicit form >>> Blog.objects.get(id=14) # __exact is implied This is for convenience, because exact lookups are the common case. iexact A case-insensitive match. So, the query: >>> Blog.objects.get(name__iexact="beatles blog") Would match a Blog titled “Beatles Blog”, “beatles blog”, or even “BeAtlES blOG”. contains Case-sensitive containment test. For example: Entry.objects.get(headline__contains=’Lennon’) Roughly translates to this SQL: SELECT ... WHERE headline LIKE ’%Lennon%’; Note this will match the headline ’Today Lennon honored’ but not ’today lennon honored’. There’s also a case-insensitive version, icontains. startswith, endswith Starts-with and ends-with search, respectively. There are also case-insensitive versions called istartswith and iendswith. Again, this only scratches the surface. A complete reference can be found in the field lookup reference. 3.2. Models and databases 101
- 106. Django Documentation, Release1.5.1 Lookups that span relationships Django offers a powerful and intuitive way to “follow” relationships in lookups, taking care of the SQL JOINs for you automatically, behind the scenes. To span a relationship, just use the field name of related fields across models, separated by double underscores, until you get to the field you want. This example retrieves all Entry objects with a Blog whose name is ’Beatles Blog’: >>> Entry.objects.filter(blog__name__exact=’Beatles Blog’) This spanning can be as deep as you’d like. It works backwards, too. To refer to a “reverse” relationship, just use the lowercase name of the model. This example retrieves all Blog objects which have at least one Entry whose headline contains ’Lennon’: >>> Blog.objects.filter(entry__headline__contains=’Lennon’) If you are filtering across multiple relationships and one of the intermediate models doesn’t have a value that meets the filter condition, Django will treat it as if there is an empty (all values are NULL), but valid, object there. All this means is that no error will be raised. For example, in this filter: Blog.objects.filter(entry__authors__name=’Lennon’) (if there was a related Author model), if there was no author associated with an entry, it would be treated as if there was also no name attached, rather than raising an error because of the missing author. Usually this is exactly what you want to have happen. The only case where it might be confusing is if you are using isnull. Thus: Blog.objects.filter(entry__authors__name__isnull=True) will return Blog objects that have an empty name on the author and also those which have an empty author on the entry. If you don’t want those latter objects, you could write: Blog.objects.filter(entry__authors__isnull=False, entry__authors__name__isnull=True) Spanning multi-valued relationships When you are filtering an object based on a ManyToManyField or a re-verse ForeignKey, there are two different sorts of filter you may be interested in. Consider the Blog/Entry relationship (Blog to Entry is a one-to-many relation). We might be interested in finding blogs that have an entry which has both “Lennon” in the headline and was published in 2008. Or we might want to find blogs that have an entry with “Lennon” in the headline as well as an entry that was published in 2008. Since there are multiple entries associated with a single Blog, both of these queries are possible and make sense in some situations. The same type of situation arises with a ManyToManyField. For example, if an Entry has a ManyToManyField called tags, we might want to find entries linked to tags called “music” and “bands” or we might want an entry that contains a tag with a name of “music” and a status of “public”. To handle both of these situations, Django has a consistent way of processing filter() and exclude() calls. Ev-erything inside a single filter() call is applied simultaneously to filter out items matching all those requirements. Successive filter() calls further restrict the set of objects, but for multi-valued relations, they apply to any object linked to the primary model, not necessarily those objects that were selected by an earlier filter() call. That may sound a bit confusing, so hopefully an example will clarify. To select all blogs that contain entries with both “Lennon” in the headline and that were published in 2008 (the same entry satisfying both conditions), we would write: Blog.objects.filter(entry__headline__contains=’Lennon’, entry__pub_date__year=2008) 102 Chapter 3. Using Django
- 107. Django Documentation, Release1.5.1 To select all blogs that contain an entry with “Lennon” in the headline as well as an entry that was published in 2008, we would write: Blog.objects.filter(entry__headline__contains=’Lennon’).filter( entry__pub_date__year=2008) Suppose there is only one blog that had both entries containing “Lennon” and entries from 2008, but that none of the entries from 2008 contained “Lennon”. The first query would not return any blogs, but the second query would return that one blog. In the second example, the first filter restricts the queryset to all those blogs linked to entries with “Lennon” in the headline. The second filter restricts the set of blogs further to those that are also linked to entries that were published in 2008. The entries selected by the second filter may or may not be the same as the entries in the first filter. We are filtering the Blog items with each filter statement, not the Entry items. All of this behavior also applies to exclude(): all the conditions in a single exclude() statement apply to a single instance (if those conditions are talking about the same multi-valued relation). Conditions in subsequent filter() or exclude() calls that refer to the same relation may end up filtering on different linked objects. Filters can reference fields on the model class F In the examples given so far, we have constructed filters that compare the value of a model field with a constant. But what if you want to compare the value of a model field with another field on the same model? Django provides the F() expressions to allow such comparisons. Instances of F() act as a reference to a model field within a query. These references can then be used in query filters to compare the values of two different fields on the same model instance. For example, to find a list of all blog entries that have had more comments than pingbacks, we construct an F() object to reference the pingback count, and use that F() object in the query: >>> from django.db.models import F >>> Entry.objects.filter(n_comments__gt=F(’n_pingbacks’)) Django supports the use of addition, subtraction, multiplication, division and modulo arithmetic with F() objects, both with constants and with other F() objects. To find all the blog entries with more than twice as many comments as pingbacks, we modify the query: >>> Entry.objects.filter(n_comments__gt=F(’n_pingbacks’) * 2) To find all the entries where the rating of the entry is less than the sum of the pingback count and comment count, we would issue the query: >>> Entry.objects.filter(rating__lt=F(’n_comments’) + F(’n_pingbacks’)) You can also use the double underscore notation to span relationships in an F() object. An F() object with a double underscore will introduce any joins needed to access the related object. For example, to retrieve all the entries where the author’s name is the same as the blog name, we could issue the query: >>> Entry.objects.filter(authors__name=F(’blog__name’)) For date and date/time fields, you can add or subtract a timedelta object. The following would return all entries that were modified more than 3 days after they were published: >>> from datetime import timedelta >>> Entry.objects.filter(mod_date__gt=F(’pub_date’) + timedelta(days=3)) 3.2. Models and databases 103
- 108. Django Documentation, Release1.5.1 New in version 1.5: .bitand() and .bitor() The F() objects now support bitwise operations by .bitand() and .bitor(), for example: >>> F(’somefield’).bitand(16) Changed in version 1.5: The previously undocumented operators & and | no longer produce bitwise operations, use .bitand() and .bitor() instead. The pk lookup shortcut For convenience, Django provides a pk lookup shortcut, which stands for “primary key”. In the example Blog model, the primary key is the id field, so these three statements are equivalent: >>> Blog.objects.get(id__exact=14) # Explicit form >>> Blog.objects.get(id=14) # __exact is implied >>> Blog.objects.get(pk=14) # pk implies id__exact The use of pk isn’t limited to __exact queries – any query term can be combined with pk to perform a query on the primary key of a model: # Get blogs entries with id 1, 4 and 7 >>> Blog.objects.filter(pk__in=[1,4,7]) # Get all blog entries with id > 14 >>> Blog.objects.filter(pk__gt=14) pk lookups also work across joins. For example, these three statements are equivalent: >>> Entry.objects.filter(blog__id__exact=3) # Explicit form >>> Entry.objects.filter(blog__id=3) # __exact is implied >>> Entry.objects.filter(blog__pk=3) # __pk implies __id__exact Escaping percent signs and underscores in LIKE statements The field lookups that equate to LIKE SQL statements (iexact, contains, icontains, startswith, istartswith, endswith and iendswith) will automatically escape the two special characters used in LIKE statements – the percent sign and the underscore. (In a LIKE statement, the percent sign signifies a multiple-character wildcard and the underscore signifies a single-character wildcard.) This means things should work intuitively, so the abstraction doesn’t leak. For example, to retrieve all the entries that contain a percent sign, just use the percent sign as any other character: >>> Entry.objects.filter(headline__contains=’%’) Django takes care of the quoting for you; the resulting SQL will look something like this: SELECT ... WHERE headline LIKE ’%%%’; Same goes for underscores. Both percentage signs and underscores are handled for you transparently. Caching and QuerySets Each QuerySet contains a cache, to minimize database access. It’s important to understand how it works, in order to write the most efficient code. 104 Chapter 3. Using Django
- 109. Django Documentation, Release1.5.1 In a newly created QuerySet, the cache is empty. The first time a QuerySet is evaluated – and, hence, a database query happens – Django saves the query results in the QuerySet‘s cache and returns the results that have been explicitly requested (e.g., the next element, if the QuerySet is being iterated over). Subsequent evaluations of the QuerySet reuse the cached results. Keep this caching behavior in mind, because it may bite you if you don’t use your QuerySets correctly. For example, the following will create two QuerySets, evaluate them, and throw them away: >>> print([e.headline for e in Entry.objects.all()]) >>> print([e.pub_date for e in Entry.objects.all()]) That means the same database query will be executed twice, effectively doubling your database load. Also, there’s a possibility the two lists may not include the same database records, because an Entry may have been added or deleted in the split second between the two requests. To avoid this problem, simply save the QuerySet and reuse it: >>> queryset = Entry.objects.all() >>> print([p.headline for p in queryset]) # Evaluate the query set. >>> print([p.pub_date for p in queryset]) # Re-use the cache from the evaluation. Complex lookups with Q objects class Q Keyword argument queries – in filter(), etc. – are “AND”ed together. If you need to execute more complex queries (for example, queries with OR statements), you can use Q objects. A Q object (django.db.models.Q) is an object used to encapsulate a collection of keyword arguments. These keyword arguments are specified as in “Field lookups” above. For example, this Q object encapsulates a single LIKE query: from django.db.models import Q Q(question__startswith=’What’) Q objects can be combined using the & and | operators. When an operator is used on two Q objects, it yields a new Q object. For example, this statement yields a single Q object that represents the “OR” of two "question__startswith" queries: Q(question__startswith=’Who’) | Q(question__startswith=’What’) This is equivalent to the following SQL WHERE clause: WHERE question LIKE ’Who%’ OR question LIKE ’What%’ You can compose statements of arbitrary complexity by combining Q objects with the & and | operators and use parenthetical grouping. Also, Q objects can be negated using the ~ operator, allowing for combined lookups that combine both a normal query and a negated (NOT) query: Q(question__startswith=’Who’) | ~Q(pub_date__year=2005) Each lookup function that takes keyword-arguments (e.g. filter(), exclude(), get()) can also be passed one or more Q objects as positional (not-named) arguments. If you provide multiple Q object arguments to a lookup function, the arguments will be “AND”ed together. For example: 3.2. Models and databases 105
- 110. Django Documentation, Release1.5.1 Poll.objects.get( Q(question__startswith=’Who’), Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)) ) ... roughly translates into the SQL: SELECT * from polls WHERE question LIKE ’Who%’ AND (pub_date = ’2005-05-02’ OR pub_date = ’2005-05-06’) Lookup functions can mix the use of Q objects and keyword arguments. All arguments provided to a lookup function (be they keyword arguments or Q objects) are “AND”ed together. However, if a Q object is provided, it must precede the definition of any keyword arguments. For example: Poll.objects.get( Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)), question__startswith=’Who’) ... would be a valid query, equivalent to the previous example; but: # INVALID QUERY Poll.objects.get( question__startswith=’Who’, Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6))) ... would not be valid. See Also: The OR lookups examples in the Django unit tests show some possible uses of Q. Comparing objects To compare two model instances, just use the standard Python comparison operator, the double equals sign: ==. Behind the scenes, that compares the primary key values of two models. Using the Entry example above, the following two statements are equivalent: >>> some_entry == other_entry >>> some_entry.id == other_entry.id If a model’s primary key isn’t called id, no problem. Comparisons will always use the primary key, whatever it’s called. For example, if a model’s primary key field is called name, these two statements are equivalent: >>> some_obj == other_obj >>> some_obj.name == other_obj.name Deleting objects The delete method, conveniently, is named delete(). This method immediately deletes the object and has no return value. Example: e.delete() You can also delete objects in bulk. Every QuerySet has a delete() method, which deletes all members of that QuerySet. For example, this deletes all Entry objects with a pub_date year of 2005: 106 Chapter 3. Using Django
- 111. Django Documentation, Release1.5.1 Entry.objects.filter(pub_date__year=2005).delete() Keep in mind that this will, whenever possible, be executed purely in SQL, and so the delete() methods of in-dividual object instances will not necessarily be called during the process. If you’ve provided a custom delete() method on a model class and want to ensure that it is called, you will need to “manually” delete instances of that model (e.g., by iterating over a QuerySet and calling delete() on each object individually) rather than using the bulk delete() method of a QuerySet. When Django deletes an object, by default it emulates the behavior of the SQL constraint ON DELETE CASCADE – in other words, any objects which had foreign keys pointing at the object to be deleted will be deleted along with it. For example: b = Blog.objects.get(pk=1) # This will delete the Blog and all of its Entry objects. b.delete() This cascade behavior is customizable via the on_delete argument to the ForeignKey. Note that delete() is the only QuerySet method that is not exposed on a Manager itself. This is a safety mechanism to prevent you from accidentally requesting Entry.objects.delete(), and deleting all the entries. If you do want to delete all the objects, then you have to explicitly request a complete query set: Entry.objects.all().delete() Copying model instances Although there is no built-in method for copying model instances, it is possible to easily create new instance with all fields’ values copied. In the simplest case, you can just set pk to None. Using our blog example: blog = Blog(name=’My blog’, tagline=’Blogging is easy’) blog.save() # blog.pk == 1 blog.pk = None blog.save() # blog.pk == 2 Things get more complicated if you use inheritance. Consider a subclass of Blog: class ThemeBlog(Blog): theme = models.CharField(max_length=200) django_blog = ThemeBlog(name=’Django’, tagline=’Django is easy’, theme=’python’) django_blog.save() # django_blog.pk == 3 Due to how inheritance works, you have to set both pk and id to None: django_blog.pk = None django_blog.id = None django_blog.save() # django_blog.pk == 4 This process does not copy related objects. If you want to copy relations, you have to write a little bit more code. In our example, Entry has a many to many field to Author: entry = Entry.objects.all()[0] # some previous entry old_authors = entry.authors.all() entry.pk = None entry.save() entry.authors = old_authors # saves new many2many relations 3.2. Models and databases 107
- 112. Django Documentation, Release1.5.1 Updating multiple objects at once Sometimes you want to set a field to a particular value for all the objects in a QuerySet. You can do this with the update() method. For example: # Update all the headlines with pub_date in 2007. Entry.objects.filter(pub_date__year=2007).update(headline=’Everything is the same’) You can only set non-relation fields and ForeignKey fields using this method. To update a non-relation field, provide the new value as a constant. To update ForeignKey fields, set the new value to be the new model instance you want to point to. For example: >>> b = Blog.objects.get(pk=1) # Change every Entry so that it belongs to this Blog. >>> Entry.objects.all().update(blog=b) The update() method is applied instantly and returns the number of rows matched by the query (which may not be equal to the number of rows updated if some rows already have the new value). The only restriction on the QuerySet that is updated is that it can only access one database table, the model’s main table. You can filter based on related fields, but you can only update columns in the model’s main table. Example: >>> b = Blog.objects.get(pk=1) # Update all the headlines belonging to this Blog. >>> Entry.objects.select_related().filter(blog=b).update(headline=’Everything is the same’) Be aware that the update() method is converted directly to an SQL statement. It is a bulk operation for direct updates. It doesn’t run any save() methods on your models, or emit the pre_save or post_save signals (which are a consequence of calling save()), or honor the auto_now field option. If you want to save every item in a QuerySet and make sure that the save() method is called on each instance, you don’t need any special function to handle that. Just loop over them and call save(): for item in my_queryset: item.save() Calls to update can also use F() objects to update one field based on the value of another field in the model. This is especially useful for incrementing counters based upon their current value. For example, to increment the pingback count for every entry in the blog: >>> Entry.objects.all().update(n_pingbacks=F(’n_pingbacks’) + 1) However, unlike F() objects in filter and exclude clauses, you can’t introduce joins when you use F() objects in an update – you can only reference fields local to the model being updated. If you attempt to introduce a join with an F() object, a FieldError will be raised: # THIS WILL RAISE A FieldError >>> Entry.objects.update(headline=F(’blog__name’)) Related objects When you define a relationship in a model (i.e., a ForeignKey, OneToOneField, or ManyToManyField), instances of that model will have a convenient API to access the related object(s). Using the models at the top of this page, for example, an Entry object e can get its associated Blog object by accessing the blog attribute: e.blog. 108 Chapter 3. Using Django
- 113. Django Documentation, Release1.5.1 (Behind the scenes, this functionality is implemented by Python descriptors. This shouldn’t really matter to you, but we point it out here for the curious.) Django also creates API accessors for the “other” side of the relationship – the link from the related model to the model that defines the relationship. For example, a Blog object b has access to a list of all related Entry objects via the entry_set attribute: b.entry_set.all(). All examples in this section use the sample Blog, Author and Entry models defined at the top of this page. One-to-many relationships Forward If a model has a ForeignKey, instances of that model will have access to the related (foreign) object via a simple attribute of the model. Example: >>> e = Entry.objects.get(id=2) >>> e.blog # Returns the related Blog object. You can get and set via a foreign-key attribute. As you may expect, changes to the foreign key aren’t saved to the database until you call save(). Example: >>> e = Entry.objects.get(id=2) >>> e.blog = some_blog >>> e.save() If a ForeignKey field has null=True set (i.e., it allows NULL values), you can assign None to it. Example: >>> e = Entry.objects.get(id=2) >>> e.blog = None >>> e.save() # "UPDATE blog_entry SET blog_id = NULL ...;" Forward access to one-to-many relationships is cached the first time the related object is accessed. Subsequent accesses to the foreign key on the same object instance are cached. Example: >>> e = Entry.objects.get(id=2) >>> print(e.blog) # Hits the database to retrieve the associated Blog. >>> print(e.blog) # Doesn’t hit the database; uses cached version. Note that the select_related() QuerySet method recursively prepopulates the cache of all one-to-many rela-tionships ahead of time. Example: >>> e = Entry.objects.select_related().get(id=2) >>> print(e.blog) # Doesn’t hit the database; uses cached version. >>> print(e.blog) # Doesn’t hit the database; uses cached version. Following relationships “backward” If a model has a ForeignKey, instances of the foreign-key model will have access to a Manager that returns all instances of the first model. By default, this Manager is named FOO_set, where FOO is the source model name, lowercased. This Manager returns QuerySets, which can be filtered and manipulated as described in the “Retrieving objects” section above. Example: >>> b = Blog.objects.get(id=1) >>> b.entry_set.all() # Returns all Entry objects related to Blog. # b.entry_set is a Manager that returns QuerySets. >>> b.entry_set.filter(headline__contains=’Lennon’) >>> b.entry_set.count() 3.2. Models and databases 109
- 114. Django Documentation, Release1.5.1 You can override the FOO_set name by setting the related_name parameter in the ForeignKey() definition. For example, if the Entry model was altered to blog = ForeignKey(Blog, related_name=’entries’), the above example code would look like this: >>> b = Blog.objects.get(id=1) >>> b.entries.all() # Returns all Entry objects related to Blog. # b.entries is a Manager that returns QuerySets. >>> b.entries.filter(headline__contains=’Lennon’) >>> b.entries.count() You cannot access a reverse ForeignKey Manager from the class; it must be accessed from an instance: >>> Blog.entry_set Traceback: ... AttributeError: "Manager must be accessed via instance". In addition to the QuerySet methods defined in “Retrieving objects” above, the ForeignKey Manager has ad-ditional methods used to handle the set of related objects. A synopsis of each is below, and complete details can be found in the related objects reference. add(obj1, obj2, ...) Adds the specified model objects to the related object set. create(**kwargs) Creates a new object, saves it and puts it in the related object set. Returns the newly created object. remove(obj1, obj2, ...) Removes the specified model objects from the related object set. clear() Removes all objects from the related object set. To assign the members of a related set in one fell swoop, just assign to it from any iterable object. The iterable can contain object instances, or just a list of primary key values. For example: b = Blog.objects.get(id=1) b.entry_set = [e1, e2] In this example, e1 and e2 can be full Entry instances, or integer primary key values. If the clear() method is available, any pre-existing objects will be removed from the entry_set before all objects in the iterable (in this case, a list) are added to the set. If the clear() method is not available, all objects in the iterable will be added without removing any existing elements. Each “reverse” operation described in this section has an immediate effect on the database. Every addition, creation and deletion is immediately and automatically saved to the database. Many-to-many relationships Both ends of a many-to-many relationship get automatic API access to the other end. The API works just as a “backward” one-to-many relationship, above. The only difference is in the attribute naming: The model that defines the ManyToManyField uses the attribute name of that field itself, whereas the “reverse” model uses the lowercased model name of the original model, plus ’_set’ (just like reverse one-to-many relationships). An example makes this easier to understand: e = Entry.objects.get(id=3) e.authors.all() # Returns all Author objects for this Entry. e.authors.count() e.authors.filter(name__contains=’John’) 110 Chapter 3. Using Django
- 115. Django Documentation, Release1.5.1 a = Author.objects.get(id=5) a.entry_set.all() # Returns all Entry objects for this Author. Like ForeignKey, ManyToManyField can specify related_name. In the above example, if the ManyToManyField in Entry had specified related_name=’entries’, then each Author instance would have an entries attribute instead of entry_set. One-to-one relationships One-to-one relationships are very similar to many-to-one relationships. If you define a OneToOneField on your model, instances of that model will have access to the related object via a simple attribute of the model. For example: class EntryDetail(models.Model): entry = models.OneToOneField(Entry) details = models.TextField() ed = EntryDetail.objects.get(id=2) ed.entry # Returns the related Entry object. The difference comes in “reverse” queries. The related model in a one-to-one relationship also has access to a Manager object, but that Manager represents a single object, rather than a collection of objects: e = Entry.objects.get(id=2) e.entrydetail # returns the related EntryDetail object If no object has been assigned to this relationship, Django will raise a DoesNotExist exception. Instances can be assigned to the reverse relationship in the same way as you would assign the forward relationship: e.entrydetail = ed How are the backward relationships possible? Other object-relational mappers require you to define relationships on both sides. The Django developers believe this is a violation of the DRY (Don’t Repeat Yourself) principle, so Django only requires you to define the relationship on one end. But how is this possible, given that a model class doesn’t know which other model classes are related to it until those other model classes are loaded? The answer lies in the INSTALLED_APPS setting. The first time any model is loaded, Django iterates over every model in INSTALLED_APPS and creates the backward relationships in memory as needed. Essentially, one of the functions of INSTALLED_APPS is to tell Django the entire model domain. Queries over related objects Queries involving related objects follow the same rules as queries involving normal value fields. When specifying the value for a query to match, you may use either an object instance itself, or the primary key value for the object. For example, if you have a Blog object b with id=5, the following three queries would be identical: Entry.objects.filter(blog=b) # Query using object instance Entry.objects.filter(blog=b.id) # Query using id from instance Entry.objects.filter(blog=5) # Query using id directly 3.2. Models and databases 111
- 116. Django Documentation, Release1.5.1 Falling back to raw SQL If you find yourself needing to write an SQL query that is too complex for Django’s database-mapper to handle, you can fall back on writing SQL by hand. Django has a couple of options for writing raw SQL queries; see Performing raw SQL queries. Finally, it’s important to note that the Django database layer is merely an interface to your database. You can access your database via other tools, programming languages or database frameworks; there’s nothing Django-specific about your database. 3.2.3 Aggregation The topic guide on Django’s database-abstraction API described the way that you can use Django queries that create, retrieve, update and delete individual objects. However, sometimes you will need to retrieve values that are derived by summarizing or aggregating a collection of objects. This topic guide describes the ways that aggregate values can be generated and returned using Django queries. Throughout this guide, we’ll refer to the following models. These models are used to track the inventory for a series of online bookstores: class Author(models.Model): name = models.CharField(max_length=100) age = models.IntegerField() class Publisher(models.Model): name = models.CharField(max_length=300) num_awards = models.IntegerField() class Book(models.Model): name = models.CharField(max_length=300) pages = models.IntegerField() price = models.DecimalField(max_digits=10, decimal_places=2) rating = models.FloatField() authors = models.ManyToManyField(Author) publisher = models.ForeignKey(Publisher) pubdate = models.DateField() class Store(models.Model): name = models.CharField(max_length=300) books = models.ManyToManyField(Book) registered_users = models.PositiveIntegerField() Cheat sheet In a hurry? Here’s how to do common aggregate queries, assuming the models above: # Total number of books. >>> Book.objects.count() 2452 # Total number of books with publisher=BaloneyPress >>> Book.objects.filter(publisher__name=’BaloneyPress’).count() 73 # Average price across all books. >>> from django.db.models import Avg 112 Chapter 3. Using Django
- 117. Django Documentation, Release1.5.1 >>> Book.objects.all().aggregate(Avg(’price’)) {’price__avg’: 34.35} # Max price across all books. >>> from django.db.models import Max >>> Book.objects.all().aggregate(Max(’price’)) {’price__max’: Decimal(’81.20’)} # All the following queries involve traversing the Book<->Publisher # many-to-many relationship backward # Each publisher, each with a count of books as a "num_books" attribute. >>> from django.db.models import Count >>> pubs = Publisher.objects.annotate(num_books=Count(’book’)) >>> pubs [<Publisher BaloneyPress>, <Publisher SalamiPress>, ...] >>> pubs[0].num_books 73 # The top 5 publishers, in order by number of books. >>> pubs = Publisher.objects.annotate(num_books=Count(’book’)).order_by(’-num_books’)[:5] >>> pubs[0].num_books 1323 Generating aggregates over a QuerySet Django provides two ways to generate aggregates. The first way is to generate summary values over an entire QuerySet. For example, say you wanted to calculate the average price of all books available for sale. Django’s query syntax provides a means for describing the set of all books: >>> Book.objects.all() What we need is a way to calculate summary values over the objects that belong to this QuerySet. This is done by appending an aggregate() clause onto the QuerySet: >>> from django.db.models import Avg >>> Book.objects.all().aggregate(Avg(’price’)) {’price__avg’: 34.35} The all() is redundant in this example, so this could be simplified to: >>> Book.objects.aggregate(Avg(’price’)) {’price__avg’: 34.35} The argument to the aggregate() clause describes the aggregate value that we want to compute - in this case, the average of the price field on the Book model. A list of the aggregate functions that are available can be found in the QuerySet reference. aggregate() is a terminal clause for a QuerySet that, when invoked, returns a dictionary of name-value pairs. The name is an identifier for the aggregate value; the value is the computed aggregate. The name is automatically generated from the name of the field and the aggregate function. If you want to manually specify a name for the aggregate value, you can do so by providing that name when you specify the aggregate clause: >>> Book.objects.aggregate(average_price=Avg(’price’)) {’average_price’: 34.35} If you want to generate more than one aggregate, you just add another argument to the aggregate() clause. So, if we also wanted to know the maximum and minimum price of all books, we would issue the query: 3.2. Models and databases 113
- 118. Django Documentation, Release1.5.1 >>> from django.db.models import Avg, Max, Min, Count >>> Book.objects.aggregate(Avg(’price’), Max(’price’), Min(’price’)) {’price__avg’: 34.35, ’price__max’: Decimal(’81.20’), ’price__min’: Decimal(’12.99’)} Generating aggregates for each item in a QuerySet The second way to generate summary values is to generate an independent summary for each object in a QuerySet. For example, if you are retrieving a list of books, you may want to know how many authors contributed to each book. Each Book has a many-to-many relationship with the Author; we want to summarize this relationship for each book in the QuerySet. Per-object summaries can be generated using the annotate() clause. When an annotate() clause is specified, each object in the QuerySet will be annotated with the specified values. The syntax for these annotations is identical to that used for the aggregate() clause. Each argument to annotate() describes an aggregate that is to be calculated. For example, to annotate books with the number of authors: # Build an annotated queryset >>> q = Book.objects.annotate(Count(’authors’)) # Interrogate the first object in the queryset >>> q[0] <Book: The Definitive Guide to Django> >>> q[0].authors__count 2 # Interrogate the second object in the queryset >>> q[1] <Book: Practical Django Projects> >>> q[1].authors__count 1 As with aggregate(), the name for the annotation is automatically derived from the name of the aggregate function and the name of the field being aggregated. You can override this default name by providing an alias when you specify the annotation: >>> q = Book.objects.annotate(num_authors=Count(’authors’)) >>> q[0].num_authors 2 >>> q[1].num_authors 1 Unlike aggregate(), annotate() is not a terminal clause. The output of the annotate() clause is a QuerySet; this QuerySet can be modified using any other QuerySet operation, including filter(), order_by(), or even additional calls to annotate(). Joins and aggregates So far, we have dealt with aggregates over fields that belong to the model being queried. However, sometimes the value you want to aggregate will belong to a model that is related to the model you are querying. When specifying the field to be aggregated in an aggregate function, Django will allow you to use the same double underscore notation that is used when referring to related fields in filters. Django will then handle any table joins that are required to retrieve and aggregate the related value. For example, to find the price range of books offered in each store, you could use the annotation: 114 Chapter 3. Using Django
- 119. Django Documentation, Release1.5.1 >>> Store.objects.annotate(min_price=Min(’books__price’), max_price=Max(’books__price’)) This tells Django to retrieve the Store model, join (through the many-to-many relationship) with the Book model, and aggregate on the price field of the book model to produce a minimum and maximum value. The same rules apply to the aggregate() clause. If you wanted to know the lowest and highest price of any book that is available for sale in a store, you could use the aggregate: >>> Store.objects.aggregate(min_price=Min(’books__price’), max_price=Max(’books__price’)) Join chains can be as deep as you require. For example, to extract the age of the youngest author of any book available for sale, you could issue the query: >>> Store.objects.aggregate(youngest_age=Min(’books__authors__age’)) Following relationships backwards In a way similar to Lookups that span relationships, aggregations and annotations on fields of models or models that are related to the one you are querying can include traversing “reverse” relationships. The lowercase name of related models and double-underscores are used here too. For example, we can ask for all publishers, annotated with their respective total book stock counters (note how we use ’book’ to specify the Publisher -> Book reverse foreign key hop): >>> from django.db.models import Count, Min, Sum, Max, Avg >>> Publisher.objects.annotate(Count(’book’)) (Every Publisher in the resulting QuerySet will have an extra attribute called book__count.) We can also ask for the oldest book of any of those managed by every publisher: >>> Publisher.objects.aggregate(oldest_pubdate=Min(’book__pubdate’)) (The resulting dictionary will have a key called ’oldest_pubdate’. If no such alias were specified, it would be the rather long ’book__pubdate__min’.) This doesn’t apply just to foreign keys. It also works with many-to-many relations. For example, we can ask for every author, annotated with the total number of pages considering all the books he/she has (co-)authored (note how we use ’book’ to specify the Author -> Book reverse many-to-many hop): >>> Author.objects.annotate(total_pages=Sum(’book__pages’)) (Every Author in the resulting QuerySet will have an extra attribute called total_pages. If no such alias were specified, it would be the rather long book__pages__sum.) Or ask for the average rating of all the books written by author(s) we have on file: >>> Author.objects.aggregate(average_rating=Avg(’book__rating’)) (The resulting dictionary will have a key called ’average__rating’. If no such alias were specified, it would be the rather long ’book__rating__avg’.) Aggregations and other QuerySet clauses filter() and exclude() Aggregates can also participate in filters. Any filter() (or exclude()) applied to normal model fields will have the effect of constraining the objects that are considered for aggregation. 3.2. Models and databases 115
- 120. Django Documentation, Release1.5.1 When used with an annotate() clause, a filter has the effect of constraining the objects for which an annotation is calculated. For example, you can generate an annotated list of all books that have a title starting with “Django” using the query: >>> Book.objects.filter(name__startswith="Django").annotate(num_authors=Count(’authors’)) When used with an aggregate() clause, a filter has the effect of constraining the objects over which the aggregate is calculated. For example, you can generate the average price of all books with a title that starts with “Django” using the query: >>> Book.objects.filter(name__startswith="Django").aggregate(Avg(’price’)) Filtering on annotations Annotated values can also be filtered. The alias for the annotation can be used in filter() and exclude() clauses in the same way as any other model field. For example, to generate a list of books that have more than one author, you can issue the query: >>> Book.objects.annotate(num_authors=Count(’authors’)).filter(num_authors__gt=1) This query generates an annotated result set, and then generates a filter based upon that annotation. Order of annotate() and filter() clauses When developing a complex query that involves both annotate() and filter() clauses, particular attention should be paid to the order in which the clauses are applied to the QuerySet. When an annotate() clause is applied to a query, the annotation is computed over the state of the query up to the point where the annotation is requested. The practical implication of this is that filter() and annotate() are not commutative operations – that is, there is a difference between the query: >>> Publisher.objects.annotate(num_books=Count(’book’)).filter(book__rating__gt=3.0) and the query: >>> Publisher.objects.filter(book__rating__gt=3.0).annotate(num_books=Count(’book’)) Both queries will return a list of publishers that have at least one good book (i.e., a book with a rating exceeding 3.0). However, the annotation in the first query will provide the total number of all books published by the publisher; the second query will only include good books in the annotated count. In the first query, the annotation precedes the filter, so the filter has no effect on the annotation. In the second query, the filter precedes the annotation, and as a result, the filter constrains the objects considered when calculating the annotation. order_by() Annotations can be used as a basis for ordering. When you define an order_by() clause, the aggregates you provide can reference any alias defined as part of an annotate() clause in the query. For example, to order a QuerySet of books by the number of authors that have contributed to the book, you could use the following query: >>> Book.objects.annotate(num_authors=Count(’authors’)).order_by(’num_authors’) values() Ordinarily, annotations are generated on a per-object basis - an annotated QuerySet will return one result for each object in the original QuerySet. However, when a values() clause is used to constrain the columns that are 116 Chapter 3. Using Django
- 121. Django Documentation, Release1.5.1 returned in the result set, the method for evaluating annotations is slightly different. Instead of returning an annotated result for each result in the original QuerySet, the original results are grouped according to the unique combinations of the fields specified in the values() clause. An annotation is then provided for each unique group; the annotation is computed over all members of the group. For example, consider an author query that attempts to find out the average rating of books written by each author: >>> Author.objects.annotate(average_rating=Avg(’book__rating’)) This will return one result for each author in the database, annotated with their average book rating. However, the result will be slightly different if you use a values() clause: >>> Author.objects.values(’name’).annotate(average_rating=Avg(’book__rating’)) In this example, the authors will be grouped by name, so you will only get an annotated result for each unique author name. This means if you have two authors with the same name, their results will be merged into a single result in the output of the query; the average will be computed as the average over the books written by both authors. Order of annotate() and values() clauses As with the filter() clause, the order in which annotate() and values() clauses are applied to a query is significant. If the values() clause precedes the annotate(), the annotation will be computed using the grouping described by the values() clause. However, if the annotate() clause precedes the values() clause, the annotations will be generated over the entire query set. In this case, the values() clause only constrains the fields that are generated on output. For example, if we reverse the order of the values() and annotate() clause from our previous example: >>> Author.objects.annotate(average_rating=Avg(’book__rating’)).values(’name’, ’average_rating’) This will now yield one unique result for each author; however, only the author’s name and the average_rating annotation will be returned in the output data. You should also note that average_rating has been explicitly included in the list of values to be returned. This is required because of the ordering of the values() and annotate() clause. If the values() clause precedes the annotate() clause, any annotations will be automatically added to the result set. However, if the values() clause is applied after the annotate() clause, you need to explicitly include the aggregate column. Interaction with default ordering or order_by() Fields that are mentioned in the order_by() part of a queryset (or which are used in the default ordering on a model) are used when selecting the output data, even if they are not otherwise specified in the values() call. These extra fields are used to group “like” results together and they can make otherwise identical result rows appear to be separate. This shows up, particularly, when counting things. By way of example, suppose you have a model like this: class Item(models.Model): name = models.CharField(max_length=10) data = models.IntegerField() class Meta: ordering = ["name"] The important part here is the default ordering on the name field. If you want to count how many times each distinct data value appears, you might try this: # Warning: not quite correct! Item.objects.values("data").annotate(Count("id")) 3.2. Models and databases 117
- 122. Django Documentation, Release1.5.1 ...which will group the Item objects by their common data values and then count the number of id values in each group. Except that it won’t quite work. The default ordering by name will also play a part in the grouping, so this query will group by distinct (data, name) pairs, which isn’t what you want. Instead, you should construct this queryset: Item.objects.values("data").annotate(Count("id")).order_by() ...clearing any ordering in the query. You could also order by, say, data without any harmful effects, since that is already playing a role in the query. This behavior is the same as that noted in the queryset documentation for distinct() and the general rule is the same: normally you won’t want extra columns playing a part in the result, so clear out the ordering, or at least make sure it’s restricted only to those fields you also select in a values() call. Note: You might reasonably ask why Django doesn’t remove the extraneous columns for you. The main reason is consistency with distinct() and other places: Django never removes ordering constraints that you have specified (and we can’t change those other methods’ behavior, as that would violate our API stability policy). Aggregating annotations You can also generate an aggregate on the result of an annotation. When you define an aggregate() clause, the aggregates you provide can reference any alias defined as part of an annotate() clause in the query. For example, if you wanted to calculate the average number of authors per book you first annotate the set of books with the author count, then aggregate that author count, referencing the annotation field: >>> Book.objects.annotate(num_authors=Count(’authors’)).aggregate(Avg(’num_authors’)) {’num_authors__avg’: 1.66} 3.2.4 Managers class Manager A Manager is the interface through which database query operations are provided to Django models. At least one Manager exists for every model in a Django application. The way Manager classes work is documented in Making queries; this document specifically touches on model options that customize Manager behavior. Manager names By default, Django adds a Manager with the name objects to every Django model class. However, if you want to use objects as a field name, or if you want to use a name other than objects for the Manager, you can rename it on a per-model basis. To rename the Manager for a given class, define a class attribute of type models.Manager() on that model. For example: from django.db import models class Person(models.Model): #... people = models.Manager() Using this example model, Person.objects will generate an AttributeError exception, but Person.people.all() will provide a list of all Person objects. 118 Chapter 3. Using Django
- 123. Django Documentation, Release1.5.1 Custom Managers You can use a custom Manager in a particular model by extending the base Manager class and instantiating your custom Manager in your model. There are two reasons you might want to customize a Manager: to add extra Manager methods, and/or to modify the initial QuerySet the Manager returns. Adding extra Manager methods Adding extra Manager methods is the preferred way to add “table-level” functionality to your models. (For “row-level” functionality – i.e., functions that act on a single instance of a model object – use Model methods, not custom Manager methods.) A custom Manager method can return anything you want. It doesn’t have to return a QuerySet. For example, this custom Manager offers a method with_counts(), which returns a list of all OpinionPoll objects, each with an extra num_responses attribute that is the result of an aggregate query: class PollManager(models.Manager): def with_counts(self): from django.db import connection cursor = connection.cursor() cursor.execute(""" SELECT p.id, p.question, p.poll_date, COUNT(*) FROM polls_opinionpoll p, polls_response r WHERE p.id = r.poll_id GROUP BY p.id, p.question, p.poll_date ORDER BY p.poll_date DESC""") result_list = [] for row in cursor.fetchall(): p = self.model(id=row[0], question=row[1], poll_date=row[2]) p.num_responses = row[3] result_list.append(p) return result_list class OpinionPoll(models.Model): question = models.CharField(max_length=200) poll_date = models.DateField() objects = PollManager() class Response(models.Model): poll = models.ForeignKey(OpinionPoll) person_name = models.CharField(max_length=50) response = models.TextField() With this example, you’d use OpinionPoll.objects.with_counts() to return that list of OpinionPoll objects with num_responses attributes. Another thing to note about this example is that Manager methods can access self.model to get the model class to which they’re attached. Modifying initial Manager QuerySets A Manager‘s base QuerySet returns all objects in the system. For example, using this model: 3.2. Models and databases 119
- 124. Django Documentation, Release1.5.1 class Book(models.Model): title = models.CharField(max_length=100) author = models.CharField(max_length=50) ...the statement Book.objects.all() will return all books in the database. You can override a Manager‘s base QuerySet by overriding the Manager.get_query_set() method. get_query_set() should return a QuerySet with the properties you require. For example, the following model has two Managers – one that returns all objects, and one that returns only the books by Roald Dahl: # First, define the Manager subclass. class DahlBookManager(models.Manager): def get_query_set(self): return super(DahlBookManager, self).get_query_set().filter(author=’Roald Dahl’) # Then hook it into the Book model explicitly. class Book(models.Model): title = models.CharField(max_length=100) author = models.CharField(max_length=50) objects = models.Manager() # The default manager. dahl_objects = DahlBookManager() # The Dahl-specific manager. With this sample model, Book.objects.all() will return all books in the database, but Book.dahl_objects.all() will only return the ones written by Roald Dahl. Of course, because get_query_set() returns a QuerySet object, you can use filter(), exclude() and all the other QuerySet methods on it. So these statements are all legal: Book.dahl_objects.all() Book.dahl_objects.filter(title=’Matilda’) Book.dahl_objects.count() This example also pointed out another interesting technique: using multiple managers on the same model. You can attach as many Manager() instances to a model as you’d like. This is an easy way to define common “filters” for your models. For example: class MaleManager(models.Manager): def get_query_set(self): return super(MaleManager, self).get_query_set().filter(sex=’M’) class FemaleManager(models.Manager): def get_query_set(self): return super(FemaleManager, self).get_query_set().filter(sex=’F’) class Person(models.Model): first_name = models.CharField(max_length=50) last_name = models.CharField(max_length=50) sex = models.CharField(max_length=1, choices=((’M’, ’Male’), (’F’, ’Female’))) people = models.Manager() men = MaleManager() women = FemaleManager() This example allows you to request Person.men.all(), Person.women.all(), and Person.people.all(), yielding predictable results. 120 Chapter 3. Using Django
- 125. Django Documentation, Release1.5.1 If you use custom Manager objects, take note that the first Manager Django encounters (in the order in which they’re defined in the model) has a special status. Django interprets the first Manager defined in a class as the “default” Manager, and several parts of Django (including dumpdata) will use that Manager exclusively for that model. As a result, it’s a good idea to be careful in your choice of default manager in order to avoid a situation where overriding get_query_set() results in an inability to retrieve objects you’d like to work with. Using managers for related object access By default, Django uses an instance of a “plain” manager class when accessing related objects (i.e. choice.poll), not the default manager on the related object. This is because Django needs to be able to retrieve the related object, even if it would otherwise be filtered out (and hence be inaccessible) by the default manager. If the normal plain manager class (django.db.models.Manager) is not appropriate for your circum-stances, you can force Django to use the same class as the default manager for your model by setting the use_for_related_fields attribute on the manager class. This is documented fully below. Custom managers and model inheritance Class inheritance and model managers aren’t quite a perfect match for each other. Managers are often specific to the classes they are defined on and inheriting them in subclasses isn’t necessarily a good idea. Also, because the first manager declared is the default manager, it is important to allow that to be controlled. So here’s how Django handles custom managers and model inheritance: 1. Managers defined on non-abstract base classes are not inherited by child classes. If you want to reuse a manager from a non-abstract base, redeclare it explicitly on the child class. These sorts of managers are likely to be fairly specific to the class they are defined on, so inheriting them can often lead to unexpected results (particularly as far as the default manager goes). Therefore, they aren’t passed onto child classes. 2. Managers from abstract base classes are always inherited by the child class, using Python’s normal name reso-lution order (names on the child class override all others; then come names on the first parent class, and so on). Abstract base classes are designed to capture information and behavior that is common to their child classes. Defining common managers is an appropriate part of this common information. 3. The default manager on a class is either the first manager declared on the class, if that exists, or the default manager of the first abstract base class in the parent hierarchy, if that exists. If no default manager is explicitly declared, Django’s normal default manager is used. These rules provide the necessary flexibility if you want to install a collection of custom managers on a group of models, via an abstract base class, but still customize the default manager. For example, suppose you have this base class: class AbstractBase(models.Model): ... objects = CustomManager() class Meta: abstract = True If you use this directly in a subclass, objects will be the default manager if you declare no managers in the base class: class ChildA(AbstractBase): ... # This class has CustomManager as the default manager. If you want to inherit from AbstractBase, but provide a different default manager, you can provide the default manager on the child class: 3.2. Models and databases 121
- 126. Django Documentation, Release1.5.1 class ChildB(AbstractBase): ... # An explicit default manager. default_manager = OtherManager() Here, default_manager is the default. The objects manager is still available, since it’s inherited. It just isn’t used as the default. Finally for this example, suppose you want to add extra managers to the child class, but still use the default from AbstractBase. You can’t add the new manager directly in the child class, as that would override the default and you would have to also explicitly include all the managers from the abstract base class. The solution is to put the extra managers in another base class and introduce it into the inheritance hierarchy after the defaults: class ExtraManager(models.Model): extra_manager = OtherManager() class Meta: abstract = True class ChildC(AbstractBase, ExtraManager): ... # Default manager is CustomManager, but OtherManager is # also available via the "extra_manager" attribute. Note that while you can define a custom manager on the abstract model, you can’t invoke any methods using the abstract model. That is: ClassA.objects.do_something() is legal, but: AbstractBase.objects.do_something() will raise an exception. This is because managers are intended to encapsulate logic for managing collections of objects. Since you can’t have a collection of abstract objects, it doesn’t make sense to be managing them. If you have function-ality that applies to the abstract model, you should put that functionality in a staticmethod or classmethod on the abstract model. Implementation concerns Whatever features you add to your custom Manager, it must be possible to make a shallow copy of a Manager instance; i.e., the following code must work: >>> import copy >>> manager = MyManager() >>> my_copy = copy.copy(manager) Django makes shallow copies of manager objects during certain queries; if your Manager cannot be copied, those queries will fail. This won’t be an issue for most custom managers. If you are just adding simple methods to your Manager, it is unlikely that you will inadvertently make instances of your Manager uncopyable. However, if you’re overriding __getattr__ or some other private method of your Manager object that controls object state, you should ensure that you don’t affect the ability of your Manager to be copied. 122 Chapter 3. Using Django
- 127. Django Documentation, Release1.5.1 Controlling automatic Manager types This document has already mentioned a couple of places where Django creates a manager class for you: default managers and the “plain” manager used to access related objects. There are other places in the implementation of Django where temporary plain managers are needed. Those automatically created managers will normally be instances of the django.db.models.Manager class. Throughout this section, we will use the term “automatic manager” to mean a manager that Django creates for you – either as a default manager on a model with no managers, or to use temporarily when accessing related objects. Sometimes this default class won’t be the right choice. One example is in the django.contrib.gis application that ships with Django itself. All gis models must use a special manager class (GeoManager) because they need a special queryset (GeoQuerySet) to be used for interacting with the database. It turns out that models which require a special manager like this need to use the same manager class wherever an automatic manager is created. Django provides a way for custom manager developers to say that their manager class should be used for automatic managers whenever it is the default manager on a model. This is done by setting the use_for_related_fields attribute on the manager class: class MyManager(models.Manager): use_for_related_fields = True ... If this attribute is set on the default manager for a model (only the default manager is considered in these situations), Django will use that class whenever it needs to automatically create a manager for the class. Otherwise, it will use django.db.models.Manager. Historical Note Given the purpose for which it’s used, the name of this attribute (use_for_related_fields) might seem a little odd. Originally, the attribute only controlled the type of manager used for related field access, which is where the name came from. As it became clear the concept was more broadly useful, the name hasn’t been changed. This is primarily so that existing code will continue to work in future Django versions. Writing correct Managers for use in automatic Manager instances As already suggested by the django.contrib.gis example, above, the use_for_related_fields feature is primarily for managers that need to return a custom QuerySet subclass. In providing this functionality in your manager, there are a couple of things to remember. Do not filter away any results in this type of manager subclass One reason an automatic manager is used is to access objects that are related to from some other model. In those situations, Django has to be able to see all the objects for the model it is fetching, so that anything which is referred to can be retrieved. If you override the get_query_set() method and filter out any rows, Django will return incorrect results. Don’t do that. A manager that filters results in get_query_set() is not appropriate for use as an automatic manager. Set use_for_related_fields when you define the class The use_for_related_fields attribute must be set on the manager class, not on an instance of the class. The earlier example shows the correct way to set it, whereas the following will not work: # BAD: Incorrect code class MyManager(models.Manager): ... 3.2. Models and databases 123
- 128. Django Documentation, Release1.5.1 # Sets the attribute on an instance of MyManager. Django will # ignore this setting. mgr = MyManager() mgr.use_for_related_fields = True class MyModel(models.Model): ... objects = mgr # End of incorrect code. You also shouldn’t change the attribute on the class object after it has been used in a model, since the attribute’s value is processed when the model class is created and not subsequently reread. Set the attribute on the manager class when it is first defined, as in the initial example of this section and everything will work smoothly. 3.2.5 Performing raw SQL queries When the model query APIs don’t go far enough, you can fall back to writing raw SQL. Django gives you two ways of performing raw SQL queries: you can use Manager.raw() to perform raw queries and return model instances, or you can avoid the model layer entirely and execute custom SQL directly. Performing raw queries The raw() manager method can be used to perform raw SQL queries that return model instances: Manager.raw(raw_query, params=None, translations=None) This method method takes a raw SQL query, executes it, and returns a django.db.models.query.RawQuerySet instance. This RawQuerySet instance can be iterated over just like an normal QuerySet to provide object instances. This is best illustrated with an example. Suppose you’ve got the following model: class Person(models.Model): first_name = models.CharField(...) last_name = models.CharField(...) birth_date = models.DateField(...) You could then execute custom SQL like so: >>> for p in Person.objects.raw(’SELECT * FROM myapp_person’): ... print(p) John Smith Jane Jones Of course, this example isn’t very exciting – it’s exactly the same as running Person.objects.all(). However, raw() has a bunch of other options that make it very powerful. Model table names Where’d the name of the Person table come from in that example? By default, Django figures out a database table name by joining the model’s “app label” – the name you used in manage.py startapp – to the model’s class name, with an underscore between them. In the example we’ve assumed that the Person model lives in an app named myapp, so its table would be myapp_person. 124 Chapter 3. Using Django
- 129. Django Documentation, Release1.5.1 For more details check out the documentation for the db_table option, which also lets you manually set the database table name. Warning: No checking is done on the SQL statement that is passed in to .raw(). Django expects that the statement will return a set of rows from the database, but does nothing to enforce that. If the query does not return rows, a (possibly cryptic) error will result. Mapping query fields to model fields raw() automatically maps fields in the query to fields on the model. The order of fields in your query doesn’t matter. In other words, both of the following queries work identically: >>> Person.objects.raw(’SELECT id, first_name, last_name, birth_date FROM myapp_person’) ... >>> Person.objects.raw(’SELECT last_name, birth_date, first_name, id FROM myapp_person’) ... Matching is done by name. This means that you can use SQL’s AS clauses to map fields in the query to model fields. So if you had some other table that had Person data in it, you could easily map it into Person instances: >>> Person.objects.raw(’’’SELECT first AS first_name, ... last AS last_name, ... bd AS birth_date, ... pk as id, ... FROM some_other_table’’’) As long as the names match, the model instances will be created correctly. Alternatively, you can map fields in the query to model fields using the translations argument to raw(). This is a dictionary mapping names of fields in the query to names of fields on the model. For example, the above query could also be written: >>> name_map = {’first’: ’first_name’, ’last’: ’last_name’, ’bd’: ’birth_date’, ’pk’: ’id’} >>> Person.objects.raw(’SELECT * FROM some_other_table’, translations=name_map) Index lookups raw() supports indexing, so if you need only the first result you can write: >>> first_person = Person.objects.raw(’SELECT * from myapp_person’)[0] However, the indexing and slicing are not performed at the database level. If you have a big amount of Person objects in your database, it is more efficient to limit the query at the SQL level: >>> first_person = Person.objects.raw(’SELECT * from myapp_person LIMIT 1’)[0] Deferring model fields Fields may also be left out: >>> people = Person.objects.raw(’SELECT id, first_name FROM myapp_person’) 3.2. Models and databases 125
- 130. Django Documentation, Release1.5.1 The Person objects returned by this query will be deferred model instances (see defer()). This means that the fields that are omitted from the query will be loaded on demand. For example: >>> for p in Person.objects.raw(’SELECT id, first_name FROM myapp_person’): ... print(p.first_name, # This will be retrieved by the original query ... p.last_name) # This will be retrieved on demand ... John Smith Jane Jones From outward appearances, this looks like the query has retrieved both the first name and last name. However, this example actually issued 3 queries. Only the first names were retrieved by the raw() query – the last names were both retrieved on demand when they were printed. There is only one field that you can’t leave out - the primary key field. Django uses the primary key to identify model instances, so it must always be included in a raw query. An InvalidQuery exception will be raised if you forget to include the primary key. Adding annotations You can also execute queries containing fields that aren’t defined on the model. For example, we could use Post-greSQL’s age() function to get a list of people with their ages calculated by the database: >>> people = Person.objects.raw(’SELECT *, age(birth_date) AS age FROM myapp_person’) >>> for p in people: ... print("%s is %s." % (p.first_name, p.age)) John is 37. Jane is 42. ... Passing parameters into raw() If you need to perform parameterized queries, you can use the params argument to raw(): >>> lname = ’Doe’ >>> Person.objects.raw(’SELECT * FROM myapp_person WHERE last_name = %s’, [lname]) params is a list of parameters. You’ll use %s placeholders in the query string (regardless of your database engine); they’ll be replaced with parameters from the params list. Warning: Do not use string formatting on raw queries! It’s tempting to write the above query as: >>> query = ’SELECT * FROM myapp_person WHERE last_name = %s’ % lname >>> Person.objects.raw(query) Don’t. Using the params list completely protects you from SQL injection attacks, a common exploit where attackers inject arbitrary SQL into your database. If you use string interpolation, sooner or later you’ll fall victim to SQL injection. As long as you remember to always use the params list you’ll be protected. 126 Chapter 3. Using Django
- 131. Django Documentation, Release1.5.1 Executing custom SQL directly Sometimes even Manager.raw() isn’t quite enough: you might need to perform queries that don’t map cleanly to models, or directly execute UPDATE, INSERT, or DELETE queries. In these cases, you can always access the database directly, routing around the model layer entirely. The object django.db.connection represents the default database connection, and django.db.transaction represents the default database transaction. To use the database connection, call connection.cursor() to get a cursor object. Then, call cursor.execute(sql, [params]) to execute the SQL and cursor.fetchone() or cursor.fetchall() to return the resulting rows. After performing a data changing operation, you should then call transaction.commit_unless_managed() to ensure your changes are committed to the database. If your query is purely a data retrieval operation, no commit is required. For example: def my_custom_sql(): from django.db import connection, transaction cursor = connection.cursor() # Data modifying operation - commit required cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz]) transaction.commit_unless_managed() # Data retrieval operation - no commit required cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz]) row = cursor.fetchone() return row If you are using more than one database, you can use django.db.connections to obtain the connection (and cursor) for a specific database. django.db.connections is a dictionary-like object that allows you to retrieve a specific connection using its alias: from django.db import connections cursor = connections[’my_db_alias’].cursor() # Your code here... transaction.commit_unless_managed(using=’my_db_alias’) By default, the Python DB API will return results without their field names, which means you end up with a list of values, rather than a dict. At a small performance cost, you can return results as a dict by using something like this: def dictfetchall(cursor): "Returns all rows from a cursor as a dict" desc = cursor.description return [ dict(zip([col[0] for col in desc], row)) for row in cursor.fetchall() ] Here is an example of the difference between the two: >>> cursor.execute("SELECT id, parent_id from test LIMIT 2"); >>> cursor.fetchall() ((54360982L, None), (54360880L, None)) >>> cursor.execute("SELECT id, parent_id from test LIMIT 2"); >>> dictfetchall(cursor) [{’parent_id’: None, ’id’: 54360982L}, {’parent_id’: None, ’id’: 54360880L}] 3.2. Models and databases 127
- 132. Django Documentation, Release1.5.1 Transactions and raw SQL When you make a raw SQL call, Django will automatically mark the current transaction as dirty. You must then ensure that the transaction containing those calls is closed correctly. See the notes on the requirements of Django’s transaction handling for more details. Connections and cursors connection and cursor mostly implement the standard Python DB-API described in PEP 249 (except when it comes to transaction handling). If you’re not familiar with the Python DB-API, note that the SQL statement in cursor.execute() uses placeholders, "%s", rather than adding parameters directly within the SQL. If you use this technique, the underlying database library will automatically add quotes and escaping to your parameter(s) as necessary. (Also note that Django expects the "%s" placeholder, not the "?" placeholder, which is used by the SQLite Python bindings. This is for the sake of consistency and sanity.) 3.2.6 Managing database transactions Django gives you a few ways to control how database transactions are managed, if you’re using a database that supports transactions. Django’s default transaction behavior Django’s default behavior is to run with an open transaction which it commits automatically when any built-in, data-altering model function is called. For example, if you call model.save() or model.delete(), the change will be committed immediately. This is much like the auto-commit setting for most databases. As soon as you perform an action that needs to write to the database, Django produces the INSERT/UPDATE/DELETE statements and then does the COMMIT. There’s no implicit ROLLBACK. Tying transactions to HTTP requests The recommended way to handle transactions in Web requests is to tie them to the request and response phases via Django’s TransactionMiddleware. It works like this: When a request starts, Django starts a transaction. If the response is produced without problems, Django commits any pending transactions. If the view function produces an exception, Django rolls back any pending transactions. To activate this feature, just add the TransactionMiddleware middleware to your MIDDLEWARE_CLASSES setting: MIDDLEWARE_CLASSES = ( ’django.middleware.cache.UpdateCacheMiddleware’, ’django.contrib.sessions.middleware.SessionMiddleware’, ’django.middleware.common.CommonMiddleware’, ’django.middleware.transaction.TransactionMiddleware’, ’django.middleware.cache.FetchFromCacheMiddleware’, ) The order is quite important. The transaction middleware applies not only to view functions, but also for all middleware modules that come after it. So if you use the session middleware after the transaction middleware, session creation will be part of the transaction. 128 Chapter 3. Using Django
- 133. Django Documentation, Release1.5.1 The various cache middlewares are an exception: CacheMiddleware, UpdateCacheMiddleware, and FetchFromCacheMiddleware are never affected. Even when using database caching, Django’s cache backend uses its own database cursor (which is mapped to its own database connection internally). Note: The TransactionMiddleware only affects the database aliased as “default” within your DATABASES setting. If you are using multiple databases and want transaction control over databases other than “default”, you will need to write your own transaction middleware. Controlling transaction management in views For most people, implicit request-based transactions work wonderfully. However, if you need more fine-grained control over how transactions are managed, you can use a set of functions in django.db.transaction to control transactions on a per-function or per-code-block basis. These functions, described in detail below, can be used in two different ways: • As a decorator on a particular function. For example: from django.db import transaction @transaction.commit_on_success def viewfunc(request): # ... # this code executes inside a transaction # ... • As a context manager around a particular block of code: from django.db import transaction def viewfunc(request): # ... # this code executes using default transaction management # ... with transaction.commit_on_success(): # ... # this code executes inside a transaction # ... Both techniques work with all supported version of Python. For maximum compatibility, all of the examples below show transactions using the decorator syntax, but all of the follow functions may be used as context managers, too. Note: Although the examples below use view functions as examples, these decorators and context managers can be used anywhere in your code that you need to deal with transactions. autocommit() Use the autocommit decorator to switch a view function to Django’s default commit behavior, regardless of the global transaction setting. Example: from django.db import transaction @transaction.autocommit 3.2. Models and databases 129
- 134. Django Documentation, Release1.5.1 def viewfunc(request): .... @transaction.autocommit(using="my_other_database") def viewfunc2(request): .... Within viewfunc(), transactions will be committed as soon as you call model.save(), model.delete(), or any other function that writes to the database. viewfunc2() will have this same behavior, but for the "my_other_database" connection. commit_on_success() Use the commit_on_success decorator to use a single transaction for all the work done in a function: from django.db import transaction @transaction.commit_on_success def viewfunc(request): .... @transaction.commit_on_success(using="my_other_database") def viewfunc2(request): .... If the function returns successfully, then Django will commit all work done within the function at that point. If the function raises an exception, though, Django will roll back the transaction. commit_manually() Use the commit_manually decorator if you need full control over transactions. It tells Django you’ll be managing the transaction on your own. Whether you are writing or simply reading from the database, you must commit() or rollback() explicitly or Django will raise a TransactionManagementError exception. This is required when reading from the database because SELECT statements may call functions which modify tables, and thus it is impossible to know if any data has been modified. Manual transaction management looks like this: from django.db import transaction @transaction.commit_manually def viewfunc(request): ... # You can commit/rollback however and whenever you want transaction.commit() ... # But you’ve got to remember to do it yourself! try: ... except: transaction.rollback() else: transaction.commit() @transaction.commit_manually(using="my_other_database") def viewfunc2(request): .... 130 Chapter 3. Using Django
- 135. Django Documentation, Release1.5.1 Requirements for transaction handling Django requires that every transaction that is opened is closed before the completion of a request. If you are using autocommit() (the default commit mode) or commit_on_success(), this will be done for you automati-cally (with the exception of executing custom SQL). However, if you are manually managing transactions (using the commit_manually() decorator), you must ensure that the transaction is either committed or rolled back before a request is completed. This applies to all database operations, not just write operations. Even if your transaction only reads from the database, the transaction must be committed or rolled back before you complete a request. How to globally deactivate transaction management Control freaks can totally disable all transaction management by setting TRANSACTIONS_MANAGED to True in the Django settings file. If you do this, Django won’t provide any automatic transaction management whatsoever. Middleware will no longer implicitly commit transactions, and you’ll need to roll management yourself. This even requires you to commit changes done by middleware somewhere else. Thus, this is best used in situations where you want to run your own transaction-controlling middleware or do some-thing really strange. In almost all situations, you’ll be better off using the default behavior, or the transaction middle-ware, and only modify selected functions as needed. Savepoints A savepoint is a marker within a transaction that enables you to roll back part of a transaction, rather than the full transaction. Savepoints are available with the PostgreSQL 8, Oracle and MySQL (when using the InnoDB stor-age engine) backends. Other backends provide the savepoint functions, but they’re empty operations – they don’t actually do anything. Changed in version 1.4: Savepoint support for the MySQL backend was added in Django 1.4. Savepoints aren’t especially useful if you are using the default autocommit behavior of Django. However, if you are using commit_on_success or commit_manually, each open transaction will build up a series of database operations, awaiting a commit or rollback. If you issue a rollback, the entire transaction is rolled back. Save-points provide the ability to perform a fine-grained rollback, rather than the full rollback that would be performed by transaction.rollback(). Each of these functions takes a using argument which should be the name of a database for which the behavior applies. If no using argument is provided then the "default" database is used. Savepoints are controlled by three methods on the transaction object: transaction.savepoint(using=None) Creates a new savepoint. This marks a point in the transaction that is known to be in a “good” state. Returns the savepoint ID (sid). transaction.savepoint_commit(sid, using=None) Updates the savepoint to include any operations that have been performed since the savepoint was created, or since the last commit. transaction.savepoint_rollback(sid, using=None) Rolls the transaction back to the last point at which the savepoint was committed. The following example demonstrates the use of savepoints: from django.db import transaction @transaction.commit_manually 3.2. Models and databases 131
- 136. Django Documentation, Release1.5.1 def viewfunc(request): a.save() # open transaction now contains a.save() sid = transaction.savepoint() b.save() # open transaction now contains a.save() and b.save() if want_to_keep_b: transaction.savepoint_commit(sid) # open transaction still contains a.save() and b.save() else: transaction.savepoint_rollback(sid) # open transaction now contains only a.save() transaction.commit() Transactions in MySQL If you’re using MySQL, your tables may or may not support transactions; it depends on your MySQL version and the table types you’re using. (By “table types,” we mean something like “InnoDB” or “MyISAM”.) MySQL transaction peculiarities are outside the scope of this article, but the MySQL site has information on MySQL transactions. If your MySQL setup does not support transactions, then Django will function in auto-commit mode: Statements will be executed and committed as soon as they’re called. If your MySQL setup does support transactions, Django will handle transactions as explained in this document. Handling exceptions within PostgreSQL transactions When a call to a PostgreSQL cursor raises an exception (typically IntegrityError), all subsequent SQL in the same transaction will fail with the error “current transaction is aborted, queries ignored until end of transaction block”. Whilst simple use of save() is unlikely to raise an exception in PostgreSQL, there are more advanced usage patterns which might, such as saving objects with unique fields, saving using the force_insert/force_update flag, or invoking custom SQL. There are several ways to recover from this sort of error. Transaction rollback The first option is to roll back the entire transaction. For example: a.save() # Succeeds, but may be undone by transaction rollback try: b.save() # Could throw exception except IntegrityError: transaction.rollback() c.save() # Succeeds, but a.save() may have been undone Calling transaction.rollback() rolls back the entire transaction. Any uncommitted database operations will be lost. In this example, the changes made by a.save() would be lost, even though that operation raised no error itself. 132 Chapter 3. Using Django
- 137. Django Documentation, Release1.5.1 Savepoint rollback If you are using PostgreSQL 8 or later, you can use savepoints to control the extent of a rollback. Before performing a database operation that could fail, you can set or update the savepoint; that way, if the operation fails, you can roll back the single offending operation, rather than the entire transaction. For example: a.save() # Succeeds, and never undone by savepoint rollback try: sid = transaction.savepoint() b.save() # Could throw exception transaction.savepoint_commit(sid) except IntegrityError: transaction.savepoint_rollback(sid) c.save() # Succeeds, and a.save() is never undone In this example, a.save() will not be undone in the case where b.save() raises an exception. Database-level autocommit With PostgreSQL 8.2 or later, there is an advanced option to run PostgreSQL with database-level autocommit. If you use this option, there is no constantly open transaction, so it is always possible to continue after catching an exception. For example: a.save() # succeeds try: b.save() # Could throw exception except IntegrityError: pass c.save() # succeeds Note: This is not the same as the autocommit decorator. When using database level autocommit there is no database transaction at all. The autocommit decorator still uses transactions, automatically committing each transaction when a database modifying operation occurs. 3.2.7 Multiple databases This topic guide describes Django’s support for interacting with multiple databases. Most of the rest of Django’s documentation assumes you are interacting with a single database. If you want to interact with multiple databases, you’ll need to take some additional steps. Defining your databases The first step to using more than one database with Django is to tell Django about the database servers you’ll be using. This is done using the DATABASES setting. This setting maps database aliases, which are a way to refer to a specific database throughout Django, to a dictionary of settings for that specific connection. The settings in the inner dictionaries are described fully in the DATABASES documentation. Databases can have any alias you choose. However, the alias default has special significance. Django uses the database with the alias of default when no other database has been selected. The following is an example settings.py snippet defining two databases – a default PostgreSQL database and a MySQL database called users: 3.2. Models and databases 133
- 138. Django Documentation, Release1.5.1 DATABASES = { ’default’: { ’NAME’: ’app_data’, ’ENGINE’: ’django.db.backends.postgresql_psycopg2’, ’USER’: ’postgres_user’, ’PASSWORD’: ’s3krit’ }, ’users’: { ’NAME’: ’user_data’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_user’, ’PASSWORD’: ’priv4te’ } } If the concept of a default database doesn’t make sense in the context of your project, you need to be careful to always specify the database that you want to use. Django requires that a default database entry be defined, but the parameters dictionary can be left blank if it will not be used. The following is an example settings.py snippet defining two non-default databases, with the default entry intentionally left empty: DATABASES = { ’default’: {}, ’users’: { ’NAME’: ’user_data’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_user’, ’PASSWORD’: ’superS3cret’ }, ’customers’: { ’NAME’: ’customer_data’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_cust’, ’PASSWORD’: ’veryPriv@ate’ } } If you attempt to access a database that you haven’t defined in your DATABASES setting, Django will raise a django.db.utils.ConnectionDoesNotExist exception. Synchronizing your databases The syncdb management command operates on one database at a time. By default, it operates on the default database, but by providing a --database argument, you can tell syncdb to synchronize a different database. So, to synchronize all models onto all databases in our example, you would need to call: $ ./manage.py syncdb $ ./manage.py syncdb --database=users If you don’t want every application to be synchronized onto a particular database, you can define a database router that implements a policy constraining the availability of particular models. Alternatively, if you want fine-grained control of synchronization, you can pipe all or part of the output of sqlall for a particular application directly into your database prompt, like this: $ ./manage.py sqlall sales | ./manage.py dbshell 134 Chapter 3. Using Django
- 139. Django Documentation, Release1.5.1 Using other management commands The other django-admin.py commands that interact with the database operate in the same way as syncdb – they only ever operate on one database at a time, using --database to control the database used. Automatic database routing The easiest way to use multiple databases is to set up a database routing scheme. The default routing scheme ensures that objects remain ‘sticky’ to their original database (i.e., an object retrieved from the foo database will be saved on the same database). The default routing scheme ensures that if a database isn’t specified, all queries fall back to the default database. You don’t have to do anything to activate the default routing scheme – it is provided ‘out of the box’ on every Django project. However, if you want to implement more interesting database allocation behaviors, you can define and install your own database routers. Database routers A database Router is a class that provides up to four methods: db_for_read(model, **hints) Suggest the database that should be used for read operations for objects of type model. If a database operation is able to provide any additional information that might assist in selecting a database, it will be provided in the hints dictionary. Details on valid hints are provided below. Returns None if there is no suggestion. db_for_write(model, **hints) Suggest the database that should be used for writes of objects of type Model. If a database operation is able to provide any additional information that might assist in selecting a database, it will be provided in the hints dictionary. Details on valid hints are provided below. Returns None if there is no suggestion. allow_relation(obj1, obj2, **hints) Return True if a relation between obj1 and obj2 should be allowed, False if the relation should be prevented, or None if the router has no opinion. This is purely a validation operation, used by foreign key and many to many operations to determine if a relation should be allowed between two objects. allow_syncdb(db, model) Determine if the model should be synchronized onto the database with alias db. Return True if the model should be synchronized, False if it should not be synchronized, or None if the router has no opinion. This method can be used to determine the availability of a model on a given database. A router doesn’t have to provide all these methods – it may omit one or more of them. If one of the methods is omitted, Django will skip that router when performing the relevant check. Hints The hints received by the database router can be used to decide which database should receive a given request. At present, the only hint that will be provided is instance, an object instance that is related to the read or write operation that is underway. This might be the instance that is being saved, or it might be an instance that is being added in a many-to-many relation. In some cases, no instance hint will be provided at all. The router checks for the existence of an instance hint, and determine if that hint should be used to alter routing behavior. 3.2. Models and databases 135
- 140. Django Documentation, Release1.5.1 Using routers Database routers are installed using the DATABASE_ROUTERS setting. This setting defines a list of class names, each specifying a router that should be used by the master router (django.db.router). The master router is used by Django’s database operations to allocate database usage. Whenever a query needs to know which database to use, it calls the master router, providing a model and a hint (if available). Django then tries each router in turn until a database suggestion can be found. If no suggestion can be found, it tries the current _state.db of the hint instance. If a hint instance wasn’t provided, or the instance doesn’t currently have database state, the master router will allocate the default database. An example Example purposes only! This example is intended as a demonstration of how the router infrastructure can be used to alter database usage. It intentionally ignores some complex issues in order to demonstrate how routers are used. This example won’t work if any of the models in myapp contain relationships to models outside of the other database. Cross-database relationships introduce referential integrity problems that Django can’t currently handle. The master/slave configuration described is also flawed – it doesn’t provide any solution for handling replication lag (i.e., query inconsistencies introduced because of the time taken for a write to propagate to the slaves). It also doesn’t consider the interaction of transactions with the database utilization strategy. So - what does this mean in practice? Let’s consider another sample configuration. This one will have several databases: one for the auth application, and all other apps using a master/slave setup with two read slaves. Here are the settings specifying these databases: DATABASES = { ’auth_db’: { ’NAME’: ’auth_db’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_user’, ’PASSWORD’: ’swordfish’, }, ’master’: { ’NAME’: ’master’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_user’, ’PASSWORD’: ’spam’, }, ’slave1’: { ’NAME’: ’slave1’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_user’, ’PASSWORD’: ’eggs’, }, ’slave2’: { ’NAME’: ’slave2’, ’ENGINE’: ’django.db.backends.mysql’, ’USER’: ’mysql_user’, ’PASSWORD’: ’bacon’, }, } 136 Chapter 3. Using Django
- 141. Django Documentation, Release1.5.1 Now we’ll need to handle routing. First we want a router that knows to send queries for the auth app to auth_db: class AuthRouter(object): """ A router to control all database operations on models in the auth application. """ def db_for_read(self, model, **hints): """ Attempts to read auth models go to auth_db. """ if model._meta.app_label == ’auth’: return ’auth_db’ return None def db_for_write(self, model, **hints): """ Attempts to write auth models go to auth_db. """ if model._meta.app_label == ’auth’: return ’auth_db’ return None def allow_relation(self, obj1, obj2, **hints): """ Allow relations if a model in the auth app is involved. """ if obj1._meta.app_label == ’auth’ or obj2._meta.app_label == ’auth’: return True return None def allow_syncdb(self, db, model): """ Make sure the auth app only appears in the ’auth_db’ database. """ if db == ’auth_db’: return model._meta.app_label == ’auth’ elif model._meta.app_label == ’auth’: return False return None And we also want a router that sends all other apps to the master/slave configuration, and randomly chooses a slave to read from: import random class MasterSlaveRouter(object): def db_for_read(self, model, **hints): """ Reads go to a randomly-chosen slave. """ return random.choice([’slave1’, ’slave2’]) def db_for_write(self, model, **hints): """ Writes always go to master. """ 3.2. Models and databases 137
- 142. Django Documentation, Release1.5.1 return ’master’ def allow_relation(self, obj1, obj2, **hints): """ Relations between objects are allowed if both objects are in the master/slave pool. """ db_list = (’master’, ’slave1’, ’slave2’) if obj1.state.db in db_list and obj2.state.db in db_list: return True return None def allow_syncdb(self, db, model): """ All non-auth models end up in this pool. """ return True Finally, in the settings file, we add the following (substituting path.to. with the actual python path to the module(s) where the routers are defined): DATABASE_ROUTERS = [’path.to.AuthRouter’, ’path.to.MasterSlaveRouter’] The order in which routers are processed is significant. Routers will be queried in the order the are listed in the DATABASE_ROUTERS setting . In this example, the AuthRouter is processed before the MasterSlaveRouter, and as a result, decisions concerning the models in auth are processed before any other decision is made. If the DATABASE_ROUTERS setting listed the two routers in the other order, MasterSlaveRouter.allow_syncdb() would be processed first. The catch-all nature of the MasterSlaveR-outer implementation would mean that all models would be available on all databases. With this setup installed, lets run some Django code: >>> # This retrieval will be performed on the ’auth_db’ database >>> fred = User.objects.get(username=’fred’) >>> fred.first_name = ’Frederick’ >>> # This save will also be directed to ’auth_db’ >>> fred.save() >>> # These retrieval will be randomly allocated to a slave database >>> dna = Person.objects.get(name=’Douglas Adams’) >>> # A new object has no database allocation when created >>> mh = Book(title=’Mostly Harmless’) >>> # This assignment will consult the router, and set mh onto >>> # the same database as the author object >>> mh.author = dna >>> # This save will force the ’mh’ instance onto the master database... >>> mh.save() >>> # ... but if we re-retrieve the object, it will come back on a slave >>> mh = Book.objects.get(title=’Mostly Harmless’) 138 Chapter 3. Using Django
- 143. Django Documentation, Release1.5.1 Manually selecting a database Django also provides an API that allows you to maintain complete control over database usage in your code. A manually specified database allocation will take priority over a database allocated by a router. Manually selecting a database for a QuerySet You can select the database for a QuerySet at any point in the QuerySet “chain.” Just call using() on the QuerySet to get another QuerySet that uses the specified database. using() takes a single argument: the alias of the database on which you want to run the query. For example: >>> # This will run on the ’default’ database. >>> Author.objects.all() >>> # So will this. >>> Author.objects.using(’default’).all() >>> # This will run on the ’other’ database. >>> Author.objects.using(’other’).all() Selecting a database for save() Use the using keyword to Model.save() to specify to which database the data should be saved. For example, to save an object to the legacy_users database, you’d use this: >>> my_object.save(using=’legacy_users’) If you don’t specify using, the save() method will save into the default database allocated by the routers. Moving an object from one database to another If you’ve saved an instance to one database, it might be tempt-ing to use save(using=...) as a way to migrate the instance to a new database. However, if you don’t take appropriate steps, this could have some unexpected consequences. Consider the following example: >>> p = Person(name=’Fred’) >>> p.save(using=’first’) # (statement 1) >>> p.save(using=’second’) # (statement 2) In statement 1, a new Person object is saved to the first database. At this time, p doesn’t have a primary key, so Django issues a SQL INSERT statement. This creates a primary key, and Django assigns that primary key to p. When the save occurs in statement 2, p already has a primary key value, and Django will attempt to use that primary key on the new database. If the primary key value isn’t in use in the second database, then you won’t have any problems – the object will be copied to the new database. However, if the primary key of p is already in use on the second database, the existing object in the second database will be overridden when p is saved. You can avoid this in two ways. First, you can clear the primary key of the instance. If an object has no primary key, Django will treat it as a new object, avoiding any loss of data on the second database: 3.2. Models and databases 139
- 144. Django Documentation, Release1.5.1 >>> p = Person(name=’Fred’) >>> p.save(using=’first’) >>> p.pk = None # Clear the primary key. >>> p.save(using=’second’) # Write a completely new object. The second option is to use the force_insert option to save() to ensure that Django does a SQL INSERT: >>> p = Person(name=’Fred’) >>> p.save(using=’first’) >>> p.save(using=’second’, force_insert=True) This will ensure that the person named Fred will have the same primary key on both databases. If that primary key is already in use when you try to save onto the second database, an error will be raised. Selecting a database to delete from By default, a call to delete an existing object will be executed on the same database that was used to retrieve the object in the first place: >>> u = User.objects.using(’legacy_users’).get(username=’fred’) >>> u.delete() # will delete from the ‘legacy_users‘ database To specify the database from which a model will be deleted, pass a using keyword argument to the Model.delete() method. This argument works just like the using keyword argument to save(). For example, if you’re migrating a user from the legacy_users database to the new_users database, you might use these commands: >>> user_obj.save(using=’new_users’) >>> user_obj.delete(using=’legacy_users’) Using managers with multiple databases Use the db_manager() method on managers to give managers access to a non-default database. For example, say you have a custom manager method that touches the database – User.objects.create_user(). Because create_user() is a manager method, not a QuerySet method, you can’t do User.objects.using(’new_users’).create_user(). (The create_user() method is only available on User.objects, the manager, not on QuerySet objects derived from the manager.) The solution is to use db_manager(), like this: User.objects.db_manager(’new_users’).create_user(...) db_manager() returns a copy of the manager bound to the database you specify. Using get_query_set() with multiple databases If you’re overriding get_query_set() on your manager, be sure to either call the method on the parent (using super()) or do the appropriate handling of the _db attribute on the manager (a string containing the name of the database to use). For example, if you want to return a custom QuerySet class from the get_query_set method, you could do this: class MyManager(models.Manager): def get_query_set(self): qs = CustomQuerySet(self.model) if self._db is not None: 140 Chapter 3. Using Django
- 145. Django Documentation, Release1.5.1 qs = qs.using(self._db) return qs Exposing multiple databases in Django’s admin interface Django’s admin doesn’t have any explicit support for multiple databases. If you want to provide an admin interface for a model on a database other than that specified by your router chain, you’ll need to write custom ModelAdmin classes that will direct the admin to use a specific database for content. ModelAdmin objects have five methods that require customization for multiple-database support: class MultiDBModelAdmin(admin.ModelAdmin): # A handy constant for the name of the alternate database. using = ’other’ def save_model(self, request, obj, form, change): # Tell Django to save objects to the ’other’ database. obj.save(using=self.using) def delete_model(self, request, obj): # Tell Django to delete objects from the ’other’ database obj.delete(using=self.using) def queryset(self, request): # Tell Django to look for objects on the ’other’ database. return super(MultiDBModelAdmin, self).queryset(request).using(self.using) def formfield_for_foreignkey(self, db_field, request=None, **kwargs): # Tell Django to populate ForeignKey widgets using a query # on the ’other’ database. return super(MultiDBModelAdmin, self).formfield_for_foreignkey(db_field, request=request, using=def formfield_for_manytomany(self, db_field, request=None, **kwargs): # Tell Django to populate ManyToMany widgets using a query # on the ’other’ database. return super(MultiDBModelAdmin, self).formfield_for_manytomany(db_field, request=request, using=The implementation provided here implements a multi-database strategy where all objects of a given type are stored on a specific database (e.g., all User objects are in the other database). If your usage of multiple databases is more complex, your ModelAdmin will need to reflect that strategy. Inlines can be handled in a similar fashion. They require three customized methods: class MultiDBTabularInline(admin.TabularInline): using = ’other’ def queryset(self, request): # Tell Django to look for inline objects on the ’other’ database. return super(MultiDBTabularInline, self).queryset(request).using(self.using) def formfield_for_foreignkey(self, db_field, request=None, **kwargs): # Tell Django to populate ForeignKey widgets using a query # on the ’other’ database. return super(MultiDBTabularInline, self).formfield_for_foreignkey(db_field, request=request, def formfield_for_manytomany(self, db_field, request=None, **kwargs): # Tell Django to populate ManyToMany widgets using a query 3.2. Models and databases 141
- 146. Django Documentation, Release1.5.1 # on the ’other’ database. return super(MultiDBTabularInline, self).formfield_for_manytomany(db_field, request=request, Once you’ve written your model admin definitions, they can be registered with any Admin instance: from django.contrib import admin # Specialize the multi-db admin objects for use with specific models. class BookInline(MultiDBTabularInline): model = Book class PublisherAdmin(MultiDBModelAdmin): inlines = [BookInline] admin.site.register(Author, MultiDBModelAdmin) admin.site.register(Publisher, PublisherAdmin) othersite = admin.AdminSite(’othersite’) othersite.register(Publisher, MultiDBModelAdmin) This example sets up two admin sites. On the first site, the Author and Publisher objects are exposed; Publisher objects have an tabular inline showing books published by that publisher. The second site exposes just publishers, without the inlines. Using raw cursors with multiple databases If you are using more than one database you can use django.db.connections to obtain the connection (and cursor) for a specific database. django.db.connections is a dictionary-like object that allows you to retrieve a specific connection using its alias: from django.db import connections cursor = connections[’my_db_alias’].cursor() Limitations of multiple databases Cross-database relations Django doesn’t currently provide any support for foreign key or many-to-many relationships spanning multiple databases. If you have used a router to partition models to different databases, any foreign key and many-to-many relationships defined by those models must be internal to a single database. This is because of referential integrity. In order to maintain a relationship between two objects, Django needs to know that the primary key of the related object is valid. If the primary key is stored on a separate database, it’s not possible to easily evaluate the validity of a primary key. If you’re using Postgres, Oracle, or MySQL with InnoDB, this is enforced at the database integrity level – database level key constraints prevent the creation of relations that can’t be validated. However, if you’re using SQLite or MySQL with MyISAM tables, there is no enforced referential integrity; as a result, you may be able to ‘fake’ cross database foreign keys. However, this configuration is not officially supported by Django. 142 Chapter 3. Using Django
- 147. Django Documentation, Release1.5.1 Behavior of contrib apps Several contrib apps include models, and some apps depend on others. Since cross-database relationships are impos-sible, this creates some restrictions on how you can split these models across databases: • each one of contenttypes.ContentType, sessions.Session and sites.Site can be stored in any database, given a suitable router. • auth models — User, Group and Permission — are linked together and linked to ContentType, so they must be stored in the same database as ContentType. • admin and comments depend on auth, so their models must be in the same database as auth. • flatpages and redirects depend on sites, so their models must be in the same database as sites. In addition, some objects are automatically created just after syncdb creates a table to hold them in a database: • a default Site, • a ContentType for each model (including those not stored in that database), • three Permission for each model (including those not stored in that database). Changed in version 1.5: Previously, ContentType and Permission instances were created only in the default database. For common setups with multiple databases, it isn’t useful to have these objects in more than one database. Common setups include master / slave and connecting to external databases. Therefore, it’s recommended: • either to run syncdb only for the default database; • or to write database router that allows synchronizing these three models only to one database. Warning: If you’re synchronizing content types to more that one database, be aware that their primary keys may not match across databases. This may result in data corruption or data loss. 3.2.8 Tablespaces A common paradigm for optimizing performance in database systems is the use of tablespaces to organize disk layout. Warning: Django does not create the tablespaces for you. Please refer to your database engine’s documentation for details on creating and managing tablespaces. Declaring tablespaces for tables A tablespace can be specified for the table generated by a model by supplying the db_tablespace option inside the model’s class Meta. This option also affects tables automatically created for ManyToManyFields in the model. You can use the DEFAULT_TABLESPACE setting to specify a default value for db_tablespace. This is useful for setting a tablespace for the built-in Django apps and other applications whose code you cannot control. Declaring tablespaces for indexes You can pass the db_tablespace option to a Field constructor to specify an alternate tablespace for the Field‘s column index. If no index would be created for the column, the option is ignored. You can use the DEFAULT_INDEX_TABLESPACE setting to specify a default value for db_tablespace. 3.2. Models and databases 143
- 148. Django Documentation, Release1.5.1 If db_tablespace isn’t specified and you didn’t set DEFAULT_INDEX_TABLESPACE, the index is created in the same tablespace as the tables. An example class TablespaceExample(models.Model): name = models.CharField(max_length=30, db_index=True, db_tablespace="indexes") data = models.CharField(max_length=255, db_index=True) edges = models.ManyToManyField(to="self", db_tablespace="indexes") class Meta: db_tablespace = "tables" In this example, the tables generated by the TablespaceExample model (i.e. the model table and the many-to- many table) would be stored in the tables tablespace. The index for the name field and the indexes on the many-to-many table would be stored in the indexes tablespace. The data field would also generate an index, but no tablespace for it is specified, so it would be stored in the model tablespace tables by default. Database support PostgreSQL and Oracle support tablespaces. SQLite and MySQL don’t. When you use a backend that lacks support for tablespaces, Django ignores all tablespace-related options. Changed in version 1.4: Since Django 1.4, the PostgreSQL backend supports tablespaces. 3.2.9 Database access optimization Django’s database layer provides various ways to help developers get the most out of their databases. This document gathers together links to the relevant documentation, and adds various tips, organized under a number of headings that outline the steps to take when attempting to optimize your database usage. Profile first As general programming practice, this goes without saying. Find out what queries you are doing and what they are costing you. You may also want to use an external project like django-debug-toolbar, or a tool that monitors your database directly. Remember that you may be optimizing for speed or memory or both, depending on your requirements. Sometimes optimizing for one will be detrimental to the other, but sometimes they will help each other. Also, work that is done by the database process might not have the same cost (to you) as the same amount of work done in your Python process. It is up to you to decide what your priorities are, where the balance must lie, and profile all of these as required since this will depend on your application and server. With everything that follows, remember to profile after every change to ensure that the change is a benefit, and a big enough benefit given the decrease in readability of your code. All of the suggestions below come with the caveat that in your circumstances the general principle might not apply, or might even be reversed. Use standard DB optimization techniques ...including: • Indexes. This is a number one priority, after you have determined from profiling what indexes should be added. Use django.db.models.Field.db_index to add these from Django. 144 Chapter 3. Using Django
- 149. Django Documentation, Release1.5.1 • Appropriate use of field types. We will assume you have done the obvious things above. The rest of this document focuses on how to use Django in such a way that you are not doing unnecessary work. This document also does not address other optimization techniques that apply to all expensive operations, such as general purpose caching. Understand QuerySets Understanding QuerySets is vital to getting good performance with simple code. In particular: Understand QuerySet evaluation To avoid performance problems, it is important to understand: • that QuerySets are lazy. • when they are evaluated. • how the data is held in memory. Understand cached attributes As well as caching of the whole QuerySet, there is caching of the result of attributes on ORM objects. In general, attributes that are not callable will be cached. For example, assuming the example Weblog models: >>> entry = Entry.objects.get(id=1) >>> entry.blog # Blog object is retrieved at this point >>> entry.blog # cached version, no DB access But in general, callable attributes cause DB lookups every time: >>> entry = Entry.objects.get(id=1) >>> entry.authors.all() # query performed >>> entry.authors.all() # query performed again Be careful when reading template code - the template system does not allow use of parentheses, but will call callables automatically, hiding the above distinction. Be careful with your own custom properties - it is up to you to implement caching. Use the with template tag To make use of the caching behavior of QuerySet, you may need to use the with template tag. Use iterator() When you have a lot of objects, the caching behavior of the QuerySet can cause a large amount of memory to be used. In this case, iterator() may help. 3.2. Models and databases 145
- 150. Django Documentation, Release1.5.1 Do database work in the database rather than in Python For instance: • At the most basic level, use filter and exclude to do filtering in the database. • Use F() object query expressions to do filtering against other fields within the same model. • Use annotate to do aggregation in the database. If these aren’t enough to generate the SQL you need: Use QuerySet.extra() A less portable but more powerful method is extra(), which allows some SQL to be explicitly added to the query. If that still isn’t powerful enough: Use raw SQL Write your own custom SQL to retrieve data or populate models. Use django.db.connection.queries to find out what Django is writing for you and start from there. Retrieve individual objects using a unique, indexed column There are two reasons to use a column with unique or db_index when using get() to retrieve individual objects. First, the query will be quicker because of the underlying database index. Also, the query could run much slower if multiple objects match the lookup; having a unique constraint on the column guarantees this will never happen. So using the example Weblog models: >>> entry = Entry.objects.get(id=10) will be quicker than: >>> entry = Entry.object.get(headline="News Item Title") because id is indexed by the database and is guaranteed to be unique. Doing the following is potentially quite slow: >>> entry = Entry.objects.get(headline__startswith="News") First of all, headline is not indexed, which will make the underlying database fetch slower. Second, the lookup doesn’t guarantee that only one object will be returned. If the query matches more than one object, it will retrieve and transfer all of them from the database. This penalty could be substantial if hundreds or thousands of records are returned. The penalty will be compounded if the database lives on a separate server, where network overhead and latency also play a factor. Retrieve everything at once if you know you will need it Hitting the database multiple times for different parts of a single ‘set’ of data that you will need all parts of is, in general, less efficient than retrieving it all in one query. This is particularly important if you have a query that is executed in a loop, and could therefore end up doing many database queries, when only one was needed. So: 146 Chapter 3. Using Django
- 151. Django Documentation, Release1.5.1 Use QuerySet.select_related() and prefetch_related() Understand select_related() and prefetch_related() thoroughly, and use them: • in view code, • and in managers and default managers where appropriate. Be aware when your manager is and is not used; sometimes this is tricky so don’t make assumptions. Don’t retrieve things you don’t need Use QuerySet.values() and values_list() When you just want a dict or list of values, and don’t need ORM model objects, make appropriate usage of values(). These can be useful for replacing model objects in template code - as long as the dicts you supply have the same attributes as those used in the template, you are fine. Use QuerySet.defer() and only() Use defer() and only() if there are database columns you know that you won’t need (or won’t need in most cases) to avoid loading them. Note that if you do use them, the ORM will have to go and get them in a separate query, making this a pessimization if you use it inappropriately. Also, be aware that there is some (small extra) overhead incurred inside Django when constructing a model with deferred fields. Don’t be too aggressive in deferring fields without profiling as the database has to read most of the non-text, non-VARCHAR data from the disk for a single row in the results, even if it ends up only using a few columns. The defer() and only() methods are most useful when you can avoid loading a lot of text data or for fields that might take a lot of processing to convert back to Python. As always, profile first, then optimize. Use QuerySet.count() ...if you only want the count, rather than doing len(queryset). Use QuerySet.exists() ...if you only want to find out if at least one result exists, rather than if queryset. But: Don’t overuse count() and exists() If you are going to need other data from the QuerySet, just evaluate it. For example, assuming an Email model that has a body attribute and a many-to-many relation to User, the following template code is optimal: {% if display_inbox %} {% with emails=user.emails.all %} {% if emails %} <p>You have {{ emails|length }} email(s)</p> {% for email in emails %} <p>{{ email.body }}</p> 3.2. Models and databases 147
- 152. Django Documentation, Release1.5.1 {% endfor %} {% else %} <p>No messages today.</p> {% endif %} {% endwith %} {% endif %} It is optimal because: 1. Since QuerySets are lazy, this does no database queries if ‘display_inbox’ is False. 2. Use of with means that we store user.emails.all in a variable for later use, allowing its cache to be re-used. 3. The line {% if emails %} causes QuerySet.__bool__() to be called, which causes the user.emails.all() query to be run on the database, and at the least the first line to be turned into an ORM object. If there aren’t any results, it will return False, otherwise True. 4. The use of {{ emails|length }} calls QuerySet.__len__(), filling out the rest of the cache without doing another query. 5. The for loop iterates over the already filled cache. In total, this code does either one or zero database queries. The only deliberate optimization performed is the use of the with tag. Using QuerySet.exists() or QuerySet.count() at any point would cause additional queries. Use QuerySet.update() and delete() Rather than retrieve a load of objects, set some values, and save them individual, use a bulk SQL UPDATE statement, via QuerySet.update(). Similarly, do bulk deletes where possible. Note, however, that these bulk update methods cannot call the save() or delete() methods of individual instances, which means that any custom behavior you have added for these methods will not be executed, including anything driven from the normal database object signals. Use foreign key values directly If you only need a foreign key value, use the foreign key value that is already on the object you’ve got, rather than getting the whole related object and taking its primary key. i.e. do: entry.blog_id instead of: entry.blog.id Insert in bulk When creating objects, where possible, use the bulk_create() method to reduce the number of SQL queries. For example: Entry.objects.bulk_create([ Entry(headline="Python 3.0 Released"), Entry(headline="Python 3.1 Planned") ]) 148 Chapter 3. Using Django
- 153. Django Documentation, Release1.5.1 ...is preferable to: Entry.objects.create(headline="Python 3.0 Released") Entry.objects.create(headline="Python 3.1 Planned") Note that there are a number of caveats to this method, so make sure it’s appropriate for your use case. This also applies to ManyToManyFields, so doing: my_band.members.add(me, my_friend) ...is preferable to: my_band.members.add(me) my_band.members.add(my_friend) ...where Bands and Artists have a many-to-many relationship. 3.2.10 Examples of model relationship API usage Many-to-many relationships To define a many-to-many relationship, use ManyToManyField. In this example, an Article can be published in multiple Publication objects, and a Publication has mul-tiple Article objects: from django.db import models class Publication(models.Model): title = models.CharField(max_length=30) def __unicode__(self): return self.title class Meta: ordering = (’title’,) class Article(models.Model): headline = models.CharField(max_length=100) publications = models.ManyToManyField(Publication) def __unicode__(self): return self.headline class Meta: ordering = (’headline’,) What follows are examples of operations that can be performed using the Python API facilities. Create a couple of Publications: >>> p1 = Publication(title=’The Python Journal’) >>> p1.save() >>> p2 = Publication(title=’Science News’) >>> p2.save() >>> p3 = Publication(title=’Science Weekly’) >>> p3.save() 3.2. Models and databases 149
- 154. Django Documentation, Release1.5.1 Create an Article: >>> a1 = Article(headline=’Django lets you build Web apps easily’) You can’t associate it with a Publication until it’s been saved: >>> a1.publications.add(p1) Traceback (most recent call last): ... ValueError: ’Article’ instance needs to have a primary key value before a many-to-many relationship can Save it! >>> a1.save() Associate the Article with a Publication: >>> a1.publications.add(p1) Create another Article, and set it to appear in both Publications: >>> a2 = Article(headline=’NASA uses Python’) >>> a2.save() >>> a2.publications.add(p1, p2) >>> a2.publications.add(p3) Adding a second time is OK: >>> a2.publications.add(p3) Adding an object of the wrong type raises TypeError: >>> a2.publications.add(a1) Traceback (most recent call last): ... TypeError: ’Publication’ instance expected Create and add a Publication to an Article in one step using create(): >>> new_publication = a2.publications.create(title=’Highlights for Children’) Article objects have access to their related Publication objects: >>> a1.publications.all() [<Publication: The Python Journal>] >>> a2.publications.all() [<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, Publication objects have access to their related Article objects: >>> p2.article_set.all() [<Article: NASA uses Python>] >>> p1.article_set.all() [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] >>> Publication.objects.get(id=4).article_set.all() [<Article: NASA uses Python>] Many-to-many relationships can be queried using lookups across relationships: >>> Article.objects.filter(publications__id__exact=1) [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] >>> Article.objects.filter(publications__pk=1) 150 Chapter 3. Using Django
- 155. Django Documentation, Release1.5.1 [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] >>> Article.objects.filter(publications=1) [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] >>> Article.objects.filter(publications=p1) [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] >>> Article.objects.filter(publications__title__startswith="Science") [<Article: NASA uses Python>, <Article: NASA uses Python>] >>> Article.objects.filter(publications__title__startswith="Science").distinct() [<Article: NASA uses Python>] The count() function respects distinct() as well: >>> Article.objects.filter(publications__title__startswith="Science").count() 2 >>> Article.objects.filter(publications__title__startswith="Science").distinct().count() 1 >>> Article.objects.filter(publications__in=[1,2]).distinct() [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] >>> Article.objects.filter(publications__in=[p1,p2]).distinct() [<Article: Django lets you build Web apps easily>, <Article: NASA uses Python>] Reverse m2m queries are supported (i.e., starting at the table that doesn’t have a ManyToManyField): >>> Publication.objects.filter(id__exact=1) [<Publication: The Python Journal>] >>> Publication.objects.filter(pk=1) [<Publication: The Python Journal>] >>> Publication.objects.filter(article__headline__startswith="NASA") [<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, >>> Publication.objects.filter(article__id__exact=1) [<Publication: The Python Journal>] >>> Publication.objects.filter(article__pk=1) [<Publication: The Python Journal>] >>> Publication.objects.filter(article=1) [<Publication: The Python Journal>] >>> Publication.objects.filter(article=a1) [<Publication: The Python Journal>] >>> Publication.objects.filter(article__in=[1,2]).distinct() [<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, >>> Publication.objects.filter(article__in=[a1,a2]).distinct() [<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>, Excluding a related item works as you would expect, too (although the SQL involved is a little complex): >>> Article.objects.exclude(publications=p2) [<Article: Django lets you build Web apps easily>] If we delete a Publication, its Articles won’t be able to access it: >>> p1.delete() >>> Publication.objects.all() [<Publication: Highlights for Children>, <Publication: Science News>, <Publication: Science Weekly>] 3.2. Models and databases 151
- 156. Django Documentation, Release1.5.1 >>> a1 = Article.objects.get(pk=1) >>> a1.publications.all() [] If we delete an Article, its Publications won’t be able to access it: >>> a2.delete() >>> Article.objects.all() [<Article: Django lets you build Web apps easily>] >>> p2.article_set.all() [] Adding via the ‘other’ end of an m2m: >>> a4 = Article(headline=’NASA finds intelligent life on Earth’) >>> a4.save() >>> p2.article_set.add(a4) >>> p2.article_set.all() [<Article: NASA finds intelligent life on Earth>] >>> a4.publications.all() [<Publication: Science News>] Adding via the other end using keywords: >>> new_article = p2.article_set.create(headline=’Oxygen-free diet works wonders’) >>> p2.article_set.all() [<Article: NASA finds intelligent life on Earth>, <Article: Oxygen-free diet works wonders>] >>> a5 = p2.article_set.all()[1] >>> a5.publications.all() [<Publication: Science News>] Removing Publication from an Article: >>> a4.publications.remove(p2) >>> p2.article_set.all() [<Article: Oxygen-free diet works wonders>] >>> a4.publications.all() [] And from the other end: >>> p2.article_set.remove(a5) >>> p2.article_set.all() [] >>> a5.publications.all() [] Relation sets can be assigned. Assignment clears any existing set members: >>> a4.publications.all() [<Publication: Science News>] >>> a4.publications = [p3] >>> a4.publications.all() [<Publication: Science Weekly>] Relation sets can be cleared: >>> p2.article_set.clear() >>> p2.article_set.all() [] 152 Chapter 3. Using Django
- 157. Django Documentation, Release1.5.1 And you can clear from the other end: >>> p2.article_set.add(a4, a5) >>> p2.article_set.all() [<Article: NASA finds intelligent life on Earth>, <Article: Oxygen-free diet works wonders>] >>> a4.publications.all() [<Publication: Science News>, <Publication: Science Weekly>] >>> a4.publications.clear() >>> a4.publications.all() [] >>> p2.article_set.all() [<Article: Oxygen-free diet works wonders>] Recreate the Article and Publication we have deleted: >>> p1 = Publication(title=’The Python Journal’) >>> p1.save() >>> a2 = Article(headline=’NASA uses Python’) >>> a2.save() >>> a2.publications.add(p1, p2, p3) Bulk delete some Publications - references to deleted publications should go: >>> Publication.objects.filter(title__startswith=’Science’).delete() >>> Publication.objects.all() [<Publication: Highlights for Children>, <Publication: The Python Journal>] >>> Article.objects.all() [<Article: Django lets you build Web apps easily>, <Article: NASA finds intelligent life on Earth>, <>>> a2.publications.all() [<Publication: The Python Journal>] Bulk delete some articles - references to deleted objects should go: >>> q = Article.objects.filter(headline__startswith=’Django’) >>> print(q) [<Article: Django lets you build Web apps easily>] >>> q.delete() After the delete(), the QuerySet cache needs to be cleared, and the referenced objects should be gone: >>> print(q) [] >>> p1.article_set.all() [<Article: NASA uses Python>] An alternate to calling clear() is to assign the empty set: >>> p1.article_set = [] >>> p1.article_set.all() [] >>> a2.publications = [p1, new_publication] >>> a2.publications.all() [<Publication: Highlights for Children>, <Publication: The Python Journal>] >>> a2.publications = [] >>> a2.publications.all() [] 3.2. Models and databases 153
- 158. Django Documentation, Release1.5.1 Many-to-one relationships To define a many-to-one relationship, use ForeignKey. from django.db import models class Reporter(models.Model): first_name = models.CharField(max_length=30) last_name = models.CharField(max_length=30) email = models.EmailField() def __unicode__(self): return u"%s %s" % (self.first_name, self.last_name) class Article(models.Model): headline = models.CharField(max_length=100) pub_date = models.DateField() reporter = models.ForeignKey(Reporter) def __unicode__(self): return self.headline class Meta: ordering = (’headline’,) What follows are examples of operations that can be performed using the Python API facilities. Create a few Reporters: >>> r = Reporter(first_name=’John’, last_name=’Smith’, email=’john@example.com’) >>> r.save() >>> r2 = Reporter(first_name=’Paul’, last_name=’Jones’, email=’paul@example.com’) >>> r2.save() Create an Article: >>> from datetime import date >>> a = Article(id=None, headline="This is a test", pub_date=date(2005, 7, 27), reporter=r) >>> a.save() >>> a.reporter.id 1 >>> a.reporter <Reporter: John Smith> Article objects have access to their related Reporter objects: >>> r = a.reporter These are strings instead of unicode strings because that’s what was used in the creation of this reporter (and we haven’t refreshed the data from the database, which always returns unicode strings): >>> r.first_name, r.last_name (’John’, ’Smith’) Create an Article via the Reporter object: 154 Chapter 3. Using Django
- 159. Django Documentation, Release1.5.1 >>> new_article = r.article_set.create(headline="John’s second story", pub_date=date(2005, 7, 29)) >>> new_article <Article: John’s second story> >>> new_article.reporter <Reporter: John Smith> >>> new_article.reporter.id 1 Create a new article, and add it to the article set: >>> new_article2 = Article(headline="Paul’s story", pub_date=date(2006, 1, 17)) >>> r.article_set.add(new_article2) >>> new_article2.reporter <Reporter: John Smith> >>> new_article2.reporter.id 1 >>> r.article_set.all() [<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>] Add the same article to a different article set - check that it moves: >>> r2.article_set.add(new_article2) >>> new_article2.reporter.id 2 >>> new_article2.reporter <Reporter: Paul Jones> Adding an object of the wrong type raises TypeError: >>> r.article_set.add(r2) Traceback (most recent call last): ... TypeError: ’Article’ instance expected >>> r.article_set.all() [<Article: John’s second story>, <Article: This is a test>] >>> r2.article_set.all() [<Article: Paul’s story>] >>> r.article_set.count() 2 >>> r2.article_set.count() 1 Note that in the last example the article has moved from John to Paul. Related managers support field lookups as well. The API automatically follows relationships as far as you need. Use double underscores to separate relationships. This works as many levels deep as you want. There’s no limit. For example: >>> r.article_set.filter(headline__startswith=’This’) [<Article: This is a test>] # Find all Articles for any Reporter whose first name is "John". >>> Article.objects.filter(reporter__first_name__exact=’John’) [<Article: John’s second story>, <Article: This is a test>] Exact match is implied here: 3.2. Models and databases 155
- 160. Django Documentation, Release1.5.1 >>> Article.objects.filter(reporter__first_name=’John’) [<Article: John’s second story>, <Article: This is a test>] Query twice over the related field. This translates to an AND condition in the WHERE clause: >>> Article.objects.filter(reporter__first_name__exact=’John’, reporter__last_name__exact=’Smith’) [<Article: John’s second story>, <Article: This is a test>] For the related lookup you can supply a primary key value or pass the related object explicitly: >>> Article.objects.filter(reporter__pk=1) [<Article: John’s second story>, <Article: This is a test>] >>> Article.objects.filter(reporter=1) [<Article: John’s second story>, <Article: This is a test>] >>> Article.objects.filter(reporter=r) [<Article: John’s second story>, <Article: This is a test>] >>> Article.objects.filter(reporter__in=[1,2]).distinct() [<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>] >>> Article.objects.filter(reporter__in=[r,r2]).distinct() [<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>] You can also use a queryset instead of a literal list of instances: >>> Article.objects.filter(reporter__in=Reporter.objects.filter(first_name=’John’)).distinct() [<Article: John’s second story>, <Article: This is a test>] Querying in the opposite direction: >>> Reporter.objects.filter(article__pk=1) [<Reporter: John Smith>] >>> Reporter.objects.filter(article=1) [<Reporter: John Smith>] >>> Reporter.objects.filter(article=a) [<Reporter: John Smith>] >>> Reporter.objects.filter(article__headline__startswith=’This’) [<Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>] >>> Reporter.objects.filter(article__headline__startswith=’This’).distinct() [<Reporter: John Smith>] Counting in the opposite direction works in conjunction with distinct(): >>> Reporter.objects.filter(article__headline__startswith=’This’).count() 3 >>> Reporter.objects.filter(article__headline__startswith=’This’).distinct().count() 1 Queries can go round in circles: >>> Reporter.objects.filter(article__reporter__first_name__startswith=’John’) [<Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>, <Reporter: John Smith>] >>> Reporter.objects.filter(article__reporter__first_name__startswith=’John’).distinct() [<Reporter: John Smith>] >>> Reporter.objects.filter(article__reporter__exact=r).distinct() [<Reporter: John Smith>] If you delete a reporter, his articles will be deleted (assuming that the ForeignKey was defined with django.db.models.ForeignKey.on_delete set to CASCADE, which is the default): 156 Chapter 3. Using Django
- 161. Django Documentation, Release1.5.1 >>> Article.objects.all() [<Article: John’s second story>, <Article: Paul’s story>, <Article: This is a test>] >>> Reporter.objects.order_by(’first_name’) [<Reporter: John Smith>, <Reporter: Paul Jones>] >>> r2.delete() >>> Article.objects.all() [<Article: John’s second story>, <Article: This is a test>] >>> Reporter.objects.order_by(’first_name’) [<Reporter: John Smith>] You can delete using a JOIN in the query: >>> Reporter.objects.filter(article__headline__startswith=’This’).delete() >>> Reporter.objects.all() [] >>> Article.objects.all() [] One-to-one relationships To define a one-to-one relationship, use OneToOneField. In this example, a Place optionally can be a Restaurant: from django.db import models, transaction, IntegrityError class Place(models.Model): name = models.CharField(max_length=50) address = models.CharField(max_length=80) def __unicode__(self): return u"%s the place" % self.name class Restaurant(models.Model): place = models.OneToOneField(Place, primary_key=True) serves_hot_dogs = models.BooleanField() serves_pizza = models.BooleanField() def __unicode__(self): return u"%s the restaurant" % self.place.name class Waiter(models.Model): restaurant = models.ForeignKey(Restaurant) name = models.CharField(max_length=50) def __unicode__(self): return u"%s the waiter at %s" % (self.name, self.restaurant) What follows are examples of operations that can be performed using the Python API facilities. Create a couple of Places: >>> p1 = Place(name=’Demon Dogs’, address=’944 W. Fullerton’) >>> p1.save() >>> p2 = Place(name=’Ace Hardware’, address=’1013 N. Ashland’) >>> p2.save() Create a Restaurant. Pass the ID of the “parent” object as this object’s ID: 3.2. Models and databases 157
- 162. Django Documentation, Release1.5.1 >>> r = Restaurant(place=p1, serves_hot_dogs=True, serves_pizza=False) >>> r.save() A Restaurant can access its place: >>> r.place <Place: Demon Dogs the place> A Place can access its restaurant, if available: >>> p1.restaurant <Restaurant: Demon Dogs the restaurant> p2 doesn’t have an associated restaurant: >>> p2.restaurant Traceback (most recent call last): ... DoesNotExist: Restaurant matching query does not exist. Set the place using assignment notation. Because place is the primary key on Restaurant, the save will create a new restaurant: >>> r.place = p2 >>> r.save() >>> p2.restaurant <Restaurant: Ace Hardware the restaurant> >>> r.place <Place: Ace Hardware the place> Set the place back again, using assignment in the reverse direction: >>> p1.restaurant = r >>> p1.restaurant <Restaurant: Demon Dogs the restaurant> Restaurant.objects.all() just returns the Restaurants, not the Places. Note that there are two restaurants - Ace Hardware the Restaurant was created in the call to r.place = p2: >>> Restaurant.objects.all() [<Restaurant: Demon Dogs the restaurant>, <Restaurant: Ace Hardware the restaurant>] Place.objects.all() returns all Places, regardless of whether they have Restaurants: >>> Place.objects.order_by(’name’) [<Place: Ace Hardware the place>, <Place: Demon Dogs the place>] You can query the models using lookups across relationships: >>> Restaurant.objects.get(place=p1) <Restaurant: Demon Dogs the restaurant> >>> Restaurant.objects.get(place__pk=1) <Restaurant: Demon Dogs the restaurant> >>> Restaurant.objects.filter(place__name__startswith="Demon") [<Restaurant: Demon Dogs the restaurant>] >>> Restaurant.objects.exclude(place__address__contains="Ashland") [<Restaurant: Demon Dogs the restaurant>] This of course works in reverse: 158 Chapter 3. Using Django
- 163. Django Documentation, Release1.5.1 >>> Place.objects.get(pk=1) <Place: Demon Dogs the place> >>> Place.objects.get(restaurant__place__exact=p1) <Place: Demon Dogs the place> >>> Place.objects.get(restaurant=r) <Place: Demon Dogs the place> >>> Place.objects.get(restaurant__place__name__startswith="Demon") <Place: Demon Dogs the place> Add a Waiter to the Restaurant: >>> w = r.waiter_set.create(name=’Joe’) >>> w.save() >>> w <Waiter: Joe the waiter at Demon Dogs the restaurant> Query the waiters: >>> Waiter.objects.filter(restaurant__place=p1) [<Waiter: Joe the waiter at Demon Dogs the restaurant>] >>> Waiter.objects.filter(restaurant__place__name__startswith="Demon") [<Waiter: Joe the waiter at Demon Dogs the restaurant>] 3.3 Handling HTTP requests Information on handling HTTP requests in Django: 3.3.1 URL dispatcher A clean, elegant URL scheme is an important detail in a high-quality Web application. Django lets you design URLs however you want, with no framework limitations. There’s no .php or .cgi required, and certainly none of that 0,2097,1-1-1928,00 nonsense. See Cool URIs don’t change, by World Wide Web creator Tim Berners-Lee, for excellent arguments on why URLs should be clean and usable. Overview To design URLs for an app, you create a Python module informally called a URLconf (URL configuration). This module is pure Python code and is a simple mapping between URL patterns (simple regular expressions) to Python functions (your views). This mapping can be as short or as long as needed. It can reference other mappings. And, because it’s pure Python code, it can be constructed dynamically. New in version 1.4: Django also provides a way to translate URLs according to the active language. See the internationalization documentation for more information. How Django processes a request When a user requests a page from your Django-powered site, this is the algorithm the system follows to determine which Python code to execute: 3.3. Handling HTTP requests 159
- 164. Django Documentation, Release1.5.1 1. Django determines the root URLconf module to use. Ordinarily, this is the value of the ROOT_URLCONF setting, but if the incoming HttpRequest object has an attribute called urlconf (set by middleware request processing), its value will be used in place of the ROOT_URLCONF setting. 2. Django loads that Python module and looks for the variable urlpatterns. This should be a Python list, in the format returned by the function django.conf.urls.patterns(). 3. Django runs through each URL pattern, in order, and stops at the first one that matches the requested URL. 4. Once one of the regexes matches, Django imports and calls the given view, which is a simple Python function (or a class based view). The view gets passed an HttpRequest as its first argument and any values captured in the regex as remaining arguments. 5. If no regex matches, or if an exception is raised during any point in this process, Django invokes an appropriate error-handling view. See Error handling below. Example Here’s a sample URLconf: from django.conf.urls import patterns, url urlpatterns = patterns(’’, url(r’^articles/2003/$’, ’news.views.special_case_2003’), url(r’^articles/(d{4})/$’, ’news.views.year_archive’), url(r’^articles/(d{4})/(d{2})/$’, ’news.views.month_archive’), url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’news.views.article_detail’), ) Notes: • To capture a value from the URL, just put parenthesis around it. • There’s no need to add a leading slash, because every URL has that. For example, it’s ^articles, not ^/articles. • The ’r’ in front of each regular expression string is optional but recommended. It tells Python that a string is “raw” – that nothing in the string should be escaped. See Dive Into Python’s explanation. Example requests: • A request to /articles/2005/03/ would match the third entry in the list. Django would call the function news.views.month_archive(request, ’2005’, ’03’). • /articles/2005/3/ would not match any URL patterns, because the third entry in the list requires two digits for the month. • /articles/2003/ would match the first pattern in the list, not the second one, because the patterns are tested in order, and the first one is the first test to pass. Feel free to exploit the ordering to insert special cases like this. • /articles/2003 would not match any of these patterns, because each pattern requires that the URL end with a slash. • /articles/2003/03/03/ would match the final pattern. Django would call the function news.views.article_detail(request, ’2003’, ’03’, ’03’). 160 Chapter 3. Using Django
- 165. Django Documentation, Release1.5.1 Named groups The above example used simple, non-named regular-expression groups (via parenthesis) to capture bits of the URL and pass them as positional arguments to a view. In more advanced usage, it’s possible to use named regular-expression groups to capture URL bits and pass them as keyword arguments to a view. In Python regular expressions, the syntax for named regular-expression groups is (?P<name>pattern), where name is the name of the group and pattern is some pattern to match. Here’s the above example URLconf, rewritten to use named groups: urlpatterns = patterns(’’, url(r’^articles/2003/$’, ’news.views.special_case_2003’), url(r’^articles/(?P<year>d{4})/$’, ’news.views.year_archive’), url(r’^articles/(?P<year>d{4})/(?P<month>d{2})/$’, ’news.views.month_archive’), url(r’^articles/(?P<year>d{4})/(?P<month>d{2})/(?P<day>d{2})/$’, ’news.views.article_detail’), ) This accomplishes exactly the same thing as the previous example, with one subtle difference: The captured values are passed to view functions as keyword arguments rather than positional arguments. For example: • A request to /articles/2005/03/ would call the function news.views.month_archive(request, year=’2005’, month=’03’), instead of news.views.month_archive(request, ’2005’, ’03’). • A request to /articles/2003/03/03/ would call the function news.views.article_detail(request, year=’2003’, month=’03’, day=’03’). In practice, this means your URLconfs are slightly more explicit and less prone to argument-order bugs – and you can reorder the arguments in your views’ function definitions. Of course, these benefits come at the cost of brevity; some developers find the named-group syntax ugly and too verbose. The matching/grouping algorithm Here’s the algorithm the URLconf parser follows, with respect to named groups vs. non-named groups in a regular expression: 1. If there are any named arguments, it will use those, ignoring non-named arguments. 2. Otherwise, it will pass all non-named arguments as positional arguments. In both cases, any extra keyword arguments that have been given as per Passing extra options to view functions (below) will also be passed to the view. What the URLconf searches against The URLconf searches against the requested URL, as a normal Python string. This does not include GET or POST parameters, or the domain name. For example, in a request to http://www.example.com/myapp/, the URLconf will look for myapp/. In a request to http://www.example.com/myapp/?page=3, the URLconf will look for myapp/. The URLconf doesn’t look at the request method. In other words, all request methods – POST, GET, HEAD, etc. – will be routed to the same function for the same URL. 3.3. Handling HTTP requests 161
- 166. Django Documentation, Release1.5.1 Notes on capturing text in URLs Each captured argument is sent to the view as a plain Python string, regardless of what sort of match the regular expression makes. For example, in this URLconf line: url(r’^articles/(?P<year>d{4})/$’, ’news.views.year_archive’), ...the year argument to news.views.year_archive() will be a string, not an integer, even though the d{4} will only match integer strings. A convenient trick is to specify default parameters for your views’ arguments. Here’s an example URLconf and view: # URLconf urlpatterns = patterns(’’, url(r’^blog/$’, ’blog.views.page’), url(r’^blog/page(?P<num>d+)/$’, ’blog.views.page’), ) # View (in blog/views.py) def page(request, num="1"): # Output the appropriate page of blog entries, according to num. ... In the above example, both URL patterns point to the same view – blog.views.page – but the first pattern doesn’t capture anything from the URL. If the first pattern matches, the page() function will use its default argument for num, "1". If the second pattern matches, page() will use whatever num value was captured by the regex. Performance Each regular expression in a urlpatterns is compiled the first time it’s accessed. This makes the system blazingly fast. Syntax of the urlpatterns variable urlpatterns should be a Python list, in the format returned by the function django.conf.urls.patterns(). Always use patterns() to create the urlpatterns variable. Error handling When Django can’t find a regex matching the requested URL, or when an exception is raised, Django will invoke an error-handling view. The views to use for these cases are specified by three variables. Their default values should suffice for most projects, but further customization is possible by assigning values to them. See the documentation on customizing error views for the full details. Such values can be set in your root URLconf. Setting these variables in any other URLconf will have no effect. Values must be callables, or strings representing the full Python import path to the view that should be called to handle the error condition at hand. The variables are: • handler404 – See django.conf.urls.handler404. • handler500 – See django.conf.urls.handler500. 162 Chapter 3. Using Django
- 167. Django Documentation, Release1.5.1 • handler403 – See django.conf.urls.handler403. New in version 1.4: handler403 is new in Django 1.4. The view prefix You can specify a common prefix in your patterns() call, to cut down on code duplication. Here’s the example URLconf from the Django overview: from django.conf.urls import patterns, url urlpatterns = patterns(’’, url(r’^articles/(d{4})/$’, ’news.views.year_archive’), url(r’^articles/(d{4})/(d{2})/$’, ’news.views.month_archive’), url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’news.views.article_detail’), ) In this example, each view has a common prefix – ’news.views’. Instead of typing that out for each entry in urlpatterns, you can use the first argument to the patterns() function to specify a prefix to apply to each view function. With this in mind, the above example can be written more concisely as: from django.conf.urls import patterns, url urlpatterns = patterns(’news.views’, url(r’^articles/(d{4})/$’, ’year_archive’), url(r’^articles/(d{4})/(d{2})/$’, ’month_archive’), url(r’^articles/(d{4})/(d{2})/(d+)/$’, ’article_detail’), ) Note that you don’t put a trailing dot (".") in the prefix. Django puts that in automatically. Multiple view prefixes In practice, you’ll probably end up mixing and matching views to the point where the views in your urlpatterns won’t have a common prefix. However, you can still take advantage of the view prefix shortcut to remove duplication. Just add multiple patterns() objects together, like this: Old: from django.conf.urls import patterns, url urlpatterns = patterns(’’, url(r’^$’, ’myapp.views.app_index’), url(r’^(?P<year>d{4})/(?P<month>[a-z]{3})/$’, ’myapp.views.month_display’), url(r’^tag/(?P<tag>w+)/$’, ’weblog.views.tag’), ) New: from django.conf.urls import patterns, url urlpatterns = patterns(’myapp.views’, url(r’^$’, ’app_index’), url(r’^(?P<year>d{4})/(?P<month>[a-z]{3})/$’,’month_display’), ) 3.3. Handling HTTP requests 163
- 168. Django Documentation, Release1.5.1 urlpatterns += patterns(’weblog.views’, url(r’^tag/(?P<tag>w+)/$’, ’tag’), ) Including other URLconfs At any point, your urlpatterns can “include” other URLconf modules. This essentially “roots” a set of URLs below other ones. For example, here’s an excerpt of the URLconf for the DjangoWeb site itself. It includes a number of other URLconfs: from django.conf.urls import include, patterns, url urlpatterns = patterns(’’, # ... snip ... url(r’^comments/’, include(’django.contrib.comments.urls’)), url(r’^community/’, include(’django_website.aggregator.urls’)), url(r’^contact/’, include(’django_website.contact.urls’)), url(r’^r/’, include(’django.conf.urls.shortcut’)), # ... snip ... ) Note that the regular expressions in this example don’t have a $ (end-of-string match character) but do include a trailing slash. Whenever Django encounters include() (django.conf.urls.include()), it chops off whatever part of the URL matched up to that point and sends the remaining string to the included URLconf for further processing. Another possibility is to include additional URL patterns not by specifying the URLconf Python module defining them as the include() argument but by using directly the pattern list as returned by patterns() instead. For example, consider this URLconf: from django.conf.urls import include, patterns, url extra_patterns = patterns(’’, url(r’^reports/(?P<id>d+)/$’, ’credit.views.report’), url(r’^charge/$’, ’credit.views.charge’), ) urlpatterns = patterns(’’, url(r’^$’, ’apps.main.views.homepage’), url(r’^help/’, include(’apps.help.urls’)), url(r’^credit/’, include(extra_patterns)), ) In this example, the /credit/reports/ URL will be handled by the credit.views.report() Django view. Captured parameters An included URLconf receives any captured parameters from parent URLconfs, so the following example is valid: # In settings/urls/main.py urlpatterns = patterns(’’, url(r’^(?P<username>w+)/blog/’, include(’foo.urls.blog’)), ) # In foo/urls/blog.py urlpatterns = patterns(’foo.views’, 164 Chapter 3. Using Django
- 169. Django Documentation, Release1.5.1 url(r’^$’, ’blog.index’), url(r’^archive/$’, ’blog.archive’), ) In the above example, the captured "username" variable is passed to the included URLconf, as expected. Passing extra options to view functions URLconfs have a hook that lets you pass extra arguments to your view functions, as a Python dictionary. The django.conf.urls.url() function can take an optional third argument which should be a dictionary of extra keyword arguments to pass to the view function. For example: urlpatterns = patterns(’blog.views’, url(r’^blog/(?P<year>d{4})/$’, ’year_archive’, {’foo’: ’bar’}), ) In this example, for a request to /blog/2005/, Django will call blog.views.year_archive(year=’2005’, foo=’bar’). This technique is used in the syndication framework to pass metadata and options to views. Dealing with conflicts It’s possible to have a URL pattern which captures named keyword arguments, and also passes arguments with the same names in its dictionary of extra arguments. When this happens, the arguments in the dictionary will be used instead of the arguments captured in the URL. Passing extra options to include() Similarly, you can pass extra options to include(). When you pass extra options to include(), each line in the included URLconf will be passed the extra options. For example, these two URLconf sets are functionally identical: Set one: # main.py urlpatterns = patterns(’’, url(r’^blog/’, include(’inner’), {’blogid’: 3}), ) # inner.py urlpatterns = patterns(’’, url(r’^archive/$’, ’mysite.views.archive’), url(r’^about/$’, ’mysite.views.about’), ) Set two: # main.py urlpatterns = patterns(’’, url(r’^blog/’, include(’inner’)), ) 3.3. Handling HTTP requests 165
- 170. Django Documentation, Release1.5.1 # inner.py urlpatterns = patterns(’’, url(r’^archive/$’, ’mysite.views.archive’, {’blogid’: 3}), url(r’^about/$’, ’mysite.views.about’, {’blogid’: 3}), ) Note that extra options will always be passed to every line in the included URLconf, regardless of whether the line’s view actually accepts those options as valid. For this reason, this technique is only useful if you’re certain that every view in the included URLconf accepts the extra options you’re passing. Passing callable objects instead of strings Some developers find it more natural to pass the actual Python function object rather than a string containing the path to its module. This alternative is supported – you can pass any callable object as the view. For example, given this URLconf in “string” notation: urlpatterns = patterns(’’, url(r’^archive/$’, ’mysite.views.archive’), url(r’^about/$’, ’mysite.views.about’), url(r’^contact/$’, ’mysite.views.contact’), ) You can accomplish the same thing by passing objects rather than strings. Just be sure to import the objects: from mysite.views import archive, about, contact urlpatterns = patterns(’’, url(r’^archive/$’, archive), url(r’^about/$’, about), url(r’^contact/$’, contact), ) The following example is functionally identical. It’s just a bit more compact because it imports the module that contains the views, rather than importing each view individually: from mysite import views urlpatterns = patterns(’’, url(r’^archive/$’, views.archive), url(r’^about/$’, views.about), url(r’^contact/$’, views.contact), ) The style you use is up to you. Note that if you use this technique – passing objects rather than strings – the view prefix (as explained in “The view prefix” above) will have no effect. Note that class based views must be imported: from mysite.views import ClassBasedView urlpatterns = patterns(’’, url(r’^myview/$’, ClassBasedView.as_view()), ) 166 Chapter 3. Using Django
- 171. Django Documentation, Release1.5.1 Reverse resolution of URLs A common need when working on a Django project is the possibility to obtain URLs in their final forms either for embedding in generated content (views and assets URLs, URLs shown to the user, etc.) or for handling of the navigation flow on the server side (redirections, etc.) It is strongly desirable not having to hard-code these URLs (a laborious, non-scalable and error-prone strategy) or having to devise ad-hoc mechanisms for generating URLs that are parallel to the design described by the URLconf and as such in danger of producing stale URLs at some point. In other words, what’s needed is a DRY mechanism. Among other advantages it would allow evolution of the URL design without having to go all over the project source code to search and replace outdated URLs. The piece of information we have available as a starting point to get a URL is an identification (e.g. the name) of the view in charge of handling it, other pieces of information that necessarily must participate in the lookup of the right URL are the types (positional, keyword) and values of the view arguments. Django provides a solution such that the URL mapper is the only repository of the URL design. You feed it with your URLconf and then it can be used in both directions: • Starting with a URL requested by the user/browser, it calls the right Django view providing any arguments it might need with their values as extracted from the URL. • Starting with the identification of the corresponding Django view plus the values of arguments that would be passed to it, obtain the associated URL. The first one is the usage we’ve been discussing in the previous sections. The second one is what is known as reverse resolution of URLs, reverse URL matching, reverse URL lookup, or simply URL reversing. Django provides tools for performing URL reversing that match the different layers where URLs are needed: • In templates: Using the url template tag. • In Python code: Using the django.core.urlresolvers.reverse() function. • In higher level code related to handling of URLs of Django model instances: The get_absolute_url() method. Examples Consider again this URLconf entry: from django.conf.urls import patterns, url urlpatterns = patterns(’’, #... url(r’^articles/(d{4})/$’, ’news.views.year_archive’), #... ) According to this design, the URL for the archive corresponding to year nnnn is /articles/nnnn/. You can obtain these in template code by using: <a href="{% url ’news.views.year_archive’ 2012 %}">2012 Archive</a> {# Or with the year in a template context variable: #} <ul> {% for yearvar in year_list %} <li><a href="{% url ’news.views.year_archive’ yearvar %}">{{ yearvar }} Archive</a></li> {% endfor %} </ul> 3.3. Handling HTTP requests 167
- 172. Django Documentation, Release1.5.1 Or in Python code: from django.core.urlresolvers import reverse from django.http import HttpResponseRedirect def redirect_to_year(request): # ... year = 2006 # ... return HttpResponseRedirect(reverse(’news.views.year_archive’, args=(year,))) If, for some reason, it was decided that the URLs where content for yearly article archives are published at should be changed then you would only need to change the entry in the URLconf. In some scenarios where views are of a generic nature, a many-to-one relationship might exist between URLs and views. For these cases the view name isn’t a good enough identificator for it when it comes the time of reversing URLs. Read the next section to know about the solution Django provides for this. Naming URL patterns It’s fairly common to use the same view function in multiple URL patterns in your URLconf. For example, these two URL patterns both point to the archive view: urlpatterns = patterns(’’, url(r’^archive/(d{4})/$’, archive), url(r’^archive-summary/(d{4})/$’, archive, {’summary’: True}), ) This is completely valid, but it leads to problems when you try to do reverse URL matching (through the reverse() function or the url template tag). Continuing this example, if you wanted to retrieve the URL for the archive view, Django’s reverse URL matcher would get confused, because two URL patterns point at that view. To solve this problem, Django supports named URL patterns. That is, you can give a name to a URL pattern in order to distinguish it from other patterns using the same view and parameters. Then, you can use this name in reverse URL matching. Here’s the above example, rewritten to use named URL patterns: urlpatterns = patterns(’’, url(r’^archive/(d{4})/$’, archive, name="full-archive"), url(r’^archive-summary/(d{4})/$’, archive, {’summary’: True}, name="arch-summary"), ) With these names in place (full-archive and arch-summary), you can target each pattern individually by using its name: {% url ’arch-summary’ 1945 %} {% url ’full-archive’ 2007 %} Even though both URL patterns refer to the archive view here, using the name parameter to django.conf.urls.url() allows you to tell them apart in templates. The string used for the URL name can contain any characters you like. You are not restricted to valid Python names. Note: When you name your URL patterns, make sure you use names that are unlikely to clash with any other application’s choice of names. If you call your URL pattern comment, and another application does the same thing, there’s no guarantee which URL will be inserted into your template when you use this name. 168 Chapter 3. Using Django
- 173. Django Documentation, Release1.5.1 Putting a prefix on your URL names, perhaps derived from the application name, will decrease the chances of collision. We recommend something like myapp-comment instead of comment. URL namespaces Introduction When you need to deploy multiple instances of a single application, it can be helpful to be able to differentiate between instances. This is especially important when using named URL patterns, since multiple instances of a single application will share named URLs. Namespaces provide a way to tell these named URLs apart. A URL namespace comes in two parts, both of which are strings: application namespace This describes the name of the application that is being deployed. Every instance of a single application will have the same application namespace. For example, Django’s admin application has the somewhat predictable application namespace of ’admin’. instance namespace This identifies a specific instance of an application. Instance namespaces should be unique across your entire project. However, an instance namespace can be the same as the application namespace. This is used to specify a default instance of an application. For example, the default Django Admin instance has an instance namespace of ’admin’. Namespaced URLs are specified using the ’:’ operator. For example, the main index page of the admin application is referenced using ’admin:index’. This indicates a namespace of ’admin’, and a named URL of ’index’. Namespaces can also be nested. The named URL ’foo:bar:whiz’ would look for a pattern named ’whiz’ in the namespace ’bar’ that is itself defined within the top-level namespace ’foo’. Reversing namespaced URLs When given a namespaced URL (e.g. ’myapp:index’) to resolve, Django splits the fully qualified name into parts, and then tries the following lookup: 1. First, Django looks for a matching application namespace (in this example, ’myapp’). This will yield a list of instances of that application. 2. If there is a current application defined, Django finds and returns the URL resolver for that instance. The current application can be specified as an attribute on the template context - applications that expect to have multiple deployments should set the current_app attribute on any Context or RequestContext that is used to render a template. The current application can also be specified manually as an argument to the django.core.urlresolvers.reverse() function. 3. If there is no current application. Django looks for a default application instance. The default application instance is the instance that has an instance namespace matching the application namespace (in this example, an instance of the myapp called ’myapp’). 4. If there is no default application instance, Django will pick the last deployed instance of the application, whatever its instance name may be. 5. If the provided namespace doesn’t match an application namespace in step 1, Django will attempt a direct lookup of the namespace as an instance namespace. If there are nested namespaces, these steps are repeated for each part of the namespace until only the view name is unresolved. The view name will then be resolved into a URL in the namespace that has been found. 3.3. Handling HTTP requests 169
- 174. Django Documentation, Release1.5.1 Example To show this resolution strategy in action, consider an example of two instances of myapp: one called ’foo’, and one called ’bar’. myapp has a main index page with a URL named ’index’. Using this setup, the following lookups are possible: • If one of the instances is current - say, if we were rendering a utility page in the instance ’bar’ - ’myapp:index’ will resolve to the index page of the instance ’bar’. • If there is no current instance - say, if we were rendering a page somewhere else on the site - ’myapp:index’ will resolve to the last registered instance of myapp. Since there is no default instance, the last instance of myapp that is registered will be used. This could be ’foo’ or ’bar’, depending on the order they are introduced into the urlpatterns of the project. • ’foo:index’ will always resolve to the index page of the instance ’foo’. If there was also a default instance - i.e., an instance named ’myapp’ - the following would happen: • If one of the instances is current - say, if we were rendering a utility page in the instance ’bar’ - ’myapp:index’ will resolve to the index page of the instance ’bar’. • If there is no current instance - say, if we were rendering a page somewhere else on the site - ’myapp:index’ will resolve to the index page of the default instance. • ’foo:index’ will again resolve to the index page of the instance ’foo’. URL namespaces and included URLconfs URL namespaces of included URLconfs can be specified in two ways. Firstly, you can provide the application and instance namespaces as arguments to django.conf.urls.include() when you construct your URL patterns. For example,: url(r’^help/’, include(’apps.help.urls’, namespace=’foo’, app_name=’bar’)), This will include the URLs defined in apps.help.urls into the application namespace ’bar’, with the instance namespace ’foo’. Secondly, you can include an object that contains embedded namespace data. If you include() an object as returned by patterns(), the URLs contained in that object will be added to the global namespace. However, you can also include() a 3-tuple containing: (<patterns object>, <application namespace>, <instance namespace>) For example: help_patterns = patterns(’’, url(r’^basic/$’, ’apps.help.views.views.basic’), url(r’^advanced/$’, ’apps.help.views.views.advanced’), ) url(r’^help/’, include(help_patterns, ’bar’, ’foo’)), This will include the nominated URL patterns into the given application and instance namespace. For example, the Django Admin is deployed as instances of AdminSite. AdminSite objects have a urls attribute: A 3-tuple that contains all the patterns in the corresponding admin site, plus the application namespace ’admin’, and the name of the admin instance. It is this urls attribute that you include() into your projects urlpatterns when you deploy an Admin instance. 170 Chapter 3. Using Django
- 175. Django Documentation, Release1.5.1 3.3.2 Writing views A view function, or view for short, is simply a Python function that takes a Web request and returns a Web response. This response can be the HTML contents of a Web page, or a redirect, or a 404 error, or an XML document, or an image . . . or anything, really. The view itself contains whatever arbitrary logic is necessary to return that response. This code can live anywhere you want, as long as it’s on your Python path. There’s no other requirement–no “magic,” so to speak. For the sake of putting the code somewhere, the convention is to put views in a file called views.py, placed in your project or application directory. A simple view Here’s a view that returns the current date and time, as an HTML document: from django.http import HttpResponse import datetime def current_datetime(request): now = datetime.datetime.now() html = "<html><body>It is now %s.</body></html>" % now return HttpResponse(html) Let’s step through this code one line at a time: • First, we import the class HttpResponse from the django.http module, along with Python’s datetime library. • Next, we define a function called current_datetime. This is the view function. Each view function takes an HttpRequest object as its first parameter, which is typically named request. Note that the name of the view function doesn’t matter; it doesn’t have to be named in a certain way in order for Django to recognize it. We’re calling it current_datetime here, because that name clearly indicates what it does. • The view returns an HttpResponse object that contains the generated response. Each view function is re-sponsible for returning an HttpResponse object. (There are exceptions, but we’ll get to those later.) Django’s Time Zone Django includes a TIME_ZONE setting that defaults to America/Chicago. This probably isn’t where you live, so you might want to change it in your settings file. Mapping URLs to views So, to recap, this view function returns an HTML page that includes the current date and time. To display this view at a particular URL, you’ll need to create a URLconf ; see URL dispatcher for instructions. Returning errors Returning HTTP error codes in Django is easy. There are subclasses of HttpResponse for a number of common HTTP status codes other than 200 (which means “OK”). You can find the full list of available subclasses in the re-quest/ response documentation. Just return an instance of one of those subclasses instead of a normal HttpResponse in order to signify an error. For example: 3.3. Handling HTTP requests 171
- 176. Django Documentation, Release1.5.1 def my_view(request): # ... if foo: return HttpResponseNotFound(’<h1>Page not found</h1>’) else: return HttpResponse(’<h1>Page was found</h1>’) There isn’t a specialized subclass for every possible HTTP response code, since many of them aren’t going to be that common. However, as documented in the HttpResponse documentation, you can also pass the HTTP status code into the constructor for HttpResponse to create a return class for any status code you like. For example: def my_view(request): # ... # Return a "created" (201) response code. return HttpResponse(status=201) Because 404 errors are by far the most common HTTP error, there’s an easier way to handle those errors. The Http404 exception class django.http.Http404 When you return an error such as HttpResponseNotFound, you’re responsible for defining the HTML of the resulting error page: return HttpResponseNotFound(’<h1>Page not found</h1>’) For convenience, and because it’s a good idea to have a consistent 404 error page across your site, Django provides an Http404 exception. If you raise Http404 at any point in a view function, Django will catch it and return the standard error page for your application, along with an HTTP error code 404. Example usage: from django.http import Http404 def detail(request, poll_id): try: p = Poll.objects.get(pk=poll_id) except Poll.DoesNotExist: raise Http404 return render_to_response(’polls/detail.html’, {’poll’: p}) In order to use the Http404 exception to its fullest, you should create a template that is displayed when a 404 error is raised. This template should be called 404.html and located in the top level of your template tree. Customizing error views The 404 (page not found) view django.views.defaults.page_not_found(request, template_name=‘404.html’) When you raise an Http404 exception, Django loads a special view devoted to handling 404 errors. By default, it’s the view django.views.defaults.page_not_found, which either produces a very simple “Not Found” message or loads and renders the template 404.html if you created it in your root template directory. 172 Chapter 3. Using Django
- 177. Django Documentation, Release1.5.1 The default 404 view will pass one variable to the template: request_path, which is the URL that resulted in the error. The page_not_found view should suffice for 99% of Web applications, but if you want to override it, you can specify handler404 in your URLconf, like so: handler404 = ’mysite.views.my_custom_404_view’ Behind the scenes, Django determines the 404 view by looking for handler404 in your root URLconf, and falling back to django.views.defaults.page_not_found if you did not define one. Three things to note about 404 views: • The 404 view is also called if Django doesn’t find a match after checking every regular expression in the URLconf. • The 404 view is passed a RequestContext and will have access to variables supplied by your TEMPLATE_CONTEXT_PROCESSORS setting (e.g., MEDIA_URL). • If DEBUG is set to True (in your settings module), then your 404 view will never be used, and your URLconf will be displayed instead, with some debug information. The 500 (server error) view Similarly, Django executes special-case behavior in the case of runtime errors in view code. If a view results in an exception, Django will, by default, call the view django.views.defaults.server_error, which either produces a very simple “Server Error” message or loads and renders the template 500.html if you created it in your root template directory. The default 500 view passes no variables to the 500.html template and is rendered with an empty Context to lessen the chance of additional errors. This server_error view should suffice for 99% ofWeb applications, but if you want to override the view, you can specify handler500 in your URLconf, like so: handler500 = ’mysite.views.my_custom_error_view’ Behind the scenes, Django determines the 500 view by looking for handler500 in your root URLconf, and falling back to django.views.defaults.server_error if you did not define one. One thing to note about 500 views: • If DEBUG is set to True (in your settings module), then your 500 view will never be used, and the traceback will be displayed instead, with some debug information. The 403 (HTTP Forbidden) view New in version 1.4. In the same vein as the 404 and 500 views, Django has a view to handle 403 Forbidden errors. If a view results in a 403 exception then Django will, by default, call the view django.views.defaults.permission_denied. This view loads and renders the template 403.html in your root template directory, or if this file does not exist, instead serves the text “403 Forbidden”, as per RFC 2616 (the HTTP 1.1 Specification). django.views.defaults.permission_denied is triggered by a PermissionDenied exception. To deny access in a view you can use code like this: 3.3. Handling HTTP requests 173
- 178. Django Documentation, Release1.5.1 from django.core.exceptions import PermissionDenied def edit(request, pk): if not request.user.is_staff: raise PermissionDenied # ... It is possible to override django.views.defaults.permission_denied in the same way you can for the 404 and 500 views by specifying a handler403 in your URLconf: handler403 = ’mysite.views.my_custom_permission_denied_view’ 3.3.3 View decorators Django provides several decorators that can be applied to views to support various HTTP features. Allowed HTTP methods The decorators in django.views.decorators.http can be used to restrict access to views based on the request method. These decorators will return a django.http.HttpResponseNotAllowed if the conditions are not met. require_http_methods(request_method_list) Decorator to require that a view only accept particular request methods. Usage: from django.views.decorators.http import require_http_methods @require_http_methods(["GET", "POST"]) def my_view(request): # I can assume now that only GET or POST requests make it this far # ... pass Note that request methods should be in uppercase. require_GET() Decorator to require that a view only accept the GET method. require_POST() Decorator to require that a view only accept the POST method. require_safe() New in version 1.4. Decorator to require that a view only accept the GET and HEAD methods. These methods are commonly considered “safe” because they should not have the significance of taking an action other than retrieving the requested resource. Note: Django will automatically strip the content of responses to HEAD requests while leaving the headers un-changed, so you may handle HEAD requests exactly like GET requests in your views. Since some software, such as link checkers, rely on HEAD requests, you might prefer using require_safe instead of require_GET. Conditional view processing The following decorators in django.views.decorators.http can be used to control caching behavior on particular views. 174 Chapter 3. Using Django
- 179. Django Documentation, Release1.5.1 condition(etag_func=None, last_modified_func=None) etag(etag_func) last_modified(last_modified_func) These decorators can be used to generate ETag and Last-Modified headers; see conditional view process-ing. GZip compression The decorators in django.views.decorators.gzip control content compression on a per-view basis. gzip_page() This decorator compresses content if the browser allows gzip compression. It sets the Vary header accordingly, so that caches will base their storage on the Accept-Encoding header. Vary headers The decorators in django.views.decorators.vary can be used to control caching based on specific request headers. vary_on_cookie(func) vary_on_headers(*headers) The Vary header defines which request headers a cache mechanism should take into account when building its cache key. See using vary headers. 3.3.4 File Uploads When Django handles a file upload, the file data ends up placed in request.FILES (for more on the request object see the documentation for request and response objects). This document explains how files are stored on disk and in memory, and how to customize the default behavior. Basic file uploads Consider a simple form containing a FileField: from django import forms class UploadFileForm(forms.Form): title = forms.CharField(max_length=50) file = forms.FileField() A view handling this form will receive the file data in request.FILES, which is a dictionary containing a key for each FileField (or ImageField, or other FileField subclass) in the form. So the data from the above form would be accessible as request.FILES[’file’]. Note that request.FILES will only contain data if the request method was POST and the <form> that posted the request has the attribute enctype="multipart/form-data". Otherwise, request.FILES will be empty. Most of the time, you’ll simply pass the file data from request into the form as described in Binding uploaded files to a form. This would look something like: 3.3. Handling HTTP requests 175
- 180. Django Documentation, Release1.5.1 from django.http import HttpResponseRedirect from django.shortcuts import render_to_response # Imaginary function to handle an uploaded file. from somewhere import handle_uploaded_file def upload_file(request): if request.method == ’POST’: form = UploadFileForm(request.POST, request.FILES) if form.is_valid(): handle_uploaded_file(request.FILES[’file’]) return HttpResponseRedirect(’/success/url/’) else: form = UploadFileForm() return render_to_response(’upload.html’, {’form’: form}) Notice that we have to pass request.FILES into the form’s constructor; this is how file data gets bound into a form. Handling uploaded files class UploadedFile The final piece of the puzzle is handling the actual file data from request.FILES. Each entry in this dictio-nary is an UploadedFile object – a simple wrapper around an uploaded file. You’ll usually use one of these methods to access the uploaded content: read() Read the entire uploaded data from the file. Be careful with this method: if the uploaded file is huge it can overwhelm your system if you try to read it into memory. You’ll probably want to use chunks() instead; see below. multiple_chunks() Returns True if the uploaded file is big enough to require reading in multiple chunks. By default this will be any file larger than 2.5 megabytes, but that’s configurable; see below. chunks() A generator returning chunks of the file. If multiple_chunks() is True, you should use this method in a loop instead of read(). In practice, it’s often easiest simply to use chunks() all the time; see the example below. name The name of the uploaded file (e.g. my_file.txt). size The size, in bytes, of the uploaded file. There are a few other methods and attributes available on UploadedFile objects; see UploadedFile objects for a complete reference. Putting it all together, here’s a common way you might handle an uploaded file: def handle_uploaded_file(f): with open(’some/file/name.txt’, ’wb+’) as destination: for chunk in f.chunks(): destination.write(chunk) Looping over UploadedFile.chunks() instead of using read() ensures that large files don’t overwhelm your system’s memory. 176 Chapter 3. Using Django
- 181. Django Documentation, Release1.5.1 Where uploaded data is stored Before you save uploaded files, the data needs to be stored somewhere. By default, if an uploaded file is smaller than 2.5 megabytes, Django will hold the entire contents of the upload in memory. This means that saving the file involves only a read from memory and a write to disk and thus is very fast. However, if an uploaded file is too large, Django will write the uploaded file to a temporary file stored in your system’s temporary directory. On a Unix-like platform this means you can expect Django to generate a file called something like /tmp/tmpzfp6I6.upload. If an upload is large enough, you can watch this file grow in size as Django streams the data onto disk. These specifics – 2.5 megabytes; /tmp; etc. – are simply “reasonable defaults”. Read on for details on how you can customize or completely replace upload behavior. Changing upload handler behavior Three settings control Django’s file upload behavior: FILE_UPLOAD_MAX_MEMORY_SIZE The maximum size, in bytes, for files that will be uploaded into memory. Files larger than FILE_UPLOAD_MAX_MEMORY_SIZE will be streamed to disk. Defaults to 2.5 megabytes. FILE_UPLOAD_TEMP_DIR The directory where uploaded files larger than FILE_UPLOAD_MAX_MEMORY_SIZE will be stored. Defaults to your system’s standard temporary directory (i.e. /tmp on most Unix-like systems). FILE_UPLOAD_PERMISSIONS The numeric mode (i.e. 0644) to set newly uploaded files to. For more informa-tion about what these modes mean, see the documentation for os.chmod(). If this isn’t given or is None, you’ll get operating-system dependent behavior. On most platforms, temporary files will have a mode of 0600, and files saved from memory will be saved using the system’s standard umask. Warning: If you’re not familiar with file modes, please note that the leading 0 is very important: it indicates an octal number, which is the way that modes must be specified. If you try to use 644, you’ll get totally incorrect behavior. Always prefix the mode with a 0. FILE_UPLOAD_HANDLERS The actual handlers for uploaded files. Changing this setting allows complete cus-tomization – even replacement – of Django’s upload process. See upload handlers, below, for details. Defaults to: ("django.core.files.uploadhandler.MemoryFileUploadHandler", "django.core.files.uploadhandler.TemporaryFileUploadHandler",) Which means “try to upload to memory first, then fall back to temporary files.” Handling uploaded files with a model If you’re saving a file on a Model with a FileField, using a ModelForm makes this process much easier. The file object will be saved to the location specified by the upload_to argument of the corresponding FileField when calling form.save(): 3.3. Handling HTTP requests 177
- 182. Django Documentation, Release1.5.1 from django.http import HttpResponseRedirect from django.shortcuts import render from .forms import ModelFormWithFileField def upload_file(request): if request.method == ’POST’: form = ModelFormWithFileField(request.POST, request.FILES) if form.is_valid(): # file is saved form.save() return HttpResponseRedirect(’/success/url/’) else: form = ModelFormWithFileField() return render(request, ’upload.html’, {’form’: form}) If you are constructing an object manually, you can simply assign the file object from request.FILES to the file field in the model: from django.http import HttpResponseRedirect from django.shortcuts import render from .forms import UploadFileForm from .models import ModelWithFileField def upload_file(request): if request.method == ’POST’: form = UploadFileForm(request.POST, request.FILES) if form.is_valid(): instance = ModelWithFileField(file_field=request.FILES[’file’]) instance.save() return HttpResponseRedirect(’/success/url/’) else: form = UploadFileForm() return render(request, ’upload.html’, {’form’: form}) UploadedFile objects In addition to those inherited from File, all UploadedFile objects define the following methods/attributes: UploadedFile.content_type The content-type header uploaded with the file (e.g. text/plain or application/pdf). Like any data supplied by the user, you shouldn’t trust that the uploaded file is actually this type. You’ll still need to validate that the file contains the content that the content-type header claims – “trust but verify.” UploadedFile.charset For text/* content-types, the character set (i.e. utf8) supplied by the browser. Again, “trust but verify” is the best policy here. UploadedFile.temporary_file_path Only files uploaded onto disk will have this method; it returns the full path to the temporary uploaded file. Note: Like regular Python files, you can read the file line-by-line simply by iterating over the uploaded file: for line in uploadedfile: do_something_with(line) However, unlike standard Python files, UploadedFile only understands n (also known as “Unix-style”) line endings. If you know that you need to handle uploaded files with different line endings, you’ll need to do so in your 178 Chapter 3. Using Django
- 183. Django Documentation, Release1.5.1 view. Upload Handlers When a user uploads a file, Django passes off the file data to an upload handler – a small class that handles file data as it gets uploaded. Upload handlers are initially defined in the FILE_UPLOAD_HANDLERS setting, which defaults to: ("django.core.files.uploadhandler.MemoryFileUploadHandler", "django.core.files.uploadhandler.TemporaryFileUploadHandler",) Together the MemoryFileUploadHandler and TemporaryFileUploadHandler provide Django’s default file upload behavior of reading small files into memory and large ones onto disk. You can write custom handlers that customize how Django handles files. You could, for example, use custom handlers to enforce user-level quotas, compress data on the fly, render progress bars, and even send data to another storage location directly without storing it locally. Modifying upload handlers on the fly Sometimes particular views require different upload behavior. In these cases, you can override upload handlers on a per-request basis by modifying request.upload_handlers. By default, this list will contain the upload handlers given by FILE_UPLOAD_HANDLERS, but you can modify the list as you would any other list. For instance, suppose you’ve written a ProgressBarUploadHandler that provides feedback on upload progress to some sort of AJAX widget. You’d add this handler to your upload handlers like this: request.upload_handlers.insert(0, ProgressBarUploadHandler()) You’d probably want to use list.insert() in this case (instead of append()) because a progress bar handler would need to run before any other handlers. Remember, the upload handlers are processed in order. If you want to replace the upload handlers completely, you can just assign a new list: request.upload_handlers = [ProgressBarUploadHandler()] Note: You can only modify upload handlers before accessing request.POST or request.FILES – it doesn’t make sense to change upload handlers after upload handling has already started. If you try to modify request.upload_handlers after reading from request.POST or request.FILES Django will throw an error. Thus, you should always modify uploading handlers as early in your view as possible. Also, request.POST is accessed by CsrfViewMiddleware which is enabled by default. This means you will need to use csrf_exempt() on your view to allow you to change the upload handlers. You will then need to use csrf_protect() on the function that actually processes the request. Note that this means that the handlers may start receiving the file upload before the CSRF checks have been done. Example code: from django.views.decorators.csrf import csrf_exempt, csrf_protect @csrf_exempt def upload_file_view(request): request.upload_handlers.insert(0, ProgressBarUploadHandler()) return _upload_file_view(request) @csrf_protect 3.3. Handling HTTP requests 179
- 184. Django Documentation, Release1.5.1 def _upload_file_view(request): ... # Process request Writing custom upload handlers All file upload handlers should be subclasses of django.core.files.uploadhandler.FileUploadHandler. You can define upload handlers wherever you wish. Required methods Custom file upload handlers must define the following methods: FileUploadHandler.receive_data_chunk(self, raw_data, start) Receives a “chunk” of data from the file upload. raw_data is a byte string containing the uploaded data. start is the position in the file where this raw_data chunk begins. The data you return will get fed into the subsequent upload handlers’ receive_data_chunk methods. In this way, one handler can be a “filter” for other handlers. Return None from receive_data_chunk to sort-circuit remaining upload handlers from getting this chunk.. This is useful if you’re storing the uploaded data yourself and don’t want future handlers to store a copy of the data. If you raise a StopUpload or a SkipFile exception, the upload will abort or the file will be completely skipped. FileUploadHandler.file_complete(self, file_size) Called when a file has finished uploading. The handler should return an UploadedFile object that will be stored in request.FILES. Handlers may also return None to indicate that the UploadedFile object should come from subsequent upload handlers. Optional methods Custom upload handlers may also define any of the following optional methods or attributes: FileUploadHandler.chunk_size Size, in bytes, of the “chunks” Django should store into mem-ory and feed into the handler. That is, this attribute controls the size of chunks fed into FileUploadHandler.receive_data_chunk. For maximum performance the chunk sizes should be divisible by 4 and should not exceed 2 GB (231 bytes) in size. When there are multiple chunk sizes provided by multiple handlers, Django will use the smallest chunk size defined by any handler. The default is 64*210 bytes, or 64 KB. FileUploadHandler.new_file(self, field_name, file_name, content_type, content_length, charset) Callback signaling that a new file upload is starting. This is called before any data has been fed to any upload handlers. field_name is a string name of the file <input> field. file_name is the unicode filename that was provided by the browser. content_type is the MIME type provided by the browser – E.g. ’image/jpeg’. content_length is the length of the image given by the browser. Sometimes this won’t be provided and will be None. charset is the character set (i.e. utf8) given by the browser. Like content_length, this sometimes won’t be provided. 180 Chapter 3. Using Django
- 185. Django Documentation, Release1.5.1 This method may raise a StopFutureHandlers exception to prevent future handlers from handling this file. FileUploadHandler.upload_complete(self) Callback signaling that the entire upload (all files) has completed. FileUploadHandler.handle_raw_input(self, input_data, META, content_length, boundary, encoding) Allows the handler to completely override the parsing of the raw HTTP input. input_data is a file-like object that supports read()-ing. META is the same object as request.META. content_length is the length of the data in input_data. Don’t read more than content_length bytes from input_data. boundary is the MIME boundary for this request. encoding is the encoding of the request. Return None if you want upload handling to continue, or a tuple of (POST, FILES) if you want to return the new data structures suitable for the request directly. 3.3.5 Django shortcut functions The package django.shortcuts collects helper functions and classes that “span” multiple levels of MVC. In other words, these functions/classes introduce controlled coupling for convenience’s sake. render render(request, template_name[, dictionary][, context_instance][, content_type][, status][, current_app]) Combines a given template with a given context dictionary and returns an HttpResponse object with that rendered text. render() is the same as a call to render_to_response() with a context_instance argument that forces the use of a RequestContext. Required arguments request The request object used to generate this response. template_name The full name of a template to use or sequence of template names. Optional arguments dictionary A dictionary of values to add to the template context. By default, this is an empty dictionary. If a value in the dictionary is callable, the view will call it just before rendering the template. context_instance The context instance to render the template with. By default, the template will be rendered with a RequestContext instance (filled with values from request and dictionary). content_type The MIME type to use for the resulting document. Defaults to the value of the DEFAULT_CONTENT_TYPE setting. Changed in version 1.5: This parameter used to be called mimetype. status The status code for the response. Defaults to 200. current_app A hint indicating which application contains the current view. See the namespaced URL resolution strategy for more information. 3.3. Handling HTTP requests 181
- 186. Django Documentation, Release1.5.1 Example The following example renders the template myapp/index.html with the MIME type application/xhtml+xml: from django.shortcuts import render def my_view(request): # View code here... return render(request, ’myapp/index.html’, {"foo": "bar"}, content_type="application/xhtml+xml") This example is equivalent to: from django.http import HttpResponse from django.template import RequestContext, loader def my_view(request): # View code here... t = loader.get_template(’myapp/template.html’) c = RequestContext(request, {’foo’: ’bar’}) return HttpResponse(t.render(c), content_type="application/xhtml+xml") render_to_response render_to_response(template_name[, dictionary][, context_instance][, content_type]) Renders a given template with a given context dictionary and returns an HttpResponse object with that rendered text. Required arguments template_name The full name of a template to use or sequence of template names. If a sequence is given, the first template that exists will be used. See the template loader documentation for more information on how templates are found. Optional arguments dictionary A dictionary of values to add to the template context. By default, this is an empty dictionary. If a value in the dictionary is callable, the view will call it just before rendering the template. context_instance The context instance to render the template with. By default, the template will be rendered with a Context instance (filled with values from dictionary). If you need to use context processors, render the template with a RequestContext instance instead. Your code might look something like this: return render_to_response(’my_template.html’, my_data_dictionary, context_instance=RequestContext(request)) content_type The MIME type to use for the resulting document. Defaults to the value of the DEFAULT_CONTENT_TYPE setting. Changed in version 1.5: This parameter used to be called mimetype. 182 Chapter 3. Using Django
- 187. Django Documentation, Release1.5.1 Example The following example renders the template myapp/index.html with the MIME type application/xhtml+xml: from django.shortcuts import render_to_response def my_view(request): # View code here... return render_to_response(’myapp/index.html’, {"foo": "bar"}, mimetype="application/xhtml+xml") This example is equivalent to: from django.http import HttpResponse from django.template import Context, loader def my_view(request): # View code here... t = loader.get_template(’myapp/template.html’) c = Context({’foo’: ’bar’}) return HttpResponse(t.render(c), content_type="application/xhtml+xml") redirect redirect(to[, permanent=False ], *args, **kwargs) Returns an HttpResponseRedirect to the appropriate URL for the arguments passed. The arguments could be: •A model: the model’s get_absolute_url() function will be called. •A view name, possibly with arguments: urlresolvers.reverse will be used to reverse-resolve the name. •A URL, which will be used as-is for the redirect location. By default issues a temporary redirect; pass permanent=True to issue a permanent redirect Examples You can use the redirect() function in a number of ways. 1. By passing some object; that object’s get_absolute_url() method will be called to figure out the redirect URL: from django.shortcuts import redirect def my_view(request): ... object = MyModel.objects.get(...) return redirect(object) 2. By passing the name of a view and optionally some positional or keyword arguments; the URL will be reverse resolved using the reverse() method: 3.3. Handling HTTP requests 183
- 188. Django Documentation, Release1.5.1 def my_view(request): ... return redirect(’some-view-name’, foo=’bar’) 3. By passing a hardcoded URL to redirect to: def my_view(request): ... return redirect(’/some/url/’) This also works with full URLs: def my_view(request): ... return redirect(’http://example.com/’) By default, redirect() returns a temporary redirect. All of the above forms accept a permanent argument; if set to True a permanent redirect will be returned: def my_view(request): ... object = MyModel.objects.get(...) return redirect(object, permanent=True) get_object_or_404 get_object_or_404(klass, *args, **kwargs) Calls get() on a given model manager, but it raises Http404 instead of the model’s DoesNotExist ex-ception. Required arguments klass A Model, Manager or QuerySet instance from which to get the object. **kwargs Lookup parameters, which should be in the format accepted by get() and filter(). Example The following example gets the object with the primary key of 1 from MyModel: from django.shortcuts import get_object_or_404 def my_view(request): my_object = get_object_or_404(MyModel, pk=1) This example is equivalent to: from django.http import Http404 def my_view(request): try: my_object = MyModel.objects.get(pk=1) except MyModel.DoesNotExist: raise Http404 Note: As with get(), a MultipleObjectsReturned exception will be raised if more than one object is found. 184 Chapter 3. Using Django
- 189. Django Documentation, Release1.5.1 get_list_or_404 get_list_or_404(klass, *args, **kwargs) Returns the result of filter() on a given model manager, raising Http404 if the resulting list is empty. Required arguments klass A Model, Manager or QuerySet instance from which to get the list. **kwargs Lookup parameters, which should be in the format accepted by get() and filter(). Example The following example gets all published objects from MyModel: from django.shortcuts import get_list_or_404 def my_view(request): my_objects = get_list_or_404(MyModel, published=True) This example is equivalent to: from django.http import Http404 def my_view(request): my_objects = list(MyModel.objects.filter(published=True)) if not my_objects: raise Http404 3.3.6 Generic views See Class-based views. 3.3.7 Middleware Middleware is a framework of hooks into Django’s request/response processing. It’s a light, low-level “plugin” system for globally altering Django’s input and/or output. Each middleware component is responsible for doing some specific function. For example, Django includes a middle-ware component, XViewMiddleware, that adds an "X-View" HTTP header to every response to a HEAD request. This document explains how middleware works, how you activate middleware, and how to write your own middleware. Django ships with some built-in middleware you can use right out of the box; they’re documented in the built-in middleware reference. Activating middleware To activate a middleware component, add it to the MIDDLEWARE_CLASSES list in your Django settings. In MIDDLEWARE_CLASSES, each middleware component is represented by a string: the full Python path to the mid-dleware’s class name. For example, here’s the default MIDDLEWARE_CLASSES created by django-admin.py startproject: 3.3. Handling HTTP requests 185
- 190. Django Documentation, Release1.5.1 MIDDLEWARE_CLASSES = ( ’django.middleware.common.CommonMiddleware’, ’django.contrib.sessions.middleware.SessionMiddleware’, ’django.middleware.csrf.CsrfViewMiddleware’, ’django.contrib.auth.middleware.AuthenticationMiddleware’, ’django.contrib.messages.middleware.MessageMiddleware’, ) During the request phases (process_request() and process_view() middleware), Django applies middleware in the order it’s defined in MIDDLEWARE_CLASSES, top-down. During the response phases (process_response() and process_exception() middleware), the classes are applied in reverse order, from the bottom up. You can think of it like an onion: each middleware class is a “layer” that wraps the view: A Django installation doesn’t require any middleware – e.g., MIDDLEWARE_CLASSES can be empty, if you’d like – but it’s strongly suggested that you at least use CommonMiddleware. Writing your own middleware Writing your own middleware is easy. Each middleware component is a single Python class that defines one or more of the following methods: 186 Chapter 3. Using Django
- 191. Django Documentation, Release1.5.1 process_request process_request(self, request) request is an HttpRequest object. This method is called on each request, before Django decides which view to execute. process_request() should return either None or an HttpResponse object. If it returns None, Django will continue processing this request, executing any other middleware and, then, the appropriate view. If it returns an HttpResponse object, Django won’t bother calling ANY other request, view or exception middleware, or the appropriate view; it’ll return that HttpResponse. Response middleware is always called on every response. process_view process_view(self, request, view_func, view_args, view_kwargs) request is an HttpRequest object. view_func is the Python function that Django is about to use. (It’s the actual function object, not the name of the function as a string.) view_args is a list of positional arguments that will be passed to the view, and view_kwargs is a dictionary of keyword arguments that will be passed to the view. Neither view_args nor view_kwargs include the first view argument (request). process_view() is called just before Django calls the view. It should return either None or an HttpResponse object. If it returns None, Django will continue processing this request, executing any other process_view() middleware and, then, the appropriate view. If it returns an HttpResponse object, Django won’t bother calling ANY other request, view or exception middleware, or the appropriate view; it’ll return that HttpResponse. Response middleware is always called on every response. Note: Accessing request.POST or request.REQUEST inside middleware from process_request or process_view will prevent any view running after the middleware from being able to modify the upload handlers for the request, and should normally be avoided. The CsrfViewMiddleware class can be considered an exception, as it provides the csrf_exempt() and csrf_protect() decorators which allow views to explicitly control at what point the CSRF validation should occur. process_template_response process_template_response(self, request, response) request is an HttpRequest object. response is a subclass of SimpleTemplateResponse (e.g. TemplateResponse) or any response object that implements a render method. process_template_response() must return a response object that implements a render method. It could alter the given response by changing response.template_name and response.context_data, or it could create and return a brand-new SimpleTemplateResponse or equivalent. process_template_response() will only be called if the response instance has a render() method, indi-cating that it is a TemplateResponse or equivalent. You don’t need to explicitly render responses – responses will be automatically rendered once all template response middleware has been called. Middleware are run in reverse order during the response phase, which includes process_template_response. 3.3. Handling HTTP requests 187
- 192. Django Documentation, Release1.5.1 process_response process_response(self, request, response) request is an HttpRequest object. response is the HttpResponse object returned by a Django view. process_response() must return an HttpResponse object. It could alter the given response, or it could create and return a brand-new HttpResponse. Unlike the process_request() and process_view() methods, the process_response() method is always called, even if the process_request() and process_view() methods of the same middleware class were skipped because an earlier middleware method returned an HttpResponse (this means that your process_response() method cannot rely on setup done in process_request(), for example). In addi-tion, during the response phase the classes are applied in reverse order, from the bottom up. This means classes defined at the end of MIDDLEWARE_CLASSES will be run first. Changed in version 1.5: response may also be an StreamingHttpResponse object. Unlike HttpResponse, StreamingHttpResponse does not have a content attribute. As a result, middleware can no longer assume that all responses will have a content attribute. If they need access to the content, they must test for streaming responses and adjust their behavior accordingly: if response.streaming: response.streaming_content = wrap_streaming_content(response.streaming_content) else: response.content = wrap_content(response.content) streaming_content should be assumed to be too large to hold in memory. Middleware may wrap it in a new generator, but must not consume it. process_exception process_exception(self, request, exception) request is an HttpRequest object. exception is an Exception object raised by the view function. Django calls process_exception() when a view raises an exception. process_exception() should return either None or an HttpResponse object. If it returns an HttpResponse object, the response will be returned to the browser. Otherwise, default exception handling kicks in. Again, middleware are run in reverse order during the response phase, which includes process_exception. If an exception middleware returns a response, the middleware classes above that middleware will not be called at all. __init__ Most middleware classes won’t need an initializer since middleware classes are essentially placeholders for the process_* methods. If you do need some global state you may use __init__ to set up. However, keep in mind a couple of caveats: • Django initializes your middleware without any arguments, so you can’t define __init__ as requiring any arguments. • Unlike the process_* methods which get called once per request, __init__ gets called only once, when the Web server responds to the first request. Marking middleware as unused It’s sometimes useful to determine at run-time whether a piece of middleware should be used. In these cases, your middleware’s __init__ method may raise django.core.exceptions.MiddlewareNotUsed. Django will then remove that piece of middleware from the middleware process. 188 Chapter 3. Using Django
- 193. Django Documentation, Release1.5.1 Guidelines • Middleware classes don’t have to subclass anything. • The middleware class can live anywhere on your Python path. All Django cares about is that the MIDDLEWARE_CLASSES setting includes the path to it. • Feel free to look at Django’s available middleware for examples. • If you write a middleware component that you think would be useful to other people, contribute to the commu-nity! Let us know, and we’ll consider adding it to Django. 3.3.8 How to use sessions Django provides full support for anonymous sessions. The session framework lets you store and retrieve arbitrary data on a per-site-visitor basis. It stores data on the server side and abstracts the sending and receiving of cookies. Cookies contain a session ID – not the data itself (unless you’re using the cookie based backend). Enabling sessions Sessions are implemented via a piece of middleware. To enable session functionality, do the following: • Edit the MIDDLEWARE_CLASSES setting and make sure it contains ’django.contrib.sessions.middleware.SessionMiddleware’. The default settings.py created by django-admin.py startproject has SessionMiddleware activated. If you don’t want to use sessions, you might as well remove the SessionMiddleware line from MIDDLEWARE_CLASSES and ’django.contrib.sessions’ from your INSTALLED_APPS. It’ll save you a small bit of overhead. Configuring the session engine By default, Django stores sessions in your database (using the model django.contrib.sessions.models.Session). Though this is convenient, in some setups it’s faster to store session data elsewhere, so Django can be configured to store session data on your filesystem or in your cache. Using database-backed sessions If you want to use a database-backed session, you need to add ’django.contrib.sessions’ to your INSTALLED_APPS setting. Once you have configured your installation, run manage.py syncdb to install the single database table that stores session data. Using cached sessions For better performance, you may want to use a cache-based session backend. To store session data using Django’s cache system, you’ll first need to make sure you’ve configured your cache; see the cache documentation for details. 3.3. Handling HTTP requests 189
- 194. Django Documentation, Release1.5.1 Warning: You should only use cache-based sessions if you’re using the Memcached cache backend. The local-memory cache backend doesn’t retain data long enough to be a good choice, and it’ll be faster to use file or database sessions directly instead of sending everything through the file or database cache backends. If you have multiple caches defined in CACHES, Django will use the default cache. To use another cache, set SESSION_CACHE_ALIAS to the name of that cache. Changed in version 1.5: The SESSION_CACHE_ALIAS setting was added. Once your cache is configured, you’ve got two choices for how to store data in the cache: • Set SESSION_ENGINE to "django.contrib.sessions.backends.cache" for a simple caching session store. Session data will be stored directly your cache. However, session data may not be persistent: cached data can be evicted if the cache fills up or if the cache server is restarted. • For persistent, cached data, set SESSION_ENGINE to "django.contrib.sessions.backends.cached_db". This uses a write-through cache – every write to the cache will also be written to the database. Session reads only use the database if the data is not already in the cache. Both session stores are quite fast, but the simple cache is faster because it disregards persistence. In most cases, the cached_db backend will be fast enough, but if you need that last bit of performance, and are willing to let session data be expunged from time to time, the cache backend is for you. If you use the cached_db session backend, you also need to follow the configuration instructions for the using database-backed sessions. Using file-based sessions To use file-based sessions, set the SESSION_ENGINE setting to "django.contrib.sessions.backends.file". You might also want to set the SESSION_FILE_PATH setting (which defaults to output from tempfile.gettempdir(), most likely /tmp) to control where Django stores session files. Be sure to check that your Web server has permissions to read and write to this location. Using cookie-based sessions New in version 1.4. To use cookies-based sessions, set the SESSION_ENGINE setting to "django.contrib.sessions.backends.signed_cookies". The session data will be stored using Django’s tools for cryptographic signing and the SECRET_KEY setting. Note: It’s recommended to leave the SESSION_COOKIE_HTTPONLY setting True to prevent tampering of the stored data from JavaScript. 190 Chapter 3. Using Django
- 195. Django Documentation, Release1.5.1 Warning: The session data is signed but not encrypted When using the cookies backend the session data can be read by the client. A MAC (Message Authentication Code) is used to protect the data against changes by the client, so that the session data will be invalidated when being tampered with. The same invalidation happens if the client storing the cookie (e.g. your user’s browser) can’t store all of the session cookie and drops data. Even though Django compresses the data, it’s still entirely possible to exceed the common limit of 4096 bytes per cookie. No freshness guarantee Note also that while the MAC can guarantee the authenticity of the data (that it was generated by your site, and not someone else), and the integrity of the data (that it is all there and correct), it cannot guarantee freshness i.e. that you are being sent back the last thing you sent to the client. This means that for some uses of session data, the cookie backend might open you up to replay attacks. Cookies will only be detected as ‘stale’ if they are older than your SESSION_COOKIE_AGE. Performance Finally, the size of a cookie can have an impact on the speed of your site. Using sessions in views When SessionMiddleware is activated, each HttpRequest object – the first argument to any Django view function – will have a session attribute, which is a dictionary-like object. You can read it and write to request.session at any point in your view. You can edit it multiple times. class backends.base.SessionBase This is the base class for all session objects. It has the following standard dictionary methods: __getitem__(key) Example: fav_color = request.session[’fav_color’] __setitem__(key, value) Example: request.session[’fav_color’] = ’blue’ __delitem__(key) Example: del request.session[’fav_color’]. This raises KeyError if the given key isn’t already in the session. __contains__(key) Example: ’fav_color’ in request.session get(key, default=None) Example: fav_color = request.session.get(’fav_color’, ’red’) pop(key) Example: fav_color = request.session.pop(’fav_color’) keys() items() setdefault() clear() It also has these methods: flush() Delete the current session data from the session and regenerate the session key value that is sent back to the user in the cookie. This is used if you want to ensure that the previous session data can’t be accessed again from the user’s browser (for example, the django.contrib.auth.logout() function calls it). 3.3. Handling HTTP requests 191
- 196. Django Documentation, Release1.5.1 set_test_cookie() Sets a test cookie to determine whether the user’s browser supports cookies. Due to the way cookies work, you won’t be able to test this until the user’s next page request. See Setting test cookies below for more information. test_cookie_worked() Returns either True or False, depending on whether the user’s browser accepted the test cookie. Due to the way cookies work, you’ll have to call set_test_cookie() on a previous, separate page request. See Setting test cookies below for more information. delete_test_cookie() Deletes the test cookie. Use this to clean up after yourself. set_expiry(value) Sets the expiration time for the session. You can pass a number of different values: •If value is an integer, the session will expire after that many seconds of inactivity. For example, calling request.session.set_expiry(300) would make the session expire in 5 minutes. •If value is a datetime or timedelta object, the session will expire at that specific date/time. •If value is 0, the user’s session cookie will expire when the user’s Web browser is closed. •If value is None, the session reverts to using the global session expiry policy. Reading a session is not considered activity for expiration purposes. Session expiration is computed from the last time the session was modified. get_expiry_age() Returns the number of seconds until this session expires. For sessions with no custom expiration (or those set to expire at browser close), this will equal SESSION_COOKIE_AGE. This function accepts two optional keyword arguments: •modification: last modification of the session, as a datetime object. Defaults to the current time. •expiry: expiry information for the session, as a datetime object, an int() (in seconds), or None. Defaults to the value stored in the session by set_expiry(), if there is one, or None. get_expiry_date() Returns the date this session will expire. For sessions with no custom expiration (or those set to expire at browser close), this will equal the date SESSION_COOKIE_AGE seconds from now. This function accepts the same keyword argumets as get_expiry_age(). get_expire_at_browser_close() Returns either True or False, depending on whether the user’s session cookie will expire when the user’s Web browser is closed. SessionBase.clear_expired() New in version 1.5. Removes expired sessions from the session store. This class method is called by clearsessions. Session object guidelines • Use normal Python strings as dictionary keys on request.session. This is more of a convention than a hard-and-fast rule. • Session dictionary keys that begin with an underscore are reserved for internal use by Django. 192 Chapter 3. Using Django
- 197. Django Documentation, Release1.5.1 • Don’t override request.session with a new object, and don’t access or set its attributes. Use it like a Python dictionary. Examples This simplistic view sets a has_commented variable to True after a user posts a comment. It doesn’t let a user post a comment more than once: def post_comment(request, new_comment): if request.session.get(’has_commented’, False): return HttpResponse("You’ve already commented.") c = comments.Comment(comment=new_comment) c.save() request.session[’has_commented’] = True return HttpResponse(’Thanks for your comment!’) This simplistic view logs in a “member” of the site: def login(request): m = Member.objects.get(username=request.POST[’username’]) if m.password == request.POST[’password’]: request.session[’member_id’] = m.id return HttpResponse("You’re logged in.") else: return HttpResponse("Your username and password didn’t match.") ...And this one logs a member out, according to login() above: def logout(request): try: del request.session[’member_id’] except KeyError: pass return HttpResponse("You’re logged out.") The standard django.contrib.auth.logout() function actually does a bit more than this to prevent inadver-tent data leakage. It calls the flush() method of request.session. We are using this example as a demonstra-tion of how to work with session objects, not as a full logout() implementation. Setting test cookies As a convenience, Django provides an easy way to test whether the user’s browser accepts cookies. Just call the set_test_cookie() method of request.session in a view, and call test_cookie_worked() in a sub-sequent view – not in the same view call. This awkward split between set_test_cookie() and test_cookie_worked() is necessary due to the way cookies work. When you set a cookie, you can’t actually tell whether a browser accepted it until the browser’s next request. It’s good practice to use delete_test_cookie() to clean up after yourself. Do this after you’ve verified that the test cookie worked. Here’s a typical usage example: def login(request): if request.method == ’POST’: if request.session.test_cookie_worked(): 3.3. Handling HTTP requests 193
- 198. Django Documentation, Release1.5.1 request.session.delete_test_cookie() return HttpResponse("You’re logged in.") else: return HttpResponse("Please enable cookies and try again.") request.session.set_test_cookie() return render_to_response(’foo/login_form.html’) Using sessions out of views An API is available to manipulate session data outside of a view: >>> from django.contrib.sessions.backends.db import SessionStore >>> import datetime >>> s = SessionStore() >>> s[’last_login’] = datetime.datetime(2005, 8, 20, 13, 35, 10) >>> s.save() >>> s.session_key ’2b1189a188b44ad18c35e113ac6ceead’ >>> s = SessionStore(session_key=’2b1189a188b44ad18c35e113ac6ceead’) >>> s[’last_login’] datetime.datetime(2005, 8, 20, 13, 35, 0) In order to prevent session fixation attacks, sessions keys that don’t exist are regenerated: >>> from django.contrib.sessions.backends.db import SessionStore >>> s = SessionStore(session_key=’no-such-session-here’) >>> s.save() >>> s.session_key ’ff882814010ccbc3c870523934fee5a2’ If you’re using the django.contrib.sessions.backends.db backend, each session is just a normal Django model. The Session model is defined in django/contrib/sessions/models.py. Because it’s a normal model, you can access sessions using the normal Django database API: >>> from django.contrib.sessions.models import Session >>> s = Session.objects.get(pk=’2b1189a188b44ad18c35e113ac6ceead’) >>> s.expire_date datetime.datetime(2005, 8, 20, 13, 35, 12) Note that you’ll need to call get_decoded() to get the session dictionary. This is necessary because the dictionary is stored in an encoded format: >>> s.session_data ’KGRwMQpTJ19hdXRoX3VzZXJfaWQnCnAyCkkxCnMuMTExY2ZjODI2Yj...’ >>> s.get_decoded() {’user_id’: 42} When sessions are saved By default, Django only saves to the session database when the session has been modified – that is if any of its dictionary values have been assigned or deleted: # Session is modified. request.session[’foo’] = ’bar’ 194 Chapter 3. Using Django
- 199. Django Documentation, Release1.5.1 # Session is modified. del request.session[’foo’] # Session is modified. request.session[’foo’] = {} # Gotcha: Session is NOT modified, because this alters # request.session[’foo’] instead of request.session. request.session[’foo’][’bar’] = ’baz’ In the last case of the above example, we can tell the session object explicitly that it has been modified by setting the modified attribute on the session object: request.session.modified = True To change this default behavior, set the SESSION_SAVE_EVERY_REQUEST setting to True. When set to True, Django will save the session to the database on every single request. Note that the session cookie is only sent when a session has been created or modified. If SESSION_SAVE_EVERY_REQUEST is True, the session cookie will be sent on every request. Similarly, the expires part of a session cookie is updated each time the session cookie is sent. Changed in version 1.5: The session is not saved if the response’s status code is 500. Browser-length sessions vs. persistent sessions You can control whether the session framework uses browser-length sessions vs. persistent sessions with the SESSION_EXPIRE_AT_BROWSER_CLOSE setting. By default, SESSION_EXPIRE_AT_BROWSER_CLOSE is set to False, which means session cookies will be stored in users’ browsers for as long as SESSION_COOKIE_AGE. Use this if you don’t want people to have to log in every time they open a browser. If SESSION_EXPIRE_AT_BROWSER_CLOSE is set to True, Django will use browser-length cookies – cookies that expire as soon as the user closes his or her browser. Use this if you want people to have to log in every time they open a browser. This setting is a global default and can be overwritten at a per-session level by explicitly calling the set_expiry() method of request.session as described above in using sessions in views. Note: Some browsers (Chrome, for example) provide settings that allow users to continue brows-ing sessions after closing and re-opening the browser. In some cases, this can interfere with the SESSION_EXPIRE_AT_BROWSER_CLOSE setting and prevent sessions from expiring on browser close. Please be aware of this while testing Django applications which have the SESSION_EXPIRE_AT_BROWSER_CLOSE set-ting enabled. Clearing the session store As users create new sessions on your website, session data can accumulate in your session store. If you’re using the database backend, the django_session database table will grow. If you’re using the file backend, your temporary directory will contain an increasing number of files. To understand this problem, consider what happens with the database backend. When a user logs in, Django adds a row to the django_session database table. Django updates this row each time the session data changes. If the user logs out manually, Django deletes the row. But if the user does not log out, the row never gets deleted. A similar process happens with the file backend. 3.3. Handling HTTP requests 195
- 200. Django Documentation, Release1.5.1 Django does not provide automatic purging of expired sessions. Therefore, it’s your job to purge expired sessions on a regular basis. Django provides a clean-up management command for this purpose: clearsessions. It’s recommended to call this command on a regular basis, for example as a daily cron job. Note that the cache backend isn’t vulnerable to this problem, because caches automatically delete stale data. Neither is the cookie backend, because the session data is stored by the users’ browsers. Settings A few Django settings give you control over session behavior: SESSION_ENGINE Default: django.contrib.sessions.backends.db Controls where Django stores session data. Valid values are: • ’django.contrib.sessions.backends.db’ • ’django.contrib.sessions.backends.file’ • ’django.contrib.sessions.backends.cache’ • ’django.contrib.sessions.backends.cached_db’ • ’django.contrib.sessions.backends.signed_cookies’ See configuring the session engine for more details. SESSION_FILE_PATH Default: /tmp/ If you’re using file-based session storage, this sets the directory in which Django will store session data. SESSION_COOKIE_AGE Default: 1209600 (2 weeks, in seconds) The age of session cookies, in seconds. SESSION_COOKIE_DOMAIN Default: None The domain to use for session cookies. Set this to a string such as ".example.com" (note the leading dot!) for cross-domain cookies, or use None for a standard domain cookie. SESSION_COOKIE_HTTPONLY Default: True Whether to use HTTPOnly flag on the session cookie. If this is set to True, client-side JavaScript will not to be able to access the session cookie. 196 Chapter 3. Using Django
- 201. Django Documentation, Release1.5.1 HTTPOnly is a flag included in a Set-Cookie HTTP response header. It is not part of the RFC 2109 standard for cookies, and it isn’t honored consistently by all browsers. However, when it is honored, it can be a useful way to mitigate the risk of client side script accessing the protected cookie data. Changed in version 1.4: The default value of the setting was changed from False to True. SESSION_COOKIE_NAME Default: ’sessionid’ The name of the cookie to use for sessions. This can be whatever you want. SESSION_COOKIE_PATH Default: ’/’ The path set on the session cookie. This should either match the URL path of your Django installation or be parent of that path. This is useful if you have multiple Django instances running under the same hostname. They can use different cookie paths, and each instance will only see its own session cookie. SESSION_COOKIE_SECURE Default: False Whether to use a secure cookie for the session cookie. If this is set to True, the cookie will be marked as “secure,” which means browsers may ensure that the cookie is only sent under an HTTPS connection. SESSION_EXPIRE_AT_BROWSER_CLOSE Default: False Whether to expire the session when the user closes his or her browser. See “Browser-length sessions vs. persistent sessions” above. SESSION_SAVE_EVERY_REQUEST Default: False Whether to save the session data on every request. If this is False (default), then the session data will only be saved if it has been modified – that is, if any of its dictionary values have been assigned or deleted. Technical details • The session dictionary should accept any pickleable Python object. See the pickle module for more informa-tion. • Session data is stored in a database table named django_session . • Django only sends a cookie if it needs to. If you don’t set any session data, it won’t send a session cookie. 3.3. Handling HTTP requests 197
- 202. Django Documentation, Release1.5.1 Session IDs in URLs The Django sessions framework is entirely, and solely, cookie-based. It does not fall back to putting session IDs in URLs as a last resort, as PHP does. This is an intentional design decision. Not only does that behavior make URLs ugly, it makes your site vulnerable to session-ID theft via the “Referer” header. 3.4 Working with forms About this document This document provides an introduction to Django’s form handling features. For a more detailed look at specific areas of the forms API, see The Forms API, Form fields, and Form and field validation. django.forms is Django’s form-handling library. While it is possible to process form submissions just using Django’s HttpRequest class, using the form library takes care of a number of common form-related tasks. Using it, you can: 1. Display an HTML form with automatically generated form widgets. 2. Check submitted data against a set of validation rules. 3. Redisplay a form in the case of validation errors. 4. Convert submitted form data to the relevant Python data types. 3.4.1 Overview The library deals with these concepts: Widget A class that corresponds to an HTML form widget, e.g. <input type="text"> or <textarea>. This handles rendering of the widget as HTML. Field A class that is responsible for doing validation, e.g. an EmailField that makes sure its data is a valid email address. Form A collection of fields that knows how to validate itself and display itself as HTML. Form Media The CSS and JavaScript resources that are required to render a form. The library is decoupled from the other Django components, such as the database layer, views and templates. It relies only on Django settings, a couple of django.utils helper functions and Django’s internationalization hooks (but you’re not required to be using internationalization features to use this library). 3.4.2 Form objects A Form object encapsulates a sequence of form fields and a collection of validation rules that must be fulfilled in order for the form to be accepted. Form classes are created as subclasses of django.forms.Form and make use of a declarative style that you’ll be familiar with if you’ve used Django’s database models. For example, consider a form used to implement “contact me” functionality on a personal Web site: from django import forms class ContactForm(forms.Form): subject = forms.CharField(max_length=100) 198 Chapter 3. Using Django
- 203. Django Documentation, Release1.5.1 message = forms.CharField() sender = forms.EmailField() cc_myself = forms.BooleanField(required=False) A form is composed of Field objects. In this case, our form has four fields: subject, message, sender and cc_myself. CharField, EmailField and BooleanField are just three of the available field types; a full list can be found in Form fields. If your form is going to be used to directly add or edit a Django model, you can use a ModelForm to avoid duplicating your model description. Using a form in a view The standard pattern for processing a form in a view looks like this: from django.shortcuts import render from django.http import HttpResponseRedirect def contact(request): if request.method == ’POST’: # If the form has been submitted... form = ContactForm(request.POST) # A form bound to the POST data if form.is_valid(): # All validation rules pass # Process the data in form.cleaned_data # ... return HttpResponseRedirect(’/thanks/’) # Redirect after POST else: form = ContactForm() # An unbound form return render(request, ’contact.html’, { ’form’: form, }) There are three possible code paths here: Form submitted? Data? What occurs Unsubmitted None yet Template gets passed unbound instance of ContactForm. Submitted Invalid data Template gets passed bound instance of ContactForm. Submitted Valid data Valid data is processed. Redirect to a “thanks” page. The distinction between Bound and unbound forms is important: • An unbound form has no data associated with it. When rendered to the user, it will be empty or will contain default values. • A bound form has submitted data, and hence can be used to tell if that data is valid. If an invalid bound form is rendered, it can include inline error messages telling the user what data to correct. Handling file uploads with a form To see how to handle file uploads with your form, see Binding uploaded files to a form. Processing the data from a form Once is_valid() returns True, the successfully validated form data will be in the form.cleaned_data dic-tionary. This data will have been converted nicely into Python types for you. 3.4. Working with forms 199
- 204. Django Documentation, Release1.5.1 Note: You can still access the unvalidated data directly from request.POST at this point, but the validated data is better. In the above example, cc_myself will be a boolean value. Likewise, fields such as IntegerField and FloatField convert values to a Python int and float respectively. Read-only fields are not available in form.cleaned_data (and setting a value in a custom clean() method won’t have any effect). These fields are displayed as text rather than as input elements, and thus are not posted back to the server. Extending the earlier example, here’s how the form data could be processed: if form.is_valid(): subject = form.cleaned_data[’subject’] message = form.cleaned_data[’message’] sender = form.cleaned_data[’sender’] cc_myself = form.cleaned_data[’cc_myself’] recipients = [’info@example.com’] if cc_myself: recipients.append(sender) from django.core.mail import send_mail send_mail(subject, message, sender, recipients) return HttpResponseRedirect(’/thanks/’) # Redirect after POST Tip: For more on sending email from Django, see Sending email. Displaying a form using a template Forms are designed to work with the Django template language. In the above example, we passed our ContactForm instance to the template using the context variable form. Here’s a simple example template: <form action="/contact/" method="post">{% csrf_token %} {{ form.as_p }} <input type="submit" value="Submit" /> </form> The form only outputs its own fields; it is up to you to provide the surrounding <form> tags and the submit button. If your form includes uploaded files, be sure to include enctype="multipart/form-data" in the form el-ement. If you wish to write a generic template that will work whether or not the form has files, you can use the is_multipart() attribute on the form: <form action="/contact/" method="post" {% if form.is_multipart %}enctype="multipart/form-data"{% endif %}> Forms and Cross Site Request Forgery protection Django ships with an easy-to-use protection against Cross Site Request Forgeries. When submitting a form via POST with CSRF protection enabled you must use the csrf_token template tag as in the preceding example. However, since CSRF protection is not directly tied to forms in templates, this tag is omitted from the following examples in this document. form.as_p will output the form with each form field and accompanying label wrapped in a paragraph. Here’s the output for our example template: 200 Chapter 3. Using Django
- 205. Django Documentation, Release1.5.1 <form action="/contact/" method="post"> <p><label for="id_subject">Subject:</label> <input id="id_subject" type="text" name="subject" maxlength="100" /></p> <p><label for="id_message">Message:</label> <input type="text" name="message" id="id_message" /></p> <p><label for="id_sender">Sender:</label> <input type="text" name="sender" id="id_sender" /></p> <p><label for="id_cc_myself">Cc myself:</label> <input type="checkbox" name="cc_myself" id="id_cc_myself" /></p> <input type="submit" value="Submit" /> </form> Note that each form field has an ID attribute set to id_<field-name>, which is referenced by the accompanying label tag. This is important for ensuring forms are accessible to assistive technology such as screen reader software. You can also customize the way in which labels and ids are generated. You can also use form.as_table to output table rows (you’ll need to provide your own <table> tags) and form.as_ul to output list items. Customizing the form template If the default generated HTML is not to your taste, you can completely customize the way a form is presented using the Django template language. Extending the above example: <form action="/contact/" method="post"> {{ form.non_field_errors }} <div class="fieldWrapper"> {{ form.subject.errors }} <label for="id_subject">Email subject:</label> {{ form.subject }} </div> <div class="fieldWrapper"> {{ form.message.errors }} <label for="id_message">Your message:</label> {{ form.message }} </div> <div class="fieldWrapper"> {{ form.sender.errors }} <label for="id_sender">Your email address:</label> {{ form.sender }} </div> <div class="fieldWrapper"> {{ form.cc_myself.errors }} <label for="id_cc_myself">CC yourself?</label> {{ form.cc_myself }} </div> <p><input type="submit" value="Send message" /></p> </form> Each named form-field can be output to the template using {{ form.name_of_field }}, which will produce the HTML needed to display the form widget. Using {{ form.name_of_field.errors }} displays a list of form errors, rendered as an unordered list. This might look like: <ul class="errorlist"> <li>Sender is required.</li> </ul> 3.4. Working with forms 201
- 206. Django Documentation, Release1.5.1 The list has a CSS class of errorlist to allow you to style its appearance. If you wish to further customize the display of errors you can do so by looping over them: {% if form.subject.errors %} <ol> {% for error in form.subject.errors %} <li><strong>{{ error|escape }}</strong></li> {% endfor %} </ol> {% endif %} Looping over the form’s fields If you’re using the same HTML for each of your form fields, you can reduce duplicate code by looping through each field in turn using a {% for %} loop: <form action="/contact/" method="post"> {% for field in form %} <div class="fieldWrapper"> {{ field.errors }} {{ field.label_tag }}: {{ field }} </div> {% endfor %} <p><input type="submit" value="Send message" /></p> </form> Within this loop, {{ field }} is an instance of BoundField. BoundField also has the following attributes, which can be useful in your templates: {{ field.label }} The label of the field, e.g. Email address. {{ field.label_tag }} The field’s label wrapped in the appropriate HTML <label> tag, e.g. <label for="id_email">Email address</label> {{ field.value }} The value of the field. e.g someone@example.com {{ field.html_name }} The name of the field that will be used in the input element’s name field. This takes the form prefix into account, if it has been set. {{ field.help_text }} Any help text that has been associated with the field. {{ field.errors }} Outputs a <ul class="errorlist"> containing any validation errors corre-sponding to this field. You can customize the presentation of the errors with a {% for error in field.errors %} loop. In this case, each object in the loop is a simple string containing the error message. {{ field.is_hidden }} This attribute is True if the form field is a hidden field and False otherwise. It’s not particularly useful as a template variable, but could be useful in conditional tests such as: {% if field.is_hidden %} {# Do something special #} {% endif %} {{ field.field }} The Field instance from the form class that this BoundField wraps. You can use it to access Field attributes , e.g. {{ char_field.field.max_length }}. Looping over hidden and visible fields If you’re manually laying out a form in a template, as opposed to relying on Django’s default form layout, you might want to treat <input type="hidden"> fields differently than non-hidden fields. For example, because hidden 202 Chapter 3. Using Django
- 207. Django Documentation, Release1.5.1 fields don’t display anything, putting error messages “next to” the field could cause confusion for your users – so errors for those fields should be handled differently. Django provides two methods on a form that allow you to loop over the hidden and visible fields independently: hidden_fields() and visible_fields(). Here’s a modification of an earlier example that uses these two methods: <form action="/contact/" method="post"> {# Include the hidden fields #} {% for hidden in form.hidden_fields %} {{ hidden }} {% endfor %} {# Include the visible fields #} {% for field in form.visible_fields %} <div class="fieldWrapper"> {{ field.errors }} {{ field.label_tag }}: {{ field }} </div> {% endfor %} <p><input type="submit" value="Send message" /></p> </form> This example does not handle any errors in the hidden fields. Usually, an error in a hidden field is a sign of form tampering, since normal form interaction won’t alter them. However, you could easily insert some error displays for those form errors, as well. Reusable form templates If your site uses the same rendering logic for forms in multiple places, you can reduce duplication by saving the form’s loop in a standalone template and using the include tag to reuse it in other templates: <form action="/contact/" method="post"> {% include "form_snippet.html" %} <p><input type="submit" value="Send message" /></p> </form> # In form_snippet.html: {% for field in form %} <div class="fieldWrapper"> {{ field.errors }} {{ field.label_tag }}: {{ field }} </div> {% endfor %} If the form object passed to a template has a different name within the context, you can alias it using the with argument of the include tag: <form action="/comments/add/" method="post"> {% include "form_snippet.html" with form=comment_form %} <p><input type="submit" value="Submit comment" /></p> </form> If you find yourself doing this often, you might consider creating a custom inclusion tag. 3.4. Working with forms 203
- 208. Django Documentation, Release1.5.1 3.4.3 Further topics This covers the basics, but forms can do a whole lot more: Formsets class django.forms.formset.BaseFormSet A formset is a layer of abstraction to work with multiple forms on the same page. It can be best compared to a data grid. Let’s say you have the following form: >>> from django import forms >>> class ArticleForm(forms.Form): ... title = forms.CharField() ... pub_date = forms.DateField() You might want to allow the user to create several articles at once. To create a formset out of an ArticleForm you would do: >>> from django.forms.formsets import formset_factory >>> ArticleFormSet = formset_factory(ArticleForm) You now have created a formset named ArticleFormSet. The formset gives you the ability to iterate over the forms in the formset and display them as you would with a regular form: >>> formset = ArticleFormSet() >>> for form in formset: ... print(form.as_table()) <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_As you can see it only displayed one empty form. The number of empty forms that is displayed is controlled by the extra parameter. By default, formset_factory defines one extra form; the following example will display two blank forms: >>> ArticleFormSet = formset_factory(ArticleForm, extra=2) Iterating over the formset will render the forms in the order they were created. You can change this order by providing an alternate implementation for the __iter__() method. Formsets can also be indexed into, which returns the corresponding form. If you override __iter__, you will need to also override __getitem__ to have matching behavior. Using initial data with a formset Initial data is what drives the main usability of a formset. As shown above you can define the number of extra forms. What this means is that you are telling the formset how many additional forms to show in addition to the number of forms it generates from the initial data. Lets take a look at an example: >>> ArticleFormSet = formset_factory(ArticleForm, extra=2) >>> formset = ArticleFormSet(initial=[ ... {’title’: u’Django is now open source’, ... ’pub_date’: datetime.date.today(),} ... ]) >>> for form in formset: ... print(form.as_table()) 204 Chapter 3. Using Django
- 209. Django Documentation, Release1.5.1 <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" id="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_<tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_There are now a total of three forms showing above. One for the initial data that was passed in and two extra forms. Also note that we are passing in a list of dictionaries as the initial data. See Also: Creating formsets from models with model formsets. Limiting the maximum number of forms The max_num parameter to formset_factory gives you the ability to limit the maximum number of empty forms the formset will display: >>> ArticleFormSet = formset_factory(ArticleForm, extra=2, max_num=1) >>> formset = ArticleFormset() >>> for form in formset: ... print(form.as_table()) <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_If the value of max_num is greater than the number of existing objects, up to extra additional blank forms will be added to the formset, so long as the total number of forms does not exceed max_num. A max_num value of None (the default) puts a high limit on the number of forms displayed (1000). In practice this is equivalent to no limit. Formset validation Validation with a formset is almost identical to a regular Form. There is an is_valid method on the formset to provide a convenient way to validate all forms in the formset: >>> ArticleFormSet = formset_factory(ArticleForm) >>> data = { ... ’form-TOTAL_FORMS’: u’1’, ... ’form-INITIAL_FORMS’: u’0’, ... ’form-MAX_NUM_FORMS’: u’’, ... } >>> formset = ArticleFormSet(data) >>> formset.is_valid() True We passed in no data to the formset which is resulting in a valid form. The formset is smart enough to ignore extra forms that were not changed. If we provide an invalid article: >>> data = { ... ’form-TOTAL_FORMS’: u’2’, ... ’form-INITIAL_FORMS’: u’0’, ... ’form-MAX_NUM_FORMS’: u’’, ... ’form-0-title’: u’Test’, ... ’form-0-pub_date’: u’1904-06-16’, ... ’form-1-title’: u’Test’, 3.4. Working with forms 205
- 210. Django Documentation, Release1.5.1 ... ’form-1-pub_date’: u’’, # <-- this date is missing but required ... } >>> formset = ArticleFormSet(data) >>> formset.is_valid() False >>> formset.errors [{}, {’pub_date’: [u’This field is required.’]}] As we can see, formset.errors is a list whose entries correspond to the forms in the formset. Validation was performed for each of the two forms, and the expected error message appears for the second item. New in version 1.4. We can also check if form data differs from the initial data (i.e. the form was sent without any data): >>> data = { ... ’form-TOTAL_FORMS’: u’1’, ... ’form-INITIAL_FORMS’: u’0’, ... ’form-MAX_NUM_FORMS’: u’’, ... ’form-0-title’: u’’, ... ’form-0-pub_date’: u’’, ... } >>> formset = ArticleFormSet(data) >>> formset.has_changed() False Understanding the ManagementForm You may have noticed the additional data (form-TOTAL_FORMS, form-INITIAL_FORMS and form-MAX_NUM_FORMS) that was required in the formset’s data above. This data is required for the ManagementForm. This form is used by the formset to manage the collection of forms contained in the formset. If you don’t provide this management data, an exception will be raised: >>> data = { ... ’form-0-title’: u’Test’, ... ’form-0-pub_date’: u’’, ... } >>> formset = ArticleFormSet(data) Traceback (most recent call last): ... django.forms.util.ValidationError: [u’ManagementForm data is missing or has been tampered with’] It is used to keep track of how many form instances are being displayed. If you are adding new forms via JavaScript, you should increment the count fields in this form as well. On the other hand, if you are using JavaScript to allow deletion of existing objects, then you need to ensure the ones being removed are properly marked for deletion by including form-#-DELETE in the POST data. It is expected that all forms are present in the POST data regardless. The management form is available as an attribute of the formset itself. When rendering a formset in a template, you can include all the management data by rendering {{ my_formset.management_form }} (substituting the name of your formset as appropriate). total_form_count and initial_form_count BaseFormSet has a couple of methods that are closely related to the ManagementForm, total_form_count and initial_form_count. total_form_count returns the total number of forms in this formset. initial_form_count returns the number of forms in the formset that were pre-filled, and is also used to determine how many forms are required. You will probably never need to override either of these methods, so please be sure you understand what they do before doing so. empty_form BaseFormSet provides an additional attribute empty_form which returns a form instance with a prefix of __prefix__ for easier use in dynamic forms with JavaScript. 206 Chapter 3. Using Django
- 211. Django Documentation, Release1.5.1 Custom formset validation A formset has a clean method similar to the one on a Form class. This is where you define your own validation that works at the formset level: >>> from django.forms.formsets import BaseFormSet >>> class BaseArticleFormSet(BaseFormSet): ... def clean(self): ... """Checks that no two articles have the same title.""" ... if any(self.errors): ... # Don’t bother validating the formset unless each form is valid on its own ... return ... titles = [] ... for i in range(0, self.total_form_count()): ... form = self.forms[i] ... title = form.cleaned_data[’title’] ... if title in titles: ... raise forms.ValidationError("Articles in a set must have distinct titles.") ... titles.append(title) >>> ArticleFormSet = formset_factory(ArticleForm, formset=BaseArticleFormSet) >>> data = { ... ’form-TOTAL_FORMS’: u’2’, ... ’form-INITIAL_FORMS’: u’0’, ... ’form-MAX_NUM_FORMS’: u’’, ... ’form-0-title’: u’Test’, ... ’form-0-pub_date’: u’1904-06-16’, ... ’form-1-title’: u’Test’, ... ’form-1-pub_date’: u’1912-06-23’, ... } >>> formset = ArticleFormSet(data) >>> formset.is_valid() False >>> formset.errors [{}, {}] >>> formset.non_form_errors() [u’Articles in a set must have distinct titles.’] The formset clean method is called after all the Form.clean methods have been called. The errors will be found using the non_form_errors() method on the formset. Dealing with ordering and deletion of forms Common use cases with a formset is dealing with ordering and deletion of the form instances. This has been dealt with for you. The formset_factory provides two optional parameters can_order and can_delete that will do the extra work of adding the extra fields and providing simpler ways of getting to that data. can_order Default: False Lets you create a formset with the ability to order: >>> ArticleFormSet = formset_factory(ArticleForm, can_order=True) >>> formset = ArticleFormSet(initial=[ ... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)}, ... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)}, ... ]) >>> for form in formset: ... print(form.as_table()) 3.4. Working with forms 207
- 212. Django Documentation, Release1.5.1 <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-ORDER">Order:</label></th><td><input type="text" name="form-0-ORDER" value="<tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" value="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_<tr><th><label for="id_form-1-ORDER">Order:</label></th><td><input type="text" name="form-1-ORDER" value="<tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_<tr><th><label for="id_form-2-ORDER">Order:</label></th><td><input type="text" name="form-2-ORDER" id="This adds an additional field to each form. This new field is named ORDER and is an forms.IntegerField. For the forms that came from the initial data it automatically assigned them a numeric value. Let’s look at what will happen when the user changes these values: >>> data = { ... ’form-TOTAL_FORMS’: u’3’, ... ’form-INITIAL_FORMS’: u’2’, ... ’form-MAX_NUM_FORMS’: u’’, ... ’form-0-title’: u’Article #1’, ... ’form-0-pub_date’: u’2008-05-10’, ... ’form-0-ORDER’: u’2’, ... ’form-1-title’: u’Article #2’, ... ’form-1-pub_date’: u’2008-05-11’, ... ’form-1-ORDER’: u’1’, ... ’form-2-title’: u’Article #3’, ... ’form-2-pub_date’: u’2008-05-01’, ... ’form-2-ORDER’: u’0’, ... } >>> formset = ArticleFormSet(data, initial=[ ... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)}, ... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)}, ... ]) >>> formset.is_valid() True >>> for form in formset.ordered_forms: ... print(form.cleaned_data) {’pub_date’: datetime.date(2008, 5, 1), ’ORDER’: 0, ’title’: u’Article #3’} {’pub_date’: datetime.date(2008, 5, 11), ’ORDER’: 1, ’title’: u’Article #2’} {’pub_date’: datetime.date(2008, 5, 10), ’ORDER’: 2, ’title’: u’Article #1’} can_delete Default: False Lets you create a formset with the ability to delete: >>> ArticleFormSet = formset_factory(ArticleForm, can_delete=True) >>> formset = ArticleFormSet(initial=[ ... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)}, ... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)}, ... ]) >>> for form in formset: .... print(form.as_table()) <input type="hidden" name="form-TOTAL_FORMS" value="3" id="id_form-TOTAL_FORMS" /><input type="hidden" <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" value="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-DELETE">Delete:</label></th><td><input type="checkbox" name="form-0-DELETE" <tr><th><label for="id_form-1-title">Title:</label></th><td><input type="text" name="form-1-title" value="<tr><th><label for="id_form-1-pub_date">Pub date:</label></th><td><input type="text" name="form-1-pub_208 Chapter 3. Using Django
- 213. Django Documentation, Release1.5.1 <tr><th><label for="id_form-1-DELETE">Delete:</label></th><td><input type="checkbox" name="form-1-DELETE" <tr><th><label for="id_form-2-title">Title:</label></th><td><input type="text" name="form-2-title" id="<tr><th><label for="id_form-2-pub_date">Pub date:</label></th><td><input type="text" name="form-2-pub_<tr><th><label for="id_form-2-DELETE">Delete:</label></th><td><input type="checkbox" name="form-2-DELETE" Similar to can_order this adds a new field to each form named DELETE and is a forms.BooleanField. When data comes through marking any of the delete fields you can access them with deleted_forms: >>> data = { ... ’form-TOTAL_FORMS’: u’3’, ... ’form-INITIAL_FORMS’: u’2’, ... ’form-MAX_NUM_FORMS’: u’’, ... ’form-0-title’: u’Article #1’, ... ’form-0-pub_date’: u’2008-05-10’, ... ’form-0-DELETE’: u’on’, ... ’form-1-title’: u’Article #2’, ... ’form-1-pub_date’: u’2008-05-11’, ... ’form-1-DELETE’: u’’, ... ’form-2-title’: u’’, ... ’form-2-pub_date’: u’’, ... ’form-2-DELETE’: u’’, ... } >>> formset = ArticleFormSet(data, initial=[ ... {’title’: u’Article #1’, ’pub_date’: datetime.date(2008, 5, 10)}, ... {’title’: u’Article #2’, ’pub_date’: datetime.date(2008, 5, 11)}, ... ]) >>> [form.cleaned_data for form in formset.deleted_forms] [{’DELETE’: True, ’pub_date’: datetime.date(2008, 5, 10), ’title’: u’Article #1’}] Adding additional fields to a formset If you need to add additional fields to the formset this can be easily accomplished. The formset base class provides an add_fields method. You can simply override this method to add your own fields or even redefine the default fields/attributes of the order and deletion fields: >>> class BaseArticleFormSet(BaseFormSet): ... def add_fields(self, form, index): ... super(BaseArticleFormSet, self).add_fields(form, index) ... form.fields["my_field"] = forms.CharField() >>> ArticleFormSet = formset_factory(ArticleForm, formset=BaseArticleFormSet) >>> formset = ArticleFormSet() >>> for form in formset: ... print(form.as_table()) <tr><th><label for="id_form-0-title">Title:</label></th><td><input type="text" name="form-0-title" id="<tr><th><label for="id_form-0-pub_date">Pub date:</label></th><td><input type="text" name="form-0-pub_<tr><th><label for="id_form-0-my_field">My field:</label></th><td><input type="text" name="form-0-my_Using a formset in views and templates Using a formset inside a view is as easy as using a regular Form class. The only thing you will want to be aware of is making sure to use the management form inside the template. Let’s look at a sample view: 3.4. Working with forms 209
- 214. Django Documentation, Release1.5.1 def manage_articles(request): ArticleFormSet = formset_factory(ArticleForm) if request.method == ’POST’: formset = ArticleFormSet(request.POST, request.FILES) if formset.is_valid(): # do something with the formset.cleaned_data pass else: formset = ArticleFormSet() return render_to_response(’manage_articles.html’, {’formset’: formset}) The manage_articles.html template might look like this: <form method="post" action=""> {{ formset.management_form }} <table> {% for form in formset %} {{ form }} {% endfor %} </table> </form> However the above can be slightly shortcutted and let the formset itself deal with the management form: <form method="post" action=""> <table> {{ formset }} </table> </form> The above ends up calling the as_table method on the formset class. Manually rendered can_delete and can_order If you manually render fields in the template, you can render can_delete parameter with {{ form.DELETE }}: <form method="post" action=""> {{ formset.management_form }} {% for form in formset %} {{ form.id }} <ul> <li>{{ form.title }}</li> {% if formset.can_delete %} <li>{{ form.DELETE }}</li> {% endif %} </ul> {% endfor %} </form> Similarly, if the formset has the ability to order (can_order=True), it is possible to render it with {{ form.ORDER }}. Using more than one formset in a view You are able to use more than one formset in a view if you like. Formsets borrow much of its behavior from forms. With that said you are able to use prefix to prefix formset form field names with a given value to allow more than one formset to be sent to a view without name clashing. Lets take a look at how this might be accomplished: 210 Chapter 3. Using Django
- 215. Django Documentation, Release1.5.1 def manage_articles(request): ArticleFormSet = formset_factory(ArticleForm) BookFormSet = formset_factory(BookForm) if request.method == ’POST’: article_formset = ArticleFormSet(request.POST, request.FILES, prefix=’articles’) book_formset = BookFormSet(request.POST, request.FILES, prefix=’books’) if article_formset.is_valid() and book_formset.is_valid(): # do something with the cleaned_data on the formsets. pass else: article_formset = ArticleFormSet(prefix=’articles’) book_formset = BookFormSet(prefix=’books’) return render_to_response(’manage_articles.html’, { ’article_formset’: article_formset, ’book_formset’: book_formset, }) You would then render the formsets as normal. It is important to point out that you need to pass prefix on both the POST and non-POST cases so that it is rendered and processed correctly. Creating forms from models ModelForm class ModelForm If you’re building a database-driven app, chances are you’ll have forms that map closely to Django models. For instance, you might have a BlogComment model, and you want to create a form that lets people submit comments. In this case, it would be redundant to define the field types in your form, because you’ve already defined the fields in your model. For this reason, Django provides a helper class that let you create a Form class from a Django model. For example: >>> from django.forms import ModelForm # Create the form class. >>> class ArticleForm(ModelForm): ... class Meta: ... model = Article # Creating a form to add an article. >>> form = ArticleForm() # Creating a form to change an existing article. >>> article = Article.objects.get(pk=1) >>> form = ArticleForm(instance=article) Field types The generated Form class will have a form field for every model field. Each model field has a corre-sponding default form field. For example, a CharField on a model is represented as a CharField on a form. A model ManyToManyField is represented as a MultipleChoiceField. Here is the full list of conversions: 3.4. Working with forms 211
- 216. Django Documentation, Release1.5.1 Model field Form field AutoField Not represented in the form BigIntegerField IntegerField with min_value set to -9223372036854775808 and max_value set to 9223372036854775807. BooleanField BooleanField CharField CharField with max_length set to the model field’s max_length CommaSeparatedIntegerCFhiaerlFdield DateField DateField DateTimeField DateTimeField DecimalField DecimalField EmailField EmailField FileField FileField FilePathField CharField FloatField FloatField ForeignKey ModelChoiceField (see below) ImageField ImageField IntegerField IntegerField IPAddressField IPAddressField GenericIPAddressFieldGenericIPAddressField ManyToManyField ModelMultipleChoiceField (see below) NullBooleanField CharField PositiveIntegerField IntegerField PositiveSmallIntegerFIinetledgerField SlugField SlugField SmallIntegerField IntegerField TextField CharField with widget=forms.Textarea TimeField TimeField URLField URLField As you might expect, the ForeignKey and ManyToManyField model field types are special cases: • ForeignKey is represented by django.forms.ModelChoiceField, which is a ChoiceField whose choices are a model QuerySet. • ManyToManyField is represented by django.forms.ModelMultipleChoiceField, which is a MultipleChoiceField whose choices are a model QuerySet. In addition, each generated form field has attributes set as follows: • If the model field has blank=True, then required is set to False on the form field. Otherwise, required=True. • The form field’s label is set to the verbose_name of the model field, with the first character capitalized. • The form field’s help_text is set to the help_text of the model field. • If the model field has choices set, then the form field’s widget will be set to Select, with choices coming from the model field’s choices. The choices will normally include the blank choice which is selected by default. If the field is required, this forces the user to make a selection. The blank choice will not be included if the model field has blank=False and an explicit default value (the default value will be initially selected instead). Finally, note that you can override the form field used for a given model field. See Overriding the default field types or widgets below. A full example Consider this set of models: 212 Chapter 3. Using Django
- 217. Django Documentation, Release1.5.1 from django.db import models from django.forms import ModelForm TITLE_CHOICES = ( (’MR’, ’Mr.’), (’MRS’, ’Mrs.’), (’MS’, ’Ms.’), ) class Author(models.Model): name = models.CharField(max_length=100) title = models.CharField(max_length=3, choices=TITLE_CHOICES) birth_date = models.DateField(blank=True, null=True) def __unicode__(self): return self.name class Book(models.Model): name = models.CharField(max_length=100) authors = models.ManyToManyField(Author) class AuthorForm(ModelForm): class Meta: model = Author class BookForm(ModelForm): class Meta: model = Book With these models, the ModelForm subclasses above would be roughly equivalent to this (the only difference being the save() method, which we’ll discuss in a moment.): from django import forms class AuthorForm(forms.Form): name = forms.CharField(max_length=100) title = forms.CharField(max_length=3, widget=forms.Select(choices=TITLE_CHOICES)) birth_date = forms.DateField(required=False) class BookForm(forms.Form): name = forms.CharField(max_length=100) authors = forms.ModelMultipleChoiceField(queryset=Author.objects.all()) The is_valid() method and errors The first time you call is_valid() or access the errors attribute of a ModelForm triggers form validation as well as model validation. This has the side-effect of cleaning the model you pass to the ModelForm constructor. For instance, calling is_valid() on your form will convert any date fields on your model to actual date objects. If form validation fails, only some of the updates may be applied. For this reason, you’ll probably want to avoid reusing the model instance passed to the form, especially if validation fails. The save() method Every form produced by ModelForm also has a save() method. This method creates and saves a database object from the data bound to the form. A subclass of ModelForm can accept an existing model instance as the keyword argument instance; if this is supplied, save() will update that instance. If it’s not supplied, save() will create a new instance of the specified model: 3.4. Working with forms 213
- 218. Django Documentation, Release1.5.1 # Create a form instance from POST data. >>> f = ArticleForm(request.POST) # Save a new Article object from the form’s data. >>> new_article = f.save() # Create a form to edit an existing Article, but use # POST data to populate the form. >>> a = Article.objects.get(pk=1) >>> f = ArticleForm(request.POST, instance=a) >>> f.save() Note that if the form hasn’t been validated, calling save() will do so by checking form.errors. A ValueError will be raised if the data in the form doesn’t validate – i.e., if form.errors evaluates to True. This save() method accepts an optional commit keyword argument, which accepts either True or False. If you call save() with commit=False, then it will return an object that hasn’t yet been saved to the database. In this case, it’s up to you to call save() on the resulting model instance. This is useful if you want to do custom processing on the object before saving it, or if you want to use one of the specialized model saving options. commit is True by default. Another side effect of using commit=False is seen when your model has a many-to-many relation with another model. If your model has a many-to-many relation and you specify commit=False when you save a form, Django cannot immediately save the form data for the many-to-many relation. This is because it isn’t possible to save many-to- many data for an instance until the instance exists in the database. To work around this problem, every time you save a form using commit=False, Django adds a save_m2m() method to your ModelForm subclass. After you’ve manually saved the instance produced by the form, you can invoke save_m2m() to save the many-to-many form data. For example: # Create a form instance with POST data. >>> f = AuthorForm(request.POST) # Create, but don’t save the new author instance. >>> new_author = f.save(commit=False) # Modify the author in some way. >>> new_author.some_field = ’some_value’ # Save the new instance. >>> new_author.save() # Now, save the many-to-many data for the form. >>> f.save_m2m() Calling save_m2m() is only required if you use save(commit=False). When you use a simple save() on a form, all data – including many-to-many data – is saved without the need for any additional method calls. For example: # Create a form instance with POST data. >>> a = Author() >>> f = AuthorForm(request.POST, instance=a) # Create and save the new author instance. There’s no need to do anything else. >>> new_author = f.save() Other than the save() and save_m2m() methods, a ModelForm works exactly the same way as any other forms form. For example, the is_valid() method is used to check for validity, the is_multipart() method is used to determine whether a form requires multipart file upload (and hence whether request.FILES must be passed to the form), etc. See Binding uploaded files to a form for more information. 214 Chapter 3. Using Django
- 219. Django Documentation, Release1.5.1 Using a subset of fields on the form In some cases, you may not want all the model fields to appear on the generated form. There are three ways of telling ModelForm to use only a subset of the model fields: 1. Set editable=False on the model field. As a result, any form created from the model via ModelForm will not include that field. 2. Use the fields attribute of the ModelForm‘s inner Meta class. This attribute, if given, should be a list of field names to include in the form. The order in which the fields names are specified in that list is respected when the form renders them. 3. Use the exclude attribute of the ModelForm‘s inner Meta class. This attribute, if given, should be a list of field names to exclude from the form. For example, if you want a form for the Author model (defined above) that includes only the name and birth_date fields, you would specify fields or exclude like this: class PartialAuthorForm(ModelForm): class Meta: model = Author fields = (’name’, ’birth_date’) class PartialAuthorForm(ModelForm): class Meta: model = Author exclude = (’title’,) Since the Author model has only 3 fields, ‘name’, ‘title’, and ‘birth_date’, the forms above will contain exactly the same fields. Note: If you specify fields or exclude when creating a form with ModelForm, then the fields that are not in the resulting form will not be set by the form’s save() method. Also, if you manually add the excluded fields back to the form, they will not be initialized from the model instance. Django will prevent any attempt to save an incomplete model, so if the model does not allow the missing fields to be empty, and does not provide a default value for the missing fields, any attempt to save() a ModelForm with missing fields will fail. To avoid this failure, you must instantiate your model with initial values for the missing, but required fields: author = Author(title=’Mr’) form = PartialAuthorForm(request.POST, instance=author) form.save() Alternatively, you can use save(commit=False) and manually set any extra required fields: form = PartialAuthorForm(request.POST) author = form.save(commit=False) author.title = ’Mr’ author.save() See the section on saving forms for more details on using save(commit=False). Overriding the default field types or widgets The default field types, as described in the Field types table above, are sensible defaults. If you have a DateField in your model, chances are you’d want that to be represented as a DateField in your form. But ModelForm gives you the flexibility of changing the form field type and widget for a given model field. To specify a custom widget for a field, use the widgets attribute of the inner Meta class. This should be a dictionary mapping field names to widget classes or instances. 3.4. Working with forms 215
- 220. Django Documentation, Release1.5.1 For example, if you want the a CharField for the name attribute of Author to be represented by a <textarea> instead of its default <input type="text">, you can override the field’s widget: from django.forms import ModelForm, Textarea class AuthorForm(ModelForm): class Meta: model = Author fields = (’name’, ’title’, ’birth_date’) widgets = { ’name’: Textarea(attrs={’cols’: 80, ’rows’: 20}), } The widgets dictionary accepts either widget instances (e.g., Textarea(...)) or classes (e.g., Textarea). If you want to further customize a field – including its type, label, etc. – you can do this by declaratively specifying fields like you would in a regular Form. Declared fields will override the default ones generated by using the model attribute. For example, if you wanted to use MyDateFormField for the pub_date field, you could do the following: class ArticleForm(ModelForm): pub_date = MyDateFormField() class Meta: model = Article If you want to override a field’s default label, then specify the label parameter when declaring the form field: >>> class ArticleForm(ModelForm): ... pub_date = DateField(label=’Publication date’) ... ... class Meta: ... model = Article Note: If you explicitly instantiate a form field like this, Django assumes that you want to completely define its behavior; therefore, default attributes (such as max_length or required) are not drawn from the corresponding model. If you want to maintain the behavior specified in the model, you must set the relevant arguments explicitly when declaring the form field. For example, if the Article model looks like this: class Article(models.Model): headline = models.CharField(max_length=200, null=True, blank=True, help_text="Use puns liberally") content = models.TextField() and you want to do some custom validation for headline, while keeping the blank and help_text values as specified, you might define ArticleForm like this: class ArticleForm(ModelForm): headline = MyFormField(max_length=200, required=False, help_text="Use puns liberally") class Meta: model = Article You must ensure that the type of the form field can be used to set the contents of the corresponding model field. When they are not compatible, you will get a ValueError as no implicit conversion takes place. See the form field documentation for more information on fields and their arguments. 216 Chapter 3. Using Django
- 221. Django Documentation, Release1.5.1 Changing the order of fields By default, a ModelForm will render fields in the same order that they are defined on the model, with ManyToManyField instances appearing last. If you want to change the order in which fields are rendered, you can use the fields attribute on the Meta class. The fields attribute defines the subset of model fields that will be rendered, and the order in which they will be rendered. For example given this model: class Book(models.Model): author = models.ForeignKey(Author) title = models.CharField(max_length=100) the author field would be rendered first. If we wanted the title field to be rendered first, we could specify the following ModelForm: >>> class BookForm(ModelForm): ... class Meta: ... model = Book ... fields = (’title’, ’author’) Overriding the clean() method You can override the clean() method on a model form to provide additional validation in the same way you can on a normal form. In this regard, model forms have two specific characteristics when compared to forms: By default the clean() method validates the uniqueness of fields that are marked as unique, unique_together or unique_for_date|month|year on the model. Therefore, if you would like to override the clean() method and maintain the default validation, you must call the parent class’s clean() method. Also, a model form instance bound to a model object will contain a self.instance attribute that gives model form methods access to that specific model instance. Form inheritance As with basic forms, you can extend and reuse ModelForms by inheriting them. This is useful if you need to declare extra fields or extra methods on a parent class for use in a number of forms derived from models. For example, using the previous ArticleForm class: >>> class EnhancedArticleForm(ArticleForm): ... def clean_pub_date(self): ... ... This creates a form that behaves identically to ArticleForm, except there’s some extra validation and cleaning for the pub_date field. You can also subclass the parent’s Meta inner class if you want to change the Meta.fields or Meta.excludes lists: >>> class RestrictedArticleForm(EnhancedArticleForm): ... class Meta(ArticleForm.Meta): ... exclude = (’body’,) This adds the extra method from the EnhancedArticleForm and modifies the original ArticleForm.Meta to remove one field. There are a couple of things to note, however. • Normal Python name resolution rules apply. If you have multiple base classes that declare a Meta inner class, only the first one will be used. This means the child’s Meta, if it exists, otherwise the Meta of the first parent, etc. 3.4. Working with forms 217
- 222. Django Documentation, Release1.5.1 • For technical reasons, a subclass cannot inherit from both a ModelForm and a Form simultaneously. Chances are these notes won’t affect you unless you’re trying to do something tricky with subclassing. Interaction with model validation As part of its validation process, ModelForm will call the clean() method of each field on your model that has a corresponding field on your form. If you have excluded any model fields, validation will not be run on those fields. See the form validation documentation for more on how field cleaning and validation work. Also, your model’s clean() method will be called before any uniqueness checks are made. See Validating objects for more information on the model’s clean() hook. ModelForm factory function You can create forms from a given model using the standalone function modelform_factory(), instead of using a class definition. This may be more convenient if you do not have many customizations to make: >>> from django.forms.models import modelform_factory >>> BookForm = modelform_factory(Book) This can also be used to make simple modifications to existing forms, for example by specifying which fields should be displayed: >>> Form = modelform_factory(Book, form=BookForm, fields=("author",)) ... or which fields should be excluded: >>> Form = modelform_factory(Book, form=BookForm, exclude=("title",)) You can also specify the widgets to be used for a given field: >>> from django.forms import Textarea >>> Form = modelform_factory(Book, form=BookForm, widgets={"title": Textarea()}) Model formsets class models.BaseModelFormSet Like regular formsets, Django provides a couple of enhanced formset classes that make it easy to work with Django models. Let’s reuse the Author model from above: >>> from django.forms.models import modelformset_factory >>> AuthorFormSet = modelformset_factory(Author) This will create a formset that is capable of working with the data associated with the Author model. It works just like a regular formset: >>> formset = AuthorFormSet() >>> print(formset) <input type="hidden" name="form-TOTAL_FORMS" value="1" id="id_form-TOTAL_FORMS" /><input type="hidden" <tr><th><label for="id_form-0-name">Name:</label></th><td><input id="id_form-0-name" type="text" name="<tr><th><label for="id_form-0-title">Title:</label></th><td><select name="form-0-title" id="id_form-0-<option value="" selected="selected">---------</option> <option value="MR">Mr.</option> <option value="MRS">Mrs.</option> <option value="MS">Ms.</option> </select></td></tr> <tr><th><label for="id_form-0-birth_date">Birth date:</label></th><td><input type="text" name="form-0-218 Chapter 3. Using Django
- 223. Django Documentation, Release1.5.1 Note: modelformset_factory() uses formset_factory to generate formsets. This means that a model formset is just an extension of a basic formset that knows how to interact with a particular model. Changing the queryset By default, when you create a formset from a model, the formset will use a queryset that includes all objects in the model (e.g., Author.objects.all()). You can override this behavior by using the queryset argument: >>> formset = AuthorFormSet(queryset=Author.objects.filter(name__startswith=’O’)) Alternatively, you can create a subclass that sets self.queryset in __init__: from django.forms.models import BaseModelFormSet class BaseAuthorFormSet(BaseModelFormSet): def __init__(self, *args, **kwargs): super(BaseAuthorFormSet, self).__init__(*args, **kwargs) self.queryset = Author.objects.filter(name__startswith=’O’) Then, pass your BaseAuthorFormSet class to the factory function: >>> AuthorFormSet = modelformset_factory(Author, formset=BaseAuthorFormSet) If you want to return a formset that doesn’t include any pre-existing instances of the model, you can specify an empty QuerySet: >>> AuthorFormSet(queryset=Author.objects.none()) Controlling which fields are used with fields and exclude By default, a model formset uses all fields in the model that are not marked with editable=False. However, this can be overridden at the formset level: >>> AuthorFormSet = modelformset_factory(Author, fields=(’name’, ’title’)) Using fields restricts the formset to use only the given fields. Alternatively, you can take an “opt-out” approach, specifying which fields to exclude: >>> AuthorFormSet = modelformset_factory(Author, exclude=(’birth_date’,)) Providing initial values New in version 1.4. As with regular formsets, it’s possible to specify initial data for forms in the formset by specifying an initial parameter when instantiating the model formset class returned by modelformset_factory(). However, with model formsets, the initial values only apply to extra forms, those that aren’t bound to an existing object instance. Saving objects in the formset As with a ModelForm, you can save the data as a model object. This is done with the formset’s save() method: # Create a formset instance with POST data. >>> formset = AuthorFormSet(request.POST) # Assuming all is valid, save the data. >>> instances = formset.save() 3.4. Working with forms 219
- 224. Django Documentation, Release1.5.1 The save() method returns the instances that have been saved to the database. If a given instance’s data didn’t change in the bound data, the instance won’t be saved to the database and won’t be included in the return value (instances, in the above example). When fields are missing from the form (for example because they have been excluded), these fields will not be set by the save() method. You can find more information about this restriction, which also holds for regular ModelForms, in Using a subset of fields on the form. Pass commit=False to return the unsaved model instances: # don’t save to the database >>> instances = formset.save(commit=False) >>> for instance in instances: ... # do something with instance ... instance.save() This gives you the ability to attach data to the instances before saving them to the database. If your formset contains a ManyToManyField, you’ll also need to call formset.save_m2m() to ensure the many-to-many relationships are saved properly. Limiting the number of editable objects As with regular formsets, you can use the max_num and extra param-eters to modelformset_factory() to limit the number of extra forms displayed. max_num does not prevent existing objects from being displayed: >>> Author.objects.order_by(’name’) [<Author: Charles Baudelaire>, <Author: Paul Verlaine>, <Author: Walt Whitman>] >>> AuthorFormSet = modelformset_factory(Author, max_num=1) >>> formset = AuthorFormSet(queryset=Author.objects.order_by(’name’)) >>> [x.name for x in formset.get_queryset()] [u’Charles Baudelaire’, u’Paul Verlaine’, u’Walt Whitman’] If the value of max_num is greater than the number of existing related objects, up to extra additional blank forms will be added to the formset, so long as the total number of forms does not exceed max_num: >>> AuthorFormSet = modelformset_factory(Author, max_num=4, extra=2) >>> formset = AuthorFormSet(queryset=Author.objects.order_by(’name’)) >>> for form in formset: ... print(form.as_table()) <tr><th><label for="id_form-0-name">Name:</label></th><td><input id="id_form-0-name" type="text" name="<tr><th><label for="id_form-1-name">Name:</label></th><td><input id="id_form-1-name" type="text" name="<tr><th><label for="id_form-2-name">Name:</label></th><td><input id="id_form-2-name" type="text" name="<tr><th><label for="id_form-3-name">Name:</label></th><td><input id="id_form-3-name" type="text" name="A max_num value of None (the default) puts a high limit on the number of forms displayed (1000). In practice this is equivalent to no limit. Using a model formset in a view Model formsets are very similar to formsets. Let’s say we want to present a formset to edit Author model instances: def manage_authors(request): AuthorFormSet = modelformset_factory(Author) if request.method == ’POST’: formset = AuthorFormSet(request.POST, request.FILES) if formset.is_valid(): formset.save() # do something. 220 Chapter 3. Using Django
- 225. Django Documentation, Release1.5.1 else: formset = AuthorFormSet() return render_to_response("manage_authors.html", { "formset": formset, }) As you can see, the view logic of a model formset isn’t drastically different than that of a “normal” formset. The only difference is that we call formset.save() to save the data into the database. (This was described above, in Saving objects in the formset.) Overiding clean() on a model_formset Just like with ModelForms, by default the clean() method of a model_formset will validate that none of the items in the formset violate the unique constraints on your model (either unique, unique_together or unique_for_date|month|year). If you want to override the clean() method on a model_formset and maintain this validation, you must call the parent class’s clean method: class MyModelFormSet(BaseModelFormSet): def clean(self): super(MyModelFormSet, self).clean() # example custom validation across forms in the formset: for form in self.forms: # your custom formset validation Using a custom queryset As stated earlier, you can override the default queryset used by the model formset: def manage_authors(request): AuthorFormSet = modelformset_factory(Author) if request.method == "POST": formset = AuthorFormSet(request.POST, request.FILES, queryset=Author.objects.filter(name__startswith=’O’)) if formset.is_valid(): formset.save() # Do something. else: formset = AuthorFormSet(queryset=Author.objects.filter(name__startswith=’O’)) return render_to_response("manage_authors.html", { "formset": formset, }) Note that we pass the queryset argument in both the POST and GET cases in this example. Using the formset in the template There are three ways to render a formset in a Django template. First, you can let the formset do most of the work: <form method="post" action=""> {{ formset }} </form> Second, you can manually render the formset, but let the form deal with itself: <form method="post" action=""> {{ formset.management_form }} {% for form in formset %} {{ form }} {% endfor %} </form> 3.4. Working with forms 221
- 226. Django Documentation, Release1.5.1 When you manually render the forms yourself, be sure to render the management form as shown above. See the management form documentation. Third, you can manually render each field: <form method="post" action=""> {{ formset.management_form }} {% for form in formset %} {% for field in form %} {{ field.label_tag }}: {{ field }} {% endfor %} {% endfor %} </form> If you opt to use this third method and you don’t iterate over the fields with a {% for %} loop, you’ll need to render the primary key field. For example, if you were rendering the name and age fields of a model: <form method="post" action=""> {{ formset.management_form }} {% for form in formset %} {{ form.id }} <ul> <li>{{ form.name }}</li> <li>{{ form.age }}</li> </ul> {% endfor %} </form> Notice how we need to explicitly render {{ form.id }}. This ensures that the model formset, in the POST case, will work correctly. (This example assumes a primary key named id. If you’ve explicitly defined your own primary key that isn’t called id, make sure it gets rendered.) Inline formsets Inline formsets is a small abstraction layer on top of model formsets. These simplify the case of working with related objects via a foreign key. Suppose you have these two models: class Author(models.Model): name = models.CharField(max_length=100) class Book(models.Model): author = models.ForeignKey(Author) title = models.CharField(max_length=100) If you want to create a formset that allows you to edit books belonging to a particular author, you could do this: >>> from django.forms.models import inlineformset_factory >>> BookFormSet = inlineformset_factory(Author, Book) >>> author = Author.objects.get(name=u’Mike Royko’) >>> formset = BookFormSet(instance=author) Note: inlineformset_factory() uses modelformset_factory() and marks can_delete=True. See Also: Manually rendered can_delete and can_order. 222 Chapter 3. Using Django
- 227. Django Documentation, Release1.5.1 More than one foreign key to the same model If your model contains more than one foreign key to the same model, you’ll need to resolve the ambiguity manually using fk_name. For example, consider the following model: class Friendship(models.Model): from_friend = models.ForeignKey(Friend) to_friend = models.ForeignKey(Friend) length_in_months = models.IntegerField() To resolve this, you can use fk_name to inlineformset_factory(): >>> FriendshipFormSet = inlineformset_factory(Friend, Friendship, fk_name="from_friend") Using an inline formset in a view You may want to provide a view that allows a user to edit the related objects of a model. Here’s how you can do that: def manage_books(request, author_id): author = Author.objects.get(pk=author_id) BookInlineFormSet = inlineformset_factory(Author, Book) if request.method == "POST": formset = BookInlineFormSet(request.POST, request.FILES, instance=author) if formset.is_valid(): formset.save() # Do something. Should generally end with a redirect. For example: return HttpResponseRedirect(author.get_absolute_url()) else: formset = BookInlineFormSet(instance=author) return render_to_response("manage_books.html", { "formset": formset, }) Notice how we pass instance in both the POST and GET cases. Form Media Rendering an attractive and easy-to-use Web form requires more than just HTML - it also requires CSS stylesheets, and if you want to use fancy “Web2.0” widgets, you may also need to include some JavaScript on each page. The exact combination of CSS and JavaScript that is required for any given page will depend upon the widgets that are in use on that page. This is where Django media definitions come in. Django allows you to associate different media files with the forms and widgets that require that media. For example, if you want to use a calendar to render DateFields, you can define a custom Calendar widget. This widget can then be associated with the CSS and JavaScript that is required to render the calendar. When the Calendar widget is used on a form, Django is able to identify the CSS and JavaScript files that are required, and provide the list of file names in a form suitable for easy inclusion on your Web page. Media and Django Admin The Django Admin application defines a number of customized widgets for calendars, filtered selections, and so on. These widgets define media requirements, and the Django Admin uses the custom widgets in place of the Django defaults. The Admin templates will only include those media files that are required to render the widgets on any given page. If you like the widgets that the Django Admin application uses, feel free to use them in your own application! They’re all stored in django.contrib.admin.widgets. 3.4. Working with forms 223
- 228. Django Documentation, Release1.5.1 Which JavaScript toolkit? Many JavaScript toolkits exist, and many of them include widgets (such as calendar widgets) that can be used to enhance your application. Django has deliberately avoided blessing any one JavaScript toolkit. Each toolkit has its own relative strengths and weaknesses - use whichever toolkit suits your requirements. Django is able to integrate with any JavaScript toolkit. Media as a static definition The easiest way to define media is as a static definition. Using this method, the media declaration is an inner class. The properties of the inner class define the media requirements. Here’s a simple example: class CalendarWidget(forms.TextInput): class Media: css = { ’all’: (’pretty.css’,) } js = (’animations.js’, ’actions.js’) This code defines a CalendarWidget, which will be based on TextInput. Every time the CalendarWid-get is used on a form, that form will be directed to include the CSS file pretty.css, and the JavaScript files animations.js and actions.js. This static media definition is converted at runtime into a widget property named media. The media for a Calendar- Widget instance can be retrieved through this property: >>> w = CalendarWidget() >>> print(w.media) <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> <script type="text/javascript" src="http://static.example.com/actions.js"></script> Here’s a list of all possible Media options. There are no required options. css A dictionary describing the CSS files required for various forms of output media. The values in the dictionary should be a tuple/list of file names. See the section on media paths for details of how to specify paths to media files. The keys in the dictionary are the output media types. These are the same types accepted by CSS files in media declarations: ‘all’, ‘aural’, ‘braille’, ‘embossed’, ‘handheld’, ‘print’, ‘projection’, ‘screen’, ‘tty’ and ‘tv’. If you need to have different stylesheets for different media types, provide a list of CSS files for each output medium. The following example would provide two CSS options – one for the screen, and one for print: class Media: css = { ’screen’: (’pretty.css’,), ’print’: (’newspaper.css’,) } If a group of CSS files are appropriate for multiple output media types, the dictionary key can be a comma separated list of output media types. In the following example, TV’s and projectors will have the same media requirements: class Media: css = { 224 Chapter 3. Using Django
- 229. Django Documentation, Release1.5.1 ’screen’: (’pretty.css’,), ’tv,projector’: (’lo_res.css’,), ’print’: (’newspaper.css’,) } If this last CSS definition were to be rendered, it would become the following HTML: <link href="http://static.example.com/pretty.css" type="text/css" media="screen" rel="stylesheet" /> <link href="http://static.example.com/lo_res.css" type="text/css" media="tv,projector" rel="stylesheet" <link href="http://static.example.com/newspaper.css" type="text/css" media="print" rel="stylesheet" /> js A tuple describing the required JavaScript files. See the section on media paths for details of how to specify paths to media files. extend A boolean defining inheritance behavior for media declarations. By default, any object using a static media definition will inherit all the media associated with the parent widget. This occurs regardless of how the parent defines its media requirements. For example, if we were to extend our basic Calendar widget from the example above: >>> class FancyCalendarWidget(CalendarWidget): ... class Media: ... css = { ... ’all’: (’fancy.css’,) ... } ... js = (’whizbang.js’,) >>> w = FancyCalendarWidget() >>> print(w.media) <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> <link href="http://static.example.com/fancy.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> <script type="text/javascript" src="http://static.example.com/actions.js"></script> <script type="text/javascript" src="http://static.example.com/whizbang.js"></script> The FancyCalendar widget inherits all the media from it’s parent widget. If you don’t want media to be inherited in this way, add an extend=False declaration to the media declaration: >>> class FancyCalendarWidget(CalendarWidget): ... class Media: ... extend = False ... css = { ... ’all’: (’fancy.css’,) ... } ... js = (’whizbang.js’,) >>> w = FancyCalendarWidget() >>> print(w.media) <link href="http://static.example.com/fancy.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/whizbang.js"></script> If you require even more control over media inheritance, define your media using a dynamic property. Dynamic properties give you complete control over which media files are inherited, and which are not. 3.4. Working with forms 225
- 230. Django Documentation, Release1.5.1 Media as a dynamic property If you need to perform some more sophisticated manipulation of media requirements, you can define the media prop-erty directly. This is done by defining a widget property that returns an instance of forms.Media. The constructor for forms.Media accepts css and js keyword arguments in the same format as that used in a static media defini-tion. For example, the static media definition for our Calendar Widget could also be defined in a dynamic fashion: class CalendarWidget(forms.TextInput): def _media(self): return forms.Media(css={’all’: (’pretty.css’,)}, js=(’animations.js’, ’actions.js’)) media = property(_media) See the section on Media objects for more details on how to construct return values for dynamic media properties. Paths in media definitions Paths used to specify media can be either relative or absolute. If a path starts with /, http:// or https://, it will be interpreted as an absolute path, and left as-is. All other paths will be prepended with the value of the appropriate prefix. As part of the introduction of the staticfiles app two new settings were added to refer to “static files” (images, CSS, Javascript, etc.) that are needed to render a complete web page: STATIC_URL and STATIC_ROOT. To find the appropriate prefix to use, Django will check if the STATIC_URL setting is not None and automatically fall back to using MEDIA_URL. For example, if the MEDIA_URL for your site was ’http://uploads.example.com/’ and STATIC_URL was None: >>> class CalendarWidget(forms.TextInput): ... class Media: ... css = { ... ’all’: (’/css/pretty.css’,), ... } ... js = (’animations.js’, ’http://othersite.com/actions.js’) >>> w = CalendarWidget() >>> print(w.media) <link href="/css/pretty.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://uploads.example.com/animations.js"></script> <script type="text/javascript" src="http://othersite.com/actions.js"></script> But if STATIC_URL is ’http://static.example.com/’: >>> w = CalendarWidget() >>> print(w.media) <link href="/css/pretty.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> <script type="text/javascript" src="http://othersite.com/actions.js"></script> Media objects When you interrogate the media attribute of a widget or form, the value that is returned is a forms.Media object. As we have already seen, the string representation of a Media object is the HTML required to include media in the <head> block of your HTML page. 226 Chapter 3. Using Django
- 231. Django Documentation, Release1.5.1 However, Media objects have some other interesting properties. Media subsets If you only want media of a particular type, you can use the subscript operator to filter out a medium of interest. For example: >>> w = CalendarWidget() >>> print(w.media) <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> <script type="text/javascript" src="http://static.example.com/actions.js"></script> >>> print(w.media)[’css’] <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> When you use the subscript operator, the value that is returned is a new Media object – but one that only contains the media of interest. Combining media objects Media objects can also be added together. When two media objects are added, the resulting Media object contains the union of the media from both files: >>> class CalendarWidget(forms.TextInput): ... class Media: ... css = { ... ’all’: (’pretty.css’,) ... } ... js = (’animations.js’, ’actions.js’) >>> class OtherWidget(forms.TextInput): ... class Media: ... js = (’whizbang.js’,) >>> w1 = CalendarWidget() >>> w2 = OtherWidget() >>> print(w1.media + w2.media) <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> <script type="text/javascript" src="http://static.example.com/actions.js"></script> <script type="text/javascript" src="http://static.example.com/whizbang.js"></script> Media on Forms Widgets aren’t the only objects that can have media definitions – forms can also define media. The rules for media definitions on forms are the same as the rules for widgets: declarations can be static or dynamic; path and inheritance rules for those declarations are exactly the same. Regardless of whether you define a media declaration, all Form objects have a media property. The default value for this property is the result of adding the media definitions for all widgets that are part of the form: >>> class ContactForm(forms.Form): ... date = DateField(widget=CalendarWidget) ... name = CharField(max_length=40, widget=OtherWidget) >>> f = ContactForm() >>> f.media <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> 3.4. Working with forms 227
- 232. Django Documentation, Release1.5.1 <script type="text/javascript" src="http://static.example.com/actions.js"></script> <script type="text/javascript" src="http://static.example.com/whizbang.js"></script> If you want to associate additional media with a form – for example, CSS for form layout – simply add a media declaration to the form: >>> class ContactForm(forms.Form): ... date = DateField(widget=CalendarWidget) ... name = CharField(max_length=40, widget=OtherWidget) ... ... class Media: ... css = { ... ’all’: (’layout.css’,) ... } >>> f = ContactForm() >>> f.media <link href="http://static.example.com/pretty.css" type="text/css" media="all" rel="stylesheet" /> <link href="http://static.example.com/layout.css" type="text/css" media="all" rel="stylesheet" /> <script type="text/javascript" src="http://static.example.com/animations.js"></script> <script type="text/javascript" src="http://static.example.com/actions.js"></script> <script type="text/javascript" src="http://static.example.com/whizbang.js"></script> See Also: The Forms Reference Covers the full API reference, including form fields, form widgets, and form and field valida-tion. 3.5 The Django template language About this document This document explains the language syntax of the Django template system. If you’re looking for a more technical perspective on how it works and how to extend it, see The Django template language: For Python programmers. Django’s template language is designed to strike a balance between power and ease. It’s designed to feel comfortable to those used to working with HTML. If you have any exposure to other text-based template languages, such as Smarty or CheetahTemplate, you should feel right at home with Django’s templates. Philosophy If you have a background in programming, or if you’re used to languages which mix programming code directly into HTML, you’ll want to bear in mind that the Django template system is not simply Python embedded into HTML. This is by design: the template system is meant to express presentation, not program logic. The Django template system provides tags which function similarly to some programming constructs – an if tag for boolean tests, a for tag for looping, etc. – but these are not simply executed as the corresponding Python code, and the template system will not execute arbitrary Python expressions. Only the tags, filters and syntax listed below are supported by default (although you can add your own extensions to the template language as needed). 3.5.1 Templates A template is simply a text file. It can generate any text-based format (HTML, XML, CSV, etc.). 228 Chapter 3. Using Django
- 233. Django Documentation, Release1.5.1 A template contains variables, which get replaced with values when the template is evaluated, and tags, which control the logic of the template. Below is a minimal template that illustrates a few basics. Each element will be explained later in this document.: {% extends "base_generic.html" %} {% block title %}{{ section.title }}{% endblock %} {% block content %} <h1>{{ section.title }}</h1> {% for story in story_list %} <h2> <a href="{{ story.get_absolute_url }}"> {{ story.headline|upper }} </a> </h2> <p>{{ story.tease|truncatewords:"100" }}</p> {% endfor %} {% endblock %} Philosophy Why use a text-based template instead of an XML-based one (like Zope’s TAL)? We wanted Django’s template lan-guage to be usable for more than just XML/HTML templates. At World Online, we use it for emails, JavaScript and CSV. You can use the template language for any text-based format. Oh, and one more thing: Making humans edit XML is sadistic! 3.5.2 Variables Variables look like this: {{ variable }}. When the template engine encounters a variable, it evaluates that variable and replaces it with the result. Variable names consist of any combination of alphanumeric characters and the underscore ("_"). The dot (".") also appears in variable sections, although that has a special meaning, as indicated below. Importantly, you cannot have spaces or punctuation characters in variable names. Use a dot (.) to access attributes of a variable. Behind the scenes Technically, when the template system encounters a dot, it tries the following lookups, in this order: • Dictionary lookup • Attribute lookup • Method call • List-index lookup This can cause some unexpected behavior with objects that override dictionary lookup. For example, consider the following code snippet that attempts to loop over a collections.defaultdict: {% for k, v in defaultdict.iteritems %} Do something with k and v here... {% endfor %} 3.5. The Django template language 229
- 234. Django Documentation, Release1.5.1 Because dictionary lookup happens first, that behavior kicks in and provides a default value instead of using the intended .iteritems() method. In this case, consider converting to a dictionary first. In the above example, {{ section.title }} will be replaced with the title attribute of the section object. If you use a variable that doesn’t exist, the template system will insert the value of the TEMPLATE_STRING_IF_INVALID setting, which is set to ” (the empty string) by default. Note that “bar” in a template expression like {{ foo.bar }} will be interpreted as a literal string and not using the value of the variable “bar”, if one exists in the template context. 3.5.3 Filters You can modify variables for display by using filters. Filters look like this: {{ name|lower }}. This displays the value of the {{ name }} variable after being filtered through the lower filter, which converts text to lowercase. Use a pipe (|) to apply a filter. Filters can be “chained.” The output of one filter is applied to the next. {{ text|escape|linebreaks }} is a common idiom for escaping text contents, then converting line breaks to <p> tags. Some filters take arguments. A filter argument looks like this: {{ bio|truncatewords:30 }}. This will display the first 30 words of the bio variable. Filter arguments that contain spaces must be quoted; for example, to join a list with commas and spaced you’d use {{ list|join:", " }}. Django provides about thirty built-in template filters. You can read all about them in the built-in filter reference. To give you a taste of what’s available, here are some of the more commonly used template filters: default If a variable is false or empty, use given default. Otherwise, use the value of the variable For example: {{ value|default:"nothing" }} If value isn’t provided or is empty, the above will display “nothing”. length Returns the length of the value. This works for both strings and lists; for example: {{ value|length }} If value is [’a’, ’b’, ’c’, ’d’], the output will be 4. striptags Strips all [X]HTML tags. For example: {{ value|striptags }} If value is "<b>Joel</b> <button>is</button> a <span>slug</span>", the output will be "Joel is a slug". Again, these are just a few examples; see the built-in filter reference for the complete list. You can also create your own custom template filters; see Custom template tags and filters. See Also: Django’s admin interface can include a complete reference of all template tags and filters available for a given site. See The Django admin documentation generator. 230 Chapter 3. Using Django
- 235. Django Documentation, Release1.5.1 3.5.4 Tags Tags look like this: {% tag %}. Tags are more complex than variables: Some create text in the output, some control flow by performing loops or logic, and some load external information into the template to be used by later variables. Some tags require beginning and ending tags (i.e. {% tag %} ... tag contents ... {% endtag %}). Django ships with about two dozen built-in template tags. You can read all about them in the built-in tag reference. To give you a taste of what’s available, here are some of the more commonly used tags: for Loop over each item in an array. For example, to display a list of athletes provided in athlete_list: <ul> {% for athlete in athlete_list %} <li>{{ athlete.name }}</li> {% endfor %} </ul> if and else Evaluates a variable, and if that variable is “true” the contents of the block are displayed: {% if athlete_list %} Number of athletes: {{ athlete_list|length }} {% else %} No athletes. {% endif %} In the above, if athlete_list is not empty, the number of athletes will be displayed by the {{ athlete_list|length }} variable. You can also use filters and various operators in the if tag: {% if athlete_list|length > 1 %} Team: {% for athlete in athlete_list %} ... {% endfor %} {% else %} Athlete: {{ athlete_list.0.name }} {% endif %} While the above example works, be aware that most template filters return strings, so mathematical comparisons using filters will generally not work as you expect. length is an exception. block and extends Set up template inheritance (see below), a powerful way of cutting down on “boilerplate” in templates. Again, the above is only a selection of the whole list; see the built-in tag reference for the complete list. You can also create your own custom template tags; see Custom template tags and filters. See Also: Django’s admin interface can include a complete reference of all template tags and filters available for a given site. See The Django admin documentation generator. 3.5.5 Comments To comment-out part of a line in a template, use the comment syntax: {# #}. For example, this template would render as ’hello’: {# greeting #}hello 3.5. The Django template language 231
- 236. Django Documentation, Release1.5.1 A comment can contain any template code, invalid or not. For example: {# {% if foo %}bar{% else %} #} This syntax can only be used for single-line comments (no newlines are permitted between the {# and #} delimiters). If you need to comment out a multiline portion of the template, see the comment tag. 3.5.6 Template inheritance The most powerful – and thus the most complex – part of Django’s template engine is template inheritance. Template inheritance allows you to build a base “skeleton” template that contains all the common elements of your site and defines blocks that child templates can override. It’s easiest to understand template inheritance by starting with an example: <!DOCTYPE html> <html lang="en"> <head> <link rel="stylesheet" href="style.css" /> <title>{% block title %}My amazing site{% endblock %}</title> </head> <body> <div id="sidebar"> {% block sidebar %} <ul> <li><a href="/">Home</a></li> <li><a href="/blog/">Blog</a></li> </ul> {% endblock %} </div> <div id="content"> {% block content %}{% endblock %} </div> </body> </html> This template, which we’ll call base.html, defines a simple HTML skeleton document that you might use for a simple two-column page. It’s the job of “child” templates to fill the empty blocks with content. In this example, the block tag defines three blocks that child templates can fill in. All the block tag does is to tell the template engine that a child template may override those portions of the template. A child template might look like this: {% extends "base.html" %} {% block title %}My amazing blog{% endblock %} {% block content %} {% for entry in blog_entries %} <h2>{{ entry.title }}</h2> <p>{{ entry.body }}</p> {% endfor %} {% endblock %} The extends tag is the key here. It tells the template engine that this template “extends” another template. When the template system evaluates this template, first it locates the parent – in this case, “base.html”. 232 Chapter 3. Using Django
- 237. Django Documentation, Release1.5.1 At that point, the template engine will notice the three block tags in base.html and replace those blocks with the contents of the child template. Depending on the value of blog_entries, the output might look like: <!DOCTYPE html> <html lang="en"> <head> <link rel="stylesheet" href="style.css" /> <title>My amazing blog</title> </head> <body> <div id="sidebar"> <ul> <li><a href="/">Home</a></li> <li><a href="/blog/">Blog</a></li> </ul> </div> <div id="content"> <h2>Entry one</h2> <p>This is my first entry.</p> <h2>Entry two</h2> <p>This is my second entry.</p> </div> </body> </html> Note that since the child template didn’t define the sidebar block, the value from the parent template is used instead. Content within a {% block %} tag in a parent template is always used as a fallback. You can use as many levels of inheritance as needed. One common way of using inheritance is the following three-level approach: • Create a base.html template that holds the main look-and-feel of your site. • Create a base_SECTIONNAME.html template for each “section” of your site. For example, base_news.html, base_sports.html. These templates all extend base.html and include section-specific styles/design. • Create individual templates for each type of page, such as a news article or blog entry. These templates extend the appropriate section template. This approach maximizes code reuse and makes it easy to add items to shared content areas, such as section-wide navigation. Here are some tips for working with inheritance: • If you use {% extends %} in a template, it must be the first template tag in that template. Template inheri-tance won’t work, otherwise. • More {% block %} tags in your base templates are better. Remember, child templates don’t have to define all parent blocks, so you can fill in reasonable defaults in a number of blocks, then only define the ones you need later. It’s better to have more hooks than fewer hooks. • If you find yourself duplicating content in a number of templates, it probably means you should move that content to a {% block %} in a parent template. • If you need to get the content of the block from the parent template, the {{ block.super }} variable will do the trick. This is useful if you want to add to the contents of a parent block instead of completely overriding it. Data inserted using {{ block.super }} will not be automatically escaped (see the next section), since it was already escaped, if necessary, in the parent template. 3.5. The Django template language 233
- 238. Django Documentation, Release1.5.1 • For extra readability, you can optionally give a name to your {% endblock %} tag. For example: {% block content %} ... {% endblock content %} In larger templates, this technique helps you see which {% block %} tags are being closed. Finally, note that you can’t define multiple block tags with the same name in the same template. This limitation exists because a block tag works in “both” directions. That is, a block tag doesn’t just provide a hole to fill – it also defines the content that fills the hole in the parent. If there were two similarly-named block tags in a template, that template’s parent wouldn’t know which one of the blocks’ content to use. 3.5.7 Automatic HTML escaping When generating HTML from templates, there’s always a risk that a variable will include characters that affect the resulting HTML. For example, consider this template fragment: Hello, {{ name }}. At first, this seems like a harmless way to display a user’s name, but consider what would happen if the user entered his name as this: <script>alert(’hello’)</script> With this name value, the template would be rendered as: Hello, <script>alert(’hello’)</script> ...which means the browser would pop-up a JavaScript alert box! Similarly, what if the name contained a ’<’ symbol, like this? <b>username That would result in a rendered template like this: Hello, <b>username ...which, in turn, would result in the remainder of the Web page being bolded! Clearly, user-submitted data shouldn’t be trusted blindly and inserted directly into yourWeb pages, because a malicious user could use this kind of hole to do potentially bad things. This type of security exploit is called a Cross Site Scripting (XSS) attack. To avoid this problem, you have two options: • One, you can make sure to run each untrusted variable through the escape filter (documented below), which converts potentially harmful HTML characters to unharmful ones. This was the default solution in Django for its first few years, but the problem is that it puts the onus on you, the developer / template author, to ensure you’re escaping everything. It’s easy to forget to escape data. • Two, you can take advantage of Django’s automatic HTML escaping. The remainder of this section describes how auto-escaping works. By default in Django, every template automatically escapes the output of every variable tag. Specifically, these five characters are escaped: • < is converted to < • > is converted to > 234 Chapter 3. Using Django
- 239. Django Documentation, Release1.5.1 • ’ (single quote) is converted to ' • " (double quote) is converted to " • & is converted to & Again, we stress that this behavior is on by default. If you’re using Django’s template system, you’re protected. How to turn it off If you don’t want data to be auto-escaped, on a per-site, per-template level or per-variable level, you can turn it off in several ways. Why would you want to turn it off? Because sometimes, template variables contain data that you intend to be rendered as raw HTML, in which case you don’t want their contents to be escaped. For example, you might store a blob of HTML in your database and want to embed that directly into your template. Or, you might be using Django’s template system to produce text that is not HTML – like an email message, for instance. For individual variables To disable auto-escaping for an individual variable, use the safe filter: This will be escaped: {{ data }} This will not be escaped: {{ data|safe }} Think of safe as shorthand for safe from further escaping or can be safely interpreted as HTML. In this example, if data contains ’<b>’, the output will be: This will be escaped: <b> This will not be escaped: <b> For template blocks To control auto-escaping for a template, wrap the template (or just a particular section of the template) in the autoescape tag, like so: {% autoescape off %} Hello {{ name }} {% endautoescape %} The autoescape tag takes either on or off as its argument. At times, you might want to force auto-escaping when it would otherwise be disabled. Here is an example template: Auto-escaping is on by default. Hello {{ name }} {% autoescape off %} This will not be auto-escaped: {{ data }}. Nor this: {{ other_data }} {% autoescape on %} Auto-escaping applies again: {{ name }} {% endautoescape %} {% endautoescape %} The auto-escaping tag passes its effect onto templates that extend the current one as well as templates included via the include tag, just like all block tags. For example: 3.5. The Django template language 235
- 240. Django Documentation, Release1.5.1 # base.html {% autoescape off %} <h1>{% block title %}{% endblock %}</h1> {% block content %} {% endblock %} {% endautoescape %} # child.html {% extends "base.html" %} {% block title %}This & that{% endblock %} {% block content %}{{ greeting }}{% endblock %} Because auto-escaping is turned off in the base template, it will also be turned off in the child template, resulting in the following rendered HTML when the greeting variable contains the string <b>Hello!</b>: <h1>This & that</h1> <b>Hello!</b> Notes Generally, template authors don’t need to worry about auto-escaping very much. Developers on the Python side (people writing views and custom filters) need to think about the cases in which data shouldn’t be escaped, and mark data appropriately, so things Just Work in the template. If you’re creating a template that might be used in situations where you’re not sure whether auto-escaping is enabled, then add an escape filter to any variable that needs escaping. When auto-escaping is on, there’s no danger of the escape filter double-escaping data – the escape filter does not affect auto-escaped variables. String literals and automatic escaping As we mentioned earlier, filter arguments can be strings: {{ data|default:"This is a string literal." }} All string literals are inserted without any automatic escaping into the template – they act as if they were all passed through the safe filter. The reasoning behind this is that the template author is in control of what goes into the string literal, so they can make sure the text is correctly escaped when the template is written. This means you would write {{ data|default:"3 < 2" }} ...rather than {{ data|default:"3 < 2" }} <-- Bad! Don’t do this. This doesn’t affect what happens to data coming from the variable itself. The variable’s contents are still automatically escaped, if necessary, because they’re beyond the control of the template author. 3.5.8 Accessing method calls Most method calls attached to objects are also available from within templates. This means that templates have access to much more than just class attributes (like field names) and variables passed in from views. For example, the Django 236 Chapter 3. Using Django
- 241. Django Documentation, Release1.5.1 ORM provides the “entry_set” syntax for finding a collection of objects related on a foreign key. Therefore, given a model called “comment” with a foreign key relationship to a model called “task” you can loop through all comments attached to a given task like this: {% for comment in task.comment_set.all %} {{ comment }} {% endfor %} Similarly, QuerySets provide a count() method to count the number of objects they contain. Therefore, you can obtain a count of all comments related to the current task with: {{ task.comment_set.all.count }} And of course you can easily access methods you’ve explicitly defined on your own models: # In model class Task(models.Model): def foo(self): return "bar" # In template {{ task.foo }} Because Django intentionally limits the amount of logic processing available in the template language, it is not possible to pass arguments to method calls accessed from within templates. Data should be calculated in views, then passed to templates for display. 3.5.9 Custom tag and filter libraries Certain applications provide custom tag and filter libraries. To access them in a template, use the load tag: {% load comments %} {% comment_form for blogs.entries entry.id with is_public yes %} In the above, the load tag loads the comments tag library, which then makes the comment_form tag available for use. Consult the documentation area in your admin to find the list of custom libraries in your installation. The load tag can take multiple library names, separated by spaces. Example: {% load comments i18n %} See Custom template tags and filters for information on writing your own custom template libraries. Custom libraries and template inheritance When you load a custom tag or filter library, the tags/filters are only made available to the current template – not any parent or child templates along the template-inheritance path. For example, if a template foo.html has {% load comments %}, a child template (e.g., one that has {% extends "foo.html" %}) will not have access to the comments template tags and filters. The child template is responsible for its own {% load comments %}. This is a feature for the sake of maintainability and sanity. 3.5. The Django template language 237
- 242. Django Documentation, Release1.5.1 3.6 Class-based views A view is a callable which takes a request and returns a response. This can be more than just a function, and Django provides an example of some classes which can be used as views. These allow you to structure your views and reuse code by harnessing inheritance and mixins. There are also some generic views for simple tasks which we’ll get to later, but you may want to design your own structure of reusable views which suits your use case. For full details, see the class-based views reference documentation. 3.6.1 Introduction to Class-based views Class-based views provide an alternative way to implement views as Python objects instead of functions. They do not replace function-based views, but have certain differences and advantages when compared to function-based views: • Organization of code related to specific HTTP methods (GET, POST, etc) can be addressed by separate methods instead of conditional branching. • Object oriented techniques such as mixins (multiple inheritance) can be used to factor code into reusable com-ponents. The relationship and history of generic views, class-based views, and class-based generic views In the beginning there was only the view function contract, Django passed your function an HttpRequest and expected back an HttpResponse. This was the extent of what Django provided. Early on it was recognized that there were common idioms and patterns found in view development. Function-based generic views were introduced to abstract these patterns and ease view development for the common cases. The problem with function-based generic views is that while they covered the simple cases well, there was no way to extend or customize them beyond some simple configuration options, limiting their usefulness in many real-world applications. Class-based generic views were created with the same objective as function-based generic views, to make view devel-opment easier. However, the way the solution is implemented, through the use of mixins, provides a toolkit that results in class-based generic views being more extensible and flexible than their function-based counterparts. If you have tried function based generic views in the past and found them lacking, you should not think of class-based generic views as simply a class-based equivalent, but rather as a fresh approach to solving the original problems that generic views were meant to solve. The toolkit of base classes and mixins that Django uses to build class-based generic views are built for maximum flexibility, and as such have many hooks in the form of default method implementations and attributes that you are unlikely to be concerned with in the simplest use cases. For example, instead of limiting you to a class based attribute for form_class, the implementation uses a get_form method, which calls a get_form_class method, which in its default implementation just returns the form_class attribute of the class. This gives you several options for specifying what form to use, from a simple attribute, to a fully dynamic, callable hook. These options seem to add hollow complexity for simple situations, but without them, more advanced designs would be limited. Using class-based views At its core, a class-based view allows you to respond to different HTTP request methods with different class instance methods, instead of with conditionally branching code inside a single view function. So where the code to handle HTTP GET in a view function would look something like: 238 Chapter 3. Using Django
- 243. Django Documentation, Release1.5.1 from django.http import HttpResponse def my_view(request): if request.method == ’GET’: # <view logic> return HttpResponse(’result’) In a class-based view, this would become: from django.http import HttpResponse from django.views.base import View class MyView(View): def get(self, request): # <view logic> return HttpResponse(’result’) Because Django’s URL resolver expects to send the request and associated arguments to a callable function, not a class, class-based views have an as_view() class method which serves as the callable entry point to your class. The as_view entry point creates an instance of your class and calls its dispatch() method. dispatch looks at the request to determine whether it is a GET, POST, etc, and relays the request to a matching method if one is defined, or raises HttpResponseNotAllowed if not: # urls.py from django.conf.urls import patterns from myapp.views import MyView urlpatterns = patterns(’’, (r’^about/’, MyView.as_view()), ) It is worth noting that what your method returns is identical to what you return from a function-based view, namely some form of HttpResponse. This means that http shortcuts or TemplateResponse objects are valid to use inside a class-based view. While a minimal class-based view does not require any class attributes to perform its job, class attributes are useful in many class-based designs, and there are two ways to configure or set class attributes. The first is the standard Python way of subclassing and overriding attributes and methods in the subclass. So that if your parent class had an attribute greeting like this: from django.http import HttpResponse from django.views.base import View class GreetingView(View): greeting = "Good Day" def get(self, request): return HttpResponse(self.greeting) You can override that in a subclass: class MorningGreetingView(GreetingView): greeting = "Morning to ya" Another option is to configure class attributes as keyword arguments to the as_view() call in the URLconf: urlpatterns = patterns(’’, (r’^about/’, GreetingView.as_view(greeting="G’day")), ) 3.6. Class-based views 239
- 244. Django Documentation, Release1.5.1 Note: While your class is instantiated for each request dispatched to it, class attributes set through the as_view() entry point are configured only once at the time your URLs are imported. Using mixins Mixins are a form of multiple inheritance where behaviors and attributes of multiple parent classes can be combined. For example, in the generic class-based views there is a mixin called TemplateResponseMixin whose primary purpose is to define the method render_to_response(). When combined with the behavior of the View base class, the result is a TemplateView class that will dispatch requests to the appropriate matching methods (a behavior defined in the View base class), and that has a render_to_response() method that uses a template_name attribute to return a TemplateResponse object (a behavior defined in the TemplateResponseMixin). Mixins are an excellent way of reusing code across multiple classes, but they come with some cost. The more your code is scattered among mixins, the harder it will be to read a child class and know what exactly it is doing, and the harder it will be to know which methods from which mixins to override if you are subclassing something that has a deep inheritance tree. Note also that you can only inherit from one generic view - that is, only one parent class may inherit from View and the rest (if any) should be mixins. Trying to inherit from more than one class that inherits from View - for example, trying to use a form at the top of a list and combining ProcessFormView and ListView - won’t work as expected. Handling forms with class-based views A basic function-based view that handles forms may look something like this: from django.http import HttpResponseRedirect from django.shortcuts import render from .forms import MyForm def myview(request): if request.method == "POST": form = MyForm(request.POST) if form.is_valid(): # <process form cleaned data> return HttpResponseRedirect(’/success/’) else: form = MyForm(initial={’key’: ’value’}) return render(request, ’form_template.html’, {’form’: form}) A similar class-based view might look like: from django.http import HttpResponseRedirect from django.shortcuts import render from .forms import MyForm class MyFormView(View): form_class = MyForm initial = {’key’: ’value’} template_name = ’form_template.html’ def get(self, request, *args, **kwargs): 240 Chapter 3. Using Django
- 245. Django Documentation, Release1.5.1 form = self.form_class(initial=self.initial) return render(request, self.template_name, {’form’: form}) def post(self, request, *args, **kwargs): form = self.form_class(request.POST) if form.is_valid(): # <process form cleaned data> return HttpResponseRedirect(’/success/’) return render(request, self.template_name, {’form’: form}) This is a very simple case, but you can see that you would then have the option of customizing this view by overriding any of the class attributes, e.g. form_class, via URLconf configuration, or subclassing and overriding one or more of the methods (or both!). Decorating class-based views The extension of class-based views isn’t limited to using mixins. You can use also use decorators. Since class-based views aren’t functions, decorating them works differently depending on if you’re using as_view or creating a subclass. Decorating in URLconf The simplest way of decorating class-based views is to decorate the result of the as_view() method. The easiest place to do this is in the URLconf where you deploy your view: from django.contrib.auth.decorators import login_required, permission_required from django.views.generic import TemplateView from .views import VoteView urlpatterns = patterns(’’, (r’^about/’, login_required(TemplateView.as_view(template_name="secret.html"))), (r’^vote/’, permission_required(’polls.can_vote’)(VoteView.as_view())), ) This approach applies the decorator on a per-instance basis. If you want every instance of a view to be decorated, you need to take a different approach. Decorating the class To decorate every instance of a class-based view, you need to decorate the class definition itself. To do this you apply the decorator to the dispatch() method of the class. A method on a class isn’t quite the same as a standalone function, so you can’t just apply a function decorator to the method – you need to transform it into a method decorator first. The method_decorator decorator transforms a function decorator into a method decorator so that it can be used on an instance method. For example: from django.contrib.auth.decorators import login_required from django.utils.decorators import method_decorator from django.views.generic import TemplateView class ProtectedView(TemplateView): template_name = ’secret.html’ 3.6. Class-based views 241
- 246. Django Documentation, Release1.5.1 @method_decorator(login_required) def dispatch(self, *args, **kwargs): return super(ProtectedView, self).dispatch(*args, **kwargs) In this example, every instance of ProtectedView will have login protection. Note: method_decorator passes *args and **kwargs as parameters to the decorated method on the class. If your method does not accept a compatible set of parameters it will raise a TypeError exception. 3.6.2 Class-based generic views WritingWeb applications can be monotonous, because we repeat certain patterns again and again. Django tries to take away some of that monotony at the model and template layers, but Web developers also experience this boredom at the view level. Django’s generic views were developed to ease that pain. They take certain common idioms and patterns found in view development and abstract them so that you can quickly write common views of data without having to write too much code. We can recognize certain common tasks, like displaying a list of objects, and write code that displays a list of any object. Then the model in question can be passed as an extra argument to the URLconf. Django ships with generic views to do the following: • Display list and detail pages for a single object. If we were creating an application to manage conferences then a TalkListView and a RegisteredUserListView would be examples of list views. A single talk page is an example of what we call a “detail” view. • Present date-based objects in year/month/day archive pages, associated detail, and “latest” pages. • Allow users to create, update, and delete objects – with or without authorization. Taken together, these views provide easy interfaces to perform the most common tasks developers encounter. Extending generic views There’s no question that using generic views can speed up development substantially. In most projects, however, there comes a moment when the generic views no longer suffice. Indeed, the most common question asked by new Django developers is how to make generic views handle a wider array of situations. This is one of the reasons generic views were redesigned for the 1.3 release - previously, they were just view functions with a bewildering array of options; now, rather than passing in a large amount of configuration in the URLconf, the recommended way to extend generic views is to subclass them, and override their attributes or methods. That said, generic views will have a limit. If you find you’re struggling to implement your view as a subclass of a generic view, then you may find it more effective to write just the code you need, using your own class-based or functional views. More examples of generic views are available in some third party applications, or you could write your own as needed. Generic views of objects TemplateView certainly is useful, but Django’s generic views really shine when it comes to presenting views of your database content. Because it’s such a common task, Django comes with a handful of built-in generic views that make generating list and detail views of objects incredibly easy. 242 Chapter 3. Using Django
- 247. Django Documentation, Release1.5.1 Let’s start by looking at some examples of showing a list of objects or an individual object. We’ll be using these models: # models.py from django.db import models class Publisher(models.Model): name = models.CharField(max_length=30) address = models.CharField(max_length=50) city = models.CharField(max_length=60) state_province = models.CharField(max_length=30) country = models.CharField(max_length=50) website = models.URLField() class Meta: ordering = ["-name"] def __unicode__(self): return self.name class Book(models.Model): title = models.CharField(max_length=100) authors = models.ManyToManyField(’Author’) publisher = models.ForeignKey(Publisher) publication_date = models.DateField() Now we need to define a view: # views.py from django.views.generic import ListView from books.models import Publisher class PublisherList(ListView): model = Publisher Finally hook that view into your urls: # urls.py from django.conf.urls import patterns, url from books.views import PublisherList urlpatterns = patterns(’’, url(r’^publishers/$’, PublisherList.as_view()), ) That’s all the Python code we need to write. We still need to write a template, however. We could explicitly tell the view which template to use by adding a template_name attribute to the view, but in the absence of an explicit template Django will infer one from the object’s name. In this case, the inferred template will be "books/publisher_list.html" – the “books” part comes from the name of the app that defines the model, while the “publisher” bit is just the lowercased version of the model’s name. Note: Thus, when (for example) the django.template.loaders.app_directories.Loader template loader is enabled in TEMPLATE_LOADERS, a template location could be: /path/to/project/books/templates/books/publisher_list.html This template will be rendered against a context containing a variable called object_list that contains all the publisher objects. A very simple template might look like the following: 3.6. Class-based views 243
- 248. Django Documentation, Release1.5.1 {% extends "base.html" %} {% block content %} <h2>Publishers</h2> <ul> {% for publisher in object_list %} <li>{{ publisher.name }}</li> {% endfor %} </ul> {% endblock %} That’s really all there is to it. All the cool features of generic views come from changing the attributes set on the generic view. The generic views reference documents all the generic views and their options in detail; the rest of this document will consider some of the common ways you might customize and extend generic views. Making “friendly” template contexts You might have noticed that our sample publisher list template stores all the publishers in a variable named object_list. While this works just fine, it isn’t all that “friendly” to template authors: they have to “just know” that they’re dealing with publishers here. Well, if you’re dealing with a model object, this is already done for you. When you are dealing with an object or queryset, Django is able to populate the context using the lower cased version of the model class’ name. This is provided in addition to the default object_list entry, but contains exactly the same data, i.e. publisher_list. If this still isn’t a good match, you can manually set the name of the context variable. The context_object_name attribute on a generic view specifies the context variable to use: # views.py from django.views.generic import ListView from books.models import Publisher class PublisherList(ListView): model = Publisher context_object_name = ’my_favourite_publishers’ Providing a useful context_object_name is always a good idea. Your coworkers who design templates will thank you. Adding extra context Often you simply need to present some extra information beyond that provided by the generic view. For example, think of showing a list of all the books on each publisher detail page. The DetailView generic view provides the publisher to the context, but how do we get additional information in that template. However, there is; you can subclass DetailView and provide your own implementation of the get_context_data method. The default implementation of this that comes with DetailView simply adds in the object being displayed to the template, but you can override it to send more: from django.views.generic import DetailView from books.models import Publisher, Book class PublisherDetail(DetailView): model = Publisher 244 Chapter 3. Using Django
- 249. Django Documentation, Release1.5.1 def get_context_data(self, **kwargs): # Call the base implementation first to get a context context = super(PublisherDetail, self).get_context_data(**kwargs) # Add in a QuerySet of all the books context[’book_list’] = Book.objects.all() return context Note: Generally, get_context_data will merge the context data of all parent classes with those of the current class. To preserve this behavior in your own classes where you want to alter the context, you should be sure to call get_context_data on the super class. When no two classes try to define the same key, this will give the expected results. However if any class attempts to override a key after parent classes have set it (after the call to super), any children of that class will also need to explictly set it after super if they want to be sure to override all parents. If you’re having trouble, review the method resolution order of your view. Viewing subsets of objects Now let’s take a closer look at the model argument we’ve been using all along. The model argument, which specifies the database model that the view will operate upon, is available on all the generic views that operate on a single object or a collection of objects. However, the model argument is not the only way to specify the objects that the view will operate upon – you can also specify the list of objects using the queryset argument: from django.views.generic import DetailView from books.models import Publisher, Book class PublisherDetail(DetailView): context_object_name = ’publisher’ queryset = Publisher.objects.all() Specifying model = Publisher is really just shorthand for saying queryset = Publisher.objects.all(). However, by using queryset to define a filtered list of objects you can be more specific about the objects that will be visible in the view (see Making queries for more information about QuerySet objects, and see the class-based views reference for the complete details). To pick a simple example, we might want to order a list of books by publication date, with the most recent first: from django.views.generic import ListView from books.models import Book class BookList(ListView): queryset = Book.objects.order_by(’-publication_date’) context_object_name = ’book_list’ That’s a pretty simple example, but it illustrates the idea nicely. Of course, you’ll usually want to do more than just reorder objects. If you want to present a list of books by a particular publisher, you can use the same technique: from django.views.generic import ListView from books.models import Book class AcmeBookList(ListView): context_object_name = ’book_list’ queryset = Book.objects.filter(publisher__name=’Acme Publishing’) template_name = ’books/acme_list.html’ 3.6. Class-based views 245
- 250. Django Documentation, Release1.5.1 Notice that along with a filtered queryset, we’re also using a custom template name. If we didn’t, the generic view would use the same template as the “vanilla” object list, which might not be what we want. Also notice that this isn’t a very elegant way of doing publisher-specific books. If we want to add another publisher page, we’d need another handful of lines in the URLconf, and more than a few publishers would get unreasonable. We’ll deal with this problem in the next section. Note: If you get a 404 when requesting /books/acme/, check to ensure you actually have a Publisher with the name ‘ACME Publishing’. Generic views have an allow_empty parameter for this case. See the class-based-views reference for more details. Dynamic filtering Another common need is to filter down the objects given in a list page by some key in the URL. Earlier we hard-coded the publisher’s name in the URLconf, but what if we wanted to write a view that displayed all the books by some arbitrary publisher? Handily, the ListView has a get_queryset() method we can override. Previously, it has just been returning the value of the queryset attribute, but now we can add more logic. The key part to making this work is that when class-based views are called, various useful things are stored on self; as well as the request (self.request) this includes the positional (self.args) and name-based (self.kwargs) arguments captured according to the URLconf. Here, we have a URLconf with a single captured group: # urls.py from books.views import PublisherBookList urlpatterns = patterns(’’, (r’^books/([w-]+)/$’, PublisherBookList.as_view()), ) Next, we’ll write the PublisherBookList view itself: # views.py from django.shortcuts import get_object_or_404 from django.views.generic import ListView from books.models import Book, Publisher class PublisherBookList(ListView): template_name = ’books/books_by_publisher.html’ def get_queryset(self): self.publisher = get_object_or_404(Publisher, name=self.args[0]) return Book.objects.filter(publisher=self.publisher) As you can see, it’s quite easy to add more logic to the queryset selection; if we wanted, we could use self.request.user to filter using the current user, or other more complex logic. We can also add the publisher into the context at the same time, so we can use it in the template: # ... def get_context_data(self, **kwargs): # Call the base implementation first to get a context context = super(PublisherBookList, self).get_context_data(**kwargs) 246 Chapter 3. Using Django
- 251. Django Documentation, Release1.5.1 # Add in the publisher context[’publisher’] = self.publisher return context Performing extra work The last common pattern we’ll look at involves doing some extra work before or after calling the generic view. Imagine we had a last_accessed field on our Author object that we were using to keep track of the last time anybody looked at that author: # models.py class Author(models.Model): salutation = models.CharField(max_length=10) name = models.CharField(max_length=200) email = models.EmailField() headshot = models.ImageField(upload_to=’/tmp’) last_accessed = models.DateTimeField() The generic DetailView class, of course, wouldn’t know anything about this field, but once again we could easily write a custom view to keep that field updated. First, we’d need to add an author detail bit in the URLconf to point to a custom view: from books.views import AuthorDetailView urlpatterns = patterns(’’, #... url(r’^authors/(?P<pk>d+)/$’, AuthorDetailView.as_view(), name=’author-detail’), ) Then we’d write our new view – get_object is the method that retrieves the object – so we simply override it and wrap the call: from django.views.generic import DetailView from django.shortcuts import get_object_or_404 from django.utils import timezone from books.models import Author class AuthorDetailView(DetailView): queryset = Author.objects.all() def get_object(self): # Call the superclass object = super(AuthorDetailView, self).get_object() # Record the last accessed date object.last_accessed = timezone.now() object.save() # Return the object return object Note: The URLconf here uses the named group pk - this name is the default name that DetailView uses to find the value of the primary key used to filter the queryset. If you want to call the group something else, you can set pk_url_kwarg on the view. More details can be found in the reference for DetailView 3.6. Class-based views 247
- 252. Django Documentation, Release1.5.1 3.6.3 Form handling with class-based views Form processing generally has 3 paths: • Initial GET (blank or prepopulated form) • POST with invalid data (typically redisplay form with errors) • POST with valid data (process the data and typically redirect) Implementing this yourself often results in a lot of repeated boilerplate code (see Using a form in a view). To help avoid this, Django provides a collection of generic class-based views for form processing. Basic Forms Given a simple contact form: # forms.py from django import forms class ContactForm(forms.Form): name = forms.CharField() message = forms.CharField(widget=forms.Textarea) def send_email(self): # send email using the self.cleaned_data dictionary pass The view can be constructed using a FormView: # views.py from myapp.forms import ContactForm from django.views.generic.edit import FormView class ContactView(FormView): template_name = ’contact.html’ form_class = ContactForm success_url = ’/thanks/’ def form_valid(self, form): # This method is called when valid form data has been POSTed. # It should return an HttpResponse. form.send_email() return super(ContactView, self).form_valid(form) Notes: • FormView inherits TemplateResponseMixin so template_name can be used here. • The default implementation for form_valid() simply redirects to the success_url. Model Forms Generic views really shine when working with models. These generic views will automatically create a ModelForm, so long as they can work out which model class to use: 248 Chapter 3. Using Django
- 253. Django Documentation, Release1.5.1 • If the model attribute is given, that model class will be used. • If get_object() returns an object, the class of that object will be used. • If a queryset is given, the model for that queryset will be used. Model form views provide a form_valid() implementation that saves the model automatically. You can override this if you have any special requirements; see below for examples. You don’t even need to provide a success_url for CreateView or UpdateView - they will use get_absolute_url() on the model object if available. If you want to use a custom ModelForm (for instance to add extra validation) simply set form_class on your view. Note: When specifying a custom form class, you must still specify the model, even though the form_class may be a ModelForm. First we need to add get_absolute_url() to our Author class: # models.py from django.core.urlresolvers import reverse from django.db import models class Author(models.Model): name = models.CharField(max_length=200) def get_absolute_url(self): return reverse(’author-detail’, kwargs={’pk’: self.pk}) Then we can use CreateView and friends to do the actual work. Notice how we’re just configuring the generic class-based views here; we don’t have to write any logic ourselves: # views.py from django.views.generic.edit import CreateView, UpdateView, DeleteView from django.core.urlresolvers import reverse_lazy from myapp.models import Author class AuthorCreate(CreateView): model = Author class AuthorUpdate(UpdateView): model = Author class AuthorDelete(DeleteView): model = Author success_url = reverse_lazy(’author-list’) Note: We have to use reverse_lazy() here, not just reverse as the urls are not loaded when the file is imported. Finally, we hook these new views into the URLconf: # urls.py from django.conf.urls import patterns, url from myapp.views import AuthorCreate, AuthorUpdate, AuthorDelete urlpatterns = patterns(’’, 3.6. Class-based views 249
- 254. Django Documentation, Release1.5.1 # ... url(r’author/add/$’, AuthorCreate.as_view(), name=’author_add’), url(r’author/(?P<pk>d+)/$’, AuthorUpdate.as_view(), name=’author_update’), url(r’author/(?P<pk>d+)/delete/$’, AuthorDelete.as_view(), name=’author_delete’), ) Note: These views inherit SingleObjectTemplateResponseMixin which uses template_name_suffix to construct the template_name based on the model. In this example: • CreateView and UpdateView use myapp/author_form.html • DeleteView uses myapp/author_confirm_delete.html If you wish to have separate templates for CreateView and UpdateView, you can set either template_name or template_name_suffix on your view class. Models and request.user To track the user that created an object using a CreateView, you can use a custom ModelForm to do this. First, add the foreign key relation to the model: # models.py from django.contrib.auth import User from django.db import models class Author(models.Model): name = models.CharField(max_length=200) created_by = models.ForeignKey(User) # ... Create a custom ModelForm in order to exclude the created_by field and prevent the user from editing it: # forms.py from django import forms from myapp.models import Author class AuthorForm(forms.ModelForm): class Meta: model = Author exclude = (’created_by’,) In the view, use the custom form_class and override form_valid() to add the user: # views.py from django.views.generic.edit import CreateView from myapp.models import Author from myapp.forms import AuthorForm class AuthorCreate(CreateView): form_class = AuthorForm model = Author def form_valid(self, form): form.instance.created_by = self.request.user return super(AuthorCreate, self).form_valid(form) 250 Chapter 3. Using Django
- 255. Django Documentation, Release1.5.1 Note that you’ll need to decorate this view using login_required(), or alternatively handle unauthorized users in the form_valid(). AJAX example Here is a simple example showing how you might go about implementing a form that works for AJAX requests as well as ‘normal’ form POSTs: import json from django.http import HttpResponse from django.views.generic.edit import CreateView class AjaxableResponseMixin(object): """ Mixin to add AJAX support to a form. Must be used with an object-based FormView (e.g. CreateView) """ def render_to_json_response(self, context, **response_kwargs): data = json.dumps(context) response_kwargs[’content_type’] = ’application/json’ return HttpResponse(data, **response_kwargs) def form_invalid(self, form): response = super(AjaxableResponseMixin, self).form_invalid(form) if self.request.is_ajax(): return self.render_to_json_response(form.errors, status=400) else: return response def form_valid(self, form): # We make sure to call the parent’s form_valid() method because # it might do some processing (in the case of CreateView, it will # call form.save() for example). response = super(AjaxableResponseMixin, self).form_valid(form) if self.request.is_ajax(): data = { ’pk’: self.object.pk, } return self.render_to_json_response(data) else: return response class AuthorCreate(AjaxableResponseMixin, CreateView): model = Author 3.6.4 Using mixins with class-based views Caution: This is an advanced topic. A working knowledge of Django’s class-based views is advised before exploring these techniques. Django’s built-in class-based views provide a lot of functionality, but some of it you may want to use separately. For instance, you may want to write a view that renders a template to make the HTTP response, but you can’t use TemplateView; perhaps you need to render a template only on POST, with GET doing something else entirely. While you could use TemplateResponse directly, this will likely result in duplicate code. 3.6. Class-based views 251
- 256. Django Documentation, Release1.5.1 For this reason, Django also provides a number of mixins that provide more discrete functionality. Template rendering, for instance, is encapsulated in the TemplateResponseMixin. The Django reference documentation contains full documentation of all the mixins. Context and template responses Two central mixins are provided that help in providing a consistent interface to working with templates in class-based views. TemplateResponseMixin Every built in view which returns a TemplateResponse will call the render_to_response() method that TemplateResponseMixin provides. Most of the time this will be called for you (for instance, it is called by the get() method implemented by both TemplateView and DetailView); similarly, it’s unlikely that you’ll need to override it, although if you want your response to return something not rendered via a Django template then you’ll want to do it. For an example of this, see the JSONResponseMixin example. render_to_response itself calls get_template_names(), which by default will just look up template_name on the class-based view; two other mixins (SingleObjectTemplateResponseMixin and MultipleObjectTemplateResponseMixin) override this to provide more flexible defaults when dealing with actual objects. New in version 1.5. ContextMixin Every built in view which needs context data, such as for rendering a template (including TemplateResponseMixin above), should call get_context_data() passing any data they want to ensure is in there as keyword arguments. get_context_data returns a dictionary; in ContextMixin it simply returns its keyword arguments, but it is common to override this to add more members to the dictionary. Building up Django’s generic class-based views Let’s look at how two of Django’s generic class-based views are built out of mixins providing discrete functionality. We’ll consider DetailView, which renders a “detail” view of an object, and ListView, which will render a list of objects, typically from a queryset, and optionally paginate them. This will introduce us to four mixins which between them provide useful functionality when working with either a single Django object, or multiple objects. There are also mixins involved in the generic edit views (FormView, and the model-specific views CreateView, UpdateView and DeleteView), and in the date-based generic views. These are covered in the mixin reference documentation. DetailView: working with a single Django object To show the detail of an object, we basically need to do two things: we need to look up the object and then we need to make a TemplateResponse with a suitable template, and that object as context. To get the object, DetailView relies on SingleObjectMixin, which provides a get_object() method that figures out the object based on the URL of the request (it looks for pk and slug keyword arguments as declared in the URLConf, and looks the object up either from the model attribute on the view, or the queryset attribute if that’s provided). SingleObjectMixin also overrides get_context_data(), which is used across all Django’s built in class-based views to supply context data for template renders. To then make a TemplateResponse, DetailView uses SingleObjectTemplateResponseMixin, which extends TemplateResponseMixin, overriding get_template_names() as discussed above. It actually provides a fairly sophisticated set of options, but the main one that most people are going to use is <app_label>/<object_name>_detail.html. The _detail part can be changed by setting 252 Chapter 3. Using Django
- 257. Django Documentation, Release1.5.1 template_name_suffix on a subclass to something else. (For instance, the generic edit views use _form for create and update views, and _confirm_delete for delete views.) ListView: working with many Django objects Lists of objects follow roughly the same pattern: we need a (possibly paginated) list of objects, typically a QuerySet, and then we need to make a TemplateResponse with a suitable template using that list of objects. To get the objects, ListView uses MultipleObjectMixin, which provides both get_queryset() and paginate_queryset(). Unlike with SingleObjectMixin, there’s no need to key off parts of the URL to figure out the queryset to work with, so the default just uses the queryset or model attribute on the view class. A common reason to override get_queryset() here would be to dynamically vary the objects, such as depending on the current user or to exclude posts in the future for a blog. MultipleObjectMixin also overrides get_context_data() to include appropriate context variables for pagination (providing dummies if pagination is disabled). It relies on object_list being passed in as a keyword argument, which ListView arranges for it. To make a TemplateResponse, ListView then uses MultipleObjectTemplateResponseMixin; as with SingleObjectTemplateResponseMixin above, this overrides get_template_names() to provide a range of options, with the most commonly-used being <app_label>/<object_name>_list.html, with the _list part again being taken from the template_name_suffix attribute. (The date based generic views use suffixes such as _archive, _archive_year and so on to use different templates for the various specialised date-based list views.) Using Django’s class-based view mixins Now we’ve seen how Django’s generic class-based views use the provided mixins, let’s look at other ways we can combine them. Of course we’re still going to be combining them with either built-in class-based views, or other generic class-based views, but there are a range of rarer problems you can solve than are provided for by Django out of the box. Warning: Not all mixins can be used together, and not all generic class based views can be used with all other mixins. Here we present a few examples that do work; if you want to bring together other functionality then you’ll have to consider interactions between attributes and methods that overlap between the different classes you’re using, and how method resolution order will affect which versions of the methods will be called in what order. The reference documentation for Django’s class-based views and class-based view mixins will help you in under-standing which attributes and methods are likely to cause conflict between different classes and mixins. If in doubt, it’s often better to back off and base your work on View or TemplateView, perhaps with SingleObjectMixin and MultipleObjectMixin. Although you will probably end up writing more code, it is more likely to be clearly understandable to someone else coming to it later, and with fewer interactions to worry about you will save yourself some thinking. (Of course, you can always dip into Django’s implementation of the generic class based views for inspiration on how to tackle problems.) Using SingleObjectMixin with View If we want to write a simple class-based view that responds only to POST, we’ll subclass View and write a post() method in the subclass. However if we want our processing to work on a particular object, identified from the URL, we’ll want the functionality provided by SingleObjectMixin. We’ll demonstrate this with the publisher modelling we used in the generic class-based views introduction. 3.6. Class-based views 253
- 258. Django Documentation, Release1.5.1 # views.py from django.http import HttpResponseForbidden, HttpResponseRedirect from django.core.urlresolvers import reverse from django.views.generic import View from django.views.generic.detail import SingleObjectMixin from books.models import Author class RecordInterest(View, SingleObjectMixin): """Records the current user’s interest in an author.""" model = Author def post(self, request, *args, **kwargs): if not request.user.is_authenticated(): return HttpResponseForbidden() # Look up the author we’re interested in. self.object = self.get_object() # Actually record interest somehow here! return HttpResponseRedirect(reverse(’author-detail’, kwargs={’pk’: self.object.pk})) In practice you’d probably want to record the interest in a key-value store rather than in a relational database, so we’ve left that bit out. The only bit of the view that needs to worry about using SingleObjectMixin is where we want to look up the author we’re interested in, which it just does with a simple call to self.get_object(). Everything else is taken care of for us by the mixin. We can hook this into our URLs easily enough: # urls.py from books.views import RecordInterest urlpatterns = patterns(’’, #... url(r’^author/(?P<pk>d+)/interest/$’, RecordInterest.as_view(), name=’author-interest’), ) Note the pk named group, which get_object() uses to look up the Author instance. You could also use a slug, or any of the other features of SingleObjectMixin. Using SingleObjectMixin with ListView ListView provides built-in pagination, but you might want to paginate a list of objects that are all linked (by a foreign key) to another object. In our publishing example, you might want to paginate through all the books by a particular publisher. One way to do this is to combine ListView with SingleObjectMixin, so that the queryset for the paginated list of books can hang off the publisher found as the single object. In order to do this, we need to have two different querysets: Publisher queryset for use in get_object We’ll set that up directly when we call get_object(). Book queryset for use by ListView We’ll figure that out ourselves in get_queryset() so we can take into ac-count the Publisher we’re looking at. Note: We have to think carefully about get_context_data(). Since both SingleObjectMixin and ListView will put things in the context data under the value of context_object_name if it’s set, we’ll instead 254 Chapter 3. Using Django
- 259. Django Documentation, Release1.5.1 explictly ensure the Publisher is in the context data. ListView will add in the suitable page_obj and paginator for us providing we remember to call super(). Now we can write a new PublisherDetail: from django.views.generic import ListView from django.views.generic.detail import SingleObjectMixin from books.models import Publisher class PublisherDetail(SingleObjectMixin, ListView): paginate_by = 2 template_name = "books/publisher_detail.html" def get_context_data(self, **kwargs): kwargs[’publisher’] = self.object return super(PublisherDetail, self).get_context_data(**kwargs) def get_queryset(self): self.object = self.get_object(Publisher.objects.all()) return self.object.book_set.all() Notice how we set self.object within get_queryset() so we can use it again later in get_context_data(). If you don’t set template_name, the template will default to the normal ListView choice, which in this case would be "books/book_list.html" because it’s a list of books; ListView knows nothing about SingleObjectMixin, so it doesn’t have any clue this view is anything to do with a Publisher. The paginate_by is deliberately small in the example so you don’t have to create lots of books to see the pagination working! Here’s the template you’d want to use: {% extends "base.html" %} {% block content %} <h2>Publisher {{ publisher.name }}</h2> <ol> {% for book in page_obj %} <li>{{ book.title }}</li> {% endfor %} </ol> <div class="pagination"> <span class="step-links"> {% if page_obj.has_previous %} <a href="?page={{ page_obj.previous_page_number }}">previous</a> {% endif %} <span class="current"> Page {{ page_obj.number }} of {{ paginator.num_pages }}. </span> {% if page_obj.has_next %} <a href="?page={{ page_obj.next_page_number }}">next</a> {% endif %} </span> </div> {% endblock %} 3.6. Class-based views 255
- 260. Django Documentation, Release1.5.1 Avoid anything more complex Generally you can use TemplateResponseMixin and SingleObjectMixin when you need their functional-ity. As shown above, with a bit of care you can even combine SingleObjectMixin with ListView. However things get increasingly complex as you try to do so, and a good rule of thumb is: Hint: Each of your views should use only mixins or views from one of the groups of generic class-based views: detail, list, editing and date. For example it’s fine to combine TemplateView (built in view) with MultipleObjectMixin (generic list), but you’re likely to have problems combining SingleObjectMixin (generic detail) with MultipleObjectMixin (generic list). To show what happens when you try to get more sophisticated, we show an example that sacrifices readability and maintainability when there is a simpler solution. First, let’s look at a naive attempt to combine DetailView with FormMixin to enable use to POST a Django Form to the same URL as we’re displaying an object using DetailView. Using FormMixin with DetailView Think back to our earlier example of using View and SingleObjectMixin together. We were recording a user’s interest in a particular author; say now that we want to let them leave a message saying why they like them. Again, let’s assume we’re not going to store this in a relational database but instead in something more esoteric that we won’t worry about here. At this point it’s natural to reach for a Form to encapsulate the information sent from the user’s browser to Django. Say also that we’re heavily invested in REST, so we want to use the same URL for displaying the author as for capturing the message from the user. Let’s rewrite our AuthorDetailView to do that. We’ll keep the GET handling from DetailView, although we’ll have to add a Form into the context data so we can render it in the template. We’ll also want to pull in form processing from FormMixin, and write a bit of code so that on POST the form gets called appropriately. Note: We use FormMixin and implement post() ourselves rather than try to mix DetailView with FormView (which provides a suitable post() already) because both of the views implement get(), and things would get much more confusing. Our new AuthorDetail looks like this: # CAUTION: you almost certainly do not want to do this. # It is provided as part of a discussion of problems you can # run into when combining different generic class-based view # functionality that is not designed to be used together. from django import forms from django.http import HttpResponseForbidden from django.core.urlresolvers import reverse from django.views.generic import DetailView from django.views.generic.edit import FormMixin class AuthorInterestForm(forms.Form): message = forms.CharField() class AuthorDetail(DetailView, FormMixin): model = Author form_class = AuthorInterestForm 256 Chapter 3. Using Django
- 261. Django Documentation, Release1.5.1 def get_success_url(self): return reverse( ’author-detail’, kwargs = {’pk’: self.object.pk}, ) def get_context_data(self, **kwargs): form_class = self.get_form_class() form = self.get_form(form_class) context = { ’form’: form } context.update(kwargs) return super(AuthorDetail, self).get_context_data(**context) def post(self, request, *args, **kwargs): form_class = self.get_form_class() form = self.get_form(form_class) if form.is_valid(): return self.form_valid(form) else: return self.form_invalid(form) def form_valid(self, form): if not self.request.user.is_authenticated(): return HttpResponseForbidden() self.object = self.get_object() # record the interest using the message in form.cleaned_data return super(AuthorDetail, self).form_valid(form) get_success_url() is just providing somewhere to redirect to, which gets used in the default implementation of form_valid(). We have to provide our own post() as noted earlier, and override get_context_data() to make the Form available in the context data. A better solution It should be obvious that the number of subtle interactions between FormMixin and DetailView is already testing our ability to manage things. It’s unlikely you’d want to write this kind of class yourself. In this case, it would be fairly easy to just write the post() method yourself, keeping DetailView as the only generic functionality, although writing Form handling code involves a lot of duplication. Alternatively, it would still be easier than the above approach to have a separate view for processing the form, which could use FormView distinct from DetailView without concerns. An alternative better solution What we’re really trying to do here is to use two different class based views from the same URL. So why not do just that? We have a very clear division here: GET requests should get the DetailView (with the Form added to the context data), and POST requests should get the FormView. Let’s set up those views first. The AuthorDisplay view is almost the same as when we first introduced AuthorDetail; we have to write our own get_context_data() to make the AuthorInterestForm available to the template. We’ll skip the get_object() override from before for clarity. 3.6. Class-based views 257
- 262. Django Documentation, Release1.5.1 from django.views.generic import DetailView from django import forms from books.models import Author class AuthorInterestForm(forms.Form): message = forms.CharField() class AuthorDisplay(DetailView): queryset = Author.objects.all() def get_context_data(self, **kwargs): context = { ’form’: AuthorInterestForm(), } context.update(kwargs) return super(AuthorDisplay, self).get_context_data(**context) Then the AuthorInterest is a simple FormView, but we have to bring in SingleObjectMixin so we can find the author we’re talking about, and we have to remember to set template_name to ensure that form errors will render the same template as AuthorDisplay is using on GET. from django.views.generic import FormView from django.views.generic.detail import SingleObjectMixin class AuthorInterest(FormView, SingleObjectMixin): template_name = ’books/author_detail.html’ form_class = AuthorInterestForm model = Author def get_context_data(self, **kwargs): context = { ’object’: self.get_object(), } return super(AuthorInterest, self).get_context_data(**context) def get_success_url(self): return reverse( ’author-detail’, kwargs = {’pk’: self.object.pk}, ) def form_valid(self, form): if not self.request.user.is_authenticated(): return HttpResponseForbidden() self.object = self.get_object() # record the interest using the message in form.cleaned_data return super(AuthorInterest, self).form_valid(form) Finally we bring this together in a new AuthorDetail view. We already know that calling as_view() on a class-based view gives us something that behaves exactly like a function based view, so we can do that at the point we choose between the two subviews. You can of course pass through keyword arguments to as_view() in the same way you would in your URLconf, such as if you wanted the AuthorInterest behavior to also appear at another URL but using a different template. from django.views.generic import View class AuthorDetail(View): 258 Chapter 3. Using Django
- 263. Django Documentation, Release1.5.1 def get(self, request, *args, **kwargs): view = AuthorDisplay.as_view() return view(request, *args, **kwargs) def post(self, request, *args, **kwargs): view = AuthorInterest.as_view() return view(request, *args, **kwargs) This approach can also be used with any other generic class-based views or your own class-based views inheriting directly from View or TemplateView, as it keeps the different views as separate as possible. More than just HTML Where class based views shine is when you want to do the same thing many times. Suppose you’re writing an API, and every view should return JSON instead of rendered HTML. We can create a mixin class to use in all of our views, handling the conversion to JSON once. For example, a simple JSON mixin might look something like this: import json from django.http import HttpResponse class JSONResponseMixin(object): """ A mixin that can be used to render a JSON response. """ response_class = HttpResponse def render_to_response(self, context, **response_kwargs): """ Returns a JSON response, transforming ’context’ to make the payload. """ response_kwargs[’content_type’] = ’application/json’ return self.response_class( self.convert_context_to_json(context), **response_kwargs ) def convert_context_to_json(self, context): "Convert the context dictionary into a JSON object" # Note: This is *EXTREMELY* naive; in reality, you’ll need # to do much more complex handling to ensure that arbitrary # objects -- such as Django model instances or querysets # -- can be serialized as JSON. return json.dumps(context) Now we mix this into the base TemplateView: from django.views.generic import TemplateView class JSONView(JSONResponseMixin, TemplateView): pass Equally we could use our mixin with one of the generic views. We can make our own version of DetailView by mixing JSONResponseMixin with the django.views.generic.detail.BaseDetailView – (the DetailView before template rendering behavior has been mixed in): 3.6. Class-based views 259
- 264. Django Documentation, Release1.5.1 class JSONDetailView(JSONResponseMixin, BaseDetailView): pass This view can then be deployed in the same way as any other DetailView, with exactly the same behavior – except for the format of the response. If you want to be really adventurous, you could even mix a DetailView subclass that is able to return both HTML and JSON content, depending on some property of the HTTP request, such as a query argument or a HTTP header. Just mix in both the JSONResponseMixin and a SingleObjectTemplateResponseMixin, and override the implementation of render_to_response() to defer to the appropriate subclass depending on the type of response that the user requested: class HybridDetailView(JSONResponseMixin, SingleObjectTemplateResponseMixin, BaseDetailView): def render_to_response(self, context): # Look for a ’format=json’ GET argument if self.request.GET.get(’format’,’html’) == ’json’: return JSONResponseMixin.render_to_response(self, context) else: return SingleObjectTemplateResponseMixin.render_to_response(self, context) Because of the way that Python resolves method overloading, the local render_to_response() implementation will override the versions provided by JSONResponseMixin and SingleObjectTemplateResponseMixin. 3.6.5 Basic examples Django provides base view classes which will suit a wide range of applications. All views inherit from the View class, which handles linking the view in to the URLs, HTTP method dispatching and other simple features. RedirectView is for a simple HTTP redirect, and TemplateView extends the base class to make it also render a template. 3.6.6 Simple usage in your URLconf The simplest way to use generic views is to create them directly in your URLconf. If you’re only changing a few simple attributes on a class-based view, you can simply pass them into the as_view() method call itself: from django.conf.urls import patterns from django.views.generic import TemplateView urlpatterns = patterns(’’, (r’^about/’, TemplateView.as_view(template_name="about.html")), ) Any arguments passed to as_view() will override attributes set on the class. In this example, we set template_name on the TemplateView. A similar overriding pattern can be used for the url attribute on RedirectView. 3.6.7 Subclassing generic views The second, more powerful way to use generic views is to inherit from an existing view and override attributes (such as the template_name) or methods (such as get_context_data) in your subclass to provide new values or methods. Consider, for example, a view that just displays one template, about.html. Django has a generic view to do this - TemplateView - so we can just subclass it, and override the template name: 260 Chapter 3. Using Django
- 265. Django Documentation, Release1.5.1 # some_app/views.py from django.views.generic import TemplateView class AboutView(TemplateView): template_name = "about.html" Then we just need to add this new view into our URLconf. ~django.views.generic.base.TemplateView is a class, not a function, so we point the URL to the as_view() class method instead, which provides a function-like entry to class-based views: # urls.py from django.conf.urls import patterns from some_app.views import AboutView urlpatterns = patterns(’’, (r’^about/’, AboutView.as_view()), ) For more information on how to use the built in generic views, consult the next topic on generic class based views. Supporting other HTTP methods Suppose somebody wants to access our book library over HTTP using the views as an API. The API client would connect every now and then and download book data for the books published since last visit. But if no new books appeared since then, it is a waste of CPU time and bandwidth to fetch the books from the database, render a full response and send it to the client. It might be preferable to ask the API when the most recent book was published. We map the URL to book list view in the URLconf: from django.conf.urls import patterns from books.views import BookListView urlpatterns = patterns(’’, (r’^books/$’, BookListView.as_view()), ) And the view: from django.http import HttpResponse from django.views.generic import ListView from books.models import Book class BookListView(ListView): model = Book def head(self, *args, **kwargs): last_book = self.get_queryset().latest(’publication_date’) response = HttpResponse(’’) # RFC 1123 date format response[’Last-Modified’] = last_book.publication_date.strftime(’%a, %d %b %Y %H:%M:%S GMT’) return response If the view is accessed from a GET request, a plain-and-simple object list is returned in the response (using book_list.html template). But if the client issues a HEAD request, the response has an empty body and the Last-Modified header indicates when the most recent book was published. Based on this information, the client may or may not download the full object list. 3.6. Class-based views 261
- 266. Django Documentation, Release1.5.1 3.7 Managing files This document describes Django’s file access APIs for files such as those uploaded by a user. The lower level APIs are general enough that you could use them for other purposes. If you want to handle “static files” (JS, CSS, etc), see Managing static files (CSS, images). By default, Django stores files locally, using the MEDIA_ROOT and MEDIA_URL settings. The examples below assume that you’re using these defaults. However, Django provides ways to write custom file storage systems that allow you to completely customize where and how Django stores files. The second half of this document describes how these storage systems work. 3.7.1 Using files in models When you use a FileField or ImageField, Django provides a set of APIs you can use to deal with that file. Consider the following model, using an ImageField to store a photo: class Car(models.Model): name = models.CharField(max_length=255) price = models.DecimalField(max_digits=5, decimal_places=2) photo = models.ImageField(upload_to=’cars’) Any Car instance will have a photo attribute that you can use to get at the details of the attached photo: >>> car = Car.objects.get(name="57 Chevy") >>> car.photo <ImageFieldFile: chevy.jpg> >>> car.photo.name u’cars/chevy.jpg’ >>> car.photo.path u’/media/cars/chevy.jpg’ >>> car.photo.url u’http://media.example.com/cars/chevy.jpg’ This object – car.photo in the example – is a File object, which means it has all the methods and attributes described below. Note: The file is saved as part of saving the model in the database, so the actual file name used on disk cannot be relied on until after the model has been saved. 3.7.2 The File object Internally, Django uses a django.core.files.File instance any time it needs to represent a file. This object is a thin wrapper around Python’s built-in file object with some Django-specific additions. Most of the time you’ll simply use a File that Django’s given you (i.e. a file attached to a model as above, or perhaps an uploaded file). If you need to construct a File yourself, the easiest way is to create one using a Python built-in file object: >>> from django.core.files import File # Create a Python file object using open() >>> f = open(’/tmp/hello.world’, ’w’) >>> myfile = File(f) 262 Chapter 3. Using Django
- 267. Django Documentation, Release1.5.1 Now you can use any of the documented attributes and methods of the File class. Be aware that files created in this way are not automatically closed. The following approach may be used to close files automatically: >>> from django.core.files import File # Create a Python file object using open() and the with statement >>> with open(’/tmp/hello.world’, ’w’) as f: >>> myfile = File(f) >>> for line in myfile: >>> print line >>> myfile.closed True >>> f.closed True Closing files is especially important when accessing file fields in a loop over a large number of objects:: If files are not manually closed after accessing them, the risk of running out of file descriptors may arise. This may lead to the following error: IOError: [Errno 24] Too many open files 3.7.3 File storage Behind the scenes, Django delegates decisions about how and where to store files to a file storage system. This is the object that actually understands things like file systems, opening and reading files, etc. Django’s default file storage is given by the DEFAULT_FILE_STORAGE setting; if you don’t explicitly provide a storage system, this is the one that will be used. See below for details of the built-in default file storage system, and see Writing a custom storage system for information on writing your own file storage system. Storage objects Though most of the time you’ll want to use a File object (which delegates to the proper storage for that file), you can use file storage systems directly. You can create an instance of some custom file storage class, or – often more useful – you can use the global default storage system: >>> from django.core.files.storage import default_storage >>> from django.core.files.base import ContentFile >>> path = default_storage.save(’/path/to/file’, ContentFile(’new content’)) >>> path u’/path/to/file’ >>> default_storage.size(path) 11 >>> default_storage.open(path).read() ’new content’ >>> default_storage.delete(path) >>> default_storage.exists(path) False See File storage API for the file storage API. 3.7. Managing files 263
- 268. Django Documentation, Release1.5.1 The built-in filesystem storage class Django ships with a built-in FileSystemStorage class (defined in django.core.files.storage) which implements basic local filesystem file storage. Its initializer takes two arguments: Argu-ment Description location Optional. Absolute path to the directory that will hold the files. If omitted, it will be set to the value of your MEDIA_ROOT setting. base_url Optional. URL that serves the files stored at this location. If omitted, it will default to the value of your MEDIA_URL setting. For example, the following code will store uploaded files under /media/photos regardless of what your MEDIA_ROOT setting is: from django.db import models from django.core.files.storage import FileSystemStorage fs = FileSystemStorage(location=’/media/photos’) class Car(models.Model): ... photo = models.ImageField(storage=fs) Custom storage systems work the same way: you can pass them in as the storage argument to a FileField. 3.8 Testing in Django 3.8.1 Testing Django applications See Also: The testing tutorial and the advanced testing topics. This document is split into two primary sections. First, we explain how to write tests with Django. Then, we explain how to run them. Writing tests Django’s unit tests use a Python standard library module: unittest. This module defines tests in class-based approach. unittest2 Python 2.7 introduced some major changes to the unittest library, adding some extremely useful features. To ensure that every Django project can benefit from these new features, Django ships with a copy of unittest2, a copy of the Python 2.7 unittest library, backported for Python 2.5 compatibility. To access this library, Django provides the django.utils.unittest module alias. If you are using Python 2.7, or you have installed unittest2 locally, Django will map the alias to the installed version of the unittest library. Otherwise, Django will use its own bundled version of unittest2. To use this alias, simply use: from django.utils import unittest 264 Chapter 3. Using Django
- 269. Django Documentation, Release1.5.1 wherever you would have historically used: import unittest If you want to continue to use the base unittest library, you can – you just won’t get any of the nice new unittest2 features. For a given Django application, the test runner looks for unit tests in two places: • The models.py file. The test runner looks for any subclass of unittest.TestCase in this module. • A file called tests.py in the application directory – i.e., the directory that holds models.py. Again, the test runner looks for any subclass of unittest.TestCase in this module. Here is an example unittest.TestCase subclass: from django.utils import unittest from myapp.models import Animal class AnimalTestCase(unittest.TestCase): def setUp(self): self.lion = Animal(name="lion", sound="roar") self.cat = Animal(name="cat", sound="meow") def test_animals_can_speak(self): """Animals that can speak are correctly identified""" self.assertEqual(self.lion.speak(), ’The lion says "roar"’) self.assertEqual(self.cat.speak(), ’The cat says "meow"’) When you run your tests, the default behavior of the test utility is to find all the test cases (that is, subclasses of unittest.TestCase) in models.py and tests.py, automatically build a test suite out of those test cases, and run that suite. There is a second way to define the test suite for a module: if you define a function called suite() in either models.py or tests.py, the Django test runner will use that function to construct the test suite for that module. This follows the suggested organization for unit tests. See the Python documentation for more details on how to construct a complex test suite. For more details about unittest, see the Python documentation. Warning: If your tests rely on database access such as creating or querying models, be sure to create your test classes as subclasses of django.test.TestCase rather than unittest.TestCase. In the example above, we instantiate some models but do not save them to the database. Using unittest.TestCase avoids the cost of running each test in a transaction and flushing the database, but for most applications the scope of tests you will be able to write this way will be fairly limited, so it’s easiest to use django.test.TestCase. Running tests Once you’ve written tests, run them using the test command of your project’s manage.py utility: $ ./manage.py test By default, this will run every test in every application in INSTALLED_APPS. If you only want to run tests for a particular application, add the application name to the command line. For example, if your INSTALLED_APPS contains ’myproject.polls’ and ’myproject.animals’, you can run the myproject.animals unit tests alone with this command: 3.8. Testing in Django 265
- 270. Django Documentation, Release1.5.1 $ ./manage.py test animals Note that we used animals, not myproject.animals. You can be even more specific by naming an individual test case. To run a single test case in an application (for example, the AnimalTestCase described in the “Writing unit tests” section), add the name of the test case to the label on the command line: $ ./manage.py test animals.AnimalTestCase And it gets even more granular than that! To run a single test method inside a test case, add the name of the test method to the label: $ ./manage.py test animals.AnimalTestCase.test_animals_can_speak You can use the same rules if you’re using doctests. Django will use the test label as a path to the test method or class that you want to run. If your models.py or tests.py has a function with a doctest, or class with a class-level doctest, you can invoke that test by appending the name of the test method or class to the label: $ ./manage.py test animals.classify If you want to run the doctest for a specific method in a class, add the name of the method to the label: $ ./manage.py test animals.Classifier.run If you’re using a __test__ dictionary to specify doctests for a module, Django will use the label as a key in the __test__ dictionary for defined in models.py and tests.py. If you press Ctrl-C while the tests are running, the test runner will wait for the currently running test to complete and then exit gracefully. During a graceful exit the test runner will output details of any test failures, report on how many tests were run and how many errors and failures were encountered, and destroy any test databases as usual. Thus pressing Ctrl-C can be very useful if you forget to pass the --failfast option, notice that some tests are unexpectedly failing, and want to get details on the failures without waiting for the full test run to complete. If you do not want to wait for the currently running test to finish, you can press Ctrl-C a second time and the test run will halt immediately, but not gracefully. No details of the tests run before the interruption will be reported, and any test databases created by the run will not be destroyed. Test with warnings enabled It’s a good idea to run your tests with Python warnings enabled: python -Wall manage.py test. The -Wall flag tells Python to display deprecation warnings. Django, like many other Python libraries, uses these warnings to flag when features are going away. It also might flag areas in your code that aren’t strictly wrong but could benefit from a better implementation. The test database Tests that require a database (namely, model tests) will not use your “real” (production) database. Separate, blank databases are created for the tests. Regardless of whether the tests pass or fail, the test databases are destroyed when all the tests have been executed. By default the test databases get their names by prepending test_ to the value of the NAME settings for the databases defined in DATABASES. When using the SQLite database engine the tests will by default use an in-memory database (i.e., the database will be created in memory, bypassing the filesystem entirely!). If you want to use a different database name, specify TEST_NAME in the dictionary for any given database in DATABASES. 266 Chapter 3. Using Django
- 271. Django Documentation, Release1.5.1 Aside from using a separate database, the test runner will otherwise use all of the same database settings you have in your settings file: ENGINE, USER, HOST, etc. The test database is created by the user specified by USER, so you’ll need to make sure that the given user account has sufficient privileges to create a new database on the system. For fine-grained control over the character encoding of your test database, use the TEST_CHARSET option. If you’re using MySQL, you can also use the TEST_COLLATION option to control the particular collation used by the test database. See the settings documentation for details of these advanced settings. Finding data from your production database when running tests? If your code attempts to access the database when its modules are compiled, this will occur before the test database is set up, with potentially unexpected results. For example, if you have a database query in module-level code and a real database exists, production data could pollute your tests. It is a bad idea to have such import-time database queries in your code anyway - rewrite your code so that it doesn’t do this. See Also: The advanced multi-db testing topics. Order in which tests are executed In order to guarantee that all TestCase code starts with a clean database, the Django test runner reorders tests in the following way: • First, all unittests (including unittest.TestCase, SimpleTestCase, TestCase and TransactionTestCase) are run with no particular ordering guaranteed nor enforced among them. • Then any other tests (e.g. doctests) that may alter the database without restoring it to its original state are run. Changed in version 1.5: Before Django 1.5, the only guarantee was that TestCase tests were always ran first, before any other tests. Note: The new ordering of tests may reveal unexpected dependencies on test case ordering. This is the case with doctests that relied on state left in the database by a given TransactionTestCase test, they must be updated to be able to run independently. Other test conditions Regardless of the value of the DEBUG setting in your configuration file, all Django tests run with DEBUG=False. This is to ensure that the observed output of your code matches what will be seen in a production setting. Caches are not cleared after each test, and running “manage.py test fooapp” can insert data from the tests into the cache of a live system if you run your tests in production because, unlike databases, a separate “test cache” is not used. This behavior may change in the future. Understanding the test output When you run your tests, you’ll see a number of messages as the test runner prepares itself. You can control the level of detail of these messages with the verbosity option on the command line: Creating test database... Creating table myapp_animal Creating table myapp_mineral 3.8. Testing in Django 267
- 272. Django Documentation, Release1.5.1 Loading ’initial_data’ fixtures... No fixtures found. This tells you that the test runner is creating a test database, as described in the previous section. Once the test database has been created, Django will run your tests. If everything goes well, you’ll see something like this: ---------------------------------------------------------------------- Ran 22 tests in 0.221s OK If there are test failures, however, you’ll see full details about which tests failed: ====================================================================== FAIL: Doctest: ellington.core.throttle.models ---------------------------------------------------------------------- Traceback (most recent call last): File "/dev/django/test/doctest.py", line 2153, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for myapp.models File "/dev/myapp/models.py", line 0, in models ---------------------------------------------------------------------- File "/dev/myapp/models.py", line 14, in myapp.models Failed example: throttle.check("actor A", "action one", limit=2, hours=1) Expected: True Got: False ---------------------------------------------------------------------- Ran 2 tests in 0.048s FAILED (failures=1) A full explanation of this error output is beyond the scope of this document, but it’s pretty intuitive. You can consult the documentation of Python’s unittest library for details. Note that the return code for the test-runner script is 1 for any number of failed and erroneous tests. If all the tests pass, the return code is 0. This feature is useful if you’re using the test-runner script in a shell script and need to test for success or failure at that level. Speeding up the tests In recent versions of Django, the default password hasher is rather slow by design. If during your tests you are authenticating many users, you may want to use a custom settings file and set the PASSWORD_HASHERS setting to a faster hashing algorithm: PASSWORD_HASHERS = ( ’django.contrib.auth.hashers.MD5PasswordHasher’, ) Don’t forget to also include in PASSWORD_HASHERS any hashing algorithm used in fixtures, if any. 268 Chapter 3. Using Django
- 273. Django Documentation, Release1.5.1 Testing tools Django provides a small set of tools that come in handy when writing tests. The test client The test client is a Python class that acts as a dummy Web browser, allowing you to test your views and interact with your Django-powered application programmatically. Some of the things you can do with the test client are: • Simulate GET and POST requests on a URL and observe the response – everything from low-level HTTP (result headers and status codes) to page content. • Test that the correct view is executed for a given URL. • Test that a given request is rendered by a given Django template, with a template context that contains certain values. Note that the test client is not intended to be a replacement for Selenium or other “in-browser” frameworks. Django’s test client has a different focus. In short: • Use Django’s test client to establish that the correct view is being called and that the view is collecting the correct context data. • Use in-browser frameworks like Selenium to test rendered HTML and the behavior of Web pages, namely JavaScript functionality. Django also provides special support for those frameworks; see the section on LiveServerTestCase for more details. A comprehensive test suite should use a combination of both test types. Overview and a quick example To use the test client, instantiate django.test.client.Client and retrieve Web pages: >>> from django.test.client import Client >>> c = Client() >>> response = c.post(’/login/’, {’username’: ’john’, ’password’: ’smith’}) >>> response.status_code 200 >>> response = c.get(’/customer/details/’) >>> response.content ’<!DOCTYPE html...’ As this example suggests, you can instantiate Client from within a session of the Python interactive interpreter. Note a few important things about how the test client works: • The test client does not require the Web server to be running. In fact, it will run just fine with no Web server running at all! That’s because it avoids the overhead of HTTP and deals directly with the Django framework. This helps make the unit tests run quickly. • When retrieving pages, remember to specify the path of the URL, not the whole domain. For example, this is correct: >>> c.get(’/login/’) This is incorrect: >>> c.get(’http://www.example.com/login/’) 3.8. Testing in Django 269
- 274. Django Documentation, Release1.5.1 The test client is not capable of retrieving Web pages that are not powered by your Django project. If you need to retrieve other Web pages, use a Python standard library module such as urllib or urllib2. • To resolve URLs, the test client uses whatever URLconf is pointed-to by your ROOT_URLCONF setting. • Although the above example would work in the Python interactive interpreter, some of the test client’s function-ality, notably the template-related functionality, is only available while tests are running. The reason for this is that Django’s test runner performs a bit of black magic in order to determine which template was loaded by a given view. This black magic (essentially a patching of Django’s template system in memory) only happens during test running. • By default, the test client will disable any CSRF checks performed by your site. If, for some reason, you want the test client to perform CSRF checks, you can create an instance of the test client that enforces CSRF checks. To do this, pass in the enforce_csrf_checks argument when you construct your client: >>> from django.test import Client >>> csrf_client = Client(enforce_csrf_checks=True) Making requests Use the django.test.client.Client class to make requests. class Client(enforce_csrf_checks=False, **defaults) It requires no arguments at time of construction. However, you can use keywords arguments to specify some default headers. For example, this will send a User-Agent HTTP header in each request: >>> c = Client(HTTP_USER_AGENT=’Mozilla/5.0’) The values from the extra keywords arguments passed to get(), post(), etc. have precedence over the defaults passed to the class constructor. The enforce_csrf_checks argument can be used to test CSRF protection (see above). Once you have a Client instance, you can call any of the following methods: get(path, data={}, follow=False, **extra) Makes a GET request on the provided path and returns a Response object, which is documented below. The key-value pairs in the data dictionary are used to create a GET data payload. For example: >>> c = Client() >>> c.get(’/customers/details/’, {’name’: ’fred’, ’age’: 7}) ...will result in the evaluation of a GET request equivalent to: /customers/details/?name=fred&age=7 The extra keyword arguments parameter can be used to specify headers to be sent in the request. For example: >>> c = Client() >>> c.get(’/customers/details/’, {’name’: ’fred’, ’age’: 7}, ... HTTP_X_REQUESTED_WITH=’XMLHttpRequest’) ...will send the HTTP header HTTP_X_REQUESTED_WITH to the details view, which is a good way to test code paths that use the django.http.HttpRequest.is_ajax() method. CGI specification 270 Chapter 3. Using Django
- 275. Django Documentation, Release1.5.1 The headers sent via **extra should follow CGI specification. For example, emulating a different “Host” header as sent in the HTTP request from the browser to the server should be passed as HTTP_HOST. If you already have the GET arguments in URL-encoded form, you can use that encoding instead of using the data argument. For example, the previous GET request could also be posed as: >>> c = Client() >>> c.get(’/customers/details/?name=fred&age=7’) If you provide a URL with both an encoded GET data and a data argument, the data argument will take precedence. If you set follow to True the client will follow any redirects and a redirect_chain attribute will be set in the response object containing tuples of the intermediate urls and status codes. If you had a URL /redirect_me/ that redirected to /next/, that redirected to /final/, this is what you’d see: >>> response = c.get(’/redirect_me/’, follow=True) >>> response.redirect_chain [(u’http://testserver/next/’, 302), (u’http://testserver/final/’, 302)] post(path, data={}, content_type=MULTIPART_CONTENT, follow=False, **extra) Makes a POST request on the provided path and returns a Response object, which is documented below. The key-value pairs in the data dictionary are used to submit POST data. For example: >>> c = Client() >>> c.post(’/login/’, {’name’: ’fred’, ’passwd’: ’secret’}) ...will result in the evaluation of a POST request to this URL: /login/ ...with this POST data: name=fred&passwd=secret If you provide content_type (e.g. text/xml for an XML payload), the contents of data will be sent as-is in the POST request, using content_type in the HTTP Content-Type header. If you don’t provide a value for content_type, the values in data will be transmitted with a con-tent type of multipart/form-data. In this case, the key-value pairs in data will be encoded as a multipart message and used to create the POST data payload. To submit multiple values for a given key – for example, to specify the selections for a <select multiple> – provide the values as a list or tuple for the required key. For example, this value of data would submit three selected values for the field named choices: {’choices’: (’a’, ’b’, ’d’)} Submitting files is a special case. To POST a file, you need only provide the file field name as a key, and a file handle to the file you wish to upload as a value. For example: >>> c = Client() >>> with open(’wishlist.doc’) as fp: ... c.post(’/customers/wishes/’, {’name’: ’fred’, ’attachment’: fp}) (The name attachment here is not relevant; use whatever name your file-processing code expects.) 3.8. Testing in Django 271
- 276. Django Documentation, Release1.5.1 Note that if you wish to use the same file handle for multiple post() calls then you will need to manually reset the file pointer between posts. The easiest way to do this is to manually close the file after it has been provided to post(), as demonstrated above. You should also ensure that the file is opened in a way that allows the data to be read. If your file contains binary data such as an image, this means you will need to open the file in rb (read binary) mode. The extra argument acts the same as for Client.get(). If the URL you request with a POST contains encoded parameters, these parameters will be made available in the request.GET data. For example, if you were to make the request: >>> c.post(’/login/?visitor=true’, {’name’: ’fred’, ’passwd’: ’secret’}) ... the view handling this request could interrogate request.POST to retrieve the username and password, and could interrogate request.GET to determine if the user was a visitor. If you set follow to True the client will follow any redirects and a redirect_chain attribute will be set in the response object containing tuples of the intermediate urls and status codes. head(path, data={}, follow=False, **extra) Makes a HEAD request on the provided path and returns a Response object. This method works just like Client.get(), including the follow and extra arguments, except it does not return a message body. options(path, data=’‘, content_type=’application/octet-stream’, follow=False, **extra) Makes an OPTIONS request on the provided path and returns a Response object. Useful for testing RESTful interfaces. When data is provided, it is used as the request body, and a Content-Type header is set to content_type. Changed in version 1.5: Client.options() used to process data like Client.get(). The follow and extra arguments act the same as for Client.get(). put(path, data=’‘, content_type=’application/octet-stream’, follow=False, **extra) Makes a PUT request on the provided path and returns a Response object. Useful for testing RESTful interfaces. When data is provided, it is used as the request body, and a Content-Type header is set to content_type. Changed in version 1.5: Client.put() used to process data like Client.post(). The follow and extra arguments act the same as for Client.get(). delete(path, data=’‘, content_type=’application/octet-stream’, follow=False, **extra) Makes an DELETE request on the provided path and returns a Response object. Useful for testing RESTful interfaces. When data is provided, it is used as the request body, and a Content-Type header is set to content_type. Changed in version 1.5: Client.delete() used to process data like Client.get(). The follow and extra arguments act the same as for Client.get(). login(**credentials) If your site uses Django’s authentication system and you deal with logging in users, you can use the test client’s login() method to simulate the effect of a user logging into the site. After you call this method, the test client will have all the cookies and session data required to pass any login-based tests that may form part of a view. The format of the credentials argument depends on which authentication backend you’re using (which is configured by your AUTHENTICATION_BACKENDS setting). If you’re using the standard authentication backend provided by Django (ModelBackend), credentials should be the user’s username and password, provided as keyword arguments: 272 Chapter 3. Using Django
- 277. Django Documentation, Release1.5.1 >>> c = Client() >>> c.login(username=’fred’, password=’secret’) # Now you can access a view that’s only available to logged-in users. If you’re using a different authentication backend, this method may require different credentials. It requires whichever credentials are required by your backend’s authenticate() method. login() returns True if it the credentials were accepted and login was successful. Finally, you’ll need to remember to create user accounts before you can use this method. As we explained above, the test runner is executed using a test database, which contains no users by default. As a result, user accounts that are valid on your production site will not work under test conditions. You’ll need to create users as part of the test suite – either manually (using the Django model API) or with a test fixture. Remember that if you want your test user to have a password, you can’t set the user’s password by setting the password attribute directly – you must use the set_password() function to store a correctly hashed password. Alternatively, you can use the create_user() helper method to create a new user with a correctly hashed password. logout() If your site uses Django’s authentication system, the logout() method can be used to simulate the effect of a user logging out of your site. After you call this method, the test client will have all the cookies and session data cleared to defaults. Subsequent requests will appear to come from an AnonymousUser. Testing responses The get() and post() methods both return a Response object. This Response object is not the same as the HttpResponse object returned Django views; the test response object has some additional data useful for test code to verify. Specifically, a Response object has the following attributes: class Response client The test client that was used to make the request that resulted in the response. content The body of the response, as a string. This is the final page content as rendered by the view, or any error message. context The template Context instance that was used to render the template that produced the response content. If the rendered page used multiple templates, then context will be a list of Context objects, in the order in which they were rendered. Regardless of the number of templates used during rendering, you can retrieve context values using the [] operator. For example, the context variable name could be retrieved using: >>> response = client.get(’/foo/’) >>> response.context[’name’] ’Arthur’ request The request data that stimulated the response. status_code The HTTP status of the response, as an integer. See RFC 2616 for a full list of HTTP status codes. 3.8. Testing in Django 273
- 278. Django Documentation, Release1.5.1 templates A list of Template instances used to render the final content, in the order they were rendered. For each template in the list, use template.name to get the template’s file name, if the template was loaded from a file. (The name is a string such as ’admin/index.html’.) You can also use dictionary syntax on the response object to query the value of any settings in the HTTP headers. For example, you could determine the content type of a response using response[’Content-Type’]. Exceptions If you point the test client at a view that raises an exception, that exception will be visible in the test case. You can then use a standard try ... except block or assertRaises() to test for exceptions. The only exceptions that are not visible to the test client are Http404, PermissionDenied and SystemExit. Django catches these exceptions internally and converts them into the appropriate HTTP response codes. In these cases, you can check response.status_code in your test. Persistent state The test client is stateful. If a response returns a cookie, then that cookie will be stored in the test client and sent with all subsequent get() and post() requests. Expiration policies for these cookies are not followed. If you want a cookie to expire, either delete it manually or create a new Client instance (which will effectively delete all cookies). A test client has two attributes that store persistent state information. You can access these properties as part of a test condition. Client.cookies A Python SimpleCookie object, containing the current values of all the client cookies. See the documentation of the Cookie module for more. Client.session A dictionary-like object containing session information. See the session documentation for full details. To modify the session and then save it, it must be stored in a variable first (because a new SessionStore is created every time this property is accessed): def test_something(self): session = self.client.session session[’somekey’] = ’test’ session.save() Example The following is a simple unit test using the test client: from django.utils import unittest from django.test.client import Client class SimpleTest(unittest.TestCase): def setUp(self): # Every test needs a client. self.client = Client() def test_details(self): # Issue a GET request. response = self.client.get(’/customer/details/’) # Check that the response is 200 OK. self.assertEqual(response.status_code, 200) # Check that the rendered context contains 5 customers. self.assertEqual(len(response.context[’customers’]), 5) 274 Chapter 3. Using Django
- 279. Django Documentation, Release1.5.1 See Also: django.test.client.RequestFactory Provided test case classes Normal Python unit test classes extend a base class of unittest.TestCase. Django provides a few extensions of this base class: Regardless of the version of Python you’re using, if you’ve installed unittest2, django.utils.unittest will point to that library. SimpleTestCase class SimpleTestCase New in version 1.4. A very thin subclass of unittest.TestCase, it extends it with some basic functionality like: • Saving and restoring the Python warning machinery state. • Checking that a callable raises a certain exception. • Testing form field rendering. • Testing server HTML responses for the presence/lack of a given fragment. • The ability to run tests with modified settings If you need any of the other more complex and heavyweight Django-specific features like: • Using the client Client. • Testing or using the ORM. • Database fixtures. • Custom test-time URL maps. • Test skipping based on database backend features. • The remaining specialized assert* methods. then you should use TransactionTestCase or TestCase instead. SimpleTestCase inherits from django.utils.unittest.TestCase. TransactionTestCase class TransactionTestCase Django TestCase classes make use of database transaction facilities, if available, to speed up the process of resetting the database to a known state at the beginning of each test. A consequence of this, however, is that the effects of transaction commit and rollback cannot be tested by a Django TestCase class. If your test requires testing of such transactional behavior, you should use a Django TransactionTestCase. TransactionTestCase and TestCase are identical except for the manner in which the database is reset to a known state and the ability for test code to test the effects of commit and rollback: • A TransactionTestCase resets the database after the test runs by truncating all tables. A TransactionTestCase may call commit and rollback and observe the effects of these calls on the database. 3.8. Testing in Django 275
- 280. Django Documentation, Release1.5.1 276 Figure 3.1: Hierarchy of Django unit testing classes Chapter 3. Using Django
- 281. Django Documentation, Release1.5.1 • A TestCase, on the other hand, does not truncate tables after a test. Instead, it encloses the test code in a database transaction that is rolled back at the end of the test. It also prevents the code under test from issuing any commit or rollback operations on the database, to ensure that the rollback at the end of the test restores the database to its initial state. When running on a database that does not support rollback (e.g. MySQL with the MyISAM storage engine), TestCase falls back to initializing the database by truncating tables and reloading initial data. Note: Changed in version 1.5. Prior to 1.5, TransactionTestCase flushed the database tables before each test. In Django 1.5, this is instead done after the test has been run. When the flush took place before the test, it was guaranteed that primary key values started at one in TransactionTestCase tests. Tests should not depend on this behaviour, but for legacy tests that do, the reset_sequences attribute can be used until the test has been properly updated. Changed in version 1.5: The order in which tests are run has changed. See Order in which tests are executed. TransactionTestCase inherits from SimpleTestCase. TransactionTestCase.reset_sequences New in version 1.5. Setting reset_sequences = True on a TransactionTestCase will make sure sequences are always reset before the test run: class TestsThatDependsOnPrimaryKeySequences(TransactionTestCase): reset_sequences = True def test_animal_pk(self): lion = Animal.objects.create(name="lion", sound="roar") # lion.pk is guaranteed to always be 1 self.assertEqual(lion.pk, 1) Unless you are explicitly testing primary keys sequence numbers, it is recommended that you do not hard code primary key values in tests. Using reset_sequences = True will slow down the test, since the primary key reset is an relatively expensive database operation. TestCase class TestCase This class provides some additional capabilities that can be useful for testing Web sites. Converting a normal unittest.TestCase to a Django TestCase is easy: Just change the base class of your test from ’unittest.TestCase’ to ’django.test.TestCase’. All of the standard Python unit test function-ality will continue to be available, but it will be augmented with some useful additions, including: • Automatic loading of fixtures. • Wraps each test in a transaction. • Creates a TestClient instance. • Django-specific assertions for testing for things like redirection and form errors. Changed in version 1.5: The order in which tests are run has changed. See Order in which tests are executed. TestCase inherits from TransactionTestCase. 3.8. Testing in Django 277
- 282. Django Documentation, Release1.5.1 LiveServerTestCase New in version 1.4. class LiveServerTestCase LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test clients other than the Django dummy client such as, for example, the Selenium client, to execute a series of functional tests inside a browser and simulate a real user’s actions. By default the live server’s address is ’localhost:8081’ and the full URL can be accessed during the tests with self.live_server_url. If you’d like to change the default address (in the case, for example, where the 8081 port is already taken) then you may pass a different one to the test command via the --liveserver option, for example: ./manage.py test --liveserver=localhost:8082 Another way of changing the default server address is by setting the DJANGO_LIVE_TEST_SERVER_ADDRESS environment variable somewhere in your code (for example, in a custom test runner): import os os.environ[’DJANGO_LIVE_TEST_SERVER_ADDRESS’] = ’localhost:8082’ In the case where the tests are run by multiple processes in parallel (for example, in the context of several simulta-neous continuous integration builds), the processes will compete for the same address, and therefore your tests might randomly fail with an “Address already in use” error. To avoid this problem, you can pass a comma-separated list of ports or ranges of ports (at least as many as the number of potential parallel processes). For example: ./manage.py test --liveserver=localhost:8082,8090-8100,9000-9200,7041 Then, during test execution, each new live test server will try every specified port until it finds one that is free and takes it. To demonstrate how to use LiveServerTestCase, let’s write a simple Selenium test. First of all, you need to install the selenium package into your Python path: pip install selenium Then, add a LiveServerTestCase-based test to your app’s tests module (for example: myapp/tests.py). The code for this test may look as follows: from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver import WebDriver class MySeleniumTests(LiveServerTestCase): fixtures = [’user-data.json’] @classmethod def setUpClass(cls): cls.selenium = WebDriver() super(MySeleniumTests, cls).setUpClass() @classmethod def tearDownClass(cls): cls.selenium.quit() super(MySeleniumTests, cls).tearDownClass() def test_login(self): self.selenium.get(’%s%s’ % (self.live_server_url, ’/login/’)) username_input = self.selenium.find_element_by_name("username") username_input.send_keys(’myuser’) 278 Chapter 3. Using Django
- 283. Django Documentation, Release1.5.1 password_input = self.selenium.find_element_by_name("password") password_input.send_keys(’secret’) self.selenium.find_element_by_xpath(’//input[@value="Log in"]’).click() Finally, you may run the test as follows: ./manage.py test myapp.MySeleniumTests.test_login This example will automatically open Firefox then go to the login page, enter the credentials and press the “Log in” button. Selenium offers other drivers in case you do not have Firefox installed or wish to use another browser. The example above is just a tiny fraction of what the Selenium client can do; check out the full reference for more details. Note: LiveServerTestCase makes use of the staticfiles contrib app so you’ll need to have your project config-ured accordingly (in particular by setting STATIC_URL). Note: When using an in-memory SQLite database to run the tests, the same database connection will be shared by two threads in parallel: the thread in which the live server is run and the thread in which the test case is run. It’s important to prevent simultaneous database queries via this shared connection by the two threads, as that may sometimes randomly cause the tests to fail. So you need to ensure that the two threads don’t access the database at the same time. In particular, this means that in some cases (for example, just after clicking a link or submitting a form), you might need to check that a response is received by Selenium and that the next page is loaded before proceeding with further test execution. Do this, for example, by making Selenium wait until the <body> HTML tag is found in the response (requires Selenium > 2.13): def test_login(self): from selenium.webdriver.support.wait import WebDriverWait timeout = 2 ... self.selenium.find_element_by_xpath(’//input[@value="Log in"]’).click() # Wait until the response is received WebDriverWait(self.selenium, timeout).until( lambda driver: driver.find_element_by_tag_name(’body’)) The tricky thing here is that there’s really no such thing as a “page load,” especially in modern Web apps that generate HTML dynamically after the server generates the initial document. So, simply checking for the presence of <body> in the response might not necessarily be appropriate for all use cases. Please refer to the Selenium FAQ and Selenium documentation for more information. Test cases features Default test client TestCase.client Every test case in a django.test.TestCase instance has access to an instance of a Django test client. This client can be accessed as self.client. This client is recreated for each test, so you don’t have to worry about state (such as cookies) carrying over from one test to another. This means, instead of instantiating a Client in each test: from django.utils import unittest from django.test.client import Client class SimpleTest(unittest.TestCase): def test_details(self): client = Client() 3.8. Testing in Django 279
- 284. Django Documentation, Release1.5.1 response = client.get(’/customer/details/’) self.assertEqual(response.status_code, 200) def test_index(self): client = Client() response = client.get(’/customer/index/’) self.assertEqual(response.status_code, 200) ...you can just refer to self.client, like so: from django.test import TestCase class SimpleTest(TestCase): def test_details(self): response = self.client.get(’/customer/details/’) self.assertEqual(response.status_code, 200) def test_index(self): response = self.client.get(’/customer/index/’) self.assertEqual(response.status_code, 200) Customizing the test client TestCase.client_class If you want to use a different Client class (for example, a subclass with customized behavior), use the client_class class attribute: from django.test import TestCase from django.test.client import Client class MyTestClient(Client): # Specialized methods for your environment... class MyTest(TestCase): client_class = MyTestClient def test_my_stuff(self): # Here self.client is an instance of MyTestClient... call_some_test_code() Fixture loading TestCase.fixtures A test case for a database-backedWeb site isn’t much use if there isn’t any data in the database. To make it easy to put test data into the database, Django’s custom TestCase class provides a way of loading fixtures. A fixture is a collection of data that Django knows how to import into a database. For example, if your site has user accounts, you might set up a fixture of fake user accounts in order to populate your database during tests. The most straightforward way of creating a fixture is to use the manage.py dumpdata command. This assumes you already have some data in your database. See the dumpdata documentation for more details. Note: If you’ve ever run manage.py syncdb, you’ve already used a fixture without even knowing it! When you call syncdb in the database for the first time, Django installs a fixture called initial_data. This gives you a way of populating a new database with any initial data, such as a default set of categories. Fixtures with other names can always be installed manually using the manage.py loaddata command. 280 Chapter 3. Using Django
- 285. Django Documentation, Release1.5.1 Initial SQL data and testing Django provides a second way to insert initial data into models – the custom SQL hook. However, this technique cannot be used to provide initial data for testing purposes. Django’s test framework flushes the contents of the test database after each test; as a result, any data added using the custom SQL hook will be lost. Once you’ve created a fixture and placed it in a fixtures directory in one of your INSTALLED_APPS, you can use it in your unit tests by specifying a fixtures class attribute on your django.test.TestCase subclass: from django.test import TestCase from myapp.models import Animal class AnimalTestCase(TestCase): fixtures = [’mammals.json’, ’birds’] def setUp(self): # Test definitions as before. call_setup_methods() def testFluffyAnimals(self): # A test that uses the fixtures. call_some_test_code() Here’s specifically what will happen: • At the start of each test case, before setUp() is run, Django will flush the database, returning the database to the state it was in directly after syncdb was called. • Then, all the named fixtures are installed. In this example, Django will install any JSON fixture named mammals, followed by any fixture named birds. See the loaddata documentation for more details on defining and installing fixtures. This flush/load procedure is repeated for each test in the test case, so you can be certain that the outcome of a test will not be affected by another test, or by the order of test execution. URLconf configuration TestCase.urls If your application provides views, you may want to include tests that use the test client to exercise those views. However, an end user is free to deploy the views in your application at any URL of their choosing. This means that your tests can’t rely upon the fact that your views will be available at a particular URL. In order to provide a reliable URL space for your test, django.test.TestCase provides the ability to customize the URLconf configuration for the duration of the execution of a test suite. If your TestCase instance defines an urls attribute, the TestCase will use the value of that attribute as the ROOT_URLCONF for the duration of that test. For example: from django.test import TestCase class TestMyViews(TestCase): urls = ’myapp.test_urls’ def testIndexPageView(self): # Here you’d test your view using ‘‘Client‘‘. call_some_test_code() This test case will use the contents of myapp.test_urls as the URLconf for the duration of the test case. 3.8. Testing in Django 281
- 286. Django Documentation, Release1.5.1 Multi-database support TestCase.multi_db Django sets up a test database corresponding to every database that is defined in the DATABASES definition in your settings file. However, a big part of the time taken to run a Django TestCase is consumed by the call to flush that ensures that you have a clean database at the start of each test run. If you have multiple databases, multiple flushes are required (one for each database), which can be a time consuming activity – especially if your tests don’t need to test multi-database activity. As an optimization, Django only flushes the default database at the start of each test run. If your setup contains multiple databases, and you have a test that requires every database to be clean, you can use the multi_db attribute on the test suite to request a full flush. For example: class TestMyViews(TestCase): multi_db = True def testIndexPageView(self): call_some_test_code() This test case will flush all the test databases before running testIndexPageView. Overriding settings TestCase.settings() New in version 1.4. For testing purposes it’s often useful to change a setting temporarily and revert to the original value after running the testing code. For this use case Django provides a standard Python context manager (see PEP 343) settings(), which can be used like this: from django.test import TestCase class LoginTestCase(TestCase): def test_login(self): # First check for the default behavior response = self.client.get(’/sekrit/’) self.assertRedirects(response, ’/accounts/login/?next=/sekrit/’) # Then override the LOGIN_URL setting with self.settings(LOGIN_URL=’/other/login/’): response = self.client.get(’/sekrit/’) self.assertRedirects(response, ’/other/login/?next=/sekrit/’) This example will override the LOGIN_URL setting for the code in the with block and reset its value to the previous state afterwards. override_settings() In case you want to override a setting for just one test method or even the whole TestCase class, Django provides the override_settings() decorator (see PEP 318). It’s used like this: from django.test import TestCase from django.test.utils import override_settings class LoginTestCase(TestCase): @override_settings(LOGIN_URL=’/other/login/’) def test_login(self): 282 Chapter 3. Using Django
- 287. Django Documentation, Release1.5.1 response = self.client.get(’/sekrit/’) self.assertRedirects(response, ’/other/login/?next=/sekrit/’) The decorator can also be applied to test case classes: from django.test import TestCase from django.test.utils import override_settings @override_settings(LOGIN_URL=’/other/login/’) class LoginTestCase(TestCase): def test_login(self): response = self.client.get(’/sekrit/’) self.assertRedirects(response, ’/other/login/?next=/sekrit/’) Note: When given a class, the decorator modifies the class directly and returns it; it doesn’t create and return a modified copy of it. So if you try to tweak the above example to assign the return value to a different name than LoginTestCase, you may be surprised to find that the original LoginTestCase is still equally affected by the decorator. When overriding settings, make sure to handle the cases in which your app’s code uses a cache or similar feature that retains state even if the setting is changed. Django provides the django.test.signals.setting_changed signal that lets you register callbacks to clean up and otherwise reset state when settings are changed. Django itself uses this signal to reset various data: Overriden settings Data reset USE_TZ, TIME_ZONE Databases timezone TEMPLATE_CONTEXT_PROCESSORS Context processors cache TEMPLATE_LOADERS Template loaders cache SERIALIZATION_MODULES Serializers cache LOCALE_PATHS, LANGUAGE_CODE Default translation and loaded translations MEDIA_ROOT, DEFAULT_FILE_STORAGE Default file storage Emptying the test outbox If you use Django’s custom TestCase class, the test runner will clear the contents of the test email outbox at the start of each test case. For more detail on email services during tests, see Email services below. Assertions As Python’s normal unittest.TestCase class implements assertion methods such as assertTrue() and assertEqual(), Django’s custom TestCase class provides a number of custom asser-tion methods that are useful for testing Web applications: The failure messages given by most of these assertion methods can be customized with the msg_prefix argument. This string will be prefixed to any failure message generated by the assertion. This allows you to provide additional details that may help you to identify the location and cause of an failure in your test suite. SimpleTestCase.assertRaisesMessage(expected_exception, expected_message, callable_obj=None, *args, **kwargs) New in version 1.4. Asserts that execution of callable callable_obj raised the expected_exception exception and that such exception has an expected_message representation. Any other outcome is reported as a failure. Similar to unittest’s assertRaisesRegexp() with the difference that expected_message isn’t a regular expression. 3.8. Testing in Django 283
- 288. Django Documentation, Release1.5.1 SimpleTestCase.assertFieldOutput(self, fieldclass, valid, invalid, field_args=None, field_kwargs=None, empty_value=u’‘) New in version 1.4. Asserts that a form field behaves correctly with various inputs. Parameters • fieldclass – the class of the field to be tested. • valid – a dictionary mapping valid inputs to their expected cleaned values. • invalid – a dictionary mapping invalid inputs to one or more raised error messages. • field_args – the args passed to instantiate the field. • field_kwargs – the kwargs passed to instantiate the field. • empty_value – the expected clean output for inputs in EMPTY_VALUES. For example, the following code tests that an EmailField accepts “a@a.com” as a valid email address, but rejects “aaa” with a reasonable error message: self.assertFieldOutput(EmailField, {’a@a.com’: ’a@a.com’}, {’aaa’: [u’Enter a valid email address.’]}) TestCase.assertContains(response, text, count=None, status_code=200, msg_prefix=’‘, html=False) Asserts that a Response instance produced the given status_code and that text appears in the content of the response. If count is provided, text must occur exactly count times in the response. New in version 1.4. Set html to True to handle text as HTML. The comparison with the response content will be based on HTML semantics instead of character-by-character equality. Whitespace is ignored in most cases, attribute ordering is not significant. See assertHTMLEqual() for more details. TestCase.assertNotContains(response, text, status_code=200, msg_prefix=’‘, html=False) Asserts that a Response instance produced the given status_code and that text does not appears in the content of the response. New in version 1.4. Set html to True to handle text as HTML. The comparison with the response content will be based on HTML semantics instead of character-by-character equality. Whitespace is ignored in most cases, attribute ordering is not significant. See assertHTMLEqual() for more details. TestCase.assertFormError(response, form, field, errors, msg_prefix=’‘) Asserts that a field on a form raises the provided list of errors when rendered on the form. form is the name the Form instance was given in the template context. field is the name of the field on the form to check. If field has a value of None, non-field errors (errors you can access via form.non_field_errors()) will be checked. errors is an error string, or a list of error strings, that are expected as a result of form validation. TestCase.assertTemplateUsed(response, template_name, msg_prefix=’‘) Asserts that the template with the given name was used in rendering the response. The name is a string such as ’admin/index.html’. New in version 1.4. You can use this as a context manager, like this: with self.assertTemplateUsed(’index.html’): render_to_string(’index.html’) with self.assertTemplateUsed(template_name=’index.html’): render_to_string(’index.html’) TestCase.assertTemplateNotUsed(response, template_name, msg_prefix=’‘) Asserts that the template with the given name was not used in rendering the response. New in version 1.4. You can use this as a context manager in the same way as assertTemplateUsed(). 284 Chapter 3. Using Django
- 289. Django Documentation, Release1.5.1 TestCase.assertRedirects(response, expected_url, status_code=302, target_status_code=200, msg_prefix=’‘) Asserts that the response return a status_code redirect status, it redirected to expected_url (including any GET data), and the final page was received with target_status_code. If your request used the follow argument, the expected_url and target_status_code will be the url and status code for the final point of the redirect chain. TestCase.assertQuerysetEqual(qs, values, transform=repr, ordered=True) Asserts that a queryset qs returns a particular list of values values. The comparison of the contents of qs and values is performed using the function transform; by default, this means that the repr() of each value is compared. Any other callable can be used if repr() doesn’t provide a unique or helpful comparison. By default, the comparison is also ordering dependent. If qs doesn’t provide an implicit ordering, you can set the ordered parameter to False, which turns the comparison into a Python set comparison. Changed in version 1.4: The ordered parameter is new in version 1.4. In earlier versions, you would need to ensure the queryset is ordered consistently, possibly via an explicit order_by() call on the queryset prior to comparison. TestCase.assertNumQueries(num, func, *args, **kwargs) Asserts that when func is called with *args and **kwargs that num database queries are executed. If a "using" key is present in kwargs it is used as the database alias for which to check the number of queries. If you wish to call a function with a using parameter you can do it by wrapping the call with a lambda to add an extra parameter: self.assertNumQueries(7, lambda: my_function(using=7)) You can also use this as a context manager: with self.assertNumQueries(2): Person.objects.create(name="Aaron") Person.objects.create(name="Daniel") SimpleTestCase.assertHTMLEqual(html1, html2, msg=None) New in version 1.4. Asserts that the strings html1 and html2 are equal. The comparison is based on HTML semantics. The comparison takes following things into account: •Whitespace before and after HTML tags is ignored. •All types of whitespace are considered equivalent. •All open tags are closed implicitly, e.g. when a surrounding tag is closed or the HTML document ends. •Empty tags are equivalent to their self-closing version. •The ordering of attributes of an HTML element is not significant. •Attributes without an argument are equal to attributes that equal in name and value (see the examples). The following examples are valid tests and don’t raise any AssertionError: self.assertHTMLEqual(’<p>Hello <b>world!</p>’, ’’’<p> Hello <b>world! <b/> </p>’’’) self.assertHTMLEqual( ’<input type="checkbox" checked="checked" id="id_accept_terms" />’, ’<input id="id_accept_terms" type=’checkbox’ checked>’) html1 and html2 must be valid HTML. An AssertionError will be raised if one of them cannot be parsed. 3.8. Testing in Django 285
- 290. Django Documentation, Release1.5.1 SimpleTestCase.assertHTMLNotEqual(html1, html2, msg=None) New in version 1.4. Asserts that the strings html1 and html2 are not equal. The comparison is based on HTML semantics. See assertHTMLEqual() for details. html1 and html2 must be valid HTML. An AssertionError will be raised if one of them cannot be parsed. SimpleTestCase.assertXMLEqual(xml1, xml2, msg=None) New in version 1.5. Asserts that the strings xml1 and xml2 are equal. The comparison is based on XML semantics. Similarily to assertHTMLEqual(), the comparison is made on parsed content, hence only se-mantic differences are considered, not syntax differences. When unvalid XML is passed in any parameter, an AssertionError is always raised, even if both string are identical. SimpleTestCase.assertXMLNotEqual(xml1, xml2, msg=None) New in version 1.5. Asserts that the strings xml1 and xml2 are not equal. The comparison is based on XML semantics. See assertXMLEqual() for details. Email services If any of your Django views send email using Django’s email functionality, you probably don’t want to send email each time you run a test using that view. For this reason, Django’s test runner automatically redirects all Django-sent email to a dummy outbox. This lets you test every aspect of sending email – from the number of messages sent to the contents of each message – without actually sending the messages. The test runner accomplishes this by transparently replacing the normal email backend with a testing backend. (Don’t worry – this has no effect on any other email senders outside of Django, such as your machine’s mail server, if you’re running one.) django.core.mail.outbox During test running, each outgoing email is saved in django.core.mail.outbox. This is a simple list of all EmailMessage instances that have been sent. The outbox attribute is a special attribute that is created only when the locmem email backend is used. It doesn’t normally exist as part of the django.core.mail module and you can’t import it directly. The code below shows how to access this attribute correctly. Here’s an example test that examines django.core.mail.outbox for length and contents: from django.core import mail from django.test import TestCase class EmailTest(TestCase): def test_send_email(self): # Send message. mail.send_mail(’Subject here’, ’Here is the message.’, ’from@example.com’, [’to@example.com’], fail_silently=False) # Test that one message has been sent. self.assertEqual(len(mail.outbox), 1) # Verify that the subject of the first message is correct. self.assertEqual(mail.outbox[0].subject, ’Subject here’) As noted previously, the test outbox is emptied at the start of every test in a Django TestCase. To empty the outbox manually, assign the empty list to mail.outbox: from django.core import mail 286 Chapter 3. Using Django
- 291. Django Documentation, Release1.5.1 # Empty the test outbox mail.outbox = [] Skipping tests The unittest library provides the @skipIf and @skipUnless decorators to allow you to skip tests if you know ahead of time that those tests are going to fail under certain conditions. For example, if your test requires a particular optional library in order to succeed, you could decorate the test case with @skipIf. Then, the test runner will report that the test wasn’t executed and why, instead of failing the test or omitting the test altogether. To supplement these test skipping behaviors, Django provides two additional skip decorators. Instead of testing a generic boolean, these decorators check the capabilities of the database, and skip the test if the database doesn’t support a specific named feature. The decorators use a string identifier to describe database features. This string corresponds to attributes of the database connection features class. See django.db.backends.BaseDatabaseFeatures class for a full list of database features that can be used as a basis for skipping tests. skipIfDBFeature(feature_name_string) Skip the decorated test if the named database feature is supported. For example, the following test will not be executed if the database supports transactions (e.g., it would not run under PostgreSQL, but it would under MySQL with MyISAM tables): class MyTests(TestCase): @skipIfDBFeature(’supports_transactions’) def test_transaction_behavior(self): # ... conditional test code skipUnlessDBFeature(feature_name_string) Skip the decorated test if the named database feature is not supported. For example, the following test will only be executed if the database supports transactions (e.g., it would run under PostgreSQL, but not under MySQL with MyISAM tables): class MyTests(TestCase): @skipUnlessDBFeature(’supports_transactions’) def test_transaction_behavior(self): # ... conditional test code 3.8.2 Django and doctests Doctests use Python’s standard doctest module, which searches your docstrings for statements that resemble a session of the Python interactive interpreter. A full explanation of how doctest works is out of the scope of this document; read Python’s official documentation for the details. What’s a docstring? A good explanation of docstrings (and some guidelines for using them effectively) can be found in PEP 257: A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. Such a docstring becomes the __doc__ special attribute of that object. For example, this function has a docstring that describes what it does: 3.8. Testing in Django 287
- 292. Django Documentation, Release1.5.1 def add_two(num): "Return the result of adding two to the provided number." return num + 2 Because tests often make great documentation, putting tests directly in your docstrings is an effective way to document and test your code. As with unit tests, for a given Django application, the test runner looks for doctests in two places: • The models.py file. You can define module-level doctests and/or a doctest for individual models. It’s com-mon practice to put application-level doctests in the module docstring and model-level doctests in the model docstrings. • A file called tests.py in the application directory – i.e., the directory that holds models.py. This file is a hook for any and all doctests you want to write that aren’t necessarily related to models. This example doctest is equivalent to the example given in the unittest section above: # models.py from django.db import models class Animal(models.Model): """ An animal that knows how to make noise # Create some animals >>> lion = Animal.objects.create(name="lion", sound="roar") >>> cat = Animal.objects.create(name="cat", sound="meow") # Make ’em speak >>> lion.speak() ’The lion says "roar"’ >>> cat.speak() ’The cat says "meow"’ """ name = models.CharField(max_length=20) sound = models.CharField(max_length=20) def speak(self): return ’The %s says "%s"’ % (self.name, self.sound) When you run your tests, the test runner will find this docstring, notice that portions of it look like an interactive Python session, and execute those lines while checking that the results match. In the case of model tests, note that the test runner takes care of creating its own test database. That is, any test that accesses a database – by creating and saving model instances, for example – will not affect your production database. However, the database is not refreshed between doctests, so if your doctest requires a certain state you should consider flushing the database or loading a fixture. (See the section on fixtures for more on this.) Note that to use this feature, the database user Django is connecting as must have CREATE DATABASE rights. For more details about doctest, see the Python documentation. 3.8.3 Advanced testing topics The request factory class RequestFactory 288 Chapter 3. Using Django
- 293. Django Documentation, Release1.5.1 The RequestFactory shares the same API as the test client. However, instead of behaving like a browser, the RequestFactory provides a way to generate a request instance that can be used as the first argument to any view. This means you can test a view function the same way as you would test any other function – as a black box, with exactly known inputs, testing for specific outputs. The API for the RequestFactory is a slightly restricted subset of the test client API: • It only has access to the HTTP methods get(), post(), put(), delete(), head() and options(). • These methods accept all the same arguments except for follows. Since this is just a factory for producing requests, it’s up to you to handle the response. • It does not support middleware. Session and authentication attributes must be supplied by the test itself if required for the view to function properly. Example The following is a simple unit test using the request factory: from django.utils import unittest from django.test.client import RequestFactory class SimpleTest(unittest.TestCase): def setUp(self): # Every test needs access to the request factory. self.factory = RequestFactory() def test_details(self): # Create an instance of a GET request. request = self.factory.get(’/customer/details’) # Test my_view() as if it were deployed at /customer/details response = my_view(request) self.assertEqual(response.status_code, 200) Tests and multiple databases Testing master/slave configurations If you’re testing a multiple database configuration with master/slave replication, this strategy of creating test databases poses a problem. When the test databases are created, there won’t be any replication, and as a result, data created on the master won’t be seen on the slave. To compensate for this, Django allows you to define that a database is a test mirror. Consider the following (simplified) example database configuration: DATABASES = { ’default’: { ’ENGINE’: ’django.db.backends.mysql’, ’NAME’: ’myproject’, ’HOST’: ’dbmaster’, # ... plus some other settings }, ’slave’: { ’ENGINE’: ’django.db.backends.mysql’, ’NAME’: ’myproject’, ’HOST’: ’dbslave’, 3.8. Testing in Django 289
- 294. Django Documentation, Release1.5.1 ’TEST_MIRROR’: ’default’ # ... plus some other settings } } In this setup, we have two database servers: dbmaster, described by the database alias default, and dbslave described by the alias slave. As you might expect, dbslave has been configured by the database administrator as a read slave of dbmaster, so in normal activity, any write to default will appear on slave. If Django created two independent test databases, this would break any tests that expected replication to occur. How-ever, the slave database has been configured as a test mirror (using the TEST_MIRROR setting), indicating that under testing, slave should be treated as a mirror of default. When the test environment is configured, a test version of slave will not be created. Instead the connection to slave will be redirected to point at default. As a result, writes to default will appear on slave – but because they are actually the same database, not because there is data replication between the two databases. Controlling creation order for test databases By default, Django will always create the default database first. However, no guarantees are made on the creation order of any other databases in your test setup. If your database configuration requires a specific creation order, you can specify the dependencies that exist using the TEST_DEPENDENCIES setting. Consider the following (simplified) example database configuration: DATABASES = { ’default’: { # ... db settings ’TEST_DEPENDENCIES’: [’diamonds’] }, ’diamonds’: { # ... db settings }, ’clubs’: { # ... db settings ’TEST_DEPENDENCIES’: [’diamonds’] }, ’spades’: { # ... db settings ’TEST_DEPENDENCIES’: [’diamonds’,’hearts’] }, ’hearts’: { # ... db settings ’TEST_DEPENDENCIES’: [’diamonds’,’clubs’] } } Under this configuration, the diamonds database will be created first, as it is the only database alias without de-pendencies. The default and clubs alias will be created next (although the order of creation of this pair is not guaranteed); then hearts; and finally spades. If there are any circular dependencies in the TEST_DEPENDENCIES definition, an ImproperlyConfigured exception will be raised. 290 Chapter 3. Using Django
- 295. Django Documentation, Release1.5.1 Running tests outside the test runner If you want to run tests outside of ./manage.py test – for example, from a shell prompt – you will need to set up the test environment first. Django provides a convenience method to do this: >>> from django.test.utils import setup_test_environment >>> setup_test_environment() This convenience method sets up the test database, and puts other Django features into modes that allow for repeatable testing. The call to setup_test_environment() is made automatically as part of the setup of ./manage.py test. You only need to manually invoke this method if you’re not using running your tests via Django’s test runner. Using different testing frameworks Clearly, doctest and unittest are not the only Python testing frameworks. While Django doesn’t provide explicit support for alternative frameworks, it does provide a way to invoke tests constructed for an alternative framework as if they were normal Django tests. When you run ./manage.py test, Django looks at the TEST_RUNNER setting to determine what to do. By default, TEST_RUNNER points to ’django.test.simple.DjangoTestSuiteRunner’. This class defines the default Django testing behavior. This behavior involves: 1. Performing global pre-test setup. 2. Looking for unit tests and doctests in the models.py and tests.py files in each installed application. 3. Creating the test databases. 4. Running syncdb to install models and initial data into the test databases. 5. Running the unit tests and doctests that are found. 6. Destroying the test databases. 7. Performing global post-test teardown. If you define your own test runner class and point TEST_RUNNER at that class, Django will execute your test runner whenever you run ./manage.py test. In this way, it is possible to use any test framework that can be executed from Python code, or to modify the Django test execution process to satisfy whatever testing requirements you may have. Defining a test runner A test runner is a class defining a run_tests() method. Django ships with a DjangoTestSuiteRunner class that defines the default Django testing behavior. This class defines the run_tests() entry point, plus a selection of other methods that are used to by run_tests() to set up, execute and tear down the test suite. class DjangoTestSuiteRunner(verbosity=1, interactive=True, failfast=True, **kwargs) verbosity determines the amount of notification and debug information that will be printed to the console; 0 is no output, 1 is normal output, and 2 is verbose output. If interactive is True, the test suite has permission to ask the user for instructions when the test suite is executed. An example of this behavior would be asking for permission to delete an existing test database. If interactive is False, the test suite must be able to run without any manual intervention. If failfast is True, the test suite will stop running after the first test failure is detected. 3.8. Testing in Django 291
- 296. Django Documentation, Release1.5.1 Django will, from time to time, extend the capabilities of the test runner by adding new arguments. The **kwargs declaration allows for this expansion. If you subclass DjangoTestSuiteRunner or write your own test runner, ensure accept and handle the **kwargs parameter. New in version 1.4. Your test runner may also define additional command-line options. If you add an option_list attribute to a subclassed test runner, those options will be added to the list of command-line options that the test command can use. Attributes DjangoTestSuiteRunner.option_list New in version 1.4. This is the tuple of optparse options which will be fed into the management command’s OptionParser for parsing arguments. See the documentation for Python’s optparse module for more details. Methods DjangoTestSuiteRunner.run_tests(test_labels, extra_tests=None, **kwargs) Run the test suite. test_labels is a list of strings describing the tests to be run. A test label can take one of three forms: •app.TestCase.test_method – Run a single test method in a test case. •app.TestCase – Run all the test methods in a test case. •app – Search for and run all tests in the named application. If test_labels has a value of None, the test runner should run search for tests in all the applications in INSTALLED_APPS. extra_tests is a list of extra TestCase instances to add to the suite that is executed by the test runner. These extra tests are run in addition to those discovered in the modules listed in test_labels. This method should return the number of tests that failed. DjangoTestSuiteRunner.setup_test_environment(**kwargs) Sets up the test environment ready for testing. DjangoTestSuiteRunner.build_suite(test_labels, extra_tests=None, **kwargs) Constructs a test suite that matches the test labels provided. test_labels is a list of strings describing the tests to be run. A test label can take one of three forms: •app.TestCase.test_method – Run a single test method in a test case. •app.TestCase – Run all the test methods in a test case. •app – Search for and run all tests in the named application. If test_labels has a value of None, the test runner should run search for tests in all the applications in INSTALLED_APPS. extra_tests is a list of extra TestCase instances to add to the suite that is executed by the test runner. These extra tests are run in addition to those discovered in the modules listed in test_labels. Returns a TestSuite instance ready to be run. DjangoTestSuiteRunner.setup_databases(**kwargs) Creates the test databases. Returns a data structure that provides enough detail to undo the changes that have been made. This data will be provided to the teardown_databases() function at the conclusion of testing. DjangoTestSuiteRunner.run_suite(suite, **kwargs) Runs the test suite. 292 Chapter 3. Using Django
- 297. Django Documentation, Release1.5.1 Returns the result produced by the running the test suite. DjangoTestSuiteRunner.teardown_databases(old_config, **kwargs) Destroys the test databases, restoring pre-test conditions. old_config is a data structure defining the changes in the database configuration that need to be reversed. It is the return value of the setup_databases() method. DjangoTestSuiteRunner.teardown_test_environment(**kwargs) Restores the pre-test environment. DjangoTestSuiteRunner.suite_result(suite, result, **kwargs) Computes and returns a return code based on a test suite, and the result from that test suite. Testing utilities To assist in the creation of your own test runner, Django provides a number of utility methods in the django.test.utils module. setup_test_environment() Performs any global pre-test setup, such as the installing the instrumentation of the template rendering system and setting up the dummy email outbox. teardown_test_environment() Performs any global post-test teardown, such as removing the black magic hooks into the template system and restoring normal email services. The creation module of the database backend (connection.creation) also provides some utilities that can be useful during testing. create_test_db([verbosity=1, autoclobber=False ]) Creates a new test database and runs syncdb against it. verbosity has the same behavior as in run_tests(). autoclobber describes the behavior that will occur if a database with the same name as the test database is discovered: •If autoclobber is False, the user will be asked to approve destroying the existing database. sys.exit is called if the user does not approve. •If autoclobber is True, the database will be destroyed without consulting the user. Returns the name of the test database that it created. create_test_db() has the side effect of modifying the value of NAME in DATABASES to match the name of the test database. destroy_test_db(old_database_name[, verbosity=1 ]) Destroys the database whose name is the value of NAME in DATABASES, and sets NAME to the value of old_database_name. The verbosity argument has the same behavior as for DjangoTestSuiteRunner. Integration with coverage.py Code coverage describes how much source code has been tested. It shows which parts of your code are being exercised by tests and which are not. It’s an important part of testing applications, so it’s strongly recommended to check the coverage of your tests. 3.8. Testing in Django 293
- 298. Django Documentation, Release1.5.1 Django can be easily integrated with coverage.py, a tool for measuring code coverage of Python programs. First, install coverage.py. Next, run the following from your project folder containing manage.py: coverage run --source=’.’ manage.py test myapp This runs your tests and collects coverage data of the executed files in your project. You can see a report of this data by typing following command: coverage report Note that some Django code was executed while running tests, but it is not listed here because of the source flag passed to the previous command. For more options like annotated HTML listings detailing missed lines, see the coverage.py docs. Automated testing is an extremely useful bug-killing tool for the modern Web developer. You can use a collection of tests – a test suite – to solve, or avoid, a number of problems: • When you’re writing new code, you can use tests to validate your code works as expected. • When you’re refactoring or modifying old code, you can use tests to ensure your changes haven’t affected your application’s behavior unexpectedly. Testing a Web application is a complex task, because a Web application is made of several layers of logic – from HTTP-level request handling, to form validation and processing, to template rendering. With Django’s test-execution framework and assorted utilities, you can simulate requests, insert test data, inspect your application’s output and generally verify your code is doing what it should be doing. The best part is, it’s really easy. 3.8.4 Unit tests v. doctests There are two primary ways to write tests with Django, corresponding to the two test frameworks that ship in the Python standard library. The two frameworks are: • Unit tests – tests that are expressed as methods on a Python class that subclasses unittest.TestCase or Django’s customized TestCase. For example: import unittest class MyFuncTestCase(unittest.TestCase): def testBasic(self): a = [’larry’, ’curly’, ’moe’] self.assertEqual(my_func(a, 0), ’larry’) self.assertEqual(my_func(a, 1), ’curly’) • Doctests – tests that are embedded in your functions’ docstrings and are written in a way that emulates a session of the Python interactive interpreter. For example: def my_func(a_list, idx): """ >>> a = [’larry’, ’curly’, ’moe’] >>> my_func(a, 0) ’larry’ >>> my_func(a, 1) ’curly’ """ return a_list[idx] 294 Chapter 3. Using Django
- 299. Django Documentation, Release1.5.1 Which should I use? Because Django supports both of the standard Python test frameworks, it’s up to you and your tastes to decide which one to use. You can even decide to use both. For developers new to testing, however, this choice can seem confusing. Here, then, are a few key differences to help you decide which approach is right for you: • If you’ve been using Python for a while, doctest will probably feel more “pythonic”. It’s designed to make writing tests as easy as possible, so it requires no overhead of writing classes or methods. You simply put tests in docstrings. This has the added advantage of serving as documentation (and correct documentation, at that!). However, while doctests are good for some simple example code, they are not very good if you want to produce either high quality, comprehensive tests or high quality documentation. Test failures are often difficult to debug as it can be unclear exactly why the test failed. Thus, doctests should generally be avoided and used primarily for documentation examples only. • The unittest framework will probably feel very familiar to developers coming from Java. unittest is inspired by Java’s JUnit, so you’ll feel at home with this method if you’ve used JUnit or any test framework inspired by JUnit. • If you need to write a bunch of tests that share similar code, then you’ll appreciate the unittest framework’s organization around classes and methods. This makes it easy to abstract common tasks into common methods. The framework also supports explicit setup and/or cleanup routines, which give you a high level of control over the environment in which your test cases are run. • If you’re writing tests for Django itself, you should use unittest. 3.8.5 Where to go from here As unit tests are preferred in Django, we treat them in detail in the Testing Django applications document. Django and doctests describes Django-specific features when using doctests. You can also use any other Python test framework, Django provides an API and tools for that kind of integration. They are described in the Using different testing frameworks section of Advanced testing topics. 3.9 User authentication in Django 3.9.1 Using the Django authentication system This document explains the usage of Django’s authentication system in its default configuration. This configuration has evolved to serve the most common project needs, handling a reasonably wide range of tasks, and has a careful implementation of passwords and permissions, and can handle many projects as is. For projects where authentication needs differ from the default, Django supports extensive extension and customization of authentication. Django authentication provides both authentication and authorization, together and is generally referred to as the authentication system, as these features somewhat coupled. User objects User objects are the core of the authentication system. They typically represent the people interacting with your site and are used to enable things like restricting access, registering user profiles, associating content with creators etc. Only one class of user exists in Django’s authentication framework, i.e., ‘superusers’ or admin ‘staff’ users are just user objects with special attributes set, not different classes of user objects. 3.9. User authentication in Django 295
- 300. Django Documentation, Release1.5.1 The primary attributes of the default user are: • username • password • email • first name • last name See the full API documentation for full reference, the documentation that follows is more task oriented. Creating users The most direct way to create users is to use the included create_user() helper function: >>> from django.contrib.auth.models import User >>> user = User.objects.create_user(’john’, ’lennon@thebeatles.com’, ’johnpassword’) # At this point, user is a User object that has already been saved # to the database. You can continue to change its attributes # if you want to change other fields. >>> user.last_name = ’Lennon’ >>> user.save() If you have the Django admin installed, you can also create users interactively. Creating superusers manage.py syncdb prompts you to create a superuser the first time you run it with ’django.contrib.auth’ in your INSTALLED_APPS. If you need to create a superuser at a later date, you can use a command line utility: manage.py createsuperuser --username=joe --email=joe@example.com You will be prompted for a password. After you enter one, the user will be created immediately. If you leave off the --username or the --email options, it will prompt you for those values. Changing passwords Django does not store raw (clear text) passwords on the user model, but only a hash (see documentation of how passwords are managed for full details). Because of this, do not attempt to manipulate the password attribute of the user directly. This is why a helper function is used when creating a user. To change a user’s password, you have several options: manage.py changepassword *username* offers a method of changing a User’s password from the com-mand line. It prompts you to change the password of a given user which you must enter twice. If they both match, the new password will be changed immediately. If you do not supply a user, the command will attempt to change the password whose username matches the current system user. You can also change a password programmatically, using set_password(): >>> from django.contrib.auth.models import User >>> u = User.objects.get(username__exact=’john’) >>> u.set_password(’new password’) >>> u.save() 296 Chapter 3. Using Django
- 301. Django Documentation, Release1.5.1 If you have the Django admin installed, you can also change user’s passwords on the authentication system’s admin pages. Django also provides views and forms that may be used to allow users to change their own passwords. Authenticating Users authenticate(**credentials) To authenticate a given username and password, use authenticate(). It takes credentials in the form of keyword arguments, for the default configuration this is username and password, and it returns a User object if the password is valid for the given username. If the password is invalid, authenticate() returns None. Example: from django.contrib.auth import authenticate user = authenticate(username=’john’, password=’secret’) if user is not None: # the password verified for the user if user.is_active: print("User is valid, active and authenticated") else: print("The password is valid, but the account has been disabled!") else: # the authentication system was unable to verify the username and password print("The username and password were incorrect.") Permissions and Authorization Django comes with a simple permissions system. It provides a way to assign permissions to specific users and groups of users. It’s used by the Django admin site, but you’re welcome to use it in your own code. The Django admin site uses permissions as follows: • Access to view the “add” form and add an object is limited to users with the “add” permission for that type of object. • Access to view the change list, view the “change” form and change an object is limited to users with the “change” permission for that type of object. • Access to delete an object is limited to users with the “delete” permission for that type of object. Permissions can be set not only per type of object, but also per specific object instance. By using the has_add_permission(), has_change_permission() and has_delete_permission() methods provided by the ModelAdmin class, it is possible to customize permissions for different object instances of the same type. User objects have two many-to-many fields: groups and user_permissions. User objects can access their related objects in the same way as any other Django model: myuser.groups = [group_list] myuser.groups.add(group, group, ...) myuser.groups.remove(group, group, ...) myuser.groups.clear() myuser.user_permissions = [permission_list] myuser.user_permissions.add(permission, permission, ...) myuser.user_permissions.remove(permission, permission, ...) myuser.user_permissions.clear() 3.9. User authentication in Django 297
- 302. Django Documentation, Release1.5.1 Default permissions When django.contrib.auth is listed in your INSTALLED_APPS setting, it will ensure that three default per-missions – add, change and delete – are created for each Django model defined in one of your installed applications. These permissions will be created when you run manage.py syncdb; the first time you run syncdb after adding django.contrib.auth to INSTALLED_APPS, the default permissions will be created for all previously-installed models, as well as for any new models being installed at that time. Afterward, it will create default permis-sions for new models each time you run manage.py syncdb. Assuming you have an application with an app_label foo and a model named Bar, to test for basic permissions you should use: • add: user.has_perm(’foo.add_bar’) • change: user.has_perm(’foo.change_bar’) • delete: user.has_perm(’foo.delete_bar’) The Permission model is rarely accessed directly. Groups django.contrib.auth.models.Group models are a generic way of categorizing users so you can apply per-missions, or some other label, to those users. A user can belong to any number of groups. A user in a group automatically has the permissions granted to that group. For example, if the group Site editors has the permission can_edit_home_page, any user in that group will have that permission. Beyond permissions, groups are a convenient way to categorize users to give them some label, or extended functional-ity. For example, you could create a group ’Special users’, and you could write code that could, say, give them access to a members-only portion of your site, or send them members-only email messages. Programmatically creating permissions While custom permissions can be defined within a model’s Meta class, you can also create permissions directly. For example, you can create the can_publish permission for a BlogPost model in myapp: from django.contrib.auth.models import Group, Permission from django.contrib.contenttypes.models import ContentType content_type = ContentType.objects.get(app_label=’myapp’, model=’BlogPost’) permission = Permission.objects.create(codename=’can_publish’, name=’Can Publish Posts’, content_type=content_type) The permission can then be assigned to a User via its user_permissions attribute or to a Group via its permissions attribute. Authentication in Web requests Django uses sessions and middleware to hook the authentication system into request objects. These provide a request.user attribute on every request which represents the current user. If the current user has not logged in, this attribute will be set to an instance of AnonymousUser, otherwise it will be an instance of User. You can tell them apart with is_authenticated(), like so: 298 Chapter 3. Using Django
- 303. Django Documentation, Release1.5.1 if request.user.is_authenticated(): # Do something for authenticated users. else: # Do something for anonymous users. How to log a user in If you have an authenticated user you want to attach to the current session - this is done with a login() function. login() To log a user in, from a view, use login(). It takes an HttpRequest object and a User object. login() saves the user’s ID in the session, using Django’s session framework. Note that any data set during the anonymous session is retained in the session after a user logs in. This example shows how you might use both authenticate() and login(): from django.contrib.auth import authenticate, login def my_view(request): username = request.POST[’username’] password = request.POST[’password’] user = authenticate(username=username, password=password) if user is not None: if user.is_active: login(request, user) # Redirect to a success page. else: # Return a ’disabled account’ error message else: # Return an ’invalid login’ error message. Calling authenticate() first When you’re manually logging a user in, you must call authenticate() before you call login(). authenticate() sets an attribute on the User noting which authentication backend successfully authenticated that user (see the backends documentation for details), and this information is needed later during the login process. An error will be raise if you try to login a user object retrieved from the database directly. How to log a user out logout() To log out a user who has been logged in via django.contrib.auth.login(), use django.contrib.auth.logout() within your view. It takes an HttpRequest object and has no return value. Example: from django.contrib.auth import logout def logout_view(request): logout(request) # Redirect to a success page. Note that logout() doesn’t throw any errors if the user wasn’t logged in. 3.9. User authentication in Django 299
- 304. Django Documentation, Release1.5.1 When you call logout(), the session data for the current request is completely cleaned out. All existing data is removed. This is to prevent another person from using the same Web browser to log in and have access to the previous user’s session data. If you want to put anything into the session that will be available to the user immediately after logging out, do that after calling django.contrib.auth.logout(). Limiting access to logged-in users The raw way The simple, raw way to limit access to pages is to check request.user.is_authenticated() and either redirect to a login page: from django.shortcuts import redirect def my_view(request): if not request.user.is_authenticated(): return redirect(’/login/?next=%s’ % request.path) # ... ...or display an error message: from django.shortcuts import render def my_view(request): if not request.user.is_authenticated(): return render(request, ’myapp/login_error.html’) # ... The login_required decorator login_required([redirect_field_name=REDIRECT_FIELD_NAME, login_url=None ]) As a shortcut, you can use the convenient login_required() decorator: from django.contrib.auth.decorators import login_required @login_required def my_view(request): ... login_required() does the following: •If the user isn’t logged in, redirect to settings.LOGIN_URL, passing the current absolute path in the query string. Example: /accounts/login/?next=/polls/3/. •If the user is logged in, execute the view normally. The view code is free to assume the user is logged in. By default, the path that the user should be redirected to upon successful authentication is stored in a query string parameter called "next". If you would prefer to use a different name for this parameter, login_required() takes an optional redirect_field_name parameter: from django.contrib.auth.decorators import login_required @login_required(redirect_field_name=’my_redirect_field’) def my_view(request): ... Note that if you provide a value to redirect_field_name, you will most likely need to customize your login template as well, since the template context variable which stores the redirect path will use the value of redirect_field_name as its key rather than "next" (the default). login_required() also takes an optional login_url parameter. Example: 300 Chapter 3. Using Django
- 305. Django Documentation, Release1.5.1 from django.contrib.auth.decorators import login_required @login_required(login_url=’/accounts/login/’) def my_view(request): ... Note that if you don’t specify the login_url parameter, you’ll need to ensure that the settings.LOGIN_URL and your login view are properly associated. For example, using the defaults, add the following line to your URLconf: (r’^accounts/login/$’, ’django.contrib.auth.views.login’), Changed in version 1.5. The settings.LOGIN_URL also accepts view function names and named URL patterns. This allows you to freely remap your login view within your URLconf without having to update the setting. Note: The login_required decorator does NOT check the is_active flag on a user. Limiting access to logged-in users that pass a test To limit access based on certain permissions or some other test, you’d do essentially the same thing as described in the previous section. The simple way is to run your test on request.user in the view directly. For example, this view checks to make sure the user has an email in the desired domain: def my_view(request): if not ’@example.com’ in request.user.email: return HttpResponse("You can’t vote in this poll.") # ... user_passes_test(func[, login_url=None ]) As a shortcut, you can use the convenient user_passes_test decorator: from django.contrib.auth.decorators import user_passes_test def email_check(user): return ’@example.com’ in user.email @user_passes_test(email_check) def my_view(request): ... user_passes_test() takes a required argument: a callable that takes a User object and returns True if the user is allowed to view the page. Note that user_passes_test() does not automatically check that the User is not anonymous. user_passes_test() takes an optional login_url argument, which lets you specify the URL for your login page (settings.LOGIN_URL by default). For example: @user_passes_test(email_check, login_url=’/login/’) def my_view(request): ... The permission_required decorator 3.9. User authentication in Django 301
- 306. Django Documentation, Release1.5.1 permission_required([login_url=None, raise_exception=False ]) It’s a relatively common task to check whether a user has a particular permission. For that reason, Django provides a shortcut for that case: the permission_required() decorator.: from django.contrib.auth.decorators import permission_required @permission_required(’polls.can_vote’) def my_view(request): ... As for the has_perm() method, permission names take the form "<app label>.<permission codename>" (i.e. polls.can_vote for a permission on a model in the polls application). Note that permission_required() also takes an optional login_url parameter. Example: from django.contrib.auth.decorators import permission_required @permission_required(’polls.can_vote’, login_url=’/loginpage/’) def my_view(request): ... As in the login_required() decorator, login_url defaults to settings.LOGIN_URL. Changed in version 1.4. Added raise_exception parameter. If given, the decorator will raise PermissionDenied, prompting the 403 (HTTP Forbidden) view instead of redirecting to the login page. Applying permissions to generic views To apply a permission to a class-based generic view, decorate the View.dispatch method on the class. See Decorating the class for details. Authentication Views Django provides several views that you can use for handling login, logout, and password management. These make use of the stock auth forms but you can pass in your own forms as well. Django provides no default template for the authentication views - however the template context is documented for each view below. New in version 1.4. The built-in views all return a TemplateResponse instance, which allows you to easily customize the response data before rendering. For more details, see the TemplateResponse documenta-tion. Most built-in authentication views provide a URL name for easier reference. See the URL documentation for details on using named URL patterns. login(request[, template_name, redirect_field_name, authentication_form ]) URL name: login See the URL documentation for details on using named URL patterns. Here’s what django.contrib.auth.views.login does: •If called via GET, it displays a login form that POSTs to the same URL. More on this in a bit. •If called via POST with user submitted credentials, it tries to log the user in. If login is suc-cessful, the view redirects to the URL specified in next. If next isn’t provided, it redirects to settings.LOGIN_REDIRECT_URL (which defaults to /accounts/profile/). If login isn’t suc-cessful, it redisplays the login form. It’s your responsibility to provide the html for the login template , called registration/login.html by default. This template gets passed four template context variables: •form: A Form object representing the AuthenticationForm. 302 Chapter 3. Using Django
- 307. Django Documentation, Release1.5.1 •next: The URL to redirect to after successful login. This may contain a query string, too. •site: The current Site, according to the SITE_ID setting. If you don’t have the site framework installed, this will be set to an instance of RequestSite, which derives the site name and domain from the current HttpRequest. •site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework. If you’d prefer not to call the template registration/login.html, you can pass the template_name parameter via the extra arguments to the view in your URLconf. For example, this URLconf line would use myapp/login.html instead: (r’^accounts/login/$’, ’django.contrib.auth.views.login’, {’template_name’: ’myapp/login.html’}), You can also specify the name of the GET field which contains the URL to redirect to after login by passing redirect_field_name to the view. By default, the field is called next. Here’s a sample registration/login.html template you can use as a starting point. It assumes you have a base.html template that defines a content block: {% extends "base.html" %} {% block content %} {% if form.errors %} <p>Your username and password didn’t match. Please try again.</p> {% endif %} <form method="post" action="{% url ’django.contrib.auth.views.login’ %}"> {% csrf_token %} <table> <tr> <td>{{ form.username.label_tag }}</td> <td>{{ form.username }}</td> </tr> <tr> <td>{{ form.password.label_tag }}</td> <td>{{ form.password }}</td> </tr> </table> <input type="submit" value="login" /> <input type="hidden" name="next" value="{{ next }}" /> </form> {% endblock %} If you have customized authentication (see Customizing Authentication) you can pass a custom authentication form to the login view via the authentication_form parameter. This form must accept a request keyword argument in its __init__ method, and provide a get_user method which returns the authenticated user object (this method is only ever called after successful form validation). logout(request[, next_page, template_name, redirect_field_name ]) Logs a user out. URL name: logout Optional arguments: •next_page: The URL to redirect to after logout. 3.9. User authentication in Django 303
- 308. Django Documentation, Release1.5.1 •template_name: The full name of a template to display after logging the user out. Defaults to registration/logged_out.html if no argument is supplied. •redirect_field_name: The name of a GET field containing the URL to redirect to after log out. Overrides next_page if the given GET parameter is passed. Template context: •title: The string “Logged out”, localized. •site: The current Site, according to the SITE_ID setting. If you don’t have the site framework installed, this will be set to an instance of RequestSite, which derives the site name and domain from the current HttpRequest. •site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework. logout_then_login(request[, login_url ]) Logs a user out, then redirects to the login page. URL name: No default URL provided Optional arguments: •login_url: The URL of the login page to redirect to. Defaults to settings.LOGIN_URL if not supplied. password_change(request[, template_name, post_change_redirect, password_change_form ]) Allows a user to change their password. URL name: password_change Optional arguments: •template_name: The full name of a template to use for displaying the password change form. Defaults to registration/password_change_form.html if not supplied. •post_change_redirect: The URL to redirect to after a successful password change. •password_change_form: A custom “change password” form which must accept a user key-word argument. The form is responsible for actually changing the user’s password. Defaults to PasswordChangeForm. Template context: •form: The password change form (see password_change_form above). password_change_done(request[, template_name ]) The page shown after a user has changed their password. URL name: password_change_done Optional arguments: •template_name: The full name of a template to use. Defaults to registration/password_change_done.html if not supplied. password_reset(request[, is_admin_site, template_name, email_template_name, password_reset_form, token_generator, post_reset_redirect, from_email ]) Allows a user to reset their password by generating a one-time use link that can be used to reset the password, and sending that link to the user’s registered email address. Changed in version 1.4: Users flagged with an unusable password (see set_unusable_password() will not be able to request a password reset to prevent misuse when using an external authentication source like LDAP. URL name: password_reset Optional arguments: 304 Chapter 3. Using Django
- 309. Django Documentation, Release1.5.1 •template_name: The full name of a template to use for displaying the password reset form. Defaults to registration/password_reset_form.html if not supplied. •email_template_name: The full name of a template to use for generating the email with the reset password link. Defaults to registration/password_reset_email.html if not supplied. •subject_template_name: The full name of a template to use for the subject of the email with the reset password link. Defaults to registration/password_reset_subject.txt if not supplied. New in version 1.4. •password_reset_form: Form that will be used to get the email of the user to reset the password for. Defaults to PasswordResetForm. •token_generator: Instance of the class to check the one time link. This will default to default_token_generator, it’s an instance of django.contrib.auth.tokens.PasswordResetTokenGenerator. •post_reset_redirect: The URL to redirect to after a successful password reset request. •from_email: A valid email address. By default Django uses the DEFAULT_FROM_EMAIL. Template context: •form: The form (see password_reset_form above) for resetting the user’s password. Email template context: •email: An alias for user.email •user: The current User, according to the email form field. Only active users are able to reset their passwords (User.is_active is True). •site_name: An alias for site.name. If you don’t have the site framework installed, this will be set to the value of request.META[’SERVER_NAME’]. For more on sites, see The “sites” framework. •domain: An alias for site.domain. If you don’t have the site framework installed, this will be set to the value of request.get_host(). •protocol: http or https •uid: The user’s id encoded in base 36. •token: Token to check that the reset link is valid. Sample registration/password_reset_email.html (email body template): Someone asked for password reset for email {{ email }}. Follow the link below: {{ protocol}}://{{ domain }}{% url ’password_reset_confirm’ uidb36=uid token=token %} The same template context is used for subject template. Subject must be single line plain text string. password_reset_done(request[, template_name ]) The page shown after a user has been emailed a link to reset their password. This view is called by default if the password_reset() view doesn’t have an explicit post_reset_redirect URL set. URL name: password_reset_done Optional arguments: •template_name: The full name of a template to use. Defaults to registration/password_reset_done.html if not supplied. password_reset_confirm(request[, uidb36, token, template_name, token_generator, set_password_form, post_reset_redirect ]) Presents a form for entering a new password. 3.9. User authentication in Django 305
- 310. Django Documentation, Release1.5.1 URL name: password_reset_confirm Optional arguments: •uidb36: The user’s id encoded in base 36. Defaults to None. •token: Token to check that the password is valid. Defaults to None. •template_name: The full name of a template to display the confirm password view. Default value is registration/password_reset_confirm.html. •token_generator: Instance of the class to check the password. This will default to default_token_generator, it’s an instance of django.contrib.auth.tokens.PasswordResetTokenGenerator. •set_password_form: Form that will be used to set the password. Defaults to SetPasswordForm •post_reset_redirect: URL to redirect after the password reset done. Defaults to None. Template context: •form: The form (see set_password_form above) for setting the new user’s password. •validlink: Boolean, True if the link (combination of uidb36 and token) is valid or unused yet. password_reset_complete(request[, template_name ]) Presents a view which informs the user that the password has been successfully changed. URL name: password_reset_complete Optional arguments: •template_name: The full name of a template to display the view. Defaults to registration/password_reset_complete.html. Helper functions redirect_to_login(next[, login_url, redirect_field_name ]) Redirects to the login page, and then back to another URL after a successful login. Required arguments: •next: The URL to redirect to after a successful login. Optional arguments: •login_url: The URL of the login page to redirect to. Defaults to settings.LOGIN_URL if not supplied. •redirect_field_name: The name of a GET field containing the URL to redirect to after log out. Overrides next if the given GET parameter is passed. Built-in forms If you don’t want to use the built-in views, but want the convenience of not having to write forms for this functionality, the authentication system provides several built-in forms located in django.contrib.auth.forms: Note: The built-in authentication forms make certain assumptions about the user model that they are working with. If you’re using a custom User model, it may be necessary to define your own forms for the authentication system. For more information, refer to the documentation about using the built-in authentication forms with custom user models. 306 Chapter 3. Using Django
- 311. Django Documentation, Release1.5.1 class AdminPasswordChangeForm A form used in the admin interface to change a user’s password. class AuthenticationForm A form for logging a user in. class PasswordChangeForm A form for allowing a user to change their password. class PasswordResetForm A form for generating and emailing a one-time use link to reset a user’s password. class SetPasswordForm A form that lets a user change his/her password without entering the old password. class UserChangeForm A form used in the admin interface to change a user’s information and permissions. class UserCreationForm A form for creating a new user. Authentication data in templates The currently logged-in user and his/her permissions are made available in the template context when you use RequestContext. Technicality Technically, these variables are only made available in the template context if you use RequestContext and your TEMPLATE_CONTEXT_PROCESSORS setting contains "django.contrib.auth.context_processors.auth", which is default. For more, see the Re-questContext docs. Users When rendering a template RequestContext, the currently logged-in user, either a User instance or an AnonymousUser instance, is stored in the template variable {{ user }}: {% if user.is_authenticated %} <p>Welcome, {{ user.username }}. Thanks for logging in.</p> {% else %} <p>Welcome, new user. Please log in.</p> {% endif %} This template context variable is not available if a RequestContext is not being used. Permissions The currently logged-in user’s permissions are stored in the template variable {{ perms }}. This is an instance of django.contrib.auth.context_processors.PermWrapper, which is a template-friendly proxy of permissions. In the {{ perms }} object, single-attribute lookup is a proxy to User.has_module_perms. This example would display True if the logged-in user had any permissions in the foo app: {{ perms.foo }} Two-level-attribute lookup is a proxy to User.has_perm. This example would display True if the logged-in user had the permission foo.can_vote: 3.9. User authentication in Django 307
- 312. Django Documentation, Release1.5.1 {{ perms.foo.can_vote }} Thus, you can check permissions in template {% if %} statements: {% if perms.foo %} <p>You have permission to do something in the foo app.</p> {% if perms.foo.can_vote %} <p>You can vote!</p> {% endif %} {% if perms.foo.can_drive %} <p>You can drive!</p> {% endif %} {% else %} <p>You don’t have permission to do anything in the foo app.</p> {% endif %} New in version 1.5: Permission lookup by “if in”. It is possible to also look permissions up by {% if in %} statements. For example: {% if ’foo’ in perms %} {% if ’foo.can_vote’ in perms %} <p>In lookup works, too.</p> {% endif %} {% endif %} Managing users in the admin When you have both django.contrib.admin and django.contrib.auth installed, the admin provides a convenient way to view and manage users, groups, and permissions. Users can be created and deleted like any Django model. Groups can be created, and permissions can be assigned to users or groups. A log of user edits to models made within the admin is also stored and displayed. Creating Users You should see a link to “Users” in the “Auth” section of the main admin index page. The “Add user” admin page is different than standard admin pages in that it requires you to choose a username and password before allowing you to edit the rest of the user’s fields. Also note: if you want a user account to be able to create users using the Django admin site, you’ll need to give them permission to add users and change users (i.e., the “Add user” and “Change user” permissions). If an account has permission to add users but not to change them, that account won’t be able to add users. Why? Because if you have permission to add users, you have the power to create superusers, which can then, in turn, change other users. So Django requires add and change permissions as a slight security measure. Changing Passwords User passwords are not displayed in the admin (nor stored in the database), but the password storage details are displayed. Included in the display of this information is a link to a password change form that allows admins to change user passwords. 308 Chapter 3. Using Django
- 313. Django Documentation, Release1.5.1 3.9.2 Password management in Django Password management is something that should generally not be reinvented unnecessarily, and Django endeavors to provide a secure and flexible set of tools for managing user passwords. This document describes how Django stores passwords, how the storage hashing can be configured, and some utilities to work with hashed passwords. How Django stores passwords New in version 1.4: Django 1.4 introduces a new flexible password storage system and uses PBKDF2 by default. Previous versions of Django used SHA1, and other algorithms couldn’t be chosen. The password attribute of a User object is a string in this format: <algorithm>$<iterations>$<salt>$<hash> Those are the components used for storing a User’s password, separated by the dollar-sign character and consist of: the hashing algorithm, the number of algorithm iterations (work factor), the random salt, and the resulting password hash. The algorithm is one of a number of one-way hashing or password storage algorithms Django can use; see below. Iterations describe the number of times the algorithm is run over the hash. Salt is the random seed used and the hash is the result of the one-way function. By default, Django uses the PBKDF2 algorithm with a SHA256 hash, a password stretching mechanism recommended by NIST. This should be sufficient for most users: it’s quite secure, requiring massive amounts of computing time to break. However, depending on your requirements, you may choose a different algorithm, or even use a custom algorithm to match your specific security situation. Again, most users shouldn’t need to do this – if you’re not sure, you probably don’t. If you do, please read on: Django chooses the algorithm to use by consulting the PASSWORD_HASHERS setting. This is a list of hashing algorithm classes that this Django installation supports. The first entry in this list (that is, settings.PASSWORD_HASHERS[0]) will be used to store passwords, and all the other entries are valid hash-ers that can be used to check existing passwords. This means that if you want to use a different algorithm, you’ll need to modify PASSWORD_HASHERS to list your preferred algorithm first in the list. The default for PASSWORD_HASHERS is: PASSWORD_HASHERS = ( ’django.contrib.auth.hashers.PBKDF2PasswordHasher’, ’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’, ’django.contrib.auth.hashers.BCryptPasswordHasher’, ’django.contrib.auth.hashers.SHA1PasswordHasher’, ’django.contrib.auth.hashers.MD5PasswordHasher’, ’django.contrib.auth.hashers.CryptPasswordHasher’, ) This means that Django will use PBKDF2 to store all passwords, but will support checking passwords stored with PBKDF2SHA1, bcrypt, SHA1, etc. The next few sections describe a couple of common ways advanced users may want to modify this setting. Using bcrypt with Django Bcrypt is a popular password storage algorithm that’s specifically designed for long-term password storage. It’s not the default used by Django since it requires the use of third-party libraries, but since many people may want to use it Django supports bcrypt with minimal effort. To use Bcrypt as your default storage algorithm, do the following: 3.9. User authentication in Django 309
- 314. Django Documentation, Release1.5.1 1. Install the py-bcrypt library (probably by running sudo pip install py-bcrypt, or downloading the library and installing it with python setup.py install). 2. Modify PASSWORD_HASHERS to list BCryptPasswordHasher first. That is, in your settings file, you’d put: PASSWORD_HASHERS = ( ’django.contrib.auth.hashers.BCryptPasswordHasher’, ’django.contrib.auth.hashers.PBKDF2PasswordHasher’, ’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’, ’django.contrib.auth.hashers.SHA1PasswordHasher’, ’django.contrib.auth.hashers.MD5PasswordHasher’, ’django.contrib.auth.hashers.CryptPasswordHasher’, ) (You need to keep the other entries in this list, or else Django won’t be able to upgrade passwords; see below). That’s it – now your Django install will use Bcrypt as the default storage algorithm. Password truncation with BCryptPasswordHasher The designers of bcrypt truncate all passwords at 72 characters which means that bcrypt(password_with_100_chars) == bcrypt(password_with_100_chars[:72]). BCryptPasswordHasher does not have any special handling and thus is also subject to this hidden pass-word length limit. The practical ramification of this truncation is pretty marginal as the average user does not have a password greater than 72 characters in length and even being truncated at 72 the compute powered required to brute force bcrypt in any useful amount of time is still astronomical. Other bcrypt implementations There are several other implementations that allow bcrypt to be used with Django. Django’s bcrypt support is NOT directly compatible with these. To upgrade, you will need to modify the hashes in your database to be in the form bcrypt$(raw bcrypt output). For example: bcrypt$$2a$12$NT0I31Sa7ihGEWpka9ASYrEFkhuTNeBQ2xfZskIiiJeyFXhRgS.Sy. Increasing the work factor The PBKDF2 and bcrypt algorithms use a number of iterations or rounds of hashing. This deliberately slows down attackers, making attacks against hashed passwords harder. However, as computing power increases, the number of iterations needs to be increased. We’ve chosen a reasonable default (and will increase it with each release of Django), but you may wish to tune it up or down, depending on your security needs and available processing power. To do so, you’ll subclass the appropriate algorithm and override the iterations parameters. For example, to increase the number of iterations used by the default PBKDF2 algorithm: 1. Create a subclass of django.contrib.auth.hashers.PBKDF2PasswordHasher: from django.contrib.auth.hashers import PBKDF2PasswordHasher class MyPBKDF2PasswordHasher(PBKDF2PasswordHasher): """ A subclass of PBKDF2PasswordHasher that uses 100 times more iterations. """ iterations = PBKDF2PasswordHasher.iterations * 100 Save this somewhere in your project. For example, you might put this in a file like myproject/hashers.py. 310 Chapter 3. Using Django
- 315. Django Documentation, Release1.5.1 2. Add your new hasher as the first entry in PASSWORD_HASHERS: PASSWORD_HASHERS = ( ’myproject.hashers.MyPBKDF2PasswordHasher’, ’django.contrib.auth.hashers.PBKDF2PasswordHasher’, ’django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher’, ’django.contrib.auth.hashers.BCryptPasswordHasher’, ’django.contrib.auth.hashers.SHA1PasswordHasher’, ’django.contrib.auth.hashers.MD5PasswordHasher’, ’django.contrib.auth.hashers.CryptPasswordHasher’, ) That’s it – now your Django install will use more iterations when it stores passwords using PBKDF2. Password upgrading When users log in, if their passwords are stored with anything other than the preferred algorithm, Django will auto-matically upgrade the algorithm to the preferred one. This means that old installs of Django will get automatically more secure as users log in, and it also means that you can switch to new (and better) storage algorithms as they get invented. However, Django can only upgrade passwords that use algorithms mentioned in PASSWORD_HASHERS, so as you upgrade to new systems you should make sure never to remove entries from this list. If you do, users using un-mentioned algorithms won’t be able to upgrade. Manually managing a user’s password New in version 1.4: The django.contrib.auth.hashers module provides a set of functions to create and validate hashed password. You can use them independently from the User model. check_password(password, encoded) New in version 1.4. If you’d like to manually authenticate a user by comparing a plain-text password to the hashed password in the database, use the convenience function check_password(). It takes two arguments: the plain-text password to check, and the full value of a user’s password field in the database to check against, and returns True if they match, False otherwise. make_password(password[, salt, hashers ]) New in version 1.4. Creates a hashed password in the format used by this application. It takes one mandatory argument: the password in plain-text. Optionally, you can provide a salt and a hashing algorithm to use, if you don’t want to use the defaults (first entry of PASSWORD_HASHERS setting). Currently supported algorithms are: ’pbkdf2_sha256’, ’pbkdf2_sha1’, ’bcrypt’ (see Using bcrypt with Django), ’sha1’, ’md5’, ’unsalted_md5’ (only for backward compatibility) and ’crypt’ if you have the crypt library installed. If the password argument is None, an unusable password is returned (a one that will be never accepted by check_password()). is_password_usable(encoded_password) New in version 1.4. Checks if the given string is a hashed password that has a chance of being verified against check_password(). 3.9.3 Customizing authentication in Django The authentication that comes with Django is good enough for most common cases, but you may have needs not met by the out-of-the-box defaults. To customize authentication to your projects needs involves understanding what points of the provided system are extendible or replaceable. This document provides details about how the auth system can be customized. 3.9. User authentication in Django 311
- 316. Django Documentation, Release1.5.1 Authentication backends provide an extensible system for when a username and password stored with the User model need to be authenticated against a different service than Django’s default. You can give your models custom permissions that can be checked through Django’s authorization system. You can extend the default User model, or substitute a completely customized model. Other authentication sources There may be times you have the need to hook into another authentication source – that is, another source of usernames and passwords or authentication methods. For example, your company may already have an LDAP setup that stores a username and password for every employee. It’d be a hassle for both the network administrator and the users themselves if users had separate accounts in LDAP and the Django-based applications. So, to handle situations like this, the Django authentication system lets you plug in other authentication sources. You can override Django’s default database-based scheme, or you can use the default system in tandem with other systems. See the authentication backend reference <authentication-backends-reference> for information on the authentication backends included with Django. Specifying authentication backends Behind the scenes, Django maintains a list of “authentication backends” that it checks for authentication. When somebody calls django.contrib.auth.authenticate() – as described in How to log a user in above – Django tries authenticating across all of its authentication backends. If the first authentication method fails, Django tries the second one, and so on, until all backends have been attempted. The list of authentication backends to use is specified in the AUTHENTICATION_BACKENDS setting. This should be a tuple of Python path names that point to Python classes that know how to authenticate. These classes can be anywhere on your Python path. By default, AUTHENTICATION_BACKENDS is set to: (’django.contrib.auth.backends.ModelBackend’,) That’s the basic authentication backend that checks the Django users database and queries the built-in permissions. It does not provide protection against brute force attacks via any rate limiting mechanism. You may either implement your own rate limiting mechanism in a custom auth backend, or use the mechanisms provided by most Web servers. The order of AUTHENTICATION_BACKENDS matters, so if the same username and password is valid in multiple backends, Django will stop processing at the first positive match. Note: Once a user has authenticated, Django stores which backend was used to authenticate the user in the user’s session, and re-uses the same backend for the duration of that session whenever access to the currently authenticated user is needed. This effectively means that authentication sources are cached on a per-session basis, so if you change AUTHENTICATION_BACKENDS, you’ll need to clear out session data if you need to force users to re-authenticate using different methods. A simple way to do that is simply to execute Session.objects.all().delete(). Writing an authentication backend An authentication backend is a class that implements two required methods: get_user(user_id) and authenticate(**credentials), as well as a set of optional permission related authorization methods. 312 Chapter 3. Using Django
- 317. Django Documentation, Release1.5.1 The get_user method takes a user_id – which could be a username, database ID or whatever – and returns a User object. The authenticate method takes credentials as keyword arguments. Most of the time, it’ll just look like this: class MyBackend(object): def authenticate(self, username=None, password=None): # Check the username/password and return a User. ... But it could also authenticate a token, like so: class MyBackend(object): def authenticate(self, token=None): # Check the token and return a User. ... Either way, authenticate should check the credentials it gets, and it should return a User object that matches those credentials, if the credentials are valid. If they’re not valid, it should return None. The Django admin system is tightly coupled to the Django User object described at the beginning of this document. For now, the best way to deal with this is to create a Django User object for each user that exists for your backend (e.g., in your LDAP directory, your external SQL database, etc.) You can either write a script to do this in advance, or your authenticate method can do it the first time a user logs in. Here’s an example backend that authenticates against a username and password variable defined in your settings.py file and creates a Django User object the first time a user authenticates: from django.conf import settings from django.contrib.auth.models import User, check_password class SettingsBackend(object): """ Authenticate against the settings ADMIN_LOGIN and ADMIN_PASSWORD. Use the login name, and a hash of the password. For example: ADMIN_LOGIN = ’admin’ ADMIN_PASSWORD = ’sha1$4e987$afbcf42e21bd417fb71db8c66b321e9fc33051de’ """ def authenticate(self, username=None, password=None): login_valid = (settings.ADMIN_LOGIN == username) pwd_valid = check_password(password, settings.ADMIN_PASSWORD) if login_valid and pwd_valid: try: user = User.objects.get(username=username) except User.DoesNotExist: # Create a new user. Note that we can set password # to anything, because it won’t be checked; the password # from settings.py will. user = User(username=username, password=’get from settings.py’) user.is_staff = True user.is_superuser = True user.save() return user return None def get_user(self, user_id): try: 3.9. User authentication in Django 313
- 318. Django Documentation, Release1.5.1 return User.objects.get(pk=user_id) except User.DoesNotExist: return None Handling authorization in custom backends Custom auth backends can provide their own permissions. The user model will delegate permission lookup functions (get_group_permissions(), get_all_permissions(), has_perm(), and has_module_perms()) to any authentication backend that implements these functions. The permissions given to the user will be the superset of all permissions returned by all backends. That is, Django grants a permission to a user that any one backend grants. The simple backend above could implement permissions for the magic admin fairly simply: class SettingsBackend(object): ... def has_perm(self, user_obj, perm, obj=None): if user_obj.username == settings.ADMIN_LOGIN: return True else: return False This gives full permissions to the user granted access in the above example. Notice that in addition to the same arguments given to the associated django.contrib.auth.models.User functions, the backend auth functions all take the user object, which may be an anonymous user, as an argument. A full authorization implementation can be found in the ModelBackend class in django/contrib/auth/backends.py, which is the default backend and queries the auth_permission table most of the time. If you wish to pro-vide custom behavior for only part of the backend API, you can take advantage of Python inheritance and subclass ModelBackend instead of implementing the complete API in a custom backend. Authorization for anonymous users An anonymous user is one that is not authenticated i.e. they have provided no valid authentication details. However, that does not necessarily mean they are not authorized to do anything. At the most basic level, most Web sites authorize anonymous users to browse most of the site, and many allow anonymous posting of comments etc. Django’s permission framework does not have a place to store permissions for anonymous users. However, the user object passed to an authentication backend may be an django.contrib.auth.models.AnonymousUser object, allowing the backend to specify custom authorization behavior for anonymous users. This is especially useful for the authors of re-usable apps, who can delegate all questions of authorization to the auth backend, rather than needing settings, for example, to control anonymous access. Authorization for inactive users An inactive user is a one that is authenticated but has its attribute is_active set to False. However this does not mean they are not authorized to do anything. For example they are allowed to activate their account. The support for anonymous users in the permission system allows for a scenario where anonymous users have permis-sions to do something while inactive authenticated users do not. Do not forget to test for the is_active attribute of the user in your own backend permission methods. 314 Chapter 3. Using Django
- 319. Django Documentation, Release1.5.1 Handling object permissions Django’s permission framework has a foundation for object permissions, though there is no implementation for it in the core. That means that checking for object permissions will always return False or an empty list (depending on the check performed). An authentication backend will receive the keyword parameters obj and user_obj for each object related authorization method and can return the object level permission as appropriate. Custom permissions To create custom permissions for a given model object, use the permissions model Meta attribute. This example Task model creates three custom permissions, i.e., actions users can or cannot do with Task instances, specific to your application: class Task(models.Model): ... class Meta: permissions = ( ("view_task", "Can see available tasks"), ("change_task_status", "Can change the status of tasks"), ("close_task", "Can remove a task by setting its status as closed"), ) The only thing this does is create those extra permissions when you run manage.py syncdb. Your code is in charge of checking the value of these permissions when an user is trying to access the functionality provided by the application (viewing tasks, changing the status of tasks, closing tasks.) Continuing the above example, the following checks if a user may view tasks: user.has_perm(’app.view_task’) Extending the existing User model There are two ways to extend the default User model without substituting your own model. If the changes you need are purely behavioral, and don’t require any change to what is stored in the database, you can create a proxy model based on User. This allows for any of the features offered by proxy models including default ordering, custom managers, or custom model methods. If you wish to store information related to User, you can use a one-to-one relationship to a model containing the fields for additional information. This one-to-one model is often called a profile model, as it might store non-auth related information about a site user. For example you might create an Employee model: from django.contrib.auth.models import User class Employee(models.Model): user = models.OneToOneField(User) department = models.CharField(max_length=100) Assuming an existing Employee Fred Smith who has both a User and Employee model, you can access the related information using Django’s standard related model conventions: >>> u = User.objects.get(username=’fsmith’) >>> freds_department = u.employee.department To add a profile model’s fields to the user page in the admin, define an InlineModelAdmin (for this example, we’ll use a StackedInline) in your app’s admin.py and add it to a UserAdmin class which is registered with the User class: 3.9. User authentication in Django 315
- 320. Django Documentation, Release1.5.1 from django.contrib import admin from django.contrib.auth.admin import UserAdmin from django.contrib.auth.models import User from my_user_profile_app.models import Employee # Define an inline admin descriptor for Employee model # which acts a bit like a singleton class EmployeeInline(admin.StackedInline): model = Employee can_delete = False verbose_name_plural = ’employee’ # Define a new User admin class UserAdmin(UserAdmin): inlines = (EmployeeInline, ) # Re-register UserAdmin admin.site.unregister(User) admin.site.register(User, UserAdmin) These profile models are not special in any way - they are just Django models that happen to have a one-to- one link with a User model. As such, they do not get auto created when a user is created, but a django.db.models.signals.post_save could be used to create or update related models as appropriate. Note that using related models results in additional queries or joins to retrieve the related data, and depending on your needs substituting the User model and adding the related fields may be your better option. However existing links to the default User model within your project’s apps may justify the extra database load. Deprecated since version 1.5: With the introduction of custom User models, the use of AUTH_PROFILE_MODULE to define a single profile model is no longer supported. See the Django 1.5 release notes for more information. Prior to 1.5, a single profile model could be specified site-wide with the setting AUTH_PROFILE_MODULE with a string consisting of the following items, separated by a dot: 1. The name of the application (case sensitive) in which the user profile model is defined (in other words, the name which was passed to manage.py startapp to create the application). 2. The name of the model (not case sensitive) class. For example, if the profile model was a class named UserProfile and was defined inside an application named accounts, the appropriate setting would be: AUTH_PROFILE_MODULE = ’accounts.UserProfile’ When a user profile model has been defined and specified in this manner, each User object will have a method – get_profile() – which returns the instance of the user profile model associated with that User. The method get_profile() does not create a profile if one does not exist. Substituting a custom User model New in version 1.5. Some kinds of projects may have authentication requirements for which Django’s built-in User model is not always appropriate. For instance, on some sites it makes more sense to use an email address as your identification token instead of a username. Django allows you to override the default User model by providing a value for the AUTH_USER_MODEL setting that references a custom model: 316 Chapter 3. Using Django
- 321. Django Documentation, Release1.5.1 AUTH_USER_MODEL = ’myapp.MyUser’ This dotted pair describes the name of the Django app (which must be in your INSTALLED_APPS), and the name of the Django model that you wish to use as your User model. Warning Changing AUTH_USER_MODEL has a big effect on your database structure. It changes the tables that are avail-able, and it will affect the construction of foreign keys and many-to-many relationships. If you intend to set AUTH_USER_MODEL, you should set it before running manage.py syncdb for the first time. If you have an existing project and you want to migrate to using a custom User model, you may need to look into using a migration tool like South to ease the transition. Referencing the User model If you reference User directly (for example, by referring to it in a foreign key), your code will not work in projects where the AUTH_USER_MODEL setting has been changed to a different User model. get_user_model() Instead of referring to User directly, you should reference the user model using django.contrib.auth.get_user_model(). This method will return the currently active User model – the custom User model if one is specified, or User otherwise. When you define a foreign key or many-to-many relations to the User model, you should specify the custom model using the AUTH_USER_MODEL setting. For example: from django.conf import settings from django.db import models class Article(models.Model): author = models.ForeignKey(settings.AUTH_USER_MODEL) Specifying a custom User model Model design considerations Think carefully before handling information not directly related to authentication in your custom User Model. It may be better to store app-specific user information in a model that has a relation with the User model. That allows each app to specify its own user data requirements without risking conflicts with other apps. On the other hand, queries to retrieve this related information will involve a database join, which may have an effect on performance. Django expects your custom User model to meet some minimum requirements. 1. Your model must have an integer primary key. 2. Your model must have a single unique field that can be used for identification purposes. This can be a username, an email address, or any other unique attribute. 3. Your model must provide a way to address the user in a “short” and “long” form. The most common interpreta-tion of this would be to use the user’s given name as the “short” identifier, and the user’s full name as the “long” identifier. However, there are no constraints on what these two methods return - if you want, they can return exactly the same value. 3.9. User authentication in Django 317
- 322. Django Documentation, Release1.5.1 The easiest way to construct a compliant custom User model is to inherit from AbstractBaseUser. AbstractBaseUser provides the core implementation of a User model, including hashed passwords and tok-enized password resets. You must then provide some key implementation details: class models.CustomUser USERNAME_FIELD A string describing the name of the field on the User model that is used as the unique identifier. This will usually be a username of some kind, but it can also be an email address, or any other unique identifier. The field must be unique (i.e., have unique=True set in its definition). In the following example, the field identifier is used as the identifying field: class MyUser(AbstractBaseUser): identifier = models.CharField(max_length=40, unique=True, db_index=True) ... USERNAME_FIELD = ’identifier’ REQUIRED_FIELDS A list of the field names that must be provided when creating a user via the createsuperuser man-agement command. The user will be prompted to supply a value for each of these fields. It should include any field for which blank is False or undefined, but may include additional fields you want prompted for when a user is created interactively. However, it will not work for ForeignKey fields. For example, here is the partial definition for a User model that defines two required fields - a date of birth and height: class MyUser(AbstractBaseUser): ... date_of_birth = models.DateField() height = models.FloatField() ... REQUIRED_FIELDS = [’date_of_birth’, ’height’] Note: REQUIRED_FIELDS must contain all required fields on your User model, but should not contain the USERNAME_FIELD. is_active A boolean attribute that indicates whether the user is considered “active”. This attribute is provided as an attribute on AbstractBaseUser defaulting to True. How you choose to implement it will de-pend on the details of your chosen auth backends. See the documentation of the attribute on the builtin user model for details. get_full_name() A longer formal identifier for the user. A common interpretation would be the full name name of the user, but it can be any string that identifies the user. get_short_name() A short, informal identifier for the user. A common interpretation would be the first name of the user, but it can be any string that identifies the user in an informal way. It may also return the same value as django.contrib.auth.models.User.get_full_name(). The following methods are available on any subclass of AbstractBaseUser: class models.AbstractBaseUser 318 Chapter 3. Using Django
- 323. Django Documentation, Release1.5.1 get_username() Returns the value of the field nominated by USERNAME_FIELD. is_anonymous() Always returns False. This is a way of differentiating from AnonymousUser objects. Generally, you should prefer using is_authenticated() to this method. is_authenticated() Always returns True. This is a way to tell if the user has been authenticated. This does not imply any permissions, and doesn’t check if the user is active - it only indicates that the user has provided a valid username and password. set_password(raw_password) Sets the user’s password to the given raw string, taking care of the password hashing. Doesn’t save the AbstractBaseUser object. check_password(raw_password) Returns True if the given raw string is the correct password for the user. (This takes care of the password hashing in making the comparison.) set_unusable_password() Marks the user as having no password set. This isn’t the same as having a blank string for a password. check_password() for this user will never return True. Doesn’t save the AbstractBaseUser object. You may need this if authentication for your application takes place against an existing external source such as an LDAP directory. has_usable_password() Returns False if set_unusable_password() has been called for this user. You should also define a custom manager for your User model. If your User model defines username and email fields the same as Django’s default User, you can just install Django’s UserManager; however, if your User model defines different fields, you will need to define a custom manager that extends BaseUserManager providing two additional methods: class models.CustomUserManager create_user(*username_field*, password=None, **other_fields) The prototype of create_user() should accept the username field, plus all required fields as argu-ments. For example, if your user model uses email as the username field, and has date_of_birth as a required fields, then create_user should be defined as: def create_user(self, email, date_of_birth, password=None): # create user here ... create_superuser(*username_field*, password, **other_fields) The prototype of create_superuser() should accept the username field, plus all required fields as arguments. For example, if your user model uses email as the username field, and has date_of_birth as a required fields, then create_superuser should be defined as: def create_superuser(self, email, date_of_birth, password): # create superuser here ... Unlike create_user(), create_superuser() must require the caller to provider a password. BaseUserManager provides the following utility methods: 3.9. User authentication in Django 319
- 324. Django Documentation, Release1.5.1 class models.BaseUserManager normalize_email(email) A classmethod that normalizes email addresses by lowercasing the domain portion of the email address. get_by_natural_key(username) Retrieves a user instance using the contents of the field nominated by USERNAME_FIELD. make_random_password(length=10, allowed_chars=’abcdefghjkmnpqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ23456789’) Returns a random password with the given length and given string of allowed characters. (Note that the default value of allowed_chars doesn’t contain letters that can cause user confusion, including: •i, l, I, and 1 (lowercase letter i, lowercase letter L, uppercase letter i, and the number one) •o, O, and 0 (uppercase letter o, lowercase letter o, and zero) Extending Django’s default User If you’re entirely happy with Django’s User model and you just want to add some additional profile information, you can simply subclass django.contrib.auth.models.AbstractUser and add your custom profile fields. This class provides the full implementation of the default User as an abstract model. Custom users and the built-in auth forms As you may expect, built-in Django’s forms and views make certain assumptions about the user model that they are working with. If your user model doesn’t follow the same assumptions, it may be necessary to define a replacement form, and pass that form in as part of the configuration of the auth views. • UserCreationForm Depends on the User model. Must be re-written for any custom user model. • UserChangeForm Depends on the User model. Must be re-written for any custom user model. • AuthenticationForm Works with any subclass of AbstractBaseUser, and will adapt to use the field defined in USER-NAME_ FIELD. • PasswordResetForm Assumes that the user model has an integer primary key, has a field named email that can be used to identify the user, and a boolean field named is_active to prevent password resets for inactive users. • SetPasswordForm Works with any subclass of AbstractBaseUser • PasswordChangeForm Works with any subclass of AbstractBaseUser • AdminPasswordChangeForm Works with any subclass of AbstractBaseUser 320 Chapter 3. Using Django
- 325. Django Documentation, Release1.5.1 Custom users and django.contrib.admin If you want your custom User model to also work with Admin, your User model must define some additional attributes and methods. These methods allow the admin to control access of the User to admin content: class models.CustomUser is_staff Returns True if the user is allowed to have access to the admin site. is_active Returns True if the user account is currently active. has_perm(perm, obj=None): Returns True if the user has the named permission. If obj is provided, the permission needs to be checked against a specific object instance. has_module_perms(app_label): Returns True if the user has permission to access models in the given app. You will also need to register your custom User model with the admin. If your custom User model extends django.contrib.auth.models.AbstractUser, you can use Django’s exist-ing django.contrib.auth.admin.UserAdmin class. However, if your User model extends AbstractBaseUser, you’ll need to define a custom ModelAdmin class. It may be possible to subclass the default django.contrib.auth.admin.UserAdmin; however, you’ll need to override any of the definitions that refer to fields on django.contrib.auth.models.AbstractUser that aren’t on your custom User class. Custom users and permissions To make it easy to include Django’s permission framework into your own User class, Django provides PermissionsMixin. This is an abstract model you can include in the class hierarchy for your User model, giving you all the methods and database fields necessary to support Django’s permission model. PermissionsMixin provides the following methods and attributes: class models.PermissionsMixin is_superuser Boolean. Designates that this user has all permissions without explicitly assigning them. get_group_permissions(obj=None) Returns a set of permission strings that the user has, through his/her groups. If obj is passed in, only returns the group permissions for this specific object. get_all_permissions(obj=None) Returns a set of permission strings that the user has, both through group and user permissions. If obj is passed in, only returns the permissions for this specific object. has_perm(perm, obj=None) Returns True if the user has the specified permission, where perm is in the format "<app label>.<permission codename>" (see permissions). If the user is inactive, this method will always return False. If obj is passed in, this method won’t check for a permission for the model, but for this specific object. 3.9. User authentication in Django 321
- 326. Django Documentation, Release1.5.1 has_perms(perm_list, obj=None) Returns True if the user has each of the specified permissions, where each perm is in the format "<app label>.<permission codename>". If the user is inactive, this method will always return False. If obj is passed in, this method won’t check for permissions for the model, but for the specific object. has_module_perms(package_name) Returns True if the user has any permissions in the given package (the Django app label). If the user is inactive, this method will always return False. ModelBackend If you don’t include the PermissionsMixin, you must ensure you don’t invoke the permissions methods on ModelBackend. ModelBackend assumes that certain fields are available on your user model. If your User model doesn’t provide those fields, you will receive database errors when you check permissions. Custom users and Proxy models One limitation of custom User models is that installing a custom User model will break any proxy model extending User. Proxy models must be based on a concrete base class; by defining a custom User model, you remove the ability of Django to reliably identify the base class. If your project uses proxy models, you must either modify the proxy to extend the User model that is currently in use in your project, or merge your proxy’s behavior into your User subclass. Custom users and signals Another limitation of custom User models is that you can’t use django.contrib.auth.get_user_model() as the sender or target of a signal handler. Instead, you must register the handler with the resulting User model. See Signals for more information on registering an sending signals. Custom users and testing/fixtures If you are writing an application that interacts with the User model, you must take some precautions to ensure that your test suite will run regardless of the User model that is being used by a project. Any test that instantiates an instance of User will fail if the User model has been swapped out. This includes any attempt to create an instance of User with a fixture. To ensure that your test suite will pass in any project configuration, django.contrib.auth.tests.utils defines a @skipIfCustomUser decorator. This decorator will cause a test case to be skipped if any User model other than the default Django user is in use. This decorator can be applied to a single test, or to an entire test class. Depending on your application, tests may also be needed to be added to ensure that the application works with any user model, not just the default User model. To assist with this, Django provides two substitute user models that can be used in test suites: • django.contrib.auth.tests.custom_user.CustomUser, a custom user model that uses an email field as the username, and has a basic admin-compliant permissions setup • django.contrib.auth.tests.custom_user.ExtensionUser, a custom user model that extends django.contrib.auth.models.AbstractUser, adding a date_of_birth field. You can then use the @override_settings decorator to make that test run with the custom User model. For example, here is a skeleton for a test that would test three possible User models – the default, plus the two User models provided by auth app: 322 Chapter 3. Using Django
- 327. Django Documentation, Release1.5.1 from django.contrib.auth.tests.utils import skipIfCustomUser from django.test import TestCase from django.test.utils import override_settings class ApplicationTestCase(TestCase): @skipIfCustomUser def test_normal_user(self): "Run tests for the normal user model" self.assertSomething() @override_settings(AUTH_USER_MODEL=’auth.CustomUser’) def test_custom_user(self): "Run tests for a custom user model with email-based authentication" self.assertSomething() @override_settings(AUTH_USER_MODEL=’auth.ExtensionUser’) def test_extension_user(self): "Run tests for a simple extension of the built-in User." self.assertSomething() A full example Here is an example of an admin-compliant custom user app. This user model uses an email address as the username, and has a required date of birth; it provides no permission checking, beyond a simple admin flag on the user account. This model would be compatible with all the built-in auth forms and views, except for the User creation forms. This example illustrates how most of the components work together, but is not intended to be copied directly into projects for production use. This code would all live in a models.py file for a custom authentication app: from django.db import models from django.contrib.auth.models import ( BaseUserManager, AbstractBaseUser ) class MyUserManager(BaseUserManager): def create_user(self, email, date_of_birth, password=None): """ Creates and saves a User with the given email, date of birth and password. """ if not email: raise ValueError(’Users must have an email address’) user = self.model( email=MyUserManager.normalize_email(email), date_of_birth=date_of_birth, ) user.set_password(password) user.save(using=self._db) return user def create_superuser(self, email, date_of_birth, password): """ 3.9. User authentication in Django 323
- 328. Django Documentation, Release1.5.1 Creates and saves a superuser with the given email, date of birth and password. """ user = self.create_user(email, password=password, date_of_birth=date_of_birth ) user.is_admin = True user.save(using=self._db) return user class MyUser(AbstractBaseUser): email = models.EmailField( verbose_name=’email address’, max_length=255, unique=True, db_index=True, ) date_of_birth = models.DateField() is_active = models.BooleanField(default=True) is_admin = models.BooleanField(default=False) objects = MyUserManager() USERNAME_FIELD = ’email’ REQUIRED_FIELDS = [’date_of_birth’] def get_full_name(self): # The user is identified by their email address return self.email def get_short_name(self): # The user is identified by their email address return self.email def __unicode__(self): return self.email def has_perm(self, perm, obj=None): "Does the user have a specific permission?" # Simplest possible answer: Yes, always return True def has_module_perms(self, app_label): "Does the user have permissions to view the app ‘app_label‘?" # Simplest possible answer: Yes, always return True @property def is_staff(self): "Is the user a member of staff?" # Simplest possible answer: All admins are staff return self.is_admin Then, to register this custom User model with Django’s admin, the following code would be required in the app’s admin.py file: 324 Chapter 3. Using Django
- 329. Django Documentation, Release1.5.1 from django import forms from django.contrib import admin from django.contrib.auth.models import Group from django.contrib.auth.admin import UserAdmin from django.contrib.auth.forms import ReadOnlyPasswordHashField from customauth.models import MyUser class UserCreationForm(forms.ModelForm): """A form for creating new users. Includes all the required fields, plus a repeated password.""" password1 = forms.CharField(label=’Password’, widget=forms.PasswordInput) password2 = forms.CharField(label=’Password confirmation’, widget=forms.PasswordInput) class Meta: model = MyUser fields = (’email’, ’date_of_birth’) def clean_password2(self): # Check that the two password entries match password1 = self.cleaned_data.get("password1") password2 = self.cleaned_data.get("password2") if password1 and password2 and password1 != password2: raise forms.ValidationError("Passwords don’t match") return password2 def save(self, commit=True): # Save the provided password in hashed format user = super(UserCreationForm, self).save(commit=False) user.set_password(self.cleaned_data["password1"]) if commit: user.save() return user class UserChangeForm(forms.ModelForm): """A form for updating users. Includes all the fields on the user, but replaces the password field with admin’s password hash display field. """ password = ReadOnlyPasswordHashField() class Meta: model = MyUser def clean_password(self): # Regardless of what the user provides, return the initial value. # This is done here, rather than on the field, because the # field does not have access to the initial value return self.initial["password"] class MyUserAdmin(UserAdmin): # The forms to add and change user instances form = UserChangeForm add_form = UserCreationForm 3.9. User authentication in Django 325
- 330. Django Documentation, Release1.5.1 # The fields to be used in displaying the User model. # These override the definitions on the base UserAdmin # that reference specific fields on auth.User. list_display = (’email’, ’date_of_birth’, ’is_admin’) list_filter = (’is_admin’,) fieldsets = ( (None, {’fields’: (’email’, ’password’)}), (’Personal info’, {’fields’: (’date_of_birth’,)}), (’Permissions’, {’fields’: (’is_admin’,)}), (’Important dates’, {’fields’: (’last_login’,)}), ) add_fieldsets = ( (None, { ’classes’: (’wide’,), ’fields’: (’email’, ’date_of_birth’, ’password1’, ’password2’)} ), ) search_fields = (’email’,) ordering = (’email’,) filter_horizontal = () # Now register the new UserAdmin... admin.site.register(MyUser, MyUserAdmin) # ... and, since we’re not using Django’s builtin permissions, # unregister the Group model from admin. admin.site.unregister(Group) Django comes with an user authentication system. It handles user accounts, groups, permissions and cookie-based user sessions. This section of the documentation explains how the default implementation works out of the box, as well as how to extend and customize it to suit your project’s needs. 3.9.4 Overview The Django authentication system handles both authentication and authorization. Briefly, authentication verifies a user is who they claim to be, and authorization determines what an authenticated user is allowed to do. Here the term authentication is used to refer to both tasks. The auth system consists of: • Users • Permissions: Binary (yes/no) flags designating whether a user may perform a certain task. • Groups: A generic way of applying labels and permissions to more than one user. • A configurable password hashing system • Forms and view tools for logging in users, or restricting content • A pluggable backend system The authentication system in Django aims to be very generic and doesn’t provide some features commonly found in web authentication systems. Solutions for some of these common problems have been implemented in third-party packages: • Password strength checking • Throttling of login attempts • Authentication against third-parties (OAuth, for example) 326 Chapter 3. Using Django
- 331. Django Documentation, Release1.5.1 3.9.5 Installation Authentication support is bundled as a Django contrib module in django.contrib.auth. By default, the required configuration is already included in the settings.py generated by django-admin.py startproject, these consist of two items listed in your INSTALLED_APPS setting: 1. ’django.contrib.auth’ contains the core of the authentication framework, and its default models. 2. ’django.contrib.contenttypes’ is the Django content type system, which allows permissions to be associated with models you create. and two items in your MIDDLEWARE_CLASSES setting: 1. SessionMiddleware manages sessions across requests. 2. AuthenticationMiddleware associates users with requests using sessions. With these settings in place, running the command manage.py syncdb creates the necessary database tables for auth related models, creates permissions for any models defined in your installed apps, and prompts you to create a superuser account the first time you run it. 3.9.6 Usage Using Django’s default implementation • Working with User objects • Permissions and authorization • Authentication in web requests • Managing users in the admin API reference for the default implementation Customizing Users and authentication Password management in Django 3.10 Django’s cache framework A fundamental trade-off in dynamic Web sites is, well, they’re dynamic. Each time a user requests a page, the Web server makes all sorts of calculations – from database queries to template rendering to business logic – to create the page that your site’s visitor sees. This is a lot more expensive, from a processing-overhead perspective, than your standard read-a-file-off-the-filesystem server arrangement. For most Web applications, this overhead isn’t a big deal. Most Web applications aren’t washingtonpost.com or slashdot.org; they’re simply small- to medium-sized sites with so-so traffic. But for medium- to high-traffic sites, it’s essential to cut as much overhead as possible. That’s where caching comes in. To cache something is to save the result of an expensive calculation so that you don’t have to perform the calculation next time. Here’s some pseudocode explaining how this would work for a dynamically generated Web page: given a URL, try finding that page in the cache if the page is in the cache: return the cached page else: generate the page 3.10. Django’s cache framework 327
- 332. Django Documentation, Release1.5.1 save the generated page in the cache (for next time) return the generated page Django comes with a robust cache system that lets you save dynamic pages so they don’t have to be calculated for each request. For convenience, Django offers different levels of cache granularity: You can cache the output of specific views, you can cache only the pieces that are difficult to produce, or you can cache your entire site. Django also works well with “upstream” caches, such as Squid and browser-based caches. These are the types of caches that you don’t directly control but to which you can provide hints (via HTTP headers) about which parts of your site should be cached, and how. 3.10.1 Setting up the cache The cache system requires a small amount of setup. Namely, you have to tell it where your cached data should live – whether in a database, on the filesystem or directly in memory. This is an important decision that affects your cache’s performance; yes, some cache types are faster than others. Your cache preference goes in the CACHES setting in your settings file. Here’s an explanation of all available values for CACHES. Memcached By far the fastest, most efficient type of cache available to Django, Memcached is an entirely memory-based cache framework originally developed to handle high loads at LiveJournal.com and subsequently open-sourced by Danga Interactive. It is used by sites such as Facebook and Wikipedia to reduce database access and dramatically increase site performance. Memcached runs as a daemon and is allotted a specified amount of RAM. All it does is provide a fast interface for adding, retrieving and deleting arbitrary data in the cache. All data is stored directly in memory, so there’s no overhead of database or filesystem usage. After installing Memcached itself, you’ll need to install a memcached binding. There are several python memcached bindings available; the two most common are python-memcached and pylibmc. To use Memcached with Django: • Set BACKEND to django.core.cache.backends.memcached.MemcachedCache or django.core.cache.backends.memcached.PyLibMCCache (depending on your chosen mem-cached binding) • Set LOCATION to ip:port values, where ip is the IP address of the Memcached daemon and port is the port on which Memcached is running, or to a unix:path value, where path is the path to a Memcached Unix socket file. In this example, Memcached is running on localhost (127.0.0.1) port 11211, using the python-memcached binding: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’, ’LOCATION’: ’127.0.0.1:11211’, } } In this example, Memcached is available through a local Unix socket file /tmp/memcached.sock using the python-memcached binding: 328 Chapter 3. Using Django
- 333. Django Documentation, Release1.5.1 CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’, ’LOCATION’: ’unix:/tmp/memcached.sock’, } } One excellent feature of Memcached is its ability to share cache over multiple servers. This means you can run Memcached daemons on multiple machines, and the program will treat the group of machines as a single cache, without the need to duplicate cache values on each machine. To take advantage of this feature, include all server addresses in LOCATION, either separated by semicolons or as a list. In this example, the cache is shared over Memcached instances running on IP address 172.19.26.240 and 172.19.26.242, both on port 11211: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’, ’LOCATION’: [ ’172.19.26.240:11211’, ’172.19.26.242:11211’, ] } } In the following example, the cache is shared over Memcached instances running on the IP addresses 172.19.26.240 (port 11211), 172.19.26.242 (port 11212), and 172.19.26.244 (port 11213): CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’, ’LOCATION’: [ ’172.19.26.240:11211’, ’172.19.26.242:11212’, ’172.19.26.244:11213’, ] } } A final point about Memcached is that memory-based caching has one disadvantage: Because the cached data is stored in memory, the data will be lost if your server crashes. Clearly, memory isn’t intended for permanent data storage, so don’t rely on memory-based caching as your only data storage. Without a doubt, none of the Django caching backends should be used for permanent storage – they’re all intended to be solutions for caching, not storage – but we point this out here because memory-based caching is particularly temporary. Database caching To use a database table as your cache backend, first create a cache table in your database by running this command: python manage.py createcachetable [cache_table_name] ...where [cache_table_name] is the name of the database table to create. (This name can be whatever you want, as long as it’s a valid table name that’s not already being used in your database.) This command creates a single table in your database that is in the proper format that Django’s database-cache system expects. Once you’ve created that database table, set your BACKEND setting to "django.core.cache.backends.db.DatabaseCache", and LOCATION to tablename – the name of the database table. In this example, the cache table’s name is my_cache_table: 3.10. Django’s cache framework 329
- 334. Django Documentation, Release1.5.1 CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.db.DatabaseCache’, ’LOCATION’: ’my_cache_table’, } } The database caching backend uses the same database as specified in your settings file. You can’t use a different database backend for your cache table. Database caching works best if you’ve got a fast, well-indexed database server. Database caching and multiple databases If you use database caching with multiple databases, you’ll also need to set up routing instructions for your database cache table. For the purposes of routing, the database cache table appears as a model named CacheEntry, in an application named django_cache. This model won’t appear in the models cache, but the model details can be used for routing purposes. For example, the following router would direct all cache read operations to cache_slave, and all write operations to cache_master. The cache table will only be synchronized onto cache_master: class CacheRouter(object): """A router to control all database cache operations""" def db_for_read(self, model, **hints): "All cache read operations go to the slave" if model._meta.app_label in (’django_cache’,): return ’cache_slave’ return None def db_for_write(self, model, **hints): "All cache write operations go to master" if model._meta.app_label in (’django_cache’,): return ’cache_master’ return None def allow_syncdb(self, db, model): "Only synchronize the cache model on master" if model._meta.app_label in (’django_cache’,): return db == ’cache_master’ return None If you don’t specify routing directions for the database cache model, the cache backend will use the default database. Of course, if you don’t use the database cache backend, you don’t need to worry about providing routing instructions for the database cache model. Filesystem caching To store cached items on a filesystem, use "django.core.cache.backends.filebased.FileBasedCache" for BACKEND. For example, to store cached data in /var/tmp/django_cache, use this setting: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.filebased.FileBasedCache’, 330 Chapter 3. Using Django
- 335. Django Documentation, Release1.5.1 ’LOCATION’: ’/var/tmp/django_cache’, } } If you’re on Windows, put the drive letter at the beginning of the path, like this: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.filebased.FileBasedCache’, ’LOCATION’: ’c:/foo/bar’, } } The directory path should be absolute – that is, it should start at the root of your filesystem. It doesn’t matter whether you put a slash at the end of the setting. Make sure the directory pointed-to by this setting exists and is readable and writable by the system user under which yourWeb server runs. Continuing the above example, if your server runs as the user apache, make sure the directory /var/tmp/django_cache exists and is readable and writable by the user apache. Each cache value will be stored as a separate file whose contents are the cache data saved in a serialized (“pickled”) format, using Python’s pickle module. Each file’s name is the cache key, escaped for safe filesystem use. Local-memory caching If you want the speed advantages of in-memory caching but don’t have the capability of running Memcached, con-sider the local-memory cache backend. This cache is multi-process and thread-safe. To use it, set BACKEND to "django.core.cache.backends.locmem.LocMemCache". For example: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.locmem.LocMemCache’, ’LOCATION’: ’unique-snowflake’ } } The cache LOCATION is used to identify individual memory stores. If you only have one locmem cache, you can omit the LOCATION; however, if you have more than one local memory cache, you will need to assign a name to at least one of them in order to keep them separate. Note that each process will have its own private cache instance, which means no cross-process caching is possible. This obviously also means the local memory cache isn’t particularly memory-efficient, so it’s probably not a good choice for production environments. It’s nice for development. Dummy caching (for development) Finally, Django comes with a “dummy” cache that doesn’t actually cache – it just implements the cache interface without doing anything. This is useful if you have a production site that uses heavy-duty caching in various places but a development/test environment where you don’t want to cache and don’t want to have to change your code to special-case the latter. To activate dummy caching, set BACKEND like so: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.dummy.DummyCache’, 3.10. Django’s cache framework 331
- 336. Django Documentation, Release1.5.1 } } Using a custom cache backend While Django includes support for a number of cache backends out-of-the-box, sometimes you might want to use a customized cache backend. To use an external cache backend with Django, use the Python import path as the BACKEND of the CACHES setting, like so: CACHES = { ’default’: { ’BACKEND’: ’path.to.backend’, } } If you’re building your own backend, you can use the standard cache backends as reference implementations. You’ll find the code in the django/core/cache/backends/ directory of the Django source. Note: Without a really compelling reason, such as a host that doesn’t support them, you should stick to the cache backends included with Django. They’ve been well-tested and are easy to use. Cache arguments In addition to the defining the engine and name of the each cache backend, each cache backend can be given additional arguments to control caching behavior. These arguments are provided as additional keys in the CACHES setting. Valid arguments are as follows: • TIMEOUT: The default timeout, in seconds, to use for the cache. This argument defaults to 300 seconds (5 minutes). • OPTIONS: Any options that should be passed to cache backend. The list options understood by each backend vary with each backend. Cache backends that implement their own culling strategy (i.e., the locmem, filesystem and database backends) will honor the following options: – MAX_ENTRIES: the maximum number of entries allowed in the cache before old values are deleted. This argument defaults to 300. – CULL_FREQUENCY: The fraction of entries that are culled when MAX_ENTRIES is reached. The ac-tual ratio is 1/CULL_FREQUENCY, so set CULL_FREQUENCY: to 2 to cull half of the entries when MAX_ENTRIES is reached. A value of 0 for CULL_FREQUENCY means that the entire cache will be dumped when MAX_ENTRIES is reached. This makes culling much faster at the expense of more cache misses. Cache backends backed by a third-party library will pass their options directly to the underlying cache library. As a result, the list of valid options depends on the library in use. • KEY_PREFIX: A string that will be automatically included (prepended by default) to all cache keys used by the Django server. See the cache documentation for more information. • VERSION: The default version number for cache keys generated by the Django server. See the cache documentation for more information. 332 Chapter 3. Using Django
- 337. Django Documentation, Release1.5.1 • KEY_FUNCTION A string containing a dotted path to a function that defines how to compose a prefix, version and key into a final cache key. See the cache documentation for more information. In this example, a filesystem backend is being configured with a timeout of 60 seconds, and a maximum capacity of 1000 items: CACHES = { ’default’: { ’BACKEND’: ’django.core.cache.backends.filebased.FileBasedCache’, ’LOCATION’: ’/var/tmp/django_cache’, ’TIMEOUT’: 60, ’OPTIONS’: { ’MAX_ENTRIES’: 1000 } } } Invalid arguments are silently ignored, as are invalid values of known arguments. 3.10.2 The per-site cache Once the cache is set up, the simplest way to use caching is to cache your entire site. You’ll need to add ’django.middleware.cache.UpdateCacheMiddleware’ and ’django.middleware.cache.FetchFromCacheMiddleware’ to your MIDDLEWARE_CLASSES setting, as in this example: MIDDLEWARE_CLASSES = ( ’django.middleware.cache.UpdateCacheMiddleware’, ’django.middleware.common.CommonMiddleware’, ’django.middleware.cache.FetchFromCacheMiddleware’, ) Note: No, that’s not a typo: the “update” middleware must be first in the list, and the “fetch” middleware must be last. The details are a bit obscure, but see Order of MIDDLEWARE_CLASSES below if you’d like the full story. Then, add the following required settings to your Django settings file: • CACHE_MIDDLEWARE_ALIAS – The cache alias to use for storage. • CACHE_MIDDLEWARE_SECONDS – The number of seconds each page should be cached. • CACHE_MIDDLEWARE_KEY_PREFIX – If the cache is shared across multiple sites using the same Django installation, set this to the name of the site, or some other string that is unique to this Django instance, to prevent key collisions. Use an empty string if you don’t care. The cache middleware caches GET and HEAD responses with status 200, where the request and response head-ers allow. Responses to requests for the same URL with different query parameters are considered to be unique pages and are cached separately. Optionally, if the CACHE_MIDDLEWARE_ANONYMOUS_ONLY set-ting is True, only anonymous requests (i.e., not those made by a logged-in user) will be cached. This is a simple and effective way of disabling caching for any user-specific pages (including Django’s admin inter-face). Note that if you use CACHE_MIDDLEWARE_ANONYMOUS_ONLY, you should make sure you’ve activated AuthenticationMiddleware. The cache middleware expects that a HEAD request is answered with the same response headers as the corresponding GET request; in which case it can return a cached GET response for HEAD request. Additionally, the cache middleware automatically sets a few headers in each HttpResponse: 3.10. Django’s cache framework 333
- 338. Django Documentation, Release1.5.1 • Sets the Last-Modified header to the current date/time when a fresh (uncached) version of the page is requested. • Sets the Expires header to the current date/time plus the defined CACHE_MIDDLEWARE_SECONDS. • Sets the Cache-Control header to give a max age for the page – again, from the CACHE_MIDDLEWARE_SECONDS setting. See Middleware for more on middleware. If a view sets its own cache expiry time (i.e. it has a max-age section in its Cache-Control header) then the page will be cached until the expiry time, rather than CACHE_MIDDLEWARE_SECONDS. Using the decorators in django.views.decorators.cache you can easily set a view’s expiry time (using the cache_control decorator) or disable caching for a view (using the never_cache decorator). See the using other headers section for more on these decorators. If USE_I18N is set to True then the generated cache key will include the name of the active language – see also How Django discovers language preference). This allows you to easily cache multilingual sites without having to create the cache key yourself. Changed in version 1.4. Cache keys also include the active language when USE_L10N is set to True and the current time zone when USE_TZ is set to True. 3.10.3 The per-view cache django.views.decorators.cache.cache_page() A more granular way to use the caching framework is by caching the output of individual views. django.views.decorators.cache defines a cache_page decorator that will automatically cache the view’s response for you. It’s easy to use: from django.views.decorators.cache import cache_page @cache_page(60 * 15) def my_view(request): ... cache_page takes a single argument: the cache timeout, in seconds. In the above example, the result of the my_view() view will be cached for 15 minutes. (Note that we’ve written it as 60 * 15 for the purpose of read-ability. 60 * 15 will be evaluated to 900 – that is, 15 minutes multiplied by 60 seconds per minute.) The per-view cache, like the per-site cache, is keyed off of the URL. If multiple URLs point at the same view, each URL will be cached separately. Continuing the my_view example, if your URLconf looks like this: urlpatterns = (’’, (r’^foo/(d{1,2})/$’, my_view), ) then requests to /foo/1/ and /foo/23/ will be cached separately, as you may expect. But once a particular URL (e.g., /foo/23/) has been requested, subsequent requests to that URL will use the cache. cache_page can also take an optional keyword argument, cache, which directs the decorator to use a specific cache (from your CACHES setting) when caching view results. By default, the default cache will be used, but you can specify any cache you want: @cache_page(60 * 15, cache="special_cache") def my_view(request): ... You can also override the cache prefix on a per-view basis. cache_page takes an optional keyword argument, key_prefix, which works in the same way as the CACHE_MIDDLEWARE_KEY_PREFIX setting for the middle-ware. It can be used like this: 334 Chapter 3. Using Django
- 339. Django Documentation, Release1.5.1 @cache_page(60 * 15, key_prefix="site1") def my_view(request): ... The two settings can also be combined. If you specify a cache and a key_prefix, you will get all the settings of the requested cache alias, but with the key_prefix overridden. Specifying per-view cache in the URLconf The examples in the previous section have hard-coded the fact that the view is cached, because cache_page alters the my_view function in place. This approach couples your view to the cache system, which is not ideal for several reasons. For instance, you might want to reuse the view functions on another, cache-less site, or you might want to distribute the views to people who might want to use them without being cached. The solution to these problems is to specify the per-view cache in the URLconf rather than next to the view functions themselves. Doing so is easy: simply wrap the view function with cache_page when you refer to it in the URLconf. Here’s the old URLconf from earlier: urlpatterns = (’’, (r’^foo/(d{1,2})/$’, my_view), ) Here’s the same thing, with my_view wrapped in cache_page: from django.views.decorators.cache import cache_page urlpatterns = (’’, (r’^foo/(d{1,2})/$’, cache_page(60 * 15)(my_view)), ) 3.10.4 Template fragment caching If you’re after even more control, you can also cache template fragments using the cache template tag. To give your template access to this tag, put {% load cache %} near the top of your template. The {% cache %} template tag caches the contents of the block for a given amount of time. It takes at least two arguments: the cache timeout, in seconds, and the name to give the cache fragment. The name will be taken as is, do not use a variable. For example: {% load cache %} {% cache 500 sidebar %} .. sidebar .. {% endcache %} Sometimes you might want to cache multiple copies of a fragment depending on some dynamic data that appears inside the fragment. For example, you might want a separate cached copy of the sidebar used in the previous example for every user of your site. Do this by passing additional arguments to the {% cache %} template tag to uniquely identify the cache fragment: {% load cache %} {% cache 500 sidebar request.user.username %} .. sidebar for logged in user .. {% endcache %} It’s perfectly fine to specify more than one argument to identify the fragment. Simply pass as many arguments to {% cache %} as you need. 3.10. Django’s cache framework 335
- 340. Django Documentation, Release1.5.1 If USE_I18N is set to True the per-site middleware cache will respect the active language. For the cache template tag you could use one of the translation-specific variables available in templates to achieve the same result: {% load i18n %} {% load cache %} {% get_current_language as LANGUAGE_CODE %} {% cache 600 welcome LANGUAGE_CODE %} {% trans "Welcome to example.com" %} {% endcache %} The cache timeout can be a template variable, as long as the template variable resolves to an integer value. For example, if the template variable my_timeout is set to the value 600, then the following two examples are equivalent: {% cache 600 sidebar %} ... {% endcache %} {% cache my_timeout sidebar %} ... {% endcache %} This feature is useful in avoiding repetition in templates. You can set the timeout in a variable, in one place, and just reuse that value. 3.10.5 The low-level cache API Sometimes, caching an entire rendered page doesn’t gain you very much and is, in fact, inconvenient overkill. Perhaps, for instance, your site includes a view whose results depend on several expensive queries, the results of which change at different intervals. In this case, it would not be ideal to use the full-page caching that the per-site or per-view cache strategies offer, because you wouldn’t want to cache the entire result (since some of the data changes often), but you’d still want to cache the results that rarely change. For cases like this, Django exposes a simple, low-level cache API. You can use this API to store objects in the cache with any level of granularity you like. You can cache any Python object that can be pickled safely: strings, dictionaries, lists of model objects, and so forth. (Most common Python objects can be pickled; refer to the Python documentation for more information about pickling.) Accessing the cache django.core.cache.get_cache(backend, **kwargs) The cache module, django.core.cache, has a cache object that’s automatically created from the ’default’ entry in the CACHES setting: >>> from django.core.cache import cache If you have multiple caches defined in CACHES, then you can use django.core.cache.get_cache() to re-trieve a cache object for any key: >>> from django.core.cache import get_cache >>> cache = get_cache(’alternate’) If the named key does not exist, InvalidCacheBackendError will be raised. Basic usage The basic interface is set(key, value, timeout) and get(key): 336 Chapter 3. Using Django
- 341. Django Documentation, Release1.5.1 >>> cache.set(’my_key’, ’hello, world!’, 30) >>> cache.get(’my_key’) ’hello, world!’ The timeout argument is optional and defaults to the timeout argument of the appropriate backend in the CACHES setting (explained above). It’s the number of seconds the value should be stored in the cache. If the object doesn’t exist in the cache, cache.get() returns None: # Wait 30 seconds for ’my_key’ to expire... >>> cache.get(’my_key’) None We advise against storing the literal value None in the cache, because you won’t be able to distinguish between your stored None value and a cache miss signified by a return value of None. cache.get() can take a default argument. This specifies which value to return if the object doesn’t exist in the cache: >>> cache.get(’my_key’, ’has expired’) ’has expired’ To add a key only if it doesn’t already exist, use the add() method. It takes the same parameters as set(), but it will not attempt to update the cache if the key specified is already present: >>> cache.set(’add_key’, ’Initial value’) >>> cache.add(’add_key’, ’New value’) >>> cache.get(’add_key’) ’Initial value’ If you need to know whether add() stored a value in the cache, you can check the return value. It will return True if the value was stored, False otherwise. There’s also a get_many() interface that only hits the cache once. get_many() returns a dictionary with all the keys you asked for that actually exist in the cache (and haven’t expired): >>> cache.set(’a’, 1) >>> cache.set(’b’, 2) >>> cache.set(’c’, 3) >>> cache.get_many([’a’, ’b’, ’c’]) {’a’: 1, ’b’: 2, ’c’: 3} To set multiple values more efficiently, use set_many() to pass a dictionary of key-value pairs: >>> cache.set_many({’a’: 1, ’b’: 2, ’c’: 3}) >>> cache.get_many([’a’, ’b’, ’c’]) {’a’: 1, ’b’: 2, ’c’: 3} Like cache.set(), set_many() takes an optional timeout parameter. You can delete keys explicitly with delete(). This is an easy way of clearing the cache for a particular object: >>> cache.delete(’a’) If you want to clear a bunch of keys at once, delete_many() can take a list of keys to be cleared: >>> cache.delete_many([’a’, ’b’, ’c’]) Finally, if you want to delete all the keys in the cache, use cache.clear(). Be careful with this; clear() will remove everything from the cache, not just the keys set by your application. 3.10. Django’s cache framework 337
- 342. Django Documentation, Release1.5.1 >>> cache.clear() You can also increment or decrement a key that already exists using the incr() or decr() methods, respectively. By default, the existing cache value will incremented or decremented by 1. Other increment/decrement values can be specified by providing an argument to the increment/decrement call. A ValueError will be raised if you attempt to increment or decrement a nonexistent cache key.: >>> cache.set(’num’, 1) >>> cache.incr(’num’) 2 >>> cache.incr(’num’, 10) 12 >>> cache.decr(’num’) 11 >>> cache.decr(’num’, 5) 6 Note: incr()/decr() methods are not guaranteed to be atomic. On those backends that support atomic in-crement/ decrement (most notably, the memcached backend), increment and decrement operations will be atomic. However, if the backend doesn’t natively provide an increment/decrement operation, it will be implemented using a two-step retrieve/update. Cache key prefixing If you are sharing a cache instance between servers, or between your production and development environments, it’s possible for data cached by one server to be used by another server. If the format of cached data is different between servers, this can lead to some very hard to diagnose problems. To prevent this, Django provides the ability to prefix all cache keys used by a server. When a particular cache key is saved or retrieved, Django will automatically prefix the cache key with the value of the KEY_PREFIX cache setting. By ensuring each Django instance has a different KEY_PREFIX, you can ensure that there will be no collisions in cache values. Cache versioning When you change running code that uses cached values, you may need to purge any existing cached values. The easiest way to do this is to flush the entire cache, but this can lead to the loss of cache values that are still valid and useful. Django provides a better way to target individual cache values. Django’s cache framework has a system-wide version identifier, specified using the VERSION cache setting. The value of this setting is automatically combined with the cache prefix and the user-provided cache key to obtain the final cache key. By default, any key request will automatically include the site default cache key version. However, the primitive cache functions all include a version argument, so you can specify a particular cache key version to set or get. For example: # Set version 2 of a cache key >>> cache.set(’my_key’, ’hello world!’, version=2) # Get the default version (assuming version=1) >>> cache.get(’my_key’) None # Get version 2 of the same key 338 Chapter 3. Using Django
- 343. Django Documentation, Release1.5.1 >>> cache.get(’my_key’, version=2) ’hello world!’ The version of a specific key can be incremented and decremented using the incr_version() and decr_version() methods. This enables specific keys to be bumped to a new version, leaving other keys unaf-fected. Continuing our previous example: # Increment the version of ’my_key’ >>> cache.incr_version(’my_key’) # The default version still isn’t available >>> cache.get(’my_key’) None # Version 2 isn’t available, either >>> cache.get(’my_key’, version=2) None # But version 3 *is* available >>> cache.get(’my_key’, version=3) ’hello world!’ Cache key transformation As described in the previous two sections, the cache key provided by a user is not used verbatim – it is combined with the cache prefix and key version to provide a final cache key. By default, the three parts are joined using colons to produce a final string: def make_key(key, key_prefix, version): return ’:’.join([key_prefix, str(version), key]) If you want to combine the parts in different ways, or apply other processing to the final key (e.g., taking a hash digest of the key parts), you can provide a custom key function. The KEY_FUNCTION cache setting specifies a dotted-path to a function matching the prototype of make_key() above. If provided, this custom key function will be used instead of the default key combining function. Cache key warnings Memcached, the most commonly-used production cache backend, does not allow cache keys longer than 250 char-acters or containing whitespace or control characters, and using such keys will cause an exception. To encour-age cache-portable code and minimize unpleasant surprises, the other built-in cache backends issue a warning (django.core.cache.backends.base.CacheKeyWarning) if a key is used that would cause an error on memcached. If you are using a production backend that can accept a wider range of keys (a custom backend, or one of the non-memcached built-in backends), and want to use this wider range without warnings, you can silence CacheKeyWarning with this code in the management module of one of your INSTALLED_APPS: import warnings from django.core.cache import CacheKeyWarning warnings.simplefilter("ignore", CacheKeyWarning) If you want to instead provide custom key validation logic for one of the built-in backends, you can subclass it, override just the validate_key method, and follow the instructions for using a custom cache backend. For instance, to do this for the locmem backend, put this code in a module: 3.10. Django’s cache framework 339
- 344. Django Documentation, Release1.5.1 from django.core.cache.backends.locmem import LocMemCache class CustomLocMemCache(LocMemCache): def validate_key(self, key): """Custom validation, raising exceptions or warnings as needed.""" # ... ...and use the dotted Python path to this class in the BACKEND portion of your CACHES setting. 3.10.6 Upstream caches So far, this document has focused on caching your own data. But another type of caching is relevant to Web devel-opment, too: caching performed by “upstream” caches. These are systems that cache pages for users even before the request reaches your Web site. Here are a few examples of upstream caches: • Your ISP may cache certain pages, so if you requested a page from http://example.com/, your ISP would send you the page without having to access example.com directly. The maintainers of example.com have no knowl-edge of this caching; the ISP sits between example.com and your Web browser, handling all of the caching transparently. • Your Django Web site may sit behind a proxy cache, such as Squid Web Proxy Cache (http://www.squid-cache. org/), that caches pages for performance. In this case, each request first would be handled by the proxy, and it would be passed to your application only if needed. • Your Web browser caches pages, too. If a Web page sends out the appropriate headers, your browser will use the local cached copy for subsequent requests to that page, without even contacting the Web page again to see whether it has changed. Upstream caching is a nice efficiency boost, but there’s a danger to it: Many Web pages’ contents differ based on authentication and a host of other variables, and cache systems that blindly save pages based purely on URLs could expose incorrect or sensitive data to subsequent visitors to those pages. For example, say you operate a Web email system, and the contents of the “inbox” page obviously depend on which user is logged in. If an ISP blindly cached your site, then the first user who logged in through that ISP would have his user-specific inbox page cached for subsequent visitors to the site. That’s not cool. Fortunately, HTTP provides a solution to this problem. A number of HTTP headers exist to instruct upstream caches to differ their cache contents depending on designated variables, and to tell caching mechanisms not to cache particular pages. We’ll look at some of these headers in the sections that follow. 3.10.7 Using Vary headers The Vary header defines which request headers a cache mechanism should take into account when building its cache key. For example, if the contents of a Web page depend on a user’s language preference, the page is said to “vary on language.” By default, Django’s cache system creates its cache keys using the requested path and query – e.g., "/stories/2005/?order_by=author". This means every request to that URL will use the same cached version, regardless of user-agent differences such as cookies or language preferences. However, if this page produces different content based on some difference in request headers – such as a cookie, or a language, or a user-agent – you’ll need to use the Vary header to tell caching mechanisms that the page output depends on those things. To do this in Django, use the convenient vary_on_headers view decorator, like so: 340 Chapter 3. Using Django
- 345. Django Documentation, Release1.5.1 from django.views.decorators.vary import vary_on_headers @vary_on_headers(’User-Agent’) def my_view(request): # ... In this case, a caching mechanism (such as Django’s own cache middleware) will cache a separate version of the page for each unique user-agent. The advantage to using the vary_on_headers decorator rather than manually setting the Vary header (using something like response[’Vary’] = ’user-agent’) is that the decorator adds to the Vary header (which may already exist), rather than setting it from scratch and potentially overriding anything that was already in there. You can pass multiple headers to vary_on_headers(): @vary_on_headers(’User-Agent’, ’Cookie’) def my_view(request): # ... This tells upstream caches to vary on both, which means each combination of user-agent and cookie will get its own cache value. For example, a request with the user-agent Mozilla and the cookie value foo=bar will be considered different from a request with the user-agent Mozilla and the cookie value foo=ham. Because varying on cookie is so common, there’s a vary_on_cookie decorator. These two views are equivalent: @vary_on_cookie def my_view(request): # ... @vary_on_headers(’Cookie’) def my_view(request): # ... The headers you pass to vary_on_headers are not case sensitive; "User-Agent" is the same thing as "user-agent". You can also use a helper function, django.utils.cache.patch_vary_headers, directly. This function sets, or adds to, the Vary header. For example: from django.utils.cache import patch_vary_headers def my_view(request): # ... response = render_to_response(’template_name’, context) patch_vary_headers(response, [’Cookie’]) return response patch_vary_headers takes an HttpResponse instance as its first argument and a list/tuple of case-insensitive header names as its second argument. For more on Vary headers, see the official Vary spec. 3.10.8 Controlling cache: Using other headers Other problems with caching are the privacy of data and the question of where data should be stored in a cascade of caches. A user usually faces two kinds of caches: his or her own browser cache (a private cache) and his or her provider’s cache (a public cache). A public cache is used by multiple users and controlled by someone else. This poses problems 3.10. Django’s cache framework 341
- 346. Django Documentation, Release1.5.1 with sensitive data–you don’t want, say, your bank account number stored in a public cache. SoWeb applications need a way to tell caches which data is private and which is public. The solution is to indicate a page’s cache should be “private.” To do this in Django, use the cache_control view decorator. Example: from django.views.decorators.cache import cache_control @cache_control(private=True) def my_view(request): # ... This decorator takes care of sending out the appropriate HTTP header behind the scenes. Note that the cache control settings “private” and “public” are mutually exclusive. The decorator ensures that the “public” directive is removed if “private” should be set (and vice versa). An example use of the two directives would be a blog site that offers both private and public entries. Public entries may be cached on any shared cache. The following code uses patch_cache_control, the manual way to modify the cache control header (it is internally called by the cache_control decorator): from django.views.decorators.cache import patch_cache_control from django.views.decorators.vary import vary_on_cookie @vary_on_cookie def list_blog_entries_view(request): if request.user.is_anonymous(): response = render_only_public_entries() patch_cache_control(response, public=True) else: response = render_private_and_public_entries(request.user) patch_cache_control(response, private=True) return response There are a few other ways to control cache parameters. For example, HTTP allows applications to do the following: • Define the maximum time a page should be cached. • Specify whether a cache should always check for newer versions, only delivering the cached content when there are no changes. (Some caches might deliver cached content even if the server page changed, simply because the cache copy isn’t yet expired.) In Django, use the cache_control view decorator to specify these cache parameters. In this example, cache_control tells caches to revalidate the cache on every access and to store cached versions for, at most, 3,600 seconds: from django.views.decorators.cache import cache_control @cache_control(must_revalidate=True, max_age=3600) def my_view(request): # ... Any valid Cache-Control HTTP directive is valid in cache_control(). Here’s a full list: • public=True • private=True • no_cache=True • no_transform=True • must_revalidate=True 342 Chapter 3. Using Django
- 347. Django Documentation, Release1.5.1 • proxy_revalidate=True • max_age=num_seconds • s_maxage=num_seconds For explanation of Cache-Control HTTP directives, see the Cache-Control spec. (Note that the caching middleware already sets the cache header’s max-age with the value of the CACHE_MIDDLEWARE_SECONDS setting. If you use a custom max_age in a cache_control decorator, the decorator will take precedence, and the header values will be merged correctly.) If you want to use headers to disable caching altogether, django.views.decorators.cache.never_cache is a view decorator that adds headers to ensure the response won’t be cached by browsers or other caches. Example: from django.views.decorators.cache import never_cache @never_cache def myview(request): # ... 3.10.9 Other optimizations Django comes with a few other pieces of middleware that can help optimize your site’s performance: • django.middleware.http.ConditionalGetMiddleware adds support for modern browsers to conditionally GET responses based on the ETag and Last-Modified headers. • django.middleware.gzip.GZipMiddleware compresses responses for all modern browsers, saving bandwidth and transfer time. 3.10.10 Order of MIDDLEWARE_CLASSES If you use caching middleware, it’s important to put each half in the right place within the MIDDLEWARE_CLASSES setting. That’s because the cache middleware needs to know which headers by which to vary the cache storage. Middleware always adds something to the Vary response header when it can. UpdateCacheMiddleware runs during the response phase, where middleware is run in reverse order, so an item at the top of the list runs last during the response phase. Thus, you need to make sure that UpdateCacheMiddleware appears before any other middleware that might add something to the Vary header. The following middleware mod-ules do so: • SessionMiddleware adds Cookie • GZipMiddleware adds Accept-Encoding • LocaleMiddleware adds Accept-Language FetchFromCacheMiddleware, on the other hand, runs during the request phase, where middleware is applied first-to-last, so an item at the top of the list runs first during the request phase. The FetchFromCacheMiddleware also needs to run after other middleware updates the Vary header, so FetchFromCacheMiddleware must be after any item that does so. 3.11 Conditional View Processing HTTP clients can send a number of headers to tell the server about copies of a resource that they have already seen. This is commonly used when retrieving a Web page (using an HTTP GET request) to avoid sending all the data for 3.11. Conditional View Processing 343
- 348. Django Documentation, Release1.5.1 something the client has already retrieved. However, the same headers can be used for all HTTP methods (POST, PUT, DELETE, etc). For each page (response) that Django sends back from a view, it might provide two HTTP headers: the ETag header and the Last-Modified header. These headers are optional on HTTP responses. They can be set by your view function, or you can rely on the CommonMiddleware middleware to set the ETag header. When the client next requests the same resource, it might send along a header such as If-modified-since, containing the date of the last modification time it was sent, or If-none-match, containing the ETag it was sent. If the current version of the page matches the ETag sent by the client, or if the resource has not been modified, a 304 status code can be sent back, instead of a full response, telling the client that nothing has changed. When you need more fine-grained control you may use per-view conditional processing functions. 3.11.1 The condition decorator Sometimes (in fact, quite often) you can create functions to rapidly compute the ETag value or the last-modified time for a resource, without needing to do all the computations needed to construct the full view. Django can then use these functions to provide an “early bailout” option for the view processing. Telling the client that the content has not been modified since the last request, perhaps. These two functions are passed as parameters the django.views.decorators.http.condition decorator. This decorator uses the two functions (you only need to supply one, if you can’t compute both quantities easily and quickly) to work out if the headers in the HTTP request match those on the resource. If they don’t match, a new copy of the resource must be computed and your normal view is called. The condition decorator’s signature looks like this: condition(etag_func=None, last_modified_func=None) The two functions, to compute the ETag and the last modified time, will be passed the incoming request object and the same parameters, in the same order, as the view function they are helping to wrap. The function passed last_modified_func should return a standard datetime value specifying the last time the resource was modified, or None if the resource doesn’t exist. The function passed to the etag decorator should return a string representing the Etag for the resource, or None if it doesn’t exist. Using this feature usefully is probably best explained with an example. Suppose you have this pair of models, repre-senting a simple blog system: import datetime from django.db import models class Blog(models.Model): ... class Entry(models.Model): blog = models.ForeignKey(Blog) published = models.DateTimeField(default=datetime.datetime.now) ... If the front page, displaying the latest blog entries, only changes when you add a new blog entry, you can compute the last modified time very quickly. You need the latest published date for every entry associated with that blog. One way to do this would be: def latest_entry(request, blog_id): return Entry.objects.filter(blog=blog_id).latest("published").published You can then use this function to provide early detection of an unchanged page for your front page view: 344 Chapter 3. Using Django
- 349. Django Documentation, Release1.5.1 from django.views.decorators.http import condition @condition(last_modified_func=latest_entry) def front_page(request, blog_id): ... 3.11.2 Shortcuts for only computing one value As a general rule, if you can provide functions to compute both the ETag and the last modified time, you should do so. You don’t know which headers any given HTTP client will send you, so be prepared to handle both. However, sometimes only one value is easy to compute and Django provides decorators that handle only ETag or only last-modified computations. The django.views.decorators.http.etag and django.views.decorators.http.last_modified decorators are passed the same type of functions as the condition decorator. Their signatures are: etag(etag_func) last_modified(last_modified_func) We could write the earlier example, which only uses a last-modified function, using one of these decorators: @last_modified(latest_entry) def front_page(request, blog_id): ... ...or: def front_page(request, blog_id): ... front_page = last_modified(latest_entry)(front_page) Use condition when testing both conditions It might look nicer to some people to try and chain the etag and last_modified decorators if you want to test both preconditions. However, this would lead to incorrect behavior. # Bad code. Don’t do this! @etag(etag_func) @last_modified(last_modified_func) def my_view(request): # ... # End of bad code. The first decorator doesn’t know anything about the second and might answer that the response is not modified even if the second decorators would determine otherwise. The condition decorator uses both callback functions simulta-neously to work out the right action to take. 3.11.3 Using the decorators with other HTTP methods The condition decorator is useful for more than only GET and HEAD requests (HEAD requests are the same as GET in this situation). It can be used also to be used to provide checking for POST, PUT and DELETE requests. In these situations, the idea isn’t to return a “not modified” response, but to tell the client that the resource they are trying to change has been altered in the meantime. For example, consider the following exchange between the client and server: 3.11. Conditional View Processing 345
- 350. Django Documentation, Release1.5.1 1. Client requests /foo/. 2. Server responds with some content with an ETag of "abcd1234". 3. Client sends an HTTP PUT request to /foo/ to update the resource. It also sends an If-Match: "abcd1234" header to specify the version it is trying to update. 4. Server checks to see if the resource has changed, by computing the ETag the same way it does for a GET request (using the same function). If the resource has changed, it will return a 412 status code code, meaning “precondition failed”. 5. Client sends a GET request to /foo/, after receiving a 412 response, to retrieve an updated version of the content before updating it. The important thing this example shows is that the same functions can be used to compute the ETag and last modifi-cation values in all situations. In fact, you should use the same functions, so that the same values are returned every time. 3.11.4 Comparison with middleware conditional processing You may notice that Django already provides simple and straightforward conditional GET handling via the django.middleware.http.ConditionalGetMiddleware and CommonMiddleware. Whilst certainly being easy to use and suitable for many situations, those pieces of middleware functionality have limitations for ad-vanced usage: • They are applied globally to all views in your project • They don’t save you from generating the response itself, which may be expensive • They are only appropriate for HTTP GET requests. You should choose the most appropriate tool for your particular problem here. If you have a way to compute ETags and modification times quickly and if some view takes a while to generate the content, you should consider using the condition decorator described in this document. If everything already runs fairly quickly, stick to using the middleware and the amount of network traffic sent back to the clients will still be reduced if the view hasn’t changed. 3.12 Cryptographic signing New in version 1.4. The golden rule of Web application security is to never trust data from untrusted sources. Some-times it can be useful to pass data through an untrusted medium. Cryptographically signed values can be passed through an untrusted channel safe in the knowledge that any tampering will be detected. Django provides both a low-level API for signing values and a high-level API for setting and reading signed cookies, one of the most common uses of signing in Web applications. You may also find signing useful for the following: • Generating “recover my account” URLs for sending to users who have lost their password. • Ensuring data stored in hidden form fields has not been tampered with. • Generating one-time secret URLs for allowing temporary access to a protected resource, for example a down-loadable file that a user has paid for. 346 Chapter 3. Using Django
- 351. Django Documentation, Release1.5.1 3.12.1 Protecting the SECRET_KEY When you create a new Django project using startproject, the settings.py file is generated automatically and gets a random SECRET_KEY value. This value is the key to securing signed data – it is vital you keep this secure, or attackers could use it to generate their own signed values. 3.12.2 Using the low-level API class Signer Django’s signing methods live in the django.core.signing module. To sign a value, first instantiate a Signer instance: >>> from django.core.signing import Signer >>> signer = Signer() >>> value = signer.sign(’My string’) >>> value ’My string:GdMGD6HNQ_qdgxYP8yBZAdAIV1w’ The signature is appended to the end of the string, following the colon. You can retrieve the original value using the unsign method: >>> original = signer.unsign(value) >>> original u’My string’ If the signature or value have been altered in any way, a django.core.signing.BadSignature exception will be raised: >>> from django.core import signing >>> value += ’m’ >>> try: ... original = signer.unsign(value) ... except signing.BadSignature: ... print("Tampering detected!") By default, the Signer class uses the SECRET_KEY setting to generate signatures. You can use a different secret by passing it to the Signer constructor: >>> signer = Signer(’my-other-secret’) >>> value = signer.sign(’My string’) >>> value ’My string:EkfQJafvGyiofrdGnuthdxImIJw’ Using the salt argument If you do not wish for every occurrence of a particular string to have the same signature hash, you can use the optional salt argument to the Signer class. Using a salt will seed the signing hash function with both the salt and your SECRET_KEY: >>> signer = Signer() >>> signer.sign(’My string’) ’My string:GdMGD6HNQ_qdgxYP8yBZAdAIV1w’ >>> signer = Signer(salt=’extra’) >>> signer.sign(’My string’) ’My string:Ee7vGi-ING6n02gkcJ-QLHg6vFw’ 3.12. Cryptographic signing 347
- 352. Django Documentation, Release1.5.1 >>> signer.unsign(’My string:Ee7vGi-ING6n02gkcJ-QLHg6vFw’) u’My string’ Using salt in this way puts the different signatures into different namespaces. A signature that comes from one namespace (a particular salt value) cannot be used to validate the same plaintext string in a different namespace that is using a different salt setting. The result is to prevent an attacker from using a signed string generated in one place in the code as input to another piece of code that is generating (and verifying) signatures using a different salt. Unlike your SECRET_KEY, your salt argument does not need to stay secret. Verifying timestamped values class TimestampSigner TimestampSigner is a subclass of Signer that appends a signed timestamp to the value. This allows you to confirm that a signed value was created within a specified period of time: >>> from django.core.signing import TimestampSigner >>> signer = TimestampSigner() >>> value = signer.sign(’hello’) >>> value ’hello:1NMg5H:oPVuCqlJWmChm1rA2lyTUtelC-c’ >>> signer.unsign(value) u’hello’ >>> signer.unsign(value, max_age=10) ... SignatureExpired: Signature age 15.5289158821 > 10 seconds >>> signer.unsign(value, max_age=20) u’hello’ Protecting complex data structures If you wish to protect a list, tuple or dictionary you can do so using the signing module’s dumps and loads functions. These imitate Python’s pickle module, but use JSON serialization under the hood. JSON ensures that even if your SECRET_KEY is stolen an attacker will not be able to execute arbitrary commands by exploiting the pickle format.: >>> from django.core import signing >>> value = signing.dumps({"foo": "bar"}) >>> value ’eyJmb28iOiJiYXIifQ:1NMg1b:zGcDE4-TCkaeGzLeW9UQwZesciI’ >>> signing.loads(value) {’foo’: ’bar’} dumps(obj, key=None, salt=’django.core.signing’, compress=False) Returns URL-safe, sha1 signed base64 compressed JSON string. loads(string, key=None, salt=’django.core.signing’, max_age=None) Reverse of dumps(), raises BadSignature if signature fails. 3.13 Sending email Although Python makes sending email relatively easy via the smtplib module, Django provides a couple of light wrappers over it. These wrappers are provided to make sending email extra quick, to make it easy to test email sending during development, and to provide support for platforms that can’t use SMTP. 348 Chapter 3. Using Django
- 353. Django Documentation, Release1.5.1 The code lives in the django.core.mail module. 3.13.1 Quick example In two lines: from django.core.mail import send_mail send_mail(’Subject here’, ’Here is the message.’, ’from@example.com’, [’to@example.com’], fail_silently=False) Mail is sent using the SMTP host and port specified in the EMAIL_HOST and EMAIL_PORT settings. The EMAIL_HOST_USER and EMAIL_HOST_PASSWORD settings, if set, are used to authenticate to the SMTP server, and the EMAIL_USE_TLS setting controls whether a secure connection is used. Note: The character set of email sent with django.core.mail will be set to the value of your DEFAULT_CHARSET setting. 3.13.2 send_mail() send_mail(subject, message, from_email, recipient_list, fail_silently=False, auth_user=None, auth_password=None, connection=None) The simplest way to send email is using django.core.mail.send_mail(). The subject, message, from_email and recipient_list parameters are required. • subject: A string. • message: A string. • from_email: A string. • recipient_list: A list of strings, each an email address. Each member of recipient_list will see the other recipients in the “To:” field of the email message. • fail_silently: A boolean. If it’s False, send_mail will raise an smtplib.SMTPException. See the smtplib docs for a list of possible exceptions, all of which are subclasses of SMTPException. • auth_user: The optional username to use to authenticate to the SMTP server. If this isn’t provided, Django will use the value of the EMAIL_HOST_USER setting. • auth_password: The optional password to use to authenticate to the SMTP server. If this isn’t provided, Django will use the value of the EMAIL_HOST_PASSWORD setting. • connection: The optional email backend to use to send the mail. If unspecified, an instance of the default backend will be used. See the documentation on Email backends for more details. 3.13.3 send_mass_mail() send_mass_mail(datatuple, fail_silently=False, auth_user=None, auth_password=None, connec-tion= None) django.core.mail.send_mass_mail() is intended to handle mass emailing. datatuple is a tuple in which each element is in this format: 3.13. Sending email 349
- 354. Django Documentation, Release1.5.1 (subject, message, from_email, recipient_list) fail_silently, auth_user and auth_password have the same functions as in send_mail(). Each separate element of datatuple results in a separate email message. As in send_mail(), recipients in the same recipient_list will all see the other addresses in the email messages’ “To:” field. For example, the following code would send two different messages to two different sets of recipients; however, only one connection to the mail server would be opened: message1 = (’Subject here’, ’Here is the message’, ’from@example.com’, [’first@example.com’, ’other@example.message2 = (’Another Subject’, ’Here is another message’, ’from@example.com’, [’second@test.com’]) send_mass_mail((message1, message2), fail_silently=False) send_mass_mail() vs. send_mail() The main difference between send_mass_mail() and send_mail() is that send_mail() opens a connec-tion to the mail server each time it’s executed, while send_mass_mail() uses a single connection for all of its messages. This makes send_mass_mail() slightly more efficient. 3.13.4 mail_admins() mail_admins(subject, message, fail_silently=False, connection=None, html_message=None) django.core.mail.mail_admins() is a shortcut for sending an email to the site admins, as defined in the ADMINS setting. mail_admins() prefixes the subject with the value of the EMAIL_SUBJECT_PREFIX setting, which is "[Django] " by default. The “From:” header of the email will be the value of the SERVER_EMAIL setting. This method exists for convenience and readability. If html_message is provided, the resulting email will be a multipart/alternative email with message as the text/plain content type and html_message as the text/html content type. 3.13.5 mail_managers() mail_managers(subject, message, fail_silently=False, connection=None, html_message=None) django.core.mail.mail_managers() is just like mail_admins(), except it sends an email to the site managers, as defined in the MANAGERS setting. 3.13.6 Examples This sends a single email to john@example.com and jane@example.com, with them both appearing in the “To:”: send_mail(’Subject’, ’Message.’, ’from@example.com’, [’john@example.com’, ’jane@example.com’]) This sends a message to john@example.com and jane@example.com, with them both receiving a separate email: 350 Chapter 3. Using Django
- 355. Django Documentation, Release1.5.1 datatuple = ( (’Subject’, ’Message.’, ’from@example.com’, [’john@example.com’]), (’Subject’, ’Message.’, ’from@example.com’, [’jane@example.com’]), ) send_mass_mail(datatuple) 3.13.7 Preventing header injection Header injection is a security exploit in which an attacker inserts extra email headers to control the “To:” and “From:” in email messages that your scripts generate. The Django email functions outlined above all protect against header injection by forbidding newlines in header values. If any subject, from_email or recipient_list contains a newline (in either Unix,Windows or Mac style), the email function (e.g. send_mail()) will raise django.core.mail.BadHeaderError (a subclass of ValueError) and, hence, will not send the email. It’s your responsibility to validate all data before passing it to the email functions. If a message contains headers at the start of the string, the headers will simply be printed as the first bit of the email message. Here’s an example view that takes a subject, message and from_email from the request’s POST data, sends that to admin@example.com and redirects to “/contact/thanks/” when it’s done: from django.core.mail import send_mail, BadHeaderError def send_email(request): subject = request.POST.get(’subject’, ’’) message = request.POST.get(’message’, ’’) from_email = request.POST.get(’from_email’, ’’) if subject and message and from_email: try: send_mail(subject, message, from_email, [’admin@example.com’]) except BadHeaderError: return HttpResponse(’Invalid header found.’) return HttpResponseRedirect(’/contact/thanks/’) else: # In reality we’d use a form class # to get proper validation errors. return HttpResponse(’Make sure all fields are entered and valid.’) 3.13.8 The EmailMessage class Django’s send_mail() and send_mass_mail() functions are actually thin wrappers that make use of the EmailMessage class. Not all features of the EmailMessage class are available through the send_mail() and related wrapper functions. If you wish to use advanced features, such as BCC’ed recipients, file attachments, or multi-part email, you’ll need to create EmailMessage instances directly. Note: This is a design feature. send_mail() and related functions were originally the only interface Django provided. However, the list of parameters they accepted was slowly growing over time. It made sense to move to a more object-oriented design for email messages and retain the original functions only for backwards compatibility. EmailMessage is responsible for creating the email message itself. The email backend is then responsible for sending the email. 3.13. Sending email 351
- 356. Django Documentation, Release1.5.1 For convenience, EmailMessage provides a simple send() method for sending a single email. If you need to send multiple messages, the email backend API provides an alternative. EmailMessage Objects class EmailMessage The EmailMessage class is initialized with the following parameters (in the given order, if positional arguments are used). All parameters are optional and can be set at any time prior to calling the send() method. • subject: The subject line of the email. • body: The body text. This should be a plain text message. • from_email: The sender’s address. Both fred@example.com and Fred <fred@example.com> forms are legal. If omitted, the DEFAULT_FROM_EMAIL setting is used. • to: A list or tuple of recipient addresses. • bcc: A list or tuple of addresses used in the “Bcc” header when sending the email. • connection: An email backend instance. Use this parameter if you want to use the same connection for multiple messages. If omitted, a new connection is created when send() is called. • attachments: A list of attachments to put on the message. These can be either email.MIMEBase.MIMEBase instances, or (filename, content, mimetype) triples. • headers: A dictionary of extra headers to put on the message. The keys are the header name, values are the header values. It’s up to the caller to ensure header names and values are in the correct format for an email message. • cc: A list or tuple of recipient addresses used in the “Cc” header when sending the email. For example: email = EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’, [’to1@example.com’, ’to2@example.com’], [’bcc@example.com’], headers = {’Reply-To’: ’another@example.com’}) The class has the following methods: • send(fail_silently=False) sends the message. If a connection was specified when the email was constructed, that connection will be used. Otherwise, an instance of the default backend will be instantiated and used. If the keyword argument fail_silently is True, exceptions raised while sending the message will be quashed. • message() constructs a django.core.mail.SafeMIMEText object (a subclass of Python’s email.MIMEText.MIMEText class) or a django.core.mail.SafeMIMEMultipart object hold-ing the message to be sent. If you ever need to extend the EmailMessage class, you’ll probably want to override this method to put the content you want into the MIME object. • recipients() returns a list of all the recipients of the message, whether they’re recorded in the to, cc or bcc attributes. This is another method you might need to override when subclassing, because the SMTP server needs to be told the full list of recipients when the message is sent. If you add another way to specify recipients in your class, they need to be returned from this method as well. • attach() creates a new file attachment and adds it to the message. There are two ways to call attach(): – You can pass it a single argument that is an email.MIMEBase.MIMEBase instance. This will be inserted directly into the resulting message. 352 Chapter 3. Using Django
- 357. Django Documentation, Release1.5.1 – Alternatively, you can pass attach() three arguments: filename, content and mimetype. filename is the name of the file attachment as it will appear in the email, content is the data that will be contained inside the attachment and mimetype is the optional MIME type for the attachment. If you omit mimetype, the MIME content type will be guessed from the filename of the attachment. For example: message.attach(’design.png’, img_data, ’image/png’) • attach_file() creates a new attachment using a file from your filesystem. Call it with the path of the file to attach and, optionally, the MIME type to use for the attachment. If the MIME type is omitted, it will be guessed from the filename. The simplest use would be: message.attach_file(’/images/weather_map.png’) Sending alternative content types It can be useful to include multiple versions of the content in an email; the classic example is to send both text and HTML versions of a message. With Django’s email library, you can do this using the EmailMultiAlternatives class. This subclass of EmailMessage has an attach_alternative() method for including extra versions of the message body in the email. All the other methods (including the class initialization) are inherited directly from EmailMessage. To send a text and HTML combination, you could write: from django.core.mail import EmailMultiAlternatives subject, from_email, to = ’hello’, ’from@example.com’, ’to@example.com’ text_content = ’This is an important message.’ html_content = ’<p>This is an <strong>important</strong> message.</p>’ msg = EmailMultiAlternatives(subject, text_content, from_email, [to]) msg.attach_alternative(html_content, "text/html") msg.send() By default, the MIME type of the body parameter in an EmailMessage is "text/plain". It is good practice to leave this alone, because it guarantees that any recipient will be able to read the email, regardless of their mail client. However, if you are confident that your recipients can handle an alternative content type, you can use the content_subtype attribute on the EmailMessage class to change the main content type. The major type will always be "text", but you can change the subtype. For example: msg = EmailMessage(subject, html_content, from_email, [to]) msg.content_subtype = "html" # Main content is now text/html msg.send() 3.13.9 Email backends The actual sending of an email is handled by the email backend. The email backend class has the following methods: • open() instantiates an long-lived email-sending connection. • close() closes the current email-sending connection. • send_messages(email_messages) sends a list of EmailMessage objects. If the connection is not open, this call will implicitly open the connection, and close the connection afterwards. If the connection is already open, it will be left open after mail has been sent. 3.13. Sending email 353
- 358. Django Documentation, Release1.5.1 Obtaining an instance of an email backend The get_connection() function in django.core.mail returns an instance of the email backend that you can use. get_connection(backend=None, fail_silently=False, *args, **kwargs) By default, a call to get_connection() will return an instance of the email backend specified in EMAIL_BACKEND. If you specify the backend argument, an instance of that backend will be instantiated. The fail_silently argument controls how the backend should handle errors. If fail_silently is True, exceptions during the email sending process will be silently ignored. All other arguments are passed directly to the constructor of the email backend. Django ships with several email sending backends. With the exception of the SMTP backend (which is the default), these backends are only useful during testing and development. If you have special email sending requirements, you can write your own email backend. SMTP backend This is the default backend. Email will be sent through a SMTP server. The server address and authentication credentials are set in the EMAIL_HOST, EMAIL_PORT, EMAIL_HOST_USER, EMAIL_HOST_PASSWORD and EMAIL_USE_TLS settings in your settings file. The SMTP backend is the default configuration inherited by Django. If you want to specify it explicitly, put the following in your settings: EMAIL_BACKEND = ’django.core.mail.backends.smtp.EmailBackend’ Console backend Instead of sending out real emails the console backend just writes the emails that would be send to the standard output. By default, the console backend writes to stdout. You can use a different stream-like object by providing the stream keyword argument when constructing the connection. To specify this backend, put the following in your settings: EMAIL_BACKEND = ’django.core.mail.backends.console.EmailBackend’ This backend is not intended for use in production – it is provided as a convenience that can be used during develop-ment. File backend The file backend writes emails to a file. A new file is created for each new session that is opened on this backend. The directory to which the files are written is either taken from the EMAIL_FILE_PATH setting or from the file_path keyword when creating a connection with get_connection(). To specify this backend, put the following in your settings: EMAIL_BACKEND = ’django.core.mail.backends.filebased.EmailBackend’ EMAIL_FILE_PATH = ’/tmp/app-messages’ # change this to a proper location This backend is not intended for use in production – it is provided as a convenience that can be used during develop-ment. 354 Chapter 3. Using Django
- 359. Django Documentation, Release1.5.1 In-memory backend The ’locmem’ backend stores messages in a special attribute of the django.core.mail module. The outbox attribute is created when the first message is sent. It’s a list with an EmailMessage instance for each message that would be send. To specify this backend, put the following in your settings: EMAIL_BACKEND = ’django.core.mail.backends.locmem.EmailBackend’ This backend is not intended for use in production – it is provided as a convenience that can be used during development and testing. Dummy backend As the name suggests the dummy backend does nothing with your messages. To specify this backend, put the following in your settings: EMAIL_BACKEND = ’django.core.mail.backends.dummy.EmailBackend’ This backend is not intended for use in production – it is provided as a convenience that can be used during develop-ment. Defining a custom email backend If you need to change how emails are sent you can write your own email backend. The EMAIL_BACKEND setting in your settings file is then the Python import path for your backend class. Custom email backends should subclass BaseEmailBackend that is located in the django.core.mail.backends.base module. A custom email backend must implement the send_messages(email_messages) method. This method receives a list of EmailMessage instances and returns the number of successfully delivered messages. If your backend has any concept of a persistent session or connection, you should also implement the open() and close() methods. Refer to smtp.EmailBackend for a reference implementation. Sending multiple emails Establishing and closing an SMTP connection (or any other network connection, for that matter) is an expensive process. If you have a lot of emails to send, it makes sense to reuse an SMTP connection, rather than creating and destroying a connection every time you want to send an email. There are two ways you tell an email backend to reuse a connection. Firstly, you can use the send_messages() method. send_messages() takes a list of EmailMessage in-stances (or subclasses), and sends them all using a single connection. For example, if you have a function called get_notification_email() that returns a list of EmailMessage objects representing some periodic email you wish to send out, you could send these emails using a single call to send_messages: from django.core import mail connection = mail.get_connection() # Use default email connection messages = get_notification_email() connection.send_messages(messages) 3.13. Sending email 355
- 360. Django Documentation, Release1.5.1 In this example, the call to send_messages() opens a connection on the backend, sends the list of messages, and then closes the connection again. The second approach is to use the open() and close() methods on the email backend to manually control the connection. send_messages() will not manually open or close the connection if it is already open, so if you manually open the connection, you can control when it is closed. For example: from django.core import mail connection = mail.get_connection() # Manually open the connection connection.open() # Construct an email message that uses the connection email1 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’, [’to1@example.com’], connection=connection) email1.send() # Send the email # Construct two more messages email2 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’, [’to2@example.com’]) email3 = mail.EmailMessage(’Hello’, ’Body goes here’, ’from@example.com’, [’to3@example.com’]) # Send the two emails in a single call - connection.send_messages([email2, email3]) # The connection was already open so send_messages() doesn’t close it. # We need to manually close the connection. connection.close() 3.13.10 Testing email sending There are times when you do not want Django to send emails at all. For example, while developing a Web site, you probably don’t want to send out thousands of emails – but you may want to validate that emails will be sent to the right people under the right conditions, and that those emails will contain the correct content. The easiest way to test your project’s use of email is to use the console email backend. This backend redirects all email to stdout, allowing you to inspect the content of mail. The file email backend can also be useful during development – this backend dumps the contents of every SMTP connection to a file that can be inspected at your leisure. Another approach is to use a “dumb” SMTP server that receives the emails locally and displays them to the terminal, but does not actually send anything. Python has a built-in way to accomplish this with a single command: python -m smtpd -n -c DebuggingServer localhost:1025 This command will start a simple SMTP server listening on port 1025 of localhost. This server simply prints to standard output all email headers and the email body. You then only need to set the EMAIL_HOST and EMAIL_PORT accordingly, and you are set. For a more detailed discussion of testing and processing of emails locally, see the Python documentation for the smtpd module. 356 Chapter 3. Using Django
- 361. Django Documentation, Release1.5.1 3.14 Internationalization and localization 3.14.1 Translation Overview In order to make a Django project translatable, you have to add a minimal amount of hooks to your Python code and templates. These hooks are called translation strings. They tell Django: “This text should be translated into the end user’s language, if a translation for this text is available in that language.” It’s your responsibility to mark translatable strings; the system can only translate strings it knows about. Django then provides utilities to extract the translation strings into a message file. This file is a convenient way for translators to provide the equivalent of the translation strings in the target language. Once the translators have filled in the message file, it must be compiled. This process relies on the GNU gettext toolset. Once this is done, Django takes care of translating Web apps on the fly in each available language, according to users’ language preferences. Django’s internationalization hooks are on by default, and that means there’s a bit of i18n-related overhead in certain places of the framework. If you don’t use internationalization, you should take the two seconds to set USE_I18N = False in your settings file. Then Django will make some optimizations so as not to load the internationalization machinery. You’ll probably also want to remove ’django.core.context_processors.i18n’ from your TEMPLATE_CONTEXT_PROCESSORS setting. Note: There is also an independent but related USE_L10N setting that controls if Django should implement format localization. See Format localization for more details. Note: Make sure you’ve activated translation for your project (the fastest way is to check if MIDDLEWARE_CLASSES includes django.middleware.locale.LocaleMiddleware). If you haven’t yet, see How Django discovers language preference. Internationalization: in Python code Standard translation Specify a translation string by using the function ugettext(). It’s convention to import this as a shorter alias, _, to save typing. Note: Python’s standard library gettext module installs _() into the global namespace, as an alias for gettext(). In Django, we have chosen not to follow this practice, for a couple of reasons: 1. For international character set (Unicode) support, ugettext() is more useful than gettext(). Sometimes, you should be using ugettext_lazy() as the default translation method for a particular file. Without _() in the global namespace, the developer has to think about which is the most appropriate translation function. 2. The underscore character (_) is used to represent “the previous result” in Python’s interactive shell and doctest tests. Installing a global _() function causes interference. Explicitly importing ugettext() as _() avoids this problem. In this example, the text "Welcome to my site." is marked as a translation string: 3.14. Internationalization and localization 357
- 362. Django Documentation, Release1.5.1 from django.utils.translation import ugettext as _ def my_view(request): output = _("Welcome to my site.") return HttpResponse(output) Obviously, you could code this without using the alias. This example is identical to the previous one: from django.utils.translation import ugettext def my_view(request): output = ugettext("Welcome to my site.") return HttpResponse(output) Translation works on computed values. This example is identical to the previous two: def my_view(request): words = [’Welcome’, ’to’, ’my’, ’site.’] output = _(’ ’.join(words)) return HttpResponse(output) Translation works on variables. Again, here’s an identical example: def my_view(request): sentence = ’Welcome to my site.’ output = _(sentence) return HttpResponse(output) (The caveat with using variables or computed values, as in the previous two examples, is that Django’s translation-string- detecting utility, django-admin.py makemessages, won’t be able to find these strings. More on makemessages later.) The strings you pass to _() or ugettext() can take placeholders, specified with Python’s standard named-string interpolation syntax. Example: def my_view(request, m, d): output = _(’Today is %(month)s %(day)s.’) % {’month’: m, ’day’: d} return HttpResponse(output) This technique lets language-specific translations reorder the placeholder text. For example, an English translation may be "Today is November 26.", while a Spanish translation may be "Hoy es 26 de Noviembre." – with the the month and the day placeholders swapped. For this reason, you should use named-string interpolation (e.g., %(day)s) instead of positional interpolation (e.g., %s or %d) whenever you have more than a single parameter. If you used positional interpolation, translations wouldn’t be able to reorder placeholder text. Comments for translators If you would like to give translators hints about a translatable string, you can add a comment prefixed with the Translators keyword on the line preceding the string, e.g.: def my_view(request): # Translators: This message appears on the home page only output = ugettext("Welcome to my site.") The comment will then appear in the resulting .po file associated with the translatable contruct located below it and should also be displayed by most translation tools. 358 Chapter 3. Using Django
- 363. Django Documentation, Release1.5.1 Note: Just for completeness, this is the corresponding fragment of the resulting .po file: #. Translators: This message appears on the home page only # path/to/python/file.py:123 msgid "Welcome to my site." msgstr "" This also works in templates. See Comments for translators in templates for more details. Marking strings as no-op Use the function django.utils.translation.ugettext_noop() to mark a string as a translation string without translating it. The string is later translated from a variable. Use this if you have constant strings that should be stored in the source language because they are exchanged over systems or users – such as strings in a database – but should be translated at the last possible point in time, such as when the string is presented to the user. Pluralization Use the function django.utils.translation.ungettext() to specify pluralized messages. ungettext takes three arguments: the singular translation string, the plural translation string and the number of objects. This function is useful when you need your Django application to be localizable to languages where the number and complexity of plural forms is greater than the two forms used in English (‘object’ for the singular and ‘objects’ for all the cases where count is different from one, irrespective of its value.) For example: from django.utils.translation import ungettext def hello_world(request, count): page = ungettext( ’there is %(count)d object’, ’there are %(count)d objects’, count) % { ’count’: count, } return HttpResponse(page) In this example the number of objects is passed to the translation languages as the count variable. Lets see a slightly more complex usage example: from django.utils.translation import ungettext count = Report.objects.count() if count == 1: name = Report._meta.verbose_name else: name = Report._meta.verbose_name_plural text = ungettext( ’There is %(count)d %(name)s available.’, 3.14. Internationalization and localization 359
- 364. Django Documentation, Release1.5.1 ’There are %(count)d %(name)s available.’, count ) % { ’count’: count, ’name’: name } Here we reuse localizable, hopefully already translated literals (contained in the verbose_name and verbose_name_plural model Meta options) for other parts of the sentence so all of it is consistently based on the cardinality of the elements at play. Note: When using this technique, make sure you use a single name for every extrapolated variable included in the literal. In the example above note how we used the name Python variable in both translation strings. This example would fail: from django.utils.translation import ungettext from myapp.models import Report count = Report.objects.count() d = { ’count’: count, ’name’: Report._meta.verbose_name, ’plural_name’: Report._meta.verbose_name_plural } text = ungettext( ’There is %(count)d %(name)s available.’, ’There are %(count)d %(plural_name)s available.’, count ) % d You would get an error when running django-admin.py compilemessages: a format specification for argument ’name’, as in ’msgstr[0]’, doesn’t exist in ’msgid’ Contextual markers Sometimes words have several meanings, such as "May" in English, which refers to a month name and to a verb. To enable translators to translate these words correctly in different contexts, you can use the django.utils.translation.pgettext() function, or the django.utils.translation.npgettext() function if the string needs pluralization. Both take a context string as the first variable. In the resulting .po file, the string will then appear as often as there are different contextual markers for the same string (the context will appear on the msgctxt line), allowing the translator to give a different translation for each of them. For example: from django.utils.translation import pgettext month = pgettext("month name", "May") or: from django.utils.translation import pgettext_lazy 360 Chapter 3. Using Django
- 365. Django Documentation, Release1.5.1 class MyThing(models.Model): name = models.CharField(help_text=pgettext_lazy( ’help text for MyThing model’, ’This is the help text’)) will appear in the .po file as: msgctxt "month name" msgid "May" msgstr "" New in version 1.4. Contextual markers are also supported by the trans and blocktrans template tags. Lazy translation Use the lazy versions of translation functions in django.utils.translation (easily recognizable by the lazy suffix in their names) to translate strings lazily – when the value is accessed rather than when they’re called. These functions store a lazy reference to the string – not the actual translation. The translation itself will be done when the string is used in a string context, such as in template rendering. This is essential when calls to these functions are located in code paths that are executed at module load time. This is something that can easily happen when defining models, forms and model forms, because Django implements these such that their fields are actually class-level attributes. For that reason, make sure to use lazy translations in the following cases: Model fields and relationships verbose_name and help_text option values For example, to translate the help text of the name field in the following model, do the following: from django.utils.translation import ugettext_lazy as _ class MyThing(models.Model): name = models.CharField(help_text=_(’This is the help text’)) You can mark names of ForeignKey, ManyTomanyField or OneToOneField relationship as translatable by using their verbose_name options: from django.utils.translation import ugettext_lazy as _ class MyThing(models.Model): kind = models.ForeignKey(ThingKind, related_name=’kinds’, verbose_name=_(’kind’)) Just like you would do in verbose_name you should provide a lowercase verbose name text for the relation as Django will automatically titlecase it when required. Model verbose names values It is recommended to always provide explicit verbose_name and verbose_name_plural options rather than relying on the fallback English-centric and somewhat naïve deter-mination of verbose names Django performs by looking at the model’s class name: from django.utils.translation import ugettext_lazy as _ class MyThing(models.Model): name = models.CharField(_(’name’), help_text=_(’This is the help text’)) class Meta: 3.14. Internationalization and localization 361
- 366. Django Documentation, Release1.5.1 verbose_name = _(’my thing’) verbose_name_plural = _(’my things’) Model methods short_description attribute values For model methods, you can provide translations to Django and the admin site with the short_description attribute: from django.utils.translation import ugettext_lazy as _ class MyThing(models.Model): kind = models.ForeignKey(ThingKind, related_name=’kinds’, verbose_name=_(’kind’)) def is_mouse(self): return self.kind.type == MOUSE_TYPE is_mouse.short_description = _(’Is it a mouse?’) Working with lazy translation objects The result of a ugettext_lazy() call can be used wherever you would use a unicode string (an object with type unicode) in Python. If you try to use it where a bytestring (a str object) is expected, things will not work as expected, since a ugettext_lazy() object doesn’t know how to convert itself to a bytestring. You can’t use a unicode string inside a bytestring, either, so this is consistent with normal Python behavior. For example: # This is fine: putting a unicode proxy into a unicode string. u"Hello %s" % ugettext_lazy("people") # This will not work, since you cannot insert a unicode object # into a bytestring (nor can you insert our unicode proxy there) "Hello %s" % ugettext_lazy("people") If you ever see output that looks like "hello <django.utils.functional...>", you have tried to insert the result of ugettext_lazy() into a bytestring. That’s a bug in your code. If you don’t like the long ugettext_lazy name, you can just alias it as _ (underscore), like so: from django.utils.translation import ugettext_lazy as _ class MyThing(models.Model): name = models.CharField(help_text=_(’This is the help text’)) Using ugettext_lazy() and ungettext_lazy() to mark strings in models and utility functions is a common operation. When you’re working with these objects elsewhere in your code, you should ensure that you don’t acci-dentally convert them to strings, because they should be converted as late as possible (so that the correct locale is in effect). This necessitates the use of the helper function described next. Joining strings: string_concat() Standard Python string joins (”.join([...])) will not work on lists contain-ing lazy translation objects. Instead, you can use django.utils.translation.string_concat(), which creates a lazy object that concatenates its contents and converts them to strings only when the result is included in a string. For example: from django.utils.translation import string_concat ... name = ugettext_lazy(’John Lennon’) instrument = ugettext_lazy(’guitar’) result = string_concat(name, ’: ’, instrument) 362 Chapter 3. Using Django
- 367. Django Documentation, Release1.5.1 In this case, the lazy translations in result will only be converted to strings when result itself is used in a string (usually at template rendering time). Other uses of lazy in delayed translations For any other case where you would like to delay the translation, but have to pass the translatable string as argument to another function, you can wrap this function inside a lazy call yourself. For example: from django.utils import six # Python 3 compatibility from django.utils.functional import lazy from django.utils.safestring import mark_safe from django.utils.translation import ugettext_lazy as _ mark_safe_lazy = lazy(mark_safe, six.text_type) And then later: lazy_string = mark_safe_lazy(_("<p>My <strong>string!</strong></p>")) Localized names of languages get_language_info() The get_language_info() function provides detailed information about languages: >>> from django.utils.translation import get_language_info >>> li = get_language_info(’de’) >>> print(li[’name’], li[’name_local’], li[’bidi’]) German Deutsch False The name and name_local attributes of the dictionary contain the name of the language in English and in the language itself, respectively. The bidi attribute is True only for bi-directional languages. The source of the language information is the django.conf.locale module. Similar access to this information is available for template code. See below. Internationalization: in template code Translations in Django templates uses two template tags and a slightly different syntax than in Python code. To give your template access to these tags, put {% load i18n %} toward the top of your template. As with all template tags, this tag needs to be loaded in all templates which use translations, even those templates that extend from other templates which have already loaded the i18n tag. trans template tag The {% trans %} template tag translates either a constant string (enclosed in single or double quotes) or variable content: <title>{% trans "This is the title." %}</title> <title>{% trans myvar %}</title> If the noop option is present, variable lookup still takes place but the translation is skipped. This is useful when “stubbing out” content that will require translation in the future: <title>{% trans "myvar" noop %}</title> 3.14. Internationalization and localization 363
- 368. Django Documentation, Release1.5.1 Internally, inline translations use an ugettext() call. In case a template var (myvar above) is passed to the tag, the tag will first resolve such variable to a string at run-time and then look up that string in the message catalogs. It’s not possible to mix a template variable inside a string within {% trans %}. If your translations require strings with variables (placeholders), use {% blocktrans %} instead. New in version 1.4. If you’d like to retrieve a translated string without displaying it, you can use the following syntax: {% trans "This is the title" as the_title %} <title>{{ the_title }}</title> <meta name="description" content="{{ the_title }}"> In practice you’ll use this to get strings that are used in multiple places or should be used as arguments for other template tags or filters: {% trans "starting point" as start %} {% trans "end point" as end %} {% trans "La Grande Boucle" as race %} <h1> <a href="/" title="{% blocktrans %}Back to ’{{ race }}’ homepage{% endblocktrans %}">{{ race }}</a> </h1> <p> {% for stage in tour_stages %} {% cycle start end %}: {{ stage }}{% if forloop.counter|divisibleby:2 %}<br />{% else %}, {% endif {% endfor %} </p> New in version 1.4. {% trans %} also supports contextual markers using the context keyword: {% trans "May" context "month name" %} blocktrans template tag Contrarily to the trans tag, the blocktrans tag allows you to mark complex sentences consisting of literals and variable content for translation by making use of placeholders: {% blocktrans %}This string will have {{ value }} inside.{% endblocktrans %} To translate a template expression – say, accessing object attributes or using template filters – you need to bind the expression to a local variable for use within the translation block. Examples: {% blocktrans with amount=article.price %} That will cost $ {{ amount }}. {% endblocktrans %} {% blocktrans with myvar=value|filter %} This will have {{ myvar }} inside. {% endblocktrans %} You can use multiple expressions inside a single blocktrans tag: {% blocktrans with book_t=book|title author_t=author|title %} This is {{ book_t }} by {{ author_t }} {% endblocktrans %} 364 Chapter 3. Using Django
- 369. Django Documentation, Release1.5.1 Note: The previous more verbose format is still supported: {% blocktrans with book|title as book_t and author|title as author_t %} Changed in version 1.4. If resolving one of the block arguments fails, blocktrans will fall back to the default language by deactivating the currently active language temporarily with the deactivate_all() function. This tag also provides for pluralization. To use it: • Designate and bind a counter value with the name count. This value will be the one used to select the right plural form. • Specify both the singular and plural forms separating them with the {% plural %} tag within the {% blocktrans %} and {% endblocktrans %} tags. An example: {% blocktrans count counter=list|length %} There is only one {{ name }} object. {% plural %} There are {{ counter }} {{ name }} objects. {% endblocktrans %} A more complex example: {% blocktrans with amount=article.price count years=i.length %} That will cost $ {{ amount }} per year. {% plural %} That will cost $ {{ amount }} per {{ years }} years. {% endblocktrans %} When you use both the pluralization feature and bind values to local variables in addition to the counter value, keep in mind that the blocktrans construct is internally converted to an ungettext call. This means the same notes regarding ungettext variables apply. Reverse URL lookups cannot be carried out within the blocktrans and should be retrieved (and stored) beforehand: {% url ’path.to.view’ arg arg2 as the_url %} {% blocktrans %} This is a URL: {{ the_url }} {% endblocktrans %} New in version 1.4. {% blocktrans %} also supports contextual markers using the context keyword: {% blocktrans with name=user.username context "greeting" %}Hi {{ name }}{% endblocktrans %} Comments for translators in templates Just like with Python code, these notes for translators can be specified using comments, either with the comment tag: {% comment %}Translators: View verb{% endcomment %} {% trans "View" %} {% comment %}Translators: Short intro blurb{% endcomment %} <p>{% blocktrans %}A multiline translatable literal.{% endblocktrans %}</p> or with the {# ... #} one-line comment constructs: 3.14. Internationalization and localization 365
- 370. Django Documentation, Release1.5.1 {# Translators: Label of a button that triggers search{% endcomment #} <button type="submit">{% trans "Go" %}</button> {# Translators: This is a text of the base template #} {% blocktrans %}Ambiguous translatable block of text{% endblocktrans %} Note: Just for completeness, these are the corresponding fragments of the resulting .po file: #. Translators: View verb # path/to/template/file.html:10 msgid "View" msgstr "" #. Translators: Short intro blurb # path/to/template/file.html:13 msgid "" "A multiline translatable" "literal." msgstr "" # ... #. Translators: Label of a button that triggers search # path/to/template/file.html:100 msgid "Go" msgstr "" #. Translators: # path/to/template/file.html:103 msgid "Ambiguous translatable block of text" msgstr "" Other tags Each RequestContext has access to three translation-specific variables: • LANGUAGES is a list of tuples in which the first element is the language code and the second is the language name (translated into the currently active locale). • LANGUAGE_CODE is the current user’s preferred language, as a string. Example: en-us. (See How Django discovers language preference.) • LANGUAGE_BIDI is the current locale’s direction. If True, it’s a right-to-left language, e.g.: Hebrew, Arabic. If False it’s a left-to-right language, e.g.: English, French, German etc. If you don’t use the RequestContext extension, you can get those values with three tags: {% get_current_language as LANGUAGE_CODE %} {% get_available_languages as LANGUAGES %} {% get_current_language_bidi as LANGUAGE_BIDI %} These tags also require a {% load i18n %}. Translation hooks are also available within any template block tag that accepts constant strings. In those cases, just use _() syntax to specify a translation string: 366 Chapter 3. Using Django
- 371. Django Documentation, Release1.5.1 {% some_special_tag _("Page not found") value|yesno:_("yes,no") %} In this case, both the tag and the filter will see the already-translated string, so they don’t need to be aware of transla-tions. Note: In this example, the translation infrastructure will be passed the string "yes,no", not the individual strings "yes" and "no". The translated string will need to contain the comma so that the filter parsing code knows how to split up the arguments. For example, a German translator might translate the string "yes,no" as "ja,nein" (keeping the comma intact). You can also retrieve information about any of the available languages using provided template tags and filters. To get information about a single language, use the {% get_language_info %} tag: {% get_language_info for LANGUAGE_CODE as lang %} {% get_language_info for "pl" as lang %} You can then access the information: Language code: {{ lang.code }}<br /> Name of language: {{ lang.name_local }}<br /> Name in English: {{ lang.name }}<br /> Bi-directional: {{ lang.bidi }} You can also use the {% get_language_info_list %} template tag to retrieve information for a list of lan-guages (e.g. active languages as specified in LANGUAGES). See the section about the set_language redirect view for an example of how to display a language selector using {% get_language_info_list %}. In addition to LANGUAGES style nested tuples, {% get_language_info_list %} supports simple lists of language codes. If you do this in your view: return render_to_response(’mytemplate.html’, { ’available_languages’: [’en’, ’es’, ’fr’], }, RequestContext(request)) you can iterate over those languages in the template: {% get_language_info_list for available_languages as langs %} {% for lang in langs %} ... {% endfor %} There are also simple filters available for convenience: • {{ LANGUAGE_CODE|language_name }} (“German”) • {{ LANGUAGE_CODE|language_name_local }} (“Deutsch”) • {{ LANGUAGE_CODE|bidi }} (False) Internationalization: in JavaScript code Adding translations to JavaScript poses some problems: • JavaScript code doesn’t have access to a gettext implementation. • JavaScript code doesn’t have access to .po or .mo files; they need to be delivered by the server. • The translation catalogs for JavaScript should be kept as small as possible. Django provides an integrated solution for these problems: It passes the translations into JavaScript, so you can call gettext, etc., from within JavaScript. 3.14. Internationalization and localization 367
- 372. Django Documentation, Release1.5.1 The javascript_catalog view javascript_catalog(request, domain=’djangojs’, packages=None) The main solution to these problems is the django.views.i18n.javascript_catalog() view, which sends out a JavaScript code library with functions that mimic the gettext interface, plus an array of translation strings. Those translation strings are taken from applications or Django core, according to what you specify in either the info_dict or the URL. Paths listed in LOCALE_PATHS are also included. You hook it up like this: js_info_dict = { ’packages’: (’your.app.package’,), } urlpatterns = patterns(’’, (r’^jsi18n/$’, ’django.views.i18n.javascript_catalog’, js_info_dict), ) Each string in packages should be in Python dotted-package syntax (the same format as the strings in INSTALLED_APPS) and should refer to a package that contains a locale directory. If you specify multiple pack-ages, all those catalogs are merged into one catalog. This is useful if you have JavaScript that uses strings from different applications. The precedence of translations is such that the packages appearing later in the packages argument have higher precedence than the ones appearing at the beginning, this is important in the case of clashing translations for the same literal. By default, the view uses the djangojs gettext domain. This can be changed by altering the domain argument. You can make the view dynamic by putting the packages into the URL pattern: urlpatterns = patterns(’’, (r’^jsi18n/(?P<packages>S+?)/$’, ’django.views.i18n.javascript_catalog’), ) With this, you specify the packages as a list of package names delimited by ‘+’ signs in the URL. This is especially use-ful if your pages use code from different apps and this changes often and you don’t want to pull in one big catalog file. As a security measure, these values can only be either django.conf or any package from the INSTALLED_APPS setting. The JavaScript translations found in the paths listed in the LOCALE_PATHS setting are also always included. To keep consistency with the translations lookup order algorithm used for Python and templates, the directories listed in LOCALE_PATHS have the highest precedence with the ones appearing first having higher precedence than the ones appearing later. Using the JavaScript translation catalog To use the catalog, just pull in the dynamically generated script like this: <script type="text/javascript" src="{% url ’django.views.i18n.javascript_catalog’ %}"></script> This uses reverse URL lookup to find the URL of the JavaScript catalog view. When the catalog is loaded, your JavaScript code can use the standard gettext interface to access it: document.write(gettext(’this is to be translated’)); There is also an ngettext interface: 368 Chapter 3. Using Django
- 373. Django Documentation, Release1.5.1 var object_cnt = 1 // or 0, or 2, or 3, ... s = ngettext(’literal for the singular case’, ’literal for the plural case’, object_cnt); and even a string interpolation function: function interpolate(fmt, obj, named); The interpolation syntax is borrowed from Python, so the interpolate function supports both positional and named interpolation: • Positional interpolation: obj contains a JavaScript Array object whose elements values are then sequentially interpolated in their corresponding fmt placeholders in the same order they appear. For example: fmts = ngettext(’There is %s object. Remaining: %s’, ’There are %s objects. Remaining: %s’, 11); s = interpolate(fmts, [11, 20]); // s is ’There are 11 objects. Remaining: 20’ • Named interpolation: This mode is selected by passing the optional boolean named parameter as true. obj contains a JavaScript object or associative array. For example: d = { count: 10, total: 50 }; fmts = ngettext(’Total: %(total)s, there is %(count)s object’, ’there are %(count)s of a total of %(total)s objects’, d.count); s = interpolate(fmts, d, true); You shouldn’t go over the top with string interpolation, though: this is still JavaScript, so the code has to make repeated regular-expression substitutions. This isn’t as fast as string interpolation in Python, so keep it to those cases where you really need it (for example, in conjunction with ngettext to produce proper pluralizations). Note on performance The javascript_catalog() view generates the catalog from .mo files on every request. Since its output is constant— at least for a given version of a site— it’s a good candidate for caching. Server-side caching will reduce CPU load. It’s easily implemented with the cache_page() decorator. To trigger cache invalidation when your translations change, provide a version-dependant key prefix, as shown in the example below, or map the view at a version-dependant URL. from django.views.decorators.cache import cache_page from django.views.i18n import javascript_catalog # The value returned by get_version() must change when translations change. @cache_page(86400, key_prefix=’js18n-%s’ % get_version()) def cached_javascript_catalog(request, domain=’djangojs’, packages=None): return javascript_catalog(request, domain, packages) Client-side caching will save bandwidth and make your site load faster. If you’re using ETags (USE_ETAGS = True), you’re already covered. Otherwise, you can apply conditional decorators. In the following example, the cache is invalidated whenever your restart your application server. from django.utils import timezone from django.views.decorators.http import last_modified 3.14. Internationalization and localization 369
- 374. Django Documentation, Release1.5.1 from django.views.i18n import javascript_catalog last_modified_date = timezone.now() @last_modified(lambda req, **kw: last_modified_date) def cached_javascript_catalog(request, domain=’djangojs’, packages=None): return javascript_catalog(request, domain, packages) You can even pre-generate the javascript catalog as part of your deployment procedure and serve it as a static file. This radical technique is implemented in django-statici18n. Internationalization: in URL patterns New in version 1.4. Django provides two mechanisms to internationalize URL patterns: • Adding the language prefix to the root of the URL patterns to make it possible for LocaleMiddleware to detect the language to activate from the requested URL. • Making URL patterns themselves translatable via the django.utils.translation.ugettext_lazy() function. Warning: Using either one of these features requires that an active language be set for each re-quest; in other words, you need to have django.middleware.locale.LocaleMiddleware in your MIDDLEWARE_CLASSES setting. Language prefix in URL patterns i18n_patterns(prefix, pattern_description, ...) This function can be used in your root URLconf as a replacement for the normal django.conf.urls.patterns() function. Django will automatically prepend the current active language code to all url patterns defined within i18n_patterns(). Example URL patterns: from django.conf.urls import patterns, include, url from django.conf.urls.i18n import i18n_patterns urlpatterns = patterns(’’, url(r’^sitemap.xml$’, ’sitemap.view’, name=’sitemap_xml’), ) news_patterns = patterns(’’, url(r’^$’, ’news.views.index’, name=’index’), url(r’^category/(?P<slug>[w-]+)/$’, ’news.views.category’, name=’category’), url(r’^(?P<slug>[w-]+)/$’, ’news.views.details’, name=’detail’), ) urlpatterns += i18n_patterns(’’, url(r’^about/$’, ’about.view’, name=’about’), url(r’^news/’, include(news_patterns, namespace=’news’)), ) After defining these URL patterns, Django will automatically add the language prefix to the URL patterns that were added by the i18n_patterns function. Example: from django.core.urlresolvers import reverse from django.utils.translation import activate 370 Chapter 3. Using Django
- 375. Django Documentation, Release1.5.1 >>> activate(’en’) >>> reverse(’sitemap_xml’) ’/sitemap.xml’ >>> reverse(’news:index’) ’/en/news/’ >>> activate(’nl’) >>> reverse(’news:detail’, kwargs={’slug’: ’news-slug’}) ’/nl/news/news-slug/’ Warning: i18n_patterns() is only allowed in your root URLconf. Using it within an included URLconf will throw an ImproperlyConfigured exception. Warning: Ensure that you don’t have non-prefixed URL patterns that might collide with an automatically-added language prefix. Translating URL patterns URL patterns can also be marked translatable using the ugettext_lazy() function. Example: from django.conf.urls import patterns, include, url from django.conf.urls.i18n import i18n_patterns from django.utils.translation import ugettext_lazy as _ urlpatterns = patterns(’’ url(r’^sitemap.xml$’, ’sitemap.view’, name=’sitemap_xml’), ) news_patterns = patterns(’’ url(r’^$’, ’news.views.index’, name=’index’), url(_(r’^category/(?P<slug>[w-]+)/$’), ’news.views.category’, name=’category’), url(r’^(?P<slug>[w-]+)/$’, ’news.views.details’, name=’detail’), ) urlpatterns += i18n_patterns(’’, url(_(r’^about/$’), ’about.view’, name=’about’), url(_(r’^news/’), include(news_patterns, namespace=’news’)), ) After you’ve created the translations, the reverse() function will return the URL in the active language. Example: from django.core.urlresolvers import reverse from django.utils.translation import activate >>> activate(’en’) >>> reverse(’news:category’, kwargs={’slug’: ’recent’}) ’/en/news/category/recent/’ >>> activate(’nl’) >>> reverse(’news:category’, kwargs={’slug’: ’recent’}) ’/nl/nieuws/categorie/recent/’ Warning: In most cases, it’s best to use translated URLs only within a language-code-prefixed block of patterns (using i18n_patterns()), to avoid the possibility that a carelessly translated URL causes a collision with a non-translated URL pattern. 3.14. Internationalization and localization 371
- 376. Django Documentation, Release1.5.1 Reversing in templates If localized URLs get reversed in templates they always use the current language. To link to a URL in another language use the language template tag. It enables the given language in the enclosed template section: {% load i18n %} {% get_available_languages as languages %} {% trans "View this category in:" %} {% for lang_code, lang_name in languages %} {% language lang_code %} <a href="{% url ’category’ slug=category.slug %}">{{ lang_name }}</a> {% endlanguage %} {% endfor %} The language tag expects the language code as the only argument. Localization: how to create language files Once the string literals of an application have been tagged for later translation, the translation themselves need to be written (or obtained). Here’s how that works. Locale restrictions Django does not support localizing your application into a locale for which Django itself has not been translated. In this case, it will ignore your translation files. If you were to try this and Django supported it, you would inevitably see a mixture of translated strings (from your application) and English strings (from Django itself). If you want to support a locale for your application that is not already part of Django, you’ll need to make at least a minimal translation of the Django core. A good starting point is to copy the Django English .po file and to translate at least some translation strings. Message files The first step is to create a message file for a new language. A message file is a plain-text file, representing a single language, that contains all available translation strings and how they should be represented in the given language. Message files have a .po file extension. Django comes with a tool, django-admin.py makemessages, that automates the creation and upkeep of these files. Gettext utilities The makemessages command (and compilemessages discussed later) use commands from the GNU gettext toolset: xgettext, msgfmt, msgmerge and msguniq. The minimum version of the gettext utilities supported is 0.15. To create or update a message file, run this command: 372 Chapter 3. Using Django
- 377. Django Documentation, Release1.5.1 django-admin.py makemessages -l de ...where de is the language code for the message file you want to create. The language code, in this case, is in locale format. For example, it’s pt_BR for Brazilian Portuguese and de_AT for Austrian German. The script should be run from one of two places: • The root directory of your Django project. • The root directory of your Django app. The script runs over your project source tree or your application source tree and pulls out all strings marked for trans-lation. It creates (or updates) a message file in the directory locale/LANG/LC_MESSAGES. In the de example, the file will be locale/de/LC_MESSAGES/django.po. By default django-admin.py makemessages examines every file that has the .html or .txt file extension. In case you want to override that default, use the --extension or -e option to specify the file extensions to examine: django-admin.py makemessages -l de -e txt Separate multiple extensions with commas and/or use -e or --extension multiple times: django-admin.py makemessages -l de -e html,txt -e xml Warning: When creating message files from JavaScript source code you need to use the special ‘djangojs’ domain, not -e js. No gettext? If you don’t have the gettext utilities installed, makemessages will create empty files. If that’s the case, either install the gettext utilities or just copy the English message file (locale/en/LC_MESSAGES/django.po) if available and use it as a starting point; it’s just an empty translation file. Working on Windows? If you’re using Windows and need to install the GNU gettext utilities so makemessages works, see gettext on Windows for more information. The format of .po files is straightforward. Each .po file contains a small bit of metadata, such as the translation maintainer’s contact information, but the bulk of the file is a list of messages – simple mappings between translation strings and the actual translated text for the particular language. For example, if your Django app contained a translation string for the text "Welcome to my site.", like so: _("Welcome to my site.") ...then django-admin.py makemessages will have created a .po file containing the following snippet – a message: #: path/to/python/module.py:23 msgid "Welcome to my site." msgstr "" A quick explanation: • msgid is the translation string, which appears in the source. Don’t change it. 3.14. Internationalization and localization 373
- 378. Django Documentation, Release1.5.1 • msgstr is where you put the language-specific translation. It starts out empty, so it’s your responsibility to change it. Make sure you keep the quotes around your translation. • As a convenience, each message includes, in the form of a comment line prefixed with # and located above the msgid line, the filename and line number from which the translation string was gleaned. Long messages are a special case. There, the first string directly after the msgstr (or msgid) is an empty string. Then the content itself will be written over the next few lines as one string per line. Those strings are directly concatenated. Don’t forget trailing spaces within the strings; otherwise, they’ll be tacked together without whitespace! Mind your charset When creating a PO file with your favorite text editor, first edit the charset line (search for "CHARSET") and set it to the charset you’ll be using to edit the content. Due to the way the gettext tools work internally and because we want to allow non-ASCII source strings in Django’s core and your applications, you must use UTF-8 as the encoding for your PO file. This means that everybody will be using the same encoding, which is important when Django processes the PO files. To reexamine all source code and templates for new translation strings and update all message files for all languages, run this: django-admin.py makemessages -a Compiling message files After you create your message file – and each time you make changes to it – you’ll need to compile it into a more efficient form, for use by gettext. Do this with the django-admin.py compilemessages utility. This tool runs over all available .po files and creates .mo files, which are binary files optimized for use by gettext. In the same directory from which you ran django-admin.py makemessages, run django-admin.py compilemessages like this: django-admin.py compilemessages That’s it. Your translations are ready for use. Working on Windows? If you’re usingWindows and need to install the GNU gettext utilities so django-admin.py compilemessages works see gettext on Windows for more information. .po files: Encoding and BOM usage. Django only supports .po files encoded in UTF-8 and without any BOM (Byte Order Mark) so if your text editor adds such marks to the beginning of files by default then you will need to reconfigure it. Creating message files from JavaScript source code You create and update the message files the same way as the other Django message files – with the django-admin.py makemessages tool. The only difference is you need to explicitly specify what in get-text parlance is known as a domain in this case the djangojs domain, by providing a -d djangojs parameter, like this: 374 Chapter 3. Using Django
- 379. Django Documentation, Release1.5.1 django-admin.py makemessages -d djangojs -l de This would create or update the message file for JavaScript for German. After updating message files, just run django-admin.py compilemessages the same way as you do with normal Django message files. gettext on Windows This is only needed for people who either want to extract message IDs or compile message files (.po). Translation work itself just involves editing existing files of this type, but if you want to create your own message files, or want to test or compile a changed message file, you will need the gettext utilities: • Download the following zip files from the GNOME servers http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/ or from one of its mirrors – gettext-runtime-X.zip – gettext-tools-X.zip X is the version number, we are requiring 0.15 or higher. • Extract the contents of the bin directories in both files to the same folder on your system (i.e. C:Program Filesgettext-utils) • Update the system PATH: – Control Panel > System > Advanced > Environment Variables. – In the System variables list, click Path, click Edit. – Add ;C:Program Filesgettext-utilsbin at the end of the Variable value field. You may also use gettext binaries you have obtained elsewhere, so long as the xgettext --version com-mand works properly. Do not attempt to use Django translation utilities with a gettext package if the command xgettext --version entered at a Windows command prompt causes a popup window saying “xgettext.exe has generated errors and will be closed by Windows”. Miscellaneous The set_language redirect view set_language(request) As a convenience, Django comes with a view, django.views.i18n.set_language(), that sets a user’s lan-guage preference and redirects to a given URL or, by default, back to the previous page. Make sure that the following item is in your TEMPLATE_CONTEXT_PROCESSORS list in your settings file: ’django.core.context_processors.i18n’ Activate this view by adding the following line to your URLconf: (r’^i18n/’, include(’django.conf.urls.i18n’)), (Note that this example makes the view available at /i18n/setlang/.) Warning: Make sure that you don’t include the above URL within i18n_patterns() - it needs to be language-independent itself to work correctly. 3.14. Internationalization and localization 375
- 380. Django Documentation, Release1.5.1 The view expects to be called via the POST method, with a language parameter set in request. If session support is enabled, the view saves the language choice in the user’s session. Otherwise, it saves the language choice in a cookie that is by default named django_language. (The name can be changed through the LANGUAGE_COOKIE_NAME setting.) After setting the language choice, Django redirects the user, following this algorithm: • Django looks for a next parameter in the POST data. • If that doesn’t exist, or is empty, Django tries the URL in the Referrer header. • If that’s empty – say, if a user’s browser suppresses that header – then the user will be redirected to / (the site root) as a fallback. Here’s example HTML template code: <form action="{% url ’set_language’ %}" method="post"> {% csrf_token %} <input name="next" type="hidden" value="{{ redirect_to }}" /> <select name="language"> {% get_language_info_list for LANGUAGES as languages %} {% for language in languages %} <option value="{{ language.code }}">{{ language.name_local }} ({{ language.code }})</option> {% endfor %} </select> <input type="submit" value="Go" /> </form> In this example, Django looks up the URL of the page to which the user will be redirected in the redirect_to context variable. Using translations outside views and templates While Django provides a rich set of i18n tools for use in views and templates, it does not restrict the usage to Django-specific code. The Django translation mechanisms can be used to translate arbitrary texts to any language that is supported by Django (as long as an appropriate translation catalog exists, of course). You can load a translation catalog, activate it and translate text to language of your choice, but remember to switch back to original language, as activating a translation catalog is done on per-thread basis and such change will affect code running in the same thread. For example: from django.utils import translation def welcome_translated(language): cur_language = translation.get_language() try: translation.activate(language) text = translation.ugettext(’welcome’) finally: translation.activate(cur_language) return text Calling this function with the value ‘de’ will give you "Willkommen", regardless of LANGUAGE_CODE and lan-guage set by middleware. Functions of particular interest are django.utils.translation.get_language() which returns the lan-guage used in the current thread, django.utils.translation.activate() which activates a translation catalog for the current thread, and django.utils.translation.check_for_language() which checks if the given language is supported by Django. 376 Chapter 3. Using Django
- 381. Django Documentation, Release1.5.1 Implementation notes Specialties of Django translation Django’s translation machinery uses the standard gettext module that comes with Python. If you know gettext, you might note these specialties in the way Django does translation: • The string domain is django or djangojs. This string domain is used to differentiate between different pro-grams that store their data in a common message-file library (usually /usr/share/locale/). The django domain is used for python and template translation strings and is loaded into the global translation catalogs. The djangojs domain is only used for JavaScript translation catalogs to make sure that those are as small as possible. • Django doesn’t use xgettext alone. It uses Python wrappers around xgettext and msgfmt. This is mostly for convenience. How Django discovers language preference Once you’ve prepared your translations – or, if you just want to use the translations that come with Django – you’ll just need to activate translation for your app. Behind the scenes, Django has a very flexible model of deciding which language should be used – installation-wide, for a particular user, or both. To set an installation-wide language preference, set LANGUAGE_CODE. Django uses this language as the default translation – the final attempt if no other translator finds a translation. If all you want to do is run Django with your native language, and a language file is available for it, all you need to do is set LANGUAGE_CODE. If you want to let each individual user specify which language he or she prefers, use LocaleMiddleware. LocaleMiddleware enables language selection based on data from the request. It customizes content for each user. To use LocaleMiddleware, add ’django.middleware.locale.LocaleMiddleware’ to your MIDDLEWARE_CLASSES setting. Because middleware order matters, you should follow these guidelines: • Make sure it’s one of the first middlewares installed. • It should come after SessionMiddleware, because LocaleMiddleware makes use of session data. And it should come before CommonMiddleware because CommonMiddleware needs an activated language in order to resolve the requested URL. • If you use CacheMiddleware, put LocaleMiddleware after it. For example, your MIDDLEWARE_CLASSES might look like this: MIDDLEWARE_CLASSES = ( ’django.contrib.sessions.middleware.SessionMiddleware’, ’django.middleware.locale.LocaleMiddleware’, ’django.middleware.common.CommonMiddleware’, ) (For more on middleware, see the middleware documentation.) LocaleMiddleware tries to determine the user’s language preference by following this algorithm: Changed in version 1.4. 3.14. Internationalization and localization 377
- 382. Django Documentation, Release1.5.1 • First, it looks for the language prefix in the requested URL. This is only performed when you are using the i18n_patterns function in your root URLconf. See Internationalization: in URL patterns for more infor-mation about the language prefix and how to internationalize URL patterns. • Failing that, it looks for a django_language key in the current user’s session. • Failing that, it looks for a cookie. The name of the cookie used is set by the LANGUAGE_COOKIE_NAME setting. (The default name is django_language.) • Failing that, it looks at the Accept-Language HTTP header. This header is sent by your browser and tells the server which language(s) you prefer, in order by priority. Django tries each language in the header until it finds one with available translations. • Failing that, it uses the global LANGUAGE_CODE setting. Notes: • In each of these places, the language preference is expected to be in the standard language format, as a string. For example, Brazilian Portuguese is pt-br. • If a base language is available but the sublanguage specified is not, Django uses the base language. For example, if a user specifies de-at (Austrian German) but Django only has de available, Django uses de. • Only languages listed in the LANGUAGES setting can be selected. If you want to restrict the language selection to a subset of provided languages (because your application doesn’t provide all those languages), set LANGUAGES to a list of languages. For example: LANGUAGES = ( (’de’, _(’German’)), (’en’, _(’English’)), ) This example restricts languages that are available for automatic selection to German and English (and any sublanguage, like de-ch or en-us). • If you define a custom LANGUAGES setting, as explained in the previous bullet, it’s OK to mark the languages as translation strings – but use a “dummy” ugettext() function, not the one in django.utils.translation. You should never import django.utils.translation from within your settings file, because that module in itself depends on the settings, and that would cause a circular import. The solution is to use a “dummy” ugettext() function. Here’s a sample settings file: ugettext = lambda s: s LANGUAGES = ( (’de’, ugettext(’German’)), (’en’, ugettext(’English’)), ) With this arrangement, django-admin.py makemessages will still find and mark these strings for trans-lation, but the translation won’t happen at runtime – so you’ll have to remember to wrap the languages in the real ugettext() in any code that uses LANGUAGES at runtime. • The LocaleMiddleware can only select languages for which there is a Django-provided base translation. If you want to provide translations for your application that aren’t already in the set of translations in Django’s source tree, you’ll want to provide at least a basic one as described in the Locale restrictions note. Once LocaleMiddleware determines the user’s preference, it makes this preference available as request.LANGUAGE_CODE for each HttpRequest. Feel free to read this value in your view code. Here’s a simple example: 378 Chapter 3. Using Django
- 383. Django Documentation, Release1.5.1 def hello_world(request, count): if request.LANGUAGE_CODE == ’de-at’: return HttpResponse("You prefer to read Austrian German.") else: return HttpResponse("You prefer to read another language.") Note that, with static (middleware-less) translation, the language is in settings.LANGUAGE_CODE, while with dynamic (middleware) translation, it’s in request.LANGUAGE_CODE. How Django discovers translations At runtime, Django builds an in-memory unified catalog of literals-translations. To achieve this it looks for translations by following this algorithm regarding the order in which it examines the different file paths to load the compiled message files (.mo) and the precedence of multiple translations for the same literal: 1. The directories listed in LOCALE_PATHS have the highest precedence, with the ones appearing first having higher precedence than the ones appearing later. 2. Then, it looks for and uses if it exists a locale directory in each of the installed apps listed in INSTALLED_APPS. The ones appearing first have higher precedence than the ones appearing later. 3. Finally, the Django-provided base translation in django/conf/locale is used as a fallback. See Also: The translations for literals included in JavaScript assets are looked up following a similar but not identical algorithm. See the javascript_catalog view documentation for more details. In all cases the name of the directory containing the translation is expected to be named using locale name notation. E.g. de, pt_BR, es_AR, etc. This way, you can write applications that include their own translations, and you can override base translations in your project. Or, you can just build a big project out of several apps and put all translations into one big common message file specific to the project you are composing. The choice is yours. All message file repositories are structured the same way. They are: • All paths listed in LOCALE_PATHS in your settings file are searched for <language>/LC_MESSAGES/django.(po|mo) • $APPPATH/locale/<language>/LC_MESSAGES/django.(po|mo) • $PYTHONPATH/django/conf/locale/<language>/LC_MESSAGES/django.(po|mo) To create message files, you use the django-admin.py makemessages tool. You only need to be in the same directory where the locale/ directory is located. And you use django-admin.py compilemessages to produce the binary .mo files that are used by gettext. You can also run django-admin.py compilemessages --settings=path.to.settings to make the compiler process all the directories in your LOCALE_PATHS setting. Finally, you should give some thought to the structure of your translation files. If your applications need to be delivered to other users and will be used in other projects, you might want to use app-specific translations. But using app-specific translations and project-specific translations could produce weird problems with makemessages: it will traverse all directories below the current path and so might put message IDs into a unified, common message file for the current project that are already in application message files. The easiest way out is to store applications that are not part of the project (and so carry their own translations) outside the project tree. That way, django-admin.py makemessages, when ran on a project level will only extract strings that are connected to your explicit project and not strings that are distributed independently. 3.14. Internationalization and localization 379
- 384. Django Documentation, Release1.5.1 3.14.2 Format localization Overview Django’s formatting system is capable to display dates, times and numbers in templates using the format specified for the current locale. It also handles localized input in forms. When it’s enabled, two users accessing the same content may see dates, times and numbers formatted in different ways, depending on the formats for their current locale. The formatting system is disabled by default. To enable it, it’s necessary to set USE_L10N = True in your settings file. Note: The default settings.py file created by django-admin.py startproject includes USE_L10N = True for convenience. Note, however, that to enable number formatting with thousand separators it is necessary to set USE_THOUSAND_SEPARATOR = True in your settings file. Alternatively, you could use intcomma to format numbers in your template. Note: There is also an independent but related USE_I18N setting that controls if Django should activate translation. See Translation for more details. Locale aware input in forms When formatting is enabled, Django can use localized formats when parsing dates, times and numbers in forms. That means it tries different formats for different locales when guessing the format used by the user when inputting data on forms. Note: Django uses different formats for displaying data to those it uses for parsing data. Most notably, the formats for parsing dates can’t use the %a (abbreviated weekday name), %A (full weekday name), %b (abbreviated month name), %B (full month name), or %p (AM/PM). To enable a form field to localize input and output data simply use its localize argument: class CashRegisterForm(forms.Form): product = forms.CharField() revenue = forms.DecimalField(max_digits=4, decimal_places=2, localize=True) Controlling localization in templates When you have enabled formatting with USE_L10N, Django will try to use a locale specific format whenever it outputs a value in a template. However, it may not always be appropriate to use localized values – for example, if you’re outputting Javascript or XML that is designed to be machine-readable, you will always want unlocalized values. You may also want to use localization in selected templates, rather than using localization everywhere. To allow for fine control over the use of localization, Django provides the l10n template library that contains the following tags and filters. 380 Chapter 3. Using Django
- 385. Django Documentation, Release1.5.1 Template tags localize Enables or disables localization of template variables in the contained block. This tag allows a more fine grained control of localization than USE_L10N. To activate or deactivate localization for a template block, use: {% load l10n %} {% localize on %} {{ value }} {% endlocalize %} {% localize off %} {{ value }} {% endlocalize %} Note: The value of USE_L10N isn’t respected inside of a {% localize %} block. See localize and unlocalize for template filters that will do the same job on a per-variable basis. Template filters localize Forces localization of a single value. For example: {% load l10n %} {{ value|localize }} To disable localization on a single value, use unlocalize. To control localization over a large section of a template, use the localize template tag. unlocalize Forces a single value to be printed without localization. For example: {% load l10n %} {{ value|unlocalize }} To force localization of a single value, use localize. To control localization over a large section of a template, use the localize template tag. Creating custom format files Django provides format definitions for many locales, but sometimes you might want to create your own, because a format files doesn’t exist for your locale, or because you want to overwrite some of the values. To use custom formats, specify the path where you’ll place format files first. To do that, just set your FORMAT_MODULE_PATH setting to the package where format files will exist, for instance: FORMAT_MODULE_PATH = ’mysite.formats’ 3.14. Internationalization and localization 381
- 386. Django Documentation, Release1.5.1 Files are not placed directly in this directory, but in a directory named as the locale, and must be named formats.py. To customize the English formats, a structure like this would be needed: mysite/ formats/ __init__.py en/ __init__.py formats.py where formats.py contains custom format definitions. For example: THOUSAND_SEPARATOR = u’xa0’ to use a non-breaking space (Unicode 00A0) as a thousand separator, instead of the default for English, a comma. Limitations of the provided locale formats Some locales use context-sensitive formats for numbers, which Django’s localization system cannot handle automati-cally. Switzerland (German) The Swiss number formatting depends on the type of number that is being formatted. For monetary values, a comma is used as the thousand separator and a decimal point for the decimal separator. For all other numbers, a comma is used as decimal separator and a space as thousand separator. The locale format provided by Django uses the generic separators, a comma for decimal and a space for thousand separators. 3.14.3 Time zones New in version 1.4. Overview When support for time zones is enabled, Django stores date and time information in UTC in the database, uses time-zone- aware datetime objects internally, and translates them to the end user’s time zone in templates and forms. This is handy if your users live in more than one time zone and you want to display date and time information according to each user’s wall clock. Even if your Web site is available in only one time zone, it’s still good practice to store data in UTC in your database. One main reason is Daylight Saving Time (DST). Many countries have a system of DST, where clocks are moved forward in spring and backward in autumn. If you’re working in local time, you’re likely to encounter errors twice a year, when the transitions happen. (The pytz documentation discusses these issues in greater detail.) This probably doesn’t matter for your blog, but it’s a problem if you over-bill or under-bill your customers by one hour, twice a year, every year. The solution to this problem is to use UTC in the code and use local time only when interacting with end users. Time zone support is disabled by default. To enable it, set USE_TZ = True in your settings file. Installing pytz is highly recommended, but not mandatory. It’s as simple as: $ sudo pip install pytz 382 Chapter 3. Using Django
- 387. Django Documentation, Release1.5.1 Note: The default settings.py file created by django-admin.py startproject includes USE_TZ = True for convenience. Note: There is also an independent but related USE_L10N setting that controls whether Django should activate format localization. See Format localization for more details. If you’re wrestling with a particular problem, start with the time zone FAQ. Concepts Naive and aware datetime objects Python’s datetime.datetime objects have a tzinfo attribute that can be used to store time zone information, represented as an instance of a subclass of datetime.tzinfo. When this attribute is set and describes an offset, a datetime object is aware. Otherwise, it’s naive. You can use is_aware() and is_naive() to determine whether datetimes are aware or naive. When time zone support is disabled, Django uses naive datetime objects in local time. This is simple and sufficient for many use cases. In this mode, to obtain the current time, you would write: import datetime now = datetime.datetime.now() When time zone support is enabled, Django uses time-zone-aware datetime objects. If your code creates datetime objects, they should be aware too. In this mode, the example above becomes: import datetime from django.utils.timezone import utc now = datetime.datetime.utcnow().replace(tzinfo=utc) Note: django.utils.timezone provides a now() function that returns a naive or aware datetime object according to the value of USE_TZ. Warning: Dealing with aware datetime objects isn’t always intuitive. For instance, the tzinfo argument of the standard datetime constructor doesn’t work reliably for time zones with DST. Using UTC is generally safe; if you’re using other time zones, you should review the pytz documentation carefully. Note: Python’s datetime.time objects also feature a tzinfo attribute, and PostgreSQL has a matching time with time zone type. However, as PostgreSQL’s docs put it, this type “exhibits properties which lead to ques-tionable usefulness”. Django only supports naive time objects and will raise an exception if you attempt to save an aware time object. 3.14. Internationalization and localization 383
- 388. Django Documentation, Release1.5.1 Interpretation of naive datetime objects When USE_TZ is True, Django still accepts naive datetime objects, in order to preserve backwards-compatibility. When the database layer receives one, it attempts to make it aware by interpreting it in the default time zone and raises a warning. Unfortunately, during DST transitions, some datetimes don’t exist or are ambiguous. In such situations, pytz raises an exception. Other tzinfo implementations, such as the local time zone used as a fallback when pytz isn’t installed, may raise an exception or return inaccurate results. That’s why you should always create aware datetime objects when time zone support is enabled. In practice, this is rarely an issue. Django gives you aware datetime objects in the models and forms, and most often, new datetime objects are created from existing ones through timedelta arithmetic. The only datetime that’s often created in application code is the current time, and timezone.now() automatically does the right thing. Default time zone and current time zone The default time zone is the time zone defined by the TIME_ZONE setting. The current time zone is the time zone that’s used for rendering. You should set the current time zone to the end user’s actual time zone with activate(). Otherwise, the default time zone is used. Note: As explained in the documentation of TIME_ZONE, Django sets environment variables so that its process runs in the default time zone. This happens regardless of the value of USE_TZ and of the current time zone. When USE_TZ is True, this is useful to preserve backwards-compatibility with applications that still rely on local time. However, as explained above, this isn’t entirely reliable, and you should always work with aware datetimes in UTC in your own code. For instance, use utcfromtimestamp() instead of fromtimestamp() – and don’t forget to set tzinfo to utc. Selecting the current time zone The current time zone is the equivalent of the current locale for translations. However, there’s no equivalent of the Accept-Language HTTP header that Django could use to determine the user’s time zone automatically. Instead, Django provides time zone selection functions. Use them to build the time zone selection logic that makes sense for you. Most Web sites that care about time zones just ask users in which time zone they live and store this information in the user’s profile. For anonymous users, they use the time zone of their primary audience or UTC. pytz provides helpers, like a list of time zones per country, that you can use to pre-select the most likely choices. Here’s an example that stores the current timezone in the session. (It skips error handling entirely for the sake of simplicity.) Add the following middleware to MIDDLEWARE_CLASSES: from django.utils import timezone class TimezoneMiddleware(object): def process_request(self, request): tz = request.session.get(’django_timezone’) if tz: timezone.activate(tz) 384 Chapter 3. Using Django
- 389. Django Documentation, Release1.5.1 Create a view that can set the current timezone: import pytz from django.shortcuts import redirect, render def set_timezone(request): if request.method == ’POST’: request.session[’django_timezone’] = pytz.timezone(request.POST[’timezone’]) return redirect(’/’) else: return render(request, ’template.html’, {’timezones’: pytz.common_timezones}) Include a form in template.html that will POST to this view: {% load tz %} <form action="{% url ’set_timezone’ %}" method="POST"> {% csrf_token %} <label for="timezone">Time zone:</label> <select name="timezone"> {% for tz in timezones %} <option value="{{ tz }}"{% if tz == TIME_ZONE %} selected="selected"{% endif %}>{{ tz }}</option> {% endfor %} </select> <input type="submit" value="Set" /> </form> Time zone aware input in forms When you enable time zone support, Django interprets datetimes entered in forms in the current time zone and returns aware datetime objects in cleaned_data. If the current time zone raises an exception for datetimes that don’t exist or are ambiguous because they fall in a DST transition (the timezones provided by pytz do this), such datetimes will be reported as invalid values. Time zone aware output in templates When you enable time zone support, Django converts aware datetime objects to the current time zone when they’re rendered in templates. This behaves very much like format localization. Warning: Django doesn’t convert naive datetime objects, because they could be ambiguous, and because your code should never produce naive datetimes when time zone support is enabled. However, you can force conversion with the template filters described below. Conversion to local time isn’t always appropriate – you may be generating output for computers rather than for humans. The following filters and tags, provided by the tz template tag library, allow you to control the time zone conversions. Template tags localtime Enables or disables conversion of aware datetime objects to the current time zone in the contained block. This tag has exactly the same effects as the USE_TZ setting as far as the template engine is concerned. It allows a more fine grained control of conversion. To activate or deactivate conversion for a template block, use: 3.14. Internationalization and localization 385
- 390. Django Documentation, Release1.5.1 {% load tz %} {% localtime on %} {{ value }} {% endlocaltime %} {% localtime off %} {{ value }} {% endlocaltime %} Note: The value of USE_TZ isn’t respected inside of a {% localtime %} block. timezone Sets or unsets the current time zone in the contained block. When the current time zone is unset, the default time zone applies. {% load tz %} {% timezone "Europe/Paris" %} Paris time: {{ value }} {% endtimezone %} {% timezone None %} Server time: {{ value }} {% endtimezone %} get_current_timezone When the django.core.context_processors.tz context processor is enabled – by default, it is – each RequestContext contains a TIME_ZONE variable that provides the name of the current time zone. If you don’t use a RequestContext, you can obtain this value with the get_current_timezone tag: {% get_current_timezone as TIME_ZONE %} Template filters These filters accept both aware and naive datetimes. For conversion purposes, they assume that naive datetimes are in the default time zone. They always return aware datetimes. localtime Forces conversion of a single value to the current time zone. For example: {% load tz %} {{ value|localtime }} utc Forces conversion of a single value to UTC. For example: {% load tz %} {{ value|utc }} 386 Chapter 3. Using Django
- 391. Django Documentation, Release1.5.1 timezone Forces conversion of a single value to an arbitrary timezone. The argument must be an instance of a tzinfo subclass or a time zone name. If it is a time zone name, pytz is required. For example: {% load tz %} {{ value|timezone:"Europe/Paris" }} Migration guide Here’s how to migrate a project that was started before Django supported time zones. Database PostgreSQL The PostgreSQL backend stores datetimes as timestamp with time zone. In practice, this means it converts datetimes from the connection’s time zone to UTC on storage, and from UTC to the connection’s time zone on retrieval. As a consequence, if you’re using PostgreSQL, you can switch between USE_TZ = False and USE_TZ = True freely. The database connection’s time zone will be set to TIME_ZONE or UTC respectively, so that Django obtains correct datetimes in all cases. You don’t need to perform any data conversions. Other databases Other backends store datetimes without time zone information. If you switch from USE_TZ = False to USE_TZ = True, you must convert your data from local time to UTC – which isn’t deterministic if your local time has DST. Code The first step is to add USE_TZ = True to your settings file and install pytz (if possible). At this point, things should mostly work. If you create naive datetime objects in your code, Django makes them aware when necessary. However, these conversions may fail around DST transitions, which means you aren’t getting the full benefits of time zone support yet. Also, you’re likely to run into a few problems because it’s impossible to compare a naive datetime with an aware datetime. Since Django now gives you aware datetimes, you’ll get exceptions wherever you compare a datetime that comes from a model or a form with a naive datetime that you’ve created in your code. So the second step is to refactor your code wherever you instantiate datetime objects to make them aware. This can be done incrementally. django.utils.timezone defines some handy helpers for compatibility code: now(), is_aware(), is_naive(), make_aware(), and make_naive(). Finally, in order to help you locate code that needs upgrading, Django raises a warning when you attempt to save a naive datetime to the database: RuntimeWarning: DateTimeField received a naive datetime (2012-01-01 00:00:00) while time zone support During development, you can turn such warnings into exceptions and get a traceback by adding the following to your settings file: 3.14. Internationalization and localization 387
- 392. Django Documentation, Release1.5.1 import warnings warnings.filterwarnings( ’error’, r"DateTimeField received a naive datetime", RuntimeWarning, r’django.db.models.fields’) Fixtures When serializing an aware datetime, the UTC offset is included, like this: "2011-09-01T13:20:30+03:00" For a naive datetime, it obviously isn’t: "2011-09-01T13:20:30" For models with DateTimeFields, this difference makes it impossible to write a fixture that works both with and without time zone support. Fixtures generated with USE_TZ = False, or before Django 1.4, use the “naive” format. If your project contains such fixtures, after you enable time zone support, you’ll see RuntimeWarnings when you load them. To get rid of the warnings, you must convert your fixtures to the “aware” format. You can regenerate fixtures with loaddata then dumpdata. Or, if they’re small enough, you can simply edit them to add the UTC offset that matches your TIME_ZONE to each serialized datetime. FAQ Setup 1. I don’t need multiple time zones. Should I enable time zone support? Yes. When time zone support is enabled, Django uses a more accurate model of local time. This shields you from subtle and unreproducible bugs around Daylight Saving Time (DST) transitions. In this regard, time zones are comparable to unicode in Python. At first it’s hard. You get encoding and decoding errors. Then you learn the rules. And some problems disappear – you never get mangled output again when your application receives non-ASCII input. When you enable time zone support, you’ll encounter some errors because you’re using naive datetimes where Django expects aware datetimes. Such errors show up when running tests and they’re easy to fix. You’ll quickly learn how to avoid invalid operations. On the other hand, bugs caused by the lack of time zone support are much harder to prevent, diagnose and fix. Anything that involves scheduled tasks or datetime arithmetic is a candidate for subtle bugs that will bite you only once or twice a year. For these reasons, time zone support is enabled by default in new projects, and you should keep it unless you have a very good reason not to. 2. I’ve enabled time zone support. Am I safe? Maybe. You’re better protected from DST-related bugs, but you can still shoot yourself in the foot by carelessly turning naive datetimes into aware datetimes, and vice-versa. If your application connects to other systems – for instance, if it queries a Web service – make sure datetimes are properly specified. To transmit datetimes safely, their representation should include the UTC offset, or their values should be in UTC (or both!). 388 Chapter 3. Using Django
- 393. Django Documentation, Release1.5.1 Finally, our calendar system contains interesting traps for computers: >>> import datetime >>> def one_year_before(value): # DON’T DO THAT! ... return value.replace(year=value.year - 1) >>> one_year_before(datetime.datetime(2012, 3, 1, 10, 0)) datetime.datetime(2011, 3, 1, 10, 0) >>> one_year_before(datetime.datetime(2012, 2, 29, 10, 0)) Traceback (most recent call last): ... ValueError: day is out of range for month (To implement this function, you must decide whether 2012-02-29 minus one year is 2011-02-28 or 2011-03-01, which depends on your business requirements.) 3. Should I install pytz? Yes. Django has a policy of not requiring external dependencies, and for this reason pytz is optional. However, it’s much safer to install it. As soon as you activate time zone support, Django needs a definition of the default time zone. When pytz is available, Django loads this definition from the tz database. This is the most accurate solution. Otherwise, it re-lies on the difference between local time and UTC, as reported by the operating system, to compute conversions. This is less reliable, especially around DST transitions. Furthermore, if you want to support users in more than one time zone, pytz is the reference for time zone definitions. Troubleshooting 1. My application crashes with TypeError: can’t compare offset-naive and offset-aware datetimes – what’s wrong? Let’s reproduce this error by comparing a naive and an aware datetime: >>> import datetime >>> from django.utils import timezone >>> naive = datetime.datetime.utcnow() >>> aware = naive.replace(tzinfo=timezone.utc) >>> naive == aware Traceback (most recent call last): ... TypeError: can’t compare offset-naive and offset-aware datetimes If you encounter this error, most likely your code is comparing these two things: • a datetime provided by Django – for instance, a value read from a form or a model field. Since you enabled time zone support, it’s aware. • a datetime generated by your code, which is naive (or you wouldn’t be reading this). Generally, the correct solution is to change your code to use an aware datetime instead. If you’re writing a pluggable application that’s expected to work independently of the value of USE_TZ, you may find django.utils.timezone.now() useful. This function returns the current date and time as a naive datetime when USE_TZ = False and as an aware datetime when USE_TZ = True. You can add or subtract datetime.timedelta as needed. 2. I see lots of RuntimeWarning: DateTimeField received a naive datetime (YYYY-MM-DD HH:MM:SS) while time zone support is active – is that bad? 3.14. Internationalization and localization 389
- 394. Django Documentation, Release1.5.1 When time zone support is enabled, the database layer expects to receive only aware datetimes from your code. This warning occurs when it receives a naive datetime. This indicates that you haven’t finished porting your code for time zone support. Please refer to the migration guide for tips on this process. In the meantime, for backwards compatibility, the datetime is considered to be in the default time zone, which is generally what you expect. 3. now.date() is yesterday! (or tomorrow) If you’ve always used naive datetimes, you probably believe that you can convert a datetime to a date by calling its date() method. You also consider that a date is a lot like a datetime, except that it’s less accurate. None of this is true in a time zone aware environment: >>> import datetime >>> import pytz >>> paris_tz = pytz.timezone("Europe/Paris") >>> new_york_tz = pytz.timezone("America/New_York") >>> paris = paris_tz.localize(datetime.datetime(2012, 3, 3, 1, 30)) # This is the correct way to convert between time zones with pytz. >>> new_york = new_york_tz.normalize(paris.astimezone(new_york_tz)) >>> paris == new_york, paris.date() == new_york.date() (True, False) >>> paris - new_york, paris.date() - new_york.date() (datetime.timedelta(0), datetime.timedelta(1)) >>> paris datetime.datetime(2012, 3, 3, 1, 30, tzinfo=<DstTzInfo ’Europe/Paris’ CET+1:00:00 STD>) >>> new_york datetime.datetime(2012, 3, 2, 19, 30, tzinfo=<DstTzInfo ’America/New_York’ EST-1 day, 19:00:00 STD>) As this example shows, the same datetime has a different date, depending on the time zone in which it is represented. But the real problem is more fundamental. A datetime represents a point in time. It’s absolute: it doesn’t depend on anything. On the contrary, a date is a calendaring concept. It’s a period of time whose bounds depend on the time zone in which the date is considered. As you can see, these two concepts are fundamentally different, and converting a datetime to a date isn’t a deterministic operation. What does this mean in practice? Generally, you should avoid converting a datetime to date. For instance, you can use the date template filter to only show the date part of a datetime. This filter will convert the datetime into the current time zone before formatting it, ensuring the results appear correctly. If you really need to do the conversion yourself, you must ensure the datetime is converted to the appropriate time zone first. Usually, this will be the current timezone: >>> from django.utils import timezone >>> timezone.activate(pytz.timezone("Asia/Singapore")) # For this example, we just set the time zone to Singapore, but here’s how # you would obtain the current time zone in the general case. >>> current_tz = timezone.get_current_timezone() # Again, this is the correct way to convert between time zones with pytz. >>> local = current_tz.normalize(paris.astimezone(current_tz)) >>> local datetime.datetime(2012, 3, 3, 8, 30, tzinfo=<DstTzInfo ’Asia/Singapore’ SGT+8:00:00 STD>) >>> local.date() datetime.date(2012, 3, 3) 390 Chapter 3. Using Django
- 395. Django Documentation, Release1.5.1 Usage 1. I have a string "2012-02-21 10:28:45" and I know it’s in the "Europe/Helsinki" time zone. How do I turn that into an aware datetime? This is exactly what pytz is for. >>> from django.utils.dateparse import parse_datetime >>> naive = parse_datetime("2012-02-21 10:28:45") >>> import pytz >>> pytz.timezone("Europe/Helsinki").localize(naive, is_dst=None) datetime.datetime(2012, 2, 21, 10, 28, 45, tzinfo=<DstTzInfo ’Europe/Helsinki’ EET+2:00:00 STD>) Note that localize is a pytz extension to the tzinfo API. Also, you may want to catch pytz.InvalidTimeError. The documentation of pytz contains more examples. You should review it before attempting to manipulate aware datetimes. 2. How can I obtain the local time in the current time zone? Well, the first question is, do you really need to? You should only use local time when you’re interacting with humans, and the template layer provides filters and tags to convert datetimes to the time zone of your choice. Furthermore, Python knows how to compare aware datetimes, taking into account UTC offsets when necessary. It’s much easier (and possibly faster) to write all your model and view code in UTC. So, in most circumstances, the datetime in UTC returned by django.utils.timezone.now() will be sufficient. For the sake of completeness, though, if you really want the local time in the current time zone, here’s how you can obtain it: >>> from django.utils import timezone >>> timezone.localtime(timezone.now()) datetime.datetime(2012, 3, 3, 20, 10, 53, 873365, tzinfo=<DstTzInfo ’Europe/Paris’ CET+1:00:00 STD>) In this example, pytz is installed and the current time zone is "Europe/Paris". 3. How can I see all available time zones? pytz provides helpers, including a list of current time zones and a list of all available time zones – some of which are only of historical interest. 3.14.4 Overview The goal of internationalization and localization is to allow a single Web application to offer its content in languages and formats tailored to the audience. Django has full support for translation of text, formatting of dates, times and numbers, and time zones. Essentially, Django does two things: • It allows developers and template authors to specify which parts of their apps should be translated or formatted for local languages and cultures. • It uses these hooks to localize Web apps for particular users according to their preferences. Obviously, translation depends on the target language, and formatting usually depends on the target country. These informations are provided by browsers in the Accept-Language header. However, the time zone isn’t readily available. 3.14. Internationalization and localization 391
- 396. Django Documentation, Release1.5.1 3.14.5 Definitions The words “internationalization” and “localization” often cause confusion; here’s a simplified definition: internationalization Preparing the software for localization. Usually done by developers. localization Writing the translations and local formats. Usually done by translators. More details can be found in the W3C Web Internationalization FAQ, the Wikipedia article or the GNU gettext docu-mentation. Warning: Translation and formatting are controlled by USE_I18N and USE_L10N settings respectively. How-ever, both features involve internationalization and localization. The names of the settings are an unfortunate result of Django’s history. Here are some other terms that will help us to handle a common language: locale name A locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC. Examples: it, de_AT, es, pt_BR. The language part is always is lower case and the country part in upper case. The separator is an underscore. language code Represents the name of a language. Browsers send the names of the languages they accept in the Accept-Language HTTP header using this format. Examples: it, de-at, es, pt-br. Both the language and the country parts are in lower case. The separator is a dash. message file A message file is a plain-text file, representing a single language, that contains all available translation strings and how they should be represented in the given language. Message files have a .po file extension. translation string A literal that can be translated. format file A format file is a Python module that defines the data formats for a given locale. 3.15 Logging 3.15.1 A quick logging primer Django uses Python’s builtin logging module to perform system logging. The usage of this module is discussed in detail in Python’s own documentation. However, if you’ve never used Python’s logging framework (or even if you have), here’s a quick primer. The cast of players A Python logging configuration consists of four parts: • Loggers • Handlers • Filters • Formatters Loggers A logger is the entry point into the logging system. Each logger is a named bucket to which messages can be written for processing. 392 Chapter 3. Using Django
- 397. Django Documentation, Release1.5.1 A logger is configured to have a log level. This log level describes the severity of the messages that the logger will handle. Python defines the following log levels: • DEBUG: Low level system information for debugging purposes • INFO: General system information • WARNING: Information describing a minor problem that has occurred. • ERROR: Information describing a major problem that has occurred. • CRITICAL: Information describing a critical problem that has occurred. Each message that is written to the logger is a Log Record. Each log record also has a log level indicating the severity of that specific message. A log record can also contain useful metadata that describes the event that is being logged. This can include details such as a stack trace or an error code. When a message is given to the logger, the log level of the message is compared to the log level of the logger. If the log level of the message meets or exceeds the log level of the logger itself, the message will undergo further processing. If it doesn’t, the message will be ignored. Once a logger has determined that a message needs to be processed, it is passed to a Handler. Handlers The handler is the engine that determines what happens to each message in a logger. It describes a particular logging behavior, such as writing a message to the screen, to a file, or to a network socket. Like loggers, handlers also have a log level. If the log level of a log record doesn’t meet or exceed the level of the handler, the handler will ignore the message. A logger can have multiple handlers, and each handler can have a different log level. In this way, it is possible to provide different forms of notification depending on the importance of a message. For example, you could install one handler that forwards ERROR and CRITICAL messages to a paging service, while a second handler logs all messages (including ERROR and CRITICAL messages) to a file for later analysis. Filters A filter is used to provide additional control over which log records are passed from logger to handler. By default, any log message that meets log level requirements will be handled. However, by installing a filter, you can place additional criteria on the logging process. For example, you could install a filter that only allows ERROR messages from a particular source to be emitted. Filters can also be used to modify the logging record prior to being emitted. For example, you could write a filter that downgrades ERROR log records to WARNING records if a particular set of criteria are met. Filters can be installed on loggers or on handlers; multiple filters can be used in a chain to perform multiple filtering actions. Formatters Ultimately, a log record needs to be rendered as text. Formatters describe the exact format of that text. A formatter usually consists of a Python formatting string; however, you can also write custom formatters to implement specific formatting behavior. 3.15. Logging 393
- 398. Django Documentation, Release1.5.1 3.15.2 Using logging Once you have configured your loggers, handlers, filters and formatters, you need to place logging calls into your code. Using the logging framework is very simple. Here’s an example: # import the logging library import logging # Get an instance of a logger logger = logging.getLogger(__name__) def my_view(request, arg1, arg): ... if bad_mojo: # Log an error message logger.error(’Something went wrong!’) And that’s it! Every time the bad_mojo condition is activated, an error log record will be written. Naming loggers The call to logging.getLogger() obtains (creating, if necessary) an instance of a logger. The logger instance is identified by a name. This name is used to identify the logger for configuration purposes. By convention, the logger name is usually __name__, the name of the python module that contains the logger. This allows you to filter and handle logging calls on a per-module basis. However, if you have some other way of organizing your logging messages, you can provide any dot-separated name to identify your logger: # Get an instance of a specific named logger logger = logging.getLogger(’project.interesting.stuff’) The dotted paths of logger names define a hierarchy. The project.interesting logger is considered to be a parent of the project.interesting.stuff logger; the project logger is a parent of the project.interesting logger. Why is the hierarchy important? Well, because loggers can be set to propagate their logging calls to their parents. In this way, you can define a single set of handlers at the root of a logger tree, and capture all logging calls in the subtree of loggers. A logging handler defined in the project namespace will catch all logging messages issued on the project.interesting and project.interesting.stuff loggers. This propagation can be controlled on a per-logger basis. If you don’t want a particular logger to propagate to it’s parents, you can turn off this behavior. Making logging calls The logger instance contains an entry method for each of the default log levels: • logger.critical() • logger.error() • logger.warning() • logger.info() • logger.debug() There are two other logging calls available: • logger.log(): Manually emits a logging message with a specific log level. 394 Chapter 3. Using Django
- 399. Django Documentation, Release1.5.1 • logger.exception(): Creates an ERROR level logging message wrapping the current exception stack frame. 3.15.3 Configuring logging Of course, it isn’t enough to just put logging calls into your code. You also need to configure the loggers, handlers, filters and formatters to ensure that logging output is output in a useful way. Python’s logging library provides several techniques to configure logging, ranging from a programmatic interface to configuration files. By default, Django uses the dictConfig format. Note: logging.dictConfig is a builtin library in Python 2.7. In order to make this library available for users of earlier Python versions, Django includes a copy as part of django.utils.log. If you have Python 2.7 or later, the system native library will be used; if you have Python 2.6, Django’s copy will be used. In order to configure logging, you use LOGGING to define a dictionary of logging settings. These settings describes the loggers, handlers, filters and formatters that you want in your logging setup, and the log levels and other properties that you want those components to have. Prior to Django 1.5, the LOGGING setting overwrote the default Django logging configuration. From Django 1.5 forward, the project’s logging configuration is merged with Django’s defaults, hence you can decide if you want to add to, or replace the existing configuration. To completely override the default configuration, set the disable_existing_loggers key to True in the LOGGING dictConfig. Alternatively you can redefine some or all of the loggers. Logging is configured as soon as settings have been loaded (either manually using configure() or when at least one setting is accessed). Since the loading of settings is one of the first things that Django does, you can be certain that loggers are always ready for use in your project code. An example The full documentation for dictConfig format is the best source of information about logging configuration dictionar-ies. However, to give you a taste of what is possible, here is an example of a fairly complex logging setup, configured using logging.config.dictConfig(): LOGGING = { ’version’: 1, ’disable_existing_loggers’: True, ’formatters’: { ’verbose’: { ’format’: ’%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s’ }, ’simple’: { ’format’: ’%(levelname)s %(message)s’ }, }, ’filters’: { ’special’: { ’()’: ’project.logging.SpecialFilter’, ’foo’: ’bar’, } }, ’handlers’: { ’null’: { ’level’: ’DEBUG’, 3.15. Logging 395
- 400. Django Documentation, Release1.5.1 ’class’: ’django.utils.log.NullHandler’, }, ’console’:{ ’level’: ’DEBUG’, ’class’: ’logging.StreamHandler’, ’formatter’: ’simple’ }, ’mail_admins’: { ’level’: ’ERROR’, ’class’: ’django.utils.log.AdminEmailHandler’, ’filters’: [’special’] } }, ’loggers’: { ’django’: { ’handlers’: [’null’], ’propagate’: True, ’level’: ’INFO’, }, ’django.request’: { ’handlers’: [’mail_admins’], ’level’: ’ERROR’, ’propagate’: False, }, ’myproject.custom’: { ’handlers’: [’console’, ’mail_admins’], ’level’: ’INFO’, ’filters’: [’special’] } } } This logging configuration does the following things: • Identifies the configuration as being in ‘dictConfig version 1’ format. At present, this is the only dictConfig format version. • Disables all existing logging configurations. • Defines two formatters: – simple, that just outputs the log level name (e.g., DEBUG) and the log message. The format string is a normal Python formatting string describing the details that are to be output on each logging line. The full list of detail that can be output can be found in the formatter documentation. – verbose, that outputs the log level name, the log message, plus the time, process, thread and module that generate the log message. • Defines one filter – project.logging.SpecialFilter, using the alias special. If this filter re-quired additional arguments at time of construction, they can be provided as additional keys in the filter con-figuration dictionary. In this case, the argument foo will be given a value of bar when instantiating the SpecialFilter. • Defines three handlers: – null, a NullHandler, which will pass any DEBUG (or higher) message to /dev/null. – console, a StreamHandler, which will print any DEBUG (or higher) message to stderr. This handler uses the simple output format. 396 Chapter 3. Using Django
- 401. Django Documentation, Release1.5.1 – mail_admins, an AdminEmailHandler, which will email any ERROR (or higher) message to the site admins. This handler uses the special filter. • Configures three loggers: – django, which passes all messages at INFO or higher to the null handler. – django.request, which passes all ERROR messages to the mail_admins handler. In addition, this logger is marked to not propagate messages. This means that log messages written to django.request will not be handled by the django logger. – myproject.custom, which passes all messages at INFO or higher that also pass the special filter to two handlers – the console, and mail_admins. This means that all INFO level messages (or higher) will be printed to the console; ERROR and CRITICAL messages will also be output via email. Custom logging configuration If you don’t want to use Python’s dictConfig format to configure your logger, you can specify your own configuration scheme. The LOGGING_CONFIG setting defines the callable that will be used to configure Django’s loggers. By default, it points at Python’s logging.config.dictConfig() function. However, if you want to use a different configu-ration process, you can use any other callable that takes a single argument. The contents of LOGGING will be provided as the value of that argument when logging is configured. Disabling logging configuration If you don’t want to configure logging at all (or you want to manually configure logging using your own approach), you can set LOGGING_CONFIG to None. This will disable the configuration process. Note: Setting LOGGING_CONFIG to None only means that the configuration process is disabled, not logging itself. If you disable the configuration process, Django will still make logging calls, falling back to whatever default logging behavior is defined. 3.15.4 Django’s logging extensions Django provides a number of utilities to handle the unique requirements of logging in Web server environment. Loggers Django provides three built-in loggers. django django is the catch-all logger. No messages are posted directly to this logger. 3.15. Logging 397
- 402. Django Documentation, Release1.5.1 django.request Log messages related to the handling of requests. 5XX responses are raised as ERROR messages; 4XX responses are raised as WARNING messages. Messages to this logger have the following extra context: • status_code: The HTTP response code associated with the request. • request: The request object that generated the logging message. django.db.backends Messages relating to the interaction of code with the database. For example, every SQL statement executed by a request is logged at the DEBUG level to this logger. Messages to this logger have the following extra context: • duration: The time taken to execute the SQL statement. • sql: The SQL statement that was executed. • params: The parameters that were used in the SQL call. For performance reasons, SQL logging is only enabled when settings.DEBUG is set to True, regardless of the logging level or handlers that are installed. Handlers Django provides one log handler in addition to those provided by the Python logging module. class AdminEmailHandler([include_html=False ]) This handler sends an email to the site admins for each log message it receives. If the log record contains a request attribute, the full details of the request will be included in the email. If the log record contains stack trace information, that stack trace will be included in the email. The include_html argument of AdminEmailHandler is used to control whether the traceback email includes an HTML attachment containing the full content of the debug Web page that would have been pro-duced if DEBUG were True. To set this value in your configuration, include it in the handler definition for django.utils.log.AdminEmailHandler, like this: ’handlers’: { ’mail_admins’: { ’level’: ’ERROR’, ’class’: ’django.utils.log.AdminEmailHandler’, ’include_html’: True, } }, Note that this HTML version of the email contains a full traceback, with names and values of local variables at each level of the stack, plus the values of your Django settings. This information is potentially very sensitive, and you may not want to send it over email. Consider using something such as Sentry to get the best of both worlds – the rich information of full tracebacks plus the security of not sending the information over email. You may also explicitly designate certain sensitive information to be filtered out of error reports – learn more on Filtering error reports. 398 Chapter 3. Using Django
- 403. Django Documentation, Release1.5.1 Filters Django provides two log filters in addition to those provided by the Python logging module. class CallbackFilter(callback) New in version 1.4. This filter accepts a callback function (which should accept a single argument, the record to be logged), and calls it for each record that passes through the filter. Handling of that record will not proceed if the callback returns False. For instance, to filter out UnreadablePostError (raised when a user cancels an upload) from the admin emails, you would create a filter function: from django.http import UnreadablePostError def skip_unreadable_post(record): if record.exc_info: exc_type, exc_value = record.exc_info[:2] if isinstance(exc_value, UnreadablePostError): return False return True and then add it to your logging config: ’filters’: { ’skip_unreadable_posts’: { ’()’: ’django.utils.log.CallbackFilter’, ’callback’: skip_unreadable_post, } }, ’handlers’: { ’mail_admins’: { ’level’: ’ERROR’, ’filters’: [’skip_unreadable_posts’], ’class’: ’django.utils.log.AdminEmailHandler’ } }, class RequireDebugFalse New in version 1.4. This filter will only pass on records when settings.DEBUG is False. This filter is used as follows in the default LOGGING configuration to ensure that the AdminEmailHandler only sends error emails to admins when DEBUG is False: ’filters’: { ’require_debug_false’: { ’()’: ’django.utils.log.RequireDebugFalse’, } }, ’handlers’: { ’mail_admins’: { ’level’: ’ERROR’, ’filters’: [’require_debug_false’], ’class’: ’django.utils.log.AdminEmailHandler’ } }, class RequireDebugTrue New in version 1.5. This filter is similar to RequireDebugFalse, except that records are passed only when DEBUG is True. 3.15. Logging 399
- 404. Django Documentation, Release1.5.1 3.15.5 Django’s default logging configuration By default, Django configures the django.request logger so that all messages with ERROR or CRITICAL level are sent to AdminEmailHandler, as long as the DEBUG setting is set to False. All messages reaching the django catch-all logger when DEBUG is True are sent to the console. They are simply discarded (sent to NullHandler) when DEBUG is False. Changed in version 1.5. See also Configuring logging to learn how you can complement or replace this default logging configuration. 3.16 Pagination Django provides a few classes that help you manage paginated data – that is, data that’s split across several pages, with “Previous/Next” links. These classes live in django/core/paginator.py. 3.16.1 Example Give Paginator a list of objects, plus the number of items you’d like to have on each page, and it gives you methods for accessing the items for each page: >>> from django.core.paginator import Paginator >>> objects = [’john’, ’paul’, ’george’, ’ringo’] >>> p = Paginator(objects, 2) >>> p.count 4 >>> p.num_pages 2 >>> p.page_range [1, 2] >>> page1 = p.page(1) >>> page1 <Page 1 of 2> >>> page1.object_list [’john’, ’paul’] >>> page2 = p.page(2) >>> page2.object_list [’george’, ’ringo’] >>> page2.has_next() False >>> page2.has_previous() True >>> page2.has_other_pages() True >>> page2.next_page_number() Traceback (most recent call last): ... EmptyPage: That page contains no results >>> page2.previous_page_number() 1 >>> page2.start_index() # The 1-based index of the first item on this page 3 >>> page2.end_index() # The 1-based index of the last item on this page 4 400 Chapter 3. Using Django
- 405. Django Documentation, Release1.5.1 >>> p.page(0) Traceback (most recent call last): ... EmptyPage: That page number is less than 1 >>> p.page(3) Traceback (most recent call last): ... EmptyPage: That page contains no results Note: Note that you can give Paginator a list/tuple, a Django QuerySet, or any other object with a count() or __len__() method. When determining the number of objects contained in the passed object, Paginator will first try calling count(), then fallback to using len() if the passed object has no count() method. This allows objects such as Django’s QuerySet to use a more efficient count() method when available. 3.16.2 Using Paginator in a view Here’s a slightly more complex example using Paginator in a view to paginate a queryset. We give both the view and the accompanying template to show how you can display the results. This example assumes you have a Contacts model that has already been imported. The view function looks like this: from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger def listing(request): contact_list = Contacts.objects.all() paginator = Paginator(contact_list, 25) # Show 25 contacts per page page = request.GET.get(’page’) try: contacts = paginator.page(page) except PageNotAnInteger: # If page is not an integer, deliver first page. contacts = paginator.page(1) except EmptyPage: # If page is out of range (e.g. 9999), deliver last page of results. contacts = paginator.page(paginator.num_pages) return render_to_response(’list.html’, {"contacts": contacts}) In the template list.html, you’ll want to include navigation between pages along with any interesting information from the objects themselves: {% for contact in contacts %} {# Each "contact" is a Contact model object. #} {{ contact.full_name|upper }}<br /> ... {% endfor %} <div class="pagination"> <span class="step-links"> {% if contacts.has_previous %} <a href="?page={{ contacts.previous_page_number }}">previous</a> {% endif %} 3.16. Pagination 401
- 406. Django Documentation, Release1.5.1 <span class="current"> Page {{ contacts.number }} of {{ contacts.paginator.num_pages }}. </span> {% if contacts.has_next %} <a href="?page={{ contacts.next_page_number }}">next</a> {% endif %} </span> </div> Changed in version 1.4: Previously, you would need to use {% for contact in contacts.object_list %}, since the Page object was not iterable. 3.16.3 Paginator objects The Paginator class has this constructor: class Paginator(object_list, per_page, orphans=0, allow_empty_first_page=True) Required arguments object_list A list, tuple, Django QuerySet, or other sliceable object with a count() or __len__() method. per_page The maximum number of items to include on a page, not including orphans (see the orphans optional argument below). Optional arguments orphans The minimum number of items allowed on the last page, defaults to zero. Use this when you don’t want to have a last page with very few items. If the last page would normally have a number of items less than or equal to orphans, then those items will be added to the previous page (which becomes the last page) instead of leaving the items on a page by themselves. For example, with 23 items, per_page=10, and orphans=3, there will be two pages; the first page with 10 items and the second (and last) page with 13 items. allow_empty_first_page Whether or not the first page is allowed to be empty. If False and object_list is empty, then an EmptyPage error will be raised. Methods Paginator.page(number) Returns a Page object with the given 1-based index. Raises InvalidPage if the given page number doesn’t exist. Attributes Paginator.count The total number of objects, across all pages. Note: When determining the number of objects contained in object_list, Paginator will first try calling object_list.count(). If object_list has no count() method, then Paginator will fallback to using len(object_list). This allows objects, such as Django’s QuerySet, to use a more efficient count() method when available. 402 Chapter 3. Using Django
- 407. Django Documentation, Release1.5.1 Paginator.num_pages The total number of pages. Paginator.page_range A 1-based range of page numbers, e.g., [1, 2, 3, 4]. 3.16.4 InvalidPage exceptions exception InvalidPage A base class for exceptions raised when a paginator is passed an invalid page number. The Paginator.page() method raises an exception if the requested page is invalid (i.e., not an integer) or contains no objects. Generally, it’s enough to trap the InvalidPage exception, but if you’d like more granularity, you can trap either of the following exceptions: exception PageNotAnInteger Raised when page() is given a value that isn’t an integer. exception EmptyPage Raised when page() is given a valid value but no objects exist on that page. Both of the exceptions are subclasses of InvalidPage, so you can handle them both with a simple except InvalidPage. 3.16.5 Page objects You usually won’t construct Page objects by hand – you’ll get them using Paginator.page(). class Page(object_list, number, paginator) New in version 1.4: A page acts like a sequence of Page.object_list when using len() or iterating it directly. Methods Page.has_next() Returns True if there’s a next page. Page.has_previous() Returns True if there’s a previous page. Page.has_other_pages() Returns True if there’s a next or previous page. Page.next_page_number() Returns the next page number. Changed in version 1.5. Raises InvalidPage if next page doesn’t exist. Page.previous_page_number() Returns the previous page number. Changed in version 1.5. Raises InvalidPage if previous page doesn’t exist. Page.start_index() Returns the 1-based index of the first object on the page, relative to all of the objects in the paginator’s list. For example, when paginating a list of 5 objects with 2 objects per page, the second page’s start_index() would return 3. 3.16. Pagination 403
- 408. Django Documentation, Release1.5.1 Page.end_index() Returns the 1-based index of the last object on the page, relative to all of the objects in the paginator’s list. For example, when paginating a list of 5 objects with 2 objects per page, the second page’s end_index() would return 4. Attributes Page.object_list The list of objects on this page. Page.number The 1-based page number for this page. Page.paginator The associated Paginator object. 3.17 Porting to Python 3 Django 1.5 is the first version of Django to support Python 3. The same code runs both on Python 2 ( 2.6.5) and Python 3 ( 3.2), thanks to the six compatibility layer. This document is primarily targeted at authors of pluggable application who want to support both Python 2 and 3. It also describes guidelines that apply to Django’s code. 3.17.1 Philosophy This document assumes that you are familiar with the changes between Python 2 and Python 3. If you aren’t, read Python’s official porting guide first. Refreshing your knowledge of unicode handling on Python 2 and 3 will help; the Pragmatic Unicode presentation is a good resource. Django uses the Python 2/3 Compatible Source strategy. Of course, you’re free to chose another strategy for your own code, especially if you don’t need to stay compatible with Python 2. But authors of pluggable applications are encouraged to use the same porting strategy as Django itself. Writing compatible code is much easier if you target Python 2.6. Django 1.5 introduces compatibility tools such as django.utils.six. For convenience, forwards-compatible aliases were introduced in Django 1.4.2. If your application takes advantage of these tools, it will require Django 1.4.2. Obviously, writing compatible source code adds some overhead, and that can cause frustration. Django’s developers have found that attempting to write Python 3 code that’s compatible with Python 2 is much more rewarding than the opposite. Not only does that make your code more future-proof, but Python 3’s advantages (like the saner string handling) start shining quickly. Dealing with Python 2 becomes a backwards compatibility requirement, and we as developers are used to dealing with such constraints. Porting tools provided by Django are inspired by this philosophy, and it’s reflected throughout this guide. 3.17.2 Porting tips Unicode literals This step consists in: 404 Chapter 3. Using Django
- 409. Django Documentation, Release1.5.1 • Adding from __future__ import unicode_literals at the top of your Python modules – it’s best to put it in each and every module, otherwise you’ll keep checking the top of your files to see which mode is in effect; • Removing the u prefix before unicode strings; • Adding a b prefix before bytestrings. Performing these changes systematically guarantees backwards compatibility. However, Django applications generally don’t need bytestrings, since Django only exposes unicode interfaces to the programmer. Python 3 discourages using bytestrings, except for binary data or byte-oriented interfaces. Python 2 makes bytestrings and unicode strings effectively interchangeable, as long as they only contain ASCII data. Take advantage of this to use unicode strings wherever possible and avoid the b prefixes. Note: Python 2’s u prefix is a syntax error in Python 3.2 but it will be allowed again in Python 3.3 thanks to PEP 414. Thus, this transformation is optional if you target Python 3.3. It’s still recommended, per the “write Python 3 code” philosophy. String handling Python 2’s unicode() type was renamed str() in Python 3, str() was renamed bytes(), and basestring() disappeared. six provides tools to deal with these changes. Django also contains several string related classes and functions in the django.utils.encoding and django.utils.safestring modules. Their names used the words str, which doesn’t mean the same thing in Python 2 and Python 3, and unicode, which doesn’t exist in Python 3. In order to avoid ambiguity and confusion these concepts were renamed bytes and text. Here are the name changes in django.utils.encoding: Old name New name smart_str smart_bytes smart_unicode smart_text force_unicode force_text For backwards compatibility, the old names still work on Python 2. Under Python 3, smart_str is an alias for smart_text. For forwards compatibility, the new names work as of Django 1.4.2. Note: django.utils.encoding was deeply refactored in Django 1.5 to provide a more consistent API. Check its documentation for more information. django.utils.safestring is mostly used via the mark_safe() and mark_for_escaping() functions, which didn’t change. In case you’re using the internals, here are the name changes: Old name New name EscapeString EscapeBytes EscapeUnicode EscapeText SafeString SafeBytes SafeUnicode SafeText For backwards compatibility, the old names still work on Python 2. Under Python 3, EscapeString and SafeString are aliases for EscapeText and SafeText respectively. For forwards compatibility, the new names work as of Django 1.4.2. 3.17. Porting to Python 3 405
- 410. Django Documentation, Release1.5.1 __str__() and __unicode__() methods In Python 2, the object model specifies __str__() and __unicode__() methods. If these methods exist, they must return str (bytes) and unicode (text) respectively. The print statement and the str() built-in call __str__() to determine the human-readable representation of an object. The unicode() built-in calls __unicode__() if it exists, and otherwise falls back to __str__() and decodes the result with the system encoding. Conversely, the Model base class automatically derives __str__() from __unicode__() by encoding to UTF-8. In Python 3, there’s simply __str__(), which must return str (text). (It is also possible to define __bytes__(), but Django application have little use for that method, because they hardly ever deal with bytes.) Django provides a simple way to define __str__() and __unicode__() methods that work on Python 2 and 3: you must define a __str__() method returning text and to apply the python_2_unicode_compatible() decorator. On Python 3, the decorator is a no-op. On Python 2, it defines appropriate __unicode__() and __str__() methods (replacing the original __str__() method in the process). Here’s an example: from __future__ import unicode_literals from django.utils.encoding import python_2_unicode_compatible @python_2_unicode_compatible class MyClass(object): def __str__(self): return Instance of my class This technique is the best match for Django’s porting philosophy. For forwards compatibility, this decorator is available as of Django 1.4.2. Finally, note that __repr__() must return a str on all versions of Python. dict and dict-like classes dict.keys(), dict.items() and dict.values() return lists in Python 2 and iterators in Python 3. QueryDict and the dict-like classes defined in django.utils.datastructures behave likewise in Python 3. six provides compatibility functions to work around this change: iterkeys(), iteritems(), and itervalues(). It also contains an undocumented iterlists function that works well for django.utils.datastructures.MultiValueDict and its subclasses. HttpRequest and HttpResponse objects According to PEP 3333: • headers are always str objects, • input and output streams are always bytes objects. Specifically, HttpResponse.content contains bytes, which may become an issue if you compare it with a str in your tests. The preferred solution is to rely on assertContains() and assertNotContains(). These methods accept a response and a unicode string as arguments. 406 Chapter 3. Using Django
- 411. Django Documentation, Release1.5.1 3.17.3 Coding guidelines The following guidelines are enforced in Django’s source code. They’re also recommended for third-party application who follow the same porting strategy. Syntax requirements Unicode In Python 3, all strings are considered Unicode by default. The unicode type from Python 2 is called str in Python 3, and str becomes bytes. You mustn’t use the u prefix before a unicode string literal because it’s a syntax error in Python 3.2. You must prefix byte strings with b. In order to enable the same behavior in Python 2, every module must import unicode_literals from __future__: from __future__ import unicode_literals my_string = This is an unicode literal my_bytestring = bThis is a bytestring If you need a byte string literal under Python 2 and a unicode string literal under Python 3, use the str() builtin: str(’my string’) In Python 3, there aren’t any automatic conversions between str and bytes, and the codecs module became more strict. str.encode() always returns bytes, and bytes.decode always returns str. As a consequence, the following pattern is sometimes necessary: value = value.encode(’ascii’, ’ignore’).decode(’ascii’) Be cautious if you have to index bytestrings. Exceptions When you capture exceptions, use the as keyword: try: ... except MyException as exc: ... This older syntax was removed in Python 3: try: ... except MyException, exc: # Don’t do that! ... The syntax to reraise an exception with a different traceback also changed. Use six.reraise(). Magic methods Use the patterns below to handle magic methods renamed in Python 3. 3.17. Porting to Python 3 407
- 412. Django Documentation, Release1.5.1 Iterators class MyIterator(six.Iterator): def __iter__(self): return self # implement some logic here def __next__(self): raise StopIteration # implement some logic here Boolean evaluation class MyBoolean(object): def __bool__(self): return True # implement some logic here def __nonzero__(self): # Python 2 compatibility return type(self).__bool__(self) Division class MyDivisible(object): def __truediv__(self, other): return self / other # implement some logic here def __div__(self, other): # Python 2 compatibility return type(self).__truediv__(self, other) def __itruediv__(self, other): return self // other # implement some logic here def __idiv__(self, other): # Python 2 compatibility return type(self).__itruediv__(self, other) Writing compatible code with six six is the canonical compatibility library for supporting Python 2 and 3 in a single codebase. Read its documentation! six is bundled with Django as of version 1.4.2. You can import it as django.utils.six. Here are the most common changes required to write compatible code. String handling The basestring and unicode types were removed in Python 3, and the meaning of str changed. To test these types, use the following idioms: isinstance(myvalue, six.string_types) # replacement for basestring isinstance(myvalue, six.text_type) # replacement for unicode isinstance(myvalue, bytes) # replacement for str Python 2.6 provides bytes as an alias for str, so you don’t need six.binary_type. 408 Chapter 3. Using Django
- 413. Django Documentation, Release1.5.1 long The long type no longer exists in Python 3. 1L is a syntax error. Use six.integer_types check if a value is an integer or a long: isinstance(myvalue, six.integer_types) # replacement for (int, long) xrange Import six.moves.xrange wherever you use xrange. Moved modules Some modules were renamed in Python 3. The django.utils.six.moves module provides a compatible loca-tion to import them. The urllib, urllib2 and urlparse modules were reworked in depth and django.utils.six.moves doesn’t handle them. Django explicitly tries both locations, as follows: try: from urllib.parse import urlparse, urlunparse except ImportError: # Python 2 from urlparse import urlparse, urlunparse PY3 If you need different code in Python 2 and Python 3, check six.PY3: if six.PY3: # do stuff Python 3-wise else: # do stuff Python 2-wise This is a last resort solution when six doesn’t provide an appropriate function. Customizations of six The version of six bundled with Django includes a few extras. assertRaisesRegex(testcase, *args, **kwargs) This replaces testcase.assertRaisesRegexp on Python 2, and testcase.assertRaisesRegex on Python 3. assertRaisesRegexp still exists in current Python3 versions, but issues a warning. In addition to six’ defaults moves, Django’s version provides thread as _thread and dummy_thread as _dummy_thread. 3.18 Security in Django This document is an overview of Django’s security features. It includes advice on securing a Django-powered site. 3.18. Security in Django 409
- 414. Django Documentation, Release1.5.1 3.18.1 Cross site scripting (XSS) protection XSS attacks allow a user to inject client side scripts into the browsers of other users. This is usually achieved by storing the malicious scripts in the database where it will be retrieved and displayed to other users, or by getting users to click a link which will cause the attacker’s JavaScript to be executed by the user’s browser. However, XSS attacks can originate from any untrusted source of data, such as cookies or Web services, whenever the data is not sufficiently sanitized before including in a page. Using Django templates protects you against the majority of XSS attacks. However, it is important to understand what protections it provides and its limitations. Django templates escape specific characters which are particularly dangerous to HTML. While this protects users from most malicious input, it is not entirely foolproof. For example, it will not protect the following: style class={{ var }}.../style If var is set to ’class1 onmouseover=javascript:func()’, this can result in unauthorized JavaScript execution, depending on how the browser renders imperfect HTML. It is also important to be particularly careful when using is_safe with custom template tags, the safe template tag, mark_safe, and when autoescape is turned off. In addition, if you are using the template system to output something other than HTML, there may be entirely separate characters and words which require escaping. You should also be very careful when storing HTML in the database, especially when that HTML is retrieved and displayed. Markup library If you use django.contrib.markup, you need to ensure that the filters are only used on trusted input, or that you have correctly configured them to ensure they do not allow raw HTML output. See the documentation of that module for more information. 3.18.2 Cross site request forgery (CSRF) protection CSRF attacks allow a malicious user to execute actions using the credentials of another user without that user’s knowl-edge or consent. Django has built-in protection against most types of CSRF attacks, providing you have enabled and used it where appropriate. However, as with any mitigation technique, there are limitations. For example, it is possible to disable the CSRF module globally or for particular views. You should only do this if you know what you are doing. There are other limitations if your site has subdomains that are outside of your control. CSRF protection works by checking for a nonce in each POST request. This ensures that a malicious user cannot simply “replay” a form POST to your Web site and have another logged in user unwittingly submit that form. The malicious user would have to know the nonce, which is user specific (using a cookie). When deployed with HTTPS, CsrfViewMiddleware will check that the HTTP referer header is set to a URL on the same origin (including subdomain and port). Because HTTPS provides additional security, it is imperative to ensure connections use HTTPS where it is available by forwarding insecure connection requests and using HSTS for supported browsers. Be very careful with marking views with the csrf_exempt decorator unless it is absolutely necessary. 410 Chapter 3. Using Django
- 415. Django Documentation, Release1.5.1 3.18.3 SQL injection protection SQL injection is a type of attack where a malicious user is able to execute arbitrary SQL code on a database. This can result in records being deleted or data leakage. By using Django’s querysets, the resulting SQL will be properly escaped by the underlying database driver. However, Django also gives developers power to write raw queries or execute custom sql. These capabilities should be used sparingly and you should always be careful to properly escape any parameters that the user can control. In addition, you should exercise caution when using extra(). 3.18.4 Clickjacking protection Clickjacking is a type of attack where a malicious site wraps another site in a frame. This attack can result in an unsuspecting user being tricked into performing unintended actions on the target site. Django contains clickjacking protection in the form of the X-Frame-Options middleware which in a support-ing browser can prevent a site from being rendered inside a frame. It is possible to disable the protection on a per view basis or to configure the exact header value sent. The middleware is strongly recommended for any site that does not need to have its pages wrapped in a frame by third party sites, or only needs to allow that for a small section of the site. 3.18.5 SSL/HTTPS It is always better for security, though not always practical in all cases, to deploy your site behind HTTPS. Without this, it is possible for malicious network users to sniff authentication credentials or any other information transferred between client and server, and in some cases – active network attackers – to alter data that is sent in either direction. If you want the protection that HTTPS provides, and have enabled it on your server, there are some additional steps you may need: • If necessary, set SECURE_PROXY_SSL_HEADER, ensuring that you have understood the warnings there thor-oughly. Failure to do this can result in CSRF vulnerabilities, and failure to do it correctly can also be dangerous! • Set up redirection so that requests over HTTP are redirected to HTTPS. This could be done using a custom middleware. Please note the caveats under SECURE_PROXY_SSL_HEADER. For the case of a reverse proxy, it may be easier or more secure to configure the main Web server to do the redirect to HTTPS. • Use ‘secure’ cookies. If a browser connects initially via HTTP, which is the default for most browsers, it is possible for ex-isting cookies to be leaked. For this reason, you should set your SESSION_COOKIE_SECURE and CSRF_COOKIE_SECURE settings to True. This instructs the browser to only send these cookies over HTTPS connections. Note that this will mean that sessions will not work over HTTP, and the CSRF protection will prevent any POST data being accepted over HTTP (which will be fine if you are redirecting all HTTP traffic to HTTPS). • Use HTTP Strict Transport Security (HSTS) HSTS is an HTTP header that informs a browser that all future connections to a particular site should always use HTTPS. Combined with redirecting requests over HTTP to HTTPS, this will ensure that connections always enjoy the added security of SSL provided one successful connection has occurred. HSTS is usually configured on the web server. 3.18. Security in Django 411
- 416. Django Documentation, Release1.5.1 3.18.6 Host header validation Django uses the Host header provided by the client to construct URLs in certain cases. While these values are sanitized to prevent Cross Site Scripting attacks, a fake Host value can be used for Cross-Site Request Forgery, cache poisoning attacks, and poisoning links in emails. Because even seemingly-secure webserver configurations are susceptible to fake Host headers, Django vali-dates Host headers against the ALLOWED_HOSTS setting in the django.http.HttpRequest.get_host() method. This validation only applies via get_host(); if your code accesses the Host header directly from request.META you are bypassing this security protection. For more details see the full ALLOWED_HOSTS documentation. Warning: Previous versions of this document recommended configuring your webserver to ensure it validates incoming HTTP Host headers. While this is still recommended, in many common webservers a configuration that seems to validate the Host header may not in fact do so. For instance, even if Apache is configured such that your Django site is served from a non-default virtual host with the ServerName set, it is still possible for an HTTP request to match this virtual host and supply a fake Host header. Thus, Django now requires that you set ALLOWED_HOSTS explicitly rather than relying on webserver configuration. Additionally, as of 1.3.1, Django requires you to explicitly enable support for the X-Forwarded-Host header (via the USE_X_FORWARDED_HOST setting) if your configuration requires it. 3.18.7 Additional security topics While Django provides good security protection out of the box, it is still important to properly deploy your application and take advantage of the security protection of the Web server, operating system and other components. • Make sure that your Python code is outside of the Web server’s root. This will ensure that your Python code is not accidentally served as plain text (or accidentally executed). • Take care with any user uploaded files. • Django does not throttle requests to authenticate users. To protect against brute-force attacks against the authen-tication system, you may consider deploying a Django plugin or Web server module to throttle these requests. • If your site accepts file uploads, it is strongly advised that you limit these uploads in your Web server configu-ration to a reasonable size in order to prevent denial of service (DOS) attacks. In Apache, this can be easily set using the LimitRequestBody directive. • Keep your SECRET_KEY a secret. • It is a good idea to limit the accessibility of your caching system and database using a firewall. 3.19 Serializing Django objects Django’s serialization framework provides a mechanism for “translating” Django models into other formats. Usually these other formats will be text-based and used for sending Django data over a wire, but it’s possible for a serializer to handle any format (text-based or not). See Also: If you just want to get some data from your tables into a serialized form, you could use the dumpdata management command. 412 Chapter 3. Using Django
- 417. Django Documentation, Release1.5.1 3.19.1 Serializing data At the highest level, serializing data is a very simple operation: from django.core import serializers data = serializers.serialize(xml, SomeModel.objects.all()) The arguments to the serialize function are the format to serialize the data to (see Serialization formats) and a QuerySet to serialize. (Actually, the second argument can be any iterator that yields Django model instances, but it’ll almost always be a QuerySet). django.core.serializers.get_serializer(format) You can also use a serializer object directly: XMLSerializer = serializers.get_serializer(xml) xml_serializer = XMLSerializer() xml_serializer.serialize(queryset) data = xml_serializer.getvalue() This is useful if you want to serialize data directly to a file-like object (which includes an HttpResponse): with open(file.xml, w) as out: xml_serializer.serialize(SomeModel.objects.all(), stream=out) Note: Calling get_serializer() with an unknown format will raise a django.core.serializers.SerializerDoesNotExist exception. Subset of fields If you only want a subset of fields to be serialized, you can specify a fields argument to the serializer: from django.core import serializers data = serializers.serialize(’xml’, SomeModel.objects.all(), fields=(’name’,’size’)) In this example, only the name and size attributes of each model will be serialized. Note: Depending on your model, you may find that it is not possible to deserialize a model that only serializes a subset of its fields. If a serialized object doesn’t specify all the fields that are required by a model, the deserializer will not be able to save deserialized instances. Inherited Models If you have a model that is defined using an abstract base class, you don’t have to do anything special to serialize that model. Just call the serializer on the object (or objects) that you want to serialize, and the output will be a complete representation of the serialized object. However, if you have a model that uses multi-table inheritance, you also need to serialize all of the base classes for the model. This is because only the fields that are locally defined on the model will be serialized. For example, consider the following models: class Place(models.Model): name = models.CharField(max_length=50) 3.19. Serializing Django objects 413
- 418. Django Documentation, Release1.5.1 class Restaurant(Place): serves_hot_dogs = models.BooleanField() If you only serialize the Restaurant model: data = serializers.serialize(’xml’, Restaurant.objects.all()) the fields on the serialized output will only contain the serves_hot_dogs attribute. The name attribute of the base class will be ignored. In order to fully serialize your Restaurant instances, you will need to serialize the Place models as well: all_objects = list(Restaurant.objects.all()) + list(Place.objects.all()) data = serializers.serialize(’xml’, all_objects) 3.19.2 Deserializing data Deserializing data is also a fairly simple operation: for obj in serializers.deserialize(xml, data): do_something_with(obj) As you can see, the deserialize function takes the same format argument as serialize, a string or stream of data, and returns an iterator. However, here it gets slightly complicated. The objects returned by the deserialize iterator aren’t simple Django objects. Instead, they are special DeserializedObject instances that wrap a created – but unsaved – object and any associated relationship data. Calling DeserializedObject.save() saves the object to the database. This ensures that deserializing is a non-destructive operation even if the data in your serialized representation doesn’t match what’s currently in the database. Usually, working with these DeserializedObject instances looks some-thing like: for deserialized_object in serializers.deserialize(xml, data): if object_should_be_saved(deserialized_object): deserialized_object.save() In other words, the usual use is to examine the deserialized objects to make sure that they are “appropriate” for saving before doing so. Of course, if you trust your data source you could just save the object and move on. The Django object itself can be inspected as deserialized_object.object. New in version 1.5. If fields in the serialized data do not exist on a model, a DeserializationError will be raised unless the ignorenonexistent argument is passed in as True: serializers.deserialize(xml, data, ignorenonexistent=True) 3.19.3 Serialization formats Django supports a number of serialization formats, some of which require you to install third-party Python modules: Identi-fier Information xml Serializes to and from a simple XML dialect. json Serializes to and from JSON. yaml Serializes to YAML (YAML Ain’t a Markup Language). This serializer is only available if PyYAML is installed. 414 Chapter 3. Using Django
- 419. Django Documentation, Release1.5.1 Notes for specific serialization formats json Be aware that not all Django output can be passed unmodified to json. In particular, lazy translation objects need a special encoder written for them. Something like this will work: import json from django.utils.functional import Promise from django.utils.encoding import force_text class LazyEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, Promise): return force_text(obj) return super(LazyEncoder, self).default(obj) 3.19.4 Natural keys The default serialization strategy for foreign keys and many-to-many relations is to serialize the value of the primary key(s) of the objects in the relation. This strategy works well for most objects, but it can cause difficulty in some circumstances. Consider the case of a list of objects that have a foreign key referencing ContentType. If you’re going to serialize an object that refers to a content type, then you need to have a way to refer to that content type to begin with. Since ContentType objects are automatically created by Django during the database synchronization process, the primary key of a given content type isn’t easy to predict; it will depend on how and when syncdb was executed. This is true for all models which automatically generate objects, notably including Permission, Group, and User. Warning: You should never include automatically generated objects in a fixture or other serialized data. By chance, the primary keys in the fixture may match those in the database and loading the fixture will have no effect. In the more likely case that they don’t match, the fixture loading will fail with an IntegrityError. There is also the matter of convenience. An integer id isn’t always the most convenient way to refer to an object; sometimes, a more natural reference would be helpful. It is for these reasons that Django provides natural keys. A natural key is a tuple of values that can be used to uniquely identify an object instance without using the primary key value. Deserialization of natural keys Consider the following two models: from django.db import models class Person(models.Model): first_name = models.CharField(max_length=100) last_name = models.CharField(max_length=100) birthdate = models.DateField() class Meta: unique_together = ((’first_name’, ’last_name’),) class Book(models.Model): 3.19. Serializing Django objects 415
- 420. Django Documentation, Release1.5.1 name = models.CharField(max_length=100) author = models.ForeignKey(Person) Ordinarily, serialized data for Book would use an integer to refer to the author. For example, in JSON, a Book might be serialized as: ... { pk: 1, model: store.book, fields: { name: Mostly Harmless, author: 42 } } ... This isn’t a particularly natural way to refer to an author. It requires that you know the primary key value for the author; it also requires that this primary key value is stable and predictable. However, if we add natural key handling to Person, the fixture becomes much more humane. To add natural key handling, you define a default Manager for Person with a get_by_natural_key() method. In the case of a Person, a good natural key might be the pair of first and last name: from django.db import models class PersonManager(models.Manager): def get_by_natural_key(self, first_name, last_name): return self.get(first_name=first_name, last_name=last_name) class Person(models.Model): objects = PersonManager() first_name = models.CharField(max_length=100) last_name = models.CharField(max_length=100) birthdate = models.DateField() class Meta: unique_together = ((’first_name’, ’last_name’),) Now books can use that natural key to refer to Person objects: ... { pk: 1, model: store.book, fields: { name: Mostly Harmless, author: [Douglas, Adams] } } ... When you try to load this serialized data, Django will use the get_by_natural_key() method to resolve [Douglas, Adams] into the primary key of an actual Person object. Note: Whatever fields you use for a natural key must be able to uniquely identify an object. This will usually mean that your model will have a uniqueness clause (either unique=True on a single field, or unique_together over multiple fields) for the field or fields in your natural key. However, uniqueness doesn’t need to be enforced at the 416 Chapter 3. Using Django
- 421. Django Documentation, Release1.5.1 database level. If you are certain that a set of fields will be effectively unique, you can still use those fields as a natural key. Serialization of natural keys So how do you get Django to emit a natural key when serializing an object? Firstly, you need to add another method – this time to the model itself: class Person(models.Model): objects = PersonManager() first_name = models.CharField(max_length=100) last_name = models.CharField(max_length=100) birthdate = models.DateField() def natural_key(self): return (self.first_name, self.last_name) class Meta: unique_together = ((’first_name’, ’last_name’),) That method should always return a natural key tuple – in this example, (first name, last name). Then, when you call serializers.serialize(), you provide a use_natural_keys=True argument: serializers.serialize(’json’, [book1, book2], indent=2, use_natural_keys=True) When use_natural_keys=True is specified, Django will use the natural_key() method to serialize any reference to objects of the type that defines the method. If you are using dumpdata to generate serialized data, you use the --natural command line flag to generate natural keys. Note: You don’t need to define both natural_key() and get_by_natural_key(). If you don’t want Django to output natural keys during serialization, but you want to retain the ability to load natural keys, then you can opt to not implement the natural_key() method. Conversely, if (for some strange reason) you want Django to output natural keys during serialization, but not be able to load those key values, just don’t define the get_by_natural_key() method. Dependencies during serialization Since natural keys rely on database lookups to resolve references, it is important that the data exists before it is referenced. You can’t make a “forward reference” with natural keys – the data you’re referencing must exist before you include a natural key reference to that data. To accommodate this limitation, calls to dumpdata that use the --natural option will serialize any model with a natural_key() method before serializing standard primary key objects. However, this may not always be enough. If your natural key refers to another object (by using a foreign key or natural key to another object as part of a natural key), then you need to be able to ensure that the objects on which a natural key depends occur in the serialized data before the natural key requires them. To control this ordering, you can define dependencies on your natural_key() methods. You do this by setting a dependencies attribute on the natural_key() method itself. 3.19. Serializing Django objects 417
- 422. Django Documentation, Release1.5.1 For example, let’s add a natural key to the Book model from the example above: class Book(models.Model): name = models.CharField(max_length=100) author = models.ForeignKey(Person) def natural_key(self): return (self.name,) + self.author.natural_key() The natural key for a Book is a combination of its name and its author. This means that Person must be serialized before Book. To define this dependency, we add one extra line: def natural_key(self): return (self.name,) + self.author.natural_key() natural_key.dependencies = [’example_app.person’] This definition ensures that all Person objects are serialized before any Book objects. In turn, any object referencing Book will be serialized after both Person and Book have been serialized. 3.20 Django settings A Django settings file contains all the configuration of your Django installation. This document explains how settings work and which settings are available. 3.20.1 The basics A settings file is just a Python module with module-level variables. Here are a couple of example settings: DEBUG = False DEFAULT_FROM_EMAIL = ’webmaster@example.com’ TEMPLATE_DIRS = (’/home/templates/mike’, ’/home/templates/john’) Because a settings file is a Python module, the following apply: • It doesn’t allow for Python syntax errors. • It can assign settings dynamically using normal Python syntax. For example: MY_SETTING = [str(i) for i in range(30)] • It can import values from other settings files. 3.20.2 Designating the settings DJANGO_SETTINGS_MODULE When you use Django, you have to tell it which settings you’re using. Do this by using an environment variable, DJANGO_SETTINGS_MODULE. The value of DJANGO_SETTINGS_MODULE should be in Python path syntax, e.g. mysite.settings. Note that the settings module should be on the Python import search path. 418 Chapter 3. Using Django
- 423. Django Documentation, Release1.5.1 The django-admin.py utility When using django-admin.py, you can either set the environment variable once, or explicitly pass in the settings module each time you run the utility. Example (Unix Bash shell): export DJANGO_SETTINGS_MODULE=mysite.settings django-admin.py runserver Example (Windows shell): set DJANGO_SETTINGS_MODULE=mysite.settings django-admin.py runserver Use the --settings command-line argument to specify the settings manually: django-admin.py runserver --settings=mysite.settings On the server (mod_wsgi) In your live server environment, you’ll need to tell your WSGI application what settings file to use. Do that with os.environ: import os os.environ[’DJANGO_SETTINGS_MODULE’] = ’mysite.settings’ Read the Django mod_wsgi documentation for more information and other common elements to a Django WSGI application. 3.20.3 Default settings A Django settings file doesn’t have to define any settings if it doesn’t need to. Each setting has a sensible default value. These defaults live in the module django/conf/global_settings.py. Here’s the algorithm Django uses in compiling settings: • Load settings from global_settings.py. • Load settings from the specified settings file, overriding the global settings as necessary. Note that a settings file should not import from global_settings, because that’s redundant. Seeing which settings you’ve changed There’s an easy way to view which of your settings deviate from the default settings. The command python manage.py diffsettings displays differences between the current settings file and Django’s default settings. For more, see the diffsettings documentation. 3.20.4 Using settings in Python code In your Django apps, use settings by importing the object django.conf.settings. Example: 3.20. Django settings 419
- 424. Django Documentation, Release1.5.1 from django.conf import settings if settings.DEBUG: # Do something Note that django.conf.settings isn’t a module – it’s an object. So importing individual settings is not possible: from django.conf.settings import DEBUG # This won’t work. Also note that your code should not import from either global_settings or your own settings file. django.conf.settings abstracts the concepts of default settings and site-specific settings; it presents a single interface. It also decouples the code that uses settings from the location of your settings. 3.20.5 Altering settings at runtime You shouldn’t alter settings in your applications at runtime. For example, don’t do this in a view: from django.conf im