Monday 25 May 2009

Pagination: Splitting Your Data Into Bite-Size Chunks For Easier Digestion

This week it's time for something nice and simple, Django has some very useful tools for handling pagination and that is what will be covered.

If you have a site that is constantly having to return long lists of results/data, then sooner or later you are probably going to want to look for a way to easily split that data over several pages so that the user can view it in easier to read chunks. Django's pagination framework is one quick and easy answer to such a problem. Take the example given below:

#File: views.py
from django.shortcuts import render_to_response
from django.template import RequestContext

def pagination_test(request, word_string):
word_chars = list(word_string)
anagrams = []
for anagram in _generate_anagrams(word_chars):
anag = "".join(anagram)
if anag not in anagrams:
anagrams.append(anag)
response_dict = {'anagrams':anagrams,
'original':word_string,
}
context_instance = RequestContext(request, response_dict)
return render_to_response('pagination_test.html',
context_instance=context_instance)

def _generate_anagrams(word_chars):
# Warning - This function is horrendously inefficient and should
# not really be used for words longer than 6 characters.
char_count = len(word_chars)
if char_count == 0:
yield []
elif char_count == 1:
yield [word_chars[0]]
else:
for char_index in range(len(word_chars)):
lone_char = [word_chars[char_index]]
other_chars=word_chars[:char_index]+word_chars[char_index+1:]
for others in _generate_anagrams(other_chars):
yield lone_char + others
{# File: pagination_test.html #}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>Anagrams of {{ original }}</title>
</head>
<body>
<h1>Anagrams of {{ original }}</h1>
<ul>
{% for anagram in anagrams %}
<li>{{ anagram }}</li>
{% endfor %}
</ul>
</body>
</html>
#File: urls.py
from django.conf.urls.defaults import *

urlpatterns = patterns('views',
# Other urls ...
url(r'^test/pagination/(?P<word_string>.+)/$,
'pagination_test', name='pagination_test')
)

Here we take a string as an input from the user and return all the possible unique combinations that can be achieved by rearranging its characters. For short strings this produces manageable numbers of results but even getting up to five character strings the list starts to increase in size dramatically (something like order n^2). What we can do here though is put it through Django's pagination framework like so:

#File: views.py
def pagination_test(request, word_string, page_index=1):
word_chars = list(word_string)
anagrams = []
for anagram in _generate_anagrams(word_chars):
anag = "".join(anagram)
if anag not in anagrams:
anagrams.append(anag)
paginator = Paginator(anagrams, 10, 5)
try:
page = paginator.page(page_index)
except (EmptyPage, InvalidPage), e:
page = paginator.page(paginator.num_pages)

response_dict = {'anagrams':page,
'original':word_string,
}
context_instance = RequestContext(request, response_dict)
return render_to_response('pagination_test.html',
context_instance=context_instance)

def _generate_anagrams(word_chars):
# Same function as before

In the code above, paginator = Paginator(anagrams, 10, 5) sets up a paginator on our results list. The first argument to Paginator says that we want the items in the anagrams list to be split into pages. The second argument says that each page should contain ten results, whilst the third argument says that if there would not be at least five results on the final page they should be added to the one before (meaning the last page will contain anything from five to fourteen results). The other addition is responsible for saying which page of results should be displayed, first of all trying the value passed in and if it's a non-existent page then using the last available page.

{# File: pagination_test.html #}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>Anagrams of {{ original }}</title>
</head>
<body>
<h1>Anagrams of {{ original }}</h1>
<div>
Results {{ anagrams.start_index }}
to {{ anagrams.end_index }}
of {{ anagrams.paginator.count }}
</div>

<ul>
{% for anagram in anagrams.object_list %}
<li>{{ anagram }}</li>
{% endfor %}
</ul>
<div>
{% if anagrams.has_previous %}
<a href="{% url paged_pagination_test original anagrams.previous_page_number %}">Previous</a>
{% endif %}
{% if anagrams.has_next %}
<a href="{% url paged_pagination_test original anagrams.next_page_number %}">Next</a>
{% endif %}
</div>

</body>
</html>

In the template the main additions are for saying which results we're viewing through the use of start_index and end_index which are functions that belong to pagination page objects representing the first and last indices of the results on display and count which is a function on the paginator itself representing the total number of results.

In the loop that iterates over the results, it needs to be told to iterate over the object_list attribute of the pagination page as this represents the actual results to be shown on this page.

Finally the section at the bottom of the page is responsible for adding Next and Prev links. has_previous and has_next are pagination page functions that do exactly what you would expect and tell you if there is a page of results available either before or after the current one. Meanwhile previous_page_number and next_page_number return the indices of the page before and page after.

#File: urls.py
from django.conf.urls.defaults import *

urlpatterns = patterns('views',
# Other urls ...
url(r'^test/pagination/(?P<word_string>[^/]+)/$',
'pagination_test', name='pagination_test'),
url(r'^test/pagination/(?P<word_string>.+)/(?P<page_index>\d+)/$',
'pagination_test', name='paged_pagination_test'),

)

An extra line in the urls file picks up any visits to our test url that has a page index included on the end.

If you now try visiting this with a shortish word like "hop" you should be presented with a single page containing all six possible anagrams. Try again with "shop" and you should get two pages worth of results with ten on the first page and fourteen on the second. Finally try with "shops" and you should get six pages of ten results.

That's it for this weeks tutorial, tune in again next week where I will be starting to look at Django's middleware system.

Sunday 17 May 2009

Signals: Letting People Know When You've Got Something To Say

This week marks a break away from the topic of templates and instead will be focussing on signals.


Signals are Django's way of allowing you to listen out for certain events that either occur by default in the Django code or that have been specified by an application developer. The signals provided by default are explained in a fair amount of detail in the Django Built-in Signal Reference (djangoproject.com) so I won't go into too much depth on what they do, but they are as follows (skip to the first example):



pre_init
Occurs at the start of the process of initialising a model;

post_init
Occurs at the end of the process of initialising a model;

pre_save
Occurs at the start of the process of saving a model;

post_save
Occurs at the end of the process of saving a model;

pre_delete
Occurs at the start of the process of deleting a model;

post_delete
Occurs at the end of the process of deleting a model;

class_prepared
Internally used, occurring when a model class has been registered with Django;

post_syncdb
Occurs after syncdb installs an application;

request_started
Occurs just before Django starts to process a request;

request_finished
Occurs just after Django processes a request;

got_request_exception
Occurs when Django encounters an exception whilst processing a request;

template_rendered
Occurs only whilst running tests when Django renders a template;

comment_will_be_posted
Occurs if you have Django's comment app installed and a comment is just about to be saved;

comment_was_posted
Occurs if you have Django's comment app installed and a comment has just been saved;

comment_was_flagged
Occurs if you have Django's comment app installed and a comment has had a flag set (i.e. a moderator just approved it).


For my first example this week I'm going to be looking at the post_save signal and how it can be used to ensure that a User object always has a profile instance associated with it.


# File: test_signal/models.py
from django.db import models
from django.contrib.auth.models import User
from django.db.models.signals import post_save

class UserProfile(models.Model):
user = models.ForeignKey(User, unique=True)
allow_our_emails = models.BooleanField(default=False)
allow_other_emails = models.BooleanField(default=False)

def ensure_profile_exists(sender, **kwargs):
if kwargs.get('created', False):
UserProfile.objects.create(user=kwargs.get('instance'))

post_save.connect(ensure_profile_exists, sender=User)

# File: (somewhere in) settings.py
AUTH_PROFILE_MODULE = "test_signal"

As you can see this defines a fairly simple profile model (supposedly for storing a user's e-mail preferences), the key part of this comes from the function ensure_profile_exists. With all signals it can only be guaranteed that the sender keyword argument will be present therefore we also need to catch everything else using **kwargs. Inside the function we use the fact that post_save is supposed to provide the arguments created and instance to check if the save was for a new object and if so to assign the created instance to the user property of a freshly created UserProfile object.


Finally we connect our function to the post_save signal with instructions to only receive signals when the sending class is User


If the test_signal app is now added to the list of installed apps in settings.py then all new users created will now have an associated profile created also.


The second example for this week demonstrates how you can create your very own signals for code to listen to.


# File: test_signal/signals.py
from django.dispatch import Signal

form_post_received = Signal(providing_args=["path","arguments",]

Here we tell Django that we want form_post_received to be a signal and that it should provide the arguments path and arguments. Now if we add the following:


# File: test_signal/views.py
from signals import form_post_received
from django.conf import settings

def view1(request):
if request.method == "POST":
form_post_received.send(sender=request, path=request.path, arguments=request.POST)
# Continue to do stuff with the form.

def view2(request):
if request.method == "POST":
form_post_received.send(sender=request, path=request.path, arguments=request.POST)
# Continue to do stuff with the form.

def log_form_received(sender, **kwargs):
print "Form Received at %(path)s with arguments %(arguments)s" % kwargs

if settings.DEBUG:
form_post_received.connect(log_form_received)

When pointing views at view1 and view2 whenever the server is in debug mode extra information should be output to the terminal informing us of whenever a form has been posted to either view.


This is done by calling send on our signal along with an argument for sender, in this case we use the request that triggered the view, along with values for each of the other arguments we specified. Django then passes them on to each function that has connected to our signal, which in this case is our logging function (but only when the server is in debug mode, otherwise connect will never be called).


That concludes this weeks tutorial on signals. Next week we shall be looking at the useful functions Django has built in for performing pagination.

UPDATE:There is now a second part to this tutorial talking about more advanced uses of signals. Check it out at Signals - Part 2: Return to sender

Sunday 10 May 2009

Custom Template Filters: Manipulating The Results

After last weeks brief distraction on the topic of inclusion tags, the time has finally come for the tutorial on template filters.


You've probably come across several of the standard filters when writing django templates, they're the things that are appended to variables though the use of the | character. Some common examples include |length which does as it says and returns the length of the variable (I.e. Number of characters if it's a string or number of elements if it's a list or dictionary), |safe which prevents any HTML within the variable being escaped and |date:"D dS M Y" which formats a date type variable using the format string provided.


The first example we're going to be looking at today will be a filter which can be used for sorting a list (by its values) or a dictionary (by its keys). The code is as follows (this should sit in the templatetags directory along with the files for custom template tags):


#File: custom_filters.py
from django import template
from django.utils.datastructures import SortedDict

register = template.Library()

@register.filter(name='sort')
def listsort(value):
if isinstance(value,dict):
new_dict = SortedDict()
key_list = value.keys()
key_list.sort()
for key in key_list:
new_dict[key] = value[key]
return new_dict
elif isinstance(value, list):
new_list = list(value)
new_list.sort()
return new_list
else:
return value
listsort.is_safe = True

If you've been following the tutorials on custom template tags over the last few weeks then most of this should look familiar. The key new points to notice are the decorator @register.filter(name='sort') which tells django that the upcoming function is a filter and that it should go by the name of sort when it is used within a template; SortedDict which is a useful data structure that is provided by django which remembers the order that you added items to it and the line listsort.is_safe = True which gives a hint to django that we've not done anything to the variable to cause it to need escaping (if the variable needed escaping beforehand however then it will still need escaping afterwards).


Pointing a url at the following view should allow you to see the filter in action. Two numbered lists should be displayed, the first being a sorted version of test_list and the second a (key) sorted version of test_dict.


#File: views.py
from django.template import Template,Context
from django.http import HttpResponse
def filter_test(request):
test_list=['Item 1','Item 2','Third', 'Initial element', 'Item 1.5']
test_dict={'a':'First', 't':'Second', 'b':'Third', 'alpha':'Fourth'}
t = Template("""{% load custom_filters %}
<html>
<head><title>Filter Test</title></head>
<body>
<ol>
{% for item in test_list %}
<li>{{ item }}</li>
{% endfor %}
</ol>
<ol>
{% with test_dict|sort as sorted_dict %}
{% for key, value in sorted_dict.items %}
<li>{{ key }} - {{ value }}</li>
{% endfor %}
{% endwith %}
</ol>
</body>
</html>""")
return HttpResponse(t.render(Context({'test_list':test_list,
'test_dict':test_dict,})))

That now shows the basics of creating custom filters, however it's also possible to create them to take extra arguments. To demonstrate this we'll look at a filter which allows you to prefix a string to the front of other string type variables. The code is as follows (append it to the existing file):


#File: custom_filters.py
from django import template
from django.template.defaultfilters import stringfilter

register = template.Library()

@register.filter
@stringfilter
def prefix(value, prefix):
return "%s%s" % (prefix, value)

In this example there's not been any arguments passed in to @register.filter, in this case django will take the name of the function being decorated to be the name of the filter (i.e. prefix). The @stringfilter decorator has been used on the function, which informs django that we only wish to operate on strings and so django will ensure that any variable passed in to the filter has been converted into a string first. Finally we do not specify that the output is safe, as there's no guarantee that the item being prefixed wont require escaping later on and so by not declaring it as safe it will force django to check that the string is escaped later on.


Now alter the view from before as follows (changes highlighted like this) and you should see the same result as before except the elements of test_list will have "Sorted! " prefixed to it.


#File: views.py
from django.template import Template,Context
from django.http import HttpResponse
def filter_test(request):
test_list=['Item 1','Item 2','Third', 'Initial element', 'Item 1.5']
test_dict={'a':'First', 't':'Second', 'b':'Third', 'alpha':'Fourth'}
t = Template("""{% load custom_filters %}
<html>
<head><title>Filter Test</title></head>
<body>
<ol>
{% for item in test_list %}
<li>{{ item|prefix:"Sorted! " }}</li>
{% endfor %}
</ol>
<ol>
{% with test_dict|sort as sorted_dict %}
{% for key, value in sorted_dict.items %}
<li>{{ key }} - {{ value }}</li>
{% endfor %}
{% endwith %}
</ol>
</body>
</html>""")
return HttpResponse(t.render(Context({'test_list':test_list,
'test_dict':test_dict,})))

That's all for the subject of custom filters. Next time we'll take a break from template related tutorials and have a look at Signals in django.

Saturday 2 May 2009

Custom Template Tags - Part 3: A Last Minute Inclusion

After finishing last week's tutorial on block style template tags I realised that there was one template tag creation tip that I had yet to mention and that's the inclusion tag. This is quite closely related to the simple template tags described previously in that it only requires you to write one function. This time however you must provide also provide a template name which will be rendered at the end of it all. Take a look at the following example (a reworking of the simple tag example)


#File: list_formatters.py
from django import template

register = template.Library()

@register.inclusion_tag('format_list_tag.html')
def format_list(input_list):
return { 'unformatted_list':input_list, }

This file is fairly straightforward, we decorate the format_list function to say that it's an inclusion style tag and that it should use the template format_list_tag.html we then return a dictionary which will be used as the context for the template when it is processed. All that's left to include is the following template file:


{# File: format_list_tag.html #}
<ul>
{% for item in unformatted_list %}
<li>{{ item }}</li>
{% endfor %}
<ul>

This file simply provides a snippet of html describing how to display the list. Pop list_formatters.py into your templatetags directory and format_list_tag.html into your templates directory and pointing a url towards the following example (repeated from the previous tutorial) you can see the tag in action.


from django.template import Template,Context
from django.http import HttpResponse
def tag_test(request):
test_list=['Item 1','Item 2','Third time lucky']
t = Template("""{% load list_formatters %}
<html>
<head><title>Tag Test</title></head>
<body>
{% format_list test_list %}
</body>
</html>""")
return HttpResponse(t.render(Context({'test_list':test_list})))

There is one other useful feature of the inclusion tag and that is the ability to access the context. This can be done as follows (changes from the original file have been highlighted):


#File: list_formatters.py
from django import template

register = template.Library()

@register.inclusion_tag('format_list_tag.html', takes_context=True)
def format_list(context):
return { 'unformatted_list':context.get(test_list,[]), }

Whilst this example isn't really the best way to do things it should illustrate the power of what can be done by the inclusion tag and should enable you to use it in your own django sites.


Next time things will be back on track with the promised tutorial on custom filters