In this tutorial we will show how to connect to MSSQL from Python/Django using pyodbc.

Installing on Mac OS X

First of all we must install two dependencies: freetds and libiodbc. Let’s use MacPorts for this (http://www.macports.org/). The MacPorts Project is an open-source community initiative to design an easy-to-use system for compiling, installing, and upgrading either command-line, X11 or Aqua based open-source software on the Mac OS X operating system.

sudo port install freetds
sudo port install libiodbc

Now we have to install pyodbc and django-pyodbc (in your virtualenv of course!)

pip install pyodbc
pip install svn+http://django-pyodbc.googlecode.com/svn/trunk/#django-pyodbc

When everything is ready we can set up odbc. To do this open ODBC Manager -> Drivers -> Add… and enter this data (change path if needed):

Driver Name: FreeTDS
Driver file: /opt/local/var/macports/software/freetds/0.82_0/opt/local/lib/libtdsodbc.so

It appears as if the system stores ODBC configuration data in /Library/ODBC, but the Mac Ports stores configuration in /opt/local/etc. So lets do some symlinks:

sudo ln -s /Library/ODBC/odbc.ini /opt/local/etc/odbc.ini
sudo ln -s /Library/ODBC/odbcinst.ini /opt/local/etc/odbcinst.ini

That’s it!

Installing on Ubuntu

First of all we must install two dependencies: freetds and libiodbc.

sudo aptitude install unixodbc unixodbc-dev freetds-dev tdsodbc

Then we install pyodbc and django-pyodbc:

pip install pyodbc
pip install svn+http://django-pyodbc.googlecode.com/svn/trunk/#django-pyodbc

When we have all in place, we must set up odbc. To do this edit odbcinst.ini file (change path if needed):

- sudo vim /etc/odbcinst.ini

and paste this configuration:

[FreeTDS]
    Description = TDS driver (Sybase/MS SQL)
    Driver = /usr/lib/odbc/libtdsodbc.so
    Setup = /usr/lib/odbc/libtdsS.so
    CPTimeout =
    CPReuse =

You are ready to use MSSQL now!

Django Settings

Last step is proper settings file. Here is an example:

DATABASES = {
    'default': {
        'ENGINE': 'sql_server.pyodbc',
        'NAME': 'DB NAME',
        'HOST': 'HOST IP',
        'USER': 'USER',
        'PASSWORD': 'PSW',
        'PORT': 1433,
        'OPTIONS': {
            'driver': 'FreeTDS',
            'host_is_server': True,
            'extra_params': "TDS_VERSION=8.0"
        }
     } 
}

Most important are ENGINE and OPTIONS.

'ENGINE' should be set to 'sql_server.pyodbc'
'driver' name is a name used in odbc configuration.
If you use remote db, you must use `'host_is_server': True` parameter.
Last important param is “TDS_VERSION=8.0“. We must declare TDS protocol version.

After all these steps, you should be able to connect MSSQL successfully.

Troubleshooting

You might encounter an error when trying to read Decimal fields: MemoryError of fetching results. – on Mac OS X
HY003 Program type out of range (SQLGetData() ) when trying read numerical field (MSSQL) – on Ubuntu

It seems that problem was fixed by wesm at github:

https://github.com/wesm/pyodbc/tree/getdata-decimal-bug

When you update pyodbc with this build (pyodbc 2.1.10-beta04) problems should dissapear! Hope this post was helpful and saved you some development time.

The Python/Django duet is fantastic, everyone knows that. However it can be even more awesome with these apps:

1. South

This is an intelligent schema and data migration tool. If you don’t use it yet, you will want to after reading this post. Okay… but what it is exactly?

When you create a Django app, you are using the “syncdb” command. After a model changes you have to delete the whole database and run “syncdb,” or manually do SQL changes directly in the database. It’s even more tragic when you already have production data. Not so fun, right?

Enter our hero: “South.” Its main objectives are to provide a simple, stable and database-independent migration layer to prevent all the hassles schema changes bring to your Django applications over time.

Simple example

Let’s create a model in our “southtut” app:

class Knight(models.Model):
    name = models.CharField(max_length=100)
    of_the_round_table = models.BooleanField()

Then let’s create our first migration:


$ ./manage.py schemamigration southtut --initial
Creating migrations directory at '/home/andrew/Programs/litret/southtut/migrations'...
Creating __init__.py in '/home/andrew/Programs/litret/southtut/migrations'...
+ Added model southtut.Knight
Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate southtut


And apply our new migration:


$ ./manage.py migrate southtut
Running migrations for southtut:
- Migrating forwards to 0001_initial.
> southtut:0001_initial
- Loading initial data for southtut.


Very similar to “syncdb,” for now. But now time for magic! Let’s modify the model:

class Knight(models.Model):
    name = models.CharField(max_length=100)
    of_the_round_table = models.BooleanField()
    dances_whenever_able = models.BooleanField()

To apply these changes we have to run these two commands:


$ ./manage.py schemamigration southtut --auto
$ ./manage.py migrate southtut


And that’s all! All data is in place with the new structure.

Read more here: http://south.aeracode.org/

2. Fabric

Fabric is the second must have app. Fabric is a Python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.

It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution.

Small example:

You are working on a Django app locally. After you finish working, you commit all changes to a repository. Now you want to deploy all the changes to staging or production server. This is what it normally looks like:

- open new console
- connect with server via SSH
- run virtual environment (if any)
- install any new requirements
- pull all changes from repository
- run syncdb, migrations (if South is used)
- collect static files
- delete *.pyc files (to be sure)
- restart apache/nginx or whatever server you have using
- restart memcache

10 steps and we are ready. But… how about doing same thing using only one line? Simple — use Fabric. With the proper configuration of all these steps we can close in this line:


$ fab production launch


Here you can read more: http://fabfile.org

3. Django compressor

What we would like to achieve in our website:

- every file is downloaded from server only ONCE (every next time cache is used)
- only one CSS and JS file is downloaded
- client browser always knows what file is actual and what file should be downloaded again

All of that is possible with Django Compressor.

Django Compressor combines and compresses linked and inline Javascript or CSS in a Django template into cacheable static files by using the “compress” template tag.

Example:

{% load compress %}

{% compress css %}

{% endcompress %}

Which would be rendered all in one file:

<link rel="stylesheet" href="/static/CACHE/css/optimized_file.css" type="text/css" media="screen">

More information here: https://github.com/jezdez/django_compressor

4. Tastypie

Sometimes we have to add an API to our application without the ability to modify the sources of that app. Tastypie is exactly what we need. It provides a convenient yet powerful and highly customizable abstraction for creating REST-style interfaces. It relies only on its own code and focuses on providing a REST-style API.
Let’s create a resource class for our model “Knight”:

from tastypie.resources import ModelResource
from southtut.models import Knight

class KnightResource(ModelResource):
    class Meta:
        queryset = Knight.objects.all()
        resource_name = 'knight'

Now we have to hook up a resource:

from django.conf.urls.defaults import *
from southtut.api import KnightResource

knight_resource = KnightResource()

urlpatterns = patterns('',
# ...
(r'^southtut/', include('southtut.urls')),
(r'^api/', include(knight_resource.urls)),
)

And voila! We have set up our REST interface. Our resource is available at these urls:

etc...


Of course, this is only one simple example. For safety, Tastypie ships with the authorization class and many more.

More information: http://tastypieapi.org/

5. Easy thumbnails

There are many apps for thumbnails. However, this one is the simplest and most powerful I’ve ever seen. We only have to add ‘easy_thumbnails’ in INSTALLED_APPS.

It’s great for quick thumbnails in template usage:

{% load thumbnail %}
{% thumbnail [source] [size] [options] %}

where:

- source must be a File object, usually an Image/FileField of a model instance,
- size in the format [width]x[height]
- list of options like sharpen, crop, and quality=90

And that’s all. The thumbnail is created dynamically (if it doesn’t exist) when the page is requested.

Of course, there is Model and Low Level usage too.

More information: https://github.com/SmileyChris/easy-thumbnails

In this post we’re going to talk specifically about the django-common app, which provides functionality commonly needed in a django application (in case it wasn’t obvious).

WWWRedirectMiddleware

WWWRedirectMiddleware redirects requests to make sure they are on the www subdomain or off the www subdomain, depending on your settings.  Besides adding ‘django_common.middleware.WWWRedirectMiddleware‘ to MIDDLEWARE_CLASSES, you need to set DOMAIN_NAME in the settings, either with www or not.  You need to also set IS_PROD to true, as you don’t want to redirect on localhost.

SSLRedirectMiddleware and NoSSLRedirectMiddleware

SSLRedirectMiddleware redirects requests to make sure that they are under https.  NoSSLRedirectMiddleware makes sure they are under http.

ssl_required decorator

If you don’t need ssl across the entire site (you oftentimes don’t) you can simply use the ssl_required decorator on the specific view that requires https.

disable_for_loaddata decorator

This decorator is used to wrap signals instead of views.  It disables signals from being fired when the loaddata command is run.  See: https://code.djangoproject.com/ticket/8399

EmailBackend

The EmailBackend is an authentication backend that works like the default ‘django.contrib.auth.backends.ModelBackend‘ except it finds users by email instead of username.  To use it simply add ‘django_common.auth_backends.EmailBackend‘ to AUTHENTICATION_BACKENDS in the settings.

These are just a few features available in the django_common app.  Stay posted for more posts on the django_common app, but of course the best way to find out all the functionality is to check out the app itself!

There are several ways to do this, and there is no absolute right or wrong way.  This is just one way to manage extra user data and we find it very easy to use and manage.

First we create a user profile model (Like to create a separate ”accounts” app to house this data).

from django.contrib.auth.models import User
from django.db.models.signals import post_save

class UserProfile(models.Model):  
    user = models.OneToOneField(User)  
    #other fields here

    def __str__(self):  
          return "%s's profile" % self.user  

def create_user_profile(sender, instance, created, **kwargs):  
    if created:  
       profile, created = UserProfile.objects.get_or_create(user=instance)  

post_save.connect(create_user_profile, sender=User) 

User.profile = property(lambda u: u.get_profile() )

As you can see above is a normal model with a foreign key to the Django User model. Then add any other fields you would want to store for a user: date of birth, website, favorate color, or whatever. Then we are going to tie a signal to the creation of users. In this case we use the django post_save signal and set the sender as the User model. When a user is saved it will then call create_user_profile.₁

On line 17 we are monkey patching the User object to have a new property. All this does is create a shortcut for our own use later.  So rather then typing my_user.get_profile().fav_color we can simply type my_user.profile.fav_color

Under the hood they are the same but I find one is more aesthetically pleasing then the other.

Lastly we need to tell django that we have a user profile so that my_user.get_profile() works. In your settings.py simply add:

AUTH_PROFILE_MODULE = 'YOURAPP.UserProfile'

You can read more about this in the django docs for storing additional information about users.  
 
 
 
₁  if created is not really needed but is a good double check along with UserProfile.objects.get_or_create. Using this we should never have an error of any user duplications or any other weird bugs/quarks your code may produce.

In the end, an application is an application. And users today want to be able to use that application anywhere, from any device, and have their personal content just magically appear.

This is as true for consumers as it is for office workers. As a consumer, I want my Amazon.com experience to be seamless on my computer, my iPad, and my iPhone. And when I’m at work I want to be able to create a spreadsheet, work on it anywhere, and collaborate on it with my office colleagues anywhere.

For developers, of course, the challenge is creating software applications which deliver a high-quality user experience across many different devices. But 2012 technologies will make it easier than ever for software engineers to “write once, run many”. New applications will no longer need a desktop version, a web version, and a mobile version. Convergence allows us to develop one version, available to users everywhere. That’s a huge opportunity.

Here are some other trends we expect to be a big part of the year ahead:

Mobile First
For most websites and applications, mobile is clearly the growth area. According to Gartner Research, smartphones and tablets will represent more than 90 percent of the growth of device adoption in the coming four years. So any new online initiative needs to start by looking at how to create a high quality experience for mobile users.

Open Standard Technologies Rule
Making a software application perform well across multiple platforms requires an absolute commitment to open standards. In 2011 we saw Flash, a proprietary (“semi-open”) technology die-off for just this reason. At Tivix our standard development stack is LAMP (P=Python), Django, HTML, CSS, and JavaScript – all open-source, standards-based technologies. In 2012 HTML5 will be the most important standard of all, bringing a whole suite of open technologies which work across all devices – from computers to tablets to mobile phones.

Infrastructure Disappears
The essential benefit of cloud computing (and its cousin, Software-as-a-Service) is that users don’t to have to purchase, install, and configure anything; computing is just delivered to them as an on-demand service. The same is true for us as developers. Today, with solutions like AWS or Linode, we can have an entire set of servers configured and running in a half-hour, with zero capital investment. This is huge, especially for the kind of development speed and agility our clients want from us today. In 2012 infrastructure will continue to become invisible, reliable, and essentially infinitely-scalable.

User Experience is More Important than Ever
The bar is set high today. Users expect even a simple website or enterprise application to have the same quality of user experience they get on Facebook or Google. On top of that, a user interface today has to work well on many different screen sizes, from large monitors to tablets to small smartphones. 2012 will see the embracing of Responsive Design techniques, where the design of the user interface automatically and fluidly adjusts for the user’s device and screen size. For some great examples of Responsive Design, see Foodsense, the Boston Globe, and Illy (open and close the width of your browser window and watch how the design automatically adjust itself for any screen size, including mobile).

2012 will feature many other tech trends as well, of course. Social will continue to be huge, as will the geolocation capabilities of mobile. And the combination of the two will continue to drive opportunities to deliver context-aware content to users. The proliferation of mobile payment solutions will mean that consumers will become increasingly comfortable using their smartphones for shopping and purchasing. And video of all kinds will continue to be the largest growth category of online content.

But mostly, 2012 will be about convergence. At Tivix, we see nothing be great opportunities ahead. It’s going to be an exciting year.