July 2006 archive
Django Magic Removal Update
I dove in and updated to the latest Django code. Django changed the fundamental API a lot in order to “remove the magic“ and make things work in a more straightforward manner. That means that to update to the latest Django code, I needed to make some major changes to my website code. Here are my rough notes of how I did it all.
My first thought was, good thing I have this all under Subversion. It was time to learn how to branch and merge. Here’s how I created my magic-removal branch:
svn copy file:///home/bryan/svnrepositories/murdockfamily/trunk \ file:///home/bryan/svnrepositories/murdockfamily/branches/magic-removal \ -m "creating branch for django magic removal update to this project"
Next I checked out the new branch:
svn co file:///home/bryan/svnrepositories/murdockfamily/branches/magic-removal \ murdockfamily-mr/murdockfamily
At this point I was able to change to the murdockfamily-mr directory and set my PYTHONPATH like so:
export PYTHONPATH=`pwd`
And change settings.py so DEBUG = True and then run the development webserver:
python manage.py runserver
So far so good. Next I renamed the directory where I have my old checkout of Django to django_src.old. Then I updated the symlink in /usr/lib/python2.4/site-packages so it still points to that. No reason to break what’s currently working, yet. Then I downloaded the latest Django code:
svn co http://code.djangoproject.com/svn/django/trunk/ django_src
The next step for me, I decided, was to create a new database that was basically a copy of my current working one. That way I could figure out all the changes that need to be made to the database, and try them out to make sure they would work when I went to do this on the production server. It would also let me take my time getting this branch all up to snuff. So first, create a new database (very simple since I have a user with the right permissions set up already):
createdb murdockfamily
Then I copied the old to this new one by using pg_dump to backup, and then psql to process the backup into the new database:
pg_dump -U <superuser> bryan > backup psql -U <superuser> murdockfamily < backup vacuumdb -U <superuser> -a -z # recommended for good measure
Next I edited settings.py to use this new database and tried it out with the development webserver. I now had a complete copy of the original website. Now I could safely begin breaking stuff.
But not just random breaking stuff. No, I had a plan to remove the magic. That’s where I learned I could have two different Django installs too, actually, and that’s the first thing I broke. I hit ctrl-c on the development server that I still had running and set my PYTHONPATH to use the newest version of django:
export PYTHONPATH=/home/bryan/web/django_src:$PYTHONPATH
When I restarted the development webserver I got all kinds of great errors, in red even! It was time to get to work. The magic removal cheatsheet seems to be a little better guide with pointers into the big magic removal document. I decided to follow it, so the first thing was to make sure I had the most up-to-date manage.py:
cp /home/bryan/web/django_src/django/conf/project_template/manage.py .
Next were the database changes. I just saved the whole big list of them from the web page to a file and piped that into psql like so:
psql murdockfamily < db-changes
It errored out because I didn’t have django_flatpages or django_redirects. No big deal, I just commented out those SQL statements and ran it again. Another thing I noticed in the documentation was the list of changes for the comments tables that were needed, and stuff that needed to be done for the authentication stuff. I ran the SQL from those sections too. It all just seemed to work.
I also decided to not follow the advice about setting the db_table attribute in my models, but to rename the tables. This website only has two models, not worth some hacky work-around like that. :-).
Next I went back over to the cheatsheet and started re-arranging my directory structure like it recommends. I almost forgot to use svn mv instead of just mv when moving stuff around.
From here it was just a matter of plowing through the main magic removal document and making all the changes necessary to the syntax of just about everything. It was hard to make sure I got the proper change made in all the files in all the directories. More judicious use of grep would have helped me a lot there! Something like as simple as this:
grep -r apps *
would have shown where I forgot to remove apps from module paths when I moved my blog app out of the deprecated project subdirectory, apps. Live and learn (over and over in my case).
There were a few things that really gave me a hard time. The middleware classes moved, and the default enabled middlewares has changed. Turns out this is documented in Removing the Magic, but I missed it. I finally figured it out by looking at the default settings.py. In fact, I found that you could learn a lot by comparing your settings.py to the default one found in django/conf/global_settings.py.
The other thing that gave me a hard time wasn’t documented in Removing the Magic, but could have been found by looking at the global_settings.py, was that the INSTALLED_APPS needed to change. Apps have been separated out and you need to include more there. I’ll let you look for yourself so you figure out how to use the global_settings.py.
Another tricky one (for me) was that I got the table name wrong for the many-to-many relationship between entries and categories. The docs led me to believe that nothing was to be left plural: entries became entry, and categories became category everywhere. Well, not quite everywhere. The table blog_entries_categories should not have become blog_entry_category, as I wrongly assumed. It should have been blog_entry_categories. I finally learned this from the command python manage.py sqlall blog, which shows what tables Django would create for the blog app.
And now that it’s all working, it’s time to merge that branch I made of the website code back into the trunk. First, figure out what revision the branch was made from, in the branch working copy type:
svn log --verbose --stop-on-copy
Now, in the trunk working copy, make sure all is up-to-date and that there are no unversioned files in there (such as .pyc files, I found that doing this on a fresh checkout worked best), and then do the merge:
svn co file:///home/bryan/svnrepositories/murdockfamily/trunk murdockfamily cd murdockfamily svn merge -r 11:HEAD file:///home/bryan/svnrepositories/murdockfamily/branches/magic-removal
If it didn’t say that it “skipped” anything then it probably worked. Do an svn st -v just to make sure it got everything right and then check it all in:
svn commit -m "Merged magic-removal-branch changes r11:27 into the trunk."
The final bit of cleanup was to update my symbolic link to use the latest django code. Phew, that only took me about three weeks of working in the evenings after putting the kids to bed. No better way to learn about Django, Subversion, and Postgresql than to break something and then try and fix it, if you ask me.
Posted by Bryan on July 22, 2006 | Filed under: About This Site Geek | 0 comments
Isaac Lost His First Tooth
At church a week ago Isaac’s primary teacher brought him into priesthood to show me that he had lost his first tooth. She wanted to make sure he didn’t misplace it, so I put it in my pocket. Both of his front bottom teeth have been very loose, and I was betting the one that was now leaning over into the middle would go first. Well, today Ily had had enough of that Leaning Tooth of Pisa:
She pulled it out! I was surprised he let her, but he was brave. So one week later, Isaac lost his second tooth!
Posted by Bryan on July 9, 2006 | Filed under: Family News | 1 comment
