Skip to content

Comments on Django’s design decisions

April 11, 2009

In my talk about what Zope did wrong, I tried to tell the world not to repeat Zope’s mistakes, and I learned at PyCon 2008 that Django have repeated some of them. Mark Ramm also did his keynote on what Django can learn from Zope at last year’s DjangoCon. So I don’t know if it was intentional, but Jacob Kaplan-Moss’ talk about Django’s design decisions at this year’s PyCon felt to me like Django trying to defend itself a bit.

And without a doubt, Django is a good web framework. If it wasn’t, it wouldn’t be so popular. And although I think it needs some big changes to be “future-proof”, I must say that this is basically not very important. One thing I have realized is that when it comes to software choice, choose what works for you, today. Future-proofing is irrelevant, because you don’t know the future. So my comments here are not designed to scare anyone away from Django. Of course, it is partly a question of me sticking the Zope technologies in your face and screaming “Hey Look at This, isn’t this Fantastic!?” and maybe to an even bigger extent just a way to say “Been There, Done That, Not a Good Idea”.

Architecture and Morality

I agree with the basic decisions of Django. They chose Python, open source, of course, having sensible defaults instead of requiring loads of configuration for everything, and also that you shouldn’t have “architecture astronauts”. There is a type of programmer who likes complexity for it’s own sake, and if you let them write a framework, it’s going to be amazing, and amazingly complex to the point of being unusable. But the risk of ignoring architecture astronauts isn’t what Kaplan-Moss says, that you risk reinventing the wheel. The risk is that you paint yourself into a corner. This is not a problem for end user software, because you can always refactor it. But it is a problem for frameworks, because here other people are using the framework and when you refactor it, you break every body else’s software. This is why corners are hard to get out of when building frameworks, because when you walk over the wet paint, it’s not just your work that gets undone, it’s everybody else’s as well. And that’s what the architecture gurus are there for, coming up with a technique of building that means you don’t end up in corners. But yes,you still should be aware of the guys that want to put in flexibility without being able to explain what the use case for that flexibility is. Doing that can often give you complexity because you have flexibility in places where you don’t need it. Zope 3’s publisher has a hook for you to replace the type of object traversal that is done (i.e how you figure out which object is meant with a particular path). And the default traverser has a hook to change how traversal is done on a particular object. And the default object traverser will use __getitem__ to figure out subobjects. Which of course is overridable. That means that you have three places where you can change what the subobjects of your object will be. That is simply not needed. I’ve yet to find a case where the simplest one, __getitem__, isn’t enough.

One common mistake between Zope 2 and Django, is that some functionality, like object security, is handled by special methods or attributes on the content class. That works well, in the beginning, but then you end up wanting to change the way the security works to make it more flexible, and you introduce more attributes, but keep backwards compatibility to not break everybody else code, and you end up with a horrible mess of magic attribute names that change the objects behavior in subtle ways. Zope 2 did this is a big way, and not only with security. I noticed last year that Django did that as well, although not in a big way. Yet.

Full stack or slim waist

I can also not really criticize them for being a full-stack framework. I do disagree with Kaplan-Moss, the dichotomy between a full-stack framework and a “best of breed” framework definitely exists. And even if you, like any good developer, tries to make your full-stack framework consist of separate parts that can be used independently of the whole, unless you make sure it is used independently of the whole, dependencies are going to start to creep in. Zope 2 and Zope 3 where also full-stack frameworks, and for the same reason as Django: There just wasn’t anything else out there. Obviously, when Django started, there was something out there, Zope 2, and Twisted. But the only part that was truly independent of Zope 2 was the ZODB, and Django decided to go for an ORM, and Twisted refused to use thread pools, making it significantly slower, as it has to start a thread for each web request.

But Jacob Kaplan-Moss then goes on to criticize the best-of breed frameworks because you need “glue code” to clue the components together. This is partly true, of course, but it doesn’t distract from the fact that the components in the glued system still works separately and can be learned separately. And that comes into play when Kaplan-Moss talks about the learning curve of Django, and how you have to almost unlearn bits. This is nothing unique to Django. With Zope 2 we have the exact same phenomenon, which is in the Zope community known as “The Z-shaped learning curve”. Jacob claims that this is something that comes with a framework. I do not agree. I think it comes with full stack framework, where there aren’t good separation between different parts, and the internals are messy. But I do not believe this is true for a well designed best of breed framework.

My way or the plugin way

And I have to really protest about Jacob Kaplan-Moss’ criticism over pluggable components. First of all he completely misses the point, and says that it’s not possible to write an application with STORM and then flip a switch and use SQLAlchemy instead. Well, of course it isn’t, unless you actually write a wrapper for them, which seems more work than it’s worth. But the point is that you shouldn’t be stuck. In Zope 2, you can in practice use two templating systems. DTML, which was Zope 2’s first one, and ZPT, which was a much better one introduced some years later. But what if you don’t like them? What if you want to use something else. You might like Zope 2 as a whole, but really detest both DTML and ZPT. Well, in Grok, we have made it very easy to plug in other templating systems. Of course you can’t switch in a project, all your ZPT templates would have to be re-written. But for the next project, you can stick in Genshi or whatever you want. (In fact you can use all of them at the same time, but that’s not true for all types of pluggable components, so it’s besides the point). Pluggability gives you flexibility.

And here Jacob Kaplan-Moss claims that by having choices is bad. That is almost shocking. His main argument is that you force the new user to make choices that he aren’t able to make, which is completely incorrect, because, as he mentioned, Django has sensible defaults. Grok has that same philosophy, and when you start a Grok project ZPT will be there by default. If you want to use Genshi instead, please go ahead and do so. But the sensible default, ZPT, is there from the beginning, and the flexibility adds no extra complexity to the user who doesn’t want to switch, and you force no-one to make choices they can’t make.

Jacob also claims Django can set their own pace, by controlling the whole stack. This is partly correct, but when you make a best-of-breed framework, you will typically have central people involved as contributors to the components you use, and as a major user of that component, you can typically influence the direction and get releases more or less when you need them. And if you aren’t a big user, and won’t have a voice, well then you probably don’t have enough developers to do everything yourself anyway.

Is Django the new Zope 2?

Jacob Kaplan-Moss acknowledges the risk that Django will become Zope. Thanks to that acknowledgement I don’t think Django will become Zope, although they clearly have gone quite a long bit on that path already. Hopefully they have stopped. I don’t know if they need to backtrack, but at least they shouldn’t go much further on the Zope path if they want to continue to be relevant. Because if you go to far, you have a big job in backtracking to become relevant again.

Oh, and Jacob: ZODB is not an ORM. It’s a real OODB. ;-)

About these ads

From → django, python, zope

6 Comments
  1. I’m sorry if I came across as defensive; it certainly wasn’t my intent. My goal with that talk isn’t to defend our decisions, but merely to look at them consciously and consider the ramifications of the trade-offs we’ve faced.

    One thing that really bothers me about the tone of most technical discussions is the implication that there’s a “right” and “wrong” choice, when in fact there’s positive and negative aspects of any decision. I think given the opportunity both of us would choose Python over Perl for nearly any task… but that doesn’t make Python “right” and Perl “wrong.” In fact, I have no doubts that if we’d implemented Django in Perl there’d be aspects about it that would be better (it’d probably be faster, for one).

    Further, each choice influences the next ones: if we’d written Django in Perl, the “whole stack” versus “best of breed” decision would have been a very different choice: having a working, established packaging ecosystem with a rich variety of choices would have made the decision to start from scratch and build a monolithic framework harder to stomach.

    If we as a community could do a better job discussing these tradeoffs without, say, accusing the other party of speaking “bullshit” we’d be better off. Frankly I’m offended that you think I’m such an evil person that I’d get up on stage at a technical conference and just lie. For the record, the idea that too much choice harms consumers has pretty broad support in the literature of cognitive science; see, for example, Steve Krug’s _Don’t Make Me Think_ (http://www.amazon.com/dp/0321344758/), Donald Norman’s _The Design of Everyday Things_ (http://www.amazon.com/dp/0465067107/), and especially Barry Schwartz’s _The Paradox of Choice_ (http://www.amazon.com/dp/0060005696/).

    Here, too, there’s plenty of evidence on both sides of this debate! For every citation “proving” that too much choice is a bad thing, there’s probably another that “proves” the opposite. My goal is to be able to have these kinds of debates without accusing the other party of being “wrong” just because we disagree with him.

    Again, I’m sorry I couldn’t do a better job make that point clear. Hopefully I’ve done so here.

  2. Can’t help but to defend Jacob on several points. I think Jacob mentioned the great book “Don’t Make Me Think” in reference to some of these choices you mention. Basically, not having choices will mean not wasting time thinking about if you’re using the best templating engine or ORM machinery.

    Secondly, my gut feeling (and having used both quite a lot) is that Grok is an abstraction of Zope3 and Django is an abstraction of Python. And if you follow Alex Martelli’s talk or Joel Spolskey’s famous article all abstractions are leaky and it’s only a matter of time until you have to go one level deeper.

    Technology aside, I definitely think Django is the new Zope2 in terms of community events.

  3. Jacob, it’s not my intention of accusing you of lying, I just wanted to say that you are wrong. And that’s also not an accusation. In any case the word bullshit was perhaps not the best choice, I’m sorry about that, and will change it. Also, my point is that you don’t need to force people to make choices. That’s what sensible defaults is about, and you can have sensible defaults when it comes to module choice as well. So if choice is good or bad is a moot point, as you are not forcing anybody to make a choice. You just give them the chance to have a choice of they want to.

  4. Peter: Well, of course you will have to go one level deeper. That does make a difference if you are a full stack framework or not. There you need good separation between the different levels, and in full-stack frameworks you typically haven’t. for the reasons I outlined. If you have that separation, then it means each layer is usable separately, and that’s more likely to happen with a best-of-breed framework, and if you don’t have that separation, you get tye Z-shaped learning curve.

  5. Martijn Faassen permalink

    Some comments:

    Python now has a working, established packaging infrastructure. It’s not perfect and plenty of people aren’t completely happy about it, but it definitely works and has been for some years (I’d say late 2007 when we had enough knowledge/infrastructure to deal with it) .

    When Django was developed this infrastructure didn’t exist yet. But when Zope was developed this certainly wasn’t there yet. Zope 3 (the packages in what we now call the Zope Toolkit) switched in 2007. Zope 2 will have switched to packaged distribution by its next release. Having to deal with independent packages definitely helped us think about dependencies, which I think is improving the whole stack.

    Concerning choices; I agree that too much choice can hurt usability of a system. But you can still make a system that allows *alternatives* but comes with a default choice that is made for you. That’s what Grok is trying to do. I think overall this is the right way to go for a software framework, so it doesn’t lock you in too much. I think that covers the topic of choice quite well.

    You can make one more argument in favor of *not* providing the potential for alternatives at all: the overall software becomes simpler and thus easier to read and learn. I think that’s a reasonable argument. A configurable framework probably never be as easy to follow as a framework with less choices: there are less indirections. With Grok we’re trying to show that you can come pretty close, but we’re not all the way there yet. I’ll also add to make a system configurable you need well-defined interfaces, and that having those can *help* understandability a lot too.

    Concerning abstractions. Django isn’t just an abstraction ‘over Python’, whatever that really means. Django is also an abstraction various web protocols and over a relational database.

    Grok is not trying to be an abstraction *over* Zope 3, though I’ll admit that it is in some ways. But what it really is, and what it is trying to be, is an alternative way to configure the components that come with the Zope Toolkit. Grok’s a primary user of the Zope Toolkit and we’re actively trying to simplify it (by clearing up dependencies and throwing out code and other efforts) to get rid of the situation where developers get the idea that Grok abstracts out over a giant mount of unseen bits.

    The Zope community (and also Grok) is in the business of building, reusing and expanding on abstractions. Abstractions are indeed leaky and the more abstractions the more indirection and the harder it may become to understand something: those are the risks of abstraction. But I believe that there is also tremendous value in reusing abstractions: less reinvention of the wheel, and a growing collection of powerful reusable components.

    The Django community’s building those too, in Django itself through Pinax, and my hope is eventually we can find a way to share useful components. Saying “import django” is one answer to that; I’ve heard it in th past. It’s not very satisfactory to me. Even though the Django community is big, the world isn’t Django. The Zope community knows this lesson pretty well, and we’re still implementing the consequences of it. So just like we try hard (with mixed success) to make pieces of Zope available in reusable form so you don’t have to commit to the whole of Zope in order to use bits of it, I prefer it I didn’t have to commit to depending on the whole of the Django codebase so I can use bits of it either.

  6. I like your wording about alternatives vs choices.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,285 other followers

%d bloggers like this: