Skip to content

Python 3000 and compatibility

June 23, 2007

Update: The practical incompatibilities between Python 2 and Python 3 in the end turned out much smaller than what was initially claimed would be the case. For more information see my free online book Porting to Python 3 which contains information on how to support both Python 2 and Python 3.

There is some discussion in Python community about the move to Python 3000. Basically it’s between people who are worried about it and those who aren’t.

There is a definite risk involved in breaking the backwards compatibility, and that is if you end up making it very difficult to make code that runs on both versions, as this is going to effectively stop people from moving to the new version. This is not unique to Python, it is a general problem, just look at Zope3, where it’s impossible to make code that runs on both systems. Some people are tired of hearing this being raised as an issue, but other people are tired of hearing it being brushed away. This IS an issue. Don’t brush it off by saying “look we’re trying our best” or “stay on Python 2.5 then if you like it so much”. Because the risk is just that people do stay on Python 2.5. And then you have TWO languages and TWO communities competing for resources. And that is a huge issue.

The Py3K project is of course completely aware of this, and is trying to address in two ways. Firstly by introducing forward compatibility in 2.6, and secondly by having a converter, so that you write Python 2 code, and run it through a converter to make it Python 3 code, and then release different releases.

The code converter is great for those who have good unit tests, and use binary eggs for releasing (as they will have to make different releases for different versions anyway). For everyone else, it adds quite a lot of extra work and complexity, and that complexity will most likely be too much for people to bother about it. And that means that this issue hinges on how easy it is to write code that runs directly under both 2.6 and 3.x without conversion. If that’s easy, all is well. If it turns out to be difficult, we have a problem. And the major issue here is this statement from the PEP 3000:

There is no requirement that Python 2.6 code will run unmodified on Python 3.0. Not even a subset. (Of course there will be a tiny subset, but it will be missing major functionality.)

There may not be a requirement in the PEP, but it IS a requirement from Python developers. If we really end up in the situation that it is practically impossible to write useful code that works under both Python 2 and Python 3, the transition to Python 3 will quite likely never happen. And that’s bad.

But that’s theory. How is the reality? Well, it looks better than PEP 3000 does. Most things in Python 3 can be introduced into Python 2 without any problems. The new I/O library and new names for CamelCased libraries should be easy. The 0o777 format for octals is also easy to support, a “from __forwards__ import print” should enable the new print() function instead of the print statement and so on.

There are some obvious problems as well. That 1/2 returns 0.5 in Python 3, but will return 0 in Python 2.6 will be a compatibility issue, but it’s solvable by writing 1//2, which will return the same thing in both. Many functions that today return lists will in Python 3 return iterables, but that can be fixed by doing list(dictionary.keys()) to make sure it’s really a list, if you use it like that.

Another issue like that is unicode strings. In Python 2.6, ‘…’ will be ascii/binary and u’…’ will be unicode. In Python 3, ‘…’ will be unicode, and b’…’ used for binary, and there will be no ascii strings as such. This is a good change, but unless the u’…’ syntax is supported in Python 3, and the b’…’ syntax in Python 2.6, you will have to type Unicode([0x7f, 0x45, 0x4c, 0x46, 0x01, 0x01, 0x01, 0x00]) and bytes([0x7f, 0x45, 0x4c, 0x46, 0x01, 0x01, 0x01, 0x00]), which is very ugly, and hard to convert to. This will practically stop anybody using either strings for bytes, or unicode in Python 2.6 to move to Python 3 without also dropping support for Python 2. It’s not clear that u’…’ is going to be supported in Python 3, and neither that b’…’ is going to be supported in 2.6, but both should be possible to support with little effort, so lets hope they are.

If this type of “pseudo-backwards compatibility” isn’t going to be supported, one alternative way forward is to use Python 2.7 and 2.8 to drop backwards compatibility in steps. That is, just as Python 2.6 introduce some 3.0 features into 2.6 while being compatible with 2.5, but not 2.4, Python 2.7 could introduce more 3.0 features, while being compatible iwth 2.6 but not 25. And 2.8 could be incompatible with 2.6, but compatible with 2.7 and 3.0. That introduces a gradual movement to 3.0, so that it is always possible to support two python versions with one set of code. This however requires a conversion process that is more than one step and is thus less desireable.

So the major issue here is how many little compatibility problems like that will pop up, and how easy it is to get around them. If there aren’t too many, then all is well. If they turn out to be many, then it’s a problem. Where the limit between “many” and “few” goes is largely a matter of taste, and therefore there is no answer to this, and there will not be an answer to this until sometime after Python 3 is released and we see how the community reacts. Basically, if the community ends up saying “I can’t write code that supports both 3.0 and 2.x”, the community will either stay on 2.x forever.

And that’s not just a problem for the community, and that doesn’t just mean that the Python 3000 effort has been wasted. It will also act as discouraging to new programmers, who may simply not get into Python at all, saying that “Python 2 is old and unsupported and Python 3 has no software”.

For me, it’s looking good. I can’t find that many obvious problems that isn’t easily fixable. But I’m not the community, I’m just me. So don’t brush the issue aside, because if you do, there is a high risk that this potential problems turns real. There HAS to be a way to go from Python 2 to Python 3 without dropping Python 2 support completely. A complete disruption WILL be damaging to Python as a whole.


From → python, python 3000

  1. maybe it could be acomplished via
    from __history__ import ? 🙂

  2. Graham permalink

    Obviously unit tests are necessary for it to work, but you can write Python 2.5/2.6 code that passes all tests under Python 2.5, while the output of *also* passes all the tests.

    Its a matter of disambiguating the few cases where 2-to-3 gets something wrong, so that it gets it right.

    Then you happily develop for 2.5, and can convert to 3.0 whenever you please just by running

Trackbacks & Pingbacks

  1. Top Posts «
  2. - The more you know - the discussion around Python 3000
  3. New project: python-incompatibility. Is Python 3.0 really incompatible? « Lennart Regebro: Plone consulting
  4. Why Python 2.6 and 3.0 compatibility would be a Very Good Thing. « Lennart Regebro: Python, Plone, Web

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: