Skip to content

Setuptools and easy_install for Python 3!

February 1, 2009

[Note: If you need Setuptools under Python 3 you can now use Distribute, which has Python 3 support.]

I spent some of my Yuletide vacation on porting setuptools to Python 3. The reason for that was simple: I want to use the Zope component architecture under Python 3. But the “official” way to develop on Zope foundations software is to use buildouts, and of course the component architecture modules have buildouts too. You can develop on them anyway, but not in the “correct” way. But before we can port buildout to Python 3 we must have setuptools, so I decided to try to port it.

So, said and done. It wasn’t easy, Phillip J Eby are way smarter than me, so there was a definite lack of comments in places where they would have helped me. Also, it needs to support versions as old as Python 2.3, so the code gets a bit hackish in some places. Also it does one thing that is tricky to port, it mixes binary and text data. This is because it sometimes doesn’t know if a file is text or not until it’s opened it and tried. But I got around most problems, and all the tests run, and you can now easy_install many packages under Python 3.

Download and test, please!

There are currently only a few packages for Python 3 that is packaged correctly with distutils and is installable with easy_install. Those are Durus – an object database, Evoque – a templating system, GChartWrapper – a wrapper for Google Chart, LEPL –  a parser Library, lockfile – a platform-independent file locking module, maildirproc – a maildir processor, pep362 – function signature objects, pg8000-py3 – a PostgreSQL interface library, pseudomethod –  call almost any function as a method, pyskein – a skein hash library, pyttk - a wrapper for Ttk, PyYAML – the name is self-explaining, qp – a web-application framework and finally qpy – xml/html content in functions (no, I don’t know what that means either).

The others aren’t correctly packaged with distutils for Python 3, so it’s not setuptools that’s buggy in those cases, as far as I can figure out.

About these ads

From → plone, python, python3, zope

25 Comments
  1. Florian Schulze permalink

    Nice work, I made some very quick tests and it seems to work.
    Now a nice option would be to be able to run 2to3 automatically on egg source distributions :)

  2. Thomas Heller permalink

    What do you mean by ‘aren’t correctly packaged with distutils for Python 3′ ?

    • Florian: Yes, that is indeed an option. In fact, I think it’s the only option if you want only source distributions, but I’m not sure.

      Thomas: I mean that you can’t install them with distutils under Python 3. Running python3.0 setup.py fails for some reason or another. (Although in fact, it’s not true for the circuits package, there I couldn’t try it because I get a connection refused from the server).

  3. Hey thanks! What kind of timeframe are we looking at for a release? It would be great if I could release LEPL 1.0 with documentation saying there is setuptools support for Python 3.0… :o)

    Cheers,
    Andrew

  4. Skip Montanaro permalink

    Seems to not work with the latest py3k branch. It relies on the cmp() builtin which will be gone in 3.1.


    File “/Users/skip/src/setuptools-0.7a1dev/pkg_resources.py”, line 1977, in __cmp__
    def __cmp__(self, other): return cmp(self.hashcmp, other.hashcmp)
    NameError: global name ‘cmp’ is not defined

  5. Skip Montanaro permalink

    Doesn’t work with the 3.0 maintenance branch either (same reason).

  6. (Florian) Now a nice option would be to be able to run 2to3 automatically on egg source distributions
    (Lennart) Yes, that is indeed an option. In fact, I think it’s the only option if you want only source distributions, but I’m not sure.

    I’d say it is certainly not be the only option… it is interesting to note that from the 14 packages you cite as properly being installable on py3 with setuptools, at least 4 of these (specifically Durus, Evoque, QP and Qpy) are running on py 2.4 thru to 3.0 from the *same* code base, i.e. no 2to3 conversion. A description of how this has been done for these packages is here:

    http://mail.mems-exchange.org/durusmail/qp/441/

    • Well, yes, avoiding 2to3 altogether is an option of course, but not always feasible. Mixing text and binary data can be complicated, doctests aren’t trivial (even with 2to3, but that’s another topic), if you want to introspect the exception data it gets very ugly, etc.

  7. Python 3.0.1 removed some builtins that *should* have been removed in 3.0 but wasn’t, for example cmp(). However, not all instances where cmp() was used was removed from the standard library. For example, it’s used in a place in distutils, which makes it almost impossible to port setuptools to 3.0.1 without having special cased code for 3.0.1. I will see what to do about this, but for the moment: Do not upgrade to 3.0.1.

  8. alex permalink

    Hi! I’m currently pretty new to Python and I was wondering what version I should learn first?

    Seems like a LOT of libraries aren’t working correctly with Python 3.0.

    Well,how the fact that cmp has been removed makes distutilsnot working under python 3.0.1? Shouldn’t be that hard to fix, no?

    Also, I’d like to know if there’s any distutils package in python 3? Or if it has been replaced by something else?

    If you could explain me :) Thank you very much!

    • You should learn Python 2. The differences are small, but they are incompatible, so 3.0 is not useful yet. The differences are so small that switching to 3.0 from a language learning standpoint is trivial.

      Yes, there is a distutils package in 3.0. Howver, in 3.0.1 it has a bug. Distutils still work, but the bug in disutils make setuptools not work under 3.0.1. I’ll work around it.

  9. I tried building the tarball under Windows and Python 3.0.1.

    I acknowledge that there are issues with Python 3.0.1 at the moment, but I believe the issues I encountered are specific to the Windows environment.

    The first issue was that cli.exe and gui.exe were not included. This caused the installer to fail when it couldn’t find them.

    I copied these exes from setuptools-0.6 svn, and encountered another error. The traceback is at http://paste.turbogears.org/paste/37902 . It appears the setup script is attempting to open cli.exe and then decode it as a cp1252-encoded text file (which fails on the 3rd byte which is 0x90).

    Has something changed as part of this patch that may have incited this behavior, or is it more likely this issue exists with the 0.7 branch in general?

    Thanks for putting together a Python 3 version; I’m finding Python basically unusable without setuptools.

  10. HI, just tried your setuptools for python3
    and get the following error:

    http://codepad.org/OlcTNZx4

    Thanks heaps!

    cheers
    James

  11. OK, I’ve updated it for Python 3.0.1. Haven’t tried on Windows yet.

  12. Lennart: Thanks for the update. I re-ran the test in Windows and it’s still failing the same way. Transcript here: http://paste.turbogears.org/paste/38469.

    I haven’t had the time to try to find a workaround. With any luck, I’ll have a chance to track it down this weekend.

  13. Thanks to Lennart’s help, I was able to build Windows installers for Python 3, both 32- and 64-bit.

    They may be downloaded here until they are released in a more formal way:

    http://dl.getdropbox.com/u/54081/setuptools-0.7a1.win32.exe

    http://dl.getdropbox.com/u/54081/setuptools-0.7a1.win-amd64.exe

    Regards,
    Jason

  14. Jason Trowbridge permalink

    I just wanted to say thanks. Your version of setup tools works just fine with the Mac OS X release of Python 3.1. While I haven’t run fully comprehensive tests on it yet, the Mako templating engine also seems to work just fine. (Mako appears to require setup tools, even with the source tarball.)

  15. Hi. i’m trying to connect mysql with python 3.2. i read in somewhere that if i want to install mysqldb module i need to install setuptools. Thus i download setuptools-0.7a1dev-r66.tar.gz [ http://code.google.com/p/python-incompatibility/downloads/detail?name=setuptools-0.7a1dev-r66.tar.gz&can=2&q= ]

    i run this command in cmd in setuptools directory.

    python setup.py install

    i got this error msg.

    Processing setuptools-0.7a1dev_r66-py3.2.egg
    Copying setuptools-0.7a1dev_r66-py3.2.egg to c:\python32\lib\site-packages
    Adding setuptools 0.7a1dev-r66 to easy-install.pth file
    Traceback (most recent call last):
    File “setup.py”, line 95, in
    scripts = [],
    File “C:\Python32\lib\distutils\core.py”, line 150, in setup
    dist.run_commands()
    File “C:\Python32\lib\distutils\dist.py”, line 919, in run_commands
    self.run_command(cmd)
    File “C:\Python32\lib\distutils\dist.py”, line 938, in run_command
    cmd_obj.run()
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\install.py”, line 73, in run
    self.do_egg_install()
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\install.py”, line 101, in do_egg_install
    cmd.run()
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 225, in run
    self.easy_install(spec, not self.no_deps)
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 441, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 492, in install_item
    self.process_distribution(spec, dist, deps)
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 511, in process_distribution
    self.install_egg_scripts(dist)
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 392, in install_egg_scripts
    self.install_wrapper_scripts(dist)
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 584, in install_wrapper_scripts
    for args in get_script_args(dist):
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 1576, in get_script_args
    header = get_script_header(“”, executable, wininst)
    File “C:\Users\Kakashi-X\Desktop\umut\setuptools-0.7a1dev-r66\setuptools\comma
    nd\easy_install.py”, line 1428, in get_script_header
    match = first_line_re.match(first)
    TypeError: can’t use a bytes pattern on a string-like object

    my os is Win7 ultimate. Turkish. Python version is 3.2.1

    Am i doing something wrong?

Trackbacks & Pingbacks

  1. python3.1にeasy_installをインストール | The Story Terrors
  2. caicono的自由世界 » 思考充分再行动 – python试水记
  3. Python3 et lxml « frabad
  4. PyGTK, PyQt4 for python3

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,339 other followers

%d bloggers like this: