Present Perfect


Picture Gallery
Present Perfect

This weekend’s yak shave

Filed under: Hacking,Python — Thomas @ 13:17


The yak shave started yesterday evening. The yak stack is actually a forked one this time, both of the forks involving pychecker.

I might not remember everything in order, but in a nutshell the stack is something like this:

  • The original goal for this weekend at the beginning of the week was to release my cd ripper, morituri
  • Something in its pychecker run did not run with pychecker 0.8.17 (the version still in Fedora, from 2006), and worked with pychecker 0.8.18 (my own build). Fork point 1.
  • While releasing moap this week, I realized that Freshmeat changed their remote API, which I should fix before I do another release of anything. Fork point 2.

Fork point 1 continues here:

  • I mailed Fedora's pychecker maintainer, offering my help, and sending a patch to update to 0.8.18, which I built and installed locally. He mailed back informing me about this pychecker bug with anaconda which was blocking the upgrade. Looking at that bug, it looked suspiciously like a bug triggered by code I added to pychecker last year.
  • However, I'd like to confirm so in the easiest way. git's got this great feature, git bisect, and wouldn'it be nice if I could do that on pychecker now ? Hey, why not add bisection to moap ?
  • CVS is actually not a very manageable VCS if you want to do fancy stuff. It costs me a few hours to figure out how I should get a more-or-less usable date from a CVS checkout. The final solution is similar to the reply to my stackoverflow question
  • moap vcs bisect run is now implemented and finds the commit that broke anaconda's pychecking. STACK POINTER IS CURRENTLY HERE.

Fork point 2 continues here:

  • moap's make check didn't work because pychecker complained about the following code:

    ef func():
    d = { 'a': 1, 'b': 2}
    print d.keys()

    which triggers, in python 2.6, the following warning:

    Object (d) has no attribute (keys)

  • I can't let myself change code in moap without a working make check, so on to figuring out what's wrong in pychecker
  • After lots of debugging and print statements, I figure out that pychecker dispatches Python opcodes, and it silently drops the ones it doesn't know about. Python 2.6 added a new opcode, STORE_MAP, and so pychecker doesn't properly handle the stack since it ignores the opcode. I should error out on those opcodes. STACK POINTER IS CURRENTLY HERE
  • Before I can fix that though, I decide I should make pychecker's test suite error out on unknown opcodes.
  • Of course, this will error out differently for different python versions, so I need different python versions on this machine.
  • I could do that by hand, but I'd also like Twisted's trial to run the testsuite, which also needs zope-interface in recent versions, which needs setuptools. So hey, why not set up jhbuild stuff to build all these python versions ?
  • Python 2.3 on my 64 bit machine doesn't work with setuptools, so the newest Twisted without it is 1.3.0, and it takes a while to figure out why trial doesn't run my testcases (it ends up being because 1.3.0 trial expects subclasses of twisted.trial.unittest.TestCase, not unittest.TestCase)

I'll blog about the useful products of my yak shave separately, for those who don't enjoy descriptions of yak shavings, only outcomes.

In general, I actually enjoy yak shaves. They're massive treasure hunts, you learn a lot, and you end up fixing a nice bunch of things all over the stack if you persevere. But it's probably more a mentality thing than anything else, and I really only indulge myself in these in my spare time.


  1. […] part of this weekend’s yakshave, I’m trying to implement a handler for STORE_MAP in pychecker. STORE_MAP is a new opcode in […]

    Pingback by thomas.apestaart.org » Python disassembly — 2009-06-27 @ 18:19

  2. […] Still on the yak shave expedition. […]

    Pingback by thomas.apestaart.org » jhbuild for python — 2009-06-27 @ 19:15

  3. […] step on this weekend’s yakshave: a first implementation of moap vcs […]

    Pingback by thomas.apestaart.org » moap vcs bisect — 2009-06-27 @ 20:32

RSS feed for comments on this post. TrackBack URL

Leave a comment