Present Perfect

Another 6 months, another Fedora release ! (I am happy Fedora is back on a 6 month schedule).

This is the first release of Fedora that I have on my computer without realizing I have it; I went checking this morning when the release would be, and apparently it was today. I checked my /etc/fedora-release, and I'm already on it, because apparently our Fedora hackers made the excellent decision that installing test3 would get you on a repo that would freeze for Fedora 8.

I think that is a great decision, it gives people more confidence to go and try the test releases because they know they will end up on a stable distro in the end.

Now mind you, I only got on the test3 release because my new T61 had some issues under F7 and I was hoping new X and kernels would fix most of them. My wireless (iwl4965 is still flaky, suspend to disk still does not work, and the video still has issues, but I digress)

Here is one question and one request though. First of all, who originally coined the name "Codec Buddy" without even writing a single line of code ? Seriously, the name is dumb, it can only remind me of the infamous paper clip or Microsoft Bob.

It's terrible that this name persists even though there is nothing in the whole thing using that name! Every time I read something with the word Codec Buddy, I decided against hacking on the thing, so it's one of the reasons it has arrived as late as it did.

So, the request, is simple: stop calling it Codec Buddy. I'm holding it hostage until you know better :)

Here's an interview about Codeina with our well-loved media nut Bastien and me. Ignore the wrong URL and the wrong titles and the multiple spelling mistakes for the word codeina.

Congrats to the Fedora team ! Now please fix my wireless and my suspend somehow :)

Another 6 months, another Fedora release ! (I am happy Fedora is back on a 6 month schedule). This is the first release of Fedora that I have on my...

Why is it that every time I get a mail sent from an iPhone, it ends in:

Sent from my iPhone ?

Funk dat!

Why is it that every time I get a mail sent from an iPhone, it ends in: Sent from my iPhone ? Funk dat!

First of all, congratulations to Joe Shaw on getting hitched! I can't believe that, only a few months ago, I was having drinks with him, and when I asked him how he was he didn't even mentioned getting married! I guess it was already a plain simple fact of life for him by the time.

But to keep it technical, I read through his reply to some Beagle comments, which resonated with me:

And lastly, having worked with Trow on a reasonably big desktop Python app, we wanted a strongly typed language. Writing real applications in Python requires a discipline that unfortunately most people (including myself) are unwilling to adhere to, and this easily leads to buggy and hard to maintain programs. You have to be very diligent about unit tests and code coverage for every line of code, because you can’t rely on the compiler to catch errors for you. We had been burned by this a bit, and wanted to get back to a strongly typed, but still easy to use language that integrated well with the desktop.

The core of this message is 100% true - you need discipline to write real apps in Python; you have to be very diligent about unit tests and code coverage.

The crux of the matter though is that this is true for *any* real application in *any* language. However old and boring it makes me feel to say so, doing these things are part of good software engineering practice, and for good reason. Since realizing this, all the projects I've started working on, as soon as they leave the single-day-hack-prototype phase, first get converted into a simple bare-bones project, and the first things I add is unit test, code coverage and API documentation infrastructure. This is very hard to retrofit on an existing codebase; you teach yourself the habit of doing so at the start so you can reap the rewards later.

Unit tests and code coverage support two separate things. The first is that they give you confidence, if done right, that overall your project code is working well. It allows you to decide to release because you have some way of measuring that the quality of your project is at least as high as on the previous release.

The second is that you can refactor correctly. Some of our hackers in the office incorrectly use the word refactor when they really mean "I don't like or understand this big section of code and it's buggy and I am going to rewrite it completely and see what happens." Refactoring is much more well-defined than that. It is the act of rewriting the internal implementation of a section of code, without changing the external behaviour at all. (A logical consequence is that refactoring code does not fix bugs - you need to produce the same bugs in your refactored code!)

How do you know if you changed the external behaviour ? You can only know so if you have unit tests which cover the code you're refactoring.

As a side note, a distinct bonus of your project having unit tests is that it makes it easier for others to hack on. Just this weekend, I found a simple bug in pychecker. Now, pychecker has a unit test setup, so that means I can add the unit test that is failing, figure out where in the code it's failing, fix the code, verify that it works with my unit test, then verify that I did not break any other functionality by running the complete unit test. The result is 15 minutes of patching work, and it's a joy to go bug fixing this way.
Getting back to Joe's post. I'm not sure Joe intended it to sound that way, but I felt the conclusion he drew from his experience with Python was "I don't use Python anymore because it forces me to think about unit tests and code coverage."

I don't know what kind of tests Beagle has as part of its codebase and release procedure. I wrote about my experience with Beagle at some point
and Joe commented to the effect that the kernel gives you 8192 inotify watches and when you run out, it acts funny.

Well, no shit. Of course I have over 8192 directories under my home directory. If Beagle is, for example (I have no idea if it does), really watching every .svn and every CVS directory, no wonder it is running out. Two questions spring to mind - how did Joe get away with not having over 8192 .svn and CVS directories, and why does the software not deal with this gracefully ? If I would learn about this limit, I would write an integration test (probably not a unit test, in this case) and test graceful handling of this boundary condition. (It's entirely possible that a test like this exists in the Beagle code, I haven't checked - I don't intend this as an attack on Beagle or Joe, for those who assume the worst in me :))

To cast a stone at myself, GStreamer for a long time only had a bunch of small ad-hoc test applications that developers would write as they hacked on some feature, then at best it would be integrated in some make check command, or at worst it would be forgotten about and bitrot. I think one of my biggest contributions to GStreamer was to reorganize the test suite into a coherent whole, and making it easy for developers to integrate into it. It's obviously not the only reason 0.10 is so much better than 0.8, but it has definitely helped a lot.

I'm not saying a program is automatically better for having unit tests and code coverage, that would be silly. A perfect programmer can knock out a perfect program with zero coverage and tests. A crap programmer can knock out a crap program with 100% test coverage. But all things equal, having tests and coverage makes your program better. These days, I definitely trust programs more, and prefer to use them more, if I've seen they come with tests, and the developers care about their quality.

The first project that really convinced me of these simple principles was the amazing Twisted project. In spite of the sometimes vicious social commentary those guys receive from their peers (bloat, overdesigned, badly documented, downright crazy ?), which makes them sound jaded sometimes, these guys have, over several years, created a consistently excellent framework. They care about their quality and their users, and have integrated so many of these good engineering practices in their daily workflow. I'm not sure they themselves realize how rare this was for a FOSS project back then, but for me, Twisted as a project was an eye opener.

Wow, this post is a lot longer than I intended it to be, after waking up way too early and still fighting the aftermath of a stomach flu. I have to get going.

Anyway, here's what I want you to take away from today's post.

I program in Python precisely because it forces me to write unit tests and think about coverage. You should be doing this anyway, regardless of the language you do it in. By choosing a language Python, you will realize why much earlier on in your project, or your project will fail spectacularly :)

First of all, congratulations to Joe Shaw on getting hitched! I can't believe that, only a few months ago, I was having drinks with him, and when I asked him...

While procrastinating about drawing up some process diagrams to visualize integration tasks, I came across graphviz. By came across, I mean, I finally really looked at it after all those brief encounters with its use our output in other tools.

While it seems incredibly handy to be able to describe graphs programatically, it does have a painful learning curve. If you disagree, try running "dotty", the visual editor :)

Anyway, here's something right off the bat I can't figure out: the documented shapes include note and tab, but when I use that as a shape attribute value, dot complains. I can't figure out any way of listing supported node shape types either.

Any graphviz expert care to throw me a bone ?

On a related note, any idea why there is no shape for the common flowchart symbol for documentation, which is a rectangle with the bottom side as a one-period sine wave ?

While procrastinating about drawing up some process diagrams to visualize integration tasks, I came across graphviz. By came across, I mean, I finally really looked at it after all those...

Like my grandmother said, "Never eat Chinese twice a day." I paid for it on the toilet last night, in full. I felt queasy all day and medication was needed.

Normally I wouldn't mention that unappetizing tidbit, because I have many much better war stories.

But in this case, I do, because there are two reasons I went to work anyway today, one of them amusing, the other maybe not.

The other reason was that there was an important meeting about my work that I did not want to weasel out of under any circumstances. This meeting was about SLA's.

The one reason was more practical. This week, they're refitting the water pipes in our building, which means that every morning, they close off the water at 8. The first night I prepared by showering late at night instead of in the morning. But last night's sudden food poisoning made the prospect of spending the day in an apartment without water very unappealing.

So, imagine this in terms of SLA's. We're building some system at work where, roughly speaking, we're selling some streaming services with SLA's at 99.5 or 99.9% uptime. (I'm simplifying, since really, good SLA's are supposed to be determined through rigorous application of probability and historic evidence, respecting the mathematical laws of probability)

Did these guys break my water SLA this week ? 8 hours, two days, that's already below 99.6% And the Chinese restaurant ? I've been told by someone that 25% of European food ingredients come from China, and do not meet the same stringent quality control as the others. What is their SLA ? 90 % maybe, based on historical evidence where I went there tence and got stung once ?

What's the SLA of the service where I'm healthy, or have water ? Answers on a postcard please.

Like my grandmother said, "Never eat Chinese twice a day." I paid for it on the toilet last night, in full. I felt queasy all day and medication was needed....