Present Perfect

Recently we got a new server for apestaart, our little hosting project between friends.

This time we decided to install CentOS, since people I trust have been saying good things about it, we use RHEL at work and it's pretty much the same, and Wiebe, the other admin, also was all for it.

This week I ran into a segfault and I wanted to debug it. Turns out CentOS ships without any debug repositories installed by default. Fedora installs the files, but cleverly disables them, as they should. But yumutils contains debuginfo-install, a handy script that allows you to install all the dependencies of a -debuginfo package, and gdb spits out useful commands like what packages to install when you're looking at a stack trace with missing symbols.

So, this doesn't work out of the box on CentOS. Fail #1.
I went on IRC, mentioned this there, and I was told that 'most CentOS users don't need debuginfo packages'. Fail #2 - neither do Fedora users, yet somehow Fedora managed to figure out how to work both for the ones that don't need them and the ones that do. So goodbye unhelpful IRC channel.
I had to manually create a .repo file based on some guy's post complaining about the same thing; something much like this:
[root@betsy ~]# cat /etc/yum.repos.d/debuggery.repo
[debuggery]
name=CentOS-$releasever - DebugInfo
baseurl=http://debuginfo.centos.org/$releasever/$basearch/
priority=1


Then I actually tried to do debuginfo-install python, and what happened ?

First of all, the python-debuginfo package for the installed version of python isn't even in that repo (though 3 others are, go figure). FAIL #3

Of the other packages that it did find, one was unsigned:

yum.Errors.YumBaseError: Package libtermcap-debuginfo-2.0.8-46.1.x86_64.rpm is not signed


FAIL #4.

CentOS, hostile to developers. Back to old-school 90's style rpm hunting and pecking on the web.

UPDATE:
As for the python debuginfo package, there is a RHEL5 package with exactly the right version and release tag, but that one was built for RHEL5, and installing it mismatches against the CRC of the installed python package. CentOS simply does not provide the debuginfo package for its shipped version of python. FAIL #5

Recently we got a new server for apestaart, our little hosting project between friends. This time we decided to install CentOS, since people I trust have been saying good things...

I'm writing some code that does something that seems straightforward but I'm trying to cover all my bases.

The code in question should perform a bunch of file/directory renames, and update some files that contain references to those changed filenames, such that the complete set stays consistent. (As a purely theoretical example, think 'a directory of sound files, and the .m3u/.cue/.log file in it).

I used to have something like this in DAD, so that you could fix spelling of tracks after ripping. Except that it was an ugly perl script and just gave up if anything went wrong in the middle (permissions, bad naming, machine crashing, ...)

This time around I want to take this conceptually simple but by design unatomic operation, and turn it into a set of resumable operations, such that if for whatever reason one of the atomic operations fails, or the machine crashes in the middle, there is enough information to resume the complete operation.

My current approach goes something like this:
- each Operation (renaming a file, replacing references in a file, ...) is an object
- the object has serialize/deserialize methods
- an Operator object gets loaded with a list of these Operations
- each Operator mains state of all tasks to be done, and which tasks are done already
- the Operator starts by serializing the todo list to disk in a given state directory
- it goes through each operation, performing the operation (which should end in an atomic step, for example moving the temporary file over the original file in the case of renaming in a file)
- after performing the operation, the operator serializes done to the state directory.
- the Operator continues performing tasks and saving done to the state directory.
- at the end of all operations, the Operator first removes the todo state, then the done state.

If, at any point, an operation errors out, then the problem can be resolved manually, and the Operator can be resumed (since it knows what the last operation was that it performed successfully).

If, at any point, the application or machine crashes, the todo and done state can be found in the state directory, so the Operator can be loaded from disk. The Operator knows which tasks have definately been performed, knows what the next task is. The only unknown is if the next task actually got performed successfully or not. Since each operation has to end in a final atomic call, it suffices to check if that final call was made correctly, and update the done state. From there it can continue.

This is the basic design I'm planning on doing. But I'm pretty tired today, lacking some sleep, so please punch holes in this approach, or tell me how overengineered this whole approach is and how it could be much better. Basically, the only even remotely similar thing I could find was this Perl module. I probably got the idea for using a state file from discussions with Jan and Arek at work about their RRD synchronization framework and its journal files, but I probably butchered their approach for this particular problem.

I could use a good reality check!

I'm writing some code that does something that seems straightforward but I'm trying to cover all my bases. The code in question should perform a bunch of file/directory renames, and...

Today the team released another development version of Flumotion! Strangely enough I still made it to the contributors list. Maybe I should look up what I am guilty of.

Here's what the guys say:

Yet another step in the long march towards a stable release. We made
sure we close more bugs than we create, hence the scarce features and
numerous fixes.

The bulk of the improvements is centered around the administration
interface. The configuration assistant gained in functionality,
stability and consistency.

More information here

Not sure why the guys decided to break with tradition and name this release after a restaurant (they probably assume it doubles as a bar). According to Flumotion tradition, micro releases are named after bars where we celebrate the release, and major/minor releases are named after the restaurants where we celebrate the release.

Today the team released another development version of Flumotion! Strangely enough I still made it to the contributors list. Maybe I should look up what I am guilty of. Here's...

Went to FNAC to get the new Yeah Yeah Yeahs album. For some reason when I left I had 80 euros worth of CD's in my bag.

The problem was I discovered a small section called 'tributes', where I picked up

• John Peel, a tribute
• Phil's Spectre, a collection of Phil Spector-produced tracks
• Kerouac - Kicks Joy Darkness
• Controversy - a Prince tribute with Soulwax, Hefner, D'Angelo and Stina Nordenstam
• Sing a song for you - tribute to Tim Buckley

I have a collector gene that keeps getting me into trouble.
Besides that, I also get the YYY one, a CD box called "Now that's what I call No. 1's" which at 10 euros for 60 tracks just represents good value for money, and some Arctic Monkeys EP.

Went to FNAC to get the new Yeah Yeah Yeahs album. For some reason when I left I had 80 euros worth of CD's in my bag. The problem was...

today give me, as a followup to yesterday's task post:

• an implementation of MultiTask that tracks progress across all tasks combined
• a new task that calculates the MusicBrainz TRM id/fingerprint of a track
• an example that uses the new MultiTask with the new TRMTask to calculate the fingerprints of a playlist given

Not bad for 90 minutes of hacking. I really like my expressiveness in Python. And all of this done while listening to beautifully mixed music with my current jukebox script. I'm actually enjoying hacking again!

The example doesn't actually save the fingerprints yet. My mini-goal with this for DAD is to fingerprint all audio on all my devices, as a basis to uniquely identify audio tracks, and then layer the rating of tracks on various machines on top of that information.

today give me, as a followup to yesterday's task post: an implementation of MultiTask that tracks progress across all tasks combined a new task that calculates the MusicBrainz TRM id/fingerprint...