Present Perfect

August is a great month for clearing the decks in Spain. Half the people at work are on holiday, and by the time they get back the other half goes away. It's a great month for making progress on all those things that've been lying around for months not getting done.

Same in my spare time. After lots of changes in my life in the last year, I've been settling down again, in a new apartment, in a new life, and I am finally in a place where I can do some hacking again. And it's a great feeling, to be back in a flow state and fixing problems.

Tonight I wanted to direct my hacking attention to my venerable N900 phone. First of all, I wanted to figure out why mushin, my couchdb-based GTD application, was not showing the right results for my shopping list. Turns out I had a broken svn snapshot running on the phone, and that was easy to fix. In the process I set up scratchbox again on my new laptop.

Then I directed my attention to erminig-ng, an application that syncs between the maemo calendar and Google calendar. It had stopped working a few months ago, giving a traceback during sync. I first tried reproducing the problem in scratchbox, but I couldn't even hit the bug after copying the sqlite databases for both erminig and calendar.

I ended up adding some well-placed prints and

import code; code.interact(local=locals())

statements on the phone's file system so I could inspect the objects provoking the crash. It turns out there was a simple call somewhere that overwrites an event's alarm time setting, but it does so with an int when the code in gdata expects a string.

Now, to submit the fix, I have the same problem as last time I submitted a fix. erminig-ng is a fork of erminig. erminig has a website that has no updates since 2008. The version number on that site is higher than the version number of erminig-ng, but erminig-ng is packaged as erminig, which is confusing for people. The bug tracker on maemo garage that the package for erminig-ng links to is only for erminig, and people get confused, so instead they're reporting all their problems in a long maemo talk conversation. It looks like the maintainer for erminig-ng is the same as for erminig, so I'm not sure why this confusion is there. It looks like it would b easy to clear up. The code is not in any public repository as far as I can tell, and people are fixing things here and there with patch files in the maemo talk thread giving instructions on how to patch files on the root file system. A mess, really. It should probably just go on github somewhere, but I want to hear from the maintainer first.

So, I slurped the latest 0.2.12 release source into my svn repository, forked it to a trunk directory, and started integrating my patch. All .py files are in the root, I had to fight the urge to not clean it up properly and put it nicely in separate directories, because that would make merging patches back harder. I added a test case for the bug I was hitting as well - there were no tests so far in the package.

After my bug was fixed, and the test worked, I rebuilt a package for my phone. That led me to updating my maemo repository, adding gpg signing to it, adding metadata info to it, and doing everything needed to make it easy to update and not trigger any errors on the phone when upgrading. Now it looks like a nice proper maemo repository (and even apt-file should work) - but if any debhead wants to give it a looksee to see if anything's wrong feel free!

So, now erminig 0.2.12.2 is available from my repository, works on my phone, and is now again properly syncing my calendar information. I dropped a line in the forum thread for a person who was probably running into the same bug as me.

Next on my radar - looking at barriosquare to see if an update to the foursquare v2 API is doable.

I felt so guilty for not doing enough maemo stuff over the last year to try and get an N950. Now I regret not applying - it's going to be the last phone in the series (yes, I'm one of those who think Nokia is making a huge mistake), the focus of what's left of Maemo will be on that phone, and I won't be able to update the programs I use on Maemo, and my phone will slowly become obsolete. At which point I probably have no choice but to switch to Android, the non-Linux Linux system. I don't know many people anymore who work for Nokia, but if any of them are listening and still want to help me get an N950, feel free to let me know.

And the kicker of my patchwork ? It turns out that, when I fixed a bug in the previous version, I had already forked off 0.2.11 into my repository at a different location. And in that checkout, I also added a unit test, a similar-but-different hack for not being able to load hildon as a module, and a HACKING and TODO file with very similar notes as today.

Hey, at least I'm consistent. And maybe I should throw this stuff on github after all, so I can use git's branch powers to track the upstream releases, and the work I did that did not get taken upstream...

August is a great month for clearing the decks in Spain. Half the people at work are on holiday, and by the time they get back the other half goes...

One of the key concepts in my rewrite of DAD is that it should be possible to relate the same track across different files and computers. I have copies of files, and different encodings of the same track, spread across machines. Various applications I use for playback seem to exist in isolation on each machine, and so I tend to rate only occasionally knowing that my ratings aren't centralized. And I get annoyed when banshee detects three copies of an album, and then orders them by track number, playing each track three times before moving on to the next one.

The logical way to do is is through acoustic fingerprinting. These are algorithms that extract certain features from an audio file and calculate an algorithm-specific 'fingerprint' for it. Usually, these fingerprints are not identical across different encodings of the same file, so you can't look up twins in a list; but the fingerprints can be compared to each other and a 'difference' within a certain confidence interval calculated.

Most fingerprinting algorithms have a library that calculates a fingerprint and then submits it to a complimentary web service where it can quickly compare it to find twins.

In the past, either the client library/application or the web service (or both) was not open enough to be of interest for most Free Software people.

But recently, someone in the #morituri channel mentioned acoustid which only consists of open components. So, that seemed interesting enough to try out!

The chromaprint client-side library consists of a library, a sample application (linked against FFMPEG), and a python module with some sample scripts.

There is also a gst-chromaprint GStreamer plug-in on github. (As a side note, amazing to see that GStreamer plug-ins these days come for free! I recall the days when we had to the work ourselves to write GStreamer plug-ins for libraries)

So, after giving them a quick test run, I packaged up the whole set and it's now available for Fedora 14 and 15 in my package repositories

The chromaprint-tools package contains fpcalc and you need to enable rpmfusion-nonfree to get its ffmpeg dependency.

And after that, I created a Task in DAD for chromaprint, and now I have:


$ dad analyze chromaprint /opt/davedina/audio/albums/Afghan\ Whigs\ -\ Gentlemen/Afghan\ Whigs\ -\ Debonair.ogg
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
/opt/davedina/audio/albums/Afghan Whigs - Gentlemen/Afghan Whigs - Debonair.ogg:
Found 1 results
- Found 4 recordings.
- musicbrainz id: 62b2952a-4605-4793-8b79-9f9745ea5da5
- artist: The Afghan Whigs
- title: Debonair
- musicbrainz id: 8ff78e73-f8bd-4d78-b562-c3e939fb93fb
- artist: The Afghan Whigs
- title: Debonair
- musicbrainz id: a0d5ced6-43e8-450a-bf11-94f1f4520b92
- artist: The Afghan Whigs
- title: Debonair
- musicbrainz id: d01ac720-874c-48d6-95c6-a2cb66f9d5d0
- artist: The Afghan Whigs
- title: Debonair


Sweet...

Now it's time to dump that in the couchdb database backend, and start identifying duplicate tracks.

Acoustid seems to be a relatively young project, but its maintainer is very active on the mailing list and it's filling a hole in the open world that I'm happy to see filled! Thank you Lukas.

One of the key concepts in my rewrite of DAD is that it should be possible to relate the same track across different files and computers. I have copies of...


[root@ana ~]# rpm -Uhv /home/thomas/rpm/RPMS/x86_64/gstreamer011-*
Preparing... ########################################### [100%]
1:gstreamer011 ########################################### [ 33%]
2:gstreamer011-devel ########################################### [ 67%]
3:gstreamer011-debuginfo ########################################### [100%]


Sweet!

[root@ana ~]# rpm -Uhv /home/thomas/rpm/RPMS/x86_64/gstreamer011-* Preparing... ########################################### [100%] 1:gstreamer011 ########################################### [ 33%] 2:gstreamer011-devel ########################################### [ 67%] 3:gstreamer011-debuginfo ########################################### [100%] Sweet!

This week at work we ran into a problem where one of our Python processes was consuming close to 3 GB of memory because it's not properly cleaning up a list. Because of other bugs this process could not be easily restarted without triggering other problems, so our core team asked for some suggestions and I told them "Why don't you try cleaning up the Python list using GDB and the Python C API ?" I had a vague recollection of someone on our team doing something like this a few years ago.

I also asked them to blog about it, because there aren't that many resources readily findable on the subject.

So here is Andoni's take on the problem.

If any Pythonista can suggest how he could have avoided the segfault during garbage collection, please let us know!

This week at work we ran into a problem where one of our Python processes was consuming close to 3 GB of memory because it's not properly cleaning up a...

I'm doing a webinar in a couple of minutes on Video In The Cloud: Live and On-demand Encoding and Delivery. A bit late to announce that if you want to catch it (I've been busy, sorry), but it will probably get recorded and be made available later on.

I'm there with people from Amazon, Sorenson and Zencoder, so that should give a good Q&A session.

It's a challenge doing that from Europe at the moment - I left the office past 23, and Neil even later - but it's a good opportunity to talk about the things we're working on.

I'm doing a webinar in a couple of minutes on Video In The Cloud: Live and On-demand Encoding and Delivery. A bit late to announce that if you want to...