Present Perfect

So, Kristien was offered a job at (what used to be) Belgium's biggest radio station before she left to China, and after some to-ing and fro-ing it was decided that the offer was too good to pass up. As a radio presenter jobs are scarce, and helping a big station back on its feet is every presenter's dream. It's all gone pretty quickly since then. She started last week.
The day before we left for Belgium to get her set up, we decided we had some things to celebrate and went out to Comerc 24.

It's a restaurant owned by one of the helper chefs of El Bulli. El Bulli is a world-famous restaurant where you have to book more than a year in advance and the kitchen is more chemistry than cooking. Comerc 24 is still expensive (we paid over 160 euro for the two of us), but slightly less experimental as well. We got a nice range of around 15 tapas, of whic the truffle-serrano-cheese bikini was probably one of the best, and the codfish foam probably the weirdest.

Part of the move involved moving Kristien's studio to Belgium. Kristien's studio consists of a PC running Linux, a Hammerfall 9652 card (which has 16 digital channels - well, actually 18), a VoiceMaster Pro for the sound processing, and a great voice microphone. The machine here has software RAID, so I took out one of the two IDE drives, and made sure that the other drive was online on the home server so I could log in later and take a peek at the config of the machine in case I messed something up somewhere. We put all the equipment in our luggage before going out for dinner, so we could just go to bed after dinner and get up at 6 in the morning to get our plane.

Sadly, when we got back from dinner, the home server had locked up. On further investigation, the machine had crashed and its hard drive had died. Have I mentioned before I have terrible luck with hard drives ? Well, since I was already planning to upgrade from FC3 to FC5 on that machine, I might as well just do it now. I had a bunch of spare sub-60GB drives lying around, and after the usual fiddling of getting a CDROM drive installed in that machine (this BIOS actually refuses to boot if a CD-ROM is a lonely slave on the IDE connector !), it was installing again.

I had stored most of the config of that machine in subversion using a great hack of a tool I wrote (more on that later), so it wasn't completely painful to restore the server back to a working condition - but it was still 4 at night when I turned in. Needless to say, I was wasted when I woke up two hours later to get a cab to the airport. We got to Kristien's new apartment (she's moving in for a couple of months with a friend of her until she knows what's happening next) and started looking at the room to put the studio in. We went out in search for a table and a chair, passing by 50 (I kid you not) furniture stores in one street, but all of them were antique or Thailand import or whatever, so no useful and affordable tables.

The next day was spent installing the machine and fiddling with the sound card and connections. The problem with digital audio is that everything has to work all at once before you actually get anything out of the system. With analog, you can connect stuff, try stuff, and listen to what the problem is, and figure out from hearing the problem how to fix it (noise, levels, ...) Digital is too expensive for a normal person to actually have lots of equipment lying around for (like other sources or input plugs), let alone testing equipment. And if everything's connected but there's no clock driving, you still don't get anything.

My particular problem was much more insidious. I had spent a good four hours fighting with ALSA and the mixer and JACK and so on to reproduce the same settings as I had at home, but not being able to get a single sound into the machine. In the end, I rebooted to start from scratch, and suddenly recording worked. I went through each of the mixer settings step by step again, noting their effect one by one - the Robert Pirsig approach - until I noticed that one toggle (innocently labeled "Channel Passthrough" or somesuch) actually made the card unable to record anything ! Yes, toggling it on and off made the card useless until I rmmod'd the kernel module. So that was the cause for my four hours of fruitless twiddling.

Seriously. It's a toggle. flipping it twice should act as if it was never flipped at all. But I guess in the quantum-mechanic world of alsa, anything's possible.

We found a decent chair and table the next day in a store in Gent, and we went back to Brussels with the chair and table under our arm on the train. In those four days of stressing to get the studio finished, we luckily also crammed in some time to go to a friend's surprise birthday party and a live karaoke evening in Gent, but that was it. Everything else was hard work.

But hey, Kristien's voice studio now works again, and sounds good, and runs on all Free Software. And Conrad, I even got her to use sweep this time !

Meanwhile, Kristien will be on-air tomorrow for the first time in a while, replacing someone else who went on a holiday to ... Barcelona ! I'll have to cut tomorrow's meeting short to be able to listen in. You go girl !

And the good thing is, since Kristien took a 60% job, and has to mainly do replacements for people going on holidays, that means she will have regular weeks off, which she can spend back home here.  Funny how things all work out on their own.

So, Kristien was offered a job at (what used to be) Belgium's biggest radio station before she left to China, and after some to-ing and fro-ing it was decided that...

It's been out for almost half a year now. In that time we've done quite a few releases of all the modules, and things are improving at an incredible rate. It seems we're collecting more and more momentum along the way, and we're getting good feedback. We get a lot of bug reports, a fair fraction of which come with patches, which is excellent. And the GStreamer team is fixing those bugs at a good rate too, so our bug total is not growing out of control.

I've been told Bastien removed the 0.8 backend from totem today, which is excellent news. When you compare the 0.8 and 0.10 versions, the difference in stability and seeking is so big that you'd never want to use the 0.8 version again. The only big thing it's still lacking is DVD support. And all that really needs is a motivated hacker. Given how Seamless, a command-line DVD player with full support for menus and subtitles, works amazingly well with 0.10, there's no real technical obstacle to finish porting the last DVD elements to 0.10 and use them from Totem.
Development of Pitivi is more focused now, and it seems to be paying off, as it's now getting more usable on my system. Jono Bacon is working on Jokosher, the multi-track audio recorder. It's good to see new outside people coming in and reporting bugs.
Jono mentions in his post that he thinks he's finding bugs because this is the first time people are writing applications for non-playback cases. This is only partly true - adder has been around since 0.4, and for the use cases people used it, it has worked fine up until 0.10. With some of the core redesign, it broke for its primary use case - adding audio tracks on the fly to be mixed. Bad, surely - but probably a side effect of 0.10's focus on achieving perfect playback. Personally, I was so busy with other stuff (Flumotion among other things) that I never got back to porting the jukebox python example to 0.10, and never noticed adder had gone south this badly. In any case, people had been using adder without problems before 0.10, although Stefan was running into problems during 0.8 for his particular use case.
By now, Wim fixed adder, and it seems to be working well, so I should finish that port of the jukebox.

This week is the first time I'm writing some actual C elements for GStreamer 0.10 myself - it's been a while since I've last written one. I'm writing a payloader/depayloader for our Data Protocol, which is used in the TCP elements to be able to connect pipelines over the network. It can still be confusing to write elements, but it's great to be able to go through all the documentation and compare with the elements you know you can trust for correct implementations in core and base.

It's also a lot more fun to develop these driven by tests I write - writing the out-of-the-box behaviour I want first, and then implementing things in the element to make the tests pass. I've written most of the elements over the last few days while on holiday in Belgium, with lots of interruptions, and mostly when being on the train or a plane, so I forced myself to write the code bookend-style (meaning, writing the code from the outside in - writing start and end of a function, making sure I mirror allocations and deallocations correctly and things like that), putting in appropriate FIXME's every time I needed to hibernate so I could dump state for later.

While writing these elements I of course ran into some corner bugs again in GStreamer, or just some things that nobody is using because they obviously do not work :) But it's getting easier and easier to slice through the multi-level GStreamer cake.

At the moment I'm working on a bug I'm triggering in multifdsink for which I need to change the way it handles streamheaders. But I'm being a good boy, and first creating a unit test for multifdsink to codify the things it already does correct for streamheaders today, before changing it to take my case into account.

The fun thing about writing tests is that you always start with the easiest possible test case, and always think "writing this test is stupid, because there's no way this can go wrong, but let's flex our mental muscles first", and invariably something does actually go wrong somewhere. In this particular case, the simplest thing I did was creating the element, setting it to PLAYING, pushing a single buffer, then going back to NULL. The element was leaking the buffer passed in.

The good thing is that most problems we find are problems like these - small leaks that happen outside of the steady state and are easy to fix once identified. In that I think our unit tests and their valgrinding are helping immensely. Instead of acting on the belief that you're working on a decent project, you can have some practical verification that your project is improving steadily.

So all in all, I'm very happy with the direction GStreamer has taken for 0.10, and the way it has started oozing quality. We still have some ways to go, but there is a very solid foundation on which we're building now. Thanks to all the GStreamer hackers over the last six-seven years !

It's been out for almost half a year now. In that time we've done quite a few releases of all the modules, and things are improving at an incredible rate....

Maybe it's the fever messing my brain, but today I found humour in strange places.
From freshmeat: Snd is a sound editor modelled loosely after Emacs.

Compiler reaches self-awareness and suggests improvements that are actually being made by others in the GStreamer codebase:
error: too few arguments to function ‘gst_event_new_new_segment’

I hope someone wakes me up when we reach twenty arguments to that function.

On an unrelated note, I bought Wired for the first time in my life. A bit fluffy, but it had a feature on the filming of A Scanner Darkly. Apparently it's ready to hit theaters this summer. Doesn't this trailer look totally *awesome* ?

Maybe it's the fever messing my brain, but today I found humour in strange places. From freshmeat: Snd is a sound editor modelled loosely after Emacs. Compiler reaches self-awareness and...

As usual, as time goes by and spring blossoms disks fill up to their maximum capacity everywhere. This past month it happened both at work and at home. This is more annoying at home since I have less time to react to it when it happens, and it messes up Kristien's ability to do recordings when it happens on the home partition of that machine.

So I decided to make a quick inventory of what I had lying around and I was shocked. Apparently, on the three machines I have running at home, I now have over a Terabyte of online storage. Wow. I still remember when we started getting Gigabyte hard drives, and how we used to marvel at news stories of Terabyte machines.

Half of this Terabyte is mirrored with software RAID as well, so underneath it's actually 1.5 Terabyte of raw storage space.
My largest drive is 400 GB, which is in my media PC in the living room, and that one was only half-used, so I could move some extra content from various /home partitions to that drive, freeing up the space necessary for more mirroring of Fedora Core stuff, and backups.

Beside that, I also seem to have collecfed 11 non-connected drives around the place. Some of them are quite useless these days - I have perfectly fine 6 and 20 GB drives for example -, one is the drive I dropped, some contain actual files, or backups, or old live systems but with a few bad blocks, and so on.

So I'm going through them now, using e2fsck to check and mark for bad blocks, so I can use some of them as emergency backups for content I already have mirrored somewhere else. I'm guessing that's pretty much the only thing slightly-broken hard drives can be good for.

Tip of the day

If you have large drives these days, remember ext2/3 reserves 5% of those drives for the root user. If you have a 200 GB drive just for, say, mirroring install trees, you're wasting 10 GB of space that never gets used. Use

tune2fs -m 1 (partition node)

to reclaim most of that space. I ended up recovering a good 100 GB all over this way, which is half a normal drive these days :) And at work, the same thing gave me a stay of execution for now.

Well, if it helps you decide on your wardrobe, I'll be wearing an "I'm with Stupid" t-shirt. As usual, as time goes by and spring blossoms disks fill up to...

Somehow the internet connection at the hotel is pretty flaky - hosts go away all the time, connections get dropped, and so on.

I couldn't post my last post because it claimed our server was done.  A traceroute showed

[root@otto qc-usb-0.6.3]# traceroute thomas.apestaart.org
traceroute to thomas.apestaart.org (64.233.185.111), 30 hops max, 40 byte packets
1  * * *
2  200.175.64.1.adsl.gvt.net.br (200.175.64.1)  17.764 ms * *
3  ge (200.175.89.129)  23.807 ms   20.764 ms   19.847 ms
4  201.47.192.33 (201.47.192.33)  18.507 ms   21.117 ms   25.016 ms
5  pos (200.139.112.86)  269.329 ms   266.176 ms pos (200.139.112.146)  33.425 ms
6  GE2 (213.140.51.237)  178.371 ms   397.465 ms   398.535 ms
7  So5 (213.140.36.65)  177.001 ms   176.764 ms   178.495 ms
8  so (213.140.43.125)  177.432 ms   179.925 ms   177.145 ms
9  Google (213.140.52.142)  392.205 ms   392.007 ms Google (213.140.52.42)  179.016 ms
10  216.239.46.19 (216.239.46.19)  178.093 ms   177.834 ms   179.362 ms
11  www.hackerteen.com.br.localdomain (72.14.236.201)  396.576 ms   391.961 ms 72.14.236.178 (72.14.236.178)  179.753 ms
12  66.249.95.149 (66.249.95.149)  189.074 ms   392.318 ms   392.449 ms
13  72.14.239.17 (72.14.239.17)  393.403 ms   393.549 ms *
14  * 72.14.238.194 (72.14.238.194)  393.850 ms   392.496 ms
15  pop.gmail.com (64.233.185.111)  180.478 ms   197.719 ms   202.391 ms

Notice how

• the local DNS server resolved my server name wrong
• not only was it wrong, but having it be so wrong as to point to gmail.com is probably not just an accident
• hop 11 has an, uhm, interesting host name

Editing /etc/hosts, adding an entry, and forcing a new traceroute allowed me to finally post my entry.

Somehow the internet connection at the hotel is pretty flaky - hosts go away all the time, connections get dropped, and so on. I couldn't post my last post because...