Present Perfect

Day two of the internal platform training sessions. Today is hacking and bugfixing day.

I wanted to take a stab at the task of creating an RPC interface to expose our Perspective Broker interface. I have very little experience with RPC systems (apart from PB, and moap's use of Trac's RPC) so this was a good opportunity to get my feet wet.

Smart hacking is lazy hacking, so I started by Googling. After some false positives, I found a Twisted JSON-RPC project, and since it was maintained by Duncan McGreggor it gave me hope that it would work.

And so it did. I wrote a simple adapter object that takes an instance of a PB root and proxies jsonrpc_* calls to remote_* calls.

This is just a proof of concept; obviously there are many caveats. The main thing being that currently you can only use it for remote_ calls with simple objects that txjsonrpc supports (although it looks like for example it supports a deferred, so sweet).

Obviously, one of the attractions of PB is that you can transfer objects. At the least this bridge could be extended to support getting references to objects and then invoking methods on them or passing them as arguments.

In any case, good enough for a one hour hack.

The code is in my tests repository and you can check out with

svn co https://thomas.apestaart.org/thomas/svn/tests/twisted/pb2jr


As for actually using it - after writing it Jan and I discussed how it should be used, and he told me he doesn't actually want to run this in the same process where we run our PB interface. Instead he wants it to acts a proxy, which would at best mean doing code inspection or importing of the PB Root in the proxy to be able to provide the JSON-RPC interface dynamically, and you wouldn't be sure if the running code's PB Root is the same as whatever you have on disk in your proxy.

I'm not very convinced about the coupling argument, because this is just a thin layer on top of the PB server, and everything will end up going through the PB server anyway, so I don't see any gain from decoupling here.

I lost interest while discussing, so I'm going to leave it in its current state for now. Although I would have preferred to just plug ahead, do some of the cooler introspection bits, provide a web UI to look at methods and invoke them, and so on.

Day two of the internal platform training sessions. Today is hacking and bugfixing day. I wanted to take a stab at the task of creating an RPC interface to expose...

Fedora comes with NetworkManager enabled out of the box. AFAICT it starts the network as part of a service script in init.d, then later on after logging in it reconnects as a user. At least, that's what it looks like to me.

The problem is, I just came home, and for some reason the media server was down. The media server also has the primary of my new DRBD setup, and bringing the machine back up didn't bring back the DRBD sync.

Digging deeper into /var/log/messages, it turned out the network wasn't active when drbd got started, and so it failed to connect to the peer.

Obviously, if this machine reboots I want drbd to Just Work.

NetworkManager's init script is S27, drbd is S70, I would have expected the network to be up by the time drbd kicks in. It looks like this wasn't the case. And I can foresee situations where NetworkManager re-connecting after my (automatic) user login not helping either if anything in the background is trying to connect to the network.

What should I be doing instead on a machine like this ? Remove NetworkManager entirely ? Is that even still possible today ? How do you set it up?

Fedora comes with NetworkManager enabled out of the box. AFAICT it starts the network as part of a service script in init.d, then later on after logging in it reconnects...

Drives fail.

Somehow they fail more often around me.

I used to swear by software RAID-1. I loved it because if either disk fails, the other disk still contains all the data and you can get to it and copy it off and so on.

Except that in reality other things get in the way when something fails. Drives may fail at the same time (because of a bad power supply, a power surge, a computer failure, ...). A particular low point was the time when my home server's RAID had a failing disk (which had all my backups on it), and I took out the drive, and added in a new drive to start copying, with both drives hanging out a little outside of the computer case, and at precisely that point a motherboard box dropped out of the shelves two meters higher, and landed with one of its points right on top of the good RAID drive, breaking it. I lost all my backups.

So I learned the hard way that most problems with RAID happen precisely at the time you need to phsyically manipulate the system when one of the drives fail.

Ever since I was wondering how I can do my storage and backup strategy better. I have some other ideas on how I want to back up my stuff from four computers in two different locations (three if you count 'laptop' as one of them) depending on what types of files they are (my pictures I need to make sure I don't lose, while most music I can re-rip if I really have to), which should go in a different post (and if you know of any good descriptions of home backup approaches for all these files we seem to collect, please share!).

But it's clear that part of the solution should involve storing files across more than one computer, preferably in a transparant way. I thought about setting up one drive in one machine, then doing a nightly rsync or dirvish snapshot of that drive, but with lots of big media files moving around it might not be the best solution.

So I was excited when I came across drbd, which implements a distributed file system that mirrors to a secondary disk over the network.

I got two new 2TB drives last week, installed them in my media server and home server (which took quite some fiddling, including blowing out a power supply which exploded in a green flash, sigh), and read through the pretty clear and decent docs, and after 50 hours of syncing empty disks, I now have this:

Sweet! Now to actually copy some files and run some failover tests before trusting my important files to it...

Drives fail. Somehow they fail more often around me. I used to swear by software RAID-1. I loved it because if either disk fails, the other disk still contains all...

Today is an exciting day at the office!

Jean-Paul Calderone (exarkun from Twisted arrived in our office this morning to give our development team an in-depth training on Twisted.

For now, the schedule is two days of Twisted training, one day of code sprinting on Twisted, and two days of consulting on our platform and various issues and projects we have.

For us it's exciting to get a training from one of the top hackers in Twisted, and I hope it is exciting for him to see a commercially successful use of the project he worked on.

But mainly I'm looking forward to a bunch of days of high-level technology talk.

Time to get started!

Today is an exciting day at the office! Jean-Paul Calderone (exarkun from Twisted arrived in our office this morning to give our development team an in-depth training on Twisted. For...

As we are deluding ourselves here into thinking it's snowing in Barcelona, I thought it appropriate to post some videos from the past few snowboarding trips.

Coincidentally, this is my first foray into the HTML5 video world - more on that later.

Let's start with my favorite, the one where I show off how years of gymnastics in my youth help me keep my body in one piece:

(Also notice the cool new orange snowboard pants that I settled on. Snowboard fashion was really boring this year, mostly grey and black only, with some ugly flashy colours as exceptions. I leave it to you to judge whether orange is one of them).

We spent eight full days in Tignes, France, with only about three days of sunny weather, and the rest filled with clouds and snow.

My goal this year was to learn how to do a 180. With the help of an instructor, that's exactly what I did! Here's an admittedly simple one - all the good ones are not caught on video.

Here's a more aggressive one with a bad ending:

A few weeks before our snowboard trip, we also had a business planning weekend which included one day of skiing. Xavier risked life and limb following me around with his iPhone to record this. It's not the most exciting descent in the world, and he ended up missing my one fall in it, but I was surprised to see how short the whole descent really is if you don't take any time to stop!

And here are Xavi and me relaxing over cheese fondue and raclette the day before the skiing:

Some notes about the HTML5 video part:

• there is an enormous difference in colour between playing Ogg in Firefox, MP4 in Safari, and MP4 in Quicktime, on the same MacBook. My pants range from a soft orange to a bright red. Something is obviously up!
• To learn about HTML5, I started with Dive into HTML5 Video, then learned about Video for Everybody, some web code that handles all of the stuff I don't know how to do for me and just makes sure the video can play on Firefox/Chrome/Safari/iPhone/...
• Then I looked for WordPress integration, and found a plugin with a long name that implemented most of Video for Everybody. I modified it a little to do something more sensible for the poster image in case it's external, and to accept .mp4 as an extension instead of .m4v (which is not suggested by Dive into HTML)
• I configured our transcoding platform to generate the three types of output file needed to support HTML5: the thumbnail, Ogg/Theora/Vorbis, and .mp4 with H264 and AAC.

The embedded video should work fine in Firefox/Safari/Chrome/iPhone/Opera (except in Aitor's "I plug mplayer into Opera" case), and work fine in Explorer too where it falls back to Flash.

I couldn't get this to work in Android. 2.0 is rumoured to support the video tag, but so far no dice, and I couldn't find a single HTML5 video page online that the Android phones over here can play. If you can see these videos embedded in Android, or know what I should to fix them, please do let me know!

As we are deluding ourselves here into thinking it's snowing in Barcelona, I thought it appropriate to post some videos from the past few snowboarding trips. Coincidentally, this is my...