Present Perfect

Weekend

My ex-boss and his wife came over, together with a couple they knew. The guy was going to ask the girl to marry her and Kristien had arranged a nice table on the beach for them. But before that, we went to the park and drank six liters of Caipirinha between the six of us. And I was pretty much on an empty stomach...

BuildBot

Warren did a new release this week, and I'm happy to see a bunch of my patches have gone in ! Also, I think our current buildbots look very spiffy.

I had started learning arch (using baz) for developing on BuildBot. What I did was create a branch off the last release point, and then create a branch per feature I was hacking on. Then I made one big branch that collected all of those features into a fluendo branch which I used for deployment. That worked out pretty well.

However, now with the new release I wanted to again branch the release version and finish up other features against the latest release. And for easier upstream merging, I also want to merge in these patches against a branch that tracks buildbot head. So I guess I'll have to redo my whole tree locally.

I'm enjoying hacking on Buildbot a lot. It's a nice project, it has fairly nice code, it just misses some documentation. There was however the initial problem of getting Warren to look at what I was doing. In the beginning I was describing what I wanted to do, I sent a few patches, and then I was waiting for comments because I didn't feel like going off very far on my own copy and then not be able to merge back because he wanted it done differently.

But meeting up in person at PyCon obviously helped. I got to explain some of the problems we deal with for GStreamer (stacked dependencies, conditional builds, ...), and get some ideas from him for those. After PyCon, Warren took some time and started looking at what I was doing. And I guess me not giving up and doing documentation work must have helped too :)

And Twisted is just sooo damn nice. The last time I had this sort of click with a project, where stuff just felt into place, and I got the feeling that here was a project that had some really good ideas, and some excellent people, was with GStreamer.

Discovering Twisted is like having that sense of wonder all over again.

Flumotion

We're looking at roping in some potential customers so I get to spend some time setting up tests and stuff. The upside of that is that we have some more stuff to play with and some nice ideas we can execute. The downside is running into the bugs you were trying to avoid in your own software :)

So today I got to buy a barebone 64bit machine to use for one of the demos, and that means I can finally start doing some of the 64 bit fixing myself. I was particularly embarassed a few months ago when Zaheer couldn't get his streaming up and running because of a bug in my code that I hadn't encountered myself in GStreamer.

As for Flumotion, currently trying to spend most of my time hacking on the RTSP stuff. It gives me ample opportunity to dig a little deeper into the internals of Twisted. Implementing from RFC is boring and rewarding at the same time. It gives you a chance to work on some other parts of the set of development skills one tries to build, because the hard work of designing is mostly done already.

Holiday
Next week Kristien and I are going to go down to the south of Spain to visit Malaga, Cordoba, Sevilla and Granada. If any of the Spanish people want to offer me some tips on what to see when I'm there, feel free !

Weekend My ex-boss and his wife came over, together with a couple they knew. The guy was going to ask the girl to marry her and Kristien had arranged a...

James, I didn't know bittorrent used rar internally. Or, let me rephrase - are you sure it does ? I've downloaded single files, and multiple files, and the multiples end up in a directory, one by one, and the first one is usable as soon as it starts writing the second one. You sure you didn't just get a torrent for a .rar file ? I mean, it could be that person hosting the torrent rar'd up his .avi files and serves it up that way. Which would be dumb, but possible.

Also, most video files are actually not streamable if you haven't downloaded them all. Unless you force index rebuilding which is really slow.

As for LCA, could anyone who's physically there please drop me a line ? I'm trying to figure out where I can follow the broadcasts, and apparently jdub thinks getting married is a reasonable excuse for being offline. Slacker !

James, I didn't know bittorrent used rar internally. Or, let me rephrase - are you sure it does ? I've downloaded single files, and multiple files, and the multiples end...

Algernon recently posted a reply to Christian's claim that flumotion is the best choice for streaming. (This is of course an example of Christian's youthful enthusiasm, crystallized in his "Promote early, promote often" motto.) He did make some good points, so I wanted to address them. I just wanted to take the time to give the anwer the time it deserves, so I'm a bit late.

Let me start by saying something that needs to be said: Icecast is *GREAT*. It's been around for quite some time, and back in the day when I was a student (my personal version of "When I was in the army...."), we put our university radio station on the Net with it as soon as it worked well enough. We were definitely the first in Belgium to stream live radio on the net, and probably the first in Europe, though it's hard to verify such a claim because there is very little info on this. But this was around 1995, I think (geez, where has the time gone :)) Also, one of the most active current hackers, Karl Heyes, is a very nice guy, always willing to help, and has given me some useful insight. Icecast has a good community.

I'll say some more about icecast later, but let me first address algernon's points and offer suggestions on how they can be done.

• Flumotion does not currently support more than one stream on the same port.

  This is correct, but a pure implementation choice because we were lazy and knew this is easy to fix. It's just that there are so many easy ways to choose from to fix this :) The main limitation here is that only one process can listen on a port. As soon as I get round to this, I'll do the most straightforward fix: have the streamer component take feeds from multiple other components, have the main thread listen to the port, and have Twisted serve one Resource per feed, on the same port, but different URL's. It's not a hard thing to do, it just needs doing. Estimated developer time: 4 hours.

• This also means - as far as I understand it - that if I want multiple streams, I need multiple flumotion instances.

  Not really - you start just one flumotion-worker binary on that server. This worker binary can fork multiple components all by itself. So you just put some more streamer components in the flow, as many as you like. Only one binary to manage. Unless by "instance" you meant to include these forks as well, but this is standard practice - look at Apache for example.

• There's no easy way to send a stream to flumotion, from a random GStreamer application. (...) For this, I would need to learn python, which I don't intend to anytime soon.
  Actually, there is. Here's what you can do to achieve this:
  1. start up Flumotion and the GUI
  2. set up a stream with only a video test producer
  3. verify that runs correctly and export the XML config
  4. edit the XML config, replace the videotest producer with a base producer type that takes pipelines
  5. use tcpclientsink for the pipeline with a given port and the GDP protocol
  6. restart flumotion with this new config
  7. producer component being hungry - nothing is feeding it data yet
  8. start up the following GStreamer pipeline: gst-launch videotestsrc ! video/x-raw-yuv,format=(fourcc)I420, framerate=5.0,width=320,height=240 ! tcpclientsink port=3333 protocol=gdp

  And there you go ! Total implementation time: 13 minutes, including feeding the cat (and at 9 am in the morning). Number of lines of Python code hacked: 0.

  I was surprised myself actually, I would have expected to have to fix at least one bug in the code. Thanks for prodding me to actually try this out. Now obviously this is not the "best" solution. If you stop the producing client, you need to restart the producer component as well, because the simple pipeline component type has no way to recover from the broken connection. And that's where you move from "quickly test something out with what's available" to "turn it into a 15-line component". Exercise for the reader.

• Flumotion does not have fallback mounts yet..

  IIRC, the fallback mounts can only be whatever the same icecast server is serving. So with the solution to the "multiple URL" problem, this can be done immediately as well. Twisted doesn't care *what* thing is providing a Resource tied to a path. So we'll probably get to this soon.

• With icecast, I can easily make relays.

  IIRC, Icecast has two types of relay. One is where you import a stream from another server that can be any HTTP streaming server: Icecast, Shoutcast, ... This one is easy to implement: you use a pipeline producer that uses gnomevfssrc to get the stream. There's only one catch at the moment - there needs to be a GStreamer element to parse the Ogg stream properly so that it can figure out what the streamheaders are. oggdemux ! oggmux should do this, but it has a bug, which I noticed while trying this just now. So, that's the fix needed. (Estimated developer time: 6 hours)

  The other mode is icecast-specific: icecast pulls in an .xml file from the other one that describes all streams, and then relays them all automatically. This is also doable, but would mean including icecast-specific support. This is something we'd really like to do - make Flumotion work really well with icecast. I see icecast more as an ally than as competion. The competition we see is Windows Media Server, RealServer, Quicktime Server (and HelixServer and Darwin Streaming Server). I'd have to look at exactly how this relay info is pulled though to say how long it would take to do this.

  (Zaheer reminds me that in Flumotion itself, it's of course even easier to do relays. You can create as many streamer components in your flow as you want, connecting to any other producing component, and the streamer can be on any machine in your network. And all of that is managed from the GUI. So in Flumotion, it's easy to do relays. We still need to make something nice though to go through firewalls transparantly - an advantage Icecast currently has because the clients are push-based).

• Icecast has this nice stuff with me being able to apply random XSLs to its XML output.

  If you tell me what this means, I'll cook this up for you. Estimated developer time: 1.5 hours

• Icecast has a usable C library that can be used to feed it

  As shown above, GStreamer is that library.

• I couldn't, for the life of me, get flumotion into a usable state under FreeBSD. It's not flumotions fault, per se, but of its dependencies.

  Ok, this is a point where I might get defensive and ranty, because it's not a fair point IMO. We all have limited resources and time. Most focus is spent on Linux, and this is the same with many projects. BSD users know this and are typically more used to - and apt to - fix problems themselves and get stuff installed. We know GStreamer and gst-python runs on FreeBSD. So does Twisted. We haven't tested it ourselves, we don't run FreeBSD. But it can't be too hard. You can't get a lot more portable than Python.

  Of course, if you don't tell us exactly where the problem is, there's nothing we can do to help. And we'd love to help it get set up. But it's not fair to blame a vague "I can't install this on FreeBSD" problem on Flumotion :) Talk to us and we'll help you get it fixed.

• While I think flumotion has a damn good design, I think icecast follows my ideals better. As the UNIX paradigm goes: it does one thing, and does that well. It doesn't try to be a complete streaming solution. It is simply a streaming server. And that I love. I know that flumotion is usable this way too (or will be, as soon as there is a flumotionsend element), but it does stuff I do not need at all. (...)That makes it harder to use for what I need, and makes the developers focus on things I will never ever use, so it lacks key features I do need.

  There are pros and cons to going either way. We think in the end that a product that can carry the task through from beginning to end is better. It's better integrated, it has a unified feel, and it allows you to do more powerful stuff. If one of my icecast streams is down, I need to leave the Realm of Icecast and go see what happened to my application producing the data. In Flumotion you just see in your GUI (which soon will have a notification area applet) what's up. However, you still can use it in "be only a server" mode, and yes, we want to make that work correctly. Saying that we focus on things you will never use, does not automatically mean we should lack key features you do need. You don't seem to be streaming AAC, yet icecast has spent time on getting that working.

• Oh, and you might probably ask why I didn't report these to the flumotion developers. That's a good question, and I admit I should have done that. However, I had deadlines, and that didn't allow me to properly report issues.

  This is indeed a crucial point. We're all strapped for time. You had a deadline so you didn't inform us of these problems you had. We have deadlines too. Yet here I am making the time to address your problems, even if it's after the fact. We care about our project. We care about people using the project. If they have a problem, we want to help them. It doesn't take much to get us to go out of our way to do so. Here I am replying to your points. Someone recently wanted to try it on SuSE and he was missing deps and built stuff by hand and something below the stack segfaulted. So I started looking for someone to make SuSE packages and now we have them (I still need to test them though, but VMWare seems to lock up when trying to install SuSE).

  You're in our IRC channel, you see us discuss. Feel free to bring up these points when you run into them, even if it's just cursory or you don't have time to go in details.

So here's the simple point: these issues you have you can let us know on IRC, on the mailing list, in our issue tracker, and we'll comment on them. When faced with our own deadlines, we don't spend the time to psionically probe all people for problems they might have with Flumotion. But we still spend time addressing valid points about it when we know what they are.

But don't take my word for it ! Try it out. Give us some stuff you want to see happen. And if you prefer to wait until we're done on our own, that's fine too. Let's meet back in a year and see where Flumotion is, feature-wise.
Flumotion is almost a year old. In software land, that's young. For a normal use case (which you indeed aren't, since you prefer to hack yourself and code up your own streaming source), I do think it's a better match than Icecast.

You don't really need relaying, you can have multiple streamers streaming the same stream, for the same effect, except that you have one less failure point. You have a nice GUI and wizard that makes it easy to set up. And the design underneath is flexible enough to handle more complex tasks, and the code is simple enough to get started with it quickly.

Now, back to icecast. I haven't used it much lately because I do not work for radio stations anymore since I started for Fluendo. The friend I originally did the streams with now works for an ISP, where they have 5 round-robin icecast servers with about 1200 streams each. He says the machines are easily pulling 60% CPU and more, which is why he needs to round-robin in the first place. One of our customers is using Flumotion to stream to 4000+ clients, pushing 80 Mbit. The CPU usage is below 5%. I wish I could test icecast in these conditions myself to see where the difference lies. Maybe they use an inefficient polling method ? Maybe something else is completely different ? But when he asked about it, they told him that they don't really have many users that serve that many streams, and they don't focus on those conditions.

One other thing that annoys me is that, while Icecast is closely affiliated with Xiph.org, it has always supported mp3 streaming, and now has added AAC streaming. You can sort of make a point for mp3 - it was around before Vorbis, it was the first thing icecast supported, ... But adding AAC does not gel well with Xiph.org's mission. Flumotion from the start has made it clear that we do not only focus on free formats. We can't do that, business-wise. But we actively support Xiph.org and the codecs they make, and we make sure they are first-class citizens in our world. And that this support is available for free. If you want non-free codecs support, you'll get non-free code. I think that's fair, and promotes free codecs better than Icecast does. We make people aware that there is a cost involved, both in money and in freedom, when using non-free codecs. And we go out of our way to make it easy to avoid that cost by using free codecs.

Flumotion is an active and young project, and I hope this shows that it's a very flexible project, easy to extend, and that as developers we are very open to other people trying out ideas and suggesting stuff we haven't thought about.

Algernon recently posted a reply to Christian's claim that flumotion is the best choice for streaming. (This is of course an example of Christian's youthful enthusiasm, crystallized in his "Promote...

How to change from a bunch of hackers to a corporation

Christian: Wim, can you mail X about Y so Z gets done ?
Wim: Sure. Write the mail and I'll send it.

Bagels

Bagels are sithspawn. They are soooo damn good with that lovely cream cheese, but there's absolutely NO WAY you can eat them without getting the cream all over your face. Two weeks after Washington and I still have a craving, I need to find me a bagel shop here.

Cat
Growing up nicely. She's been very playful since day two, and has started exploring the rooms beyond the living room. We're still not letting her on the terrace though. She has made her dumps in the litter box every single time. The only problem left is that she starts digging and running through the litterbox afterwards, smearing out the dumps and getting lots of poo on her body. So we stand by and clean her up when she gets out. And we clean out the litterbox in the morning and pray to God the place is still clean when we get back.

Stien
Had a nice little birthday party last weekend. First BBQ of the year ! Christian seems to be under the mistaken impression that it's perfectly fine to not call to cancel your coming when you are invited as long as you think it's not a birthday party. Also, he thinks it's fine to not think it's not a birthday party when he actually got an invite that clearly said it's a birthday party. Can someone please clue him in on how girls think on this subject so I can get unstuck out of the middle ?

Kristien has taken on a part-time at CitiBank, the black hole for Belgians in Barcelona. Two weeks of 8 hour training with only two fifteen minutes break and an hour lunch break. I feel for her. (Well, I already did of course) Hang in there honey !

Birthdays
Sometimes I get that strong feeling that someone I know is having his or her birthday coming up, or today. The thing is, I know exactly what date the birthday is on. The trouble is of course remembering who's birthday it is. So, we had Heleens birthday on April 11 (Heleen is an ex-girlfriend who by coincidence is also living in Barcelona) and Kristien's on April 9. Those were accounted for. But who was the person that April 10 was trying to ring a bell for ?

Some thinking later I realized that I had another ex-girlfriend who has her birthday on the 10th. Eerie. And they match in chronological descending order. Coincidence ?

Maybe we shouldn't let too many people in on this secret though - next thing you know all the women born on April 8th will try to break Kristien and me up. And I'm quite happy with my current girlfriend, thank you very much.

Music

I put in my LCD Sound System disc-that-can't-be-called-CD-because-of-the-copy-protection-scheme and the computer tried to make me believe it was Bloc Party. Coincidence ?
Of course, the damn copy protection scheme also makes the drive unusable - it has errors mounting the data part and after that it fails to eject.

Firewire
Pete seems to not be very convinced of my patch-steal-fu when it comes to kernels :) However, after some culling and fixing from the -mm branch, this patch fixes it. The test case runs 1000 open/grab frame/close iterations, and not a single traceback or lockup. Incidentally, Pete, why do you hope and expect Firewire to die in the marketplace ? I don't see that happening any time soon, and moreover, I'm not sure what you suggest the alternative would be.

Now, being a kernel n00b, can anyone tell me who I should poke to make sure this goes into Fedora Core 4. I have heard the word "upstream" being mentioned, combined with the words "haha" and "chaos". Somebody please enlighten me :)

How to change from a bunch of hackers to a corporation Christian: Wim, can you mail X about Y so Z gets done ? Wim: Sure. Write the mail and...

Alunya

So, Kristien went to get the cat yesterday. He's about five weeks old, has grey-whitish stripes and a few cute gingery spots on the top of his head. He's more active than we expected - in one night he's eaten three times already (out of the zero times projected by the woman handing over the cat) and he was already sleeping happily on our belly while people were being shot on TV in the Sopranos. Pictures up soon.

How not to do it
Yesterday some guy came into the #gstreamer channel. This was the start of the discussion:

<NickWithheldToProtectTheInnocent> I'm almost finished reading the documentation,
and I must say that the terminology of gstreamer is very lousy, much better metaphor names
could be used that would map to better real world object. Like the ones used in home theater
example. Don't you guys think so ?
<thomasvs> NickWithheldToProtectTheInnocent: no, I don't think so.
<thomasvs> NickWithheldToProtectTheInnocent: but feel free to write some documentation
that uses home theater terminology.
<NickWithheldToProtectTheInnocent> how does "bin" and "sink" make sense to you
without reading the documentation.
<thomasvs> NickWithheldToProtectTheInnocent: how do you pee cleanly without
your daddy telling you ?
<thomasvs> NickWithheldToProtectTheInnocent: how do you learn that it's not
acceptable to poo in public ?

Now, suppose you're having an open house day, and someone walks in, walks over to the group of people standing in your living room talking to each other, and then says loud enough for everyone to hear "This room is very ugly, it could look a lot better. And who puts a fishtank in the living room when everyone can see a pool table would be a much better fit for this room ?"

It's one thing to deliberately start off on the wrong foot with people you don't know at all. If you do, you'd better have a personality that can make up for it. Now later on, this guy goes on to say that he would use "metaphors that could map to real world objects, that would ease the learning process obviously. Like signals and slots of Qt. something that you can connect other thing to."

Now, I had before not taken a look at exactly what slots were in Qt, but I always wondered what they would be, since the only real-world image I had of slots was "some opening you can slide a wide and thin object into". Like, a slot machine, or a letterbox. So he starts off by saying that our terminology is not real-world enough (though to me, a bin is something you put stuff in, and a sink is something in which a stream disappears - I have one of those in my kitchen and bathroom), but a slot is fine.

It might be possible that Vegas has slot machines connected to the ground somehow to prevent from getting stolen, but in the end, a slot being basically the absence of something in something else makes it really hard to *connect* something to it.(1)

Anyway, if you're new to a project:
- be nice when you get in
- if you're not nice, but think you have a valid point, defend your point when people question it
- make sure you use a valid example to make your point. be prepared to back up if you chose an invalid example
- don't call one of the project people "stereotypical" when they challenge your point's validity. It's very stereotypical.
- don't tell them to stop wasting your time. you just came in and wasted theirs *without* having a history of good will on your side.
- don't expect to not have to read *any* documentation. Saying that "bin" and "element" should mean something to you without reading any docs is just dumb. How do you know that these concepts exist in first place ?

On a related note, I hope the cat will learn about the acceptability of pooing in public.

kernel

I got fed up about the fact that both our FC3 kernels and matthias's production kernels cause a kernel fault every time you start or stop capturing from firewire, and sometimes they even lock up hard. It keeps us from hacking on nice Flumotion bits related to Firewire. So I decided it was time for me to do the hard work and start kernel debugging. Boy, what a world of pain :)

I managed to get a serial cable linked up between the test box (which sadly is also our firewall and svn repository) and the workstation. After some tweaking, kernels started spewing all sorts of info to my other machine. But apparently firewire dies without an oops or anything. The machine just locks up and that's it.

So I started hunting for info, similarities in bug reports, kernel patches, ... I also managed to rebuild Fedora Core kernels with my own patches in a fairly sane way, but it's still painful. Yesterday night I tweaked the config a little to give me more debug info, and this morning I will bother my co-workers again with on-off internet. So if you see us go under ...

(1) This is no attack on Qt. It's a fine framework afaict. The guy gave "slot" as an example, and it just happens to be a Qt thing.

and the talking leads to touchingand the touching leads to sexand then there is no mystery left Alunya So, Kristien went to get the cat yesterday. He's about five weeks...