Present Perfect

The last few months news about streaming to iPhone 3.0 has been making the rounds. I've been holding off commenting on it for a while since I didn't actually look into it much and didn't want to base anything on hearsay. And I don't even have - or want - an iPhone!

Last week I took some time to read the IETF draft and the Apple developer introduction.

On my next plane ride I quickly hacked together a simple segmenter in Python, and tried it the next day at work to see that it sort-of-worked for about a minute.

And yesterday evening, during Nerd Night, I changed my original plans (since Wiebe cancelled, I wasn't going to work on the Spykee robot yet) and decided to go back to the iPhone streaming hacking.

After tweaking mpegtsmux to do something useful with GStreamer's GST_BUFFER_FLAG_DELTA_UNIT and teaching the segmenter to always start a new segment on a non-delta-unit, and after switching to a black videotestrc with a timeoverlay (the normal one seems to trigger a weird encoder bug in our H264 encoder, need some help from our Fluendo codec gurus for that), I started a simple stream last night:
71118

I left it running for the night.

And this morning when I got up, it was still going strong, and I left it pass the 10 hour mark:

71122

So, a good first step.

Hope to finish up some loose ends across the week to make this work inside Flumotion.

I'll leave you with my first impressions on this Apple creation:

• Naming a draft 'HTTP Live Streaming' pretending this is something new after years of Shoutcast - Icecast - Flumotion is either plain ignorance or typical Apple hubris. At least qualify the name with something like 'segmented', 'TS', or 'high-latency', Apple. Come on, play nice for once.
• The streaming system is very different from your typical streaming system. Effectively, this approach creates a live stream by segmenting a live feed into a sequence of MPEG Transport Stream segments at a regular interval. This has some benefits and drawbacks.
• The key concept is now the playlist file, an extension of .m3u called .m3u8. This playlist file is the entry point into the stream, as it lists the segments that make up the stream.
• This playlist file can reference other playlist files. This is what enables adaptive bandwidth streaming.
• One clear benefit that Apple was aiming for is that they effectively managed to separate the preparation part from the streaming part - the actual streaming can be handled by any old web server that can serve up files. I'm sure this is the main benefit they had in mind. The benefit is two-fold: first of all, it's easy and cheap to install web servers, and second, you get all the benefits of using a bog-standard protocol like HTTP: firewall acceptance, proxy and caching support, edge caching, ... Take for example the fact that a company like Akamai charges more for some streaming protocols because they have to deploy specific servers and can't use all their edge infrastructure for it.
• Another benefit is that you are generating the data for your live and ondemand streaming at the same time. The transport segments can be reused as is for ondemand .m3u8 streams. This blending of live and ondemand is something we started thinking about with the developers at Flumotion too.
• A third benefit is how easy this system would make it to do load balancing on a platform. In most streaming services, a connection is long-lived, and hard to migrate between servers. Since in Apple's live HTTP streaming the stream consists of several short files, you can switch servers by updating the playlists, effectively migrating the streaming sessions to another machine within a minute.
• As for drawbacks, the biggest drawback I see is the latency. In this system, the latency is at least the segmentation interval times three. This is because the playlist should only contain finished segments, and the spec mandates that the player have at least three segments loaded (one playing, two preloaded) to work. So, the recommended interval of 10 seconds gives you at best a 30 second latency. I don't really understand why they didn't work around this limitation somehow (for example, by allowing a growing transport stream in the playlist, marked as such, or referencing future files, marked as such), because this is where live iPhone streaming is going to catch the biggest amount of flak, if our customers' opinion about latency in general is anything to go by.
• Another possible drawback is the typical problem with most HTTP streaming systems - no synchronization of server and client clocks. Computer clocks typically don't match in speed, so in practice this usually means that the client's buffer will eventually underrun (causing pauses) or overrun (usually causing players to stop). In practice this is not that big of a deal, and I doubt on the iPhone sessions will be long enough to really make this a problem.

Whether this will become a general-purpose streaming protocol remains to be seen. I would assume that Apple is at least going to make this work in a future update of OSX. For us though it is an exciting development, allowing us to showcase the flexibility of our design to this new protocol. And while I saw some fellow GStreamer developers griping about this new way of streaming, there as well it should be seen as an advantage, since (in theory at least) the flexible GStreamer design should make it possible to write a source element for this protocol that abstracts the streaming implementation and just feeds the re-assembled transport stream much like a dvb or firewire element would do.

The last few months news about streaming to iPhone 3.0 has been making the rounds. I've been holding off commenting on it for a while since I didn't actually look...

Today we had some friends over for probably the last barbecue this summer...

This was probably our best this year. Well-prepared, all the preparations ready before people arrived, nice chunks of meat, ... We started off with frozen strawberry daiquiris and this fresh ceviche made yesterday evening: 71113

The pièce de résistance was a nice chunk of côte à l'os of 1.5 kg:
7110771110

Good food, good company, our first bottle of Perfum de Vi Blanc, the great wine we got in the Penedes, and no stress. Even Kristien was surprised - she hardly did anything and still everything went smooth!

Today we had some friends over for probably the last barbecue this summer... This was probably our best this year. Well-prepared, all the preparations ready before people arrived, nice chunks...

The phone doesn't take great pictures, but I'm sure you get the idea:

71095

A nice, inspiring view.

The phone doesn't take great pictures, but I'm sure you get the idea: 71095 A nice, inspiring view.

Woke up around 10.30 at my parent's place in Gent, after a birthday. Went to Evergem to have a brunch with my godson and his parents. Drove to Beerse, in the Kempen, for the baby shower of Willeke's fourth kid. Gorged myself on candy.

Drove back to Mariakerke for a tango evening at the enchanging Parkkaffee, a place where I used to play with the boy scouts as a kid. Our teachers were there too. Kids recognizing Kristien were all nice and polite and either asked for autographs or came to bring them pictures they made.

Then drove off to the coast to the seaside house my parents bought last year, which they've done up wonderfully, and now has a little garden area with a fishpond, and a bunch of sewage tubes where they grow their herbs, an idea I definitely must steal.

Belgium is so wonderfully tiny, you can get pretty much anywhere in under two hours!

Tomorrow I will work from the first time from this house by the sea, and it looks like everything is ready and working here to do so. Awesome!

Bonus: there's a maintained pool here and because the weather's been so good, they will keep it open next week for one more week than usual.

Woke up around 10.30 at my parent's place in Gent, after a birthday. Went to Evergem to have a brunch with my godson and his parents. Drove to Beerse, in...

I used to make fun of people making new year's resolutions. Why decide some loftly goals quickly on the last day of the year only to break them a month after ? How many people that quit smoking on New Year's Day actually make it through a year ? Why put yourself through the ridicule of your friends like that ?

But last year on one of those rare weekend days I was taking stock of my life I did come up with some things that I wanted to get done or improve on. And as the end of the year was approaching, I was wondering how I could make these goals attainable. How could I make the ridiculous concept of New Year's Resolutions work for me ?

I theorized that, for them to work, I needed:

• goals that I actually want, that are missing from my life
• some alignation with where I want my life to go
• some concrete target for that goal
• a way to check my progress against the goal
• a commitment (to myself or others) that these goals are important and take precedence

For long-term life goals, I want to enjoy more culture, enjoy life, and be healthier.

Roughly, these were the resolutions I made:

• standup comedy: I've always been interested in going to see it, and for some reason I never do. And when I see stuff on TV I keep thinking I should go. Culture and enjoying life.
• dance more tango: a requirement before we go to Argentina IMO. Culture and enjoying life.
• cook more: I very much enjoy cooking, and I'm not bad at it either, but I want to get better at it. Culture and enjoying life.
• rerip and organise my music collection: since moving to Barcelona I have bought 400 CD's that aren't in any computer, and everything else is all over the place. Enjoying life.
• learn catalan. Culture.
• see more movies Culture.
• sleep before midnight: I am a perpetual late sleeper, and I need to start changing my sleeping patterns. Health.
• improve my GTD use: I'm getting good at it, but I'm sure I can make this work better for me. Health (mentally) and enjoying life.
• sports - play basketball again. Health.

Why does it help to make these resolutions ? Because it turns into an easy list that helps you make decisions at those times where you need to decide something quickly or are making plans for the week.

Do you stay at home reading books or find someone to go to the movies with ? Movies is a goal, so go. Do I accept a job interview meeting on Tuesday at 15:00 or do I stick to the basketball planned ? If I'm behind on basketball, then the meeting needs to move. Shall I eat out tonight ? If I'm behind on cooking, then no. It's actually liberating for an eternal undecider like me to just decide based on goals set earlier.

So, with that list in mind, I formulated some more concrete goals for some of them:

• standup comedy: go see four shows this year. Let's start gentle, you have to get tickets well in advance over here.
• cook more: cook at least one meal a week. Ambitious, given my life, but worth trying.
• rerip and organise my music collection: no concrete goal set, since this is a project that's been on my mind for a long time, and code will need to be written.
• tango: since dancing requires Kristien taking lessons, let's start there. 2 courses of tango classes this year. Good goal since there are only three courses.
• learn catalan: no concrete goal set. The voice in the back of my head suggested I spend at least an hour reading through language courses, but I didn't actually set that as a goal. Probably shows a lack of real commitment to the goal!
• see 40 movies
• sleep before midnight: do so half of the working week days. Not that easy a goal at all, since my two flights a week usually have me land pretty close to that mark. To make it easier to track, and allow to make up in weekends, this comes down to 2.5 days in 52 weeks, or 130 days.
• improve my GTD use: lower number of open tasks over time.
• basketball: play 26 times in a year

An important part for me is to take stock once in a while. Where was I when July started, halfway through the year ?

• standup comedy: 3 shows seen, 2 planned for July: 100% completed!
• cook 52 meals: 18 times. I cheated by also counting the 6 meal course on New Year's Eve (only as 1 though). so, 69% there, and I need to step it up to make it.
• rerip and reorganise music: even though there is no concrete goal set, I'm well on my way. I have various projects going on, from writing a correct ripper (which Linux still lacks) to getting Lego Mindstorms sets to create a CD ripping robot to reviving the old DAD project into a new Django-based one to finally writing a decent mixer... Wondering if I should set as a goal something like 'rip all newly bought CD's to FLAC using a robot and importing and rating them in the new DAD'. I'd rate myself halfway there right now, but some stepping up is required.
• learn catalan. As I said, no goal set. Practically speaking, didn't do anything realistically useful. I need to either drop the goal or set a concrete task.
• tango: one class taken, one class booked for september onwards. so, on track for 100%
• see 40 movies: saw exactly 20 in the first half of the year! on track for 100%
• sleep before midnight: as I feared, pretty pathetic. I don't have a complete tracking, but looks like I managed 18 times out of 140. That's like 36% by the end of the year if I keep this up! However, I seem to be cutting it close all the time: 20 times between midnight and 00:15, and 17 times between 00:15 and 00:30. I could have made the goal if I went half an hour to bed earlier, on average. Need more resolve!
• improve GTD use: I was missing the tools to track this goal, so I wrote some. I need to fine-tune them a little, so I can better see how I'm doing each day. Sadly, for now, they seem to indicate my task list is growing. I need to better define the goal I'm working towards to - something like 'over time reduce the number of U:3 to U:5 tasks in the queue'
• basketball: we finally got together at work with a few people, and now rent a court weekly. I managed to get there 10 times, so 77% so far. I specifically set the bar low to only half of the weeks, knowing that I have all sorts of roadblocks for going to play: emergency meetings, not being in Barcelona, Spanish fickleness (like, the guy not showing up at all because it rained a little in the morning), and so on. Still, I should be able to make this goal by the end of the year, possibly by playing some more here in Belgium on the court in front of the apartment.

So, half-year status:

• 1 goal already completed! (comedy)
• 3 goals clearly on track for completion (tango and movies), a third not well defined but probably getting there (music)
• 2 goals around 70% (cooking and basket), could still be made if I really step it up
• flunking 2 goals clearly; learning catalan and sleeping before midnight. Need to seriously consider what to do about them.
• 1 goal did not have the tools required to measure properly; now need to actually track the task again.

Conclusions:

• the measurable goals seem to be easier to attain - setting a clear way to track progress clearly helps following through on your commitment.
• it is important to track those measurable goals - I wouldn't have guessed I made the movies one for example from memory. Also, writing them down makes it feel like an achievement.
• the tasks that seem to be set more by my conscience than by my desires seem to languish (sleep and catalan). Clearly I need a better conscience or re-evaluate the tasks.
• one task without concrete goals is getting done anyway, just because it's been a long desire to get it done (music). Still, it would help me focus to set a concrete goal.
• concrete objectives can be re-evaluated in light of the desired goal. For example, the original goal was to be healthier by doing more sports. I picked basketball because I always liked it - but if needed, I could redefine the goal in terms of another sport if that makes it more attainable - as long as I don't cheat by making the goal easier.

So, next things to consider for me:

• start thinking up new or extended resolutions for next year
• decide what to do with Catalan - leaning towards forcing myself an hour a week for the next weeks
• do end of September evalutation
• cook more than once a week.
• get to bed just a little bit earlier!

Anal-retentive - moi ?

But enough about me, let's talk about you. What goals did you set this year ? How are you doing on them ? What helps you stick to them ?

PS: completing this blog post marks another U:4 task of my GTD list, yay for me!

I used to make fun of people making new year's resolutions. Why decide some loftly goals quickly on the last day of the year only to break them a month...