Present Perfect

I just received confirmation of my ticket to FISL 7.0 in Porto Alegre, Brazil.  If everything goes according to plan, I will be there to give two presentations - one about GStreamer and one about Flumotion - though the latter one now seems to have disappeared from the website.

I seem to be in some illustrous company, so I should make sure I do well if I manage to get there.

People I am sure I will run into over there are Johan Dahlin (we can have ourselves a little Fluendo starters reunion), Jeff and Pia, Nat and Miguel, and Lucas Rocha, as well as Pablo and Fabricio who got me there.  I don't know if it's still going to happen as planned, but the idea was that a lot of the rooms would be streamed live, and some of them with simultaneous translation, to have Spanish, Portuguese, and English.  That would be pretty cool.
If you're there, let me know if you want to discuss about GStreamer, Fluendo, Flumotion, GNOME, Fedora, or anything else.  I have no idea what to expect, except that the schedule looks good, it seems there will be a lot of people and it can only prove interesting.

I just received confirmation of my ticket to FISL 7.0 in Porto Alegre, Brazil.  If everything goes according to plan, I will be there to give two presentations - one...

So, I've been delaying a certain problem for a few months now, hoping that any of my fellow monkeys at Apestaart would address the issue first. Of course, they were doing the exact same thing. And I guess I'm the first to cave in.

The issue is simple - our previous server was running Fedora Core 1, and I had a hacked-up Gallery 1.3 install in my website. Some of the others also were running various Gallery versions. At some point, our hosting company went bankrupt and got taken over by another one, and in the process we migrated to a Fedora Core 4 server. That migration was mostly succesful, save for two problems - Gallery 1.3 doesn't work with PHP 5 (the version shipped in FC4), and all my personal CVS history was deleted.

Having more or less dealt with the second issue before, it was now time to attack the first. Have I mentioned before how much I hate web software ?

So, after browsing the docs for Gallery, and noticing they have a largely rewritten Gallery2 which looks very nice, has unit tests (!), and seems to be decently put together, I started looking for upgrading documentation. And after some puzzling, it seems that my best bet is to upgrade from 1.3 to 1.5, then verify integrity, and then jump to version 2.

There's one slight problem - for the upgrades to work, I need to actually verify the current version works, and then run the upgrade to 1.5 through PHP - but the current site doesn't run on PHP5. The images take over 5GB at the moment, so copying everything over to a different machine and do the upgrade there isn't feasible either.

While pondering if I should rebuild apache and php by hand, it hit me that I could just use mach to set up an FC3 chroot, install apache and php, copy my site into the chroot, run apache on a different port, and do the upgrades step by step. Ten minutes later, I had completed the first step of the upgrade.

For future reference, here's what I did:

yum install mach
mach -r f3c setup
mach -r f3c yum install apache php vim-enhanced
mach -r f3c chroot

- copy site into chroot, change /etc/httpd/conf/httpd.conf to run on port 8080 and point to the copied site

httpd -X

(because doing 'service httpd restart' would make it kill the host's httpd as well)
- access the site, notice that it stops httpd, check the log and figure out why

cd /dev
mknod random c 1 8
mknod urandom c 1 9
httpd -X

- access the site again, notice that everything works except logging in
- edit php.ini and turn register_globals On, login then works
- overwrite current gallery with gallery-1.5.2pl2
- go to site, notice gallery tells me to upgrade, but that gives me a blank page
- edit config.php, remove some of my customizations, redefine some vars, run again, upgrade now works
- turn off register_globals (because the upgrade wizard then complains about it, good)
- notice errors in the config, and:

yum install netpbm-tools unzip

- Finish the upgrade wizard
- go to gallery page and see the "httpd -X" crash again
- look at error log, notice it ran out of memory, and update php.ini to use 32M instead of 16M
- access site again, Gallery informs me my albums are out of date and I should upgrade them, which I do
- after some waiting, the upgrade is done, and (beside the horrible theme) my gallery seems to be ok, just like before
- go to tools/find_orphans.php, repair
- go to tools/validate_albums.php; this ran fine

At this point, I had a working Gallery 1.5, and it was time to move the whole tree back into the original host's web directory, and finish the migration to Gallery 2.1 there.

- use the multisite option to put a gallery in my website dev tree
- create MySQL tables (Gallery 1.3 worked completely database-free, but this is not possible anymore - too bad)
- disable GD as requested by the upgrade manual
- create test album, upload test photo, notice thumbnail missing
- this is because by disabling GD, none of the graphics toolkits were activated. Activate ImageMagick
- Gallery warns me about a vulnerability, which it links to. Nice ! I double-check if my ImageMagick RPM has a patch for this (rpm -q --changelog ImageMagick) - it does: "fix a denial of service in the xwd coder (#158791, CAN-2005-1739)"
- proceed with this version
- recreate thumbnails, twiddle with settings, until there are thumbnails for the test photo

At this point, Gallery 2 is working correctly, and now it's time for the big upgrade. MAKE SURE YOU HAVE A BACKUP.

- For the import, increase your PHP memory limit. I used 64M (because an initial run with 16M stopped running after a minute, leaving me with only part of the albums imported, annoying)
- In Site Admin, choose "import from Gallery 1"
- I chose to set Title from blank, and Summary from Caption (because a test import had some captions that were too long to serve as title and got truncated)
- Nice progress bar, everything works well, and the upgrade finishes with around 33MB used for close to 4000 pictures.

All in all, this worked a hell of a lot better than I expected it to. Gallery is a pretty impressive web application, and the user experience I got from it was amazing - I was able to upgrade from a version from 2002 all the way to the latest without too many problems. I don't think I've ever seen a web app take this much care of its users.

Next on my list - migration of Rombocop and other Apestaart sites with Gallery.

So, I've been delaying a certain problem for a few months now, hoping that any of my fellow monkeys at Apestaart would address the issue first. Of course, they were...

Somehow I always end up delaying releases of mach until a few weeks after a new Fedora Core release, even though I always add the relevant files before the release. This time was no exception. One of the main reasons is because I don't have a smooth release process as I have for GStreamer, and that always annoys me enough to not properly do a release.

Anyway, using the prototype tools I cobbled together over the weekend I sent out the release announcement mail and updated the freshmeat record.

As for mach itself, features in this release include:
- added DOAP file (Thomas)
- added Fedora Core 5 (Thomas)
- fix build ordering when building multiple packages (Thomas)
- always evaluate spec file inside the root (Thomas)
- add an option for md5summing of results (Matthias)
- pass 'buildopts' from a root config to rpmbuild (Thomas)
- add 'excludearch' property to root config,
fixes yum's semi-random behaviour on x86_64 (Thomas)
- store yum cache data per-dist, shared for flavors (Thomas)
- clean up yum metadata for local repositories (Thomas)

Enjoy, and let me know if you run into anyissues.

Meanwhile, the mach3 rewrite is getting functionality added step by step as I have some days off to work on them. I added a bunch more stuff the past weekend, and I now have a working mach binary that can go through all the setup steps. That's probably going to stimulate me more than anything else to go back to hacking on it - even though developing the classes correctly and writing unit tests for them on its own is fun as well, as long as there isn't a mach binary to play with it still feels as if it's years away from completion.

Somehow I always end up delaying releases of mach until a few weeks after a new Fedora Core release, even though I always add the relevant files before the release....

As my Friday project of last friday, I wanted to take a look at Description of a Project and see if it would fit my needs as a release maintainer of GStreamer, and possibly other projects.

There are a number of problems I'd like to solve; some of them are:

1. use the DOAP file for a project to centralize the short and long descriptions used everywhere (website, feeds, tracker sites, packages, ...)
2. submit to freshmeat from the doap file
3. send out release announcements
4. keep track of releases and their dates on the various branches
5. possibly keep track of the future release schedule
6. generate an iCal file and an rss feed to get the history
7. merge multiple files to get a combined feed for all GStreamer projects

All items of this list related to keeping track of multiple releases on the same branch seem to be outside of the scope of DOAP; sadly, it seems most things I would like to be able to do with it need this. I don't understand RDF well enough yet to know if I'm doing something wrong by stuffing multiple Version nodes with the same branch inside the DOAP file (Apache seems to be doing it, not sure if that counts) and possibly I should learn to extend DOAP in some compliant way so that I'm doing things correctly.

However, that makes it grow a little out of scope for a Friday project :)

In any case, I started off by trying to install the Redland RDF library and its python bindings. I took the .src.rpm's, rebuilt them for FC4 (I was trying on one of the other machines at the office), had to tweak here and there, and then tried a python example. That crashed with a double free. I took some time to look through the code and found a double fclose() which fixed my problem when I patched it. Submitted a patch and everything, only to find out that the same patch had already been sent to the mailing list (but not logged in the bug tracker). Well anyway, at least I can fix bugs in a completely new-to-me library.

Due to some other interruptions, Friday went quickly by.

After more experimenting, I ran into another segfault with the 1.0.3 version. I decided to clean up the spec files completely today, and build the 1.0.2 version on my laptop - which runs FC5, to see if that worked better. Yes, it did - no more crashing. So now I could focus on writing some actual programs.

RDF is something completely new to me - I thought the world had ended at XML and XSLT. Now it seems I have a new format to learn that looks a lot like XML, but apparently you can't just use XSLT on it. Not sure yet if I will then be able to integrate DOAP stuff with our current GStreamer site, which is completely built with XML and XSLT.

In any case, after some hacking, I cobbled together some basic tests which manage to generate an RSS feed and an .ics file, as well as submit project releases to freshmeat.

The rss 1.0 feed works fine in straw, but not in RSSOwl; the rss 2.0 feed works fine in straw and RSSOwl; and the .ics file works fine if I import it locally in Evolution, but importing it as on the web (and changing webcal:// to http://) makes evolution hang - not sure what's going on there.

All in all, a good first stab at trying to solve some of my releasing problems.

As my Friday project of last friday, I wanted to take a look at Description of a Project and see if it would fit my needs as a release maintainer...

Pretty much every time I upgrade Fedora something has changed in the way USB is handled and/or my camera (A Canon Powershot G2) can't talk to my computer anymore. I really don't know what is going on this time. Who are these usb hackers, and is it too much to ask to keep things working for more than six months? If I were paranoid I'd think they're out to get me.

I'm sure my mood will get better once I figure out, *again*, what the magic hoops are I have to jump through to get it working for this Fedora release.

Pretty much every time I upgrade Fedora something has changed in the way USB is handled and/or my camera (A Canon Powershot G2) can't talk to my computer anymore. I...