Present Perfect

Time to unravel some tufts of the yak hair lying around from this weekend.

I wanted to work on making my GTD application (using CouchDB) a little more friendly.

I started out by learning GreaseMonkey so I could modify Trac pages to include a link converting the ticket into a thing in my application.

The basics of that were easy, but I need to learn a bunch more Javascript before it will actually work well. So, I switched to finally figuring out if desktopcouch might be a good solution for the general problem for users to set up a couchdb database.

desktopcouch's tagline describes it nicely 'A CouchDB for every desktop.' From what I can tell, it has:

• a way to automatically start a per-user couchdb instance
• oauth enabled by default for it, so only you can get to it
• some gnomekeyring integration
• a pairing tool that makes it easy to pair with other desktopcouch instances on your network
• a python module on top of python-couchdb that layers a 'record' concept over CouchDB documents using record_type and record_revision keys
• some pre-defined record types for notes, contacts, and bookmarks
• possible integration with the UbuntuOne service

At this point I don't know how intertwined this record concept is with the rest of desktopcouch. As at the moment I want to use desktopcouch simply as a reliable way to get a secure, per-user, replicated couchdb database, I'd prefer to not deal with records at all at this point.

In the process of getting acquainted with desktopcouch and packaging it for Fedora, I ran into a few little niggles:

• First off, bdist_rpm didn't work for desktopcouch because of the man pages being compressed.
• My first run failed. Furthermore, the unhandled exception left my $HOME/.local/share/desktopcouch in a half-done state; I ran it again, and it worked, but it didn't generate a couchdb.html as it should have because of this.
• Some rpmlint warnings

As for the package, if you're on Fedora 11, just install tao-release for F-11 i386 then type 'yum -y install desktopcouch'

After that, you can start it by asking it for what port is running:


dbus-send --session --dest=org.desktopcouch.CouchDB --print-reply --type=method_call / org.desktopcouch.CouchDB.getPort


If all goes well that should come back with a number.

You can now access your desktopcouch through Futon with:

firefox $HOME/.local/share/desktop-couch/couchdb.html


Let me know if this worked for you or not !

I have a separate post about packaging ubuntuone stuff, which was much more complicated - and I'm still not sure if I will be able to use it as an online server for my couchdb app.

Now, for some questions to you desktopcouch-savvy people out there:

• Looking at the generated couchdb.html, I see that I can now access the user-specific couchdb with a generated username and password. If I want to push my GTD app to this couchdb, I can do so using this username and password. So, obvious questions: a) should I push with couchapp this way to get my app in desktopcouch ? b) is there a better way to get this username/password ? I assume desktopcouch does this in the python modules using gnome-keyring c) is it an intended mode of operation to be able to push my app into the user couchdb ? I am working on the web interface part served by couchdb and I want it to be one of the ways in which to access the data.

Is it possible to pair with a different user on the same machine ? I was on a laptop with no other linux computer around, so I was not able to test the pairing. I did set it up for a different user, but neither account seemed to see the other's couchdb running.

• Will I be able to sync non-record databases to ubuntuone ? If not, what else could I do to have an online couchdb to sync between my various work machines ?

Time to unravel some tufts of the yak hair lying around from this weekend. I wanted to work on making my GTD application (using CouchDB) a little more friendly. I...

Had a migraine attack this weekend so I took it relatively easy on the hacking side. I finally finished setting up some new buildbots, and as part of that I rebuilt some packages for CentOS/RHEL 5.

So I also updated my package page that now has direct download links for release packages for my repo, to make it easier for you guys to install stuff from there if you're interested.

I use these repositories for prereleases and releases of the projects I work on, as well as packages that I have submitted to Fedora or otherwise needed and haven't had time to submit properly yet. You can still browse the repositories to take a peek.

Had a migraine attack this weekend so I took it relatively easy on the hacking side. I finally finished setting up some new buildbots, and as part of that I...

This morning I wanted to upgrade my laptop to F11 after both my work and home desktops had been upgraded to F11 64-bit a while back.

Since a few releases I've been maintaining and extending my own upgrade guide which makes the process decidedly less painful. In the past I would have my flow interrupted for a few weeks as I ran into yet another thing that wasn't installed or set up. Now I can just put myself in 'machine upgrade' mode and follow the instructions, and adapt them as I go.

Two particular things were annoying this time so I fixed them permanently and crossed off some todo items from my GTD list in the process.

First of all, since the laptop is i386 as opposed to the x86_64 installs I had previously, I hadn't yet rebuilt packages that I maintain for i386. So I started rebuilding them. And then I realized that I shouldn't rebuilt noarch packages, I should fix my packaging symlink script to take those into account too.

So, converted my ugly shell script into a slightly less ugly python script that finally is smart enough to put any noarch rpm into the package repos for all archs.

Then I rebuilt the arch packages, and finally created a tao-release rpm so that enabling my repo can now be a one-click install. Should have done this a long time ago.

As you might expect, no promises on quality in that repo, but I use it myself over a bunch of machines, and drop packages there of projects I work on, or packages I still need to submit to Fedora.

While I was on a roll, I also finally updated those package pages and links, dropped the links to the pre-Fedora-9 packages (though they're all still there if you browse), upgraded from WordPress 2.8.0 to 2.8.4 (still a huge drag).

Anyways, a good start to a Saturday after the worst migraine attack I've ever had last night. My back feels like it's welded shut, but hey, today seems to be an ok day.

This morning I wanted to upgrade my laptop to F11 after both my work and home desktops had been upgraded to F11 64-bit a while back. Since a few releases...

mach allows you to set up clean roots from scratch for any distribution or distribution variation supported.

This release of mach contains fixes for Python 2.6, and adds Fedora 10 and 11, while fixing the archived Fedora locations.

Get it from the mach project page.

mach allows you to set up clean roots from scratch for any distribution or distribution variation supported. This release of mach contains fixes for Python 2.6, and adds Fedora 10...

Last week, after upgrading my home desktop to F11, I had palimpsest tell me one of my disks was broken on the desktop machine. The desktop is running on two 250 GB drives in software raid. It was time to get new drives.

After a weekend of fiddling with new 1 TB disks for my home desktop, trying failure scenarios, making sure the system can boot from each of the two drives, and waiting for the 4 hour resync of the software RAID in between each step, I finally closed up the desktop machine and cleaned up under my desk again, thinking I was done with my halfyearly messing about with broken disks.

I guess I was tempting faith anyway. Doing a routine operation on my home server after all the configuration stuff I'd done to set up asterisk last week, suddenly an rsync aborted, a journal errored out, a partition changed to being mounted read-only, and the log was full of scary drive errors. Ouch.

Well, that's why I keep around a big box of old drives - for when some drive fails and I want to tempt fate even more by reusing an old drive that's probably going to fail real soon too. And anyway, I had just spent my hard drive piggybank on the new desktop drives.

Luckily, I seemed to have a 400 GB SATA drive lying around that used to belong to my media center. I don't remember why I swapped it out, given that the media center has a 160GB drive for the OS (and two 1.5 TB raid drives for the data, of course), but this was a lucky break. I booted with a rescue cd, and tried copying the root filesystem of my CentOS 5.2 home server partition to this new drive. Which worked fine, except that /var was where I triggered an Input/Output error and some more drive errors in the kernel log.

So, powered off, took out the broken drive, and put it in a USB chassis. The advantage of a USB chassis is that you can easily just replug the drive to try again, instead of locking up your system terribly and having to reboot. Sadly, /var was broken beyond repair. I ran an e2fsck hoping to recover the contents, and that partly worked, but some of the important stuff is missing even from lost+found (apart from the annoying situation where you have to reconstruct file names, which I usually end up not bothering with).

But really, how important can /var be ? Turns out, rather important. As in, you need it to boot in the first place. And also, it holds your rpm database. Crap.

Some Googling gave me some posts on how to reconstruct your rpm database from log files (using --justdb --noscripts --notriggers). But to use those, you actually need those log files. Where are those ? On /var as well. Crap. And they're not in lost+found either.

Ok, so time to get creative. Here's what I ended up doing:

• create /var/lib/rpm, and run rpm --rebuilddb to end up with an empty rpm database
• Based on the contents of /usr/bin, figure out what packages ought to be installed:
  
rpm -qf /etc/* | grep 'not owned' | cut -f2 -d' ' > /tmp/unowned
yum --enablerepo=c5-media --disablerepo=base --disablerepo=updates --disablerepo=addons --disablerepo=extras whatprovides `cat /tmp/unowned` | cut -f1 -d' ' | sort | uniq > /tmp/missing
yum --enablerepo=c5-media --disablerepo=base --disablerepo=updates --disablerepo=addons --disablerepo=extras install `cat /tmp/missing`


  This works by first listing all files that are not owned by rpm (on the first run, that's all of them), figure out what packages can provide these files, then installing those packages.

• Repeat the process for other important directories, like /bin, /sbin, /usr/sbin, /usr/lib, /usr/include, ...
• Clean up .rpmnew files that don't actually contain differences:
  
find / -name *.rpmnew | sed s/.rpmnew//g > /tmp/rpmnew
for c in `cat /tmp/rpmnew`; do echo $c; diff $c $c.rpmnew && mv -f $c.rpmnew $c; done


• Same for *.rpmorig:
  
find / -name *.rpmorig | sed s/.rpmorig//g > /tmp/rpmorig
for c in `cat /tmp/rpmorig`; do echo $c; diff $c $c.rpmorig && mv -f $c.rpmorig $c; done

• Inspect the remaining ones, and merge changes.

While it's not an experience I hope to repeat any time soon, it worked out surprisingly well!

Last week, after upgrading my home desktop to F11, I had palimpsest tell me one of my disks was broken on the desktop machine. The desktop is running on two...