Present Perfect

August is a great month for clearing the decks in Spain. Half the people at work are on holiday, and by the time they get back the other half goes away. It's a great month for making progress on all those things that've been lying around for months not getting done.

Same in my spare time. After lots of changes in my life in the last year, I've been settling down again, in a new apartment, in a new life, and I am finally in a place where I can do some hacking again. And it's a great feeling, to be back in a flow state and fixing problems.

Tonight I wanted to direct my hacking attention to my venerable N900 phone. First of all, I wanted to figure out why mushin, my couchdb-based GTD application, was not showing the right results for my shopping list. Turns out I had a broken svn snapshot running on the phone, and that was easy to fix. In the process I set up scratchbox again on my new laptop.

Then I directed my attention to erminig-ng, an application that syncs between the maemo calendar and Google calendar. It had stopped working a few months ago, giving a traceback during sync. I first tried reproducing the problem in scratchbox, but I couldn't even hit the bug after copying the sqlite databases for both erminig and calendar.

I ended up adding some well-placed prints and

import code; code.interact(local=locals())

statements on the phone's file system so I could inspect the objects provoking the crash. It turns out there was a simple call somewhere that overwrites an event's alarm time setting, but it does so with an int when the code in gdata expects a string.

Now, to submit the fix, I have the same problem as last time I submitted a fix. erminig-ng is a fork of erminig. erminig has a website that has no updates since 2008. The version number on that site is higher than the version number of erminig-ng, but erminig-ng is packaged as erminig, which is confusing for people. The bug tracker on maemo garage that the package for erminig-ng links to is only for erminig, and people get confused, so instead they're reporting all their problems in a long maemo talk conversation. It looks like the maintainer for erminig-ng is the same as for erminig, so I'm not sure why this confusion is there. It looks like it would b easy to clear up. The code is not in any public repository as far as I can tell, and people are fixing things here and there with patch files in the maemo talk thread giving instructions on how to patch files on the root file system. A mess, really. It should probably just go on github somewhere, but I want to hear from the maintainer first.

So, I slurped the latest 0.2.12 release source into my svn repository, forked it to a trunk directory, and started integrating my patch. All .py files are in the root, I had to fight the urge to not clean it up properly and put it nicely in separate directories, because that would make merging patches back harder. I added a test case for the bug I was hitting as well - there were no tests so far in the package.

After my bug was fixed, and the test worked, I rebuilt a package for my phone. That led me to updating my maemo repository, adding gpg signing to it, adding metadata info to it, and doing everything needed to make it easy to update and not trigger any errors on the phone when upgrading. Now it looks like a nice proper maemo repository (and even apt-file should work) - but if any debhead wants to give it a looksee to see if anything's wrong feel free!

So, now erminig 0.2.12.2 is available from my repository, works on my phone, and is now again properly syncing my calendar information. I dropped a line in the forum thread for a person who was probably running into the same bug as me.

Next on my radar - looking at barriosquare to see if an update to the foursquare v2 API is doable.

I felt so guilty for not doing enough maemo stuff over the last year to try and get an N950. Now I regret not applying - it's going to be the last phone in the series (yes, I'm one of those who think Nokia is making a huge mistake), the focus of what's left of Maemo will be on that phone, and I won't be able to update the programs I use on Maemo, and my phone will slowly become obsolete. At which point I probably have no choice but to switch to Android, the non-Linux Linux system. I don't know many people anymore who work for Nokia, but if any of them are listening and still want to help me get an N950, feel free to let me know.

And the kicker of my patchwork ? It turns out that, when I fixed a bug in the previous version, I had already forked off 0.2.11 into my repository at a different location. And in that checkout, I also added a unit test, a similar-but-different hack for not being able to load hildon as a module, and a HACKING and TODO file with very similar notes as today.

Hey, at least I'm consistent. And maybe I should throw this stuff on github after all, so I can use git's branch powers to track the upstream releases, and the work I did that did not get taken upstream...

August is a great month for clearing the decks in Spain. Half the people at work are on holiday, and by the time they get back the other half goes...

I got my N900 back. The micro-USB port had broken off cleanly. Apparently that's a known thing. I brought it to a Nokia Support Centre and they fixed it in two weeks. So far for the good news.

The bad news is that, even if they only reattached a connector, they still wiped the phone itself. I had a backup from the backup application and a dirvish backup of /home/user

But obviously that's not enough. First of all, restoring from the backup takes a while - it reinstalled 110 apps, 20 of which stopped the installation because they feel I should click ok to install. Sigh.

Transferring 30 GB of old data is slow no matter which way you cut it.

I lost my Angry Birds levels and the Sygic Mobile Maps app, still trying to figure out if I can get those back easily.

But at the moment I'm mostly annoyed at my problems figuring out why I can't ssh as user@ to the N900 anymore.

So, here are the steps I took:

$ sudo gainroot
# apt-get install sysklogd
# vi /etc/syslog.conf
(uncomment the line that says: auth,authpriv.* /var/log/auth.log)
# vi /etc/ssh/sshd_config
change LogLevel to DEBUG3
# killall sshd
(doing /etc/init.d/ssh restart does not actually get the job done; neither does initctl stop sshd)
# tail -f /var/log/auth.log
(finally the log is there)
try and log in

Now the log tells me:
Apr 7 12:51:31 Nokia-N900 sshd[2266]: User user not allowed because account is locked

And a quick look at /etc/passwd shows ! as the password, meaning the account is locked out.

So...

# passwd user
(pick a password)

then try and log in again using ssh keys, and now it works.

I had no idea a user needed a password before being allowed to log in with ssh keys (not using the password).

After this, don't forget to set logging back to INFO lest you fill up your limited disk space with useless debug info.

I got my N900 back. The micro-USB port had broken off cleanly. Apparently that's a known thing. I brought it to a Nokia Support Centre and they fixed it in...

One of the things I really love about my N900 is the ease with which I could share photos. You take the picture, click a few buttons, and there it is, your photo on flickr on Facebook. I'm sure other devices offer a similar experience, but this really is the first time I've been able to appreciate

Since a few weeks my Facebook sharing has stopped working. At first it only seemed like a missing icon and broken config. But I had a really really hard time to figure out the problem, much harder than it should have been for a mostly open platform. Of course, for some reason this sharing part is closed, which doesn't make much sense at all. What secrets can Nokia possibly have invested in some code that pushes photos to flickr or Facebook.

So, as part of the debugging process, over various weeks, I've seen and done the following:

• When sharing through the Facebook account, the little 'I'm sharing' icon that pops up in the status area went away almost immediately, compared to sharing through Flickr
• Sharing stores outgoing stuff in $HOME/.sharing/outbox - files were piling up there
• At first I thought I had some login details problems, so I went to Settings > Sharing accounts and clicked on the Facebook account. That didn't do anything at all, no dialog popped up. Since the Delete button is in that popup, I also couldn't use the UI to recreate my Facebook sharing account.
• I started learning about where the N900 stores account info, following some dubious posts until I found out about signond, a daemon running as root that stores login information. This daemon uses the file $HOME/.signon/user_db.xml (You get no points at all for correctly guessing whether this file contains XML data, but please do go look for yourself!)
• I moved this file around to force recreation of data, which seemed to work after a reboot, since my Flickr data was now gone, as well as the Facebook one. I was able to click Flickr in Settings > Sharing accounts, but the Facebook still didn't pop up a dialog. At least this narrows down the problem I should solve for now: I can't even configure the service from scratch even when deleted, so let's focus on that first.
• I reinstalled the package that contained the facebook sharing plugin. This restored the icon, but the popup still didn't work.
• Usually I would just strace the relevant binary, but I considered the friction too high at first since there is no strace in fremantle, and I would have to set up a chroot for this.
• But what binary ? I didn't even know what program handles the settings. After comparing a few ps listings, I figured out the program was controlpanel.
• Interestingly, you can just ssh into the device as the user, and execute controlpanel, and it will pop up the binary on the phone (even with X forwarding), which I didn't expect, but is fine by me. This gave me a message like "sharing-accountstore.c 516 sharing_account_store_add_account Last added id exists: 35
  " which I wrongly took to mean that adding a facebook account was failing because it was still somehow present in the config. This is when I figured out that this code is closed, because I wasn't able to figure out where to get accountstore.c and look at what it is doing. I'm pretty sure it's not storing its settings in gconf (at least I couldn't find anything related to facebook sharing when walking the gconf tree), and it doesn't seem to be in any dot folders in my home dir. Anyone know ?
• Finally, I gave up, set up the chroot, and straced that binary. And I should have known this would have led me to the right solution. Among all the noise, there were some calls trying to open a library. It's normal for a bunch of calls to fail as it searches the whole LD_LIBRARY_PATH, but usually the last one of those in an strace log should succeed. But they weren't; apparently /usr/lib/libfacebookcommon.so.0 was missing!
• Looking for that file on disk confirmed that it wasn't there, and doing dpkg -L feedservice-plugin-fb-common showed that it should in fact be there. This reminded me of a pet peeve I have with those people who claim Debian's packaging system to be far superior to rpm - apparently dpkg doesn't have any equivalent of rpm -qv which allows you to verify that the files that should be installed by a package are indeed on disk. This would have saved me *a lot* of time figuring out this problem, and is typically the first thing I do on an rpm-based system where things act funny.
• With my first real clue in hand, I used ldd to verify that indeed there were libraries missing:
  

  ldd -r /usr/lib/sharing/plugins/libfacebooksharingservice.so
  ...
  undefined symbol: facebook_credentials_free (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: facebook_request_reset (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: network_utils_post_multipart_with_progress (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: facebook_get_email (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: facebook_login (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: generate_signature (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: facebook_store_credentials_to_gconf (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: facebook_request_new (/usr/lib/sharing/plugins/libfacebooksharingservice.so)
  undefined symbol: facebook_request_free (/usr/lib/sharing/plugins/libfacebooksharingservice.so)

• Using dpkg -S to figure out what package this belonged to, I apt-get remove'd that package, then reinstalled it and dependencies.
• Still didn't work, so repeated the process, this time libfeeserviceutils.so.0 was missing, so reinstall feedservice-utils and dependencies.

And finally! The sharing plugin can again be configured, and all is well. But this experience was needlessly painful...

For the record, I didn't tinker with any package files by hand (anyone who knows me a little knows my stance on packages and /usr), and I have no idea what I did to get into a situation where files that should be on disk aren't. And I'm worried about what else is missing, so if anyone can point me to some resources explaining how to verify installed package manifests, that would be awesome.

One of the things I really love about my N900 is the ease with which I could share photos. You take the picture, click a few buttons, and there it...

Today I ran into a weird problem on my N900. I was clearing some space to put on more music, and to do so I wanted to delete some root images I had on the device.

I had used qchroot to mount each of them to decide whether I could delete them or not. The combination of mount, qchroot and closechroot however seem to have some issues; I'm not 100% convinced closechroot actually properly cleans up. It seems to do so, but running mount after it shows most chroot mounts still active, although you can't actually access them or unmount them by hand.

Anyway, I unmounted them as best as I could, and deleted 3 of them, for a total of 8 GB. However, the disk space wasn't actually freed. Deleting other files worked fine and reclaimed disk space as expected.

I rebooted, thinking that there probably were active references to the blocks, but this didn't fix anything either.

I started digging around for possibly hidden Trash folders, and someone on #maemo suggested that there should be a /home/user/MyDocs/.Trash-1000 but it wasn't there on my device.

Eventually, I found about dosfsck, which is installed on the device, and it said something hopeful:

Reclaimed 109128 unused clusters (7151812608 bytes).
Free cluster summary wrong (5274 vs. really 114402)
1) Correct
2) Don't correct

and after some fiddling to actually make it apply the changes, it dropped the following files in my MyDocs:

FSCK0000.REC
FSCK0001.REC
FSCK0002.REC

They were all about the same size as the images from before, so I deleted these files and voila!

Since I didn't find anything pointing to this using Google, I'm posting it here so that I can find it next time, as well as others that may need it..

Today I ran into a weird problem on my N900. I was clearing some space to put on more music, and to do so I wanted to delete some root...

It took quite a bit of blood, sweat, tears, and dirty hacks, but finally:


Nokia-N900-42-11:~# apt-get install desktopcouch
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
couchdb-bin erlang-base erlang-crypto erlang-inets erlang-mnesia
erlang-public-key erlang-runtime-tools erlang-ssl erlang-xmerl libjs
python-desktopcouch python-desktopcouch-records python-keyring
Suggested packages:
couchdb erlang-tools erlang erlang-manpages erlang-doc-html
python-keyring-gnome python-keyring-kwallet
Recommended packages:
erlang-syntax-tools
The following NEW packages will be installed:
couchdb-bin desktopcouch erlang-base erlang-crypto erlang-inets
erlang-mnesia erlang-public-key erlang-runtime-tools erlang-ssl erlang-xmerl
libjs python-desktopcouch python-desktopcouch-records python-keyring
0 upgraded, 14 newly installed, 0 to remove and 9 not upgraded.
Need to get 0B/12,5MB of archives.
After this operation, 27,9MB of additional disk space will be used.
Do you want to continue [Y/n]?
WARNING: The following packages cannot be authenticated!
erlang-base erlang-crypto erlang-mnesia erlang-runtime-tools
erlang-public-key erlang-ssl erlang-inets erlang-xmerl libjs couchdb-bin
python-keyring python-desktopcouch python-desktopcouch-records desktopcouch
Install these packages without verification [y/N]? y
Selecting previously deselected package erlang-base.
(Reading database ... 27651 files and directories currently installed.)
Unpacking erlang-base (from .../erlang-base_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-crypto.
Unpacking erlang-crypto (from .../erlang-crypto_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-mnesia.
Unpacking erlang-mnesia (from .../erlang-mnesia_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-runtime-tools.
Unpacking erlang-runtime-tools (from .../erlang-runtime-tools_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-public-key.
Unpacking erlang-public-key (from .../erlang-public-key_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-ssl.
Unpacking erlang-ssl (from .../erlang-ssl_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-inets.
Unpacking erlang-inets (from .../erlang-inets_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package erlang-xmerl.
Unpacking erlang-xmerl (from .../erlang-xmerl_1%3a13.b.2.1-dfsg-3maemo2_armel.deb) ...
Selecting previously deselected package libjs.
Unpacking libjs (from .../libjs_1.8.0-rc1-1_armel.deb) ...
Selecting previously deselected package couchdb-bin.
Unpacking couchdb-bin (from .../couchdb-bin_0.10.0-0maemo3_armel.deb) ...
Selecting previously deselected package python-keyring.
Unpacking python-keyring (from .../python-keyring_0.2-3_all.deb) ...
Selecting previously deselected package python-desktopcouch.
Unpacking python-desktopcouch (from .../python-desktopcouch_0.5.1-0maemo1_all.deb) ...
Selecting previously deselected package python-desktopcouch-records.
Unpacking python-desktopcouch-records (from .../python-desktopcouch-records_0.5.1-0maemo1_all.deb) ...
Selecting previously deselected package desktopcouch.
Unpacking desktopcouch (from .../desktopcouch_0.5.1-0maemo1_all.deb) ...
Setting up erlang-base (1:13.b.2.1-dfsg-3maemo2) ...
Searching for services which depend on erlang and should be started...none found.
Setting up erlang-crypto (1:13.b.2.1-dfsg-3maemo2) ...
Setting up erlang-mnesia (1:13.b.2.1-dfsg-3maemo2) ...
Setting up erlang-runtime-tools (1:13.b.2.1-dfsg-3maemo2) ...
Setting up erlang-public-key (1:13.b.2.1-dfsg-3maemo2) ...
Setting up erlang-ssl (1:13.b.2.1-dfsg-3maemo2) ...
Setting up erlang-inets (1:13.b.2.1-dfsg-3maemo2) ...
Setting up erlang-xmerl (1:13.b.2.1-dfsg-3maemo2) ...
Setting up libjs (1.8.0-rc1-1) ...
Setting up couchdb-bin (0.10.0-0maemo3) ...
Setting up python-keyring (0.2-3) ...
Setting up python-desktopcouch (0.5.1-0maemo1) ...
Setting up python-desktopcouch-records (0.5.1-0maemo1) ...
Setting up desktopcouch (0.5.1-0maemo1) ...
Nokia-N900-42-11:~# dbus-send --session --dest=org.desktopcouch.CouchDB \
> --print-reply --type=method_call / \
> org.desktopcouch.CouchDB.getPort
method return sender=:1.516 -> dest=:1.515 reply_serial=2
int32 53864
Nokia-N900-42-11:~#


Among other things, I had to:

• port desktopcouch to use python-keyring as suggested by the desktopcouch hackers, although I'm not sure what they now want to do with it since there's no way to make it compatible with their current way of storing in gnome-keyring
• Build the arm parts of erlang and couchdb on my phone in a chroot, since qemu-arm-sb actually hangs when executing erlc. Had to build it in an actual maemo chroot as well; the easy-deb chroot was not a solution since it comes with newer C and ssl libraries.
• figure out how to package spidermonkey, instead of depending on xulrunner the way the debian couchdb does
• package lots of missing packages; for example, while maemo has avahi, it didn't have the python bindings built.

For the adventurous, I think you should be able to download tao.install on your phone, then go to console and run the same apt-get install command as I did. Please let me know if it worked for you !

Now, need to figure out (or get told by the desktopcouch hackers) how I can set up pairing with UbuntuOne for my data without needing to port the whole gnome applet...

It took quite a bit of blood, sweat, tears, and dirty hacks, but finally: Nokia-N900-42-11:~# apt-get install desktopcouch Reading package lists... Done Building dependency tree Reading state information... Done The...