The short version is 'I can finally live with the favicons in my delicious extension thanks to a script I wrote and after upgrading to beta 2.1.005.
The long version is, well, longer.
This morning's yak shave story begins a bit over 5 years ago. That was the time when I was inbetween jobs, and already had my laptop for the new job, which was going to evolve into being 'employee number 1 at Fluendo'. I had decided I wanted to try the 'living life from only one computer' approach to working and hacking. Some people do this instinctively, some people don't choose and end up with crap on all their machines, but me, I like inventing processes where there is no need to and sticking to it.
So, for a long time, I mostly worked only from my laptop, and this served me fine in a lot of cases. My main motivations were 'I want to be able to work anywhere I am' and 'If I'm used to a laptop I will always be able to be productive'. The first proved very useful whenever I was travelling, and the second is what made me not want or need a second screen at work when all developers around me were making it a political point.
The biggest problem, however, is that often the work you do really does need the higher CPU, the more RAM, or the bigger disk that your average desktop easily has over your laptop as soon as your laptop becomes a year old. Also, your laptop wears out a lot faster if you have it running all the time. So, at some point, after knowing for sure that I am able to work constructively on my laptop, I decided that I should also make it possible to work from a home desktop and a work desktop.
Make no mistake, to me this is an incredible paradigm shift. While on its own there is no real problem to habitually work on three computers, to someone like me it is incredibly annoying to have my flow broken by various small details that made up my previous 'work environment' on my laptop that I have not yet successfully migrated to the other machines. If you have only one machine, adapting it to your workflow continuously is easy. If you have three, you constantly bump into the things you forgot to synchronize across all three at the moment where you least want it.
From day 1 the plan was thus to make sure that everything that is important to me would be synchronized across all three machines as often as I needed it. Additionally, I would make sure not to start using any new tools that weren't able to work this way, and move my current set of applications in a direction where they would be easy to sync. It would also be important to make sure I can work offline as much as possible - I spend time in travel and on planes that I want to put to good use.
So, over the years, I've:
- created an svn repository for all my private notes, documents, tests, and code
- created a public svn repository for the same stuff that can be public
- picked a Getting Things Done application called yagtdthat uses a text backend so that I can put the data in subversion to keep it synced across all machines but still work offline
- tried to move my contacts to an LDAP server so that Evolution can use it (which I still consider a 'fail' because Evolution's LDAP backend is still crashy, hangy, and practically unusable for everyday use)
- moved my calendar to various Google Calendars (up to having a public calendar called 'Where is Thomas' so friends and collagues know what country to expect me in)
- actually paid for an application on my phone - the wonderful GooSync Zaheer recommended me which allows me to synchronize all my Google calendars to my phone, in two directions
- moved all of my bookmarks to Delicious for obvious reasons
- used the Delicious firefox extension to replace my bookmarks folder toolbar - Instead, I use the 'Favorite tags' view and use delicious tags like '@daily' (borrowing the at symbol from GTD's contexts) to have quick access to all the sites I use on a daily basis. This works great because as soon as you drag a bookmark to one of those favorite tags on the bookmark bar, it is instantly available on all three computers where I have this set up.
- put all of my passwords into a revelation database that I put in my subversion repo (not 100% ideal, since managing conflicts is impossible, but hey...)
- wrote two scripts, called 'start' and 'stop', which I execute when I start or stop working on a machine, and which sync stuff from subversion and commit stuff to subversion that is important to my workflow (my GTD list, the password database, ...)
None of this is perfect or ideal, but most of it is about as good as it gets for my usual workflow. Lately, I've started focusing on improving these individual elements.
For example, contacts. I really wanted all my contacts to be available in Evolution on all my computers, and on my phone, syncing every way. And bonus points for anything that would finally let me know when my friends have their birthday, a pet peeve for like the last 10 years.
I read some Evolution code and figured out a lot of reasons why the LDAP code is bad, but most of my inquiries (over IRC, on the mailing list, in bugzilla, or poking some friendly hackers (who will remain anonymous)) gave me no idea about why the code was the way it was, who is responsible for maintaining it, and if my ideas for improving were actually any good. I did get a lot of feedback from people agreeing that it sucked or wanting me to work on their pet feature for LDAP support though.
Similarly, calendars. Around the same point, I tried the Google Calendar backend for Evolution as well, because I was getting fed up with not being able to use Evolution for this, and what seemed to be touted as a major feature for a new GNOME release didn't actually work very well. I couldn't figure out at all if it actually supported more calendars than just your primary Google calendar (which I hardly use - I have 10 others), and it didn't work very well at all.
I found a java daemon that is supposed to synchronize your Google calendar with local .ics files which should then work for Evolution, but in practice works very badly for two-way sync and Evolution doesn't seem to notice the file changing anyway. All fixable, but fixing hacky solutions isn't very motivating. And as I mentioned before, while I usually set myself up with a process then stick to it with an almost religious fervor, as I had done for the last years with Evolution - for the first time I was seriously considering completely dropping Evolution.
I saw that Conduit seemed to have some Google support, but after talking to John it seems that the support for Google Calendar got broken and removed in a recent refactoring (I abstained from noting that, by definition, if you broke or removed code then it isn't a refactoring), so it wouldn't work at the moment.
Moving back to contacts, it came to the point where I deactivated my LDAP backends, and tried to go back to using the local one and figuring out other ways to sync. I noticed that GooSync offered the option to sync to Google Contacts, which I had never used before, so I tried that. I found that Conduit did have support for Google Contacts, but I could never get Conduit to sync more than 20 contacts. If I were to hack on anything to fix it, it would probably be Conduit, because I have a feeling it wouldn't be too hard to get it fixed. While my Evolution contacts were being updated on the fly through conduit, I could never get them all. And after a few 3-way syncs I ended up with a bunch of triplicate contacts in my Google Contacts.
But at the same time I noticed something utterly astounding that I would never expect from any Google product. And sure, it's a detail. But somehow I have this opinion of Google that they are so good at getting some of the small details right in their applications. I was so surprised to find out that Google Contacts does not allow you to store birthdays. It's just not there. Even my phone lets me do that and shows the birthdays on its calendar. I was sure it was me who was unable to figure out where they store it, until I found a whole bunch of posts from users sharing the same disbelief. Amazing.
Here's a company with some of the smartest engineers in the world, who've made a successful web mail client, a successful calendaring application, and some technology for storing and using contacts across all of their applications. Yet no one in that company, with all their 20% times, thought it would be a good idea to have birthdays in contacts and have Google Calendar show you a calendar of birthdays automatically all the time ?
I left the whole 'synchronizing contacts' plan on the top shelf right now, while I try and figure out whether I'm going to:
- ignore birthdays and clean up my Google Contacts
- figure out if I can write enough code to make Conduit sync 3-way between my phone, all my Evolutions, and a collection of vcard files in my subversion repository.
So, after that disappointment, there was one more really annoying problem in my 'synchronize everything' approach - since upgrading to Firefox 3 favourite icons have been a huge mess. I had about a 30% chance of getting correct favicons, with the other 70% split between 'not getting a favicon' or 'getting the wrong one, forever'. The delicious extension was even worse at it, usually not able to show more than 5 of them correctly in my toolbar. Seems like a very small detail, but when you have a lot of bookmarks the icons really help in finding the right one consistently.
After various betas and updates of the extension that never actually fixed the problem, a few posts to the yahoo group, little more than promises from the developers that 'things are changing', this week they launched a new beta that takes the first step in a new direction. It wasn't clear to me that it would actually solve the problems, but the notes said that they moved from an RDF file to an sqlite database for storing the synchronized bookmarks. And A developer replied that this new beta should fix my particular problem.
So I updated, tried, and at first nothing changed. I stopped Firefox, opened up places.sqlite and dropped the moz_favicons table. Restarted Firefox, and it refetched a bunch of favicons (some of them wrong - my homepage got the GNOME foot for example), and the delicious extension did get a bunch more favicons, but not all of them by a long shot.
If the extension had been open source I would for sure already have taken the time to look into the code, and fix the bugs - that's how much this irritates me.
But, with the information that now everything was stored in an sqlite backend, I decided to look for the database and see what was wrong. And after 2 hours of hacking (double the time it took to write this overly long blog post) I have a simple python script that adds or fixes the favicons in the ybookmarks.sqlite database.
It's probably dangerous, eats your babies, breaks your database (so back it up first), and takes ages to run as it downloads each page in your delicious account, but for the first time in two years I can finally see that last.fm favicon in my @daily bookmarks folder!
It's Saturday morning going onto 13:15, and finally I can start doing what I was actually planning to do today, and in no way related with the final straw that caused me to fix my favicons.
The short version is 'I can finally live with the favicons in my delicious extension thanks to a script I wrote and after upgrading to beta 2.1.005. The long version...
![[lang]](/images/eye.png)