Present Perfect

So, my GUADEC proposal got accepted, and now I'm left with the stress of actually gathering enough content for my talk on Practical Project Maintenance.  I'm trying to find my random collection of notes I have collected ideas in for this talk.  As Andy puts it, "You are the perfect target for Beagle".  He's probably right, but somehow I always end up turning off Beagle after another instance of beagled using 100% CPU on something.  It is one of those cool technologies that you actually need to put effort into to learn enough of it so you can troubleshoot yourself out of a corner - like I had to do with SELinux for example.

But I digress.  One of the topics I wanted to bring up in the talk is how to get your project known.  As a good engineer, I want to measure before I change and affect, so I wanted to write a simple program to figure out where some of my projects turn up in Google if you search for relevant keywords.

Sadly, it turns out that the Google Web SOAP API (for which there is a python module) is deprecated, and no API keys are being handed out anymore.  The only interface left is the AJAX API (which I guess you can only sensibly use from a web page in the first place).  If I understand correctly Google's EULA also does not allow me to screenscrape.  Doing a simple wget confirms this - though it's easy to fool around with the user agent of course.

I'm no expert on business but I'm going to guess that Google closing this SOAP API is part of the move from search as their core business to advertisement.

I'm going to play around with various alternatives - probably no simple program using BeautifulSoup to screenscrape (I'm saying "no" because my lawyer tells me to), using this amusing EvilAPI website, or using Yahoo Search API instead.  I want to be practical, so anything that gets the job done.

If any of you have a Google API key left that they're not using, feel free to hook me up.  I promise I will Do No Evil with it...

So, my GUADEC proposal got accepted, and now I'm left with the stress of actually gathering enough content for my talk on Practical Project Maintenance.  I'm trying to find my...