Present Perfect

After a visit from the unicode police I'm gearing up for a first morituri release.

If you're on F-11, you can just install my repository and then install the latest snapshot with
yum -y install morituri

After that, pop in a relatively well-known CD, then type:

rip offset find

Hopefully, it will be able to tell you the read offset of your CD drive.

Then do

rip cd rip --offset (the number from above)

and it should proceed to look up the CD on MusicBrainz, rip and verify the tracks to flac, and then do an AccurateRip verification.

Let me know if that works out for you. And if you're not on Fedora 11, you can always check out the source code directly, it's not much harder to get running.

After a visit from the unicode police I'm gearing up for a first morituri release. If you're on F-11, you can just install my repository and then install the latest...

The last few weeks I spent most of my spare time learning a little Django and rewriting the old DAD (Digital Audio Database) code (which was written in PHP) to something a little more modern and malleable.

All these topics deserve separate posts, but for now I'm just content hooking up the various pieces of infrastructure for the overall project.

So, I was happy as a child when I was able to do a query for 'all tracks that contain a hidden track' (which I look up by doing level analysis on the files, and figuring out where there are sections with a reasonably large section of silence).

The full list for my collection is below, but it definately picked out the ones I expected, like 'All Apologies' (I can't count the times I was annoyed when a music player played it then shut up for 10 minutes), Lift To Experience's "Into the Storm", the Auteurs track, and the beautiful beautiful hidden track at the end of Placebo's first album.

Pretty soon this player will play these hidden tracks correctly, and DAD will allow you to rate tracks separately.

Here's the current list (some audio files are repeated because they contain more separate slices):

In [8]: for s in models.Slice.objects.filter(start__gt=0): print s.audiofile
...:
Alanis Morissette - You Oughta Know (Extended).ogg
Auteurs - Home Again.ogg
Andrew Dorff - Angel Puppets.ogg
At The Close Of Every Day - Lower World.ogg
Autour De Lucie - Chanson Sans Issue (Remix).ogg
Ash - Darkside Lightside.ogg
Astrid - Say What You Mean.ogg
And You Will Know Us By The Trail Of Dead - Sigh Your Children.ogg
And You Will Know Us By The Trail Of Dead - Source Tags And Codes.ogg
Better Than Ezra - Coyote.ogg
Ben Folds Five - Evaporated.ogg
Beck - Blackhole.ogg
Blur - To The End (La Comedie).ogg
Blur - Essex Dogs.ogg
Beck - Debra.ogg
Counting Crows - St Robinson In His Cadillac Dream.ogg
Chris Whitley - Ultraglide.ogg
Cowboy Junkies - Those Final Feet.ogg
Chocolate Genius - It's All Good.ogg
Coldplay - Everything's Not Lost.ogg
Drugstore - The Funeral (But Most of All).ogg
Damien Rice - Eskimo.ogg
Damien Rice - Eskimo.ogg
Deftones - Mx.ogg
Deftones - Mx.ogg
Denis Leary - Lock 'N Load.ogg
Drugstore - Flying Down To Rio.ogg
Ed Harcourt - Like Only Lovers Can.ogg
Elvis Presley - Rip It Up.ogg
Everclear - Like A California King.ogg
Fun Lovin' Criminals - Little Song.ogg
Green Day - FOD.ogg
Gabriel Rios - Badman.ogg
Geneva - Have You Seen The Horizon Lately.ogg
Grandaddy - Lawn And So On.ogg
Janet Jackson - Special.ogg
Kyuss - Spaceship Landing.ogg
Kyuss - Spaceship Landing.ogg
Krezip - Fine.ogg
Korn - My Gift To You.ogg
K's Choice - All.ogg
Lemonheads - The Jello Fund.ogg
Lemonheads - The Jello Fund.ogg
Lemonheads - The Jello Fund.ogg
Lemonheads - The Jello Fund.ogg
Lamb - Feela.ogg
Lamb - Just Is.ogg
Lo Fidelity Allstars - Dark Is Easy.ogg
Lift To Experience - Into The Storm.ogg
Magnapop - Voice Without A Sound.ogg
Monster Magnet - Vertigo.ogg
Mansun - Dark Mavis.ogg
Manic Street Preachers - Freedom Of Speech Won't Feed My Children.ogg
Metallica - The More I See.ogg
Nirvana - Something In The Way.ogg
Nemo - The Headphone Song.ogg
Nirvana - All Apologies.ogg
Offspring - Smash.ogg
Placebo - Swallow.ogg
Poe - Fly Away.ogg
Polar - Kill My Fears.ogg
Phoenix - Alphabetical.ogg
Patti Smith - Notes To The Future (live).ogg
Placebo - Burger Queen.ogg
Pearl Jam - All Those Yesterdays.ogg
Queens Of The Stone Age - A Song For The Deaf.ogg
Robbie Williams - Suprême.ogg
Radiohead - Motion Picture Soundtrack.ogg
Robbie Williams - Beyond The Sea.ogg
Starsailor - Coming Down.ogg
Sense Field - Haunted.ogg
Smashing Pumpkins - Daydream.ogg
Sunzoo Manley - Taxidriver.ogg
Sparklehorse - Babies On The Sun.ogg
Sonic Youth - Sweet Shine.ogg
Soulwax - Acapulco Gold.ogg
Therapy - Sister.ogg
Tool - Flood.ogg
Tool - Opiate.ogg
Throwing Muses - Shark.ogg
Tanya Donelly - The Shadow.ogg
Throwing Muses - Fever Few.ogg
Travis - Walking Down The Hill.ogg
Turin Brakes - The Optimist.ogg
Tracy Chapman - I'm Ready.ogg
Thrills - The Irish Keep Gate-crashing.ogg
U2 - All I Want Is You.ogg
U2 - The Wanderer.ogg
Teddy Thompson - Missing Children.ogg
Verve - Come On.ogg
Jesus And Mary Chain - Just Like Honey.ogg
Nits - Meisje Van 16.ogg
Whiskeytown - Bar Lights.ogg
Whipping Boy - Morning Rise-A Natural.ogg
Yeah Yeah Yeahs - Modern Romance.ogg
Beyoncé - Bonnie And Clyde '03.ogg
Sigur Rós - Untitled 4.ogg

The last few weeks I spent most of my spare time learning a little Django and rewriting the old DAD (Digital Audio Database) code (which was written in PHP) to...

While working on morituri I provided some patches to pycdio, the python bindings for libcdio, to handle CD-Text. I was assuming it was the only library or program capable of reading it (turns out I was wrong, cdrdao also extracts it). But those functions weren't wrapped, so I added them. Upstream was using git, so that was another opportunity to test my git workflow across my 3 computers.

Anyway, it was nice to get feedback that said:

Your code has been checked into pycdio. Looks very competently done and is very thorough. Thanks.

For various reasons I won't get into I don't always enjoy the results of coding contributions at work to Flumotion, the project I started. And just as much in the open source world, my contributions aren't necessarily always valued, or treated with the respect I would think they deserve :)

That usually isn't that big of a deal to me, because that's not why I do it, but getting this succint positive remark reminds me that being nice invites being nice. Something I need to remind myself of once in a while to make sure I try and act the same way.

Anyways, a roundabout way of saying I can add another project to my list of projects I've touched.

Meanwhile, on my last plane trip I took the time to autotool morituri. I've finally caved in and went with the familiar, adding a simple frontend command using my python Command class
Another tree is born. Here's the first command's output:
[morituri-trunk] [gst-git] [thomas@ana trunk]$ rip offset find
Trying read offset 0 ...
Trying read offset 6 ...
Trying read offset 12 ...
Trying read offset 48 ...
Offset of device is likely 48, confirming ...

Read offset of device is: 48.

Now, on to ripping and drive selection and TOC caching and such niceties. Someday soon I should be able to actually use this...

While working on morituri I provided some patches to pycdio, the python bindings for libcdio, to handle CD-Text. I was assuming it was the only library or program capable of...

Today was not a particularly good day - I woke up with a huge headache around 4, took a dafalgan, went back to bed, and woke up around 8:30, still with a headache. This sort of thing happens once in a while to me, though it happens a lot less since I reduced my disorganization stress a few years ago.

Anyways, that's not the point. On days like these, I can only do small incremental changes, which I guess is good for polishing. My brain doesn't let me do much more.

So, I finished some loose ends in my ripping code. Here's a file listing of a directory:

[gst-git] [thomas@ana trunk]$ ls -l Bloc\ Party\ -\ Silent\ Alarm
total 559364
-rw------- 1 thomas thomas 35797484 2009-05-15 16:25 00. Bloc Party - Hidden Track One Audio.wav
-rw------- 1 thomas thomas 46214492 2009-05-15 13:01 01. Bloc Party - Like Eating Glass.wav
-rw------- 1 thomas thomas 38838620 2009-05-15 13:02 02. Bloc Party - Helicopter.wav
-rw------- 1 thomas thomas 41531660 2009-05-15 13:03 03. Bloc Party - Positive Tension.wav
-rw------- 1 thomas thomas 35519948 2009-05-15 13:03 04. Bloc Party - Banquet.wav
-rw------- 1 thomas thomas 29480012 2009-05-15 13:04 05. Bloc Party - Blue Light.wav
-rw------- 1 thomas thomas 36928796 2009-05-15 13:05 06. Bloc Party - She's Hearing Voices.wav
-rw------- 1 thomas thomas 46854236 2009-05-15 13:06 07. Bloc Party - This Modern Love.wav
-rw------- 1 thomas thomas 37977788 2009-05-15 13:06 08. Bloc Party - The Pioneers.wav
-rw------- 1 thomas thomas 45809948 2009-05-15 13:07 09. Bloc Party - Price of Gasoline.wav
-rw------- 1 thomas thomas 41082428 2009-05-15 13:08 10. Bloc Party - So Here We Are.wav
-rw------- 1 thomas thomas 41846828 2009-05-15 13:08 11. Bloc Party - Luno.wav
-rw------- 1 thomas thomas 44151788 2009-05-15 13:09 12. Bloc Party - Plans.wav
-rw------- 1 thomas thomas 50010620 2009-05-15 13:10 13. Bloc Party - Compliments.wav
-rw-rw-r-- 1 thomas thomas 1769 2009-05-15 22:33 Bloc Party - Silent Alarm.cue
-rw-rw-r-- 1 thomas thomas 1002 2009-05-15 16:27 Bloc Party - Silent Alarm.m3u


Notice the first file - it's the hidden track one audio. The .cue file references it properly as well. The metadata is retrieved from MusicBrainz (that got added some night last week), and the files are named according to a template (I'm still not sure I like having the track number in my file names, but a certain site demands it). The files match the AccurateRip database, so this rip is basically as good as you can get with EAC. CDText and ISRC codes also get written to the .cue file (although this particular disc doesn't have CDText).

Things still left to be done, in no particular order:

• write a command-line frontend
• write a GUI
• have the second step of the read-and-verify task encode directly to flac (or another lossless format)
• write out a log file (involves interpreting cdparanoia's stderr-progress output, which looks a little tricky
• add code to allow choosing a device
• add some kind of config file so that, for each device (by name), an offset can be stored

That's quite enough for tonight. Hopefully I feel better tomorrow.

Today was not a particularly good day - I woke up with a huge headache around 4, took a dafalgan, went back to bed, and woke up around 8:30, still...

While I'm working on the ripping software, I find myself going back and forth between various references to figure out the small details and the pieces that subtly get interpreted differently between them. As is the case with other projects, I can easily see myself forgetting about these details soon, and cursing myself a year from now for not having written down my clear understanding of today.

So, in an effort to appease my future self, I've started writing down a condensed form of the important information I've come across.

On that page, I'm also comparing various ripping programs and how they handle the various details I consider important for correct ripping. I'll use that information and that chart as the basis for the features of my ripping program.

I'm trying to stay as objective as possible on that page, so feel free to tell me about mistakes, omissions, software I should be adding, ...

By now, I have a good set of goals for my ripping program:

• lossless ripping
• accuracy is the number one goal
• speed is always second to accuracy
• hands-off one-click/command ripping
• separate ripping from metadata fixing
• rip hidden track one audio automatically

With this in mind, I thought yesterday how I could figure out the drive's read offset the way EAC does it. I've come up with a simple program that:

• checks if the current CD is in the AccurateRip database
• if it is, rip the first track with various offsets
• if any of the AccurateRip checksums match, that is most likely the offset for your drive

It took longer to test the program than to write it, since my AccurateRip checksum calculation is currently done purely in Python and thus rather slow.

In any case, using Bat For Lashes' "Fur and Gold":

[gst-git] [thomas@ana trunk]$ PYTHONPATH=$PYTHONPATH:`pwd` python examples/ARcalibrate.py
CDDB disc id 8a0aa10b
AccurateRip URL http://www.accuraterip.com/accuraterip/9/f/f/dBAR-011-00112ff9-00976269-8a0aa10b.bin
4 AccurateRip reponses found
ripping track 1 with offset 46
AR checksum calculated: b880421e
ripping track 1 with offset 47
AR checksum calculated: 4a29a173
ripping track 1 with offset 48
AR checksum calculated: 903b390e
MATCHED against response 3
offset of device is 48
ripping track 1 with offset 49
AR checksum calculated: e7c008f1
[gst-git] [thomas@ana trunk]$


I made the program scan from 46 to 49, knowing that my drive has a +48 read offset. Now I'm going to add an option to choose the range, an option to start with the most common offsets, and think about including using online databases of drive features to start with the one most likely to be correct for your drive.

While I'm working on the ripping software, I find myself going back and forth between various references to figure out the small details and the pieces that subtly get interpreted...