[lang]

Present Perfect

Personal
Projects
Packages
Patches
Presents
Linux

Picture Gallery
Present Perfect

Amazing Marvin and KeyCombiner

Filed under: friction — Thomas @ 3:27 am

2021-6-3
3:27 am

I recently came across an excellent tool called KeyCombiner that helps you practice keyboard shortcuts (3 sets for free, $29/6 months for more sets). I spent some time to create a set for Amazing Marvin, my current todo manager of choice.

The shareable URL to use in KeyCombiner is https://keycombiner.com/collecting/collections/shared/f1f78977-0920-4888-a86d-d00a7201502e

I generated it from the printed PDF version of Marvin’s keyboard guide and a bunch of manual editing, in a google sheet.

Keyboard shortcuts are great timesavers and help reduce friction, but it’s getting harder to learn them properly, and this tool has been a great help for some other apps, and for figuring out common shortcuts across apps, and for picking custom shortcuts (in other apps) that don’t conflict. If this is a problem you recognize, give KeyCombiner a try.

Recursive storytelling for kids

Filed under: Life,New York City — Thomas @ 3:14 am

2018-11-20
3:14 am

Most mornings I take Phoenix to school, as his school is two blocks away from work.

We take the subway to school, having about a half hour window to leave as the school has a half-hour play window before school really starts, which inevitably gets eaten up by collecting all the things, putting on all the clothes, picking the mode of transportation (no, not the stroller; please take the step so we can go fast), and getting out the door.

At the time we make it out, the subway is usually full of people, as are the cars, so we shuffle in and Phoenix searches for a seat, which is not available, but as long as he gets close enough to a pole and a person who looks like they’d be willing to give up a seat once they pay attention, he seems to get his way more often than not. And sometimes, the person next to them also offers up their seat to me. Which is when the fun begins.

Because, like any parent knows these days, as soon as you sit down next to each other, that one question will come:

“Papa, papa, papa… mag ik jouw telefoon?” (Can I have your phone? – Phoenix and I speak Dutch exclusively to each other. Well, I do to him.)

At which point, as a tired parent in the morning, you have a choice – let them have that Instrument of Brain Decay which even Silicon Valley parents don’t let their toddlers use, or push yourself to make every single subway ride an engaging and entertaining fun-filled program for the rest of eternity.

Or maybe… there is a middle way. Which is how, every morning, Phoenix and I engage in the same routine. I answer: “Natuurlijk mag jij mijn telefoon… als je éérst een verhaaltje vertelt.” (Of course you can have my phone – if you first tell me a story.)

Phoenix furrows his brows, and asks the only logical follow-up question there is – “Welk verhaaltje?” (Which story?)

And I say “Ik wil het verhaaltje horen van het jongetje en zijn vader die met de metro naar school gaan” (I want to hear the story of the little boy and his dad who take the subway to school.)

And he looks at me with big eyes and says, “Dat verhaaltje ken ik niet.” (I don’t know that story)

And I begin to tell the story:

“Er was eens… een jongetje en zijn vader.” (Once upon a time, there was a little boy and his father. Phoenix already knows the first three words of any story.)
“En op een dag… gingen dat jongetje en zijn vader met de metro naar school.” (And one day… the little boy and his father took the subway to school. The way he says “op een dag” whenever he pretends to read a story from a book is so endearing it is now part of our family tradition.)

“Maar toen de jongen en zijn vader op de metro stapten zat de metro vol met mensen. En het jongetje wou zitten, maar er was geen plaats. Tot er een vriendelijke mevrouw opstond en haar plaats gaf aan het jongetje, en het jongetje ging zitten. En toen stond de meneer naast de mevrouw ook recht en de papa ging naast het jongetje zitten.” (But when the little boy and his father got on the subway, it was full of people. And the little boy wanted to sit but there was no room. Until a friendly woman stood up and gave up her seat to the little boy, so the little boy sat down. And then the man next to the woman also stood up and his father sat down next to him.)

“En toen de jongen op de stoel zat, zei het jongetje, Papa papa papa papa papa papa papa…”(And when the boy sat down on the chair, he said Papa papa papa papa papa papa)

“Ja?, zei papa.” (Yes?, said papa.)

“Papa, mag ik jouw telefoon”? (Papa, can I have your phone?)

“Natuurlijk jongen….. als je éérst een verhaaltje vertelt.” (Of course son… if you first tell me a story.)

At which point, the story folds in on itself and recurses, and Phoenix’s eyes light up as he mouths parts of the sentences he already remembers, and joins me in telling the next level of recursion of the story.

I apologize in advance to all the closing parentheses left dangling like the terrible lisp programmer I’ve never given myself the chance to be, but making that train ride be phoneless every single time so far is worth it.

A morning in San Francisco

Filed under: General — Thomas @ 5:57 am

2017-1-29
5:57 am

This morning in San Francisco, I check out from the hotel and walk to Bodega, a place I discovered last time I was here. I walk past a Chinese man swinging his arms slowly and deliberately, celebrating a secret of health us Westerners will never know. It is Chinese New Year, and I pass bigger groups celebrating and children singing. My phone takes a picture of a forest of phones taking pictures.

I get to the corner hoping the place is still in business. The sign outside asks “Can it all be so simple?” The place is open, so at least for today, the answer is yes. I take a seat at the bar, and I’m flattered when the owner recognizes me even if it’s only my second time here. I ask her if her sister made it to New York to study – but no, she is trekking around Columbia after helping out at the bodega every weekend for the past few months. I get a coffee and a hibiscus mimosa as I ponder the menu.

The man next to me turns out to be her cousin, Amir. He took a plane to San Francisco from Iran yesterday after hearing an executive order might get signed banning people with visas from seven countries to enter the US. The order was signed two hours before his plane landed. He made it through immigration. The fact sheet arrived on the immigration officer’s desks right after he passed through, and the next man in his queue, coming from Turkey, did not make it through. Needles and eyes.

Now he is planning to get a job, and get a lawyer to find a way to bring over his wife and 4 year old child who are now officially banned from following him for 120 days or more. In Iran he does business strategy and teaches at University. It hits home really hard that we are not that different, him and I, and how undeservedly lucky I am that I won’t ever be faced with such a horrible choice to make. Paria, the owner, chimes in, saying that she’s a Iranian Muslim who came to the US 15 years ago with her family, and they all can’t believe what’s happening right now.

The church bell chimes a song over Washington Square Park and breaks the spell, telling me it’s eleven o’clock and time to get going to the airport.

Puppet/puppetdb/storeconfigs validation issues

Filed under: puppet,sysadmin — Thomas @ 9:31 pm

2016-10-9
9:31 pm

Over the past year I’ve chipped away at setting up new servers for apestaart and managing the deployment in puppet as opposed to a by now years old manual single server configuration that would be hard to replicate if the drives fail (one of which did recently, making this more urgent).

It’s been a while since I felt like I was good enough at puppet to love and hate it in equal parts, but mostly manage to control a deployment of around ten servers at a previous job.

Things were progressing an hour or two here and there at a time, and accelerated when a friend in our collective was launching a new business for which I wanted to make sure he had a decent redundancy setup.

I was saving the hardest part for last – setting up Nagios monitoring with Matthias Saou’s puppet-nagios module, which needs External Resources and storeconfigs working.

Even on the previous server setup based on CentOS 6, that was a pain to set up – needing MySQL and ruby’s ActiveRecord. But it sorta worked.

It seems that for newer puppet setups, you’re now supposed to use something called PuppetDB, which is not in fact a database on its own as the name suggests, but requires another database. Of course, it chose to need a different one – Postgres. Oh, and PuppetDB itself is in Java – now you get the cost of two runtimes when you use puppet!

So, to add useful Nagios monitoring to my puppet deploys, which without it are quite happy to be simple puppet apply runs from a local git checkout on each server, I now need storedconfigs which needs puppetdb which pulls in Java and Postgres. And that’s just so a system that handles distributed configuration can actually be told about the results of that distributed configuration and create a useful feedback cycle allowing it to do useful things to the observed result.

Since I test these deployments on local vagrant/VirtualBox machines, I had to double their RAM because of this – even just the puppetdb java server by default starts with 192MB reserved out of the box.

But enough complaining about these expensive changes – at least there was a working puppetdb module that managed to set things up well enough.

It was easy enough to get the first host monitored, and apart from some minor changes (like updating the default Nagios config template from 3.x to 4.x), I had a familiar Nagios view working showing results from the server running Nagios itself. Success!

But all runs from the other vm’s did not trigger adding any exported resources, and I couldn’t find anything wrong in the logs. In fact, I could not find /var/log/puppetdb/puppetdb.log at all…

fun with utf-8

After a long night of experimenting and head scratching, I chased down a first clue in /var/log/messages saying puppet-master[17702]: Ignoring invalid UTF-8 byte sequences in data to be sent to PuppetDB

I traced that down to puppetdb/char_encoding.rb, and with my limited ruby skills, I got a dump of the offending byte sequence by adding this code:


Puppet.warning "Ignoring invalid UTF-8 byte sequences in data to be sent to PuppetDB"
File.open('/tmp/ruby', 'w') { |file| file.write(str) }
Puppet.warning "THOMAS: is here"

(I tend to use my name in debugging to have something easy to grep for, and I wanted some verification that the File dump wasn’t triggering any errors)
It took a little time at 3AM to remember where these /tmp files end up thanks to systemd, but once found, I saw it was a json blob with a command to “replace catalog”. That could explain why my puppetdb didn’t have any catalogs for other hosts. But file told me this was a plain ASCII file, so that didn’t help me narrow it down.

I brute forced it by just checking my whole puppet tree:


find . -type f -exec file {} \; > /tmp/puppetfile
grep -v ASCII /tmp/puppetfile | grep -v git

This turned up a few UTF-8 candidates. Googling around, I was reminded about how terrible utf-8 handling was in ruby 1.8, and saw information that puppet recommended using ASCII only in most of the manifests and files to avoid issues.

It turned out to be a config from a webalizer module:


webalizer/templates/webalizer.conf.erb: UTF-8 Unicode text

While it was written by a Jesús with a unicode name, the file itself didn’t have his name in it, and I couldn’t obviously find where the UTF-8 chars were hiding. One StackOverflow post later, I had nailed it down – UTF-8 spaces!


00004ba0 2e 0a 23 c2 a0 4e 6f 74 65 20 66 6f 72 20 74 68 |..#..Note for th|
00004bb0 69 73 20 74 6f 20 77 6f 72 6b 20 79 6f 75 20 6e |is to work you n|

The offending character is c2 a0 – the non-breaking space

I have no idea how that slipped into a comment in a config file, but I changed the spaces and got rid of the error.

Puppet’s error was vague, did not provide any context whatsoever (Where do the bytes come from? Dump the part that is parseable? Dump the hex representation? Tell me the position in it where the problem is?), did not give any indication of the potential impact, and in a sea of spurious puppet warnings that you simply have to live with, is easy to miss. One down.

However, still no catalogs on the server, so still only one host being monitored. What next?

users, groups, and permissions

Chasing my next lead turned out to be my own fault. After turning off SELinux temporarily, checking all permissions on all puppetdb files to make sure that they were group-owned by puppetdb and writable for puppet, I took the last step of switching to that user role and trying to write the log file myself. And it failed. Huh? And then id told me why – while /var/log/puppetdb/ was group-writeable and owned by puppetdb group, my puppetdb user was actually in the www-data group.

It turns out that I had tried to move some uids and gids around after the automatic assignment puppet does gave different results on two hosts (a problem I still don’t have a satisfying answer for, as I don’t want to hard-code uids/gids for system accounts in other people’s modules), and clearly I did one of them wrong.

I think a server that for whatever reason cannot log should simply not start, as this is a critical error if you want a defensive system.

After fixing that properly, I now had a puppetdb log file.

resource titles

Now I was staring at an actual exception:


2016-10-09 14:39:33,957 ERROR [c.p.p.command] [85bae55f-671c-43cf-9a54-c149cede
c659] [replace catalog] Fatal error on attempt 0
java.lang.IllegalArgumentException: Resource '{:type "File", :title "/var/lib/p
uppet/concat/thomas_vimrc/fragments/75_thomas_vimrc-\" allow adding additional
config through .vimrc.local_if filereadable(glob(\"~_.vimrc.local\"))_\tsource
~_.vimrc.local_endif_"}' has an invalid tag 'thomas:vimrc-" allow adding additi
onal config through .vimrc.local
if filereadable(glob("~/.vimrc.local"))
source ~/.vimrc.local
endif
'. Tags must match the pattern /\A[a-z0-9_][a-z0-9_:\-.]*\Z/.
at com.puppetlabs.puppetdb.catalogs$validate_resources.invoke(catalogs.
clj:331) ~[na:na]

Given the name of the command (replace catalog), I felt certain this was going to be the problem standing between me and multiple hosts being monitored.

The problem was a few levels deep, but essentially I had code creating fragments of vimrc files using the concat module, and was naming the resources with file content as part of the title. That’s not a great idea, admittedly, but no other part of puppet had ever complained about it before. Even the files on my file system that store the fragments, which get their filename from these titles, happily stored with a double quote in its name.

So yet again, puppet’s lax approach to specifying types of variables at any of its layers (hiera, puppet code, ruby code, ruby templates, puppetdb) in any of its data formats (yaml, json, bytes for strings without encoding information) triggers errors somewhere in the stack without informing whatever triggered that error (ie, the agent run on the client didn’t complain or fail).

Once again, puppet has given me plenty of reasons to hate it with a passion, tipping the balance.

I couldn’t imagine doing server management without a tool like puppet. But you love it when you don’t have to tweak it much, and you hate it when you’re actually making extensive changes. Hopefully after today I can get back to the loving it part.

Media unit for geeks with kids?

Filed under: General — Thomas @ 4:13 am

2015-9-21
4:13 am

Phoenix is growing up quickly and pretty soon he’ll be crawling around the house. So it’s time for babyproofing.

For the past year, I’ve been looking all over the internet for decent media units that we could get. IKEA used to have some good ones, but it doesn’t look like they have any decent model anymore.

So I turn to the geeky side of the internet, as I’m sure there’s lots of people out there who’ve gone through the same problem with an infant growing up.

So far, I’m thinking:

  • closed at the front, except for a big slot large enough to fit my central speaker (I admit I went large with a PolkAudio A4)
  • thick glass – the kind that lets IR through, but not babies when they smash into it
  • plenty of holes out the back for ventilation – in fact, mostly open
  • useful leads for cables if possible
  • 50-60 inch wide because the TV needs to go on top
  • high enough – at least 80 cm. So many units are low, why?
  • deep enough – so many media units do not even fit a standard AV receiver, let alone leave enough space for air to circulate so the unit doesn’t burn up
  • cubby holes/shelves high enough so said unit fits as well
  • not butt ugly or escaped from the eighties
  • can hold A/V receiver, standard Digital TV unit, router, a NAS, a PS3, and an Atari VCS 2600. Bonus points for space left over for a future Megadrive or NES.
  • easy to attach to a wall
  • built-in custom rack for Atari VCS 2600 cartridges (though I’d begrudgingly accept a unit that ticks all the other boxes)

Any requirements I’m missing? Anyone want to share which unit made them happy?

Update: if it matters, this is for a smallish appartment in Manhattan – preference for no DIY.

Next Page »
picture