Votes for talks at open source conferences |
2013-05-07
|
I've never been a fan of voting for talks, because it tends to be poorly implemented under the guise of democracy. Of course it's easy for me to talk, I've never organized anything at that scale.
I'll give two examples on why I feel this way, one of which triggering today's blog post.
First off, my colleague Marek submitted a talk to Djangocon. The talk was about how to use feat (a toolkit we wrote for livetranscoding) to serve Django pages, but in such a way that they can use Deferreds to remove the concurrency bottleneck of "1 request at a time" per process running Django.
Personally, to me, this is one of the most irritating design choices of Django - from the ground up it was built synchronously (which could have been fine in most places). But the fact that, when you get a request, you have to always synchronously respond to it (and block every other request for that process in the meantime) is a design choice that could have easily been avoided.
In our particular use case, it was really painful. If our website has to do an API request to some other service we don't control that can easily take 30 seconds, our process throughput suddenly becomes 2 pages per minute. All the while, the server is sitting there waiting.
Yes, you can throw RAM at the problem and start 30 times more processes; or thread out API requests; or farm it out to Celery, and do some back-and-forthing to see when the call's done. Or do any other number of workarounds for a fundamental design choice.
Since we like Twisted, we preferred to throw Twisted at the problem, and ended up with something that worked.
Anyway, that's a lot of setup to explain what the talk was about. Marek submitted the talk to DjangoCon, and honestly I didn't expect it to get much traction because, when you're inside Django, you think like Django, and you don't really realize that this is a real problem. Most people who do realize it switch away to something else.
But to my surprise, Marek's talk was the most-voted talk! I wish I could link to the results, but of course that vote site is no longer online.
I guess I expected that would mean he'd be presenting at DjangoCon this year. So I asked him today when his talk was, and he said "Oh that's right. I did not get accepted."
Well, that was a surprise. Of course, the organising committee reserves the right to decide on their own - maybe they just didn't like the talk. But if you ask your potential visitors to vote, you'd expect the most-voted talk to make it on the schedule no ?
The feedback Marek got from them was surprising too, though. Their first response was that this talk was too similar to another talk, titled "How to combine JavaScript & Django in a smart way". Now, I'm not a JavaScript expert, but from the title alone I can already tell that it's very unlikely that these two talks have many similarities beyond the word 'Django'.
After refuting that point, their second reason was that they wanted more experienced speakers (but they didn't ask Marek for his experience), and their third reason was that the talk was in previous editions of DjangoCon US/EU (it's unclear whether they meant his talk or the JavaScript one, but Marek's definitely wasn't, and we couldn't find any mention of the other talk in previous conferences. I'm also not sure why that even matters one way or the other. This email thread was in Polish, so I have to rely on Marek's interpretation of it)
Personally, my reaction would have been to complain to the organizers or Django maintainers. Marek's flegmatic attitude was much better though - after such an exchange, he simply doesn't want to have anything to do with the conference.
He's probably right - it's hard to argue with someone who doesn't want to invite you and is lying about the reasons.
The second example is BCNDevCon, a great conference here in Barcelona, organized by a guy who used to work for Flumotion who I have enormous respect for. I've never seen anyone create such a big conference over so little time.
He believes strongly in the democratic aspect, and as far as I can tell constructs the schedule solely based on the votes.
Sadly I didn't go to the last one, and the reason is simply because I felt that the talks that made it were too obviously corporate. A lot of talks were about Microsoft products, and you could tell that they won votes because people's coworkers voted on talks. I'm not saying that's necessarily wrong - given that he worked at our company and has friends here, I'm sure people working here presenting at his conference have also done vote tending. It's natural to do so. But there should be a way to balance that out.
I think the idea of voting is good, but implementation matters too. Ideally, you would only want people that actually are going to show up to vote. I have no idea how you can ensure that, though. Do you ask people to pre-pay ? Do you ask them to commit to pay if at least 50% of their votes make it in the final schedule, kickstarter-style ?
These two examples are on opposite extremes of voting. One conference simply disregards completely what people vote on. If I had voted or bought a ticket, I would feel lied to. Why waste the time of so many people? The other conference puts so much stock in the vote, that I feel the final result was strongly affected. I seriously doubt all those Windows 8 voters actually showed up.
Does anyone have good experiences with conference voting that did work? Feel free to share!
I definitely see what you mean there. For Flock (attendance is free), I think this is less of an issue. We are trying to mitigate this by having the voting be anonymized (you can’t see who is giving the talk, just the title and abstract) and having a committee with members from the Fedora Board, FESCo, and FAMSCo on it to make the final call on who makes the cut (based on the voting, but not strictly copied from it).
But hey, it might still suffer from that problem. We’re breaking new-ish ground here.
Comment by Tom Callaway — 2013-05-07 @ 20:39
That could definitely help. Although I assume a presenter can easily tell his/her friends the name of the talk. But at least other people aren’t necessarily selecting talks simply on names they recognize.
Maybe another thing that could help is a minimum amount of votes – so that you know that the person in voting at least took the effort in going through more abstracts than just the one they know about.
Comment by Thomas — 2013-05-07 @ 21:58
I had a very similar experience with DjangoCon. I submitted several talks back in 2011. None were accepted, and the reasoning for one made me not want to participate again. Although I was not privy to the exact discussion, reviewers and organizers felt a previous talk I had presented at PyCon in 2011 presented “non-standard and controversial practices were framed as the normal and accepted ways of doing thingsâ€. Obviously this “raised concerns of misleading less experienced developersâ€.
Comment by Corey Oordt — 2013-05-08 @ 12:44
Open Source Bridge provides a way for individuals to mark a proposed session with a favstar, and the organizers take those stars into account when accepting or rejecting proposals, but no one (outside the conference committee) can see which proposals have gotten the most stars. There’s also a way to contact the organizers privately about any proposal to offer feedback. I think this is a good solution, especially because the system retains the favstars once the final schedule goes up — and that way you can easily see what talks you’d like to attend and remember to go to them.
Comment by Sumana Harihareswara — 2013-05-08 @ 22:45