technology

A Fascinating, Forward-Thinking Statistical Decision Making System

It’s commonly known that statistics often work well in aggregate — for instance, the central limit theorem tells us that averages of random variables follow a specific distribution. I don’t wanna give that Gauss dude any more royalties, he’s already balling outta control, so I’m just gonna call it the normal distribution. Plus I feel like saying Gaussian sounds sorta like you think you’re smart, right?

Over the last few days, I learned about a software company that operated a while back. They made some social media stuff and various apps. The company’s decision making process is so ingenious that I don’t even want to name the company. Let’s just call it eJunk, which made various electronic junk of course. But the innovation was really in how they exploited statistical aggregation to make product decisions.

Here’s how that process worked. There were about 100 people in the company, divided unequally into 5 teams. Two teams  (A & B) were size 30, two were size 15 (C & D), and one size 10 (E).

eJunk’s CEO, Elaine Benz, said one day “ya know, we oughta to vote on each important decision. Let’s make each decision be based on a popular vote”.

Then the CPO, let’s call him Homer Simpson, scratched his balding head for a minute, and in his infinite wisdom, responded, “Well, but what if, say, Team E’s votes are kinda overwhelmed by others, cuz they’re a small but important team? … oh, I know, let’s assign a weight to each one of the team’s votes.”

The CTO Qarl, starting with a Q, loved the idea of doing weights; he had done some work in grad school where he weighted some forecasts. “I’ll devise the weighting scheme. Since we don’t want team E to be undervalued let’s do this”  (then he wrote this up on the board):

Team Members Votes
A 30 10
B 30 10
C 15 8
D 15 8
E 10 7

Everyone was pretty convinced at this point.

Elaine then said, “well, okay but let’s make sure each team comes to a consensus within their team, to promote the process of consensus, which is always good”.

Homer: “In that case, within teams, let’s have a popular vote. Whatever the outcome is within that team, all their votes will count toward that outcome. So, uh, like, say we’re voting on whether this new feature should go out or not. Then say in team C, let’s say 9 people vote yes and 6 vote no, then but like, okay, we count ALL their 8 team votes as ‘yes’, cuz the overall consensus of that team was basically yes … it just makes more sense.”

Elaine, who had been drinking earlier that day, unbeknownst to everyone else, emphatically said “That’ll promote unity! I LIKE it”, smashed her gavel on the table (for some reason eJunk’s executives conducted their business in robes, in a courtroom), effectively concluding the meeting.

A decision was made based on this system for the first time. Everything seemed ok. Some more were made. Time passed. Trees grew and all that stuff they show in movies where time passes.

Then a few really odd (and in hindsight, maybe bad) decisions got made. One included buying a lot of stake in MySpace; another included only letting people do Thumbs Up on Facebook; another one was deciding to promote Rebecca Black’s song “Friday”. But most people carried on as usual, trusting the system.

Years later, an internal statistician (her name doesn’t matter, nobody within eJunk liked her) started doing some analysis on the Rebecca Black decision. When that decision was made, the teams were a lot different in size than they had been originally — but it struck her that they had the same, original number of votes per team. The decision results (which somebody had stored conveniently and for apparent reason other than for me to write this post) broke down like this.

Team Members Breakdown of Individual Votes — yes / no Team Votes allocated to YES decision on promoting masterpiece Friday song Team Votes allocated to NO decision on promoting masterpiece Friday song
A 70 20 / 50 0 10
B 60 10 / 50 0 10
C 50 26 / 24 8 0
D 30 17 / 13 8 0
E 10 8 / 2 7 0
Total 220 86 / 139 23 20

She was amazed when she saw that, even though overall, the total popular vote was 139 – 86, in favor of not promoting the masterpiece, the team votes ended up as 23-20, in the opposite direction!

She took it to her manager and said, “look! It looks like we need to change this process, the voting doesn’t make any sense anymore. I found out that 62% of people voted against that Rebecca Black song, but it worked out so that 53% of team votes ended up being FOR it… A bunch of people on Team E have a specific animosity toward the rest of the company and they kinda mess up decisions sometimes because of this system.”

Her manager, Don, replied: “It’s hard to listen to what you’re saying, using all these numbers …. And I love that song, forgot about it…. But  …. Well, we have a well defined process for changing any process. If you want to try, fill out these forms.”

He continued, “And within 6-8 business years, upper management will take a look.”

“A process to change the process? And what’s a business year?”

Don: “..yep…. Yea a business year is like, 365 days, but take out weekends and holidays, so I guess thats like 10 regular people years or something” He went back to solitaire.

She filled the forms out. But management never got around to to looking at the stack of papers, because they ended up making so many more amazing decisions in the next couple business years that some investors decided they really didn’t need to make any more.

So whatever happened to Homer Simpson’s idea? Most tech companies don’t use it now.

But two things happened.

First, someone took a slightly more pure version of it (where you don’t even need to weight the votes!) and published it as Simpson’s paradox.

Second, someone learned about it and said: “Hey, I know, let’s call this brilliant statistical scheme ‘the electoral college’  … and let’s make this the basis for deciding who will be the next leader of the free world.”

trump blue steelben stiller blue steel

The fruitful electoral college, a product of decades of careful statistical thinking

 

Discussion

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s