Strategy. Innovation. Brand.
Inference, Frequentism, and Sex
Statistically significant.
I understand inferential statistics reasonably well but I’m a rookie at Bayesian statistics. What’s the difference? Well, I’m glad you asked.
Inferential statistics allow us to infer some conclusion. Let’s say we want to study a hypothesis that Scientific American put forth last year: Men who do more housework have less sex. That’s an interesting thought and one that men – especially married men – might want to investigate.
Where do we start? First, we need to define the population. Let’s say we’re only interested in men in America. So, the hypothesis becomes:
Men in America who do more housework have less sex.
Now, all we have to do is interview all the men in America about the chores they do and how often they have sex. There are roughly 147 million men in America so this is going to take some time. Indeed, it will take so long that our attitudes toward sex and housework might change in the interim. The study would be useless.
We need a faster way. We might take a random sample of men and interview them. We gather the data and find that the hypothesis is true in the sample. The men we interviewed who did more chores also had less sex.
Now we have to ask ourselves, what’s the probability that the sample accurately represents the population? We found an inverse correlation (more housework, less sex) in the sample. Can we infer that this is also true in the population?
To do this, we need to play with probabilities. We randomly selected the men in the sample, which means the sample probably represents the population (but maybe not). Generally speaking, larger random samples are more likely to accurately represent the population.
We can also calculate the probability that the correlation found in the sample also exists in the population. By general agreement, if it’s more than 95% probable (less than 5% improbable), we declare that the finding is statistically significant. In other words, we believe that the finding is real and not caused by errors in the way we chose the sample. We infer that it exists in the population as well as the sample.
Two things to note here:
- Statistical significance has to do with probability, not size. It’s not the same as saying, “Tom is significantly smarter than Joe”. A statistically significant difference may be quite small.
- Five per cent of the time – one time out of twenty – the finding is flat out wrong. Yikes! The five per cent threshold is generally used only in the social sciences. In the medical sciences, we normally use a one per cent or one-tenth of one per cent threshold to declare statistical significance.
Note that our finding – which is called a frequentist probability — represents a point in time. It’s essentially a snap shot. We noted that, if our study takes too long, conditions might change and invalidate the study. Indeed the Scientific American study cites data collected from 1992 to 1994. Perhaps conditions have changed since then. So how accurate is this?
That’s where Bayesian statistics come in to play. They allow us to add in new information as it becomes available. Let’s say, for instance, that women’s attitudes toward men who do house work evolve over time. We could factor in the new information and recalculate the probabilities.
Bayesian statistics are complicated and hard to compute. We’ve only been using them widely in the recent past as more powerful computers have become available. Still, they can help us work out complex problems that used to be way beyond our capabilities.
I’ll write more about Bayesian probabilities as I learn more. In the meantime, I’m going to sell the vacuum cleaner.
My Good Judgment
Who’s the better forecaster?
We’re all familiar with the idea of placing a bet on a football match. You can bet on many different outcomes: which team will win; by how much; how many total goals will be scored on so on. With a large number of bettors, the aggregate prediction is often remarkably accurate. It’s what James Surowiecki calls The Wisdom of Crowds.
Prediction markets aim to do the same thing but broaden the scope. Instead of betting on sports, they bet on political or economic or natural events. For instance, What’s the probability that: Greece will exit the Euro in 2015; or that nuclear weapons will be used in the India/Pakistan conflict before 2018; or that Miami will have more than 100 flood days by 2020?
The forecasting questions are quite precise and always bounded by a time limit. There should be no question whether the event happens or not. In other words, we can actually judge how accurate the forecasts are.
Why is judging so important? As Philip Tetlock pointed out in Expert Political Judgment: How Good Is It? How Can We Know?, we traditionally don’t measure the accuracy of expert political predictions. Pundits make predictions and nobody checks them. Indeed, Tetlock argues that most pundits make predictions as a way to advertise their consulting businesses. The bolder the prediction, the more powerful the ad.
When Tetlock actually measured the accuracy of expert political predictions, he discovered they were essentially useless. Tetlock writes, “The results were startling. The average expert did only slightly better than random guessing.” Remember that the next time you read an expert prediction.
You may remember that I wrote about a prediction market – InTrade – during the 2012 elections in the United States. Based in Ireland, InTrade allowed people all over the world to place bets on who would win the presidential election, as well as various Senate, gubernatorial, and congressional elections. InTrade’s electoral predictions were remarkably accurate. (It did less well in predicting Supreme Court decisions).
Unfortunately, the U.S. government saw InTrade as a form of online gambling. As such, it needed to be tightly regulated or perhaps even suppressed. It’s a complicated story — and may have involved “financial irregularities” on InTrade’s part — but, in 2013, InTrade decided to close its doors.
So, how can we use prediction markets in the United States? In its wisdom, another agency of the federal government, the Intelligence Advanced Reesarch Projects Activity (IARPA), started a prediction tournament called Aggregative Contingent Estimation (ACE). IARPA/ACE has run a prediction tournament for the past three years. Various teams – mainly from academic institutions – participate for the honor of being named the most accurate forecaster.
And who wins these tournaments? A team called The Good Judgment Project (GJP) put together by none other than Philip Tetlock. GJP selects several thousand volunteers, gives them some training on how to make forecasts, and asks them to forecast the several hundred questions included in the IARPA/ACE tournament.
The Good Judgment project wins the tournament consistently. They must be doing something right. And who is the newest forecaster on the Good Judgment team? Well… with all due modesty, it’s me.
To say the least, I’m excited to participate – and I expect to write about my experiences over the coming months. I can predict with 70% confidence that I won’t be a world class forecaster in the first go round. But I may just learn a thing or two and improve my accuracy over time. Wish me luck.
How To Have A Good Idea – Part 1
Idea generator.
We often think that good ideas are random events. We might say, for instance, that “the idea just popped into my head” or it “just occurred to me.” We may convince ourselves that we’re not the authors of our own ideas. They just happen.
But having a good idea is really no different than any other skill. The more we practice, the better we become. We can train ourselves to create more ideas. The more ideas we produce, the more chance we have of producing good ones. As Scott Adams, the creator of Dilbert, has remarked, “Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.”
So how do you create ideas and, especially, how do you create good ones? The first step is to pay attention. We all have good ideas from time to time. Pay attention to what you’re doing when you have a good idea. Then do more of that.
With that thought in mind, I’ve asked the students in several editions of my Critical Thinking class what they were doing the last time they had a good idea. Here’s a list, in no particular order.
Taking a walk – I haven’t kept precise count but this is certainly one of the most frequently mentioned activities. It’s also my favorite. Whenever I need a good idea, I go for a walk.
Out for a run – like walking, only faster.
Going for a drive – also fairly popular and may be related to highway hypnosis.
Taking a shower – another frequent one.
Reading – this one comes up fairly often, especially reading in bed before falling asleep. As I’ve noted before, sleepy people seem more creative.
Meditation – most of my students don’t meditate but the ones who do often mention it as a time when they get good ideas.
On the subway or bus, commuting – a routine action repeated most every business day; apparently the mind wanders.
On the elevator – this was a student from New York where I gather they take long elevator rides.
Lifting weights at the gym – I suspect that System 1 is lifting the weights while System 2 is rummaging around in more interesting topics.
Dreaming – this comes up occasionally. As one student put it, “I had a fabulous idea for work and had to wake up and write it down.”
Listening to others and listening to colleagues – nobody has told me that they have a good idea when talking. A number of students have told me the ideas come when they’re listening.
Mowing the lawn – it’s a good time to let your mind wander.
Over a cup of coffee in the quiet morning before the kids get up – what a delicious time.
Sitting on the couch talking and strategizing the next day’s plan with my wife – another lovely moment.
I’m struck by how many of the idea-generating activities involve motion. We’re walking or running or driving the car or riding the subway. We’re moving through space and there’s a lot to stimulate our System 1. Yet we don’t have to do much consciously; our System 2 is free to wander. We’re daydreaming but we don’t realize it.
Many of the activities find us alone – like driving or meditating – or alone in a crowd – like riding the subway. We seem to need some stimulus but we also need to withdraw into a quiet, inner space.
Not many students have mentioned having good ideas while in a meeting. But when they do, they often mention the alone-in-a-crowd experience. They may not know many people in the meeting or they’ve mentally checked out for a while. There’s some stimulus but the mind is wandering.
It occurs to me that most of these activities are a form of meditation. We don’t think of it as such but we’re occupying System 1 with routine activities – not unlike chanting a mantra – and allowing System 2 to wander. We don’t think of ourselves as a nation of meditators but perhaps we are.
So, how about it? What were you doing the last time you had a good idea?
When Should You Name Your Baby?
University of Guanajuato
In 1980, Suellen and I moved to Mexico where I was invited to teach at the University of Guanajuato. A lovely mountain town about 200 miles northwest of Mexico City, Guanajuato is rich in history, culture, and tradition. It’s the heart of the Bajío, a broad, fertile region known as the breadbasket of Mexico. Graham Greene’s novel, The Power and The Glory, uses the Bajío as a backdrop.
My students were in their twenties and getting married and having babies. During our year in Guanajuato, Suellen and I were invited to any number of “baby welcoming” parties. About a month after a baby arrived, the parents would host a party to introduce the baby to family, friends, and neighbors. It was like being a very small debutante.
The parties were quite touching. Attendees took note of the fact that a new member of the community had arrived. We implicitly agreed to help the child grow and prosper. We also told stories, offered advice, and gave presents.
At many of the parties, the baby did not yet have a name. I often asked about this: “Why haven’t you named the baby yet?” The more-or-less standard response: “We’ve only just met him. We need to live with him for a while to learn which name fits him best.”
I sometimes noted that, in the United States, we typically named our babies well before they arrived. To which one of my students remarked, “No wonder you gringos are so screwed up. You all have the wrong names!”
So when should you name your baby? It was a question I had never considered before… and that’s the point here. A central tenet of critical thinking is that we should question our own assumptions. As the world has changed, have our assumptions changed as well? Are they still valid? Were they ever?
But how do you question your assumptions if you don’t realize that you’re making assumptions? I assumed that one should name a baby before it arrives. I never questioned it. Why would I? It’s the natural order of things, isn’t it?
So, how do you uncover your unknown assumptions? Here are a few things that have worked for me:
- Learn a new language – many assumptions are bound up in our language; especially the metaphors we use. Learn a new language and you’ll pick up new metaphors and identify hidden assumptions.
- Travel to a different country – paraphrasing Yogi Berra, you can learn a lot just by observing.
- Talk to people who aren’t like you – living in a diverse community (or working in a diverse company) helps you think more clearly. You probably won’t suffer from groupthink.
- Study history – it’s amazing what people used to think.
- Ask yourself a simple question – why do I think that?
Questioning your assumptions doesn’t necessarily mean that you should change them. You may find that they still work quite well. We didn’t change our baby naming assumptions, for instance. We named Elliot well before he arrived and he turned out just fine (I assume).
How Are You? Your Keyboard Knows.
Know-it-all.
Some months ago I took an online course — a MOOC — offered through Coursera. To identify me, Coursera’s security system asked me to type in approximately three sentences of text. Whenever the system needed to identify me again, it sampled my keystrokes. It could tell by the way I type that I was indeed the one and only G. Travis White.
That’s a pretty neat trick. The Coursera system doesn’t need to read my fingerprint or do an eye scan. It just needs to observe my typing skills. The system can easily distinguish me from all the other students in class and, conceivably, from every other human on earth. It’s simple, cheap, and hard to mimic.
So, what else can a keyboard do? Normally, when we interact with a computer, we’re transferring information. We’re asking questions and receiving answers. We’re issuing command and expecting responses. Whether we type fast or slow or hard or soft, we expect the computer to react the same way to the same input. We’re transferring information and nothing else.
But if a keyboard can uniquely identify us, could it also do more? Could it detect our emotions? And, if so, could it change the computer’s behavior based on the emotions it detects?
These are questions that several researchers at the Islamic University of Technology in Bangladesh investigated in an article recently published in Behaviour & Information Technology. As the authors point out, “Affective computing is the field that detects user emotion… [and if a machine]… can detect user emotions and change its behavior accordingly, then using machines can be more effective and friendly.”
So, how do you teach a machine to detect emotions? The researchers chose keystrokes for the same reasons that Coursera did: they’re cheap and available. The researchers also chose to combine two different methods of analysis that had previously been studied:
- Keyboard dynamics – including dwell time (how long your finger stays on a key), flight time (how long it takes to get to the next key), and content attributes (number of deletes, backspaces, etc.).
- Text pattern analysis – this usually involves identifying “affective content” by spotting specific keywords and analyzing syntax as the user chats with other users.
The researchers aimed to identify seven different emotions – anger, disgust, fear, guilt, joy, sadness, and shame as defined in the International Survey on Emotion Antecedents and Reactions (ISEAR).
And how did it work? Remarkably well. Using the two methods together produced better results than either method independently. Better yet, the results were surprisingly consistent across the range of emotions. Here’s how often the system detected an emotion correctly:
Joy 87%
Anger 81%
Guilt 77%
Disgust 75%
Sadness 71%
Shame 69%
Fear 67%
What’s next? How about a computer that responds to your emotional state by changing its behavior in a variety of subtle and not-so-subtle ways? In other words, it becomes a true personal assistant rather than merely mechanical device. Imagine the possibilities.
My Social Media
