But that doesn’t mean we don’t like you.

Partly, it’s a function of culture. It’s hard to imagine a society more biased against introverts than modern America. Reticence to engage in group activities is seen as a failing, even a potential sign of deeper psychological problems. (Has the phrase “he was a loner, kept to himself” ever been uttered in a positive context?) Admitting “I’m not really a people person” is enough to tank a job interview. And if you do get hired, opting to head home after work rather than hit the bar with colleagues can lead to accusations of shyness or snobbery.

Misunderstandings between introverts and the extroverts who love them can even be hard on marriages. In his book The Audacity of Hope, Obama describes the adjustments that he and Michelle went through early in their life together. In addition to leaving the butter out after breakfast and forgetting to “twist the little tie around the bread bag,” the future president liked time alone to work and think—to the point that he left his new wife feeling neglected. Obama would “often spend the evening holed up in my office in the back of our railroad apartment; what I considered normal often left Michelle feeling lonely.”

Accounts like that are what lead journalists and armchair psychologists to speculate on Obama’s introversion. Michael Lewis’ recent Vanity Fair profile of the president, while largely flattering, also helped paint a picture of an introvert hurting from his loss of privacy. (Obama enjoys being in the Oval Office on the weekends, for instance, when his aides aren’t there.)

Wednesday night’s lackluster debate performance added to the impression that the president’s personality may get in the way of his political prowess. “Obama behaved like an introvert, and he got steamrolled by the extroverted energy of Mitt Romney,” writes Sophia Dembling, whose book The Introvert’s Way: Living a Quiet Life in a Noisy World is due out in December.

But can Obama—or anyone else, for that matter—truly be “typed” as an introvert or extrovert? Personality tests like the Myers-Briggs exam I was subjected to (technically called the Myers-Briggs Type Indicator) are widely used in corporate settings. By some estimates, 2.5 million Americans take it every year. Yet it’s been widely questioned in psychological circles, and a 1991 review by the National Research Council found it largely ineffective. Still, the questionable validity of personality tests doesn’t mean that introversion itself isn’t a real phenomenon, and the observable evidence certainly suggests that Obama qualifies for the “Scarlet I.”

Critics in the so-called chattering class (that description alone tells you it must be packed with extroverts) suggest that Obama’s reluctance to chat up congressmen or hobnob with donors has hurt his legislative agenda and cost him valuable campaign dollars. Extroverts don’t necessarily make better leaders, writes Susan Cain, author of Quiet: The Power of Introverts in a World That Can't Stop Talking, in a recent New York Times op-ed. But they certainly have an easier time standing up to the constant exposure of modern political life. As Wednesday night showed, the president’s tendency to remain calm and low key can cut both ways: Where some observers see a cool head and quiet confidence, others perceive him as detached and coldly unemotional, cowed by a bombastic opponent—or worse yet, imperious and annoyed that he has to stand up there and explain himself to the rest of the world.

If that performance costs him votes, the president might finally get the privacy we introverts all crave—four years earlier than he hopes.