Alex Blaze

Journalist Investigates 'LGBT Population Is 9 Million' Claim

Filed By Alex Blaze | April 18, 2011 11:00 AM | comments

Filed in: Media
Tags: Gary Gates, math, stistics, survey, Wall Street Journal, Williams Institute

A couple of weeks ago I posted about that Williams Institute brief that said that there were nine million LGBT people in the US. Gary Gates averaged results from several problematic studies that asked people what their sexuality was and arrived at 3.5% of the population being LGBT, assuming that averaging several untruths of unknown merit produces the truth.

paris-pride.JPGUnsurprisingly, the Wall Street Journal found that other demographers don't agree with his methods:

One problem they cited with Dr. Gates's findings is that they combine results from surveys with different sample sizes and interview formats. The California Health Interview Survey canvassed about 50,000 Californians in 2009 by phone, finding that 3.2% identified themselves as gay, lesbian or bisexual. In contrast, roughly 5,900 people took Indiana University's online National Survey of Sexual Health and Behavior in 2009, and nearly twice as many-- 5.6%--identified themselves that way.

"I think there are a lot of problems with every one of those data sets," says Randall Sell, associate professor at Drexel University's school of public health. A concern, he says, is that people are more likely to reveal their sexual identity via computer than by phone or in person.

There are problems with every one of those studies that Gates averaged together, said Professor Sell. The Wall Street Journal put together a small graphic to show just how much variance can be caused by methodology:

gay-graphic.png

The lowest result in Gates's average was based on face-to-face interviews. Is anyone surprised that people are much less likely to tell someone they're talking to that they're queer instead of an online poll?

Also, does it make them any less gay or bisexual if they're not willing to say it in a face-to-face interview?

A companion piece in the Journal brings up those Witeck-Combs polls we hear about every now and then, which are also online polls (conducted by Harris Interactive). They produce another number, a number which, for no given reason, wasn't averaged into Gates's final number:

Other estimates are even higher than the top of the range Gates included. Witeck-Combs Communication, a Washington, D.C., marketing and public-relations company, has since 2000 been surveying gays and lesbians in a partnership with Harris Interactive. Based on roughly 150 surveys, Bob Witeck, chief executive and co-founder of Witeck-Combs, estimates that 6.7% of American adults are gay, lesbian, bisexual or transgendered. However, LGBT people also tend to be early adopters of technology, according to the firm's surveys -- which might also make them more comfortable joining an online polling panel such as Harris's. "It's possible" that explains the higher numbers, Witeck said.

"There's some risk and all forms of bias creep into methodologies and sampling," Witeck elaborated in an email. "Nonetheless, I think the Harris model gives us a high degree of confidence that any bias in this direction is helped by their propensity weighting process."

6.7% is a lot higher than 3.5%. Does anyone think that Harris Interactive is deliberately fudging the data? Finding how many people are LGBT is impossible for a variety of reasons I laid out in my previous post, but is there an accusation against Harris Interactive of adding another layer of conscious bias? If there is, I haven't heard about it. And yet their online survey produced different results than the online survey Gates cited.

How does Gates respond to these charges against his brief?

Dr. Gates says without more information about the validity of each survey, averaging the results is the best compromise. "You can make an argument they're all credible," he says.

If you ever wondered why people who study the hard sciences (physics, biology...) scoff at the social sciences and use quotation marks around the "science" part, it's because of statements like the one above. Falsehoods don't compromise to produce the truth. Studies asking different questions don't compromise to answer another question. It's like how teaching both evolution and creationism to students isn't a compromise - they both can't be literally correct at the same time.

Gates cites five surveys on sexuality and health in the US that asked people their sexual orientation. The percent who identified as gay, lesbian, and bisexual (combined) was, from top to bottom, 5.6%, 3.7%, 3.2%, 2.9%, and 1.7%. When it came to just bisexual people, the answers ranged from 0.7% to 3.1%. For just gays and lesbians, it ranged from 1.0% to 2.5%.

With transgender people, Gates cited studies that ranged from 0.1% of the population being transgender to 2.0%, based on what exactly was asked. Trans people agree about the meaning of the word "transgender" even less than gay people agree about the meaning of "homosexual," and lots of people we'd think would be included in a survey of transgender people simply identify as their actual gender while others don't have the language to describe how they're feeling.

If one makes the assumption that the LGB population is stable and clearly-defined enough to be found (I'm not convinced but I'll assume for the sake of argument), then it can't possibly be 5.6% and 1.7% at the same time. It's one or the other or neither, but there's no reason to believe it's the average of the two.

If one doesn't make such an assumption, then they don't put out press releases saying that the LGBT population is nine million.

I also take issue with Gates saying that "they're all credible." One of the studies focused just on California - do we hold presidential elections in California and then extrapolate the rest of the states out under the assumption that they'd all vote Democratic at the same rate? It'd sure be cheaper to run elections that way. Another only asked people ages 18-44; would it be democratic to just let those people vote in elections under the assumption that older people would vote the same way?

I haven't read them so I don't know how credible each is (Gates surely has and could have made a judgment about which was the most credible), but at least two weren't applicable.


The Journal's companion piece also provides another reason producing an number of LGBT people is impossible that I had never heard about before:

Many people's confusion with terms surrounding sexual orientation contributes to the difficulty in producing solid estimates. Randall Sell, associate professor at Drexel University's school of public health and administrator of the informational website GayData.org, has conducted cognitive testing among older Americans who say they are bisexual because they aren't familiar with the term -- "bi means two, and that must mean man and woman," Sell said, describing their reaction.

In our country of 310,000,000 people and thousands of subcultures, there are lots of people who don't know what the word "bisexual" means. You can hand them a paper to fill out and then get a number of how many of them say they're bisexual, but that number isn't going to be worth anything. It'll still be a number, though, and journalists are always willing to print numbers!


In the days since I first posted, lots of news sources have been reprinting the nine million number without questions and with few caveats. It reminds me of this column from the Washington Post's ombudsman about how journalists are notoriously bad at math, which cited the many mathematical errors that appeared in the paper and concluded:

Many newsrooms provide remedial math training, but that's not been done at The Post. It should be considered. And given the increasing usage of numbers in reporting and graphics, The Post should pay heightened attention to math and statistical literacy when evaluating prospective hires.

We read the press corps we have, not the press corps we might want or wish to have at a later time.

But the issue isn't just math, it's credulousness. While we're willing to challenge people on moral or philosophical grounds, journalists take numbers from someone with a big degree without question. Math is seen as a mysterious subject and numbers must all be taken as real because to question one means to question them all, and the world falls apart when the latter happens. (Plus it's hard to make a deadline if you read the fine print.)

The ombudsman also wrote about statistics specifically:

"I think what's going on is that when journalists see a number, they take it at face value and don't question it," Maier said. "With numbers, I think journalists tend to abdicate that scrutiny."

Martell agreed, explaining that those intimidated by math tend to "panic" when forced to deal with numbers.

"You don't really have to know that much about statistics to read a statistical paper critically," she said, adding that reporters often cite numbers and statistics touted in news releases without questioning their accuracy.

Why do journalists "abdicate that scrutiny" when they see a number? It's probably because, as one person interviewed put it: "I think we have a culture where it's okay to say, 'I'm a journalist, which means I'm terrible at math.'" A database manager said that it's "charming" for Post journalists to say they "can't do math" and that math errors are tolerated while spelling errors are not.

That's pretty much what happened regarding this number. The Wall Street Journal was willing to look more closely because they have a column devoted to statistics, but that's about it. Even though it seems on the low end to a lot of people and doesn't take into account all the people who would benefit from LGBT legislation, expect it to become an un-cited truth in future media reports.

img Alex Blaze


Recent Entries Filed under Media:

Leave a comment

We want to know your opinion on this issue! While arguing about an opinion or idea is encouraged, personal attacks will not be tolerated. Please be respectful of others.

The editorial team will delete a comment that is off-topic, abusive, exceptionally incoherent, includes a slur or is soliciting and/or advertising. Repeated violations of the policy will result in revocation of your user account. Please keep in mind that this is our online home; ill-mannered house guests will be shown the door.


Jill Davidson | April 18, 2011 11:49 AM

I guess it depends on how much estimation error you are willing to tolerate. Of course, that is another tendency among journalists and much of the general public - they are more interested in central tendency, but not variability. Sometimes variability is the more interesting question. Variability comes from measurement error, as well as real differences among subjects and how they answer questions. The real number of LGBT people is going to be difficult to pin down because people identify differently. I have a close friend who is lesbian but says she doesn't identify with lesbians nor feel part of the "lesbian community". Transgender is even more difficult to pin down, since so many of us disappear into GLB or straight. It would be great to be able to have a number that says, "Look we're here, damnit, and there's a lot of us, so pay attention", but there's never going to be a lot of us.

I thought "bisexual" meant you liked to have sex with two partners at once.

Another part of the problem Jill pointed out is the number of people who don't use labels. My wife and I are an example: she's CIS-gender, and I'm transgender. We both identify as female. Does that make us lesbian? Or, since at various times in our lives, we have each had sex with men, are we bisexual?

A dear friend summed it up for us: "Stacy is Robyn-sexual, and Robyn is Stacey-sexual. You're both oriented to each other, and that's all that matters."

Honestly, I'm waiting (and working) for the day that the only label ANYONE needs is "human."

great point. It really shows how meanings of identities we take for granted as being straight-forward are so personal

I think that's more fodder for behavior-based measures.

Aside from how terms like MSM, WSW, or MSMW (yeah), have been integrated into socio-political identities, their primary goal is to address behavior independent of social identification.

You can ask a "straight" man if he's had sex with another man in the past year (and throw in number of times, etc. to get a sense of frequency). You can likewise ask a "gay" woman if she's had sex with men. Those will tell you more about who will benefit than the dubiousness of social labels.

Likewise, you can ask about cis or trans status by asking about birth sex and whether it is the same as your current gender identity. You may also want to just throw in label questions so that you can run an analysis on disparity between social identity and behavior.

And the whole "human is all that matters in terms of labels" thing is true...socially. But in terms of addressing needs, behavioral labels are important. If they weren't we'd be spending equal amounts of money creating HIV/AIDS awareness services for lesbian women, when the real need and risk is higher in people of color, men who have sex with men, and women who have sex with men who have sex with men (yes).

It may be nice to say that all that matters is I'm [insert partner's name]-sexual, but in terms of the person at the court house looks at you and your partner and deciding to stamp "No" on your marriage certificate, knowing why and how to address that in the future is important. That scenario shouldn't be simply categorized as "two people were denied a marriage license". If it was, we wouldn't notice a trend that same-sex couples are the ones being told that they cannot receive marriage licenses.

We also wouldn't know that queer teens are the ones committing the majority of suicides and need the most support, or that men who have sex with men are the ones being mostly turned away from blood donation.

The question itself "how many LGBT people are there in America?" is poorly thought out.

That's partly the fault of the LGBT community, who have deployed the argument
'It's not our fault, we were born that way." since it's beginnings with Magnus Hirshfeld. That argument leave dangling the question of homosexuality's fault itself, even as it deflects it from homosexuality's practitioners.

If we reject the idea that "we are born that way": that we all get one 'true' identity at birth-- simple and constant-- that is either accepted or rejected by our culture values, and instead recognize all human individual sexuality as having aspects of and potential for a variety of attractions and possibilities. Then the question becomes more akin of asking if one is a Republican or Democrat: an affiliation so fexible that we recognize it needs to be asked repeatedly as cultural and personal pressures and tastes shift with time.

Sometime between the 1960s and the 2000s, the idea of liberating all folks to pursue their sexualities as they saw fit was trampled by a new emphasis on identifying those 'born that way' and giving them 'rights' if and only if they conform to the values of their oppressors. They demand we embrace monogamy and lifelong marriage-- which is difficult to pursue unless one also embraces the 'stablizing' forces of home-ownership, middle-class careers, SUV, and lawn care. Only then does one deserve all the priviledges that straight folks have.

Already those queers among us who don't embrace these middle-class values are getting labeled as (sexaually and emotionally) "immature", "sluts", and "bad examples" that are undermining the movement.

Seconded. And well written too.

If you're ever interested in penning a guest post on the subject, send us an email, ditchhook.

I've recently gotten in a discussion with some other trans people about the possibility of putting a 'transgender' checkbox on the next census. My point is against that (or, at least seeing it has a very complex issue which shouldn't be taken lightly) is a huge percentage of people under the 'transgender umbrella'... education campaign or no, are not going to check it for a multitude of reasons. Which means a considerable undercount. And we all know, once statistics are out there in the media, they will be repeated as fact and will be used to minimize communities to make a point or to deny representation for years afterward.

Gates' stats were a joke and would have been seen as such by anyone who bothered to read his flawed methodology. Yet they've been referenced and reprinted ad nauseam because they are repeatable numbers and especially because they picture the LGBT community with smaller numbers than previously guesstimated. Do I think Gates purposely wanted to undercount the community... no. But in his own hubris to sound authoritative and to pretend his bizarre guesses at numbers had any correlation to reality, IMO, he's sold us out.

Robyn
Your right on your Bisexual definition!
As long as you're not Buysexual "Means you payed for it"
Me, I'm Trysexual (Example)"I might want to try that!"

Ginasf How about we just use the "T" On the Checkbox?

Those people from the Tea party checking "T" would inflate our groups presence drastically? It would be an easy sell to those Tea party people! they do not read critically, if they read at all!

It's perfectly normal and expected that different samples will yield different means, even if the populations sampled really are the same.

There are simple statistical tests to see how likely it is these numbers truly are different.

Don't disregard the results just because the numbers "seem" really different. That's just as ignorant of statistics as you claim journalists are.

If I flip two coins I might not get one head and one tail (50% each as we know is the true probability). If I flip two coins 1000 times I can combine the results to get close to the true probability. But each individual sample will have huge variation -- two tails, two heads, one of each.

This doesn't mean there was anything wrong the coins or the way they were flipped.

Hope that explains it better. Enough good samples will "regress toward the mean", assuming the same population is sampled (maybe California has more gay people? Don't know.) The more samples combined, the closer we'll get to a small range of where the real number is.

Your coin example:

1. repeats the same methodology multiple times
2. has a theoretically known correct answer
3. has a simple, uniform means of distinguishing different responses

The studies cited in the brief:

1. have different methodologies
2. have no theoretically known correct answer
3. have different ways of distinguishing between different responses

If California does have more gay people, then including it in the studies averaged means that the number produced will be too high. If there are fewer people out in the under 44 category, then that means that the number will be too low.

Or, to extend your coin analogy, you flip a quarter 500 times and find out how many heads (let's say 261). Then you flip a nickel 500 times and find out how many heads you get (let's say 245). Last, you flip a two-headed quarter from a toy store 500 times and record how many heads you get (let's say 500). The average is 335, which means that a coin, when flipped, would average 335 heads and 165 tails.

The thing is there, most people know that heads appears half the time and can easily spot the problem with your coins. Here, people really have no clue how many LGBT people there are so they're not so quick to spot the problem.

You're definitely right. I wasn't saying that the studies are definitely valid.

I was referring to the scare tactics used in some of the quotes to try to discredit the surveys because of non-problems.

For example:

"One problem they cited with Dr. Gates's findings is that they combine results from surveys with different sample sizes"

There's no problem with combining surveys of different sample sizes. That's a non-problem.

Then before the graphic: "The Wall Street Journal put together a small graphic to show just how much variance can be caused by methodology"

The percentages "look" very different, but no actual measures of variance are presented. They're encouraging people to qualitatively evaluate the numbers, when simple statistical tests can be used. The variance might have NOTHING to do with differing methodologies. I was trying to explain that even with exactly the same sample, you'd usually expect different results each time.

Another quote from above relies on this poor reasoning: "6.7% is a lot higher than 3.5%." It might not be! This is an ignorant thing to say.

Here's another example of terrible statistical ignorance:

"If one makes the assumption that the LGB population is stable and clearly-defined enough to be found...then it can't possibly be 5.6% and 1.7% at the same time. It's one or the other or neither, but there's no reason to believe it's the average of the two."

These samples are somewhat small (given the US population). There is EVERY reason to expect the means to be different for each sample. Unless the samples are enormous (millions of people?) then the probability of their being the same is ZERO.

If both samples are valid & measure the same population, then we should certainly believe an average of the two samples, taking into account variance and sample size, will get us closer to the true number.


Basically, this analysis acts to discredit true concerns about methodology by including a bunch of ignorant analysis of the numbers.


Pat Clements | April 19, 2011 7:36 AM

If we debate the number, we fall into the trap our opponents want. Bottom line: it doesn't MATTER how many lgbt people there are, rights are rights. After all, nobody challenges protections for any other group this way.

While there's no need to play oppression politics, one group does have its protections challenged constantly - even before being granted them.

Immigrants, particularly the undocumented, and especially the lowest on the ladder, like day workers (the people who make it possible for us to enjoy our cheap orange juice) are probably far more demonised today than anyone else. Consider Arizona, for starters. They're also the most badly treated, with conditions approaching servitude in many instances.

That's pretty doggone disturbing to find that journalists are lousy at math.

I guess I should negate my journalism studies altogether. It does, however, seem awfully peculiar that I'm actually able to see that 1+1=2 or that 4326 + 2615 = 6941 without using a calculator, because my brain is sufficient.

From reading all of the commentary, it seems like the statistical argument can go on for days and days, without much resolution. I've nodded my head and said "yeah, that is sensible" to a few disparate comments already, so I guess that everyone has a valid argument in this very fluid sort of analysis.

Journalists who wish to report on a subject owe their readers an informed opinion, a thorough analysis, and a commitment to fact-checking not only their own work, but that of those on whom they report.

It it also interesting to me that none of the percentages presented approach the commonly-touted one-in-ten figure.

That aside, I pretty much stopped reading the moment the word "oppressors" crossed my eyes. Ta!