No polling is perfect, we all know that, but the problems present in general polling are only exaggerated in polling LGBT populations. While these press releases and news articles about LGBT surveys hit my inbox at least weekly, I'm more likely to ignore them than to publish their results.
This new study from Clark, Martire & Bartolomeo, on behalf of PlanetOut Inc. and advertising agency Prime Access, about gay and lesbian spending habits is a good example of many of the common pitfalls in studying LGBT people. Here's the lede from the press release:
Bravo, Apple, Showtime, HBO, Absolut, and Levi's, are the gay-friendliest brands, while WalMart, Dunkin Donuts, Cracker Barrel, Exxon Mobil, and Samsung earn the lowest marks from gay and lesbian consumers, according to the 2008 Prime Access/PlanetOut Gay and Lesbian Consumer Study released today.
Without saying that the results are wrong (I have no idea), let's take a look at the methodology, after the jump.
Usually the problems with these studies start with the distance between the conclusions drawn from them and the study's methodology. Polling "gay and lesbian" people turns into "LGBT," polling Advocate subscribers or HRC donors turns into the "gay and lesbian community," and terms like "gay," "lesbian," "bisexual," and "transgender" are rarely defined for or by the respondents. All these terms have specific meanings, meanings that change from person to person (how bisexual is "bisexual"? does "transgender" include pre-op, post-op, and/or non-op trans people? are "gay" and "lesbian" based on public identity or personal identity, and are those identities based on attractions or actions?), and they each come with limitations that need to be kept in mind.
This particular study and press release are pretty good about sticking to "gay and lesbian." But the first question with any sort of LGBT polling should be: where the heck did they get their list of queers?
Unless a poll is done in such gargantuan proportions that they can ask everyone how they identify over the phone (still some problems there with closet cases) and then have a subset of LGBT data large enough to reach conclusions from, they're getting their gays from somewhere. Here's what the Clark, Martire & Bartolomeo survey says, right in the press release:
The respondents were sourced from both the Harris Interactive general population panel, one of the largest online panels available today, and from Harris Interactive's GLBT specialty panel.[...]
The Harris Interactive GLBT specialty panel is recruited from various sources, and is not specifically enlisted from gay and lesbian websites.
Harris Interactive is not synonymous with the more rigorous Harris Polls of public opinion. Harris Interactive is a marketing firm that studies online panelists, as the release specifically says.
What does this mean? First, the people responding to the poll are going to have to have internet access, which mean that they're overall wealthier and better educated than the general population. Second, it means that they're more self-selecting than telephone surveys, as people with more time on their hands and more interest in responding to these sorts of polls are going to be more likely to do them. Again, probably more educated and wealthier.
This particular poll at least found a control group of straight respondents. They don't always do that.
Also, there's a slippage there between the "GLBT specialty panel" and "gay and lesbian consumers," but I personally don't know where that's coming from.
Here's the methodology of the second part of the poll:
In addition, 3,156 PlanetOut subscriber and reader respondents, drawn from email promotable lists provided by PlanetOut, were also surveyed. The PlanetOut "universe" includes readers/subscribers of Out, The Advocate, gay.com, out.com, planetout.com and outtraveler.com.
Well, I'm sure you all already have opinions about the people who frequent those sites. They're not equivalent to "gay and lesbian" consumers as a whole.
Changing the way a question is asked can affect the results as well. While that problem exists in general population polling, consider this:
"Gays and lesbians have many similarities to straight people, with one pronounced difference: they're more powerful consumers," said Aiken. "In virtually every category -- from financial services to fragrance -- the study shows that gays and lesbians tend to be ahead of the curve when it comes to embracing new products and trends. They are early adopters that their peers look to for advice, opinions, and ideas. As a result of their influence, they impact many more purchases than just their own."
For example, Aiken said the study revealed that gays and lesbians are almost twice as likely (60% vs. 34%) as their straight counterparts to say people seek their advice.
I remember reading a poll (I can't find it at all right now, but it was pretty old, like 1970's old) that said that gay men were twice as likely as straight men to have responded that their mothers were overbearing. My first thought was that I'm twice as likely as my straight brother to say that our mother was overbearing, but I don't know if that proves Freud right at all.
While Aiken expands what should be "gay and lesbian survey respondents" to "gays and lesbians," again, at least he had a control group of straights to compare his results to.
Lots of these polls get published all the time, to find marketing info, LGBT opinions on politics, and a source for sexual orientation, the problems with finding a random sampling of queer people with a consistent definition of who's being studied and who isn't are generally large and they should be taken with a grain of salt for anyone other than their intended audience. Here, advertisers looking to buy space in PlanetOut media will have more of a use for the data than anyone else.
But I don't know if this poll is right that "69% percent of gays and lesbians are Democrats, while 7% are Republicans," ("registered Democrat" or just "agree with them more"?), but it's still fun to read about.