Thursday, November 1, 2018

When Your Research Methodology Automatically Excludes Those Who Won't Stereotype Your Science Inevitably Supports Stereotpying

The piece I posted a week ago today about the Pew study in which they asked people to evaluate different religious identities for how warm or cold they felt about them, I ended by pointing out that if someone asked me to evaluate entire groups, sometimes not even actual groups but a collection of groups on the basis of whether I liked them or not, I'd have to refuse because all of those groups are comprised of individuals who might or might not be as I thought about the artificial "typical" member of that group.  There are individuals who match that artificial identity or those who deviate and even dissent from it but who are contained in that grouping by their own identification or the rules of the group.  And those are the Pew groupings that are, themselves, an artificial amalgamation of distinctly different bodies, "white Evangelicals" "Mainline Protestant" "Black Protestant".

You can't pretend that the members of different churches, each having their own range of viewpoints and beliefs within them, are of some uniform character that can morally or rationally be identified in terms of preference or dislike. It's even true of the identities in the study which, in fact, are uniform in membership of one church.

There are Roman Catholics who match the most insanely rigid old-line pseudo-medieval, Piux X era Integralists (a Catholic version of the worst of Fundamentalists) and there are those who the Integralists would love to kick out of the Church, excommunicate and destroy.  Something which even very conservative Catholics have been trying to recover from for the past century - literally.   I wouldn't be surprised if there are Integralists who are at war with other Integralists over the most obscure points of loyalty to what Pius X or other Vatican hacks of his term wrote.  There are even pre-Pius X style Integralists who are, if anything, worse.

One of the things I've been reading about recently was the reported feud between two of the greatest Catholic theologians of the 20th century who agreed with each other about much.  When Hans Kung wrote Infallible? An Inquiry,  his exhaustive and, from my perspective, entirely convincing refutation of the Vatican I dogma of Papal Infallibility,  perhaps the most eminent Catholic theologian of the 20th century, Karl Rahner publicly and fulsomely opposed his book.  They shared an interesting exchange between those two giants which I'm still in the process of teasing out.  As much as I admire Rahner in some things, in this disagreement Kung had the goods to make his case.  There is enormous diversity even within the most informed of intellectuals in the Catholic Church, even within those licensed to teach Catholic theology at Catholic institutions.  Even among those who are later canonized as saints.

But the reason I'm giving these examples is that it occurred to me this morning that  I would refuse to participate in the Pew survey because I refuse to characterize people I didn't know and people within a group who I knew didn't match the "typical" character of that group, there must be other people like me who would refuse to do what the surveyors asked.   I would refuse to stereotype for what is purported to be science.

I don't have any idea how big that group of non-cooperators on that basis would be though if it occurred to me, I'm sure it's occurred to other people, too.  But I can be pretty sure those who would participate in such a survey were either people who had no qualms about stereotyping to the degree they'd give a ranking for a group because they didn't care about diversity within that group or those who would participate never thought about it seriously enough so they wouldn't even realize that was what they were doing.

Whatever else you can say about that Pew study or so many like it presented as "science" it would exclude people whose thinking on it was much closer to real, lived reality than the people who would participate in such an exercise in stereotyping and evaluation based on stereotyping.   That seems to me to be an insurmountable obstacle that people wanting to do such surveys cannot overcome by the very nature of what they want to do.  If they tried to cover every possible or even a good range of possible variations represented among groups, their data would probably become a generalized mush of points with no reasonable, general, claims made about them.  I doubt that the confidence levels given for any of this stuff is honest, not because what their respondents say isn't what they said but because what they said is meaningless due to it being divorced from complex reality.

1 comment:

  1. I'm less and less interested in polls, involving was Walt Kelly said "the buckshot use of the curved question." Political polling always declares itself accurate after the fact, obscuring the number of polls that didn't "predict" the electoral outcome, or missed it by a few points or a mile, and then how much those polls might have influenced turnout (or encouraged it) and so became a part of what it purportedly, taking all into account, measures.

    I like to follow the aggregate of polls at 538.com on Trump's approval (consistently underwater by 10 points) but is it even vaguely accurate? Who knows? I like to think so, but who can say? Polls have notoriously missed electoral outcome by 10 points or more (depending on which ones you count after the election, or which ones you aggregate and how), and yet close enough is good enough, and let's do it again in two year!

    Polls ask me to be the person they are looking for, not to tell them who I am. Not nearly so interested in that as the pollsters think I should be. (And that's the problem with the polls right now. Texas early voting turnout already exceeds the entirety of early voting turnout for 2014. Nationwide it may reach the level of the turnout in 1966, or even 1914. Models for "likely voters" are useless in such events, and yet, here we are, learning daily who is "leading in the polls", and then when it all falls apart, pollsters will just shake their heads, collect their money, and go back to those phones. Nice work, if you can get it.)

    ReplyDelete