The Pitfalls of the Pew Religious Landscape Study Results for the LCMS

The Pew Research Center for Religion and Public Life recently released its Religious Landscape Study. It is a formidable project that attempts to provide a high level snapshot of America’s religious status, and to record changes over time.

The survey results are valuable in mapping America’s shifting religious coastlines, but it provides very limited information about the denominational currents below the surface. It is clear that America is becoming less religious and less orthodox. What is less clear is exactly what is going on within the smaller church bodies.

For example, the findings for the Lutheran Church — Missouri Synod make for miserable reading. Two hair pulling examples :

  1. Generally: Even though Pew correctly categorizes the LCMS and ELCA differently — Evangelical Protestant vs Mainline Protestant respectively, the survey results are quite similar for the two denominations despite the massive doctrinal gulf.
  2. Specifically: LCMS members don’t appear to have a clue what happens at church. 47% of LCMS members attend “religious services” once a week, yet only 23% engage in prayer, Bible study or “religious education” with the same frequency. Only 36% say they read scripture once a week. What is going on during a divine service that 10% of the respondents say they don’t read scripture in that time?

However, anecdotal evidence from peers and pastors suggests that the Pew results for the LCMS are flawed to the point of being meaningless.  There are several problems:

  1. Imprecise questions: the most obvious problem with the survey is the questions. Since the survey attempts the widest coverage, the questions have to be tailored accordingly. Asking Lutherans a yes or no question about whether Scripture should be taken literally is not going to produce a good result without some qualifiers. Similarly, a question about meditation is tone deaf in a Lutheran context.
  2. Margin of error: the sample size was 459 out of a population of 2,097,250 LCMS members as at 31 Dec 2014. At a confidence level of 95% the margin of error is 5%. Using the example of frequency of church attendance, this means that there is a 95% probability that between 42% to 52% of respondents attend church once a week. That is a 10% spread, which is very large.
  3. Distribution: basic information is given about the geographic distribution of LCMS survey respondents, but it does not match well with the distribution of actual LCMS membership. For example, 5% of the North Dakota respondents to Pew claimed LCMS membership, but only ~1% of the LCMS total membership is from North Dakota. Compounded with legacy “heirloom Lutheranism”, many small rural congregations in the state, but significant per capita charitable contributions, the reliability of the results degrades further.
  4. Response rate: somewhat related to distribution, no information was provided about the response rate. Given perceptions that Pew tends to have a liberal bias, it is quite likely that many respondents who were well qualified to give good answers declined to participate. It’s the statistical equivalent of Gresham’s Law – the best qualified respondents are eclipsed by the worst ones.
  5. Misidentified: Pastor Lincoln Winters notes that 16% of respondents seldom if ever attend church. So how does that make them LCMS members whose opinions on, say, creation and marriage have equal weight with a steadfast attendee?
  6. Statistical significance: the cumulative impact of these problems means that there is a reasonable probability of getting the same results from a random sample of respondents in the general population.

This means that the Pew results for the LCMS are interesting and possibly even entertaining, but they should not be relied on to reach any conclusions.

Brothers of John the Steadfast will be releasing its own LCMS survey soon in an effort to provide more reliable results from which to draw meaningful conclusions for reflection and action. Our goal is to secure results from at least 2,000 respondents (3% margin of error) with a good distribution across all districts.


Comments

The Pitfalls of the Pew Religious Landscape Study Results for the LCMS — 7 Comments

  1. I’m glad that BJS will try its hand at a survey. Do you know if the survey will include all LCMS’ers or will be limited to those who self-identify as “Confessional?”

  2. @RevJimO #1 Hi Pastor – no, we will not restrict it. We want to make it totally neutral to get the most representative sample possible. To that end, the plan is to release the raw data publicly once we have completed the survey. That will allow everyone to slice and dice it as they see fit, and I’m sure we will see some competing conclusions.

  3. Interesting piece. Will be very interested in your survey. Thanks for taking on this task.

  4. Pastor Wood,

    I am a recovering stats dork, and would be happy to help you. I also have done a fair amount of survey design, which is VERY important to getting good data.

    Eric

  5. They have several segments on Issues, etc. every year reviewing Pew and other surveys, and showing the problems with them.

    I think it is part of the liberal conspiracy to not use stats correctly. While it is interesting to know the attendance, for any meaningful data about “beliefs” they should filter ONLY by people who attend church at least 1/month. However, when I checked the more in depth data, it gets better for more frequent attenders… but not that much better. It’s shameful.

    * What the heck — more people “pray” daily than go to church weekly!? Of course, I know a lot of libs and lapsed Christians who go to church 2x a year, say they pray all the time.
    * “Frequency of feeling wonder about the universe among members of the Lutheran Church-Missouri Synod” — what does does this have to do with anything?
    * We do very badly versus other denoms on the absolute morals questions and Hell, worse on Bible reading!
    * I disagree with the caveat about the “literal” interpretation of Scripture. It’s not that hard, especially if you have any common sense. If you answer anything but literally, you are heterodox (maybe heretical), not too bright, or splitting hairs on the questions. Properly worded questions are very important, but the respondent has to have some sort of discernment and common sense, too.
    * 46% think abortion is just fine, and women want to kill babies more than the men (64% vs. 35%). Once again, this includes all church attendance frequencies, but even among those that go to church 1/week, 36% think killing babies should be legal is all/most cases?
    * 45% want sodomite marriage (39% of >=1/week attenders).
    * Evolution = 18% believe in it. How bizarre that this is so much LOWER than the support for abortion and homosexuality? Why would that be? 32% believe in “theistic” evolution, so maybe that is the answer. We are the WORST of all denoms in the data set.

Leave a Reply

Your email address will not be published. Required fields are marked *

Notify me of followup comments via e-mail. You can also subscribe without commenting.