The Pew Research Center for Religion and Public Life recently released its Religious Landscape Study. It is a formidable project that attempts to provide a high level snapshot of America’s religious status, and to record changes over time.
The survey results are valuable in mapping America’s shifting religious coastlines, but it provides very limited information about the denominational currents below the surface. It is clear that America is becoming less religious and less orthodox. What is less clear is exactly what is going on within the smaller church bodies.
For example, the findings for the Lutheran Church — Missouri Synod make for miserable reading. Two hair pulling examples :
- Generally: Even though Pew correctly categorizes the LCMS and ELCA differently — Evangelical Protestant vs Mainline Protestant respectively, the survey results are quite similar for the two denominations despite the massive doctrinal gulf.
- Specifically: LCMS members don’t appear to have a clue what happens at church. 47% of LCMS members attend “religious services” once a week, yet only 23% engage in prayer, Bible study or “religious education” with the same frequency. Only 36% say they read scripture once a week. What is going on during a divine service that 10% of the respondents say they don’t read scripture in that time?
However, anecdotal evidence from peers and pastors suggests that the Pew results for the LCMS are flawed to the point of being meaningless. There are several problems:
- Imprecise questions: the most obvious problem with the survey is the questions. Since the survey attempts the widest coverage, the questions have to be tailored accordingly. Asking Lutherans a yes or no question about whether Scripture should be taken literally is not going to produce a good result without some qualifiers. Similarly, a question about meditation is tone deaf in a Lutheran context.
- Margin of error: the sample size was 459 out of a population of 2,097,250 LCMS members as at 31 Dec 2014. At a confidence level of 95% the margin of error is 5%. Using the example of frequency of church attendance, this means that there is a 95% probability that between 42% to 52% of respondents attend church once a week. That is a 10% spread, which is very large.
- Distribution: basic information is given about the geographic distribution of LCMS survey respondents, but it does not match well with the distribution of actual LCMS membership. For example, 5% of the North Dakota respondents to Pew claimed LCMS membership, but only ~1% of the LCMS total membership is from North Dakota. Compounded with legacy “heirloom Lutheranism”, many small rural congregations in the state, but significant per capita charitable contributions, the reliability of the results degrades further.
- Response rate: somewhat related to distribution, no information was provided about the response rate. Given perceptions that Pew tends to have a liberal bias, it is quite likely that many respondents who were well qualified to give good answers declined to participate. It’s the statistical equivalent of Gresham’s Law – the best qualified respondents are eclipsed by the worst ones.
- Misidentified: Pastor Lincoln Winters notes that 16% of respondents seldom if ever attend church. So how does that make them LCMS members whose opinions on, say, creation and marriage have equal weight with a steadfast attendee?
- Statistical significance: the cumulative impact of these problems means that there is a reasonable probability of getting the same results from a random sample of respondents in the general population.
This means that the Pew results for the LCMS are interesting and possibly even entertaining, but they should not be relied on to reach any conclusions.
Brothers of John the Steadfast will be releasing its own LCMS survey soon in an effort to provide more reliable results from which to draw meaningful conclusions for reflection and action. Our goal is to secure results from at least 2,000 respondents (3% margin of error) with a good distribution across all districts.