Sermon responses and preferences in Pentecostal and ... · the need for evaluations of sermon...

Pre-print version, Sermon responses

Pre-print, post refereed version

Sermon responses and preferences

in Pentecostal and Mainline churches.

by

William Vaughan Jenkins and Heather Kavan

Formatted version in Journal of Empirical Theology, 22(2), 142-161

Pre-print version, Sermon responses 1

Sermon responses and preferences

in Pentecostal and Mainline churches.

William Vaughan Jenkins and Heather Kavan

ABSTRACT

While research over the last 30 years has shown that sermons have a minimal

influence on listeners, denominational differences in sermon effectiveness have

been unexplored. We surveyed Anglican and Elim members (representing

Mainline and Pentecostal Christians respectively) to compare their responses to

sermons, and the sermon features and themes they prefer. The results revealed

higher sermon effectiveness responses from Elim participants. The data also

showed that Anglicans desired significant intellectual content and discussion of

social issues compared with Elim members who gave higher ratings on almost

all other features and themes. However, participants from both churches

responded to sermons in a predominantly emotional (rather than cognitive or

behavioural) way, and most wanted to hear sermons on grace and forgiveness.

We conclude by noting the need for evaluations of sermon effectiveness that

reflect preachers’ and listeners’ motivations.

Key words

sermon effectiveness, sermon themes, preaching, Pentecostal, Anglican


1. INTRODUCTION

Sermons are usually goal orientated: they are intended to induce and

reinforce the experience of being a Christian. Yet researchers continue to

question the effectiveness of the sermon as a communication tool, and several

studies have shown that sermons have a minimal influence on listeners (e.g.,

Greene, 1998; Price, Terry & Johnston, 1980).

To determine the factors that may impact on sermon effectiveness, some

researchers have concentrated on characteristics of the sermon and the preacher

(Maybury & Chickering, 2001; Joseph & Thompson, 2004), but most have

emphasised the characteristics of the receiver (e.g., Howden, 1989; Newman &

Wright, 1980; Pargament & Silverman, 1982). To date, no study has examined

denomination as a factor, even though there are strong variations in theology

and preaching styles across denominations. Given that tailoring messages to

listeners is essential to good communication, it is important to know if people

from different church affiliations, (especially fundamentalist and liberal ones)

vary in their responses to sermons, and the types of sermons they prefer.

This study examines the extent to which affiliation to a Mainline or

Pentecostal denomination affects reports of sermon effectiveness, and preferred


features and themes.1 By Mainline we refer to large historic denominations,

such as Anglican, Methodist, Presbyterian and Baptist, which were brought to

the United States by the early European settlers. By Pentecostal, we mean

churches that arose from the revivals of glossolalia at the turn of the 20th

century in the Unites States and Wales, which encourage converts to seek the

baptism of the Spirit and speak in tongues.

Specifically, we compare the responses to sermons of two Anglican and

two Elim churches. The Anglican church is the third largest Christian

community worldwide and represents a middle ground between Roman

Catholicism and Protestantism. Elim is a Pentecostal denomination which has

almost 9,000 churches worldwide. We look at how the two churches differ in:

(1) sermon effectiveness, measured by reported responses to sermons, (2)

members’ desire for features commonly used in sermons, e.g. gestures, stories,

anecdotes, intellectual content, and (3) members’ desire for sermon themes, e.g.

the grace and forgiveness of God, social justice and peace issues, the believer in

the work place.

The paper takes the following format: First we discuss the research on

sermon impact. Then we present the reasons why we expect higher sermon

1 The authors acknowledge Dr Nicola Murray and the referees for their helpful suggestions to

the manuscript.


ratings from Pentecostals than Mainline members, and the types of themes and

features that members of each religious affiliation may favour. Next, we outline

the New Zealand context of our research, followed by our method in which we

present our sermon inventories. In our results section we examine both the

mean rank differences and median scores of sermon responses and preferred

features and themes in the respective denominations. We close by discussing

the need for evaluations of sermon effectiveness that reflect preachers’ and

listeners’ motivations.

2. SERMON IMPACT

Over the last 30 years researchers have not found any evidence to

suggest that sermons are significantly beneficial to the majority of listeners.

Greene found in a survey of 400 English parishioners who were actively

involved in their churches that 47% viewed the preaching as irrelevant and

lacking in life-giving insight (1998). Newman and Wright (1980) developed a

sermon effectiveness questionnaire, which they used to survey Catholic

participants’ responses to sermons in areas such as Bible reading, prayer and

social service. They found that sermons had only a poor to moderate effect,

with 75% of their sample “never, seldom or sometimes” responding to sermons.

Similarly, Price, Terry and Johnston (1980) found that sermons are relatively

ineffective at changing knowledge and behaviour, even over short periods of


time. Also, studies testing sermon recall show that listeners have poor to

modest recollections of the messages (Joseph & Thompson, 2004; Pargament &

De Rosa, 1985).

Nevertheless, several church members rate sermons positively, and these

tend to be people with strong religious beliefs and high religiosity who are

more familiar with religious messages—although listeners often distort

messages to make them more acceptable (Eckstein, 2005; Pargament & De Rosa,

1985; Ragsdale & Durham, 1986). Those who are single, less highly educated

and more involved in parish activities also give positive ratings (Newman &

Wright, 1980). Sex and age are additional factors, with older people and

females evaluating sermons more positively (Howden, 1989). The sex of the

speaker makes a difference too, with males rating high status females lowly

(Maybury & Chickering, 2001).

Features of the sermon itself also contribute to its effect. The content

must be personally relevant to the listeners’ life experiences if they are to

respond positively (Eckstein, 2005; Pargament & Silverman, 1982). Related to

this, listeners are more likely to remember and be persuaded by sermons that

are vivid (Joseph & Thompson, 2004). Also, the perceived preaching style of the

clergy—speaking at a natural pace, good tone, eye contact, a clear central topic


and good preparation—are highly predictive of sermon impact (Pargament &

Silverman, 1982).

Both Newman and Wright (1980) and Pargament and Silverman (1982)

recommend researching a wider range of variables than the usual demographic

ones when measuring sermon impact and effectiveness. In particular, Newman

and Wright suggest that their sermon effectiveness questionnaire should be

replicated among Protestants, for whom the Bible and preaching are typically

given greater emphasis and higher levels of sermon effectiveness might be

expected. They also predict that this effectiveness would be greatest in less

liturgical denominations. These comments point to a need for research that uses

denomination as an independent variable to reflect the churches’ differences in

theology, preaching and worship styles.

3. PENTECOSTAL AND MAINLINE DIFFERENCES

In this section we provide the background to our hypotheses. We begin

by outlining the key differences between Mainline and Pentecostal churches.

Then we explain why we expect higher sermon ratings from Pentecostals than

Mainline members, and why we anticipate that participants from each religious

affiliation will favour certain types of sermon themes and features.


The central difference between Mainline churches and other churches

(both Pentecostal and non-Pentecostal) is that they are less likely to have a

conversion message as their focal point. They tend to be at the ideological left

of Christianity, more inclined toward social service than proselytising. As they

are large, hierarchical institutions, they are more liturgical ― prayers, scripture

readings and the order of service for a given day may be prescribed by service

books. Sermons similarly may be drawn from written materials, and tend to

reinforce the church’s role as a community of believers who uphold ideals of

love, hope and peace.

In contrast, Pentecostals view the ‘born again’ experience as the heart of

Christianity, and Pentecostal leaders are more concerned with winning people

to Christ than with welfare provision. Few would use the word ‘liturgy’. Their

tradition is primarily oral, and church services are relatively non-hierarchical,

characterised by spontaneous utterances from participants, including

glossolalia, singing in the Spirit, prophecy, praise, prayer, and testimonies.

Similarly, sermons are usually extemporaneous, and the messages aim to

reinforce the participants’ experiences of the church as a spiritually empowered

community.


3.1. Sermon effectiveness

We expected higher sermon ratings from Pentecostals than Mainline

members for several reasons. First, non-Mainline Christians tend to have

higher religiosity, as measured by frequency of prayer, church attendance, and

self ratings of the importance of God in their lives (Webster & Perry, 1989), and

highly religious people tend to give more positive sermon ratings.

Second, Pentecostal preaching is optimistic: the preacher approaches the

task with “awe and wonder for what God will do” and the congregation has an

expectation that preaching will be accompanied by “signs and wonders”

(Gordy, 2001, p. 94). As people who listen to optimistic sermons are likely to be

more optimistic themselves (Sethi & Seligman, 1993), we expected that

Pentecostals would be inclined to rate sermons positively.

Third, Pentecostal preaching is directed towards the perceived needs of

the audience. Speakers are expected to be open to the Spirit’s leading and are

therefore more likely to change direction when they feel that the Spirit is

steering them to topics that listeners need. Pentecostal preachers often walk

around engaging the audience in a dialogue about their relationship with God

(Taiwo, 2005), albeit a dialogue in which the audience’s responses may be

limited to affirmations like ‘Amen!’ and ‘Praise the Lord!’ As people tend to


respond to sermons that they believe fill their needs (Eckstein, 2005) we

anticipated relatively high sermon ratings from Pentecostals.

3.2. Sermon features

We also expected Pentecostals to prefer strong emotional appeals, and

Mainline members to prefer more intellectual content, because the Pentecostal

preaching style is more experiential and active than Mainline styles. Most

Pentecostal preaching contains little exegesis and incorporates vivid language

and an imploring altar call. As scholar of church growth, C. Peter Wagner

wrote, “The result of hearing Pentecostal preaching is not that you learn more,

but rather that you feel better" (Wagner, 1973, p. 118).

It also seemed likely that Pentecostals would be less interested in

intellectual content because religiosity is negatively related to complexity of

thought for existential issues (Eckstein, 2005; Hunsenberger, Pratt & Pancer,

1994). Also, Pentecostals rely heavily on the New Testament Acts of the

Apostles, which is descriptive and not overtly reflective. While an intellectual

Pentecostalism has emerged (with Pentecostal scholars like Gordon Fee and

Mark McLean), by far the majority of Pentecostals would be more interested in

experiencing the Holy Spirit than intellectual analysis.


We further anticipated that Pentecostals would favour sermons with

significant Biblical content more than Mainline members would favour them

because of the high status the ‘Word of God’ is given in Pentecostal churches.

People tend to prefer the types of sermons that they are used to (Eckstein, 2005),

and Pentecostal preachers often use Bible verses as proof texts and re-enact

stories from scripture with various additions to make them more entertaining

and accentuate the message.

3.3. Sermon themes

We expected Pentecostals to prefer spiritual themes in sermons and to be

less interested in themes of worldly engagement, such as social justice and the

transformation of society, unless the themes were framed in the context of

Biblical authority, such as “Biblical attitudes towards the poor”. It would seem

unlikely that Pentecostals would be highly interested in social justice issues as

most believe that the best way to solve problems is by personal salvation

through Jesus. Rokeach (1970) found that people who strongly value salvation

usually do not want the church to be involved in social justice issues. Although

Rokeach’s finding is dated, and Pentecostals today are less ardently millennial

and therefore less inclined to simply wait for Jesus to return and set right all

injustices, Pentecostal ideology still gives a stronger priority to church planting

than to political and economic efforts to alleviate oppression.


Regarding the theme of “hell and damnation”, we could not predict

whether Pentecostals would like this incorporated in sermons because there is

no consensus in the literature. Ragsdale and Durham (1986) found that

fundamentalists rate such messages more positively than non-fundamentalists;

however Eckstein (2005) found that although participants believe the messages

are important for others to hear, they themselves are bored by them.

4. THE NEW ZEALAND CONTEXT

This study was carried out in New Zealand, a country that is officially

secular, but is mainly comprised of nominal Christians, i.e. people who self-

identify as belonging to a Christian denomination but do not attend church. In

the early 1900s, 41% of the population were Anglicans, and—even though

membership drops by tens of thousands each census—the Anglican church is

still the largest denomination in New Zealand, comprising 13.3% of the

population (Statistics New Zealand, 2006).

In contrast, only 2% of the New Zealand population identify a

Pentecostal church as their religious affiliation. However, Pentecostals are one

of the fastest growing religious groups in New Zealand, and it is likely that

Pentecostals are among the additional 4.5% of New Zealanders who describe


their religion simply as ‘Christian’ (Statistics New Zealand, 2006). Of the

Pentecostal denominations in New Zealand, Elim is one of the fastest growing.

Elim has Welsh rather than American roots (like the Apostolic church), and

these two churches in New Zealand tend to have less exuberant services, but

they are theologically similar to Pentecostal churches of American lineage.

New Zealand Anglican churches have more theological diversity than

most Mainline churches, ranging from High Anglican (similar to Catholic) to

evangelical, Charismatic, or liberal (Brookes, 1998), with most members in the

middle ground. In large Anglican churches there can be four services on a

Sunday, each with a different style. Anglicans who experience Pentecostal

phenomena such as glossolalia, but prefer Anglican services, may not need to

move to Pentecostal churches as there is tolerance within their own

denomination for Christians with diverse theologies and experiences. In fact,

most New Zealand Mainline members who experience Pentecostal phenomena

stay within Mainline Christianity, although they may change churches or

supplement services by joining Charismatic groups, such as the Full Gospel

Business Men’s Fellowship or Women’s Aglow (Kavan, 1995).


5. METHOD

5.1. Hypotheses

Our hypotheses are:

1. Pentecostals will rate sermons within their denomination more highly than

Mainline members rate sermons within their denomination.

2 a. Pentecostals will give a higher rating than Mainline members for preferring

strong emotional appeals in sermons.

b. Mainline members will give a higher rating than Pentecostals for

preferring significant intellectual content in sermons.

c. Pentecostals will give a higher rating than Mainline members for preferring

significant Biblical content in sermons.

3a. Pentecostals will show a higher preference than Mainline members for

spiritual themes in sermons.

b. Mainline members will show a higher preference than Pentecostals for

themes of social justice and peace issues, and the transformation

of society.

5.2. Sample

The sample comprised 148 church members, 18 years of age and over,

who were present at the two Anglican and two Elim churches on the days that

the research was conducted. The churches are in adjacent cities in the Hawkes


Bay region of New Zealand. The two Anglican churches had a membership of

100 members each, while the two Elim churches comprised 60 and 160 members

respectively.

There were 71 Anglican and 77 Elim participants. 42.5% were male and

57.5% were female. 2.7% were aged 18 to 24, 12.8% were 25 to 34, 22.3% were

35 to 49, 23.7% were 50 to 64, and 38.5% were 65 and over. 46% of the sample

had a university qualification and 64.2% were married. 66.2% were not

working—they were either home makers or retired—and this is reflected in

their relatively high church attendance: 59.5% reported that they attend church

more than three times a fortnight.

5.3. Materials

A questionnaire was developed, which contained sections on: (1) sermon

effectiveness, (2) preferred features, (3) preferred themes, and (4)

demographics. For the first three sections, a seven point Likert scale was

employed for all three measures, ranging from 1 = never to 7 = always. In

between these polarities were: 2 = very rarely, 3 = occasionally, 4 = about half of

the time, 5 = quite often, and 6 = frequently.


Sermon effectiveness was measured by Newman and Wright’s (1980)

scale, consisting of 10 questions measuring how frequently people respond to

sermons. These included questions about how people respond emotionally, for

example, “Do sermons provide you with a sense of God’s love?” and “Has a

sermon made you want to praise and thank God?” There were also questions

about cognitive responses, such as: “Do sermons answer your important

questions about God and religion?” and “Do you think that your knowledge of

Jesus Christ has improved due to sermons?” Finally, there were questions

about behavioural responses, for example, “Do the guidelines in sermons

actually change the way you live?” and “Have you ever read a part of the Bible

because of a sermon you heard?”

The preferred sermon features were assessed by an inventory of 12

features generated from Pargament and Silverman’s (1982) research, plus

interviews with church leaders and members. Participants were asked how

often they would like to see these features in sermons. Three of the features

explicitly referred to content: “a lot of Biblical content”, “significant intellectual

content”, and “the gospel followed by an altar call”. The remainder referred to

presentation: “good eye contact between the speaker and the congregation”, “a

clear central topic with easy to follow points”, “strong emotional appeals”,

frequent use of gestures”, “drama and other visual presentations”,


“opportunity for questions and other two way forms of feedback”, and “a

sermon containing humour” .

The preferred sermon themes were assessed by an inventory of 12

themes, developed from online searches of sermon material and interviews

with church leaders and members. Six of these were spiritual and Biblical

themes, (“grace and forgiveness of God”, “life and times of Jesus Christ”, “work

of the Holy Spirit”, “judgement of God”, “heaven and the afterlife”, “hell and

damnation”). Two were worldly (“social justice and peace issues”,

“transformation of society”) and three were worldly but Biblically framed

(“Biblical attitudes to sex”, “Biblical attitudes to money”, and “Biblical attitudes

to the poor”). “The believer in the workplace” was added to take into account

Greene’s (1998) finding that church members wanted sermons to be more

relevant to everyday life.

The questionnaire concluded with a demographic section containing

questions on age, gender, occupation, educational qualifications, marital status,

denomination, and number of church meetings per fortnight. Participants were

also asked to identify the theological outlook that best described them, for

example: Liberal (willing to interpret the Bible as a cultural document rather

than a statement of facts), Evangelical (primarily focused on spreading the

gospel and personal salvation through Jesus), Fundamentalist (primarily


focused on the Bible as a historically and scientifically accurate document),

Pentecostal/Charismatic (incorporating the experience of the baptism of the

Spirit and spiritual gifts) or Orthodox (following traditional church beliefs and

practices).

5.4. Procedure

The lead researcher attended morning services at each of the four

churches, and during the meetings the Minister gave him the opportunity to

explain the research purpose and invite members to fill out a questionnaire at

the end of the service. The researcher explained that the questionnaire would

take about ten minutes to complete. He emphasised that he was not asking

participants to rate the sermon they were going to hear, but to rate sermons in

general within their denomination. At the end of the services he distributed

the questionnaires and collected the data.

However, at Elim church A a ‘move of the Spirit’ at closing time resulted

in an extended altar call, and many participants were too immersed in spiritual

experiences to fill out a questionnaire. Therefore most of the members of this

church received and returned the questionnaires by post. The written

instructions reinforced the request that participants focus on sermons in general

within their denomination.


Almost half of the participants of all four churches responded. Therefore

the response rate was in keeping with typical survey response rates: 54.3% for

Elim members and 44.1% for Anglicans.

6. RESULTS

6.1. Data analysis

SPSS version 16 was used to analyse the data. Demographic differences

between the Anglican and Elim samples were determined using Chi-square

tests of independence, with Yates continuity correction used for the gender

difference. The differences between the two denominations’ responses to

sermons and sermon preferences were analysed by Mann-Whitney U tests. For

all data p = .05 was used as the level of statistical significance. Effect sizes were

also calculated to measure the magnitude of the observed effects of the

variables.

6.2. Demographic differences

Looking first at the comparison between Elim and Anglican participants,

the Chi-square tests of independence revealed no significant differences for age,

occupation, educational qualifications and marital status.


There was, however, a significant denominational difference in

theological outlook: χ2 (4, n = 139) = 122.6, p = .001, Cramer’s V = .94. Not

unexpectedly, all Elim participants described their theological outlook as

Pentecostal/Charismatic, Fundamentalist or Evangelical, compared with

Anglicans who tended to describe themselves as either Liberal or Orthodox. So

strong was the magnitude of the relationship between theological outlook and

denomination shown by the Cramer V score, that the two variables almost

seemed to be measuring the same concept. However, the similarity was less

marked in Anglican church B where 20% of participants identified themselves

as Evangelical or Pentecostal/Charismatic, compared with Anglican church A,

which had no Evangelical, Pentecostal/Charismatic or Fundamentalist

participants.

There was also a strong significant difference in how frequently Anglican

and Elim participants attended church, χ2 (3, n = 146) = 28.82, p = .001, Cramer’s

V = .44. Elim members were more likely than Anglicans to report attending

church frequently, and 21% of Elim participants claimed to attend church more

than six times a fortnight.

Additionally, there was a significant (but moderate) gender difference in

the denominational samples: χ2 (1, n = 148) = 8.43, p = .004, phi = -.25. Elim


congregations had a higher proportion of males than the Anglican

congregations (54.5% and 29.6% respectively). These percentages are

disproportionate to the census statistics of Pentecostal and Anglican adherents

in New Zealand, in which there are 45.4% males and 44% males respectively

(Statistics New Zealand, 2006). However, they are in keeping with Kavan’s

(1995) research based primarily on fieldwork and questionnaires distributed in

New Zealand Pentecostal and Mainline churches.

6.3. Sermon effectiveness

On all variables, Elim members rated sermons as more effective than

Anglican members rated them, and on all but two items (“Do sermons provide

you with a sense of God’s love?” and “In reality, do sermons affect the way you

see God?”) these differences were statistically significant. Table 1 shows the

mean rank sermon effectiveness scores for the denominations and the statistical

significance of the differences (measured by Mann-Whitney U tests). Of the

statistically significant items, the effect size was small to moderate, ranging

from -.194 to -.432.

(Table 1 here – see page 34)


While on average Elim members chose the high end of the sermon

response scale, the spread of Anglican and Elim responses was similar.

Participants from both denominations gave high ratings when answering

questions about the emotional effects of sermons. Even though, by self report,

participants did not greatly change their attitudes or behaviours after hearing

sermons, 83% reported that sermons “provided [them] with a sense of God’s

love” and made them “want to praise and thank God” half of the time or more.

With the exception of the item “Do you think that your knowledge of

Jesus Christ has improved due to sermons?” there were relatively moderate

ratings for items assessing cognitive responses to sermons, such as “Do sermons

answer your important questions about God and religion?” The lowest ratings

were for behavioural responses, such as “Have you ever read part of the Bible

because of a sermon you heard?” and “Do the guidelines on sermons actually

change the way you live?” and these were especially lowly rated by Anglicans.

Table 2 compares the sermon effectiveness scores with the Catholic data

obtained by Newman and Wright (1980). Unfortunately, the comparison is not

exact because Newman and Wright’s Likert scale is unknown. However,

Newman and Wright’s data summary shows the broad categories they used,

and we have grouped our data into similar categories. The results show that

the Elim sermon ratings were the highest of the three denominations. The


Anglican ratings were second, (with Anglican church B’s responses close to

Elim church A’s ratings.) Newman and Wright’s Catholic sample gave the

lowest sermon ratings.


6.4. Desired features

Table 3 shows the denominations’ median scores and mean rank scores

for each desired sermon feature. As with the sermon effectiveness ratings, Elim

members chose the high end of the sermon preference scale and were more

likely than Anglicans to desire the listed sermon features “quite often” and

“frequently”. The only exception was the item “significant intellectual

content”, which Anglican participants were significantly more likely than Elim

participants to desire. The strongest difference between Anglican and Elim

participants was Elim members’ higher preference for sermons that include the

presentation of the gospel followed by an altar call, which was significant at the

p < .01 level and had a large effect size. Of the ten remaining Elim preferences,

seven were statistically significant, and the effect sizes were small to moderate,

ranging from r = -.163 to r = -.403.



The two features that participants most desired in sermons were the

same for both denominations. Both Elim and Anglican members most wanted a

presentation of a clear central topic with easy to follow points, and good eye

contact between the speaker and the congregation (the latter feature was

especially important to Elim participants). There was also common agreement

that participants were keen for sermons to include a lot of Biblical content,

storytelling, anecdotes and humour.

6.5. Desired themes

Table 4 shows Anglican and Elim members’ median and mean rank

ratings for each sermon theme. The Mann-Whitney U tests demonstrated that

Elim members were significantly more likely than Anglicans to desire sermons

on spiritual themes: God’s grace and forgiveness, Holy Spirit, judgement,

heaven and the afterlife, and hell and damnation. Elim members also rated

two of the three items that were Biblically framed significantly higher: Biblical

attitudes to money and Biblical attitudes towards sexuality. However, there

was no significant difference in preferences for sermons on Biblical attitudes to

the poor.


In contrast, Anglicans showed a stronger preference than Elim

participants for sermons on social justice and peace issues and questions

concerning the transformation of society, although the latter was not

statistically significant.


The most and least desired sermon themes were the same across

denominations: both Elim and Anglican members most wanted to hear sermons

on the grace and forgiveness of God and the work of the Holy Spirit in the

believer’s life. Despite the large effect size of the item “hell and damnation”,

participants from both denominations least wanted to hear sermons on this

theme—with Anglicans more inclined to choose the “never” or “very rarely”

option, and Elim members tending to choose the “occasionally” option.

The item “questions concerning the believer in the workplace” did not

shed light on the desire for sermon relevance as hoped because an unexpectedly

high proportion of the sample (66.2%) were not in the work force. Both

Anglican and Elim members rated this theme moderately (with median ratings

of “about half of the time” and “quite often” respectively).


6.6. Variations within denominations

Analysis of the individual church responses reveals that the responses

from the Elim churches were similar. This suggests that the results were not

significantly affected by the unanticipated ‘move of the Spirit’ that delayed the

questionnaire distribution at Elim Church A. However, the responses from the

Anglican churches showed more variation. Anglican church B (where 20% of

the participants described themselves as Evangelical or

Pentecostal/Charismatic) had five significant differences in sermon responses

and preferences to Elim church A and eight to Elim church B. Conversely,

Anglican church A (which had a strong liberal membership) had 23 significant

differences to Elim church A and 28 significant differences to Elim church B.

Nevertheless, members of both Anglican churches gave higher ratings

than Elim members for preferring sermons with significant intellectual content

and social justice and peace themes, while the Elim ratings were higher for

almost all other variables.

7. DISCUSSION

The findings provided support for our prediction that Mainline and


Pentecostal Christians would differ in their evaluations of sermon effectiveness

and in the types of features and themes they prefer in sermons.

Our first hypothesis, that Pentecostals would rate sermons within their

denomination more highly than Mainline members rate sermons within their

denomination, was supported for all variables. The high Pentecostal sermon

ratings are reflected in frequent Pentecostal church attendance and increasing

numbers nationwide, highlighting the usefulness of research on sermon

effectiveness to those concerned with church growth.

Hypotheses 2A , 2B and 2C were also supported. Pentecostals gave a

higher rating than Mainline members for preferring strong emotional appeals in

sermons and significant Biblical content, and Mainline members gave a higher

rating than Pentecostals for preferring significant intellectual content in

sermons. That this trend occurred despite (a) no significant differences in the

educational qualifications of the members of each denomination, and (b) 8.5%

of Mainline members identifying their theology as Evangelical or

Pentecostal/Charismatic, lends support to Eckstein’s (2005) finding that people

prefer the types of sermons they are used to.

Hypotheses 3A and 3B were also supported. Pentecostals showed a

higher preference than Mainline members for spiritual themes in sermons, and


Mainline members showed a higher preference than Pentecostals for preferring

themes of social justice and peace issues. Although, as predicted, Anglicans

rated the item “transformation of society” higher than Elim members rated it,

the difference was not statistically significant. Most likely this reflects the

enthusiasm of the Pentecostal sample. With the exception of the two themes

(“social justice and peace issues” and “the transformation of society”), Elim

participants wanted sermons on every theme—spiritual and this-worldly—

either equally or more frequently than Anglican members wanted them. An

additional explanation is that the phrase “transformation of society” is open to

interpretation and could be construed, for example, in apocalyptic terms.

Despite the differences in denominational mean rank scores, when the

data was interpreted by median scores there were considerable parallels.

Participants from all the churches showed a relatively high emotional (rather

than cognitive or behavioural) response to sermons. They also wanted to hear

most often sermons on “the grace and forgiveness of God” and sermons that

were comprehensible, direct, and made them feel good (features we would

expect to be preferred in most types of communications, religious or secular.)

The high preference for sermons on grace and forgiveness may explain why

participants from both denominations least wanted to hear sermons on hell and

damnation. Most likely members who believe in hell did not favour these

messages because hell and damnation are the opposite of the grace and


forgiveness sermon themes they desire. The lack of personal relevance may

also be a factor, given that none of the participants were likely to believe they

were going to hell.

An unexpected finding was that 83% of the whole sample reported that

sermons “filled [them] with a sense of God’s love” and made them “want to

praise and thank God” half of the time or more. This positive response presents

a challenge to 30 years of scholarship in which researchers have not found any

evidence to suggest that sermons are significantly beneficial to the majority of

listeners. This indicates that research on sermon effectiveness, with its focus on

recall and behavioural change, does not fully capture the value of sermons for

participants. Religious speech, perhaps more than other communications,

depends strongly on what scholars call the communication climate (i.e. the

emotional atmosphere that is experienced instinctively), which participants

might call the ‘movement of the Spirit’. This variable does not lend itself easily

to academic research and is often beyond the scope of sermon studies.

8. CONCLUSION

In this study we attempted to move beyond sermon effectiveness studies

based on characteristics of the speaker and listener, and explore denomination

as a factor reflecting different theologies, preaching and worship styles. The


first contribution of this paper is that we have offered preliminary evidence that

sermons are not as ineffective as previously supposed. The second contribution

is that we have provided material on preferred sermon features and themes that

can be used by preachers who wish to be sensitive to the theological

underpinnings of the faith traditions. Our third contribution is that we have

extended Newman and Wright’s study of Catholics to Anglicans and

Pentecostals, and provided an empirical base for their theory that reported

sermon effectiveness will be higher in the less liturgical churches.

Several limitations to this study preclude in depth interpretation. The

data was based on a low non-representative sample of churches, and small sizes

in the demographic sub-samples made it difficult to fully assess the impact of

the different theological outlooks. There was also a variation in the

denominational response rates within the study and limited information on

how participants interpreted sermon themes. Additionally, there is no way of

knowing whether participants really did rate sermons in general within their

denomination and not their specific experience during the sermon they had just

heard. Replication and refinement of this kind of study across churches is

needed to validate, extend and develop the findings.

Further study is especially important because the major research of

Pargament and Silverman (1982) and Newman and Wright (1980), is now over


25 years old. There have been significant changes since the 1980s: more

involvement of the laity, increased ordination of women, expanded access to

other services via televangelism, the rise of the electronic church, and the

growth of the Charismatic movement.

One of the paradoxes of previous studies showing that sermons have

minimal effects on listeners is that millions of people throughout the world

voluntarily listen to sermons and believe that they are benefiting from them.

Our relatively positive findings suggest that this contradiction may be due, in

part, to the discrepancy between what researchers and participants believe to be

the purpose of sermons. We submit that, to be truly beneficial, future

discussions of sermon effectiveness should take into account both the speakers’

aims and the listeners’ hopes and expectations. If Christians perceive sermons

not as lessons to be remembered and applied to daily life, but in terms of

spiritual, emotional and mental stimulation, and primarily as opportunities for

divine love to fill their hearts, then the findings may offer insights not just for

preachers, but for anyone concerned with people’s existential needs.


REFERENCES

Brookes, N. (1998). Shaping a future, incorporating lifting the lid on the New Zealand

church. Auckland, N.Z.: Omega Distributors.

Eckstein, J. L. (2005). “Conversion conundrums: Listener perceptions of

affective influence attempts as mediated by personality and individual

differences”. Communication Quarterly, 53(3), 401-19.

Gordy, J. (2001). “Toward a theology of Pentecostal preaching”. Journal of

Pentecostal Theology, 10(1), 81-97.

Greene, M. (1998). Is anybody listening? Retrieved Sept. 12, 2006, from:

http://www.licc.org.uk/pdfs/isanybodylistening.pdf

Howden, W. D. (1989). “’Good sermon, preacher’: The effects of age, sex, and

education on hearing response to preaching”. Review of Religious

Research, 31(2), 196-207.

Hunsenberger, B., Pratt, M., & Pancer, S. M. (1994). “Religious fundamentalism

and integrative complexity of thought: A relationship for existential

content only?” Journal for the Scientific Study of Religion, 33(4), 335-46.

Joseph, S. A., & Thompson, T. L. (2004). “The effect of vividness on the

memorability and persuasiveness of a sermon: A test of the elaboration

likelihood model”. Journal of Communication and Religion, 27, 217-44.

Kavan, H. (1995). “Glossolalia and altered states of consciousness”. PhD diss.,

Victoria University, Wellington, New Zealand.


Maybury, K., & Chickering, S. (2001). “The influence of pastor status and sex on

evaluations of sermons”. Review of Religious Research, 42(4), 415–24.

Newman, W. M., & Wright, S. A. (1980). “The effects of sermons among lay

Catholics: An exploratory study”. Review of Religious Research, 22(1), 54–9.

Pargament, K. I., & DeRosa, D. V. (1985). “What was that sermon about?

Predicting memory for religious messages from cognitive psychology

theory”. Journal for the Scientific Study of Religion, 24(2), 119–236.

Pargament, K. I. & Silverman, W. H. (1982). “Exploring some correlates of

sermon impact on Catholic Parishioners”. Review of Religious Research,

24(1), 33–9.

Price, D. L., Terry, W. R. & Johnston, B. C. (1980). “The Measurement of the

Effect of Preaching and Preaching plus Small Group Dialogue in one

Baptist church”. Journal for the Scientific Study of Religion, 19(2), 186–97.

Ragsdale, J. D., & Durham, K. R. (1986). “Audience response to religious fear

appeals”. Review of Religious Research, 28, 40-50.

Rokeach, M. (1970). “Faith, hope and bigotry”. Psychology Today, April 1970, 33-

58.

Sethi, S., & Seligman, M. E. P. (1993). “Optimism and fundamentalism”.

Psychological Science, 4, 256–9.

Statistics New Zealand. (2006). “Quick stats about culture and identity”. Table

28 (Religious affiliation) & Table 29 (Age group and sex by religious

affiliation). Retrieved Nov. 15, 2008, from:


http://www.stats.govt.nz/census/2006-census-data/quickstats-about-

culture-identity/quickstats-about-culture-and-

identity.htm?page=para014Master

Taiwo, R. (2005). “Forms and functions of interrogation in Charismatic

Christian pulpit discourse”. Nebula, 2(4), 117-131.

Wagner, C. P. (1973). What are we missing? Carol Stream, IL: Creation House.

Webster, A. C. & Perry, P. (1989). The religious factor in New Zealand society.

Palmerston North, New Zealand: Alpha Publications.


Table 1: Sermon effectiveness: Anglican and Elim.†

Sermon effectiveness

questions

Anglican

median

Anglican

mean

rank

Elim

median

Elim

mean

rank

Significance

p =

Effect

size#

r =

Has a sermon made you

want to praise and thank

God?

Do you think your

knowledge of Jesus

Christ has improved due

to sermons?

Do sermons provide you

with a sense of God’s

love?

Do sermons relate the

message of Christ to the

problems in your life?

In reality, do sermons

affect the way you see

God?

Do sermons answer

your important

questions about God

and religion?

After a sermon do you

ever feel like telling

others about God’s love?

Have you ever changed

your attitude to another

person because of a

sermon you heard?

Have you ever read a

part of the Bible because

of a sermon you heard?

Do the guidelines in

sermons actually change

the way you live?

5

5

5

4

5

4

3

3

3

3

58.11

64.62

71.30

62.44

68.42

65.99

56.74

58.79

55.76

59.58

6

6

5

5

5

5

5

5

5

5

89.62

83.61

77.45

85.62

80.10

82.35

90.88

88.99

91.78

88.25

.000*

.006*

.366

.001*

.091

.018*

.000*

.000*

.000*

.000*

-.379

-.227

-.074

-.278

-.139

-.194

-.405

-.367

-.432

-.345

† 1 = never, 2 = very rarely, 3 = occasionally, 4 = about half of the time, 5 = quite often, 6 =

frequently, 7 = always.

# r < .10 no effect, r = .11 - .29 small effect, r = .30 – .49 moderate effect, r > .50 large effect.

* Significant result at the p. < 05 level.


Table 2: Sermon effectiveness: Anglican and Elim, compared with Newman and Wright’s Catholic responses

Anglican A† Anglican B† Elim A† Elim B† Catholic*

1 - 2 3 – 4 5-7 1 – 2 3 – 4 5-7 1 – 2 3 – 4 5-7 1 - 2 3-4 5-7 1 – 3 4 - 5 6 - 7

n % n % n % n % n % n % n % n % n % n % n % n % n % n % n %

praise 2 5 20 49 19 46 0 0 5 17 25 83 0 0 8 27 22 73 0 0 5 11 42 89 259 48 216 40 60 11

knowledge 3 7 15 37 23 56 4 13 6 20 20 67 2 7 7 23 21 70 0 0 8 17 39 83 179 34 271 50 85 16

love 6 15 9 22 26 64 0 0 3 10 27 90 3 10 7 23 20 67 0 0 6 13 41 87 129 24 234 44 173 32

problems 2 5 26 63 13 32 1 3 10 33 19 63 0 0 10 33 20 67 0 0 9 19 38 81 133 25 234 42 178 33

see 5 12 18 44 18 4 3 10 5 17 22 73 3 10 9 30 18 60 2 4 11 23 34 72 251 47 183 34 100 19

questions 6 15 23 56 12 29 1 3 9 30 20 67 3 10 13 43 14 47 1 2 11 23 35 75 147 28 205 38 182 34

tell 18 44 17 42 6 15 3 10 15 50 12 40 2 7 12 40 16 53 3 6 15 32 29 62 253 48 170 32 113 21

attitude 7 17 28 68 6 15 5 17 14 47 11 37 2 7 9 30 19 63 4 9 18 38 25 53 229 43 170 32 136 25

read 20 49 16 39 5 12 4 13 16 53 10 33 2 7 16 53 12 40 2 4 16 34 29 62 187 35 182 34 91 17

live 10 24 25 61 6 15 5 17 11 37 14 47 2 7 14 47 14 47 3 6 17 36 27 57 244 45 158 29 57 10

% mean 19% 48% 33% 9% 31% 60% 7% 36% 59% 3% 25% 72% 36% 38% 22%

N.B. Some proportion figures do not add exactly to 100% due to rounding.

† 1 = never, 2 = very rarely, 3 = occasionally, 4 = about half of the time, 5 = quite often, 6 = frequently, 7 = always.

* 1 - 3 = never or seldom, 4 – 5 = sometimes, 6 - 7 = often or very often.


Table 3: Desired sermon features: Anglican and Elim†

Desired features

Anglican

median

Anglican

mean

rank

Elim

median

Elim

mean

rank

Significance

p =

Effect

size#

r =

Good eye contact

between the speaker

and the congregation.

6

67.58

6

80.88

.046*

-.163

The presentation of a

clear central topic

with easy to follow

points.

6

77.53

6

71.71

.380

-.072

A sermon with a lot

of Biblical content.

5 60.27 6 87.62 .000* -.327

The presentation of

the Gospel, followed

by an altar call.

2

48.94

6

98.06

.000*

-.580

The use of story

telling and other

anecdotes during the

sermon.

5

69.38

5

79.22

.143

-.120

A sermon containing

humour.

5

74.03

5

74.94

.894

-.010

Wide-ranging

variation in the

speaker’s voice.

4

65.82

5

81.64

.021*

-.189

A sermon with

significant intellectual

content.

5

87.27

5

62.72

.000*

-.292

A sermon containing

strong emotional

appeals.

3

64.87

4

83.38

.007*

-.222

The frequent use of

gestures.

3 63.96 5 84.21 .003* -.242

The use of drama and

other visual

presentations during

the sermon.

3

66.23

4

82.13

.019*

-.192

Opportunity for

questions and other

two way forms of

feedback.

3

57.13

4

90.51

.000*

-.403




* Significant result at the p < .05 level.


Table 4: Desired sermon themes: Anglican and Elim†

Desired themes

Anglican

median

Anglican

mean

rank

Elim

median

Elim

mean

rank

Significance

p =

Effect

size#

r =

The grace and

forgiveness of God.

5 67.00 6 81.42

.036* -.017

The work of the Holy

Spirit in the believer’s

life.

5 63.29 6 84.84 .002* -.025

The life and times of

Jesus Christ.

5 70.96 5 77.76 .322 -.08

Biblical attitudes

towards the poor.

5 73.42 5 75.50 .761 -.02

Questions concerning

the believer in the

workplace.

4 70.68 5 78.02 .283 -.089

Biblical attitudes

towards money.

3 63.79 4 84.38 .002* -.025

The judgement of God. 3 62.14 4 85.90 .000* -.029

Questions concerning

the transformation of

society.

4 77.96 4 71.31 .334 -.079

Heaven and the

afterlife.

3 61.19 3 86.77 .000* -.319

Biblical attitudes

towards sexuality.

3 60.83 3 87.10 .000* -.327

Social justice and

peace issues.

4 87.16 3 62.82 .000* -.290

Hell and damnation. 2 51.27 3 95.92 .000* -.537




* Significant result at p < .05 level.

Sermon responses and preferences in Pentecostal and ... · the need for evaluations of sermon...

Documents

Transcript of Sermon responses and preferences in Pentecostal and ... · the need for evaluations of sermon...