QUT Digital Repository: http://eprints.qut.edu.au/

Lennie, June (2005) An evaluation capacity-building process for sustainable community IT initiatives. Evaluation, 11(4). pp. 390-414.

© Copyright 2005 Sage Publications

1

An Evaluation Capacity-Building Process for Sustainable Community IT

Initiatives: Empowering and Disempowering Impacts

June Lennie

Paper to be published in Evaluation. The International Journal of Theory,

Research and Practice, 11 (4), 390-414.

Introduction

Participatory forms of evaluation aim to produce a range of empowering outcomes and

impacts, including increased community capacities in planning and conducting evaluations and

broader stakeholder participation in decision-making. Other outcomes include improved

communication and trust among stakeholders, and constant improvement of initiatives in ways

that meet community or client needs. In her extremely positive assessment of participatory

evaluation, Diez (2001: 907) suggests that this methodology can be a useful tool to ‘mobilise

communities for regional action, empower local agents and enhance learning capacity’.

Many researchers argue that building community capacities and fostering empowerment are

more effective ways of achieving sustainable community development than programs and

success indicators imposed by outside experts (Harrison, 1998; Mobbs, 1998). They point out

that outside experts usually have limited knowledge and understanding of the particular

context, needs and issues of a community. Local solutions to achieving sustainable community

and economic development are therefore seen as important outcomes of a capacity building

approach to evaluation.

Nevertheless, evaluation methodologies which explicitly aim to be participatory raise many

complex theoretical, methodological and ethical issues, as Gregory (2000), McKie (2003),

Rebien (1996) and others point out. They include issues related to stakeholder

representativeness, the greater level of time, energy and resources required to conduct

evaluations and develop trust, the potential for dependence on the facilitator/professional

evaluation consultant, and for conflicting agendas and perspectives of various stakeholder

groups to hinder success (Gregory, 2000; Papineau and Kiely, 1996; Rebien, 1996). Rebien

(1996) also notes the conceptual weakness of the concept of participation, while the contested

2

concept of ‘empowerment’ is often used in uncritical ways. Indeed those advocating

participatory approaches to research and evaluation often make idealistic or naïve assumptions

that community participation will automatically lead to empowerment.

Thus, while power is a central issue in participatory forms of evaluation, it is often ignored

(Gregory, 2000). Vanderplaat (1995: 85) suggests that even the more critical models of

evaluation have failed ‘to deal, in any meaningful way, with the concept of relative power, or

more specifically the unequal distribution of discursive power, a central construct in

empowerment-based social programming’. As McKie highlights in her recent paper on the

concept of ‘rhetorical spaces’ (McKie, 2003), there is a need to focus on the communicative

and relational dimensions of participatory evaluations which can affect their outcomes in

unintended ways. This requires rigorous analysis of both the intended and potentially

unintended or negative impacts and effects of participatory evaluations.

This paper presents outcomes of a detailed analysis of the empowering and, at times,

disempowering impacts of the implementation of a framework known as ‘the LEARNERS

process’ (Learning, Evaluation, Action & Reflection for New technologies, Empowerment and

Rural Sustainability) for participants in two Australian rural communities. A key aim of the

LEARNERS project, which trialled this framework, was to build community capacities in

evaluating rural communication and information technology (C&IT) initiatives such as

community websites and information literacy programs. Using participatory evaluation and

participatory action research (PAR) methodologies (McTaggart, 1991; Wadsworth, 1998)

informed by praxis feminist theories and methodologies (Lather, 1991; Stanley and Wise,

1990), the project implemented and undertook an ongoing meta-evaluation and critique of the

implementation and use of the LEARNERS process in the two participating communities.

This capacity building framework was considered innovative in that it took a ‘whole of

community’ systems approach, used PAR and participatory evaluation methods, encouraged

analysis of differences such as gender, age, ethnicity and skills with C&IT, and sought to

develop ‘learning communities’ (Faris, 2001). The objectives of learning communities are

closely related to the goals of community capacity building. The process involves community

members from every sector working together to enhance the social, economic, cultural and

environmental conditions of their community and the promotion of lifelong learning (Faris,

3

2001). The implementation of the LEARNERS process also aimed to increase collaboration

and cooperation between diverse community organisations and groups and to generate new

awareness, knowledge and ideas about the use of C&IT for sustainable community and

economic development. The project was considered relatively unique in that few C&IT-based

projects in the community development field appear to have used a combination of PAR and

participatory evaluation methods.

Community members and representatives of community organisations in the Tara and

Stanthorpe Shires in Southern Queensland and representatives of five public sector

organisations participated in the project, which was conducted by a research team from

Queensland University of Technology (QUT) in Brisbane. The trial of the LEARNERS process

aimed to assist in enhancing the sustainability and the C&IT initiatives evaluated by groups in

the participating communities, and to increase community participation, empowerment and

leadership. The project also sought to increase recognition of the importance of women in

Australian rural communities in the uptake of new C&IT and in community development and

informal leadership activities (Grace and Lennie, 1998; The Rural Women and ICTs Research

Team, 1999).

After an overview of the development of the LEARNERS process, the aims of this capacity

building process and the methods involved in implementing the process in the two

participating communities are outlined. A brief summary of case studies of the implementation

of the process is then provided, followed by outcomes of a detailed analysis of the impacts of

the project for participants and their communities. While this impact assessment identified a

range of empowering effects of the LEARNERS project, a number of unintended and

disempowering impacts and other limitations of the project and the LEARNERS process were

also identified, as well as a number of barriers to participation. Drawing on these findings, the

strengths and limitations of the LEARNERS project and process are considered, as well as

lessons learned in the project for other evaluation capacity building projects.

Participatory approaches to research and evaluation

The emergence of participatory approaches to research and evaluation

Greater use of participatory and action research methodologies was made from the 1980’s

onwards as organisations started taking more inclusive, social justice-based approaches to the

4

design, implementation and assessment of community-based interventions. Evaluation

discourses shifted from those of the positivist social scientist to those of the empowerment-

oriented critical adult educator and social action researcher (Vanderplaat, 1995: 83). The

emphasis also shifted from knowledge created through social science to knowledge generated

through lived experience (Vanderplaat, 1995: 83).

Participatory action research aims to develop equal partnerships between researchers and

participants and to create knowledge that leads to action and positive social change.

Knowledge is seen as related to power and power is related to change. It is a political process

because it involves people making changes together that affect others (McTaggart, 1991: 177).

Critical reflection is a crucial step in each PAR cycle of planning, acting, observing and

reflecting. As in participatory evaluations, PAR encourages the active involvement of

participants and stakeholders in the design and conduct of projects and supports capacity

building processes. It can be a valuable method for involving a diversity of people in projects,

generating appropriate action, new ideas and long term visions, fostering ongoing change and

improvement, and regularly reflecting on outcomes (McTaggart, 1991). Thomas (2000: 112)

argues that PAR ‘challenges the hegemony of orthodox evaluation research methods’ and

therefore offers more opportunity to develop and evaluate long-term strategies for widening

participation in education and lifelong learning.

PAR has been successfully used in a large number of community development projects

conducted in developing countries (Garaway, 1995), as well as in community-based projects in

developed countries. It has been employed in a diversity of fields, including community

development, agricultural extension, education, health, and organisational management.

Participatory evaluation

Participatory evaluation methodologies emerged from the extension of PAR to evaluation

(Garaway, 1995) and the growing interest in evaluation as an action learning and capacity

building process. Collaborative or participatory forms of evaluation are seen as particularly

useful in assessing the impacts of ‘more complex system change and comprehensive

community initiatives’ (The Kellogg Foundation, 1998:5). Such initiatives include those

involving the use of new communication technologies. Participatory evaluations have been

argued to contribute to the long-term sustainability and success of community and economic

5

development programs (Brunner and Guzman, 1989; Diez, 2001; Dugan, 1996; Fetterman et

al.., 1996; Hudson, 2001; Papineau and Kiely, 1996; Rebien, 1996). They have been conducted

in a wide diversity of fields since the 1970s, including agriculture and rural development,

education, social services and health. However, this methodology has not been widely used to

evaluate C&IT projects.

Three main reasons have been proposed for increasing the involvement of participants and

stakeholders in evaluations:

(1) to increase utilisation of evaluation results; (2) to represent the values and concerns

of the multiple groups involved in decision-making; (3) to promote the empowerment

of disenfranchised stakeholder groups previously left out of the process (Papineau and

Kiely, 1996: 81).

As this quotation indicates, empowerment is often equated with participation in participatory

evaluations. However, different forms of participatory evaluation emphasise different levels of

participation. Empowerment evaluation (Fetterman et al., 1996) is notable in that it encourages

active involvement in all stages of the evaluation. Both empowerment and participation are,

however, contested concepts. Humphries (1994) points out that the concept of empowerment

can be used to justify oppressive practices; while the forms of participation identified range

from co-option to collective action (Martin, 2000: 200).

For an evaluation to be considered participatory, Rebien (1996: 160) suggests that stakeholders

must have an active role in the evaluation process, that at least representatives of stakeholders

should participate, and that stakeholders should participate in at least three evaluation phases:

‘designing terms of reference, interpreting data, and using evaluation information’. However,

Gregory (2000: 184) suggests that these criteria are insufficiently defined and ‘indeed, may

promote practices which actually have a negative impact on participation, rather than a positive

one’.

The methods and ethos of participatory evaluation thus contrast markedly with traditional

evaluation methods such as quasi-experimental impact assessments. In traditional approaches,

the key questions and methods to be used are decided by the evaluator or funding body, the

evaluator is expected to adopt an impartial and objective perspective, and program activities

6

are reduced to measurable indicators (Vanderplaat, 1995). In contrast, participant-oriented

forms of evaluation tend to use more ‘naturalistic’ inquiry methods that aim to reflect ‘the

complexities of everyday reality and the different perspectives of those engaged in providing

services’ (Worthen, Sanders and Fitzpatrick, 1997: 154). The scientific ideal of objectivity is

usually rejected in favour of a more holistic approach that openly acknowledges and takes into

account the diverse perspectives, agendas and values of participants, stakeholders and

evaluation consultants. However, evaluation rigor and the validation of findings is obtained

through using multiple methods of data collection and analysis, triangulation of data, peer

review, and engaging in critical reflexivity to enhance the ‘trustworthiness’ of results (Chess,

2000; Guba and Lincoln, 1989).

Questioning assumptions about community participation and empowerment

The research team working on the LEARNERS project adopted a critical approach which

questioned assumptions about community participation, empowerment and the sustainability of

C&IT projects. This approach recognised the often complex and contradictory nature of these

processes that can affect the outcome of participatory evaluations, as well as the importance of

paying attention to the communicative and relational dimensions of evaluation.

Taking barriers to participation into account

Research suggests that community participation methods that aim to be inclusive and

empowering and involve the use of new C&IT raise many complex issues. A range of social,

cultural, organisational, technological and economic factors and barriers have been found to

limit the effectiveness of participatory processes involving diverse community members and

groups (Boyce, 2001; Lennie, 2001, 2002b; McKie, 2003). These factors include the time,

energy and costs involved, people’s level of familiarity with participatory processes, lack of

effective access to C&IT, and the different agendas and power relations among various

stakeholder groups. Differences in knowledge about new C&IT, and concepts such as

‘community capacity building’ are other factors that need to be considered. In projects

involving academics and bureaucrats from urban areas and people and organisations in rural

communities, these factors are often highly relevant to the success or otherwise of strategies for

empowerment and inclusion in participatory evaluations (Lennie, 2001, 2002b). Analysis of

the impacts of the LEARNERS project took these complex issues and factors into account.

7

Critically evaluating claims for empowerment

Feminist critiques suggest that research projects which aim to be empowering can have

contradictory effects and that claims that research and evaluations have produced empowerment

require closer examination (Anderson, 1996; Gore, 1992; Lennie, 2002a; Lennie, Hatcher and

Morgan, 2003). This entails adopting a more critical and self-reflexive approach and using

more rigorous methods to evaluate such claims. Such an approach was taken in the meta-

evaluation and critique of the LEARNERS project, which included a rigorous analysis of the

impacts of the LEARNERS project. This analysis employed a framework of rural women’s

empowerment which was developed as part of my doctoral project (Lennie, 2001). Building on

the work of Friedmann (1992), this framework comprises four interrelated forms of power:

social, technological, political and psychological (see Figure 2 later in this paper).

Corresponding forms of disempowerment were also analysed, and gender and other differences

between participants were taken into account.

The various capacity building methods used in the LEARNERS project were seen as

underpinned by positive ‘power to’ and ‘power with’ models in which power is considered as

social and cooperative, rather than negative and related to domination and control (Deutchman,

1991). Empowerment was viewed as a process that can happen at the level of the individual,

the group, and/or the community (Claridge, 1996). Friedmann (1992: 33) suggests that

psychological empowerment can result in an increased sense of power, confidence and control,

which is often the result of successful action. Empowerment was seen as a long-term process

that people undertake for themselves, rather than something that is done to or for another

person (Lather, 1991: 4). However, outsiders, such as evaluation consultants, were considered

to play important roles in the process of empowerment by providing information and support,

and encouraging less empowered participants to gain confidence and ‘free themselves of

traditional dependency’ (Friedmann, 1992: 77).

Building community capacities in evaluation

The trial of the LEARNERS process aimed to build community and organisational awareness,

skills, confidence and capacities in participatory planning and evaluation and PAR.

Community capacity building has been defined as ‘strengthening the knowledge, skills and

attitudes of people so that they can establish and sustain their area’s development’ (Mannion,

8

1996: 2). It is seen as the ability of communities to solve their own problems, make their own

decisions and plan their own futures. Community capacity building programs aim to increase

community participation in planning and decision-making, to facilitate sustainable

development by building on existing community strengths, and to create communities that are

more inclusive, cooperative and self-reliant. In the LEARNERS project, capacity building also

referred to the process of building the capacity of organisations and community members to

undertake ongoing evaluations of local C&IT projects.

Our previous pilot research project indicated that rural community members and staff of

organisations that work closely with rural communities in Queensland often have limited skills

and experience in participatory forms of research, planning and evaluation (Lennie, Lundin and

Simpson, 2000a). The need to build community and organisational capacities in these

processes has been increasingly recognised by others working in this field (Fetterman et al.,

1996; Wadsworth, 1997; Boyle and Lemaire, 1999; O’Sullivan and O’Sullivan, 1998; Khan,

1998). PAR, empowerment evaluation, and participatory evaluation have been found to be

effective in developing these capacities.

Sustainability of C&IT initiatives

Governments and rural industry organisations in Australia have positioned new technologies

such as the Internet as being at the forefront of community and economic development in rural

and regional areas. Rural people are expected to benefit economically from new technologies

because they will provide new employment opportunities, the potential to buy and sell online,

an increase in services, and access to education and training, among other impacts (Da Rin and

Groves 1999; Groves and Da Rin 1999a, 1999b; Hearn et al., 2005).

However, many C&IT initiatives in rural Australia have failed (Geiselhart, 2004). This is due

to factors such as the small, highly scattered populations in many rural areas, limited funding

and resources, lack of access to training and technical support, and reliance on enthusiastic

‘champions’ and volunteers to successfully maintain initiatives. The long-term sustainability

of these initiatives is therefore a significant issue for rural communities.

Given that major Federal government funding programs for C&IT projects in Australia have

wound down in recent years, the continuing feasibility of these projects requires local

9

communities to find ways to make them self-sufficient and economically viable (DCITA,

2003). An approach to evaluation and impact assessment that takes a wide range of social,

cultural, economic, technological and environmental factors into account is therefore required.

This type of ‘whole of community’ systems approach was an important feature of the

LEARNERS process.

Development of the LEARNERS process

The LEARNERS process was based on a framework for planning and evaluating C&IT

projects which was designed and developed as part of a pilot research project. This project was

conducted from 1999 - 2000 by three QUT researchers in collaboration with two public sector

organisations (see Lennie et al., 2000a and 2000b).1 Development of this framework drew on

findings from substantial earlier action research projects undertaken by QUT researchers

(including the author) in partnership with people in rural, regional and remote Queensland

communities and several government and industry partners.

A multidisciplinary and iterative approach was taken in developing the original framework.

This involved the following research activities:

• A comprehensive search and critical review of relevant literature in the fields of

community development, C&IT impacts, evaluation and impact assessment methodologies,

and community participation and inclusion.

• Ongoing meetings and discussions that drew on the knowledge and expertise of the

research team and collaborating partners in diverse areas such as rural community

development, rural women’s access to and use of C&IT, and evaluating the use of C&IT in

education and training programs.

• Drawing on this literature, the outcomes of earlier research projects, and the ideas and

knowledge of personnel in the collaborating organisations, to begin collaborative

development of the framework.

• Ongoing redesign of the framework.

• Seeking feedback on the framework from diverse participants in two focus groups held in a

Queensland rural town with a total of eight women and three men and in a teleconference

that linked five women and three men in six rural and regional communities in Queensland,

10

most of whom were local coordinators of QOLN2, one of the collaborating organisations,

which provides support to distance education students.

• Making revisions to the framework based on this feedback.

This earlier research suggested a need to find new ways of making rural C&IT projects more

sustainable; a need to build community capacities in planning and evaluation; and a need to

take diversity, difference and inclusion into greater account in these processes. It also

suggested that the direct and more indirect impacts and ‘ripple effects’ of C&IT initiatives

needed to be identified, and that a ‘whole of community’ systems approach to analysis could

provide a more comprehensive understanding of these impacts. Participatory methods were

seen as valuable in undertaking these activities. This pilot research indicated that there had

been little formalised and coordinated planning, needs assessments and evaluation of C&IT

initiatives in rural and regional Queensland, and that ad hoc or informal processes were often

used.

Overview of the LEARNERS process and project

The LEARNERS project, which implemented and evaluated the trial of the framework in the

Tara and Stanthorpe Shires, was conducted from 2001 - 2004. Full details about the project are

provided in Lennie et al. (2004). The project commenced with a workshop involving the

research team and representatives of the project’s five collaborating partners which critically

assessed the original framework and recommended revisions. This revised framework, which

was subsequently labelled ‘the LEARNERS process’, is shown in Figure 1.

As discussed, the implementation and use of the LEARNERS process aimed to achieve the

following outcomes:

• the long-term sustainability of C&IT initiatives;

• increased collaboration and cooperation between people and organisations from various

sectors in rural communities;

• enhanced community capacity in participatory planning and evaluation methods;

• increased community participation and inclusion in planning and evaluation processes;

• giving value to empowering forms of leadership based on collaboration, sharing

information, networking, encouraging participation, and supporting people; and

11

• the development of learning communities.

The whole of community approach of the LEARNERS process encouraged the inclusion of a

broad diversity of people from different sectors of the community in planning and evaluating

the C&IT initiatives and collaboration and cooperation between community groups and

organisations from a diversity of sectors such as local government, education, and business.

The aim was to assess the direct and more indirect impacts of C&IT initiatives on different

groups and sectors of the community. Local project coordinators in each of the participating

communities were therefore encouraged to invite a broad diversity of community members and

representatives of a range of community organisations and groups to workshops and other

activities. They targeted people involved in or affected by the local C&IT initiatives that

community members chose to evaluate.

Figure 1 outlines the goals and outcomes of the LEARNERS process and the proposed PAR

cycles and steps involved. A document outlining possible strategies and questions to consider

at each of the nine steps involved was also prepared and distributed to participants to further

explain the process as well as a range of other relevant information and resources. This

information was placed on the LEARNERS project website: http://www.learners.qut.edu.au.

ADD FIGURE 1 ABOUT HERE

As well as using an inclusive, whole of community approach, the methods and processes

involved in implementing the ‘ideal’ LEARNERS process included:

• Identifying and building on existing community skills, knowledge and resources.

• Engaging in continuous PAR cycles of planning, acting, observing and critically reflecting

on the actions taken.

• Ensuring that all participants’ ideas, comments and feedback are included and taken into

account.

• Collaboratively planning and conducting evaluations that enable ongoing learning about

and constant improvements to C&IT initiatives.

• Gathering data that enables analysis of relevant differences such as gender, age, and levels

of skills, knowledge and access to C&IT.

12

Implementation of the LEARNERS process

Using feminist approaches to PAR (Gatenby and Humphries, 1996; Maguire, 1996), the

methodology used in the LEARNERS project involved:

• Drawing on local knowledge to identify key issues related to C&IT in the participating

communities and relevant contextual information about gender and leadership, community

networks, and other issues.

• Community participants collaboratively deciding on the C&IT initiatives which they

wanted to evaluate and, based on the outcomes of their evaluations, democratically

deciding on strategies that would improve their initiatives and increase community

participation and awareness.

• The research team and community participants collaboratively deciding on the format and

content of project activities such as workshops, and designing project activities in ways that

aimed to meet the diverse needs of participants and to be inclusive and empowering.

• Taking gender and power issues into account in designing and evaluating project activities.

• Community participants, the research team and staff of the collaborating industry partners

actively participating in project activities, including fieldwork and teleconferences, and

regularly sharing knowledge and information.

Communication processes included: holding regular meetings, workshops, teleconferences and

videoconferences involving community participants and collaborating partners; and sharing

information through two email discussion lists. Feedback on the project and suggested

revisions to the LEARNERS process were collected via email and the project’s online

discussion lists. Critical reflection workshops involving the research team, collaborating

partners, local project coordinators and interested participants were conducted both face to face

and via interactive conferencing technology in each year of the project.

While the participatory methodologies used in the project aimed to create an equal partnership

between the community participants, research team, collaborating partners and other

stakeholders, we recognised that this ideal was not easy to achieve in practice. We also

recognised that PAR requires sufficient time, energy and resources to be effective and that they

would not always be readily available to everyone involved in the project.

13

An ‘open enquiry’ approach to evaluation (Wadsworth, 1997) was used that involved regularly

seeking feedback from participants on what worked well, what did not work so well and how

various project activities could be improved, and taking this feedback into account in designing

future activities. Multiple qualitative and quantitative methods were used to evaluate project

activities and to undertake the meta evaluation and critique of the project. Evaluation methods

included: distributing short feedback forms and questionnaires at community workshops,

holding focus group discussions and semi-structured individual interviews with a diversity of

participants (ie men and women with different occupations, active and less active participants,

volunteers and paid workers etc), conducting participant observations of project activities, and

making entries in a fieldwork diary.

Community workshops used participatory methods such as small group discussions,

brainstorming and ranking of priority actions and strategies emerging from evaluations.

Information sheets about various forms of evaluation and the techniques used during the

workshops were prepared and distributed to workshop participants. They were later published

on the LEARNERS project website, along with a wide assortment of other evaluation

resources. Detailed notes from all community workshops and meetings were prepared and

distributed to all participants and collaborating partners.

Most of the qualitative data gathered during the project was entered into the QSR NVivo

program which was used to code text into broad themes and issues. The process of coding and

analysing the impacts of the project used the framework of rural women’s empowerment

described in Figure 1. The use of multiple participation, research and evaluation methods

provided a variety of rich research data and ongoing feedback. Together with the data coding

and analysis process, this enabled rigorous validation of the findings and the data analysis

results. To enhance the trustworthiness of the evaluation results, feedback and critical

comments were also sought from staff of collaborating partner organisations and community

participants on case studies, data analysis, and project reports and resources. Revisions were

made, based on this feedback.

14

Summary of the case studies

A brief summary of the case studies of the implementation of the LEARNERS process in the

Tara and Stanthorpe Shires is provided to provide some context for the analysis of project

impacts. The full case studies are found in Lennie et al. (2004).

When the project commenced, the larger Tara Shire had a small, highly scattered population of

approximately 3,800 people, while the smaller Stanthorpe Shire had a population of nearly

10,400 people. Elected leaders and senior staff of local Councils in both Shires were keen to

support and participate in the project. While the Tara Shire was considered to be a

disadvantaged community with a significant lack of services, particularly in communication,

the Stanthorpe Shire was more advantaged in terms of communication, education, housing and

other services. Both communities had already implemented new C&IT initiatives, including

community websites, computer and Internet training programs, and Learning Network

Queensland centres which provided support to distance education students.

Community leaders in the Tara Shire were interested in using the LEARNERS process to assist

the community to work together to reach its goals, and to engage in more effective planning

and evaluation. They hoped that participation in the trial might improve communication across

the Shire, as well as training and access to C&IT. Community leaders in the Stanthorpe Shire

were initially interested in finding ways to increase the sustainability of ‘GraniteNet’, a virtual

community project managed by the Shire Council that had established 80 online community

interest groups and enabled people to start their own interest group or build their own website.

In the Tara Shire, participants selected the Tara Shire Community Website, and IT training and

access across the Shire as the C&IT projects to be evaluated, while, following negotiations, the

core group in Stanthorpe elected to conduct an evaluation of the Lifelong Learning Group’s

website on GraniteNet as a pilot project.

In both communities, a larger number of people participated in initial workshops while a

smaller core group continued their involvement, including planning and conducting the

evaluations of local C&IT projects. The maximum number participating in workshops in Tara

was 23, while a maximum of 18 people took part in Stanthorpe workshops. As in the

community capacity building project reported by O’Meara, Chesters and Han (2004), involving

a wide diversity of community members and organisations was problematic. The majority of

15

active participants were women in the 40-59 age group with a white and/or Anglo-Celtic

ethnicity. Many participants worked in the areas of community development, education and

training or local government, in both paid and voluntary positions. Some participants in both

communities held formal leadership positions in local government, community or business

groups and organisations.

McKie (2003: 310) points out that the language and terms used in research and evaluation tend

to ‘exclude those who do not understand or use these terms’. Problems were initially

experienced in both communities with generating understanding about the project. The

terminology and methods used to explain the process were barriers to understanding for some

participants, particularly in the Tara community. While groups such as school teachers and

principals understood PAR and the notion of evaluation as an action learning process, it was

unfamiliar to other community members who were seeking more concrete outcomes and

greater direction from the research team.

A key outcome in the Stanthorpe Shire was the formation of a group who enthusiastically

commenced a related project that aimed to develop the Shire as a Learning Community.

However, many Stanthorpe participants became confused about the relationship between the

LEARNERS project and their local Learning Community project. This affected participation

and outcomes of the LEARNERS project in this area. Nevertheless, various positive impacts of

the project were reported in each community, including improved communication and

networking, and increased skills and knowledge in participatory planning and evaluation.

Revisions to the LEARNERS process

Feedback from community participants about their need for a simpler, easier to understand

version of the LEARNERS process and more case studies and examples, led the research team

to redesign the LEARNERS process from late 2003 to early 2004. The revised process that

emerged was a simple four step evaluation process with key questions and a comprehensive

case study of the whole process, based on outcomes from the LEARNERS project. Following

extensive discussions, the term ‘review’ was mainly used in preference to ‘evaluation’. The

four steps in the ‘EvaluateIT’ process are:

1. Plan the review.

16

2. Involve people in the review.

3. Do the review.

4. Review results and make the changes.

Details of these four steps were published with a variety of other information and evaluation

resources in the online EvaluateIT resource kit.3 Feedback on the prototype kit was sought

through questionnaires, workshops and focus groups, which were held in the participating

communities and in two other regional communities in North Queensland in July 2004 (see

Lennie et al., 2004). The final version of the kit is located at http://www.evaluateit.org.

Impacts of the LEARNERS project In addition to activities related to implementing the LEARNERS process, the research team

conducted an ongoing meta-process of evaluation, reflection, critique and analysis of the

LEARNERS project and its impacts on participants and their organisations and communities.

Impacts are understood as the changes that occur as a result of a project or intervention. They

include intended and unintended, positive and negative, empowering and disempowering

impacts. My earlier research in this field indicates that both empowering and disempowering

impacts and outcomes can be anticipated in PAR projects involving people from diverse

backgrounds who have different levels of status, power and knowledge, unequal access to

C&IT, and different goals and agendas (Lennie, 2001, 2002a; Lennie, Hatcher and Morgan,

2003). While empowerment and disempowerment were seen as separate and distinct for the

purpose of this analysis, this earlier research demonstrates that there is an inter-relationship

between these two processes.

Details of the multiple sources of qualitative and quantitative data on project impacts that were

used in this analysis and the data analysis and validation process are provided in Lennie et al.

(2004). The framework of empowerment and disempowerment used in the analysis is shown in

Figure 2. Most of the analysis reported below was obtained from feedback and information

provided by fifteen rural participants during interviews, focus group discussions and critical

reflection workshops. They comprised nine participants (eight women and one man) from

Stanthorpe Shire and six participants (four women and two men) from Tara Shire. Data from

workshop feedback questionnaires, fieldwork notes, and critical reflection workshops were

also included in the analysis.

17

ADD FIGURE 2 ABOUT HERE

Empowering impacts

The analysis indicated that participation in the project had a range of empowering impacts and

effects on participants in both communities. The social, technological, political and

psychological forms of empowerment that interviewees and other participants’ were

considered to have experienced are considered below.

Social empowerment

The analysis indicated that many participants in each community experienced various forms of

social empowerment, which is connected to the development of social capital (Woolcock,

Renton and Cavaye, 2004). The most significant forms of social empowerment identified were:

Gaining new knowledge or understanding of participatory planning and evaluation: The

majority of participants increased their knowledge and understanding in this area. Feedback on

participants’ use of the EvaluateIT kit indicated that it assisted in further developing

knowledge and understanding of participatory evaluation. The four step EvaluateIT process

was seen as simple and easy to understand for people without significant evaluation

experience, inclusive and flexible. Ongoing use of this resource is expected to continue helping

to increase community capacities in evaluation.

Increased evaluation and communication skills: Six interviewees (three women in Stanthorpe

and one woman and one man in Tara) considered that they or others had developed or

enhanced their skills in areas such as small group facilitation, brainstorming, and designing and

conducting evaluations. Some also improved their general skills in leadership, communication

and interaction.

Obtaining or sharing new information: Several participants and collaborating partners

indicated that they valued much of the information and resources provided during the project,

which was often forwarded to others via community websites and networks. This information

sharing was also seen as helping people ‘appreciate the value of other perspectives’.

Appying new knowledge, information and skills to other contexts: Nine participants (five

women and one man in Stanthorpe and two women and one man in Tara) reported that they

18

and others had applied their new or enhanced skills and knowledge in other contexts such as

their paid work in community development with youth or other community groups, or in their

voluntary positions in community organisations.

Networking with others: Seven participants (four women in Stanthorpe and two women and

one man in Tara) identified the opportunity to network and interact with others in their

community and elsewhere as a valuable outcome of the project. Networking among the

Learning Community Project Group in Stanthorpe via email and face to face meetings was

seen as a significant outcome of the project.

Participating in various group activities: Six interviewees (four women in Stanthorpe and one

woman and one man in Tara), as well as several workshop participants in both communities,

reported that they valued and often enjoyed the opportunity to take part in the various

groupwork activities such as workshops, meetings and videoconferences.

Other impacts related to social empowerment, mentioned by a smaller number of participants

and interviewees in both communities, included:

• The ‘better direction’ provided by the research team.

• The sense of collaboration and cooperation among community participants and the

‘community empowerment’.

• Feeling as if they were on an equal standing with the research team and industry partners

and other participants who were seen as having a higher status.

• The opportunity to reflect on the community and the issues raised in the project.

• Gaining a broader perspective on the community.

Technological empowerment

Eleven of the fifteen interviewees (nine women and two men) indicated that they and others in

their community had developed a new awareness or knowledge of new C&IT, or local C&IT

initiatives and issues. For example, staff and elected members of the Tara Shire Council

gained a significantly increased awareness and developed new ideas about the innovative use

of email, the Internet and other technologies. This led to the redesign of the Tara Shire website

and greater use of C&IT such as teleconferencing by the Council.

19

Eight interviewees (three Stanthorpe women, and three women and two men in Tara) also

provided examples of new uses, and greater use and confidence in using C&IT, or the

development of new skills in using various technologies. This was particularly evident in Tara.

Participants from both communities also gained useful skills, experience and knowledge in

using an interactive conferencing system to participate in three major LEARNERS project

events.

Political empowerment

While the project did not involve participants in providing a direct input into policy-making, it

produced political empowerment to some extent. Participation in workshops enabled

community members to ‘have their say’ about issues of concern, and to take collective action

on issues and the promotion or improvement of new C&IT initiatives that aimed to benefit the

community, or could help improve sustainability.

Participants in the Tara Shire saw the project as assisting in bringing the community together

to work towards a common goal. The formation of an active Learning Community Project

Group was considered as a significant outcome of the project by seven of the nine Stanthorpe

interviewees (six women and one man). This group has continued to meet on a regular basis to

plan and organise various lifelong learning activities.

Psychological empowerment

Self-confidence and self-esteem are essential first steps to empowerment (Anderson, 1996;

Claridge, 1996). The majority of participants appeared to have a fairly high existing level of

psychological empowerment. However, two less confident female participants reported an

increase in their level of confidence. Most participants also reported feeling very comfortable

participating in workshops and other activities as they were viewed as ‘non-threatening’,

‘informal, safe and respectful of the individuals participating’, and participants did not feel

pressured.

One of the critical reflection workshops that linked participants in Tara and Stanthorpe with the

research team and collaborating partners in Brisbane via interactive conferencing technology,

was seen as particularly empowering for participants in Stanthorpe. The local Stanthorpe

facilitator reported that this technology enabled participants to feel more ‘free to speak their

20

minds’ about what had not worked well in the project. This process was seen as ‘really

effective’.

Disempowering impacts

While the analysis suggested that the project resulted in a wide range of empowering and

positive impacts and effects, interviewees also reported some disempowering, negative or

unintended effects. Some of these effects appeared to have been experienced to a greater extent

by participants in Stanthorpe than in Tara. This appeared to be due to a number of factors,

particularly the initial misunderstandings and confusion about the project and its relationship to

the Stanthorpe Learning Community project.

Social disempowerment

The two most significant areas of social disempowerment identified by several participants in

both communities were:

• Not gaining sufficient knowledge or understanding about the project and the LEARNERS

process. Factors included the unfamiliar language that was used, the perceived complexity

of the diagram and other information that was used to explain the LEARNERS process, and

the purpose of the project being open to multiple interpretations.

• Initial confusion about the purpose of the project, which was greater in Stanthorpe than in

Tara.

In addition, the ongoing, cyclical nature of the LEARNERS process did not meet the needs of

some participants who were more ‘outcome driven’. Others were disappointed about the

perceived lack of immediate outcomes from the project. Understanding about the project

increased over time for those who remained active participants but (along with factors such as

lack of time, relevance and interest and living at a distance from workshop venues and lack of

initial understanding) was likely to have been a factor in several of the early participants’

decision not to continue involvement. These negative effects could have contributed to some

participants considering that the LEARNERS process did not meet their needs and/or that there

was a lack of outcomes.

21

Technological disempowerment

Participants also experienced various forms of technological disempowerment during the

project at times. However, some of these experiences were not directly related to project

activities. The main issues reported were:

• Frustration due to technological problems encountered while participating in some

workshops and meetings via conferencing technology.

• A lack of confidence with using computers, email and conferencing technologies which

were unfamiliar to some participants.

• Limited access to computers and email facilities.

• Lack of local IT training and support, which was a particular issue in the Tara Shire,

following the departure of the IT Support Officer, who was not replaced by the Council.

Political disempowerment

Some participants and project partners thought that certain project activities had been

controlled by particular participants and stakeholders, while one woman reported that she and

others in her community had experienced a lack of influence and voice. These two effects

appeared to result from different perceptions about ownership of the project and control of the

agenda, and other factors. The control of activities was considered to have impacted on the

effectiveness of the project in one of the communities in particular.

One reason for these outcomes was the pre-existing formal and informal networks that existed

between key stakeholders within one of the communities. These networks and alliances

affected community perceptions of the project, especially about who controlled its agenda and

what outcomes were expected. One of the learnings from the project was that such issues need

to be dealt with in a proactive but sympathetic manner through strategies such as choosing

local ‘champions’ and coordinators carefully and personally approaching key leaders and

stakeholders in the community (Lennie et al., 2004: 53-54).

Psychological disempowerment

A few participants reported experiencing a lack of confidence to participate in project activities

at times. Some participants also experienced frustration due to factors such as difficulties in

22

attracting new people, the confusion about the project, or having a lack of time and capacity to

participate. One participant also expressed concerns about the perceived value of her

contribution as an unpaid volunteer, compared with those in paid community development

roles.

Conclusions: The strengths and limitations of participatory evaluation

This paper has provided some critical insights into the complexity of building community

capacities in evaluating rural C&IT projects and the unintended or contradictory outcomes that

can emerge. The research team’s critical reflections on the LEARNERS project and process,

and our analysis of the impacts of the project, suggest that both the project and the

participatory evaluation process had several strengths, but also some limitations, which are

summarised in Table 1. Several barriers to community participation and inclusion are also

identified in Table 1.

Table 1: Strengths and limitations of the LEARNERS process and project

Strengths Limitations

Builds on the existing community strengths and capacities, including skills, knowledge, social capital, and human and technological infrastructure.

Greater time and resources were required to adequately identify the existing skills and knowledge of community members and to encourage their involvement. Loss of key community champions affected progress.

The process was effective in producing various forms of empowerment and building capacities in participatory planning and evaluation. Some participants transferred these skills to other contexts.

The project had some unintended and disempowering impacts on participants which affected its success. Capacity building was limited to a small core group of participants in each community.

The involvement of a broad diversity of community members and groups was encouraged. The process encouraged consideration of the needs of various groups, such as older and younger people, people in remote townships, and from non English speaking backgrounds.

Obtaining ongoing participation and involvement from the whole diversity of community members and sectors was problematic. The process attracted mainly women, people involved in community development and education, and reflective learners.

The processes used were flexible, open and transparent and could be readily adapted to meet the needs of different community groups and projects. The participatory methods used in the project were seen as applicable to other non-C&IT projects.

The LEARNERS process was initially considered complex and confusing by many participants. In addition, a paradox of the process is that it is not interventionist enough. This can result in the agenda being dominated by particular community members and groups.

The processes used in the project helped the community groups better develop and maintain a focus, define clearer objectives, and decide on priorities for action. The LEARNERS process

Participatory evaluation and PAR methods appear to fit better with some people’s values, needs and agendas than with others. Some participants preferred a more outcome-oriented process.

23

provided a useful framework for evaluation, planning, needs and impact assessments and critical reflection. All input from participants was considered valuable.

The process helped community groups to maintain interest and motivation through the positive feedback and support from other interested community leaders and members and the research team, which was provided through the evaluation process.

Planning, organising, conducting and evaluating project activities was very time consuming. The process required a commitment to regularly participate in activities and to develop new ideas for improving C&IT projects and generating community interest. There was some dependence on the research team to facilitate activities and encourage action.

Conferencing technologies and email were effectively used in the project to critically reflect on the project, maintain communication and provide information.

Poor quality technological infrastructure and lack of C&IT access in the communities affected participation and communication at times.

The project’s rigorous ongoing evaluation process provided a strong test of theories about community empowerment, inclusion and participation and the role of C&IT in these processes. The use of gender analysis highlighted the important leadership roles of rural women.

The funding, time and resources required to effectively undertake the ongoing evaluation and impact assessment process was limited. This created difficulties in effectively meeting all the project objectives and deadlines, and providing adequate training and support to participants.

The project generated good mutual learning and understanding for the community participants, industry partners and the research team.

Only the participants and industry partners who were actively involved for the duration of the project developed new learnings and understanding.

As this table indicates, the ideal LEARNERS framework and the reality of implementing the

process were somewhat different. The framework provided a flexible and inclusive whole of

community process for participation in the evaluation of rural C&IT projects. It built on

existing community capacities, resources and goals, generated mutual trust and understanding

between the diverse groups involved, and produced various forms of empowerment. The

participatory evaluation and PAR processes that were used assisted in identifying strategies to

increase the sustainability of C&IT projects in the two communities. Using a range of new

technologies to communicate with and support distant participants assisting in producing

various empowering impacts. Outcomes of the project were particularly positive in the Tara

Shire which was considered to be a disadvantaged community with a high level of need for

improved communication systems and better community networking and cohesion.

However, due to inequalities in power and knowledge, the different values and agendas of the

participants and researchers, the pre-existing relationships and networks within the

communities, and other complex issues, the project also had a number of unintended and

disempowering impacts. They included a perceived lack of ownership and control of some

24

project activities, and confusion and misunderstandings about the project and the LEARNERS

process. Several other barriers to community participation, empowerment and capacity

building were identified, such as a lack of time and/or capacity to participate, the loss of key

‘champions’ in the community, and lack of access to or limited experience with C&IT. Similar

barriers have been found in other community-based projects (Boyce, 2001; Lennie, 2002a;

O’Meara et al., 2004).

Many of these limitations and issues have also been identified in the participatory evaluation

literature. However, as Gregory (2000) points out, the central issue of power is often neglected

by those using participatory evaluation methodologies. This paper highlights the need for a

more open, honest, critical and reflexive approach to evaluating claims for the empowering

impacts of evaluation capacity building projects. This approach requires the rigorous analysis

of both the intended and unintended impacts of such projects. Gender and power issues also

need to be taken into account in such evaluations, particularly issues related to leadership,

communication and control. The framework of empowerment and disempowerment that was

used to analyse the impacts of the LEARNERS project was considered to provide a useful

means of conducting such an analysis. A more in-depth analysis of the gendered power

relations enacted in the project, using feminist discourse analysis methods, was originally

planned, but became beyond the scope of the project.

The outcomes of the LEARNERS project indicate that participatory evaluations and PAR are

political processes that can have unintended impacts and effects. These methodologies also fit

better with some people’s values and needs than with others (Lennie and Hearn, 2003). The

project mainly attracted women, people involved with community development and education,

and reflective learners. The dominance of women in the project was unsurprising, given that

women in Australian rural communities are often very active in community development

activities and have been identified as early adopters of C&IT (The Rural Women and ICTs

Research Team, 1999). However, questions about the ‘gendered’ nature of the participatory

evaluation methodology arose during the project which may need to be taken into account in

other capacity building projects that aim to involve a diversity of participants.

McKie (2003: 320) argues that ‘Whether we like it or not evaluation has created a language

and modus operandi that can be excluding’. Developing effective strategies for including a

diversity of community members in participatory evaluations clearly requires that issues

25

related to the appropriateness of the language used and the perceived value and relevance of

participation and evaluation to various groups are addressed. Otherwise some participants may

see evaluation as a judgemental process that could have negative implications for their

employment or the funding of their projects, rather than an ongoing learning process that can

help to improve projects in ways that better meet community goals and needs. Our research

suggests that the promotion and use of practical and user friendly resources such as the

EvaluateIT kit could be a valuable means of demystifying participatory evaluation and PAR

and involving a broader diversity of community groups and stakeholders in evaluations.

Notes

1. This project was a collaboration between three QUT researchers (Dr June Lennie, Dr Roy

Lundin and Lyn Simpson) and senior staff of the Queensland Open Learning Network

(QOLN) and the Queensland Department of Employment, Training and Industrial

Relations. It was funded through a QUT/Industry Collaborative Research Grant.

2. QOLN is now known as Learning Network Queensland.

3. A significant contribution to the development of the EvaluateIT resource kit was provided

by a research project funded by the State Library of Queensland and QUT through a QUT

Strategic Links with Industry grant. The design of the EvaluateIT website was funded by a

QUT Community Service grant. I acknowledge and thank Dr Mary Hanrahan for her

valuable assistance in developing the EvaluateIT kit and conducting the community

consultations.

Acknowledgements

The LEARNERS project was funded and supported by the Australian Research Council, the

Commonwealth Department of Family and Community Services, Learning Network

Queensland, the Queensland Department of Natural Resources and Mines, the Office for

Women in the Queensland Department of Local Government, Planning, Sport and Recreation,

and Legal Aid Queensland. I acknowledge and thank the other Chief Investigators in the

project, Professor Greg Hearn and Associate Professor Lyn Simpson and the project’s

Research Assistants Kitty van Vuuren and Emma Kennedy da Silva. I also acknowledge and

26

thank the people from the Tara and Stanthorpe Shires and the collaborating partners who

participated in the project and provided valuable feedback and input.

References

Anderson, J. (1996) ‘Yes, but is it empowerment? Initiation, implementation and outcomes of

community action’, in B. Humphries (ed). Critical Perspectives on Empowerment, pp 69–83.

Birmingham: Venture Press.

Boyce, W. (2001) ‘Disadvantaged persons’ participation in heath promotion projects: Some

structural dimensions’, Social Science and Medicine 52: 1551-1564.

Boyle, R. and D. Lemaire, (eds.) (1999) Building Effective Evaluation Capacity. London:

Transaction Publishers.

Brunner, I. and A. Guzman (1989) ‘Participatory evaluation: A tool to assess projects and

empower people’, in R.F. Conner and M. Hendricks (eds.), International Innovations in

Evaluation and Methodology: New Directions for Program Evaluation, pp.9-17. San

Francisco: Jossey-Bass.

Chess, C. (2000) ‘Evaluating environmental public participation: Methodological questions’,

Journal of Environmental Planning and Management 43(6): 769-784.

Claridge, C. (1996) Women, Development and the Environment: A Method to Facilitate

Women’s Empowerment, PhD thesis, Brisbane: Department of Agriculture, The University of

Queensland.

Da Rin, J. and J. Groves (1999) Demand and supply of Internet content for Australian farm

businesses, Canberra: Rural Industries Research and Development Corporation.

Department of Communications, Information Technology and the Arts (DCITA). (2003)

Maintaining the viability of online access centres in regional, rural and remote Australia,

Discussion paper.

(http://www.dcita.gov.au/tel/regional,_ rural_and_remote_communications/?a=7789)

Deutchman, I. (1991) ‘The politics of empowerment’, Women and Politics 11(2): 1–18.

27

Diez, M.A. (2001) ‘The evaluation of regional innovation and cluster policies: Towards a

participatory approach’. European Planning Studies, 9 (7): 907-923.

Dugan, M.A. (1996) ‘Participatory and empowerment evaluation: Lessons learned in training

and technical assistance’, in D. Fetterman, S. Kaftarian and A. Wandersman (eds.)

Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability, pp.

277-303. Thousand Oaks, California: Sage.

Faris, R. (2001) Learning Communities: Villages, Neighbourhoods, Towns, Cities and Regions

Preparing for a Knowledge-Based Society. (http://members.shaw.ca/rfaris/docs/LCdigest.pdf)

Fetterman, D., S. Kaftarian, and A. Wandersman, eds (1996) Empowerment Evaluation.

Thousand Oaks: Sage.

Friedmann, J. (1992) Empowerment: The Politics of Alternative Development. Cambridge:

Blackwell.

Garaway, G.B. (1995) ‘Participatory evaluation’, Studies in Educational Evaluation 21 (1): 85-

102.

Gatenby, B. and M. Humphries (1996) ‘Feminist commitments in organisational

communication: Participatory action research as feminist praxis’, Australian Journal of

Communication 23 (2): 73–87.

Geiselhart, K. (2004) The Electronic Canary: Sustainability Solutions for Australian

Teleservice Centres, Wangaratta, Victoria: Centre for Online Regional Research.

(http://www.teleservices.net.au/CTSA_Viability_Report%20Final.pdf)

Gore, J. (1992) ‘What we can do for you! What can “we” do for “you”?: Struggling over

empowerment in critical and feminist pedagogy’, in C. Luke and J. Gore (eds) Feminisms and

Critical Pedagogy, pp.54–73. New York: Routledge.

Grace, M. and J. Lennie (1998) ‘Constructing and reconstructing rural women in Australia: the

politics of change, diversity and identity’, Sociologia Ruralis 38(3): 351–370.

Gregory, A. (2000) ‘Problematising participation. A critical review of approaches to

participation in evaluation theory’, Evaluation 6 (2): 179-199.

28

Groves, J. and J. Da Rin (1999a) Buying and Selling Online: The Opportunities for Electronic

Commerce for Australian Farm Businesses, Canberra: Rural Industries Research and

Development Corporation.

Groves, J. and J. Da Rin (1999b) Economic and Social Impacts of Farm Internet Use,

Canberra: Rural Industries Research and Development Corporation.

Guba, E. and Y. Lincoln (1989) Fourth Generation Evaluation. Newbury Park: Sage.

Harrison, L. (1998) ‘Using community learning to identify stores of social capital: Have we

found the “right” community?’, in Learning Communities, Regional Sustainability and the

Learning Society, Vol II. Launceston: Centre for Research and Learning.

Hearn, G., M. Kimber, J. Lennie and L. Simpson (2005) ‘ICTs and regional sustainability:

Critical reflections using action research’, Journal of Community Informatics, 1 (2): 18-31.

Hudson, H.E. (2001) ‘The Acacia programme: Developing evaluation and learning systems for

African Telecentres’, in C. Latchem and D. Walker (eds.) Telecentres: Case Studies and Key

Issues. Vancouver, Canada, The Commonwealth of Learning

(http://www.col.org/telecentres/chapter%2015.pdf).

Humphries, B. (1994) ‘Empowerment and social research: Elements for an analytic

framework’, in B. Humphries and C. Truman (eds) Re-Thinking Social Research. Anti-

Discriminatory Approaches in Research Methodology, pp.185–204. Aldershot: Avebury.

Kellogg Foundation (1998) Chapter 2. Evaluation Handbook.

(http://www.wkkf.org/Publications/evalhdbk/)

Khan, M. (1998) ‘Evaluation capacity building’, Evaluation 4 (3): 310-328.

Lather, P. (1991). Getting Smart. Feminist Research and Pedagogy With/in the Postmodern.

New York: Routledge.

Lennie, J. (2001) Troubling Empowerment: An Evaluation and Critique of a Feminist Action

Research Project Involving Rural Women and Interactive Communication Technologies. PhD

thesis. Brisbane: The Communication Centre, Faculty of Business, Queensland University of

Technology.

29

Lennie, J. (2002a) ‘Rural women’s empowerment in a communication technology project:

Some contradictory effects’, Rural Society 12 (3): 224-245.

Lennie, J. (2002b) ‘Including a diversity of rural women in a communication technology

access project: Taking the macro and micro contexts into account’, in G. Johanson, and L.

Stillman (eds.), Proceedings, Electronic Networks - Building Community: 5th Community

Networking Conference, Monash University, Melbourne, 3-5 July 2002.

Lennie, J., C. Hatcher and M. Morgan (2003) ‘Feminist discourses of (dis)empowerment in an

action research project involving rural women and communication technologies’, Action

Research, 1(1): 57-80.

Lennie, J. and Hearn, G. (2003) ‘The potential of PAR and participatory evaluation for

increasing the sustainability and success of community development initiatives using new

communication technologies’. Proceedings, Action Learning, Action Research & Process

Management and Participatory Action Research Congress, University of Pretoria, South

Africa, 21-24 September, 2003.

http://www.education.up.ac.za/alarpm/PRP_pdf/Lenny&Hearn.PDF)

Lennie, J., G. Hearn, L. Simpson, E. Kennedy da Silva, M. Kimber and M. Hanrahan (2004)

Building Community Capacity in Evaluating IT Projects: Outcomes of The LEARNERS

Project, Brisbane: Creative Industries Research and Applications Centre and Service

Leadership and Innovation Research Program, Queensland University of Technology.

(http://www.learners.qut.edu.au/reports.php)

Lennie, J., R. Lundin, and L. Simpson (2000a) Development of a Participatory Evaluation,

Assessment and Planning Framework for Sustainable Rural Community Development That

Uses Interactive Communication Technologies. Brisbane: The Communication Centre,

Queensland University of Technology.

Lennie, J., R. Lundin, and L. Simpson (2000b) A Participatory Evaluation, Assessment and

Planning Framework for Sustainable Rural Community Development That Uses Interactive

Communication Technologies. Brisbane: The Communication Centre, Queensland University

of Technology.

30

Maguire, P. (1996) ‘Proposing a more feminist participatory research: Knowing and being

embraced openly’, in K. de Koning and M. Martin (eds.) Participatory Research in Health.

Issues and Experience, pp.27–39. London and New Jersey: Zed Books.

Mannion, J. (1996) ‘Partnership, participation and capacity building: Rural development based

on local bottom-up strategies’, Leader Magazine 12, (October): 6-10.

Martin, M. (2000) ‘Critical education for participatory research’, in C. Truman, D. M. Mertens

and B. Humphries (eds.) Research and Inequality, pp.191-204. New York: UCL Press.

McKie, L. (2003) ‘Rhetorical spaces: Participation and pragmatism in the evaluation of

community health work’, Evaluation 9 (3): 307-324.

McTaggart, R. (1991) ‘Principles for participatory action research’, Adult Education Quarterly

41(3): 168-187.

Mobbs, R. (1998) ‘The self empowerment of communities: A systems strategy for rural

community assessment (SSRCA)’, in Learning Communities, Regional Sustainability and the

Learning Society. Vol II. Launceston: Centre for Research and Learning.

O’Meara, P., J. Chesters and G-S Han (2004) ‘Outside - looking in: Evaluating a community

capacity building project’, Rural Society, 14(2): 126-141.

O’Sullivan, J.M. and R.G. O’Sullivan (1998) ‘Evaluation voices: Promoting evaluation from

within programs through collaboration’, Evaluation and Program Planning 21(1): 21-29.

Papineau, D. and M. Kiely (1996) ‘Participatory evaluation in a community organization:

Fostering stakeholder empowerment and utilization’, Evaluation and Program Planning 19

(1): 79-93.

Rebien, C. (1996) ‘Participatory evaluation of development assistance’, Evaluation 2(2): 151-

171.

Stanley, L and S. Wise (1990) ‘Method, methodology and epistemology in feminist research

process’, in L. Stanley (ed.) Feminist Praxis. Research, Theory and Epistemology in Feminist

Sociology, pp.20-62. London: Routledge.

31

The Rural Women and ICTs Research Team (1999) The New Pioneers: Women in Rural

Queensland Collaboratively Exploring The Potential of Communication and Information

Technologies for Personal, Business and Community Development. Brisbane: The

Communication Centre, Queensland University of Technology.

Thomas, L. (2000) ‘Bums on seats; or “Listening to Voices”: Evaluating widening

participation initiatives using participatory action research’, Studies in Continuing Education

22 (1): 95-113.

Vanderplaat, M. (1995) ‘Beyond technique. Issues in evaluating for empowerment’,

Evaluation, 1 (1): 81-96.

Wadsworth, Y. (1997) Everyday Evaluation on the Run. St Leonards, New South Wales:

Allen and Unwin.

Wadsworth, Y. (1998) What is participatory action research?

(http://www.scu.edu.au/schools/gcm/ar/ari/p-ywadsworth98.html)

Woolcock, G., D. Renton and J. Cavaye (2004) What Makes Communities Tick? Local

Government & Social Capital Action Research Project. Ipswich, Queensland: Community

Service & Research Centre, The University of Queensland and Local Government Association of

Queensland. (http://www.uq.edu.au/csrc/lcandsc/final%20report.pdf)

Worthen, B., J. Sanders and J. Fitzpatrick (1997) Program Evaluation: Alternative Approaches

and Practical Guidelines. 2nd ed. New York: Longman.

32

Key goals/ outcomes

Partnerships/

collaborations/

cooperation

Enhanced

community

capacity

Long-term

sustainability

of C&IT

initiatives

Community

empowerment

Value new

forms of

leadership

Develop

learning

communities

Increased

community involvement,

ownership participation &

inclusion

Key methods

Participatory action

research and action

learning

Participatory

evaluation and

assessment of needs

and impacts

Ongoing, in-built

evaluation and

assessment processes

Whole of community

systems approach to

analysis and

planning

Analysis of differences

(gender, age, ethnicity,

ability etc.)

Process#

*

Analyse skills development and learning needs

Identify and engage

stakeholders

Assess evaluation

learning needs

Assess strengths, resources & information

needs

Share learnings and outcomes

Design programs and processes for

evaluation & assessment

Implement evaluation &

impact assessment

Begin learning sessions to use

LEARNERS process

Plan

Reflect Act

Observe

Review and redesign

processes and initiatives

Existing community strengths and resources

33

# All of these steps do not have to be followed and do not have to be undertaken in this sequence. They represent activities that could potentially be undertaken by stakeholders and participants. * Each of the steps involves a cycle of planning, acting, observing and reflecting

Figure 1: The LEARNERS Process

34

Figure 2: The framework of empowerment used to analyse the impacts of the

LEARNERS project (from Lennie, 2001)

Social empowerment

Political empowerment

Technological empowerment

Psychological empowerment

• Having a voice and being listened to • Participating in policy making • Taking action to change your life or your

community • Networking and lobbying • Changing stereotypes about rural women

• Knowledge about ICTs • Awareness and understanding about ICTs • Skills and competence in using new ICTs • Ongoing support and advice in using ICTs • Access to high quality technologies • Confidence to use and speak about ICTs

• Self confidence and self esteem • Feeling more valued and respected • Motivation, interest and enthusiasm • Freedom to do things or express yourself • Feelings of belonging • Wellbeing and happiness

• New knowledge and information • Awareness and understanding of issues • Skills, abilities and competence • Support, friendship and inspiration • Participating in group activities with women • Networking

35