Session S1G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S1G-1

Leading Change in the Internal Quality Assurance

System Implantation of a European Technical

University

Edmundo Tovar, Gonzalo Carrillo, José Domingo Carrillo Edmundo.tovar@upm.es, Gonzalo.carrillo@atosorigin.com, jcarrillo@fi.upm.es

Abstract - The Universidad Politécnica de Madrid’s

School of Computer Engineering is implementing its

internal quality assurance system from European

directives and other sources, quality policies, and its

underlying process map. One the main success critical

factor implementing these processes is to obtain a real

involvement of the different stakeholders. Change

management is a good practice used in organizations as a

way to provide a detailed road map, with caution signs

pointing out the potential dangers. This paper presents

the adaptation of a change management methodology to

technical universities and explains the road explored

implementing a key strategic process as the educational

program improvement review process. We show, as an

example, success results about of how a student workload

study is developed and used.

Index Terms – Quality Assurance, European Higher

Education Area, Change Management

INTRODUCTION

The Universidad Politécnica de Madrid’s School of

Computer Engineering, under the European Higher

Education Area and as part of the accreditation process of

new titles [1], has opted to develop its internal quality

assurance system (QAS). This process starts from

conformance with European directives and other sources

taken into account by the institution and end with the

drafting of quality policies and their underlying process map,

the design and implementation of these processes [2]. Thus,

university autonomy requires that each institution decides on

its standards in the context of its mission and goals [3].

One of the major risks to success in QAS

implementation lies beyond the proper design of procedures,

concretely in a real involvement of stakeholders. The

solution given in many organizations is leading change as a

way to provide a detailed road map, with caution signs

pointing out the potential dangers [4].

This paper presents an innovative approach to deal with

these issues, applying to this process a change management

methodology adapted to technical universities in the

implementation of QAS processes that ensures that those

most affected by the proposed changes are ready, motivated

and able to adapt their behavior to new requirements to

operate. The actions were aimed at ensuring that those

affected by the new processes were willing and ready for

adoption. Thus, we sought:

• Communicate to affected communities on the new

processes

• Involve key persons of the University, to serve as

change agents to disclose the change in the organization.

The process selected is a general procedure for

assessing students’ learning progress and outcomes in the

Internal QAS set out in the Educational programme

outcomes measurement and improvement process. This

process describes the mechanisms used by the Computer

Engineering School to assure the quality of each of the

building blocks of the educational programmes, including

the objectives of the degree, and the developed competences,

as well as to properly maintain and remodel its educational

offer, and approve, control and review the above

programmes and their outcomes.

This process is a clear example of implication to several

stakeholders. As part of one of the tasks of this process, the

Office of the Associate Dean for Quality and Strategic

Planning drafts a report containing the results for each

degree offered tuned to the diverse audiences interested in

tits results: Dean, Expert Committee on Degrees, department

managers, students, tutors, members of horizontal

commissions... Conclusions extracted from this experience

are exposed.

ASSESSING STUDENTS’ LEARNING PROGRESS AND

OUTCOMES PROCESS

Promoting and assuring the quality of Spanish higher

education institutions is an important part of higher

education policy, the goals of which include the

“measurement of the performance of the public higher

education system” and the “improvement of teaching and

research and the management of higher education

institutions”. Higher education institutions should set out to

improve the training of their graduates to assure that they are

capable of adapting to the demands of both society and the

scientific and technological system. The general procedure

for assessing students’ learning progress and outcomes in the

Internal Quality Assurance System in place at the

Universidad Politécnica de Madrid’s School of Computer

Session S1G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S1G-2

Engineering (SGIC-FIUPM) is set out in the Educational

programme outcomes measurement and improvement

process, PR/ES/2/003

This process describes the mechanisms used by the

Computer Engineering School to assure the quality of each

of the building blocks of the educational programmes,

including the objectives of the degree, and the developed

competences, as well as to properly maintain and remodel its

educational offer, and approve, control and review the above

programmes and their outcomes.

As part of one of the tasks of this process, the Office of

the Associate Dean for Quality and Strategic Planning drafts

a report containing the results for each degree offered by the

Computer Engineering school. This report is then submitted

to the Dean of the Computer Engineering school and the

Expert Committee on Degrees.

In view of the volume of data that need to be gathered,

the report is, in practice, composed of a series of studies.

One of these studies is called the “Student workload study”.

The purpose of this study is to serve as in instrument of

analysis, based on objective data, to help the higher

education community to assess the learning progress and

outcomes of students enrolled in the 2009/2010 academic

year in any first- or second-year subject of the degree of

Bachelor in Informatics Engineering taught by the

Universidad Politécnica de Madrid. This study sets out not

merely to list the aggregate student grades but also aims to

analyse the many factors that can have an impact on student

attainment.

This evaluation is based on facts and data, as opposed to

personal opinions or ratings. Its aim is to detect the strengths

and opportunities for improvement of the different subjects

taught at the Universidad Politécnica de Madrid’s Computer

Engineering and their scheduled activities. Note that this

process of evaluation does not conform to the notion of

control or inspection, but is an instrument that should be

used as a diagnostic tool.

Another aim of this study is to clarify how students

organize the workload of each subject that they are to take

and how they all rate the different educational activities

developed in each second-semester subject in the 2009/2010

academic. This way, the policy makers of the higher

education community can use the information gathered to

make the right decisions.

A change management approach applied to the

implementation of processes of a SQA in Informatics

Engineering degree. Change management is about

facilitating the process whereby individuals and

organizations can improve their performance through

effective engagement and commitment to delivering the

benefits of change ensuring successful project

implementation. It is equally important for all types of

transformation, whether driven by process, IT or strategy.

Change management should not be considered an isolated

activity but more a series of embedded and inter-related

activities that support all aspects of the human elements of

change from short-term transition through to long-term

sustainable transformation.

Then the reason why it is so important for this

institution to manage changing is because increasingly

competitive environment change is no longer an option but a

imperative. The rate at which organizations need to react to

events - both internally and externally - is increasing, and the

time that we have to adapt to change is decreasing. This

means to design, build and roll out projects as quickly and

cost effectively as possible, with maximum commitment

from those affected by the change. The only way to achieve

this is to ensure that the human elements of change are

addressed well as part of the process.

The business now - the current position

There is a high organisational commitment to the current

state – the status quo. In order for people to change the way

they work they must understand that the cost of remaining in

the present state is too high, not only for the organisation but

also for them personally. This is the business imperative for

change. Understanding brings motivation to discover what

change really means.

The business future - improved organization performance.

The future performance expectations must be clear and

desirable if people are to be motivated. The communication

strategies and plans for the project need to be tailored to the

different groups, to show them clearly what they can gain

from it personally and organisationally.

Engagement - facilitating the change.

In order to maximise levels of success, it is important to

build understanding, skills and motivation at all levels to

deliver the changes. It is important for people to experience

in real life what the change will mean. By tailoring

engagement programmes to the needs, culture and

communications styles of each major stakeholder group, we

ensure that people gain the knowledge of change that will

result in positive action and ultimately in improved business

performance.

Thus, the successful implementation of Quality Assurance

Systems depends not only on the efficiency of the processes

design, but primarily on the involvement and conviction of

its actors [5]. In the event that this paper presents, the

application of a methodology for the period October 2009 to

July 2010 for the implementation of the processes associated

with the new integrated management system of the quality of

Informatics Faculty Polytechnic University. In this first

phase the following processes have been considered:

Time Record by the students

Maintenance Management

Incident Management

The actions were aimed at ensuring that those affected

by the new processes were willing and ready for adoption.

Thus, we sought [6]:

Session S1G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S1G-3

Communicate to affected communities on the new

processes

Involve key persons of the University, to serve as

change agents to disclose the change in the

organization.

Change management is the process that ensures that

those most affected by the proposed changes are ready,

motivated and able to adapt their behavior to new

requirements to operate [7].

For the preparation of this project has been used change

management methodology that follows (Figure 1):

FIGURE 1. PHASES OF THE METHODOLOGY APPLIED

Phase I. Change Analysis

This phase is to realize the necessity and scope of

change and to analyze the impact on the groups involved. It

is built with the participation of areas who lead the initiative.

It is used as a pillar in the dissemination of key messages to

all affected groups.

For the change management of the process identified,

this phase culminated in:

The identification of the affected groups,

distinguishing between those with influence to

change and those who will be impacted (user

processes). We identified the following groups:

- Students. As main users affected by the

registration of hours

- Teachers. As recipients of information from the

Time Record process. Additionally, they were

also identified as a collective interest in their

influence on students.

- Directors of Departments. As influential staff

to communicate the proposed changes among

the group of teachers

- Delegation of students. As a group to influence

and communicate the change on students.

- Personnel Administration. As a user group of

the processes of maintenance management and

incidents.

- Dean of quality, as sponsor and responsible for

the initiative

- Administration Manager, responsible for the

implementation of the administrative processes

of the center (in this case, incident

management and maintenance)

- Committee for horizontal coordination, as the

recipient of information from the Time Record

process.

Thus, from this collective identification, the

following matrix was constructed. It served as

guide for the identification and implementation

of the actions to include in the change

management plan (Table I)

TABLE I

STAKEHOLDERS MATRIX

Students

Personnel administration

Delegation of students

Administration Manager

Teachers

Secretary of Students

Directors of Departments

Committee for horizontal

coordination Dean of quality

Identifying the level of involvement required for each of

the groups, for the successful implementation and the

associated risks (Figure 2).

FIGURE 2. STAKEHOLDERS IMPLICATION DEGREE

The main risk identified was the failure to use the time

recording process by the students, since it meant a new

operation previously undeveloped. So the awareness and

communication effort should be higher.

Identification of the case for change. The case for change

articulated the key messages to convey to implement the

desired transformation and puts into context the reasons why

do we need change. A case for change was articulated to

implement the new processes, through the implementation of

the quality plan that the Computer Engineering School is

developing as a result of new curricula related to Bolonia.

Additionally, for the actual process of the Time Record, it

was associated with the new model of horizontal

competencies, where planning and time management is a

specific skill to be developed by students So that students

understand why this new process was required.

Phase II. Planning and implementation of change

This phase includes the actions to take in terms of

communication, training and involvement actions to deliver

IV. Control and

monitoringIII. Change

implementation

• Current and future situation

analysis

• Impact analysis:

• Stakeholders

impacted

• Risk assessment

• Change network design

• Communication Plan

• Training Plan

• Kick-off and deployment

• Communications events

• Training sessions

• Metrics definition

• Change monitoring

• Users acceptance

PH

AS

ES

TA

SK

S

II. Change planningI. Change Analysis

AWARENESS

UNDERSTANDING

SUPPORT

INVOLVEMENT

COMMITMENT

Quadrant 1 Quadrant 2 Quadrant 3 Quadrant 4

Keep

informed

Address

concerns

Partial

involvement

Active full

time involvement

Change

impact

Change Influence

Session S1G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S1G-4

with the various groups affected by the change.

Additionally, it defines the responsibility and resources for

the implementation

Development of the network change. Identified those

who should take a key role in the development of

change activities, assuming the roles and

responsibilities:

- Sponsor of change. He leads, promotes and

supports the change. Set posts and guidelines. In

this case, the sponsor of the change was the Dean of

Quality and Strategic Planning of the University.

- Project leaders: knowing the type of change

proposed, the current and future operational model.

They help to the operational implementation of

change. This role was assumed by the School

Administrator.

- Change agents: opinion makers, people of reference

for the target group for change. They are the vehicle

for the transmission of change. It was decided to

formalize the Director of Department of Software

and Systems and Software Engineering as change

agents for the group of teachers and other

department heads, and the Student Representative

as change agents for the group of students.

- Additionally, we had an outside consultant, an

expert in organizational change management, to

guide the working group.

A monthly monitoring committee was defined to

maintain this change network, - composed of by the

people indicated – This committee defined and planned

the actions to be taken, and evaluated the results.

Communication actions. The communication actions

to be performed for each group were defined based on

the "ladder " of involvement. Thus, we defined the

following awareness initiatives:

- Signage on the new Quality Management Program

and the time record process. We used the bulletin

boards and displays to raise awareness of the new

quality management system and, specifically, the

process of recording the hours

- Publication on the intranet about the information to

proceed with the the new time record process

- Sending individual letters to students explaining the

new registration process for hours and its link with

the assessment of horizontal competencies.

- Outreach to teachers of information on the new

processes.

The actions of understanding included specific sessions

with students to explain the system of integrated quality

management, the new registration process for hours and

its relation to the competency assessment.

The actions of involvement and to reach commitment:

- Monthly monitoring of the management committee

with the identified change agents, to gather

"feedback" about the change process among groups

represented (students + teachers).

And finally, in terms of training, any specific action was

needed, beyond the production of manuals to use the

tool.

Phase III. Control and monitoring

Definition of KPIs of to measure and follow-up

the change implementation, and to determine

the success of the implementation of the new

model from different perspectives.

The indicators were defined in terms of users

of the new processes and the degree of

compliance with the information requested in

the process.

These indicators were consolidated into annual

reports, and distributed internally within the

School.

RESULTS OBTAINED. STUDY TARGETING SUBJECT

COORDINATORS: EXAMPLE OF KEY TASKS IN A STRATEGIC

PROCESS OF THIS INSTITUTION

As result of the implementation of this process, and

according the methodology applied, two kind of results have

been obtained:

1. Data Exploitation for FACULTY. Educational Program

Outcomes Measurement Process: Student workload study

The study aims to analyse the overall outcomes attained

by students in each subject for which they enrolled, placing

the emphasis on aspects such as a comparison between the

expected workload in hours set out in the subject learning

guides and the actual workload in hours recorded by students

for each subject; a comparison between the hours of work

planned by students and the actual hours employed; a rating

by students of each analysed subject and, finally, an analysis

of the reasons given by students for not completing the

weekly activities schedule or failing to attend lectures.

Finally, an analysis of student attainment fitted to the model

developed by the Office of the Associate Dean for Quality

and Strategic Planning is shown. We conclude with a short

graphic report on the key aspects of each subject and the

educational activities they each contain.

A. Preparation of the first report: population and

participation level

The study was completed for the first time with bachelor

of informatics engineering students. This degree has been

imparted as of the 2009/2010 academic year. The study

population was composed of second-semester students

enrolled in the 10II degree programme. As regards

participation, note that the percentage cooperation by weeks

ranged from 11% in the last week of the semester, mainly

explained by the commencement of the examination period,

Session S1G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S1G-5

16 1718 20 20 21 21 21 21 22 22 22 22 22 22 22 23

2426

29 29

0

10

20

30

40

AD

MIN

ISTR

AC

ION

Y G

ESTI

ON

DE

EMP

RES

AS

CO

NC

UR

REN

CIA

PR

OG

RA

MA

CIO

N II

BA

SES

DE

DA

TOS

PR

OB

AB

ILID

AD

ES Y

EST

AD

ISTI

CA

I

CA

LCU

LO

PR

OG

RA

MA

CIO

N P

AR

A S

ISTE

MA

S

ESTR

UC

TUR

A D

E C

OM

PU

TAD

OR

ES

RED

ES D

E C

OM

PU

TAD

OR

ES

SIST

EMA

S D

IGIT

ALE

S

SIST

EMA

S O

PER

ATI

VO

S

PR

OB

AB

ILID

AD

ES Y

EST

AD

ISTI

CA

II

ALG

OR

ITM

ICA

NU

MER

ICA

LOG

ICA

AR

QU

ITEC

TUR

A D

E C

OM

PU

TAD

OR

ES

MA

TEM

ATI

CA

DIS

CR

ETA

I

ALG

EBR

A L

INEA

L

FUN

DA

MEN

TOS

FISI

CO

S Y

TE

CN

OLO

GIC

OS

DE

LA IN

FOR

MA

TIC

A

ALG

OR

ITM

OS

Y E

STR

UC

TUR

A D

E D

ATO

S

LEN

GU

AJE

S FO

RM

ALE

S, A

UTO

MA

TAS

Y C

OM

PU

TAB

ILID

AD

PR

OG

RA

MA

CIO

N I

to 50%, reached in week 13 (from 03/05/2010 to

09/05/2010) (Figure 3).

FIGURE 3. PERCENTAGE PARTICIPATION BY WEEKS (SOURCE:

DATA SUPPLIED BY THE COMPUTER CENTRE)

A total of 281 students of the 412 enrolled in the

2009/2010 academic year participated throughout the 20

week-period amounted to.

To be exact, the report contains the following sections

related to the overall outcomes of the degree course:

1) Comparison of the expected workload and actual

workload: One of the key issues to be analysed is the

comparison between the hours that students recorded as

having employed to acquire the competences required

by each subject with the hours scheduled in the learning

guides for each subject. In most subjects, specifically all

but five, the average time that students record as having

spent on the subject is greater than the semester

schedule planned in the learning guides.

2) Comparison of actual workload and workload planned

by students themselves: Planning is a methodical and

reasoned decision made in the present with the definite

aim of steering the future in the desired direction. Based

on this definition, this section aims to analyse the

students’ ability to work effectively as an individual,

organizing and planning their own work, independently

or as a member of a team, making arrangements, plans

and decisions, negotiating and solving conflicts,

entering into relations and adopting a critical and self-

critical attitude.

3) Reasons given for not completing scheduled activities

and not attending lectures: A noteworthy factor be

analysed to explore the failure to meet the planned

schedule is to examine the main reasons given by

students for not completing the scheduled activities or

not attending lectures. According to data supplied by the

students themselves, over 25% of students participating

in the study confess to not having completed a

scheduled activity or not having attended lectures in

every single week of the second semester analysed

(weeks 19 to 20 were omitted from the analysis because

there were few entries due to the imminent

commencement of the June examination period).

4) Overall rating by subject (scale 0-40). A new question

was added to the questionnaire of the last survey

administered at the end of the term. This new item

aimed to have the students rate, on a scale of 1 to 10, the

extent to which each scheduled educational activity

helped them to achieve the planned learning outcomes,

where 0 indicates not at all helpful and 10 extremely

helpful. Figure 5 shows the rating by subject. This is the

sum of the average ratings for the respective educational

activities, taking into account only the average ratings

of “theory classes”, “individual independent study and

work”, “practical classes” and “individual and group

practical assignments”, as they are the activities

common to all subjects. Therefore, the maximum rating

per subject would, in theory, be 40 points.

FIGURE 4. OVERALL STUDENT RATING BY SUBJECT (SOURCE:

DATA SUPPLIED BY COMPUTER CENTRE)

5) Overall rating by activity (scale 0-10). Another point

that we considered worth analysing was the overall

student rating of educational activities, that is, the sum

of the average scores given by students to each

educational activity in the different subjects to discover

if any educational activity at the Computer Engineering

School has any particular defect. This analysis accounts

for the average score given by students over the sum of

average scores by educational activity for analysed

subjects that have a score for the activity/subject

combination in question. We assume that this analysis

has weaknesses. For example, some students may have

made the mistake of rating an activity that does not in a

particular subject. This significantly lowers the sum of

the average ratings of the activity in question. Even so,

we consider that this information is worth exhibiting.

As Figure 4 shows, the Seminars/Workshops

educational activity receives a significantly lower rating

than the others. Another point to be taken into account

is that the student ratings of whether or not the activities

were helpful for achieving the scheduled learning

outcomes in Tutorials, Projects and Laboratory Work

are far from optimal. On the other hand, let us stress that

students give a high score to the Individual Independent

Study and Work educational activity.

Session S1G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S1G-6

FIGURE 5. AVERAGE SCORE OVER SUM OF AVERAGE RATINGS FOR EACH EDUCATIONAL ACTIVITY ACROSS ALL SUBJECTS.

6) Monitoring of the success rate and attainment rate

indicators for both each first- and second-semester

subject with the aim of getting a snapshot of student

attainment in each subject. Monitoring is major factor

considering that, on page 187, the Bachelor in

Informatics Engineering report submitted to ANECA

gave an estimate of the percentage of students that

would pass the whole first year in one year, as well as

the ratio between the number of credits fresher students

enrolled for and the number of curriculum credits

multiplied by the number of fresher students that

enrolled for any of the first-year subjects. Conformance

to the above estimates will be a key justification for the

future accreditation of the degree.

2. Data exploitation for STUDENTS: Planning and self

organization competence.

Individual report to students have been elaborated and

submitted reporting several objectives about data related to

metrics with respect to:

Verifying the student activity

Verifying if the student take corrective actions

Verifying the student efficiency managing the

time

Assessing the effort applied.

CONCLUSION

The application of leading change as a way to provide a

detailed road map, with caution signs pointing out the

potential dangers has been determinant to avoid difficulties

in the implementation of the first processes of the QAS

implemented in our school. A change management

methodology has ensured that those stakeholders most

affected by the proposed changes are ready, motivated and

able to adapt their behavior to new requirements to operate.

The participation of an outside consultant, an expert in

organizational change management, has been key to guide

the working group and to learn how to face to the futures

processes implementation.

REFERENCES

[1] ANECA. “Directrices, definición y documentación de Sistemas de

Garantía Interna de Calidad de la formación universitaria”. Documento 02. PROGRAMA AUDIT. www.aneca.es. 2007

[2] E. Tovar, J. Carrillo, Creating transparency for mutual recognition in

technical teachings through Internal Quality Assurance Systems, 38th ASEE/IEEE Frontiers in Education Conference, ISBN 978-1-4244-

1970-8/08, October 22 – 25, 2008, Saratoga Springs, NY.

[3] L. Wilson, Common instruments for assessment and accreditation in Europe, Seminar on Methodological common instruments for

assessment and accreditation in the European framework, Santander,

July 28th - 30th 2004.

[4] Ford, Jeffrey D.; Ford, Laurie W.; & McNamara, Randall T. 2002

Resistance and the background conversations of change. Journal of

Organizational Change Management; 2002, Vol. 15 Issue 2, p105, [Online]. Available from:

http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&

db=buh&AN=6881523&site=eds-live&scope=site Accessed: 20 January 2011

[5] Leppitt, N. 2006-1 Challenging the code of change: Part 1. Praxis

does not make perfect. Journal of Change Management, Volume 6, Issue 2 June 2006 , pages 121 – 142 . DOI:

10.1080/14697010600683104. [Online]. Available from:

http://www.informaworld.com.ezproxy.liv.ac.uk/smpp/ftinterface~content=a748001674~fulltext=713240930~frm=section Accessed: 7

February 2011

[6] Sirkin, Harold;Keenan, Perry;Jackson, Alan; 2005. THE HARD SIDE OF CHANGE MANAGEMENT. Harvard Business Review; Oct2005,

Vol. 83 Issue 10, p108-118, 11p

[7] Stensaker, Inger; Falkenberg, Joyce; and Grønhaug, Kjell; 2008. Implementation Activities and Organizational Sensemaking. Journal

of Applied Behavioral Science 2008 44: 162 originally published

online 5 March 2008 DOI: 10.1177/0021886307313794

AUTHOR INFORMATION

Edmundo Tovar Caro, FIE International Co-Chair, Europe,

Universidad Politécnica de Madrid, etovar@fi.upm.es

Gonzalo Carrillo, Atos Consulting Director, in charge of

the Transformation & Business Innovation line, gonzalo.

carrillo@atosorigin.com

Jose Domingo Carrillo, Languages and Systems and

Software Engineering Department, Universidad Politécnica

de Madrid, jcarrillo@upm.es

5.4

2.7

6.1

4.7

3.3 3.9

4,9 5.6

3.7

0

1

2

3

4

5

6

7

8

9

10

Le

ctu

res

Seminar…

Individua…

Practical…

Tu

torin

g

Laborato…

Group…

Assignm…

Pro

jects