Presentation to the Library Assessment Conference

August 5, 2008Seattle, Washington

Presenter: Dr. John B. Harer, East Carolina UniversityConsultant to Presenter: Dr. Shannon Gibson, College of Business, ECU

• Traditional: Job Satisfaction Surveys:• Measures of motivator’s and hygiene factors

(Herzberg) and work issues relevant to satisfaction• More currently: Organizational climate and

culture surveys• Definition: Some sources indicate it is difficult to

define. Can be defined as “the process of quantifying the culture of an organization” (Reichers and Schneider, 1990)

• Schneider (1994) “…(the need for climate and culture surveys) is caused by the seeming failure of both business educators…and business managers to appreciate the fact that organization members interpret all that happens to and around them.”

• ClimateQual™, formerly Organizational Climate and Diversity Assessment (OCDA):• “…a means of collecting information about staff

perceptions about how well the Libraries were doing in achieving the climate for diversity and organizational health.” (www.lib.umd.edu/ocda/)

• This current study discovered other institutional culture and climate surveys:– University of Arizona– SUNY Buffalo– Florida State University– University of Washington

This study is grounded in two theoretical approaches: Theoretical Approach #1◦ Employee assessment helps the

organization achieve its goals: “In order to compete effectively, organizations need to recruit and retain good employees, have their employees work hard on behalf of the organization, and draw on the experiences and expertise of employees to move the organization forward. Research has shown that job satisfaction plays an important role in maintaining well-staffed, vital, and healthy organizations that contribute to the bottom line as well as the personal well-being of employees” (Amplitude Research, 2005).

• Continuous Quality Improvement (CQI) principle that employees are customers and customers judge quality:

◦ What makes top performing companies different is their organisational arrangements. Specifically, they are better organised to meet the needs of their people, so that they attract better people than their competitors do and their people are more greatly motivated to do a superior job, whatever it is they do. They are better organised to meet the needs of customers so that they are either more innovative in anticipating customer needs, more reliable in meeting customer expectations, better able to deliver their product or service more cheaply, or some combination of the above”. Robert Waterman, 1994, p. 48 (co-author, In Search of Excellence, with Tom Peters)

LibQual+® stated purpose:◦ “assess and improve library services, change

organizational culture, and market the library”. Cook and Heath (2001).

◦ “Service marketing has identified the customer or user as the most critical voice in assessing service quality” (p. 549).

The Homer Simpson Effect

What is the Primary Goal of an Organization: ☺ Improved production? Productivity : “The

rate at which goods or services are produced especially output per unit of labor.”

☺ Satisfied employees? ☺ High quality library goods and services?

Quality: “the degree or grade of excellence”

LibQual+®: Quality of library services from the external customer perspective Three dimensions of measurement:

Affect of Service Information Control Library as Place

This assessment links its results to quality improvement.

Motivators and hygiene factors:◦ Pay, benefits, contingent rewards, promotion◦ Work relationships: management and peers◦ Work processes: policies, procedures, communication, nature of

the work This assessment links its results to employee

satisfaction improvement “For many years, organizational behaviorists have

debated and analyzed the elements affecting job performance. A review of their results leads to the conclusion that there is a connection between work-related attitudes and performance.” (Siggins, 1992, p. 302)

ClimateQual™ measures.◦ job satisfaction◦ fair treatment ◦ relationship or task conflict◦ continuous learning◦ managerial practices◦ ethnic or gender harassment

This assessment links its results to improving the conditions of the workplace, employee satisfaction, diversity and safety.

Maslow’s Hierarchy of Needs

The Volvo Corporation example

Physiological

Safety

Love/belonging

Self esteem

SA

The focus on external customer assessment suggests that the internal customer (employee) has less to do about quality assessment

Quality judgment of the external customer: ◦ The finished product only (automobile story here)◦ Some external customers are sophisticated users and

some aren’t Quality judgment of the internal customer

◦ Knowledge of input quality (quality of input products, process variables, supplier relations, process flow

◦ Knowledge of work process, at least their small part of it: ◦ Employees are customers of the library’s goods and

services too.

Daugherty (2002): Library staffs are much more knowledgeable than is often appreciated. The true wisdom embedded in an organization will emerge from its staff when they all share the same base of information and are thus able to see the whole picture. When information is shared, it is the staff who will find the best answers for most of their own challenges.”

Heath (2002): “When Professor Parasuraman addresses this topic, he observes that there must be a dozen different ways of listening to or evaluating an organization. It is important to realize that LibQual+™ only addresses one or two of those. I recommend that you take a look at his books…to get a feel for the many other ways, such as focus groups, employee surveys (emphasis this author’s), and the like.”

Nitecki (2001): “Conceiving service quality as existing solely in the eye of the beholder – the recipient of the library services – also has its shortcomings…Service quality goes beyond satisfying customers, it reflects the interactive relationship between the library and the recipients of its services and should not be viewed from one perspective.”

Assessment of Quality by External Customer

Input Work Process Output

External customer judges quality of output

Assessment of Quality by Internal Customer

Input Work Process Output

Internal customer judges quality of input, process and output

Purpose: Two facets:◦ to discover if current practices in employee

assessment in academic libraries addressed employees’ perceptions of quality and quality improvement

◦ To discover what metrics exist and begin to develop those metrics for library employee assessment of quality and quality improvement

Gathered existing instruments of employee assessment:◦ Academic libraries that are members of ARL

Distinct set of institutions for research question Large enough to expect existing documents in use 113 total

Letter of request mailed:◦ Asked for documents that met this criteria and

suggested four types: Job satisfaction surveys Employee exit interview forms Employee self-assessment forms (typically used in performance evaluation

process Dean/Director/Management surveys

Institutional human resources manager identified ◦ Letter addressed HR manager directly◦ SASE envelope provided for return◦ Check-off provided to indicate that no such documents were

used or available Response Rate:

◦ 30 institutions; 26.5% Documents received:◈ One employee satisfaction survey ◈ Four organizational climate surveys◈ Eight Exit interview questionnaires ◈ Eleven employee self-assessments◈ Three Supervisor/Director/Dean evaluations distributed to employees◈ Thirteen Letters returned indicating that such assessment documents were

used or available

Two widely accepted Continuous Quality Improvement (CQI) publications were used to inform the analysis:◦ Malcolm Baldrige National Quality Award for

Education (MBNQA) criteria◦ The Certified Quality Manager Handbook, edited

by Duke Okes and Russell T. Westcott Used to inform the content analysis by

providing directions for metrics and language of quality and quality improvement

MBNQA Criteria: Of seven criteria categories, Category 5: Workforce Focus was most relevant◦ Two sub-categories:

Workforce Enrichment Workforce Environment

◦ Workforce enrichment criteria included Workforce engagement Workforce and leader development Assessment of workforce engagement

◦ The MBNQA criteria emphasizes quality improvement; how organizations plan for and conduct quality improvement

Workforce engagement items◦ How do you determine the key factors of workforce engagement◦ How do you foster an organizational culture conducive to high performance and a

motivated workforce to accomplish the following: Cooperation, effective communication and skill sharing among all faculty and

staff Effective information flow and two-way communication Individual goal setting, empowerment, and initiative Innovation in the work environment The ability to benefit from diverse ideas, cultures, and thinking of your workforce

◦ How does your workforce performance management system support high-performance work and workforce engagement?

How does your workforce performance management system support high-performance work and workforce engagement by addressing:◦ core competencies, strategic challenges, and accomplishment of your action plans◦ organizational performance improvement, technological change and innovation◦ the reinforcement of new skills and knowledge on the job

Central to what Duke and Oakes consider important for employee assessment is also important to employee empowerment: “One of the core components of quality management is that of having everyone in the organization involved in managing and improving quality.” (2001, p. 29)

More specifically, address:◦ How well does the company’s culture fit with its strategy◦ Are skills and rewards appropriate for the industry and strategic direction◦ Are management systems properly aligned to the espoused vision, mission, and values◦ Do employees have the authority needed to make decisions for which they are held

responsible◦ Do employees understand the company strategy and are they committed to the

changes necessary◦ Is there sufficient trust and communication to enable the organization to work

effectively◦ Was past strategy implementation seen as appropriately deployed (2001, p. 66)

The study PIs also theorized:◦ Language, directly or indirectly, rating the quality of

inputs◦ Language rating quality in the production process◦ Language to improve quality of a given process, e.g.:

Are measures for improving inputs (and production processes) in place

Are employees’ judgment of inputs (and production processes) gathered effectively?

Are employees judgment of inputs (and production processes) respected by superiors?

◦ Measures that allowed employees to rate the quality of actual products and services completed

As noted above, the CQI resources guided the review of the measures in the documents gathered

Examined each item on these documents for content addressing:◦ specific work roles or responsibilities◦ the process of producing goods or services◦ work flow descriptions◦ provisions for gauging and/or improving work

responsibilities◦ performance and/or procedures in work

responsibilities

Language that mirrored the production model, i.e. input, process, output

Words or phrases describing suppliers and their role, resources and materials, process and flow of input

Words or phrases describing the specific role and its responsibilities including:◦ Design◦ flow between units in the process◦ cross-functional responsibilities or connections◦ Resources◦ volume and complexity of the work load

Additionally, quality words and concepts ◦ Skills and skill sharing Flexibility Rapid Response◦ Complexity of learning work responsibilities Strategy Implementation◦ Authority to make decisions within their role Respect for innovation ◦ Methods in place for improving work performed

Examined the Four Organizational Culture and Climate Surveys◦ Every item from these four documents

individually placed on a post card◦ Each card read and assigned a label

Label coincided with terminology identified from examination of the CQI sources

Three iterations of reading and categorizing each card

Iterations allowed changes to: Eliminate duplicate or nearly duplicate labels Ensure the labels were relevant to the terminology

Seventy items were deemed appropriate content:◦ 32 from the University of Arizona◦ 20 from the Florida State University◦ 16 from SUNY Buffalo◦ 2 from the University of Washington

Only the SUNY Buffalo survey asked employees about the quality of results (output).◦ survey contained twenty-one questions, including:

quality ratings about the library’s collections quality ratings about the library’s services, such as

ILL

Theme or category Number of Items

1. Sharing skills or knowledge 42.Knowledge, information, expectations needed for specific job

4

3. Workload 13 a. appropriate and fair amount b. obstacles to workload c. Value of workload assigned4. Support for performing specific job 17 a. adequate equipment b. Tech support c. resources needed to complete the job’s duties d. Administrative support 5. Alignment of job to library’s vision, mission, strategies

3

6. Training for specific skills needed for the job 77. Work with cross-functional, teams, departments, units

8

8. Employee advice, input, suggestions, complaints respected

9

9. Flow in a work process 310. Striking a balance between job and personal life 2

Results

It has been posited that:◦ Customers judge quality◦ Employees are customers who are capable of and should

judge quality and quality improvement ◦ Employee judgments of quality and quality improvement

Are linked to employee satisfaction through the higher order of needs of self-esteem and self-actualization

Are measures of the employees needs and expectations Whose results are directly linked to quality improvement

Results of the analysis of the documents gathered:◦ Show that measures of quality do exist in some form◦ That further research is needed

Contact information:◦ Dr. John B. Harer◦ Assistant Professor of Library Science◦ Department of Library Science◦ East Carolina University◦ Greenville, NC 27858◦ (252) 328-4389 - work◦ (252) 328-4368 – Fax◦ harerj@ecu.edu - email