1  

ISBN 978-988-16168-2-1

Working Paper No. 10

The Chinese University of Hong Kong The Development of Critical Thinking Skills as a Graduate Attribute for Undergraduate and Postgraduate Students: A Quantitative and Qualitative Approach Keith Thomas Beatrice Lok Carmel McNaught Centre for Learning Enhancement And Research

Funded by University Grants Committee as a Teaching Development Grant, 2009–12

2011

Citation: Thomas, K., Lok, B., & McNaught. (2011). The development of critical-thinking skills as a graduate attribute for undergraduate and postgraduate students: A quantitative and qualitative approach. Working Paper 10. Hong Kong: Centre for Learning Enhancement And Research, The Chinese University of Hong Kong.

2  

Contents

Section Page Introduction 3 Brief review of literature 3 Objectives 4 Methodology 5 CT defined 6 CT – The Delphi report 7Assessing CT 8 Utility of CT tests 9 Selected tests 10 Comparison of selected tests 10 CT test study method 11 Findings 13

Objective 1: Identifying what CT means to students, teachers and alumni 13 Disciplinary traits

Objective 2: Evidence of achievement of outcomes in terms of how CT is assessed and reported

16

Objective 3: Aligning learning outcomes and practices in case studies of best practice by academic discipline

20

Conclusions 22 References 23 Appendix 1: Sample cases illustrating evidence of CT 27 Appendix 2: Interview questions (Academic staff) 28 Appendix 3: Interview questions (Students) 29 Figure 1: CT constructs conceptualised 13 Figure 2: CUHK students’ disposition towards CT 16 Figure 3: CUHK students’ CT skills 17 Figure 4: CUHK Ug students’ disposition towards CT for different Faculties 17 Figure 5: CUHK Ug students’ CT skills for different Faculties 18 Figure 6: The overall performance of CUHK Pg students towards CT disposition 18 Figure 7: CUHK Masters students’ CT skills 19 Figure 8: CUHK PhD students’ CT skills on CCT-G835 19 Table 1: Brief information about the tests 6 Table 2: CT defined 6 Table 3: Summary of skills and disposition 7 Table 4: CT assessment tests and design features 8 Table 5: A brief summary of different CT tests 8 Table 6: Factual comparison of the selected tests 11 Table 7: Summary of data-collection methods and purposes for test selection 12 Table 8: Disciplinary traits 14 Table 9: Comparison of teacher and student conceptions 15 Table 10: Disciplinary groupings 16 Table 11: Good practices by disciplinary groups 20 Table 12: Improvement (development) strategies by disciplinary groups 21

3  

Introduction Critical thinking (CT) is a much-discussed and written-about attribute; however, there is no one agreed definition of CT (Gibbs & Simpson, 2004). Nonetheless, given the widespread focus on CT, there is a pressing need for some conceptual clarity (Green, Hammer, & Star, 2009). In general, it is a term used to describe thinking that is reasoned, purposeful and goal-directed; the kind of thinking that is involved in solving problems, formulating inferences and making decisions in a particular context (Halpern, 2003). It is seen as a necessary thinking skill for graduates in the 21st century and many universities in the United States and other countries require students to take a course in CT as part of their general education programme (Halpern, 1998). In an academic context, there is a broad assumption that CT includes identifying, interpreting, analysing and evaluating arguments and the word ‘critical’ appears frequently in descriptors of undergraduate (Ug) and postgraduate (Pg) courses and learning outcomes. For example, ‘a student is required to demonstrate a critical understanding of principles and subject matter’, and ‘evaluate critically methodologies and current research’ (Moore, Faltin, & Wright, 2003).

There have been a number of efforts to understand how CT can be embedded in degree programmes (MacPherson, 1999; Tapper, 2004). Some issues identified include concerns related to the effect on teaching delivery (Thompson & Rebeschi, 1999; Angeli & Valanides, 2009), and how CT can be reflected in the quality of professional decision-making (Baron, 2000; Hastie & Dawes, 2001). More pertinently, as Moore et al. (2003) suggested, definitions of CT may be linked to cultural and political assumptions and values, and thus it may also be reasonable to ask if these models are the most appropriate modes of thinking in an international context. Bearing this ambiguity and related issues in mind, this project set out to explore the development of CT skills as a graduate attribute at The Chinese University of Hong Kong (CUHK). The wider context behind the imperative to develop this skill is the reality of unprecedented change in today’s information- and knowledge-based workplace, which means graduates need to be aware, be able to think on their feet, question their practice and apply concepts flexibly as they face daily challenges. Brief review of literature CT as a cognitive skill has a long tradition (Fisher, 2001). However, despite this long tradition and the recent wide discussion in education and education reform (Moore, 2011), there is no clear agreement on the concept (McMillian, 1987; Cheung, Rudowicz, Kwan, & Yue, 2002). Described as what Whitehead (1929) called the ability to ‘think critically’ (cited in Ramsden, 1992) or more simply as Dewey (1916) said ‘[displaying] good habits of thinking’ (cited in Davies, 2011), CT is in Williams’s (1976) words ‘a most difficult’ term (cited in Moore, 2011). Some also argue that the concept seems to have a much narrower focus than when it was conceived earlier (Davies, 2011). Nonetheless, CT is accepted as a most important generic competency in both high-school and tertiary curricula worldwide (Ku, 2009). A second issue relates to disposition. As Ennis (1996) and others reported, CT ability may not be enough. Rather, citing earlier authors such as Dewey (1930), Facione and Facione (1992), Norris and Ennis (1989), and Paul (1990), Ennis suggested that CT dispositions are also necessary. In all, there are seven dispositions that are identified by Facione in the California Critical Disposition Inventory (CCTDI), which is one of the test instruments used in this study. These include: analyticity (anticipating both good and bad outcomes); CT self-confidence (trusting the use of reason and reflective thinking to solve problems); inquisitiveness (knowing things, even if not immediately useful); maturity of judgement (considering multiple evidence sources, and standing firm or adjusting as appropriate); open-

4  

mindedness (tolerance of opinions of others); systematicity (approaching problems in a disciplined way); and truth-seeking (desiring the best possible understanding).

Underlying this definitional difficulty is another matter, the issue of ‘generalisability’. The issue falls broadly into two groups, generic and specific. For the former, as Ennis (1992) argued, CT can be distilled down to a discrete set of skills that can be learned in a systematic way and applied across all academic disciplines. Because it is a ‘general’ skill, it is also argued that CT can be taught as an area of study, separate from any specific discipline-based content. McPeck (1992), however, was sceptical of such a generalist stance, arguing instead that CT is contextual and closely tied to subject matter. From a specifist’s perspective then, CT skills are highly dependent on content knowledge and development of the appropriate skills consequently is best pursued within the context of their studies. While the debate is viewed as being somewhat at an impasse, the issue is not trivial or simply a theoretical one. The reason is that how the concept is understood and conceptualised has a major bearing on curriculum design and on education policy (Barnett, 2004; Barrie & Prosser, 2004).

Offering a way forward, as Moore (2011) noted, the debate is not rooted in

educational reality or founded on any empirical evidence about the nature and achievement of CT. Consequently, two linked needs emerge:

First, there is logic in closing this empirical gap, particularly in the way CT is assessed, where, consistent with the increased attention by scholars and educational practitioners, the development of assessment tools to measure attainment has also grown significantly (Paul, 1985; Garrison, Anderson, & Archer, 2001).

Second, actual achievement needs to be ascertained. Internal CUHK surveys of student and graduate experience, via the Student Experience Questionnaire (SEQ) and the Graduate Capabilities Questionnaire (GCQ), suggest that students have a positive perception of development of this skill and of related capabilities such as creative thinking, problem-solving and communication skills. However, as the concept of CT (concerned with the ability to cast a sceptical mental eye over claims, arguments and assertions) invites, there is still a need to provide substantive evidence of successful practice, ideally mapped against discipline-based best practice.

Objectives Helping students improve their ability to think is undoubtedly one of the most important tasks in university education and developing CT skills, specific to a study discipline, is invaluable. Consistent with this objective, the Strategic Plan of the Chinese University of Hong Kong (CUHK, 2006) emphasises that all graduates should learn to be critical and independent and possess analytic skills. The objective of this study is to find the ways CT is both articulated and exemplified as a graduate attribute across disciplines.

The objectives of this study are to: 1. identify what CT means to students, teachers and alumni; 2. seek evidence of achievement of outcomes in terms of how CT is assessed and

reported; and 3. align learning outcomes and practice in case studies of best practice by academic

discipline.

By exploring different conceptions of CT, this study will seek to throw light on CUHK practices in teaching and learning. The key outcome will be a conceptual summary of literature that identifies the critical attributes and associated skills to support teaching by

5  

specific discipline. Usefully articulated learning activities and outcomes will allow academic staff in the University to take a proactive role in making an informed and distinctive contribution to curriculum development. This summary report will also include guidelines for curriculum development and for quality assurance. Ideally, sharing the findings of this study will contribute to the further successful development of CT (and related cognitive skills) at CUHK, as well as building awareness of how to embed CT in the curriculum in order to assist students gain a powerful tool for life. Methodology In seeking to understand CT in practice, the study used both a quantitative and qualitative approach to explore CT conceptions of senior academic managers, teachers, students and alumni, and discipline-based characteristics of CT in learning and teaching practice at CUHK. Highlighting group similarities and differences, the analysis will focus primarily on the experiences of Ug students; however, Pg students are also included in the study. The study design is in three parts, as detailed below. Objective 1: Identifying what CT means to students, teachers and alumni

focus-group and semi-structured interviews with teachers, Ug students and Pg students o Teachers – gain understanding of CT from a teaching perspective Associate Deans (Education) from Faculties of Arts, Business Administration,

Engineering, Medicine, Science, and Social Science; and teaching staff from Faculties of Education, and Law, revealing viewpoints from Ug programmes.

Teacher from Department of Computer Science and Engineering, representing a Pg programme as a whole.

Tool (Ug): ten open-ended questions to capture teachers’ understanding of CT, disciplinary CT traits, some ways to foster students’ CT in various programmes, some constraints, and some suggestions for further improvement.

Tool (Pg): nine open-ended questions. Research method: interview. Use of language: English.

o Ug students – gain understanding of CT from a learning perspective Subjects: eight groups of randomly selected students from each Faculty; some had

taken the tests, some had not. Tool: nine open-ended questions in responses to teachers’ interview questions. Research method: Focus-group interviews (three to six students in a group). Use of language: Cantonese mixed with English.

o Pg students – gain understanding of CT from a learning perspective Subjects: two groups of Pg students. Tool: eight open-ended questions in responses to Pg teachers’ interview questions. Research method: Focus-group interviews (five to six students in a group). Use of language: English (Some students are non-local students and thus the

mutual language of English was used.)

6  

Objective 2 – Evidence of achievement of outcomes in terms of how CT is assessed and reported

Conducting a pilot test for justification. Based on findings (see below), in general students believe that the California CT (CCTST) assessment is more suitable for CUHK students.

Administering a diagnostic test (CCTST + CCTDI selected) Subjects: 169 final-year Ug students (approximately 5% of the final-year students in

each Faculty) + 47 final-year Pg students across disciplines.

Table 1: Brief information about the tests

CCTDI CCTST Objective Disposition Skill Duration 20 minutes 50 minutes Paper type Paper test Paper test Question format Ug + masters: 75 MC* questions Ug + masters: 34 MC questions

PhD: 75 MCQs** PhD: 41 MCQs* multiple-choice ** multiple-choice questions

Objective 3 – Align learning outcomes and practice in case studies of best practice by academic discipline through

Triangulating interview data (teachers + students) with test results; Using content analysis of programme design and goals in relation to CT; Presenting data from Objective 2; and Conducting case studies of selected programmes.

CT defined Although the definition of CT varies among different authors and different disciplines, CT has been regarded as a set of metacognition skills that is incorporated with disposition, application, information and factual content. Most authors agree that CT consists of cognitive skills and dispositions; however, the contents and combination of different core CT skills remain controversial. Table 2 below reveals the range of views in authors’ definitions of CT and identifies the similarities among various definitions (√ indicates the attribute is mentioned).

Table 2: CT defined

Author(s) Evaluation Self-

regulation Interpretation Analysis Explanation Inference

Dewey (1909) √ √ √ Glaser (1941) √ √ √ √ √ Brookfield (1987) √ √ Beyer (1988) √ √ √ √Ennis (1989) √ √ √Facione (1990) √ √ √ √ √ √ Chaffee (1992) √ √ √ √ Paul (1993) √ √ √ √ Freeley (1993) √ √ √ √ Fischer & Scriven (1997) √ √ √ √ Pithers & Soden (2000) √ √ √Fisher (2001) √ √ √

7  

Watson & Glaser (2002) √ √ √ Facione (2009) √ √ √ √ √ √

CT disposition can be described as a willingness to think in a particular way regarding

specific evidence (Norris, 1994). It is a term that represents certain patterns of intellectual behaviours. In order to provide a more precise description of the nature of these thinking patterns, some scholars catergorise CT dispositions into a number of sub-dispositions (e.g. Ennis, 1994; Costa, 1991; Facione, Sanchez, Facione, & Gainen, 1995; Tishman & Andrade, 1997). A summary of different CT dispositions is in Table 3 (√ indicates the attribute is mentioned).

Table 3: Summary of skills and disposition

Disposition item Ennis (1994)

Costa (1991)

Paul (1993)

Perkins, Jay, & Tishman

(1994)

Facione et al.

(1995) Being clear about the intended meaning

√

Systematicity √ √ √ √ √ Taking the total situation into account

√

Analyticity √ √ √ √Inquisitiveness √ √ √ √Looking for alternatives

√

Seeking precision as the situation requires

√

Consciousness √ √ √ Open-mindedness √ √ √ √Truth-seeking √ √ √ Using one’s CT abilities

√

Being intellectually careful

√ √

Being metacognitive √ √ √ √ Efficacy √ Self-confidence √ √ Maturity √ √ CT – The Delphi report Among the above different definitions, the one developed in the Delphi report (Facione, 1990), a study endorsed by the American Philosophical Association, seems to be the most widely recognized one in the field. This report, based on the Delphi research method that requires the formation of an interactive panel of experts, also provides one strong methodological reference for our project to develop an operational definition of CT within the CUHK context. As indicated in Table 2, the experts consulted in this study by Facione found good CT to include both a skill dimension and a dispositional dimension. The experts characterized certain cognitive skills as central or core CT skills. The experts were virtually unanimous (N>95%) on including analysis, evaluation, and inference as central to CT. Strong consensus (N>87%) exists that interpretation, explanation and self-regulation are also central to CT. However, as the report stated, the experts did not say that a person must be proficient at every skill to be perceived as having CT ability (Facione, 1990). One implication the

8  

experts drew from their analysis of CT skills is that: “while CT skills themselves transcend specific subjects or disciplines, exercising them successfully in certain contexts demands domain-specific knowledge, some of which may concern specific methods and techniques used to make reasonable judgments in those specific contexts” (Facione, 1990, p. 5). Finally, as regards development of CT skills, according to the Delphi panel of experts, CT was not regarded as a body of knowledge to be delivered to students as one more subject along with others. Much like reading and writing, the report suggested CT had applications in all areas of life and learning and that like developing reading and writing, CT development and instruction could occur in programmes rich with discipline-specific content or in programmes that relied on events in everyday life. Assessing CT Table 4 (√ indicates type of design feature included) highlights some CT assessment tests and compares the designs among different tests.

Table 4: CT assessment tests and design features

Name of the test Question

type Discipline-

specific Type of analysis

Disposition measure

Skills measure

Watson-Glaser CT Appraisal (WGCTA)

MC Quantitative √ √

California CT Tests (CCTST + CCTDI)

MC √ Quantitative √ √

Ennis-Weir CT Essay Test (EWCTET)

Essay Content analysis

√

Cornell CT Tests (CCTT) MC Quantitative √ √Halpern CT Assessment Using Everyday Situations (HCTAES)

Essay + MC

Quantitative

+ Qualitative

√ √

International CT test (ICT) √ √

There are a large number of CT tests in the field, with instruments varying in design and scoring methods. Table 5 is a summary of some tests and their related designs.

Table 5: A brief summary of different CT tests

Name of the test Question

type Discipline-specific

Type of analysis

Disposition measure

Skills measure

Knowledgemeasure

Watson-Glaser CT Appraisal (WGCTA)

MC Quantitative √ √

California CT Tests (CCTST + CCTDL)

MC √ Quantitative √ √

Ennis-Weir CT Essay Test (EWCTET)

Essay Qualitative √

Cornell CT Tests (CCTT) MC Quantitative √ √ International CT Basic concepts and understanding online test (ICT)

MC Quantitative √

Halpern CT Assessment using everyday situations (HCTA)

Essay + MC

Quantitative + Qualitative

√ √

9  

Some well known tests included in this summary include the Watson-Glaser CT

Appraisal (WGCTA; Watson & Glaser, 1980); the Ennis-Weir CT Essay Test (EWCTET; Ennis & Weir, 1985); and the California CT Test (CCTT) that comprises two instruments, a skills test (CCTST) and a disposition inventory (CCTDI; Facione, 1990). There are several other newer and relatively less well-known instruments. Two such instruments are the International CT Basic Concepts and Understanding Online Test (ICT; Paul & Elder, 2006) and the Halpern CT Assessment Using Everyday Situations (HCTA; Halpern, 2003), both of which are included for examination in this study.

As several scholars have noted, few studies have examined the characteristics of tests

available on the market (Ku, 2009; Adams et al., 1996). More important, perhaps, is the currently limited research that compares attributes of tests and, more importantly, their contribution to skill development and performance from a student’s perspective. As Gibbs and Simpson (2004) noted, valid assessment should, at least in an educational context, also provide developmental feedback that can contribute to the enhancement of, in this case, CT ability. Given the emphasis on CT in higher-education curriculums, this is an important consideration, as is understanding how students, both Ug and Pg, receive the different designs of CT assessments and perceive the usefulness of any feedback, particularly in terms of individual skills development. Utility of CT tests Summarising literature related to CT tests, as Cheung et al., (2002) noted, few studies review design and content. There are also few recent comparative studies; hence, some of the relatively newer tests such as the ICT and HCTA noted earlier are not included. Moreover, as Adams et al. (1996) reported in their review of four CT tests, evaluative research is useful because this enables educational institutions to select the test instrument appropriate to their specific programme need. Notably, none of the reported studies so far appears to be based on empirical studies; nor do these studies capture the student’s view. Rather, the research focus of those reported studies is on examining the content of tests from a researcher’s perspective.

Ku (2009), for example, compared the purpose, format and context of a number of

tests including the CCTT, the WGCTA, the HCTA, the Ennis-Weir CT Essay Test and the Cornell CT Test. This study concluded that a combination of MC and open-ended questions was a more effective evaluation of students’ CT performance, as this combination allowed an assessment of the correct response, as well as of spontaneous metacognitive processes. Such a conclusion would be substantive if it were supported by empirical data based on test recipients. Empirical support aside, as Kline (1986) noted, appropriateness of the test is crucial, because it directly affects the validity of the result. Hence, student perception of and attitude towards the different tests formats is important. These are some of the gaps in the state of understanding of CT tests currently available, and given the curricular emphasis on CT, further studies that examine these tests in terms of design, purpose and content, as well as developmental effect, are useful.

Selected tests For this study, three tests were selected from the range of tests shown in Table 1 – the CCTT, HCTA and ICT. The CCTT is one of the most popular tests in research-based investigations (Jacobs, 1999; Fawkes, O’Meara, Weber, & Flage, 2005). It provides a useful benchmark for the other two tests, which are both as yet relatively understudied and which display some unusual features in both design and question type. Looking first to the CCTT, Khalili and

10  

Hossein Zadeh (2003) asserted the validity of this test and its usefulness as a reliable research tool, because the design of the (skills) test enabled the researcher to distinguish levels of competence across individuals. However, this view is not unanimous. Linking to the generic/ specific debate, Stone, Davidson, Evans, and Hansen, (2001), for example, questioned the validity of the CCTT for assessing nursing students’ skills, arguing rather that the CCTT could not fully reflect the competencies of nursing students. In addition, in an argument based on empirical evidence, Stone et al., (2001) suggested there was no significant relationship between the two components of the CCTT – the Disposition Inventory (CCTDI) and the Thinking Skills Test (CCTST).

If the general utility of the CCTST in assessing discipline-specific CT skills

competence is doubtful for Nursing, there is reason to question its general suitability to test the CT competence of Hong Kong (HK) Ug students from a range of different disciplines. Beyond the appropriateness of design of tests such as the CCTT, Fawkes et al. (2005), who examined the choice of questions posted in the CCTST, questioned the accuracy of these questions and suggested, in effect, that the CCTST was only able to assess a minimal level of CT competence. To put these findings into some perspective, as no comparative tests were conducted it is difficult to comment on the relative strengths and weaknesses of the CCTST against any of the other currently available tests in higher education.

Compared with studies based on the CCTT, literature on the HCTA and the ICT is

relatively limited. A common feature in the few studies of the HCTA is the positive response to the use of both open-ended and MC-type responses (Ku, 2009; Hau, Ho, & Halpern, 2006). However, as Neito and Saiz (2008) have noted, while open-ended questions may be an effective way to evaluate CT competence, these questions are time-consuming to mark. Similarly, assuring both reliability and validity are likely to be other major concerns, particularly in any large-scale application of one or more tests in an educational institution. Importantly, however, as Hau et al. (2006) reported, open-ended questions appear to have no significant influence on the performance of bilingual test takers, using either Chinese or English versions of the HCTA. This finding is crucial as test-takers need to express their thoughts in words and so written skills can influence reported performance. Again, the limitation in this study by Hau et al. (2006) is that there is no comparison study of test takers with different levels of language proficiency.

The third test selected for this study is the ICT, which is an online instrument

developed by Paul and Elder (2007). This test is different from the other tests as it focuses on foundational CT knowledge and principles as they define and illustrate it in their model. This is a notable divergence from other tests and, while education practitioners generally agree that a good conceptual understanding can enhance an individual’s awareness of and application of CT skills, they also suggest a sound conceptual understanding is not a reliable indicator of CT competence (Browne & Freeman, 2000; Fisher, 2001). That aside, the ICT test does not appear to allow for individual disposition or the capacity to translate knowledge into action, although it does address perceived obstacles to effective performance. Overall, there are very few independent examinations of the ICT; also, reliability and face validity (internal consistency) of the ICT appears to be understudied. Comparison of selected tests As a factual comparison of the three tests in Table 6 shows, each of the three tests has unique features in terms of test content, questionnaire design and scoring method. Both the CCTT and ICT use an MC question format, while the HCTA uses both MC and open-ended

11  

questions. The three tests also vary in terms of format, with the CCTT tests available in both paper and online format, while the HCTA and ICT are only available in an online format. The method of scoring also varies across the three tests. The CCTT, unlike the HCTA and ICT, is based on a confidential and centralized scoring system that is closely managed by the parent organisation. This practice, undoubtedly for commercial reasons, does, however, leave the validity and reliability of judgments on student performance in the hands of an external testing agency.

Table 6: Factual comparison of the selected tests

Test CCTT HCTA ICT

Aspects covered

CCTST CCTDI Verbal reasoning skills

Argument analysis skills

Skills in thinking as hypothesis testing

Using likelihood & uncertainty

Decision-making & problem-solving skills

Attitudes

Basic CT insight Elements of thought Intellectual standard Intellectual

trait/virtue Obstacles to CT

Analysis Inference Evaluation of

deductive reasoning

Inductive reasoning

Truth-seeking Open-

mindedness Analyticity Systematicity CT self-

confidence Inquisitiveness Cognitive

maturity

Design A variety of questions (e.g. situations, charts, numerical) in MC format

25 sets of scenario-based questions (including constructed and forced responses)

Theoretical questions in MC format

No. of questions 109 (34+75) 56 100 Attitude measurement

Included Included Excluded

Duration 70 mins (CCTST + CCTDI) 2 hours 30 minutes

Scoring standards Unknown (restricted by copyright)

Known for the 25 sets of questions but unknown for disposition

Known

Full score CCTST: 34/ CCTDI: 420 194 100 Chinese version Yes Yes No CT test study method The study design is based on comparative evaluation of the three selected tests (CCTT, HCTA, ICT) using Ug student volunteers. Thirty participants, all final-year Ug students in CUHK, registered for the test session and 23 of them completed the study. These student participants were from a variety of disciplines including arts, humanities and science. No student had previously completed a similar CT type of assessment. The tests were administered in staggered sequence at the one setting: test group 1 – completing the HCTA, ICT and then the CCTT; test group 2 – the ICT, followed by the CCTT and then the HCTA; and test group 3 – the CCTT, followed by the HCTA and then the ICT. This sequenced approach was intended to reduce the influence of test order variation and ensure greater balance in reported experience. Test fatigue was an expected limitation, compensated for by adequate refreshments and short breaks between each test. Given the length of each test, the

12  

overall process (test and questionnaire) took approximately four hours, including a lunch break.

Following the tests, participants were asked to complete a questionnaire and later

participate in a small-group interview to gather feedback on the design and perceived efficacy of the tests. In all, 23 students (11 female, 12 male) successfully completed the full suite of tests and follow-up questionnaire that focused on participants’ overall attitude toward the tests. The primary purpose of this questionnaire was to obtain immediate summative comments of the different tests from a student’s perspective. The questionnaire also asked students to rate their overall opinion of the three tests. The group interviews that followed some weeks later looked to provide developmental feedback based in the test results and explore participant views on the design and contents of each test. Table 7 is a summary of the collected data.

Table 7: Summary of data-collection methods and purposes for test selection

Research method

Purpose Research questions Data collected

Questionnaires with 90% of participants

To gain background information of the participant group

To explore participants’ general perception of the three tests

To seek the participants’ willingness to participate in further exploration of this study

Which is the most appropriate test for CUHK Ug students?

What are perceived strengths and weaknesses of the tests?

20 completed questionnaires with 16 closed questions and one open-ended question.

Three CT tests: results with 100% of participants

To examine the participants’ performance via different measurements

What is the CT performance of CUHK Ug students?

69 completed tests with 69 results

Group interviews with 40% of participants

To collect the general views of students on tests

To identify the positive and negative features of each individual test

To identify the criteria for a good test based on a student’s perspective

Reflecting on the three tests, what are students’ general view of these tests?

What were students’ views of the positive and/or negative features of the three selected tests?

In students’ views, what are the criteria for a good CT test?

Five group interviews lasted approximately from 60 to 75 minutes.

All tests were completed in English, while the responses collected in the

questionnaires were mainly in English, with some written Chinese. All numeric data collected from the questionnaires were coded and analysed using SPSS 17, while the interview data were collected and transcribed in Cantonese for content analysis by a native Cantonese speaker who is also fluent in English. Selected comments were translated into English for

13  

reporting purposes, with the data from questionnaires and interviews cross-matched to enhance validity and support subsequent analysis. Findings Objective 1: Identifying what CT means to students, teachers and alumni There is a great deal of debate over CT being a generic or specific skill. Embedded in different ideas from literature, a conceptual CT model (Figure 1) was developed for the use within CUHK context, based on three broad components, foundational knowledge (the knowledge base that is founded on understood facts, information and experience), personal attributes or ego-resiliency (personality disposition and situational attributes) and actual skills (that include a reflective component). Intuitively, it makes sense that CT skills will be significantly better if there is a base or foundational knowledge. Similarly, as some of the literature reviewed identified, disposition or willingness to commit the effort to apply CT is a crucial attribute that precedes and shapes the quality of performance of CT. This attribute also includes personality aspects such as self-confidence and self-esteem, as well as the situation (or context), which can shape behaviour. The third category includes skills of applying CT in various situations, the area where there is a good deal of debate.

Figure 1: CT constructs conceptualised

The organizing framework for operationalising the level and type of performance

outcome is facilitated by the conceptual triangle on the right side of this illustration. Briefly, the three defined levels of performance vary with complexity across a continuum described by Reason (1990). Simple automatic routines or actions require little conscious attention. These are skill-based actions. Slightly more complex is rule-based behaviour that involves pre-defined behaviors when an appropriate rule is applied. For example, if the symptoms are X, then Y is the issue and Z is the action. Finally, the last automated performance level is knowledge-based performance where there is likely greater complexity and ambiguity and often improvisation is required in unfamiliar environments, with no rules or routines to govern how to handle the situation. In a knowledge-based mode, a person will carry out the

14  

task in an almost conscious manner – a beginner performing a new task or an experienced worker facing a novel situation. Importantly, all three attributes can be operationalised at the one time, as is illustrated in the example of driving a car. Normal steering is an automated skill-based activity, managing driving distances between vehicles a largely rule-based matter, while driving at night and in snowy conditions will require improvisation and careful management. The construct developed for safety and risk management by Reason (1990) provides a useful framework to evaluate performance.

In order to explore the underlying beliefs and thus to confirm the model, interviews

were conducted with academics (Associate Dean (Education) or teachers) and students from each Faculty. In our findings, we found that CT is neither a generic skill, which can apply in the same way to all disciplines, nor a specific skill, which is only needed for particular specialisms. Instead, different specialisms seem to have different notions of what CT is. Academics in all Faculties mentioned that CT skills embraced some generic skills, which are applicable for general public, as well as some disciplinary specific skills, which are particularly important for individual disciplines. Disciplinary traits Interview findings indicate there is a conceptual common ground on the understanding of CT among teaching colleagues from different Faculties, whereas their focus on CT and the ways of fostering CT can vary widely. Table 8 lists an overview of shared understanding of CT across different Faculties.

Table 8: Disciplinary traits

CT understanding Arts Business Education Engineer-

ing Law Medicine Science

Social Science

No universal definition

√ √ √ √ √ √ √ √

CT is important in daily and academic life

√ √ √ √ √ √ √ √

Embraced several competences

√ √ √ √ √ √ √

Learnt through implicit training

√ √ √ √

Learnt from integrated approach

√ √

Creativity as an operator

√

Teacher as facilitator √ √ √ √ √ √ Language affects one’s CT

√ √ √ √

Time is needed √ √ √ (diligent) √

CT is cultivated in environment

√ √

Practice in simulated environment

√ √ √

In order to understand the similarities and differences in the understanding of CT

between teachers and students, interview data were also compared and contrasted between

15  

the views of teachers and students. Table 9 illustrates the different understanding of CT between teachers and students in CUHK.

Table 9: Comparison of teacher (N=10) and student (N=41) conceptions

Teacher Students

Gen

eric

com

pete

nce

Skills Evaluation: E.g. avoiding political

propaganda and presuppositions Providing alternatives Knowledge Understanding surrounding phenomenon

knowledge as backup to provide alternatives

Skills Valid and rational thinking Evaluation: Provide reasons; arguments need to

be connected and well-organized; clear logic Diverse thinking: Understanding from different

perspectives Considering different arguments Understanding one’s own limitations Disposition

Willingness to develop CT: Motivation and prioritizing thinking to a higher position

Open-mindedness: Considering different information before making judgment

Persisting in thinking; being willing to ask for assistance

Spec

ific

com

pete

nces

Skills Evaluation: Assessing validity; seeking

logical ambiguity; sufficient reasoning of underlying assumptions; exploring presuppositions

Sympathetic understanding: Understanding others’ limitations

Providing alternatives Self-evaluation: Understanding one’s own

limits Disposition Appreciation of others’ achievement (being

critical is not the same as criticizing) Foundational Knowledge Understanding of relevant information

Skills Data collection (by observation or literature) Analysis and interpretation: Decoding

significance and how results are connected Test validity of selected observations or

examples Evaluation

Based on the different CT focus of each Faculty teaching representative, the various

understandings of CT were further grouped into three discipline categories: 1) skills-based; 2) knowledge-based; and 3) rule-based, using the broad disciplinary grouping identified by Becher (1987) and Becher and Trowler (2001). These studies conceived of disciplines as having recognizable identities and cultural attributes. This study adapted the four-part rubric based on hard-soft and pure-applied categories using disciplinary culture and nature of knowledge to place disciplines within categories that emphasise skills, knowledge or rule-based activity. There were no participants from the hard-pure category.

16  

We found that the Faculties of Arts, Business Administration and Social Sciences had a similar understanding of CT due to similar disciplinary characteristics. Considering the overlap between Science and Engineering, we found that the understandings of CT between these two Faculties are more similar than different from each other. It is interesting to note that although the Faculties of Law, Education and Medicine seem to diverge in many aspects, their views towards CT were relatively aligned. One of their shared learning outcomes – developing students’ professional career in a specific area – might be an explanation.

Table 10: Disciplinary grouping

 Disciplinary

grouping Hard-applied skills-based

discipline Soft-applied knowledge-

based discipline Soft-applied rule-based

discipline

Faculties Science and Engineering

Arts, Business Administration and Social Science

Education, Law and Medicine

Characteristics

Science and technology; study of some hard knowledge, e.g. mathematics; mainly assessed on calculation.

Study about soft knowledge; person- or society-oriented; resulting in understanding or interpretation.

Concerned with [semi-] professional practice; exercise/ practice disciplinary knowledge in actual or simulated environment is necessary; resulting in protocols/ procedures.

Objective 2: Evidence of achievement of outcomes in terms of how CT is assessed and reported We turn next to examine results from the CT test conducted by representative CUHK students using the CCTDI (CT disposition) and CCTST (skills) tests. The top dotted line is what is called the high-cut line (ratings above this line suggest a strong performance), while the lower solid line is the low-cut line (ratings below this line suggest a weaker performance).

Figure 2: CUHK students’ disposition towards CT (N=169)

Figure 2 shows the overall performance of CT disposition in our sample of CUHK

students. The scores illustrate that all of our participants are positively and strongly disposed towards CT. Indeed, they showed a particularly strong desire to learn and eager to acquire new knowledge/ inquisitiveness. However, their attitudes toward truth-seeking were

17  

relatively low, as they tended to have less interest in exploring the best possible understandings such as alternative reasons and additional evidence for any given situation.

Figure 3: CUHK students’ CT skills (N=169)

Figure 3 shows the overall CT skills performance of our participating students. The

results reflect that the CT skills of our students are much stronger than the norm reference group. In fact, CUHK students appear especially good at drawing conclusions based on a wide variety of reasons and evidence/ inference; analyzing and developing both inductive and deductive arguments.

Figure 4: CUHK Ug students’ disposition towards CT for different Faculties (N=169)

Figure 4 shows the different performance of CT dispositions among eight Faculties. It

is interesting to note that although some Faculties tended to put a stronger emphasis on CT

18  

into their curriculum than the others did, there are no significant differences in the students’ performance on the CCTDT among the Faculties. This implies that all selected students recognize the importance of a positive attitude towards CT and they all seem to have an eagerness to become a critical thinker, regardless of the discipline of study.

Figure 5: CUHK Ug students’ CT skills for different Faculties (N=169)

Figure 5 reports the students’ performance on the CCTST of individual Faculty.

Students from the Faculty of Business Administration obtain a higher total score than students from the other Faculties. However, it is important to note that the difference is not found to be statistically significant and thus we conclude that all selected CUHK students performed generally well in the CCTST and there is no particular Faculty where students performed better than any other.

Figure 6: The overall performance of CUHK Pg students towards CT disposition (N=47)

19  

Figure 6 illustrates the overall CT disposition of our selected Pg participants in CUHK. It indicates that our Pg students show a positive and strong disposition towards CT, though have lower truth-seeking, which is below the low-cut line. Similar to the results of our Ug participants (see Figure 2), their results on the CCTDI reflect a strong motivation to acquire new knowledge and learn explanations for new things; while a relatively weak attitude towards truth-seeking indicates that they tend to be less interested in seeking all possible evidence and reasons.

Postgraduate students’ performance on CCTST

0

2

4

6

8

10

12

Analysis Inference Evaluation Inductive Reasoning Deductive Reasoning

Figure 7: CUHK masters students’ CT skills (N=41)

Figure 7 reports the CT skills performance of our participating master students on the CCTST. Comparing their performance with the scores of the norm reference group, it is found that our master students are strong in analytical and interpretive skills. They seem good at examining ideas, gathering information and identifying reasons and assumptions.

76

78

80

82

84

86

88

90

Analysis Inference Evaluation Inductive

Reasoning

Deductive

Reasoning

Figure 8: CUHK PhD students’ CT skills on CCT-G835 (N=6)

There is a different version of CCTST to measure the CT skills of PhD-level candidates, which is called CCT-G835. Due to the different test design and rating scale, Figure 8 illustrates the CT skills performance of PhD students that is separate from the Pg student results listed in Figure 7. Comparing with the scores of the norm reference group provided by the testing authority (Insight Assessment), it is interesting to note that our selected PhD participants are particularly strong in analytical and deductive reasoning skills, which are both above the given high-cut point. This implies that our PhD students are not only good at analyzing different sources of information and identifying precise meaning from various sources of data, but also skilled in evaluating general information down to specific meaning for a given purpose/ situation and constructing arguments through a set of premises.

20  

Overall, the University findings can be described as follows: Based on the students’ test reports, the relative performance of CUHK students

compared with the US student mean, show no significant difference between Faculties. Also, on average, CUHK students perform well.

A basic statistical analysis on students’ test performance in relation to socio-cultural and training background revealed that the amount of time spent on talking to professors was significantly correlated to students’ CT attributes.

The level of CT confidence is highly correlated with CT performance (which indicates that the test instrument reflects what students think CT is).

Objective 3: Align learning outcomes and practices in case studies of best practice by academic discipline Briefly, this objective remains to be fully actioned and is suited to follow-up institution-level work. Sample cases illustrating evidence of CT in Departments are shown in Appendix 1. Good practices extracted from transcribed responses by teachers are shown according to the broad themes of enquiry that were identified based on the questions asked of Associate Deans (Education) or their representatives, as well as from students from the various Faculties (see Appendices 2 and 3, respectively). Two sample questions include: ‘To what extent does your Faculty foster the development of your critical thinking?’ and ‘What would you do to enhance your critical thinking development other than by doing your core courses?’ Table 11 illustrates good practices in CT by broad disciplinary groups.

Table 11: Good practices by disciplinary groups

Classroom teaching

Hard-applied discipline Soft-applied knowledge-based discipline

Soft-applied rule-based discipline

Classroom presentation Question probing in

classroom

Interactive elements: guided discussion

Explicate the rationale and methods in teaching

Interactive elements: debate/ discussion

Encourage students to reflect on action/ decision and to articulate/ present their thinking

Assessment Facilitate student–student

interaction through group projects

Emphasis on evaluation and justification in essay-type question or project

Require analytical thinking and reflection in essay-type question or case studies

Outside classroom facilitation Experiential learning (e.g.

exchange, internship) Encourage students to share

their knowledge via web forums

Provide free-to-discuss environment

Experiential learning (e.g. exchange, field study)

Seminar where external speakers shared their real-world experience

Actual practice in simulated environment

Experiential learning (e.g. exchange, internship)

Curriculum design Require students to apply CT

in solving problems Provide diverse exposure to

cultivate students’ attitude via field study or site visit

No model answers

Involved research and reflective elements

21  

After consolidating research data gathered in interviews with Faculty staff and focus-group interviews with participating students, some useful development strategies were identified. These are outlined in Table 12. The feedback gathered was further categorised by level: individual teacher level, at Faculty level and, more generally, in terms of the learning environment and categorised as at University level. Notably, the perspectives offered do not assert a specific requirement for a course to teaching CT skills. Rather, and consistent with the Delphi panel of experts, CT is not regarded as a body of knowledge to be delivered to students in a subject along with others (Facione, 1990, p. 5). Instead, CT and related skills development were described in a similar way to reading and writing, as occurring in terms of discipline-specific content or in programmes that relied on events in everyday life.

Table 12: Improvement (development) strategies by disciplinary groups

Individual teacher level Hard-applied discipline Soft-applied knowledge-based

discipline Soft-applied rule-based

discipline Involve interactive elements

in class Encourage students to voice

out their opinions Reduce the emphasis on

memorisation; emphasise thinking and reflective elements in assessment

Less MC, more essay-type questions or question with no standard answer

Give quality feedback on the reasoning process

More time for discussion Encourage more students to

voice out their views

Give more quality feedback Include more reflective

elements Explicate the rationale and

reasons More time for discussion Give more case studies in

class

Faculty level Involve CT elements in every

course Less exam-oriented in

assessment scheme Create a more free-to-discuss

environment and cultivate students’ CT attitude

More exchange opportunities Raise the awareness of CT in

curriculum

Involve CT elements in every course

Standardise the assessment method within the Faculty

More interdisciplinary courses/ training/ activities

Involve CT elements in every course

More different stimulated exercise/ environment to practice

University level Create free-to-discuss environment to foster students’ opinion exchange Increase the opportunities of exchange, internship or residential accommodation to allow students

having more different exposure Encourage teachers to give timely and useful feedback Encourage outside-class teacher–student interaction Strengthen and raise the importance of General Education courses which are beneficial for students

from ALL disciplines Reduce the focus on content and memorisation, leave more time for thinking and reflection Raise the CT awareness of both teacher and students Offer more CT workshops or seminars presenting solid CT conceptions More interdisciplinary learning opportunities such as interdisciplinary courses or FYP allowing

students to hear opinions from different perspectives/ disciplines

22  

General comments relating to good practice from follow-up interviews include: Contextual constraints (that appear common to both teacher and students)

o Time constraints (study time and curriculum scope) o Less [opportunity for] communication outside classroom o Less interactive elements inside classroom o Teaching of CT is not explicated (fostering strategy is not obvious) o No-one has specific responsibility to teach CT

Both teacher and students believe that other learning activities (e.g. exchange, internship, placement … etc.) can help CT development.

General education is reported as useful in facilitating several CT skills attributes There were no significant results supporting how assessment methods facilitated

students’ CT performance Other general suggestions include:

o Use College activities (assemblies, debates, college final-year project … etc.) o Create a free-to-discuss atmosphere

Conclusions Critical thinking is a valuable capability and an essential component in both Ug and Pg studies. In this study, all teacher and student participants recognized its importance to one’s personal, as well as professional development. Research on the development of CT skills as a graduate attribute for Ug and Pg students provides evidence of the CT competence of our CUHK students. Our analysis highlights the diversified methods for teaching and fostering CT development into different disciplines in the CUHK context. Based on the reported results, this study finds that the CT development of students tend to be hindered by their overwhelmingly busy curriculum and it can be costly for them to be a critical thinker at all times. The interview data with teaching colleagues also shows that they sometimes have difficulties in facilitating their students’ CT development within the discipline as their interaction with students is constrained by the size of the class and workload-related issues.

With an additional Ug year from 2012, it is suggested that more work on fostering

students’ CT development could be done in the coming normative four-year curriculum and more CT-related courses can be introduced through general education. Noting the emphasis given to CT capability in higher-education curricula, however, there is clear need for a greater focus on the evaluation of CT (and disposition). This process must include the twin aspects of assessment and developmental feedback. A key limitation in the design of this capability is the actual definition of CT, as this has a heavy influence on the subsequent design of the test instrument. Given the lack of consensus on this matter, it is also difficult to reach an agreement about how best to measure this capability. The conceptual illustration presented in this study (Figure 1) is ideal as a basis for use by teachers. A crucial point: student participants took a unanimous view that CT was a generic skill and that the test should include both open-ended and closed-ended (MC-type) questions. Participants also highlighted the importance of bringing socially related content into the test in order to trigger CT performance in realistic daily contexts. Students also expressed a concern over the need to balance numeric conditional and natural language conditional type of questions. Feedback suggests that students currently perceive a discipline bias in the tests, while based on the pilot study students appear to think the California-based tests (CCTST and CCTDI) are the most suitable test out of the three tests evaluated.

For the future, it is hoped that this study might encourage further research into a key area of educational practice – with the objective of developing a feasible and readily

23  

accessible CT skills and disposition test. This is a high-value goal, given CT is a widely endorsed capability in Ug programmes. An important corollary of this goal, a useful test, is to shift the focus in design of CT assessment tools from the domain of scholars and industry to the Ug (and Pg) student body. Given that this is a major customer group, it seems appropriate that the perceptions and pragmatic concerns of these prospective users and test-takers be included as legitimate considerations in the design and application of future CT tests. References Angeli, C., & Valanides, N. (2009). Instructional effects on critical thinking: Performance on

ill-defined issues. Learning and Instruction, 19(4), 322–334. Adams, M. H., Whitlow, J. F., Stover L. M., & Johnson, K. W. (1996). Critical thinking as an

educational outcome: An evaluation of current tools of measurement, Nurse Educator, 21(3), 23–32.

Barnett, R. (2004). Learning for an unknown future. Higher Education Research and Development, 23(3), 247–260.

Baron, J. (2000). Thinking and deciding (3rd ed.). New York, NY: Cambridge University Press.

Barrie, S., & Prosser, M. (2004). Generic graduate attributes: citizens for an uncertain future. Higher Education Research and Development, 23(3), 243–246.

Becher, T. (1987). The disciplinary shaping of the profession. In B. R. Clark (Ed.) The academic profession. Berkeley, CA: University of California Press.

Becher, T., & Trowler, P. (2001). Academic tribes and territories: Intellectual enquiry and the cultures of disciplines (2nd ed.). Buckingham: Open University Press/SRHE.

Beyer, B. K. (1988). Critical thinking. Indiana, IN: The Phi Delta Kappa Educational Foundation.

Brookfield, S. D. (1987). Developing critical thinking: Challenging adults to explore alternative ways of thinking and acting. Milton Keynes: Open University Press.

Browne, M. N., & Freeman, K. (2000). Distinguishing features of critical thinking classrooms. Teaching in Higher Education, 5(3), 301–309.

Chaffee, J. (1992). Critical thinking skills: The cornerstone of developmental education. Journal of Developmental Education, 15(3), 2–8, 39.

Cheung, C., Rudowicz, E., Kwan, A. S. F., & Yue, X. D. (2002). Assessing university students’ general and specific critical thinking. College Student Journal, 36. Retrieved March 18, 2012, from http://www.tighturl.com/2ye1

Costa, A. (1991). The school as a home for the mind. Palatine, IL: Skylights. Davies, M. (2011). Introduction to the special issue on critical thinking in higher education.

Higher Education Research & Development, 30(3), 255–260. Dewey, J. (1909). How we think. Charleston: SC: Forgotten Books. Dewey, J. (1930). Human nature and conduct. New York, NY: The Modern Library. Ennis, R. H. (1989). Critical thinking and subject specificity. Educational Research, 18(3),

4–10. Ennis, R. H. (1992). The degree to which critical thinking is subject specific: Clarification

and needed research. In S.P. Norris (Ed.), The generalizability of critical thinking: Multiple perspectives on an educational ideal (pp. 21–37). New York, NY: Teachers College Press.

Ennis, R. H. (1994, April). Assessing critical thinking dispositions: Theoretical considerations. Paper presented at Annual Meeting of the American Education Research Association. New Orleans, LA.

Ennis, R. H. (1996). Critical thinking dispositions: Their nature and assessability. Informal Logic, 18(2), 165–182.

24  

Ennis, R. H., & Weir, E. (1985). The Ennis-Weir Critical Thinking Essay Test. Pacific Grove, CA: Midwest.

Facione, P. A. (1990). Executive summary – Critical thinking. A statement of expert consensus for purposes of educational assessment and instruction. The California Academic Press, Millbrae, CA. Retrieved March, 18, 2012, from http://assessment.aas.duke.edu/documents/Delphi_Report.pdf

Facione, P. A. (2009). Critical thinking: What it is and why it counts. Insight Assessment. Retrieved March, 18, 2012, from http://www.insightassessment.com/content/download/1176/7580/file/what%26why2010.pdf

Facione, P. A., & Facione, N. C. (1992). The California critical thinking dispositions inventory. Milbrae, CA: California Academic Press.

Facione, P. A., Sanchez, C. A., Facione, N. C., & Gainen, J. (1995). The disposition toward critical thinking. Journal of General Education, 44(1), 1–25.

Fawkes, D., O’Meara, B., Weber, D., & Flage, D. (2005). Examining the exam: A critical look at the California Critical Thinking Skills Test, Science & Education, 14, 117–135.

Fisher, A. (2001). Critical thinking: An introduction. Cambridge, UK: Cambridge University Press.

Fisher, A., & Scriven, M. (1997). Critical thinking: Its definition and assessment. Norwich: Centre for Research in Critical Thinking

Freeley, A. J. (1993). Argumentation and debate: Critical thinking for reasoned decision making. (8th ed.). Belmont, CA: Wadsworth.

Garrison, D. R., Anderson, T., & Archer W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23.

Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports learning. Learning and Teaching in Higher Education, 1, 3-31

Glaser, E. M. (1941). An experiment in the development of critical thinking. New York, NY: Teachers College Press.

Green, W., Hammer, S., & Star, C. (2009). Facing up to the challenge: Why is it so hard to develop graduate attributes? Higher Education Research and Development, 28(1), 17–29.

Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Dispositions, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449–455.

Halpern, D. F. (2003). Thought and knowledge: An introduction to critical thinking (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Publishers.

Hau, K. T., Ho, I. T., & Halpern, D. (2006, November). Assessment of critical thinking in Chinese and U.S. students: Cross-cultural construct validation. Paper presented at the International Conference on Educational Evaluation Conference, Taipei, Taiwan, Taiwan Normal University.

Hastie, R., & Dawes, R.W. (2001). Rational choice in an uncertain world: The psychology of judgment and decision making. Thousand Oaks, CA: Sage.

Jacobs S. S. (1999). The equivalence of forms A and B of the California Critical Thinking skills test, Measurement and Evaluation in Counseling and Development, 31, 211–222.

Khalili, H., & Hossein Zadeh, M., (2003) Investigation of reliability, validity and normality Persian version of the California Critical Thinking skills test; Form B (CCTST). Journal of Medical Education. 3(1), 29–32.

25  

Kline, P. (1986). A handbook of test construction: Introduction to psychometric design. London: Methuen & Co. Ltd.

Ku, K. (2009). Assessing students’ critical thinking performance: Urging for measures using multi-response format. Thinking Skills and Creativity, 4, 70-76.

MacPherson, K. (1999). The development of critical thinking skills in undergraduate supervisory management units: Efficacy of student peer assessment. Assessment & Evaluation in Higher Education, 24(3), 273–84.

McMillian, J. H, (1987). Enhancing college students’ critical thinking: A review of studies. Research in Higher Education, 26(1), 3–29.

McPeck, J. (1992). Thought on subject specificity. In S. Norris (Ed.), The generalizability of critical thinking: Multiple perspective on an educational ideal (pp. 198–205). New York, NY: Teachers College Press.

Moore, T. J. (2011). Critical thinking and disciplinary thinking: a continuing debate. Higher Education Research & Development, 30(3), 261–274.

Moore, Z. B., Faltin, L., & Wright, M. (2003). Critical thinking and international postgraduate students. Discourse: Learning and Teaching in Philosophical and Religious Studies, 3(1), 63–64.

Nieto, A. M., & Saiz, C. (2008) Evaluation of Halpern’s “structural component” for improving critical thinking. The Spanish Journal of Psychology, 11, 266–274.

Norris, S. P. (1994). The meaning of critical thinking test performance: The effects of abilities and dispositions on scores. In D. Fasko Jr. (Ed.), Critical thinking: Current research, theory, and practice. Dordecht, The Netherlands: Kluwer.

Norris, S. P., & Ennis, R. H. (1989). Evaluating critical thinking. Pacific Grove, CA: Midwest.

Paul, R. W. (1990). Critical thinking: What every person needs to survive in a rapidly changing world. Rohnert Park, CA: Center for Critical Thinking and Moral Critique, Sonoma State University.

Paul, R. W. (1985). The Critical-Thinking movement, National Forum, 65, 2–3. Paul, R. W. (1993). The logic of creative and critical thinking. The American Behavioral Sc Paul, R., & Elder, L. (2006). A guide for educators to critical thinking competency standards:

Standards, principles, performance, indicators, and outcomes with a critical thinking master rubric. Retrieved March, 18, 2012, from http://www.criticalthinking.org/files/CTcompetencies2005-DC.pdf

Paul, R., & Elder, L. (2007). A critical thinker’s guide to educational fads. Retrieved March, 18, 2012, from http://www.criticalthinking.org/files/Educational_Fads_2007DC.pdf

Perkins, D. N., Jay, E., & Tishman, S. (1994). Assessing thinking: A framework for measuring critical thinking and problem solving skills at the college level. In A. Greenwood (Ed.), The national assessment of college student learning: Identification of the skills to be taught, learned, and assessed (pp. 65–98). Washington DC: U.S. Department of Education, Office of Educational Research and Improvement.

Pithers, R. T., & Soden, R. (2000). Critical thinking in education: A review. Educational Research, 42(3), 237–249.

Ramsden, P. (1992). Learning to teach in higher education. London: Routledge. Reason, J. (1990). Human error. New York: Cambridge University Press. Stone, C., Davidson, L., Evans, J., & Hansen, M., (2001). Validity evidence holistic nursing

practice, July, 65–74. Tapper, J. (2004). Student perceptions of how critical thinking is embedded in a degree

program, Higher Education Research and Development, 23(2), 199–222. The Chinese University of Hong Kong (CUHK) (2006). Strategic Plan. Retrieved March, 18,

2012, from http://www.cuhk.edu.hk/strategicplan/english/ourtenyearvision.html

26  

Thompson, C., & Rebeschi, L. M. (1999). Critical thinking skills of baccalaureate nursing students at program entry and exit. Nursing and Health Care Perspectives, 20(5), 248–52.

Tishman, S., & Andrade, A. (1997). Critical squares: Games of critical thinking and understanding. Englewood, CO: Libraries Unlimited/Teacher Idea Press.

Watson G., & Glaser, E. M. (1980). Watson–Glaser critical thinking appraisal. Cleveland, OH: Psychological Corporation.

Watson, G., & Glaser, E. (2002). Critical thinking appraisal. London: The Psychological Corporation.

27  

Appendix 1

Sample cases illustrating evidence of critical thinking

Pharmacy (opinions from Professors) 1. Prevent adverse drug events 2. So what is best for the patient? 3. Evaluate medical literature and apply this to practice 4. Explore ethical issues in practice 5. Give suggestions to patients re drug-related problems Religious Studies (opinions from Professors and TAs) 1. Understand others’ presuppositions 2. Respect for knowledge 3. Appreciate others’ opinions/ thoughts/ assumptions 4. Analysis and evaluation 5. Creativity 6. Open-mindedness 7. Observation 8. Having one’s own viewpoint, with avoiding bias Computer Science Engineering (opinions from Professor) 1. Problem-solving 2. Creativity 3. Able to identify the problem and give corresponding solution 4. Analysis 5. Able to identify constraints and requirements

28  

Appendix 2

Interview questions (Academic staff)

Centre for Learning Enhancement And Research (CLEAR)

1. In your view, what are the particular components to critical thinking that are essential to

your students?

2. Based upon your experience, do you think students in your Faculty are critical enough? Why and why not?

3. Are your students ready to apply critical thinking in their studies? Why and why not?

4. Are there any specific critical thinking competences required in your field? If yes, what are they?

5. Do you think your students are keen to develop their critical thinking? Why and why not?

6. To what extent, how does your Faculty foster the development of students’ critical thinking?

7. Does the Faculty encounter any difficulty teaching for critical thinking in your undergraduate programmes? What are they?

8. Do you think our CUHK context is favorable to the students’ development of critical thinking? Why and why not?

9. Can you think of any ideas for enhancing critical thinking in your discipline?

29  

Appendix 3

Interview questions (Students)

Centre for Learning Enhancement And Research (CLEAR) 1. As a CUHK graduating student, what would you suggest to enhance critical thinking in

your discipline?

2. In your view, what are some particular components to critical thinking that are essential?

3. Do you know are there any specific critical thinking competences required in your field? If yes, what are they?

4. Do you think you and your fellow colleagues are critical thinkers? Why or why not? How can you demonstrate your critical thinking in your studies?

5. In your view, how important critical thinking is for your studies or daily life?

6. Do you think you and your fellows developed well enough critical thinking skills in your undergraduate studies?

7. What would you do to help enhancing your critical thinking development other than doing your core courses?

8. To what extent does your Faculty foster the development of your critical thinking?

9. In general, are there any constraints in the learning environment (including inside class and outside class) hinder your development of critical thinking?