Page 1 of 28 - Carleton University€¦ · That Senate approve the Final Assessment Report and...

2

2. THAT SAPC recommends to Senate the approval of the Final Assessment Report and Executive Summary of the Cyclical Review of Carleton University’s Bachelor of Journalism program.

Senate Motions

1. That Senate approve the Final Assessment Report and Executive Summary of the Cyclical Review of Carleton University’s graduate programs in Public Policy and Administration.

2. That Senate approve the Final Assessment Report and Executive Summary of the Cyclical Review of Carleton University’s Bachelor of Journalism program.

of 28

1 | P a g e

CARLETON UNIVERSITY COMMITTEE ON QUALITY ASSURANCE

Cyclical Review of the Bachelor of Journalism Program

Final Assessment Report and Executive Summary

This Executive Summary and Final Assessment Report of the cyclical review of Carleton’s Bachelor of Journalism (BJ) program is provided pursuant to articles 4.2.5-4.2.6 of the provincial Quality Assurance Framework and articles 5.1.9.23-24 and 5.1.9.26-27 of Carleton’s Institutional Quality Assurance Process (IQAP). EXECUTIVE SUMMARY Carleton University’s Bachelor of Journalism program is administered by the University’s School of Journalism and Communication, an academic unit in the Faculty of Public Affairs (FPA). The program was categorised by the Carleton University Committee on Quality Assurance (CUCQA) as being of good quality with report. The program was judged to be of good quality. However, a number of concerns were identified. These concerns were:

1. The appropriateness of the curriculum for the profession and the discipline of journalism studies at this place and time given the integrated digital medial landscape.

2. That some of the learning outcomes are not currently being achieved, particularly in the area of multimedia and digital journalism.

3. An historic admissions practice that yields twice the number of students in first year who can be admitted to second year.

4. Lack of appropriate technical support. 5. Challenges faced by faculty with respect to staying abreast of developments in the

professional field of journalism and in having their professional contributions better recognised by the University.

Responding to these concerns has provided opportunities for program enhancement. CUCQA feels the School of Journalism and Communication has taken advantage of these opportunities in a satisfactory manner through initiatives identified in its response to the report of the Review Committee and the Action Plan that details the implementation of these and other initiatives. The Provost and Vice-President (Academic) and Senate of Carleton have approved this Final Assessment Report and Executive Summary and the Action Plan it contains.

Senate - February 28, 2014 Attachment A

of 28

2 | P a g e

FINAL ASSESSMENT REPORT Introduction Carleton University’s Bachelor of Journalism program is administered by the University’s School of Journalism and Communication, an academic unit in the Faculty of Public Affairs (FPA). This Final Assessment Report provides a summary of: The outcome of the review with respect to the strengths of the program and the

opportunities for program improvement and enhancement. Initiatives to be implemented for program improvement and enhancement.

This summary draws on eight documents: The self-study developed by the program (please see Carleton’s IQAP 5.1.1-5.1.5)

(Appendix B). The discussant’s initial report to the Carleton University Committee on Quality

Assurance (CUCQA) on the self-study submitted by the program (IQAP 5.1.9.1.-5.1.9.3) (Appendix C). It should be noted that the discussant had access to the external reviewers’ report from the previous UPR review as well as the response of the program to this report.

The communication to the external reviewers containing the supplementary questions (IQAP 5.1.9.3-5.1.9.6) (Appendix D).

The response of the program to the supplementary questions (Appendix E) The report of the review committee (IQAP 5.1.9.9) (Appendix F). The response of the program and the relevant dean to this report (IQAP 5.1.9.10)

(Appendix G). The discussant’s recommendation report (IQAP 5.1.9.11) (Appendix H). The communication regarding the outcome of the review (IQAP 5.1.9.13) (Appendix I).

Appendix J contains brief biographies of the members of the Review Committee. The Final Assessment Report contains as an Appendix (Appendix A) the Action Plan required by the IQAP (5.1.9.13-14). This Plan was agreed to by the program and the relevant dean (IQAP 5.1.9.18), and provides details on the implementation of plans and recommendations for program improvement and enhancement to have been advanced as a consequence of the cyclical program review process. CUCQA was not satisfied that the Action Plan adequately addressed two serious concerns raised by the external reviewers. This outcome was communicated to the Director of the School of Journalism, the Dean of FPA and the Provost. A subsequent meeting was held between the Provost, the Vice-Provost (Chair of CUCQA), the Dean and the Director of the School. As a consequence, an addendum to the Action Plan was


of 28

3 | P a g e

supplied by the Director with the agreement of the Dean. The communication from CUCQA and the addendum are included as part of Appendix A1. The Final Assessment Report and Executive Summary were approved by CUCQA at its meeting of February 5th, 2014. As required by the IQAP (5.1.9.24), the Final Assessment Report and Executive Summary, together with the Action Plan, was submitted to the Provost for his approval. The Provost approved the Final Assessment Report on February 6, 2014. On February 13, the Senate Academic Program Committee recommended to Carleton’s Senate that the Final Assessment Report and Executive Summary be approved. Senate approved the Final Assessment Report and Executive Summary at its meeting of February 28th, 2014. The Action Plan is summarised in the penultimate section of this Report. Review Outcome: Strengths and Weaknesses The external reviewers reported that the ‘undergraduate program [is] generally strong.’ They also reported that ‘individual courses are strong,’ and that ‘the level of educational and activity expectations [are] appropriate for an undergraduate . . . level curriculum.’ However, the external assessors expressed concerns about a number of aspects of the BJ program. These concerns were:

1. The appropriateness of the curriculum ‘for the profession and the discipline of journalism studies at this place and time give the integrated digital medial landscape.’

2. That ‘some of the learning outcomes are not currently being achieved, particularly in the area of multimedia and digital journalism.’

3. ‘An historic admissions practice that yields twice the number of students in first year who can be admitted to second year.’

4. Lack of appropriate technical support. 5. Challenges faced by faculty with respect to staying abreast of developments in the

professional field of journalism and in having their professional contributions better recognised by the University.

The response of the School to the external reviewers’ report takes issue with one aspect of the conduct of the site visit and with some aspects of the report. The response observes that ‘there was very little discussion of curriculum in meetings that the Director [of the School] attended.’ Surprise was expressed that ‘given their interest in curriculum the external assessors offered no comments or mention of the significant undergraduate curriculum review the School undertook in the fall of 2011’ (the 62-page report included as Appendix D in the self-study). In addition, the School reports that, given the fast-changing and mercurial world of professional journalism, it has adopted a process of annual and therefore evolutionary review of its curriculum that it feels better responds to this environment than a one-time overall review.


of 28

4 | P a g e

CUCQA felt that the observations of the School were well taken, and that the School had responded positively to the external reviewers’ recommendations. Accordingly, CUCQA categorised the BJ as being of good quality with report. The communication from CUCQA to the School of October 17th, 2013 regarding this outcome is attached as Appendix H. It should be noted that, with respect to the report required on or before June 1st, 2015, CUCQA was encouraged by the participation of the School in the University’s pilot project on learning outcomes assessment. However, CUCQA was nonetheless of the view that the concerns of the external reviewers remained legitimate and were in some cases serious. This view is reflected in the requirements of the two reports to be submitted, respectively, by June 1st, 2014 and June 1st, 2015. The Action Plan The Action Plan (Appendix A) was received by CUCQA in mid-December 2013. CUCQA was not satisfied with the initial actions proposed with respect to the lack of technical support and the current admissions process. This dissatisfaction was communicated to the School on January 9th, 2014 (Appendix A1). The meeting requested by CUCQA between the Vice-Provost and Associate Vice-President (Academic) (Chair of CUCQA), the Provost and Vice-President (Academic), the Dean of the Faculty of Public Affairs and the Director of the School was held on January 17th, 2014. An update to the Action Plan was received by CUCQA on January 20 (Appendix A2). As a consequence of this communication, CUCQA approved the Action Plan at its meeting of January 22nd. It should be noted that the deadline for the submission of a report on the admissions process is now due on June 30th, 2015 rather than June 30th, 2014, as required in the outcome of the review process communicated to the School on October 17th, 2013 (Appendix H). CUCQA was in agreement that this deferred deadline was acceptable given the seriousness of the issue and the challenges to be overcome in addressing it. The Next Cyclical Review The next cyclical review of the BJ program, together with that of the MJ programs, will be conducted during the 2020-2021 academic year.


of 28

CARLETON UNIVERSITY School of Journalism and Communication MEMORANDUM DATE: December 12, 2013 TO: John Shepherd Chair Carleton University Committee on Quality Assurance FROM: Christopher Waddell Director School of Journalism and Communication CC: Andre Plourde Dean Faculty of Public Affairs Re: Action plan in response to Outcome of the cyclical review of the Bachelor of Journalism program ——————————————————————————————————————— This note outlines the School of Journalism’s action plan in response to the CUCQA October memo which included a series of requests and notes that two follow-up reports will be filed with CUCQA by June 30, 2014 and June 30, 2015. It is easiest to address the issues raised in the October 17 memo one by one, indicating the action taken or to be taken and expected time of completion for each. Report for June 30, 2014 The addition of technical support to enhance the classroom experience.

With significant financial support from the Dean’s office, the School used some of its own funds to staff the television facilities in the summer of 2013 to upgrade software, solve problems dis-covered during the year, clean up servers from the academic year, prepare for the fall term and make the television studio rentable by outside organizations. Had we not done that, the facility would have sat closed and empty all summer and we would have been unprepared for the fall term. Our sole television technician (compared to five IMS staff dedicated to Journalism when our television facilities were in the Southam Building) is an employee of Instructional Media Services seconded to Journalism from September to the end of April.

Senate - February 28, 2014 Attachment B

of 28

In addition to that sole person, in September the School hired using its own budget a second technician to work part-time for some days of the week providing additional technical support to television (as well as assisting our one IT person who reports to the Dean’s office). This second technician combines that work with a part-time function overseeing equipment distribution in television.

These steps are all stop gap measures until the technical support issue can be addressed more substantively in the School’s 2014-15 budget.

The School’s budget submission for the coming year will include two requests that are essential to provide sufficient technical support for both the undergraduate and graduate courses taught in the School. Both groups of students spend considerable time in television and in multimedia journalism courses that require ongoing technical support.

The budget submission will propose that the School’s television technician (seconded from IMS) be transferred to the School’s books as a full-time member of the School staff, working all year long not just during the academic year. A second person would be hired on a full-time basis and would split his or her time between additional support for the television facilities and providing more general IT support for courses in the program and for faculty members.

While providing essential IT support to current undergraduate and graduate courses, this second person can also play a role in a proposed new Bachelor of Civic Media program now under dis-cussion within the School (see below). Technical support for skills training and teaching in online activities, social media etc. would be needed for this new degree program but in many ways it is an outgrowth of what is currently taught within Journalism, mainly in support of their teaching-related activities.

The Dean of Public Affairs has signaled his support for this proposed restructuring and increase in technical support. He has indicated, however, that there are not the necessary unallocated base funds in the Faculty’s current budgetary allocation to make this much needed investment in tech-nical support.

Finally the School has started to generate revenue from renting its television facilities to external production houses at times when the studio and related equipment are not in use for teaching. We expect this to be a growing source of revenue in coming years that can contribute to the cost of additional technical support for the facility.

Plans to respond to the concern over the admission process for the BJ program in the light of the condition for students to enter second year.

The School is in the early states of developing a proposal for a new degree - a Bachelor of Civic Media - to be delivered jointly by Journalism and Communication faculty within the School. The School’s management committee enthusiastically supports such a new degree. A faculty commit-tee will complete a study by the end of 2013 that will make a preliminary assessment of the fea-


of 28

sibility of such a degree, how it would operate and what additional teaching, infrastructure and technical resources such a degree would require.

Among the elements under consideration for inclusion in such a program of study are how digital media are changing social interactions, politics, communications and communities and how such media are used to engage local, national and international communities on a wide range of issues and causes. The program might also include innovations in communications, new technologies and new uses for existing technologies. It could also offer a professional skills component and workshops designed to test social media and communications theories in journalism as well as in communication and public relations, as well as more traditional academic study. Components of the degree might also involve participation from or courses offered by other units within the Faculty of Public Affairs.

Although it is too early to know the committee’s recommendations, the concept is predicated on the ability of students to transfer into the program after taking the one first-year journalism course (JOUR1000) or from the first year Communication Studies course (COMM1101). We believe this will be an attractive alternative for those who don’t continue in Journalism beyond first year. We also see it as a potential stream of additional students for the School, independent of those admitted to either Journalism or Communication Studies.

The report with its recommendations will be taken to a School faculty meeting in early January for a decision on next steps.

Plans based on the Journalism’s tenure and promotion criteria to support professional fac-ulty members in career development in a research institution. It is expected that research be conducted to identify how other journalism schools understand the contribution of dis-tinguished, standard setting practitioners and how appropriate career development can contribute effectively to continuous curriculum development in journalism programs.

The School’s tenure and promotion criteria written in the spring of 2013 and approved by the Provost in July 2013 detail for the first time a journalism-specific set of requirements for a Jour-nalism faculty member to be promoted to Full Professor. With these guidelines in place, the Di-rector will meet in the winter term 2014 with individual journalism faculty members at the Asso-ciate Professor level to encourage them to pursue activities that would qualify them for promo-tion to Full Professor under the guidelines. At the same time during the winter term 2014 a committee of faculty members will conduct a survey of other journalism schools in Canada and the United States to determine the number of full professors they may have and how they achieved that to provide additional input into discussions about career development the Director will have with faculty members.

Report for June 15, 2015


of 28

The completion of a strategic plan for the BJ program that encompasses a review of the program’s curriculum in the light of the external reviewers’ observation that there is a need for the BJ program ‘to articulate Carleton’s contribution to journalism education in the twenty-first century with degree level expectations that are consistent with emergent professional expectations.’ It is the view of the Committee that an important first step in this process will be the BJ program’s participation in the pilot project on learning out-comes assessment and the basis this provides for curriculum mapping and annual curricu-lum review.

It is expected that a new strategic plan for the journalism program will be completed in the 2014-15 academic year under the leadership of the program’s new Director or Associate Director. The current Director’s term ends on June 30, 2014.

In preparation for that a committee composed of three faculty members and two undergraduate students has begun work as part of the pilot project on learning outcomes and assessment sup-ported by Carleton’s Office of Quality Assurance. The group has met twice and is drafting new learning outcomes that will then lead to curriculum mapping as preparation for curriculum re-views. The outcomes and mapping exercises should be completed in the first half of the winter 2014 term.

Response to recommendations from the review committee

We recommend the classroom experience would be enhanced with additional technical support. As noted above, the School intends to request an additional full-time technical support person in its 2014-15 budget submission to our Dean in January, as well as requesting the current televi-sion technician academic-year secondment be transferred to the School from IMS as a full-time employee. We recommend that the School of Journalism would benefit from developing a strategic plan that clearly articulates Carleton’s contribution to journalism education in the twenty-first century and that degree level expectations are consistent with emergent professional expectations. As noted above, preparations that will help lead to the development of a strategic plan are now under way but much of the work will be done in the 2014-15 academic year with a proposed completion date of June 2015. A learning outcomes committee is at work on learning outcomes in preparation for curriculum mapping in the winter 2014 term. At a minimum, we recommend the School of Journalism engage an external consultant to support the faculty to systematically review and renew its curriculum.


of 28

The School believes that its participation in Carleton’s learning outcomes pilot project will give it a good base from which to review its curriculum systematically. As that process proceeds, en-gaging an external consultant remains a possibility should it be determined that it would assist in reviewing the curriculum and in developing a curriculum review process. We strongly urge Carleton to change its undergraduate admission system. While there are no plans to change the undergraduate admission system for the Bachelor of Jour-nalism degree, the Schools recognizes the benefits to be gained from providing a journalism-related option for those who do not have the grades required to continue in Journalism beyond first year. (Most now continue at Carleton in a BA program.) For that reason a joint Journalism and Communication faculty committee is currently exploring the idea of a Bachelor of Civic Media degree. We believe it would be attractive to those who can not continue in Journalism as well as appealing to some students now in Communication Studies. The program could also at-tract a third group of students from outside those in either program to increase our overall School enrolment. We strongly recommend that the School engage in some comparative research with other journalism schools in North American in order to identify how other schools understand the contribution of distinguished, standard-setting practitioners and support professional faculty members in their career development in a research institution. A committee of School faculty will conduct this research in the winter term 2014 with the results compared to the School’s new tenure and promotion standards for Full Professor. The results of this comparison will be incorporated in career discussions held by the Director with faculty members during the winter 2014 term.


of 28

421 Tory Building, 1125 Colonel By Drive, Ottawa ON K1S 5B6

DATE: January 9th, 2014 TO: Chris Waddell, Director, School of Journalism and Communication FROM: John Shepherd, Chair, Carleton University Committee on Quality Assurance RE: Outcome of the Review of the Action Plan for the Bachelor of

Journalism Program I am writing to convey the outcome of the review of the Action Plan for the Bachelor of Journalism program by the Carleton University Committee on Quality Assurance at its meeting of January 8th. There were two issues addressed in the Plan where the Committee felt that the course of action indicated did not match the seriousness of the concerns identified by the external reviewers. The Committee shares the external reviewers’ sense of the seriousness of these issues. The first has to do with the addition of technical support for the program. The Committee is of the view that resolving this issue of technical support on a satisfactory and continuing basis is crucial to the program’s quality. While the Committee appreciates that some stopgap measures have been taken, that the School intends to make a budget request for the 2014-2015 cycle which, if successful, will resolve the situation, and that the Dean of the Faculty of Public Affairs has signalled his support for this request, the Committee is not encouraged to learn that the Dean ‘has indicated that there are not the necessary unallocated base funds in the Faculty’s current budgetary allocation to make this much-needed investment in technical support.’ Given the seriousness of the issue, the Committee feels that a course of action with a higher likelihood of a positive outcome needs to be agreed by the Dean and the University. The Committee has asked that I take up this matter with the Dean and the Provost. The second issue has to do with the current admissions process that yields twice the number of students in first year who can then be admitted to second year. While the Committee appreciates that the creation of a bachelor’s program in Civic Media will impart useful journalistic and digital media skills to students who may move on to careers other than journalism, the proposed program will nonetheless replicate the role that other programs at Carleton currently play in accepting students who cannot continue into the second year of the Bachelor of Journalism program. The Committee believes the current admissions practice to be of an ethically dubious character, and one which undermines the creation and maintenance of a healthy environment for


of 28

421 Tory Building, 1125 Colonel By Drive, Ottawa ON K1S 5B6

first-year students. The Committee does not believe that the following suggestion represents an ideal solution. However, it would nonetheless propose to the School that it admit only 100 students to first-year journalism, offering places to students who cannot be so admitted in the proposed bachelor’s program in Civic Media. The opportunity would then be provided for a modest number of students who perform well in the first year of this proposed new program to transfer into second-year journalism. The Committee has also asked that I take up this matter with the Dean and the Provost. There are, in addition, three issues identified by the external reviewers that the Committee feels the School should address. While the Committee does not expect that academic units act on all recommendations made by external reviewers, it does expect that all issues will be addressed in Action Plans with an explanation, as and when necessary, of why the unit does not feel it appropriate to act on some of them. The three recommendations are as follows:

1. ‘More than ever, the future of journalism will reside in a multi-platform, digital environment requiring new skills and an entrepreneurial spirit often found in newer organisations. That is also where many of the future jobs will be. Carleton’s School needs to forge closer relationships with these new organizations as a way of introduction to its students.’

2. ‘We did have one concern about the long tenure of the undergraduate director. We recommend that this position rotate among faculty on a regular, scheduled basis.’

3. ‘ . . . other decisions such as funding and faculty-student ratios appear to be having a negative impact on the large number of sessionals in the program. We recommend that the School address the number of sessionals through a strategic plan as well as funding from the Dean’s office.’

I will be attempting to broker a meeting with the Provost, the Dean and you in the immediate future. The Committee would appreciate receiving a revised Action Plan by January 24th. c.c: Dr. Peter Ricketts, Provost and Vice-President (Academic) Dr. André Plourde, Dean, Faculty of Public Affairs Carleton University Committee on Quality Assurance Jessica DeVries, Interim Manager, Office of Quality Assurance Vicki Haydon, Quality Assurance Officer


of 28

CARLETON UNIVERSITY School of Journalism and Communication MEMORANDUM DATE: January 20, 2014 TO: John Shepherd Chair Carleton University Committee on Quality Assurance FROM: Christopher Waddell Director School of Journalism and Communication CC: Andre Plourde Dean Faculty of Public Affairs Re: Response to Review of Action Plan for the Bachelor of Journalism - January 9, 2014 ——————————————————————————————————————— I am writing in response to the above-noted memo outlining the response of the Carleton Univer-sity Committee on Quality Assurance (CUCQA) at its meeting on January 8, 2014 to the action plan for the Bachelor of Journalism program submitted in December 2013 by the School of Jour-nalism and Communication. There are two issues that I think require further clarification and I trust this will assist the com-mittee in understating the School’s planned response to the review of our program and the report of the external assessors. On the question of obtaining additional technical support for the School, as the action plan noted, the School will include a request for additional technical support in his budget submission this month for the 2014-2015 academic year. If successful, we believe this will address our current problems. The Dean of the Faculty of Public Affairs has signalled his support for this request. What I did not make clear in the action plan was that the Dean will include the request in the base budget proposal for the Faculty for 2014-15 that he puts forward in the coming weeks, indi-cating that it reflects the strong recommendation from the external assessors of the Bachelor of Journalism program supported by the Carleton University Committee on Quality Assurance in its review of the journalism program. On the second issue raised in the memo, the current admission process has long been a subject of discussion and debate within the School of Journalism and Communication. There are ad-vantages and some obvious drawbacks to the decisions about how many students are admitted


of 28

into first year of the Bachelor of Journalism program annually and how many move on to the up-per years of the program after first year. At the same time the media world into which our graduates go has been changing quite dramati-cally. Cutbacks, layoffs and concerns about the survival of individual news organizations now dominate the news about the media. At the same time the number of applicants for journalism has been falling in recent years. Still about 800 apply for the 200 first-year spaces taken by those with high school averages of 85 per cent or higher but application numbers seem in a slow but steady decline. We believe that is largely in response to perceptions of the post-graduation em-ployment environment. Six or seven years ago almost 1,300 were applying for first-year admis-sion to journalism. That combination of factors suggests it is time for the School to conduct an examination and re-view of the related issues of first year admission numbers and advancement to second year of the program. The School will undertake such a review with a goal of producing recommendations by June 30, 2015. This review will be conducted in conjunction with our Dean and the Provost and obviously must address the resource implications of any possible changes that might be recom-mended. The January 9 memo also noted three other recommendations contained in the body of the exter-nal assessors report but not included in their summary of recommendations. The School has been aware of and has been acting on these issues. That will continue as resources and circumstances allow. In summary here are the steps the School will take as part of its action plan in response to the external assessors’ report and the reply from CUCQA. Date Action Participants

January 2014

Include request for additional technical support for School in budget submission: Dean to in-clude it in Faculty budget sub-mission

Director and Dean

Winter 2014 Meet with individual faculty and review promotion criteria for promotion to full professor

Director

Winter 2014 Review guidelines at other insti-tutions for promotion to full pro-fessor

School committee appointed by the Director

June 30, 2015

Strategic plan for the journalism program

School committee appointed by the Director

June 30, 2015

Review and recommend future admissions process and ad-vancement to second year of journalism program

School committee appointed by the Director with the involvement of the Dean and Provost


of 28

CARLETON UNIVERSITY COMMITTEE ON QUALITY ASSURANCE

Cyclical Review of the Graduate Programs in Public Policy and Administration

Final Assessment Report and Executive Summary This Executive Summary and Final Assessment Report of the cyclical review of Carleton’s graduate programs in Public Policy and Administration is provided pursuant to articles 4.2.5-4.2.6 of the provincial Quality Assurance Framework and articles 5.1.9.23-24 and 5.1.9.26-27 of Carleton’s Institutional Quality Assurance Process (IQAP). EXECUTIVE SUMMARY Carleton University offers a diploma, master’s and doctoral program in Public Policy and Administration. These programs are administered by the University’s School of Public Policy and Administration (SPPA), an academic unit in the Faculty of Public Affairs (FPA). As a consequence of the review, the programs were categorised by the Carleton University Committee on Quality Assurance (CUCQA) as being of good quality with national presence and report (Carleton’s IQAP 5.1.9.12). The report of the external reviewers (the Review Committee) spoke glowingly of the School’s national profile and reputation, the support the School and its programs have received from the senior administration at Carleton, the quality of the programs, the quality of course material, the performance of the faculty, the quality of the Schools’ management, and the appreciation of students for the School’s programs and its faculty. Nonetheless, the Review Committee felt that while ‘the School will continue to satisfy quality requirements’ without considering program enhancements, it would ‘run the risk of forfeiting its leadership role in the country, as other programs innovate and improve in response to changing demands, student expectations and new opportunities.’ Accordingly, the Review Committee identified a number of opportunities for program improvement and enhancement: the clarification of learning outcomes; strengthening the professional components of master’s offerings; rethinking core offerings in the master’s program; re-thinking written exercises to better approximate assignments anticipated in government; involving practitioners more in classroom and extra-curricular activities; expanding co-op opportunities; better synthesizing the disciplines of political science and economics in the core offerings of the doctoral program; acquainting doctoral students with the research tools current among public servants; and clarifying the objectives of the diploma program and its relationship to accreditation as a processional evaluator.

1 | P a g e

Senate - February 28, 2014 Attachment C

of 28

CUCQA was impressed with both the comprehensiveness and detail of SPPA’s response to the opportunities for program improvement and enhancement identified by the Review Committee as evidenced in SPPA’s response to the report of the Review Committee and the School’s Action Plan. CUCQA was impressed also by the ambitious character of the Action Plan in terms of initiatives, timelines, and the amount of work that had already been achieved at the time this Final Assessment Report and Executive Summary was drafted.

2 | P a g e


of 28

FINAL ASSESSMENT REPORT Introduction Carleton University’s graduate programs in Public Policy and Administration are administered by the University’s School of public Policy and Administration (SPPA), an academic unit in the Faculty of Public Affairs (FPA). This Final Assessment Report provides a summary of: The strengths of the above program Opportunities for program improvement and enhancement Initiatives to be implemented for program improvement and enhancement.

This summary draws on eight documents: The self-study developed by SPPA (please see Carleton’s IQAP 5.1.1-5.1.5) (Appendix B). The discussant’s initial report to the Carleton University Committee on Quality

Assurance (CUCQA) on the self-study submitted by SPPA (IQAP 5.1.9.1.-5.1.9.3) (Appendix C). It should be noted that the discussant had access to the external reviewers’ report from the previous OCGS review as well as the response of the program to this report.

The communication to the external reviewers containing the supplementary questions (IQAP 5.1.9.3-5.1.9.6) (Appendix D).

The response of SPPA to the supplementary questions (Appendix E). The report of the review committee (IQAP 5.1.9.9) (Appendix F). The response of SPPA and the relevant deans to this report (IQAP 5.1.9.10) (Appendix

G). The discussant’s recommendation report (IQAP 5.1.9.11) (Appendix H). The communication regarding the outcome of the review (IQAP 5.1.9.13) (Appendix I).

Appendix J contains brief biographies of the members of the Review Committee. The Final Assessment Report contains as an Appendix (Appendix A) the Action Plan agreed to by the program and the relevant deans regarding implementation of plans and recommendations for program improvement and enhancement to have been advanced as a consequence of the cyclical program review process. The Action Plan provides an account of who will be responsible for implementing the selected plans and recommendations, as well as of the timelines for implementation and reporting.

3 | P a g e


of 28

Overall Strengths of the School and its Programs

1. ‘No other School can claim to have a longer or more distinguished history of contributions to the public sector in Canada. Furthermore, Carleton’s School has shown an awareness of and responsiveness to new situations and opportunities as demonstrated by its willingness to experiment with new programming.’ (Appendix F, pp. 1-2)

2. ‘The senior administration at Carleton clearly supports the School and its programs and

seeks to sustain Carleton’s enviable reputation in policy and administration.’ (Appendix F, p. 2)

3. ‘From a quality perspective . . . the programs are meeting their objectives, the

management of the School is excellent, and faculty performance is strong.’ (Appendix F, p. 2)

4. ‘Overall, the quality of the course material is high and there are no deep deficits in resources or commitment.’ (Appendix F, p. 2)

5. ‘[Students] were articulate and forthcoming regarding their experiences and, for the most part, quick to praise the program and their professors.‘ (Appendix F, p. 2)

Opportunities for Program Improvements and Enhancements General Observation The Review Committee felt that while ‘the School will continue to satisfy quality requirements’ without considering program enhancements, it would ‘run the risk of forfeiting its leadership role in the country, as other programs innovate and improve in response to changing demands, student expectations and new opportunities.’ (Appendix F, pp. 2-3) The following opportunities for program enhancement should be understood within this context. Much of the text that follows is drawn from the report of the Review Committee. Master’s Program

1. Clearly identify the learning outcomes of the master’s program in the context of its dual objectives of professional and academic education.

2. Strengthen the professional content of the program and test its relevance by consulting

practitioners on the skills and knowledge currently in demand in the public sector.

4 | P a g e


of 28

3. Consider offering a shorter program in addition to the current two-year master’s program that would provide students with the option of a more intensive professional experience.

4. Reconsider the core offerings in the light of clear program learning outcomes, in particular the number and orientation of courses that are intended to remediate deficiencies in the backgrounds of master’s students. Establish clear learning outcomes for these courses.

5. Ensure that courses central to the educational objectives of students are taught in a consistent manner with a minimum of variation in structure and assignments from instructor to instructor.

6. Reconsider the number and character of the streams in the program, understanding that students need to be conversant with both policy and management.

7. Respond to student desire for writing exercises that better approximate assignments anticipated in government.

8. Increase the involvement of practitioners in the classroom and in extra-curricular activities as a means of enhancing student experience and ensuring course relevance. Alumnae from the program constitute a valuable source of practitioners.

9. Expand the co-op placement offerings available to students and improve the placement process.

Doctoral Program

1. Clarify the program learning outcomes.

2. Within this context, create a genuinely interdisciplinary synthesis of the core disciplines of political science and economics: organize core courses around theoretically compelling problems of policy and administration rather than around topics that map onto these two disciplines.

3. Acquaint students with the research tools that are in current use among public servants: respond to the student desire to acquire at least an acquaintanceship with contemporary methodologies, their assumptions, and their use in the field.

Diploma Program

1. Clarify the objectives and learning outcomes of the program.

5 | P a g e


of 28

2. Clarify the relationship between the program and accreditation as a professional evaluator.

Initiatives to Be Implemented for Program Improvement and Enhancement CUCQA was impressed with both the comprehensiveness and detail of SPPA’s response to the opportunities for program improvement and enhancement identified by the Review Committee as evidenced in SPPA’s response to the report of the Review Committee (Appendix G) and the School’s Action Plan (Appendix A). CUCQA was impressed also by the ambitious character of the Action Plan in terms of initiatives, timelines, and the amount of work that had already been achieved at the time this Final Assessment Report and Executive Summary was drafted. Staff of Carleton’s Office of Quality Assurance has acted as facilitators for some of the workshops identified in the Action Plan. Conclusion As a consequence of the review, the program was categorised by the Carleton University Committee on Quality Assurance as being of good quality with national presence and report (Carleton’s IQAP 5.1.9.12). The Next Cyclical Review The next cyclical review of the graduate programs of the School of Public Policy and Administration will be conducted during the 2020-2021 academic year.

Dr. John Shepherd, FRSC Vice-Provost and Associate Vice-President (Academic)

Chair, Carleton University Committee on Quality Assurance February 7th, 2014

6 | P a g e


of 28

1

QUALITY ASSURANCE ACTION PLAN SCHOOL OF PUBLIC POLICY AND ADMINISTRATION

January 2014 The Quality Assurance review has been of enormous benefit to the School of Public Policy and

Administration (SPPA) in undertaking a process of renewal that strengthens each of the programs

that were the subject of cyclical review in 2013: the Master of Arts in Public Administration (MAPA),

PhD in Public Policy, and Graduate Diploma in Policy and Program Evaluation (DPE). Many of the

issues raised by the consultants, and reinforced by the Carleton University Committee on Quality

Assurance (CUCQA), were concerns that had been raised internally during the review process, and

SPPA has begun steps to act upon the recommendations.

This Action Plan outlines the goals, process, and schedule of how SPPA intends to have a report on

curriculum redesign (with the establishment of clear learning outcomes) submitted to CUCQA by

June 2014 that addresses its primary recommendations and identifies any resources or changes to

governance structures needed to support such change. Each program is addressed separately, as

the process is different for each, and the overall timing is summarized in a final table.

Master of Arts in Public Administration

The primary recommendations, paraphrasing the external reviewers’ report, are:

1. Identity and better communicate the primary learning outcomes of the program,

particularly the balance (or choice) between its theoretical vs. practical orientation, and how

this translates into skills and knowledge in its core courses, the name of the degree and its

interdisciplinary nature (points 1 & 4);

2. Reassess the length, content and pedagogy of the ‘core’ courses given the considerable

dissatisfaction expressed by students, and address the lack of consistency across sections of the

same core courses (points 2 & 3);

3. Address the configuration of the ‘streams’ or concentrations of electives, particularly whether

the current distinction between the ‘policy’ and ‘management’ concentrations is meaningful and

useful (point 5);

4. Review the pedagogy and opportunities for experiential learning, addressing the desire by

students to have more problem-oriented approaches that are relevant to careers in the public

sector and the potential for some kind of capstone experience (point 6);

5. Make better use and integration of adjunct faculty who are experienced professionals into core

and elective courses (point 7); and

Senate - February 28, 2014 Appendix D

of 28

2

6. Ensure the strength of the coop program is maintained as an important vehicle for recruitment

and a means for students to gain professional experience, with a view to reducing reliance on

the federal government and seeking placements in other orders of government and in the

nonprofit sector (point 8).

In taking seriously the consultants’ recommendation that, while the program is of high quality, the

MAPA requires ‘attention beyond tinkering,’ a process that promotes genuine innovation has already

begun. This process is led by the SPPA Review Committee (RC), but is a participatory process engaging

faculty at all stages of the process and students at ‘critical moments’ of reflection.

The phases of the strategy are to: 1) identify the essential skills and knowledge that graduates of the

program need; 2) frame these as learning outcomes; 3) translate into overall curriculum design and

pedagogy; 4) develop specific curriculum content; and 5) review and enhance pedagogy and consistency

of delivery. The intent is to do this a way that is student-centred, rather than being based on the existing

curriculum: that is, to undertake a creative process that enables ‘more than tinkering.’ This strategy will

be implemented in five phases:

1) Identify essential skills and knowledge for graduates (November – December 2013):

Debrief with student representatives (by the Director and at an SPPA Management Committee)

to ensure their concerns presented to the consultants were accurately reflected in the report

and understood by the RC. This was done in November 2013, and it confirmed the concerns

about the content and pedagogy of the core, the need to reconsider concentrations, and the

desire for more experiential learning and problem-solving content.

Compare ourselves with ‘aspirational’ programs internationally, drawing on the research by

Chancellor’s Professor Leslie Pal for his SSHRC grant on international public management and

public policy curricula. Faculty were asked to review and discuss the features of other programs

they felt could be particularly relevant to the MAPA. See

http://portal.publicpolicy.utoronto.ca/en/MPPMPAPrograms/Pages/default.aspx.

Reflect upon the knowledge and skills for future leaders in the public sector as identified by

alumni who have made particularly successful careers in the public sector. Assessments from six

such alumni have been shared with faculty and students. In addition, a number of recent reports

on public sector reform in Canada and internationally have been shared with faculty and

students and used as an input into subsequent discussions.

‘Crowdsource’ ideas about learning outcomes and curriculum reform. A discussion board has

been established on CULearn in which all faculty and student representatives have been

‘enrolled,’ with encouragement to be creative.

Identify and prioritize essential skills and knowledge. A half-day discussion with faculty

(facilitated by Professor Emeritus Ann Clark-Okah and Andrea Thompson from CUCQA) was

held on November 29, 2013. A list of essential skills/knowledge was developed and the top

priorities were ‘voted’ on. A summary of this discussion prepared by the facilitators was

circulated to faculty.


of 28

http://portal.publicpolicy.utoronto.ca/en/MPPMPAPrograms/Pages/default.aspx

3

2) Develop and Review Program Learning Outcomes (January – mid February 2014)

A creative approach to articulating program learning outcomes entails working from the essential

knowledge and skills, rather than from the existing program, in several steps:

Identify Program Learning Outcomes. A full-day faculty retreat (again facilitated by Clark-Okah

and Thompson) was held on January 10, 2014 at which: 1) consensus was reached on the

priority skills and knowledge identified at the November session; 2) draft program outcomes

were written (after a lesson on writing learning outcomes); and 3) implications for curriculum

design discussed.

Refine and write learning outcomes: A follow up session on January 24 will continue the

discussion about curriculum redesign, leading to refined draft learning outcomes; the RC will

circulate a draft; feedback from students will be incorporated.

3) Translate Program Learning Outcomes into Curriculum Design (February – mid March 2014)

Some of the implications of the re-formulated learning outcomes have already become apparent and

these will be articulated and agreed upon by faculty and tested with students by mid March, as led by

the RC. By mid March, it is expected that parameters for the core of the program, configuration of

concentrations and potential for experiential learning will be identified. A mandate for more detailed

work by sub-committees will be provided.

4) Develop Specific Curriculum Content (March – May 2014)

This step involves identification of specific courses and their learning outcomes led by sub-committees

of faculty with subject area expertise. This will occur in two stages:

Content and pedagogy for the core of the MAPA, from March to mid-April

Review and revision of elective courses, with identification of their primary learning outcomes,

from late March to late April, with precise timing depending on progress of the first stage. A

facilitated discussion in late April should reach agreement on these; formal agreement by the

SPPA Management Committee would follow.

On this basis, the final report (prepared by the RC and reviewed by MC) would be ready in mid

May, be reviewed again, to go forward to CUCQA by the end of May.

With knowledge of the revised, curriculum, the School can identify and recruit Contract

Instructors, Adjunct Professors and ‘Senior Fellows.’ Plans for the launch of a ‘fellows’ program

that would recruit 3-4 high profile professionals who will take an active part in the life of the

School/Faculty will be developed during this period, including discussions with the Dean of FPA

as to coordination with other units and any possible resourcing implications (as part of the SPPA

2014-15 budget proposal).

The future of the coop program, with a view to diversifying it, will be discussed with the Deans

of FPA and FGPA, and potentially then senior administration and/or the Coop Office.

An issue that may need to be addressed, depending on the specifics of curriculum redesign, is the

adequacy of teaching resources for management-related (or other) courses. There is no expectation

that net new teaching resources are needed, but a strategic hiring plan for faculty replacements should

be prepared, and curriculum redesign may affect the recruitment of specific Contract Instructors.


of 28

4

5) Review and Enhancement of Pedagogy and Consistency of Delivery (May – July 2014)

The final stage is a more informal, ongoing process that involves coordination among faculty who teach

multiple sections of the core courses, refinement of course content and pedagogy, and intersections

with the other SPPA programs. This Committees of the core instructors will review specific content and

syllabi beginning in May and this process needs to be undertaken on an ongoing basis, but particularly

during the preparation of course syllabi in summer (for fall term) and late fall (for the winter term). The

idea of creating a teaching improvement committee to encourage sharing of teaching innovations

among faculty has been discussed as well.

This schedule would enable all of the points raised in the QA review to be addressed, produce

curriculum redesign of the Master’s program, including the establishment of clear learning

outcomes, to report by the deadline of June 1, 2014 that has been established by CUCQA.

PhD in Public Policy

The QA review recommendations for the PhD are not as extensive as for the MAPA. The specific

recommendations are that:

1. A more coherent interdisciplinary experience needs to be created, that may involve more

integration of core courses and genuine team teaching and that address compelling problems of

policy and administration. The remedy for this is partly curriculum redesign, but also lies in

pedagogy and how the culture of the School is communicated; and

2. Inclusion of a course (or other means) to provide students with suitable research tools.

On the creation of a more satisfying interdisciplinary experience, which is the much broader

recommendation, there is much that can be learned from the Masters redesign which will have

addressed similar issues by the end of February. A PhD subcommittee of the RC (including faculty and

students) was formed during the QA process to take the lead on some of the changes we had already

recognized needed to be made. By late January this subcommittee will be expanded to better address

the first recommendation. Most of its work will occur after the bulk of the Masters review so as to draw

on what is learned through that process.

The plan is:

Late March: The PhD Subcommittee circulates a draft proposal that articulates the program

learning outcomes for the PhD, including proposals for creating a more coherent,

interdisciplinary approach.

Early April: The proposal is discussed by faculty and PhD students at a session following

Management Committee.

Late April: The Subcommittee presents to Management Committee a revised proposal with

learning outcomes; courses (including research methods/tools); and pedagogy.


of 28

5

May: The proposal is revised, as needed, with final review by Management Committee in late

May and submission by to CUCQA by the deadline.

Graduate Diploma in Policy and Program Evaluation

The DPE curriculum and pedagogy have undergone annual (informal) review and adjustments for several

years, and we believe is currently working quite well. No major curriculum review or overhaul was

recommended by the external reviewers. Their recommendations for the DPE are one very specific

recommendation, which can be readily accomplished, and a comment on future development which

does not directly pertain to quality improvement of the program.

The reviewers noted that the relationship between the program and certification as a professional

evaluator (which is the responsibility of the Canadian Evaluation Society) is unclear to students. Clearer

information on the website, in materials provided to students and in the orientation for new students

should eliminate this confusion. The website and other relevant materials can be updated before June

2014.

The second point pertains to the future of the DPE, which is to continue with the in-class delivery format

primarily aimed at domestic students (as it currently operates) and to develop an online format for

delivery primary to an international audience, building on the success of IPDET. This is not an

improvement to the current program but an extension of it to a different audience with a different

delivery mode using the same overall curriculum but with revision to content. It is likely that SPPA could

develop an online format in 2015-16, prefaced by market research, but this would take additional

instructional and course development resources and requires substantial time. Given all the other

major initiatives currently being undertaken by SPPA, a detailed implementation plan cannot be

prepared before June 2014, but its potential will be explored over the next year.

Changes in Organization, Policy or Governance

No major changes to the organization or governance of the programs or the School are needed to

implement these recommendations. The proposals for reform are being developed in a participatory

manner with guidance from the Review Committee and receive formal approval by the SPPA

Management Committee, as is the School’s longstanding practice. Each program has a Graduate

Supervisor who will manage and monitor implementation, reporting to the SPPA Director. The current

director’s term will end on June 30th 2014, so for purposes of continuity, it will be helpful to have the

plan for curriculum redesign completed by then, with implementation the responsibility of the incoming

director. In the transition process, any recommendations for improved governance will be discussed

between outgoing and incoming directors, so that changes may be introduced later in 2014.

Better marketing: The need for much better marketing is noted in the QA recommendations and fully

recognized by the School. Small improvements to marketing – those that can be achieved with no

additional financial resources – are already underway with the assistance of FGPA. This includes social

media ads and a video featuring alumni as part of the celebration of SPPA’s 60th anniversary in March

2014. The consultants suggest that the school be provided with one-time funds to better market its


of 28

6

degrees and to align its messaging with the new university plan. This would be a welcome injection,

particularly if major curriculum redesign is to be announced, and will be discussed with both Deans.

Summary of the Timeline

Program / Timing

Vehicle/ Responsibility Objective / Deliverable

Masters

November - December 2013

Identify essential skills and knowledge

Early November Debrief / feedback from student representatives

Briefing to MC on extent to which the views shared with consultants are widespread

December 1 Feedback from alumni on essential knowledge

Assessment shared with faculty and students

Available Notes by Professor Pal on ‘aspirational’ programs

‘Atlas’ used to inform discussion and ongoing resource

Ongoing Crowdsourcing ideas (online forum)

Innovative list of ideas and options

November 29 Half day facilitated discussion

List and identify of priority skills and knowledge; faculty agreement on priorities

January – Mid February 2014

Develop and Review Program Learning Outcomes

January 10 Full day, facilitated faculty retreat (with lesson on writing outcomes)

Concept for program learning outcomes; initial ideas for curriculum redesign

January 24 Half day faculty discussion Continued discussion of concept for curriculum

Early February Review Committee Draft plan for curriculum design

Mid February Review Committee Test ideas with students

Feedback incorporated

March Translate Program Learning Outcomes into Curriculum Design

Early March Review Committee Draft of curriculum design with a focus on the core, incorporating student feedback; Mandate for subcommittees

March - May Develop Specific Curriculum Content

Mid April RC Subcommittees Proposal for core courses with draft learning outcomes

Late April Review Committee Proposal for concentrations (drawing on earlier discussions)

Late April Faculty discussion, testing with students

Agreement on proposal for curriculum redesign and primary pedagogy

Late April Management Committee Agreement on proposal for curriculum redesign

Mid May Review Committee Adjustments as required; package ready for CUCQA, review with faculty and students

May Director with input from faculty

Develop and recruit Senior Fellows

May Director with Graduate Discussions with Deans (and Coop Office) regarding


of 28

7

Supervisor Coop

May 30 Review Committee Submission of plan to CUCQA

May - July Review and Enhancement of Pedagogy and Consistency of Delivery

June - ongoing Individual faculty/instructors

Coordinate consistency across core courses; enhance pedagogical approaches; refine course content

May - Ongoing Better Marketing

PhD

Late January PhD Subcommittee Current PhD subcommitee of RC expanded, and follows the Masters process in regard to interdisciplinarity and pedagogy

Early March PhD Subcommittee Draft proposal with learning outcomes and core curriculum design

Mid April Faculty and student discussion

Feedback to Subcommittee; proposal revised as needed

Late April Management Committee Agreement on proposal

Mid May PhD Subcommittee Proposal revised, as needed; curriculum content and learning outcomes proposed; final agreement

May 30 Review Committee Submission of report to CUCQA

DPE

February - March DPE Supervisor Revisions to website and other materials to clarify certification responsibilities

April - June DPE Supervisor Review and adjustment to DPE curriculum (no major changes planned that require CUCQA report)

June - December Director/DPE Supervisor Potential for online delivery assessed


of 28

Page 1 of 28 - Carleton University€¦ · That Senate approve the Final Assessment Report and...

Documents

Transcript of Page 1 of 28 - Carleton University€¦ · That Senate approve the Final Assessment Report and...