TEACHING THE ART OE ECONOMIC RESEARCH IN A SENIOR...

TEACHING THE ART OE ECONOMIC RESEARCHIN A SENIOR SEMINAR

by Jessica Wolpaw Reyes*

Abstract

In many academic settings, undergraduate economics majors are asked to do an independent project orthesis in their final year. Because doing research is very different from leaming economics in aclassroom, students are frequently unprepared for this endeavor. This article describes a seminarcourse intended to ease this transition by preparing students to embark on such a project. The firstgoal of the course is to assist students in developing a sound economic research project. The secondgoal is to help them develop the research skills to execute their project. In essence, the course is anabbreviated and accelerated version of some of the leaming that takes place alongside the standardcurriculum in graduate school. The article describes a course that has been designed for a liberal artssetting, and also discusses other possible variations.

Keywords: Capstone, Project, Curriculum, Senior, Thesis, Research

Introduction

An undergraduate honors thesis can provide avaluable opportunity for students to build on theknowledge they have gained in college, to ask aninteresting question, and to use the tools of theirdiscipline to answer that question. As an econom-ics professor at a small liberal arts college, I findthe honors thesis a particularly exciting moment inour students' education. While many studentscome to economics out of their curiosity aboutreal-world events (govemment policy, financialmarkets, development), it is not necessarily so easyfor them to bring the economics they have leamedback to those same real-world events. Through ourown experience, we are all aware of the simplereason for this difficulty: actually doing economicresearch is markedly different from learning eco-nomics in a classroom.

Undergraduate economic majors spend most oftheir time absorbing and understanding well-estab-lished economic theory. They spend some of their

time doing problem sets and other assignments toimprove their understanding of that theory, and tobe able to use the theoretical tools of economicsto answer well-defined questions. They spend amuch smaller portion of their time questioningestablished theory and debating unresolved eco-nomic problems. And, finally, they spend littletime, if any, formulating and answering researchquestions of their own. In many academic settings,economics majors do not embark upon an indepen-dent research project until their senior year.

Thus, while undergraduates leam a lot abouteconomics, most do not leam how to actually doeconomics. They have not been taught how to for-mulate a well-defined and interesting researchquestion, how to design a feasible project to ad-dress the question, and how to go about testinghypotheses to answer the question. Many econo-mists would argue that "doing economics" is notsomething that can be taught - rather, it falls moreunder the rubric of leaming-by-doing. Further-more, many would say it does not happen in the

Associate Professor of Economics, Amherst College, Address: Department of Economies, AC 2201,Amherst College, Amherst, MA 01002-5000, Telephone: (413) 542-8517, Email: jwreyes(5)amherst.edu or [email protected]. I would most like to thank my students in the senior honorsseminars at Amherst College since 2004. I would also like to acknowledge the support and col-laboration of my colleagues at Amherst, particularly Daniel Barbezat, Jun Ishii, Susan Edwards,Walter Nicholson, and Jeanne Reinle. Thanks also to René Reyes. Any remaining enors and omissionsare my own.

Vol. 55, No. 2 (Fall 2010) 111

classroom, but rather in the first years of graduateschool or in a research assistant job.

However, despite the reality of when and howthis leaming does or does not happen, manydepartments expect students to be able to do theirown research as senior majors. Many institutionsoffer accomplished students the option of doing asenior thesis, an honors thesis, or an independentproject in the senior year. Some institutions evenrequire such a project of all economics majors. Inthese situations, it is inevitable that students stmg-gle with making the difficult transition to doingoriginal research, and that faculty stmggle to assiststudents in this endeavor. We all persist in thiseffort because we acknowledge the importance ofthis capstone experience, the knowledge and expe-rience to be gained from actually completing aresearch project of one's own from start to finish.While that motivates the students and the faculty,it does not make the path much easier.

One approach to bridging this gap is to offer ajunior or senior research seminar, or a "thesis"seminar — a course specifically intended to assiststudents in embarking on this research project. Ifwell-designed and well-executed, I believe such acourse can do a good job initiating students intothe art of doing independent economic research.This article describes the specific seminar that Ihave developed for the senior honors thesis writersat a small liberal arts college. This course aimsexplicitly at bridging the gap between passivelyconsuming economics and actively producing eco-nomics. The course is essentially an acceleratedversion of some of the leaming that takes placealongside the standard curriculum in the first yearsof a Ph.D. program in economics. It is essentiallyleaming-by-doing, but stmctured and with help.There are of course many possible approaches; thisis just one approach that has worked in one partic-ular setting. My hope is that this approach, orpieces of it, can be adapted to serve similar needsin other settings.

Course Goals

The senior seminar course has two main goals.The first goal of the course is for each student todevelop a feasible and interesting economic re-search project, one that she or he will pursue as anhonors thesis or senior project. The second goal isto help students develop the skills to do research in

economics and thereby execute their chosen proj-ect effectively. These goals are pursued in parallelthroughout the semester. At heart, the course's oneoverarching goal is to initiate students into the artof doing independent economic research.

The first task - the development of a thesis top-ic' - is done throughout the semester by the student,in consultation with the professor for the course,their peers, and the larger economics faculty. Ideally,students will start the semester with one or two areasof interest (e.g., microfinance in India, health carefor poor children) and even particular questions thatpique their curiosity. A very few students will comein with well-formed project proposals. Through aseries of proposal drafts and opportunities for pre-sentation and discussion, the students work to devel-op a clear, feasible, and interesting thesis proposal.

The second task - the acquisition and develop-ment of economic research skills - is the morefundamental goal of the seminar, but is also bynature more unusual and less tangible. The re-search skills targeted include: formulating a goodresearch question, designing a sound project,researching the literature, critically reading journalarticles, formulating economic theory, analyzingdata, interpreting results, and conveying ideasclearly. The idea is to move students closer tobeing able to do economic research independently.

These goals are clear enough, and I presentthem in this manner to the students right at theoutset. I emphasize that doing research is differentfrom most things they have done before, and thatactually doing research is the best way to leamhow to do it. I try to make it clear to students thatmore is expected of them in this setting, and thattheir own motivation and interest are crucial: thesenior seminar is an active endeavor in which stu-dents must fully engage, not a passive coursewhere they will sit and absorb knowledge.

Laying the Groundwork

The official time at which a senior project beginscan vary widely across institutions. In many set-tings, a senior project takes place almost exclusivelyduring the senior year. In order to prepare studentsto make the best use of the time they have, I believeit can be helpful if the department is able to estab-lish some stmctures that will lay the groundwork,months or years earlier. This can greatly ease the

112 THE AMERICAN ECONOMIST

transition, assisting students in focusing their inter-ests and developing their research skills.

As a foundational experience, nothing comparesto working as a research assistant or doing an inde-pendent research project in some field of econom-ics. Departments can establish simple structures toprovide, facilitate, and publicize such opportunities.At smaller institutions, it can be especially valuablefor students to work as an RA for an economist atanother institution in the summer. Early researchexperience can be instrumental in focusing a stu-dent's interests, as can conversations with facultyat their home institution and elsewhere.

It is also a good idea to make the thesis orhonors track more visible to economics majors intheir junior and sophomore years. One way to dothis is to encourage students to read past theses,possibly even by assigning past theses as readingand discussion material in électives and seminars.Another possibility is to directly involve juniormajors in the thesis process, by encouraging oreven requiring that they attend the presentationsdone by students in the senior seminar. This givesstudents additional insight into what a thesis is andhow it evolves, and they can get a glimpse into thethesis process which they may embark upon in thenext year. This can be supplemented by a meetingor panel in which faculty and thesis writers candiscuss the thesis process with junior majors.

Lastly, it makes sense to set the bar high rightfrom the start, by asking students to read a selec-tion of articles in all fields of economics carefullybefore the semester starts. The list I send usuallyincludes five to ten articles, ranging from acknowl-edged classics to relatively new work. I include thefollowing suggestions with the reading list:

Things to think about while reading: What isthe fundamental economic theory? How doesthat fit in with what you know? What is theempirical strategy? Does that seem like agood idea? What are the results - do theyseem plausible? What do you leam from thepaper? Does it give you other ideas? Gener-ally, you should read carefully, actively, andcritically. Try to get a good sense of what thepaper does.

This is intended to get them started thinking likeeconomists, and ready to start the semester work-ing hard.

Course Structure

Ideally, the course will meet as a small group ofstudents in a seminar format. Because each studenthas his or her own research topic, entirely differentfrom the others and spanning a wide range offields, having a small class size - preferably lessthan 15 students - is essential to making the semi-nar work smoothly. It is also helpful to meet in aseminar-style room, in which students are not sit-ting in small student chairs facing the professor,but rather sitting around a table in a more collabo-rative setting. Structuring the physical environ-ment in a less hierarchical and more collaborativeway can help create an environment conducive tostudent participation and active engagement. Twoclass meetings per week, each lasting an hour anda half, seem to provide sufficient time for thecourse.

Table 1 shows an example of a course schedulefor a course that meets twice a week for thirteenweeks. The course is divided into six two-weekchunks, altemating between "tools" and "topics."This is intended to keep the two principal objec-tives - developing the proposal and developing theskills - proceeding in parallel, giving students timeto develop both their projects and the researchskills needed to complete their projects in thesubsequent semester(s).

Starting Out

In the first class, I set out the two goals of thecourse - leaming to do research and developing athesis proposal. I generally try to give the students arealistic picture of what they are facing by impress-ing upon them that doing economic research is quitedifferent from merely leaming about economic re-search that has already been done. I also emphasizethat, while they should aim to create economicknowledge, they need not completely reworkgrowth theory or do something similarly ground-breaking: it should be interesting and exciting, nottoo easy but not too hard.'' I outline the skills theywill need to do research, and explain how the workof the semester is intended to help them developthose skills: researching the literature, structuringtheir thoughts, thinking like an economist, findingdata, performing empirical analysis, and presentingtheir work.

Vol. 55, No. 2 (Fall 2010) 113

TABLE 1.Sample Course Schedule

Section Week Tuesday Thursday

Intro 1 Introduction to the course- Overview of the course- Discuss topicsBeginning the process- Overview of the research process- How to develop a projectDiscuss summer reading

Topics - discuss- How to read papers (brief)* preliminary proposal and preliminary

literature review due next Monday

Topics and Proposals- How to develop a project- How to write a proposal- Match with faculty, meet w/ facultyTopics - discuss

Library Research- How to write a literature review- Leam about research strategies, findingarticles, developing a bibliography, reviewingthe literature

* proposal l'̂ ' draft due next Monday

Topics

Tools

Topics

Tools

Topics

3

4

5

6

7

8

9

10

11

12

13

14

15

Topics - Proposal draft 1Topics - discuss* EndNote orientation tonight

Topics - discussArticles - how to read

Article discussion (panels 1, 2)* prepare panel presentations and

response papers

(no class)

Article discussion (panels 7, 8)

Topics - Proposal draft 2Topics - discuss

Topics - discuss* continue refining proposal

Data analysis- overview and intro- begin data project in computer lab

* Data analysis or theory project due

(no class. Thanksgiving week)

Topics - Propo.sal 3 (final)Topics - discuss- prepare for presentations

Proposals - Presentations* presentations to the department

Wrap up

Data- overview of economic data- library resources for Data

* work with faculty: meet weekly, set workplan, refine proposal

Article discussion (panels 3, 4)* literature review due

Article discussion (panels 5, 6)

* proposal 2"'' draft due (with faculty comments)

Topics - discuss* find data for data analysis project or

plan theory project

Topics - di.scuss

STATA- introduction to STATA- continue projects

Discuss projects* continue developing proposals

(no class. Thanksgiving week)* final proposal due next Monday

Topics - discuss

Proposals - Presentations* presentations to the department

Note: assignments are marked with an asterisk and listed close to their due date.


After laying out the framework for the course,we take a few minutes for each person to discusstheir research interests and goals for the semester.Students can arrive at the course with a very spe-cific research plan or with only a general idea ofthe area in which they might want to work. This istheir first chance to discuss these thoughts with thegroup. I also find it is helpful if students write thisinformation down on an index card that they passin, adding any particular questions or concems.I encourage students to find one or two other stu-dents with similar interests, and to use these infor-mal groupings throughout the semester as anadditional place to share ideas and seek advice.

The most important purpose of the first class, asin most courses, is to set the right tone for thesemester. In this case, that is one of intellectualengagement, self-motivation, and hard work. I re-inforce this by asking students in the flrst class towrite a shon response paper, answering a questionon one or two of the anieles on the summerreading list. My hope is that their answers willshow that they have started thinking criticallyabout economic research, and also provide somesubstantive material around which to start a discus-sion. I realize that some students will not havedone the reading, or will not have done it carefullyenough. Accordingly, the secondary purpose ofthis assignment is to shock some of the studentsinto realizing that this course is different, in casethey hadn't already figured that out.

The first class can also be a good time to brieflyraise the question of whether they want to do athesis or senior project at all. I emphasize to thestudents that the keys to success, aside from soundeconomic reasoning and hard work, are their owninterest and motivation. Students can gain someinsight into their own preferences by talking topeers, meeting with faculty, and looking at pasttheses in the first week of the semester. If there isindeed a choice to be made, it can help immeasur-ably to select a group of students who are moti-vated and committed and winnow out those whoare uninterested or unprepared.

Beginning the Research Process

In the next few classes, 1 endeavor to give stu-dents basic tools and get them moving on theirprojects. We discuss the process of doing research

in economics more generally, and of writing a the-sis in particular. As we all know, doing research issomething of a leamed skill, and one that under-graduates are unlikely to have acquired throughtheir previous coursework. I set out the steps ofthe research process in broad strokes: the researchquestion, literature review, theory, empirics, inter-pretation, and presentation. I try to give them thelarger picture of how to do research and how todevelop ideas into interesting and feasible projects.I also take time to discuss the scientific method,primarily as a method of structuring the process ina disciplined manner to arrive at a valid answer.Students have a variety of experiences with aca-demic honesty, so it may be wise to take this op-ponunity to lay out basic guidelines for ethical andresponsible research.

Students seem to find this broad overview of thestructure and rhythm of research extremely help-ful, however sketchy and incomplete it may be. Ingraduate school, budding economists are expectedto pick this up as they go along, and most do. Butundergraduates rarely have the luxury of time to dothat, nor are they enmeshed in an active researchcommunity that provides an appropriate leamingenvironment. I believe this is the key innovationof a senior seminar course of this type: to condenseand facilitate this type of leaming - leaming thatusually takes place alongside (and outside) thegraduate curriculum in economics. It is an essentialbuilding block for graduate students, and is no lessessential for undergraduates who are trying to dohigh quality independent research. Undergraduatesseem most surprised by the non-linear and circui-tous path of research: they read carefully honedand edited articles where all the bumps have beensmoothed away, and are blissfully unaware of thecomplex path that was traversed.

Another area in which undergraduates need sub-stantial additional assistance is in finding a goodresearch topic. Many young economists say thatfinding the topic for their dissertation was one ofthe biggest challenges of graduate school. Under-graduates have less of almost everything - time,experience, knowledge, motivation - and so findthis task even more challenging. One school ofthought holds that undergraduates are so unpre-pared to choose a good topic that they should begiven one by a faculty member: the student's con-tribution is then the execution of the research, notthe design. Another school of thought holds that

Vol. 55, No. 2 (Eall 2010) 115

students should be appropriately guided to developthe topic themselves, so that the entire project is oftheir making. Depending on the academic settingand culture, either approach can work.

My own preference has been for the secondapproach. Since fostering student independence isin the spirit of this course, I think it is worthwhileto enable and assist students in developing a goodresearch question. It helps to provide students witha brief roadmap as they embark into this unchartedtenitory: picking a topic area of interest, readingrelated articles, thinking about the issues, talkingto faculty and peers. Many students find it helpfulto keep a research joumal as they develop andfocus their ideas, and I encourage them to do thisin a publicly-accessible blog. The goal is to honein on a well-defined, interesting, and feasible ques-tion. Of course, having an idea of what makesa good project can help structure this process, andI emphasize the following elements: an interestingquestion, a testable hypothesis, clear theory, gooddata, and a methodology that provides insight.With this guidance, most students will find theirway to a topic of their own.

Course Elements: Tbe Tbesis Proposal

The development of a thesis topic is a primarygoal of the course, and it is done throughout thesemester by the student in consultation with the pro-fessor for the course, their peers, and the economicsfaculty. Ideally, students start the semester with oneor two areas of interest and even particular questionsthat they might like to explore. As discussed, stu-dents are given guidance in how to form their ideasinto a feasible research project. They are also givenan outline to follow, and are encouraged to highlightpotentially difficult issues rather than smoothingthem over, in order to facilitate discussion and proj-ect development. Through a series of proposal drafts

and opportunities for presentation and discussion,the students work to develop a clear, feasible, andinteresting thesis proposal.

In the spirit of jumping right in, I ask studentsto write a preliminary proposal in the flrst weekof class: a one-page proposal that has a clear ques-tion, working hypothesis, brief background, explana-tion of why it is of interest, and list of challenges.Accompanying this, I ask them to write a prelimi-nary literature review, making a few substantivepoints by drawing on three to flve articles in thearea. This is a modest, if abmpt, start, andsubsequent drafts of the thesis proposal and the asso-ciated literature review are more extensive and so-phisticated. Figure 1 shows the proposal outline theyare asked to follow. I ask them to keep the proposalsto under two pages. When reading the successivedrafts, I focus more and more on the section whichidentifies challenges and cunent questions.

The discussion of the thesis proposals - ques-tioning, interacting, reading, thinking - is wheremost of the real work happens. This interaction iscrucial to helping the students develop their abilityto "think like economists." Fostering the students'own ability and courage to comment constructivelyon each others' work is an important underlyinggoal of the course, and as the semester progressesthe students do substantially more of that. Mycomments are aimed at helping students focus theirtopics, become familiar with the relevant literature,think carefully about the fundamental economictheory, and consider empirical feasibility.'* Onlinediscussion can take place using wikis, blogs, ordiscussion boards within course-management sites.Blogs and wikis are much less linear, and allow amore fluid and vibrant online discussion to takeplace. A wiki, in particular, allows the thesis pro-posals to be dynamic documents that evolvethroughout the semester based on input from theparticipants in the class. Each student is also

Question: What are you asking, or what is your topic?Area: What is the larger topic area, and how does this fit in?Relevance: Why is it interesting?Theory: What is the analytical framework?Methods: How will you do it? Model, data, analysis.Issues: Any particular challenges, concems, thoughts, or questions?References: List relevant articles.

FIGURE 1. Thesis Proposal Template.


matched with one or two other faculty memberswhose interests line up well with the student's,and expected to consult with those faculty mem-bers at various points during the semester.

By the end of the semester (usually earlier), stu-dents should have developed a finely-tuned thesisproposal that sets out an interesting and feasibleresearch project. At this point they are asked to pres-ent their thesis proposal to the entire economics fac-ulty. Since most students have neither made norobserved many research presentations, we prepareby practicing in class and discussing the elements ofa good presentation. Guidelines, such as the "TopTen Tips for an Effective Presentation" provided bythe Committee on the Status of Women in the Eco-nomics Profession, provide a good foundation. Stu-dents present to the class (a familiar and comfortableaudience), and we provide comments on strengthsand weaknesses. By late in the semester, studentshave generally become sufficiently comfortable inthe group and confident in their own skills to talkand comment freely. This small preparation can pro-duce presentations that are clearer and more focused,with students showing greater confidence in theirability to talk with ease about their projects, answerquestions, and think on their feet.

Course Elements: Tools forEconomic Research

The acquiring of economic research skills is thesecond major goal of the seminar. It is more basic,and also more elusive. These skills include: formu-lating a good research question, framing a goodproject, researching the literature, critically readingjoumal articles, formulating economic theory, ana-lyzing data, interpreting results, and conveying ideasclearly orally and in writing. While these are skillsthat graduate students generally pick up along theway, this course tries to stmcture a process throughwhich undergraduate students leam these skills via acombination of instmction, guidance, and practice.The tools elements can be interspersed with thethesis topic development in a variety of ways.

Thinking like an Economist

Reading, responding to, and discussing economicjoumal articles is an important piece of the course.This is a moment in which students move from

being passive consumers of economic theory tocritical and active consumers who are ready to beproducers. Students read journal articles, write re-sponse papers to those articles, and engage in alively class discussion of the theoretical and empir-ical issues at hand. The idea is to move the stu-dents towards thinking, talking, and acting likeeconomists. I don't want them to say "Yes, I readthat paper, it says x y z." I want them to say "Yes, Iread that paper, and I found x compelling, but I'mconcemed about y and would like to see theauthors do z. Maybe we should read a, b, and cand check with person K to see what he thinks."The response papers provide good material forthese discussions, and two or three students arealso asked to be "on panel" for each article; thismeans they are expected to be the resident expertsduring the discussion.

The reading and discussion of journal articles isinstmmental in teaching students to think like econ-omists. A well-chosen and diverse group of articlescan provide numerous and varied opportunities toleam about context, theory, identification, and pre-sentation. The faculty member can choose somearticles herself or himself, colleagues often haveexcellent suggestions, and students can be asked tosuggest articles of interest to them. I tend to seekout articles that will provide leaming opportunities,not simply classic articles that have stood the test oftime and are universally admired. It's really aboutgetting inside the research process. Students areparticularly intrigued when they are given a chanceto look inside an ongoing research project of theprofessor or an economist on campus to give aseminar; they can read the paper (or even multipledrafts thereof) and hear about the process first-hand.

Analyzing Data

The second major skill we focus on is data anal-ysis. All students will have taken at least oneeconometrics course in which they have learnedhow to test economic hypotheses using data. How-ever, given the constraints on the econometricscurriculum in most institutions, they have general-ly solved well-defined problems using clean data.If they do an empirical thesis, they will need tofind data and design a set of econometric tests, aprocess that will be much messier. The data analy-sis project in this course is designed like a moredifficult lab experiment - students are asked to

Vol. 55, No. 2 (Fall 2010) 117

investigate a very simple economic research ques-tion from start to finish. The assignment is to find adata set (hopefully relevant to their thesis), sum-marize the data, state a hypothesis, and test thathypothesis using the data.

To the uninitiated students, this assignmentsounds simple. As they proceed, students discoverthat each step presents its own challenges, and theyare able to leam techniques for confronting thosechallenges. This assignment also gets studentsstarted using an econometrics package (such asSTATA), and provides some an opportunity to dopreliminary analysis for their project early on.^ Wedo this assignment in a collaborative way - I pro-vide initial advice on strategies and techniques, andthen circulate among the students making com-ments as they work. At the end, we discuss the finalprojects in class, trying to draw general lessonsfrom the experience. Subject to the limits of interestand time, students can be given an opportunity toproduce a revised project based on comments theyhave received. There are other possible variationson this assignment - e.g., asking students to answera set research question using data provided to themor data that they find from a particular source suchas the Inter-University Consortium for Political andSocial Research (ICPSR). However it is done, thisproject moves the students a step closer to produc-ing economics, and often provides a useful buildingblock for their larger project.

Creating Theory

While most students pursue empirical projects,some are interested in more theoretical pursuits. Inrecent years, several students have completed ahands-on theory project that mirrors the data anal-ysis assignment. This is of a much more variednature; it must draw its structure from the particu-lar theoretical question under investigation. Stu-dents can develop a simple proposal where theyintend to investigate a small bit of theory, such asrederiving a theoretical result under slightly differ-ent assumptions, or solving a game under differentincentives or with different players. This is a dif-ferent sort of challenge than the data analysis proj-ect, and requires a bit more creativity and vision.With a focused approach and positive guidancefrom a faculty member, it can work well to movea student along on a theoretical project.

Just as students doing theoretical projects needsome exposure to data analysis, so too do studentsdoing empirical projects need some exposure totheory. This is often begun in class discussion orindividual meetings with faculty, where studentsare forced to make some hard choices and comeup with a simple yet workable model.

Library Research and the Literature Review

There are a number of other research skills thatare important for the students to acquire. Libraryresearch skills can be a good place to start, espe-cially for a generation of students whose mainresearch tools are Google and Wikipedia. Librar-ians are often eager to introduce students to libraryresources, explain how to conduct a literature re-view, and discuss methods for locating and acquir-ing data. In some institutions, librarians may evenbe interested in meeting one-on-one with students,assisting them with their research, and advisingthem on potential avenues.

I have been fortunate to have found a willingcollaborator in the library, and we generally con-duct several sessions together with the class. Thefirst introduces students to basic tools and methodsfor searching the literature, such as EconLit or theSocial Science Citation Index. This is followedclosely by an optional session on how to use bib-liographic management software such as EndNote.I coordinate these sessions with a literature reviewassignment, in order to give the students the toolsthey need to complete the assignment. Interesting-ly, it has worked out well to have the students tryto do a preliminary literature review - 3 to 5 rele-vant articles - before leaming anything from thelibrarian. It seems that the experience of doing itwithout much guidance makes them realize thatthere actually is something useful to leam abouthow to find information in a stmctured manner.Actually writing the literature review providesthem an opportunity to test and develop their criti-cal reading skills; they are asked to provide somestructure and find a narrative thread in what theyread, not merely to summarize blandly.

The second session with the librarian addressesfinding data, and can be held several weeks intothe semester. Students rarely have first-hand expe-rience with the challenges of locating and acquir-ing usable data. Working through some of the


common search methodologies can give themsome idea where to start. Obviously, it is alsoimportant to get them to think clearly about whatthey are seeking - ideal data, serviceable data,unacceptable data.

To be sure, there are other library-related re-search skills, beyond reviewing the literature andfinding data, that students can leam. Moreover,students may come into the class with a broadrange of skills - these may be more substantial asmore institutions make "information literacy" ahigher priority. A course can be flexible enough toaccommodate this diversity, and collaboration witha knowledgeable librarian or someone else from aresearch staff can work out very well.

Writing and Presentation

The final act of economic research is the pre-sentation of the work and the results, generallyorally and in writing. As discussed above, oralpresentation skills can be cultivated through dis-cussion and practice towards the end of the semes-ter. I usually offer straightforward advice abouthow to constmct a clear and compelling presenta-tion, how to deliver remarks with clarity and confi-dence, and how to answer questions intelligentlyand respectfully.

Students generally have had more experiencewriting papers, but not necessarily economics re-search papers. Their writing skills for this purposecan be developed by working through the drafts ofthe thesis proposal and response papers; a smallseminar makes it possible for the professor to pro-vide detailed comments and to have individualconversations with students about how to presenttheir work. Many institutions have writing advisorsor a writing center, and these resources can poten-tially be incorporated. For some students, it is im-portant to emphasize that presentation of the workis not an afterthought, and that good economicsand good writing will reinforce each other. DeirdreMcCloskey's book Economical Writing expressesthis and other choice advice eloquently.

Eeedback and Evaluation

Grades in the seminar are based on all of thework discussed above. In establishing the structureof the coursework and the grading scheme, one

must balance the oft-conflicting goals of providingenough independence so that students are able tofollow their own interests but also motivating stu-dents to do key pieces of assigned work. In mycourse, I have based 40% of the grade on the thesisproposals, 40% on the various "tools" assignments,and the remaining 20% on class participation.

The Thesis Proposals

The thesis proposal is clearly a central piece ofthe course. Accordingly, the thesis proposal repre-sents 40% of the grade, with 10% of the gradeassigned to each of three drafts and 10% assignedto the final presentation. The thesis proposals gen-erally develop continuously from one draft to an-other, although there are usually a few studentswho try out two different proposals at the begin-ning or who switch topics entirely between ver-sions. Proposals are graded based on the quality ofthe research project and the clarity of exposition.Essentially, the proposals are graded based on howthey are progressing towards a feasible and inter-esting economic research project. Accordingly,grading standards evolve as the semester pro-gresses - a proposal that is very good for a firstproposal is probably not as good for a third propos-al. Particularly because students are encouraged tohighlight potential difficulties (such as a weaknessin their theory, a problem with the data, etc.),I grade them not on how polished the proposal isbut on whether it is developing well.

The final presentation is graded separatelyfrom the final proposal, based on the extent towhich the presentation is a clear exposition ofa viable and interesting economic research ques-tion. Students are also expected to handle ques-tions intelligently and be able to engage in adiscussion with the audience about any issues thatarise.

Response Papers

Reading, responding to, and discussing economicjoumal articles is an important activity in thecourse. Students are asked to write brief one-pageresponse papers to the articles under discussionand to write a longer two-page response paperwhen they are "on panel." The response paperscan represent from 20 to 30% of the total grade.

Vol. 55, No. 2 (Eall 2010) 119

The primary criterion for evaluating the responsepapers is the student's intellectual engagementwith the material, and application of strongcritical thinking, sound economic theory, and solideconometrics. Students are not expected to engagewith the anicle on every level and every issue, butmerely to raise a single methodological or theoret-ical issue and show that they are reading activelyand applying their knowledge of economics. If theresponse papers are posted online, students canengage in further online discussion, which can bevery fmitful.

Analyzing Data or Creating Theory

The small project that asks students to analyzesome data or develop a bit of theory is really ask-ing them to do a tiny paper in itself. I grade thisquite simply based on the quality of what they'vedone - not the quality of the results, which areoften largely absent - but the quality of the meth-ods and the effon. I want to see that they've beenable to explain their data by highlighting imponantfeatures without overwhelming the reader with ir-relevant detail, and that they have clearly statedand tested a hypothesis that is at least somewhatinteresting. Given the narrow range of the assign-ment and the shon time-frame, discussion of theshoncomings of the data and the econometric ap-proach are cenainly appropriate (in lieu of actuallyaddressing those shoncomings.) A theoretical piececan be assessed by considering to what extent theexercise was handled in a careful and rigorous man-ner that indicates an understanding of the theoreticalissues at hand.

Class Participation

As is the case in many seminar courses, engage-ment in class discussion is as imponant as it isdifficult to grade. One possible approach is tograde each student based on the extent to whichhe or she seems to be engaging in the whole en-deavor of developing a thesis topic and leaminghow to do research. While this engagement cantake many different forms depending on each stu-dent's character, it is not extremely difficult toassess whether the student is very engaged, some-what engaged, or unengaged. This is my personalmetric, and I try to assess their panicipation in

class discussion, online, and outside of class. Iassign each student a letter grade for each two tofour week period (standards change with therhythm of the semester), and average these at theend, to comprise approximately 20% of the overallgrade.

Student Perceptions

In an ideal worid, students would see the pur-pose of each pedagogical venture with perfect clar-ity. In reality, that is not the case. In this course,students may have different reactions to beingasked to take responsibility for their own researchproject, and to working independently. Many areaccustomed to being guided through a set of care-fully orchestrated educational experiences. Whilethis course is certainly orchestrated, the freedomand personal responsibility can obscure the organi-zation. Not having had the experience of beingthrown into research without any preparation, theymay be more likely to perceive the lack of struc-ture (relative to previous courses) rather than thepresence of structure (relative to being thrown intoresearch alone.) Most students, however, seem toappreciate the structure of the course:^

The expectations were made very clear in thesyllabus and we were always encouraged toask questions and discuss economic issues inclass. I think the class went from being rathersilent and uncertain of how to approach eachother's topics to being very open and con-stmctively critical. Comments went from be-ing general, vague, and not always relevantto being pointed and concise so I think we allleamed to think better in economic terms.

I think the discussions of each other's topics,and the data project, made me feel as if I wasmaking real progress instead of flounderingaround trying to get a foothold on the process.

The challenge, however, is for the students them-selves to create the time to devote to the develop-ment of their own projects. Even with severaldeadlines and assignments, students are alwaysshon of time and can easily shirk this task. It is noteasy to create an appropriate incentive mechanismthat will elicit the optimal effon on their part. In-deed, if one places a high a value on the student's


own motivation and ownership of their independentproject, such a mechanism is not even desirable.There is simply an unavoidable tension betweenproviding incentives and giving them independence.I have tried to address this by exhorting the studentsto make the project their own, to do somethingexciting, to work harder than they have workedbefore. Many seem to appreciate this type ofencouragement and confidence in their abilities:

Aggressive deadlines were very helpful inkeeping us on track.

[HJigh expectations in the beginning meantthat I was always prepared to work hard.I think [the professor] managed to achieve acoherence to the class, despite the great vari-ety of topics and interests of the students.

Thus, while explanations about the ambiguity ofthe research progress are useful, they cannot re-place the actual experience of pursuing an inde-pendent project fraught with its unique challenges.Even at the end of the semester, with little time togive them perspective on the experience, most stu-dents seem to appreciate the value of the courseand what they have achieved:

I think I'm more ready to write a thesis thanI would have been without the seminar - itwas especially helpful for thinking abouteconometric and data issues.

[The class was] great. Leamed how to reallydelve into economics papers, got a good starton a thesis topic that I enjoy, classroom envi-ronment was stimulating and fun... I appre-ciate how [the professor] challenged me.Critical reading of econ papers was very funand helpful.

I enjoyed this class very much, not least be-cause it gave me confidence in my ability toapply economics I have leamed since Eco-nomics 11 and be comfortable discussing itwith my peers.

Overall, the students seem to think they have ac-complished the two primary objectives of thecourse: they have developed decent thesis propo-sals and they have leamed useful research skills.Many of them also seem to have gained an appre-ciation for the joys and challenges of economic

research, and gained confidence in their ownability to bring their economics training to bear oninteresting questions.

Situating tbe Course

The foregoing text describes one way of mount-ing an undergraduate research seminar of this kind.There are, indeed, many many variations on thistheme. No course will work perfectly for each stu-dent, each professor, or each institution; hopefullya course can be flexible enough to work reasonablywell for most. This model can easily be adapted toother institutional settings or cultures, while main-taining a good degree of flexibility. In this section,I discuss some issues related to situating the coursein other contexts.

Regardless of the institutional setting, preparingstudents early and laying a strong foundation canmake things go much more smoothly, if only byoffering students some of these leaming opportu-nities and teaching them some of these skills earlieron. Structures that connect students with researchopportunities either on-campus or off-campus, en-courage them to take challenging seminars, provideopportunities to hear guest lecturers or attend con-ferences, and generally spark their interest in eco-nomic research, can all work to lay a strongfoundation. These goals can be accomplished via aformal "honors track" or more informal means,depending on the institutional culture.

Setting out these opportunities can also providea means for selecting students into the thesis semi-nar or the honors track. Universities and collegesvary widely in the number of economics majorsthey serve, the academic goals of those students,and in the faculty's interest in selecting a certaingroup for this endeavor. All sorts of selectionmechanisms can come into play, from a grade cut-off, to some indication of preparatory work, orbetter yet some sign of commitment to the effort.Whatever the specific process, students willbeneflt greatly from knowing what it means to doa senior project or thesis at that institution - theycan learn this by attending thesis presentations,reading past theses, talking with faculty, or talkingwith alumni who have gone through the process.

The reason selection may be necessary is thatthe course, as described, really does need to be lessthan 15 students to work well. Research is very

Vol. 55, No. 2 (Fall 2010) 121

individualized, and the seminar needs to be able tomake enough space for all of those separate topics.It would be possible to have a larger group andseveral break-out groups, and to incorporate grad-uate student mentors as well. If there are enoughstudents to separate the course by field (even justmicro vs. macro), that could potentially improvethe ability of the group to focus on the particulartopics. The broad point is that this collectiveleaming of somewhat intangible skills, combinedwith the development of specific individual pro-jects, best happens in a small group that can becohesive and supportive.

An additional reason to think about laying afoundation and selecting students is to be able todetermine where exactly to start the seminar. Ifstudents are sufficiently prepared, the course couldbe offered in the junior year. If students have donefoundational work and have started their projects tosome degree, the course can take this additionalpreparation into account. At a minimum, studentsshould have completed their core theory courses; itis advantageous if they have also taken advancedtheory courses and an upper-level seminar in whichthey have had the opportunity to engage with cur-rent research and possibly write an independentpaper. Timing and pre-requisites can vary greatly.

In most institutions it makes sense to build onthe foundation established by the economics majorand any broad university-wide requirements, aswell as to draw on institutional resources. Semi-nars, brown-bag lunches, writing tutoring, computertraining, data support, library training - all of thesecan be brought into service to the economics seniorseminar and project, with very little direct effort bythe professor for the course.

Other resources can also be helpful and can beworked in depending on the professor's preferencesand the students' interests. A small library of rele-vant books - such as Steven Greenlaw's DoingEconomics, Thomas Wyrick's The Economist'sHandbook, or Peter Kennedy's A Guide To Econo-metrics - can be made available to students at spe-cific junctures or just more generally. Students canfind chapters or passages that align with their per-sonal approach or help them with their own individ-ual challenges. A course website can provide aspace for discussion of assigned articles, studentblogging about their projects, or general discussion.Students can be sorted (or sort themselves) intosmall "research groups" with similar interests.

giving each other comments on their proposals andideas as they develop. Other faculty can easilycheck in on student progress if it is posted regularlyonline, and feedback can be shared to whatever ex-tent the various parties are so inclined.^

The underlying theme here is that nearly allelements of the course - timing, specific assign-ments, skills addressed, resources used - can bevaried or modified. The beauty of this is that thecourse can have a fluid stmcture that adapts to thefaculty member's preferences, student preparationand interests, and institutional culture. Some ele-ments may be extraneous in some settings, othersmay be essential, still others may be entirely miss-ing from the above discussion. The course canevolve to meet the needs of the students and theinstitution, with the broad goal of getting themstarted in the right direction with some confidenceand a few useful tools in hand.

Conclusion

While undergraduate economics majors leam alot about economics, they do not generally leamhow to do economics. They have probably notbeen taught how to formulate a well-defined andinteresting research question, how to design a fea-sible research project to answer that question, andhow to go about testing a hypothesis. It is inevita-ble that students struggle with making the difficulttransition to doing original research, and that fac-ulty stmggle to assist students in this endeavor.Those of us who persist in this effort do so becausewe acknowledge the importance of this capstoneexperience, the knowledge and experience to begained from actually completing a research projectof one's own. The collaboration of students andfaculty in this research endeavor can be a stimulat-ing, if challenging, culmination of an undergradu-ate education in economics.

The seminar described above, quite simply, ismeant to help where help is needed. It is intendedto prepare students to do their own independentresearch and thereby to ease this transition andimprove the experience. While many economistswould argue that "the art of doing economics" mustbe Ieamed along the way and cannot be taught, itseems worthwhile to give it a try. The courseI describe aims to be a rigorous and dynamic struc-ture for preparing students to do economic research


and to do their best independent work. The idea isto give the students a strong foundation in most ofwhat they need, help them leam how to figure otherthings out for themselves, and get them on theirway to doing sound economic research.

Notes

1. For most of this paper, I refer to the seniorproject as a thesis, although it of course neednot have that particular designation.

2. For example, a student in a social policy semi-nar once used a former student's thesis (a theo-retical model of homelessness) as the topic of apaper assignment. This was not assigned, butrather found by the student following the pro-fessor's suggestion.

3. Admittedly, "interesting" is an ambiguous andsubjective term. I mean it to be broadly de-fined, primarily indicating that the project hassome economically substantive content and issomething about which the student himself orherself (at a minimum) is curious. Examples ofinteresting projects range widely: the impactof globalization on markets for English foot-ball, a study of home bias in equity markets,intellectual property protection in developingcountries, an investigation of low take-up ratesfor public health insurance.

4. My comments are of various kinds. For the ear-lier drafts, I often ask students to focus theirtopic further, and give them suggestions forhow to do that. I will frequently suggest specificjoumal articles to read, or point them towardstextbooks, chapters in a Handbook, or reviewarticles in the Journal of Economic Perspec-tives. This is also facilitated by their one-on-one meetings with faculty whose interests areclose to their panicular topic. I often push stu-dents to think carefully about the fundamentaleconomic theory - what is the utility function inthe model, who is maximizing what, what is inthe production function, who are the economicactors, what is the nature of the information, etc.I also make many comments about empiricalfeasibility, and often find myself reminding stu-dents that a strikingly interesting question thatremains unanswered may be so because it isextraordinarily difficult to test empirically.

5. I also give them the opportunity for more for-mal training in econometrics packages, whichmany find helpful after they've struggled ontheir own for a while.

6. The quotes in this section are excerpted fromanonymous course evaluations completed bystudetits in the course over the past severalyears. At the end of the semester, studentswere asked open-ended and general questionsabout the course and the professor. These eva-luations were not made available to the profes-sor until after grades had been completed, andwere typed to preserve anonymity.

7. In some ways, these resources can serve as a"grab-bag" of sorts, from which students canpick what works well for them.

References

Bates College Department of Economics. "PreparingPapers and Theses: A Guide for EconomicsStudents." (Online at http://abacus.bates.edu/acad/depts/econ/the_writing_guide.htm)

Committee on the Status of Women in the Eco-nomics Profession. 2005. "Top 10 List: Givingan Effective Presentation," CSWEP Newsletter,Spring/Summer 2005.

Davis, Barbara Gross. 2001. Tools for Teaching.San Francisco, CA: John Wiley & Sons.

Elliott, Donald, John Meisel, and Warren Richards.1998. "The Senior Project: Using the Literatureof Distinguished Economists." Journal of Eco-nomic Education, pp. 312-20.

Greenlaw, Steven A. 2005. Doing Economics:A Guide to Understanding and Carrying OutEconomic Research. Boston: Houghton MifflinCompany.

Kennedy, Peter. 2003. A Guide to Econometrics,5"' edition. Cambridge, MA: MIT Press.

McCloskey, Deirdre. 1999. Economical Writing,2'"' edition. Waveland Press.

Thomson, William. 2001. A Guide for the YoungEconomist. Cambridge, Massachusetts: MIT Press.

Walstad, William B. and Phillip Saunders. 1997.Teaching Undergraduate Economics: A Hand-book for Instructors. Irwin McGraw-Hill.

Wyrick, Thomas L. 2005. The Economist's Hand-book: A Research and Writing Guide. Stamford,CT: Thomson.

Vol. 55, No. 2 (Fall 2010) 123

Copyright of American Economist is the property of American Economist and its content may not be copied or

emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.

However, users may print, download, or email articles for individual use.

TEACHING THE ART OE ECONOMIC RESEARCH IN A SENIOR...

Documents

Transcript of TEACHING THE ART OE ECONOMIC RESEARCH IN A SENIOR...