Long Assignment for Non-Experimental Research Methods and Statistics

Psychology 303

David Buck Psychology Department

Introduction for Faculty Colleagues

This is a term-long writing assignment that is designed to provide a structure to the Non-experimental Research Methods course. The objectives for the course are for the students to learn the fundamentals of research design in psychology and statistical analysis of quantitative data through a highly structured and closely mentored experiential learning opportunity. The course itself will be designed to mirror the research process. That is, the beginning of the term will focus on how to read psychological research, conduct background research, and develop hypotheses. From there, the focus will shift to how to design a study to test a specific hypothesis, then how to analyze different types of data, and finally how to interpret results and draw conclusions from results. Lecture time will be devoted to discussing each step of the research process, and separate handouts will be made available to students that describe each step of the assignment in more detail. My goal is for students to apply the lessons learned in class to their projects and to learn through first-hand experience about the decisions that psychologists have to make when testing a hypothesis and interpreting results. By allowing them to conduct research on a topic of their choosing, I am hoping that students will become more engaged with course material. This is a group assignment for two reasons. The first is logistical; I want students to actually collect data, and multiple students working on the same project makes that more feasible in a one-term course. The second reason is because research is very rarely conducted in isolation. Collaboration is very common and can be tremendously helpful for developing ideas or coming up with new ways of looking at old ideas. Below, I’ve included the handout that students will receive with the syllabus that generally describes the long assignment and the rubric for the final manuscript. Each step of the process will have its own handout that describes the specific deliverable in more detail, along with how it will be evaluated. I’ve included the information sheet for the annotated bibliography as an example of what these handouts will be like.

The Assignment I Distribute to Students

PSY 303 - Non Experimental Methods and Statistics Research Project

Assignment Over the course of the term, you, along with three to four other students, will be taking on the role of research psychologists, designing and conducting a study, analyzing data, and writing an APA style manuscript suitable for publication in a psychology journal. Process Writing the paper will be a process made up of several smaller assignments and drafts, which you will turn in for feedback at each stage. Specific assignment handouts will be provided for each of the component assignments. This project will require significant out of class work. You are expected to meet with your group on a regular basis in order to keep up with deadlines. You are required to meet at least seven times outside of class and record minutes during these meetings, which will be turned in for credit with the final manuscript (see the meeting minutes handout for more information). The final draft of the paper will be turned in at the end of the term. It might be helpful for you to consult your syllabus for due dates, and write them in the spaces below so that you have a convenient reference here.

Step Skill Assignment Points Due Date

1 Conducting background research

Annotated bibliography and Hypothesis 10

2 Using evidence to support a hypothesis Introduction 10

3 Designing a study Research Prospectus 10

4 Analyzing and Interpreting Data Results 10

5 Drawing conclusions and evaluating your work Discussion 10

6 Putting it all together Research Manuscript 100

Effective collaboration Meeting Minutes 35

Learning Objectives Through this assignment, you will learn a variety of skills necessary for a research psychologist. You will summarize a body of research, use existing research as evidence to explain the rationale for a hypothesis, describe a method for testing your hypothesis, interpret data, draw conclusions based on your data, evaluate your research and its limitations, and identify future directions for study. Final Product The final product that you turn in will be the completed research manuscript. It will be due at the end of the term. It will need to be in APA format, and it will need to include a title page, an abstract, at least one APA style table, a references list, and the four main content sections: Introduction, Methods, Results, and Discussion (a general description of these sections can be found below; more details on the recommended content and structure of each section will be provided in class and in the reading). You will need to include a minimum of four peer reviewed journal articles in your references.

Section 1 – Introduction The introduction is where you identify the problem that you plan to study, and present the reader with some background on the topic. You should think of the introduction as a position paper. You start by introducing a problem or research question. Then you propose an answer to the research question. This is your hypothesis. The body of the introduction is used to summarize existing literature related to the problem, and to explain how this literature leads you to believe that your hypothesis is true. By the end of the paper, the reader should be able to follow your rationale and understand how you came to your conclusion. Section 2 – Methods The methods section is where you tell the story of what you did to test your hypothesis. This section is written so that another researcher could, with the information that you provide, replicate your study. There are three parts to a methods section: Participants, Procedure, and Materials. The participants section is where you describe your sample, including sample size, relevant demographic information, and how they were recruited. The procedure section is where you describe what you had the participants do. This begins with them signing an informed consent and ends with them being debriefed. The materials section is where you describe your variables. Section 3 – Results The results section is where you explain how you analyzed your data and interpret the resulting statistics. Run your analyses and then write a

draft of this section without including any numbers. Once you have written the draft without numbers, go back and insert the appropriate statistics. Section 4 – Discussion In your discussion section, you should offer your interpretation the results and what they mean. You should also highlight how your findings relate to the previous literature, “next steps” in terms of research on the topic, and any problems (i.e., limitations) you see with the study.

Step 1 – Annotated Bibliography and Hypothesis At this point, you should have done some initial background research and developed a hypothesis with the other members of your group. Now you need to begin formally constructing the argument in support of your hypothesis, which will comprise the introduction of your paper. The first step is to go back and do a thorough search of the literature to find other research related to your topic. Look for research that is related to your hypothesis or the variables that you are going to measure. Assignment Summary This assignment should be conducted independently. Each group member should conduct their own literature search and turn in their own bibliography.

1. State your group’s research hypothesis 2. Conduct background research, using Psych Info, and find at least four

relevant articles 3. Create a citation for each article in APA format 4. Beneath each citation, write a brief annotation (~250 words each) for

each article. 5. Print the first page of each article that you reference to turn in with

you bibliography 6. Bring two copies of your annotated bibliography with you to class on

the due date.

Objectives By writing annotations for the references that you list, you will be forced to evaluate your references and to think about how they explicitly relate back to your hypothesis. If you can’t put into words how a reference supports your hypothesis, then it probably isn’t a good reference to include. The purposes of this assignment are for you to develop your ability to find relevant background research, evaluate sources for quality and applicability, and summarize research concisely, accurately, and in your own words. Process After discussing with your group, you should have a clear research hypothesis. Write this at the top of your paper. It should be clear what type of relationship you are trying to predict, and what your variables are. This portion of the assignment should be the same for everyone in your group.

Once you have your hypothesis, you should split up and use Psych Info to search for peer-reviewed articles that support your hypothesis. Find at least four peer-reviewed journal articles that you think support your hypothesis in some way, and begin writing your annotations. Your annotations don’t need to be very long, but each should briefly summarize the research presented in the article and describe how you might use this article as evidence to support your hypothesis. Your annotations should not include direct quotes or paraphrases. The audience for these annotations includes myself and the other members of your group, and while they are not intended to be formal essays, they should be written in complete sentences and should be understandable when read by someone with background in Psychology. Bring two copies of your annotated bibliography to class with you on the date due. One copy will be given to me, along with the first pages of the articles referenced. The other copy will be shared with your group. As a group, you should discuss the references that you have found. Maybe you found something great that nobody else did, or maybe you found a reference that other group members did too (suggesting it might be pretty relevant). You are not required to use all the references that you and your group members found in your final paper (remember, you only need a minimum of four in the final document), but the background research and annotations that you’ve created should start to give you a good idea of how you are going to structure your introduction. Grading (10 points) Four points

To receive the first four points, all of the following criteria must be met. ü The research hypothesis is clearly stated at the top.

ü There are a minimum of four articles referenced

ü The citations are in correct APA style

ü The first page of each article referenced is included.

Six points The remaining points will be awarded based on how well your annotations meet the following criteria:

ü The annotations clearly, concisely, and accurately summarize the research in the student’s own words

Very successful (3pts)

Somewhat successful (2pts) Unsuccessful or incomplete (1pt)

ü The annotations explicitly state how the research in the article could be used to support the group’s hypothesis

Very successful (3pts) Somewhat successful (2pts) Unsuccessful or incomplete (1pt)

Step 2 – Introduction For your first group assignment, you will need to write a draft of the introduction for your research manuscript. Writing a paper as a group can be challenging. This will be your first test of how well you can collaborate. Discuss with each other what your strategy is going to before you start working. You can adopt whatever strategy you like, but I recommend dividing up the project into smaller tasks that can be assigned to each group member. Assignment Summary You will be writing the introduction to the manuscript that you plan to submit at the end of the term. This should be written in APA style, with 1” margins on all sides. The Introduction must incorporate a minimum of four peer reviewed journal articles as references, cited appropriately, within the text. I don’t like to give minimum page requirements. The paper should be as long as it needs to be to do what it needs to do and no longer. That being said, you probably can’t achieve that goal in two pages, and you will probably overshoot your target with more than five. The paper you turn in should contain a title page, the introduction section, and a correctly formatted reference list at the end of the document. Objectives Through this assignment, you are expected to develop your ability to

• Select and summarize relevant research from primary sources • Create and articulate a psychological hypothesis • Present a reasoned argument to support your hypothesis • Use research as evidence to support your points • Appropriately cite relevant work

Process You will be expected to have a rough draft of your group’s introduction ready for peer review in class the week before the final draft is due to me.

Rough Draft Due: ______________ Final Copy Due: ______________

On the peer review day, you will need a hard copy of the draft for each group member. If an individual does not have a hard copy, then he or she

will not be able to participate in the peer review activity and will not receive credit for the in class activity. A single hard copy of the Introduction will be turned in to me the following week. What is an Introduction? The APA style manual states that an introduction should serve three purposes (in this order): 1. It should introduce the problem being studied.

The first paragraph or two should be devoted to describing the general problem that you are studying. It can either put the problem in the context of the real world or a larger disciplinary debate. These paragraphs should conclude by introducing your hypothesis.

2. It should summarize the relevant background literature.

The middle of the introduction is where you provide some background for the reader. Here you describe the concepts that are important to your hypothesis and how the existing research in the field leads you to your current research. This does not need to be an exhaustive review of all the literature in the field, but it should be enough background that the reader can understand the rationale behind your hypothesis.

3. It should state the purpose of the current work.

The closing paragraph should be a general description of how you plan to test your hypothesis. This does not need to be very detailed (the next section of the paper will provide the details). This is more of a way of transitioning from a discussion of previous work to the discussion of your current work.

I understand that this is a very general description, which is why we will be devoting an entire day of class time to going over this in more detail. In addition to your class notes there are many other resources available to you:

• The APA publication manual goes into more detail, and should be available at the library if you do not own a copy or know someone who does.

• Chapter 16 of the text provides some discussion of the different sections of an APA paper, and a sample paper is included in Appendix D.

• I have also posted an article written by Daryl Bem on the Moodle site for the course, which provides information about the structure and style of empirical research manuscripts.

• Any psych article published in an APA style journal can serve as an example of an introduction that’s passed the peer review process.

Introduction Section Rubric

Insufficient

or not present

Poor Needs Work Satisfactory Exceeds

Expectations

The research question is clearly described and the importance of the topic is established at the beginning of the paper.

1 2 3 4 5 6 7 8 9 10

Background research presents the state of psychological research related to the topic area.

1 2 3 4 5 6 7 8 9 10

Research is effectively used to guide the reader to the authors’ hypothesis.

1 2 3 4 5 6 7 8 9 10

The paper closes with a clear statement of the hypothesis and a general description of how the current research will test the hypothesis.

1 2 3 4 5 6 7 8 9 10

The paper is formatted and organized according to APA style guidelines, including internal citation and a references list.

1 2 3 4 5 6 7 8 9 10

The prose is clear, precise, concise, and logically developed throughout

1 2 3 4 5 6 7 8 9 10

The paper is free of spelling, grammatical, and typographical errors.

1 2 3 4 5 6 7 8 9 10

Step 3 – Methods Proposal Now that you have conducted the background research and come up with a hypothesis to test, your next step is to plan exactly how you will test it. A methods section tells a story. It’s the story of what you did in your study. The goal is to provide enough detail that another researcher could replicate what you did. This paper will differ a bit from a traditional methods section because instead of writing about what you did, you are going to be writing a proposal for what you plan to do. Assignment Summary This assignment asks you to write a research proposal. This is very similar to a methods section. In fact, the practically speaking, the biggest differences are the verb tense you use (you’ll be using future tense since you haven’t actually done anything yet) and the stats you include (which are none at the moment because you have no data yet). Like with the previous section, this paper should follow APA formatting guidelines. It should include a title page, the research proposal itself, and a reference page. Objectives Through this assignment, you are expected to develop your ability to

• Design a study to test a hypothesis • Describe a research protocol with enough detail that another

researcher could replicate it • Follow APA guidelines for a methods section • Appropriately use and cite materials

Process You will not be turning in a rough draft of your methods proposal. However, you will be expected to let me know what you are doing before your draft of the methods section is due. We will have an in-class activity the week before your methods proposal is due in which you will be asked to answer specific questions about your proposed research. Be prepared to give me a description of what you plan to do and the measures that you plan to use. This means that before you start writing your methods section, you are going to want to discuss your plan for research as a group and collect and/or create the measures you are going to use. I will be talking to each group during this activity to ensure that your research plan is both sound and ethical. This step is intended to help you make sure that you have all the important content ready before you spend the time writing up the proposal.

Once you have an outline of what you plan to do, you can start writing the methods proposal.

In class study design activity: ___________ Methods proposal due: ___________

What is a Methods section? A methods section is typically the most formulaic part of a research report. It contains three sections with their own headings: Participants, Procedure, and Materials. These sections outline the who, what, and how of your research

1. Participants: This is the “who” section. In a typical methods section, you would use this to give details about your sample. In addition to sample size, this includes relevant demographic characteristics of your sample, a description of how they were recruited, and how they were compensated. For your proposal, you won’t have this information, but you should have an idea of how you are going to recruit and compensate your participants, and you should have a goal for your sample size.

2. Procedure: This is the “what” section. Here you will describe what you had participants do. Remember, this section is about the participants’ experience, not yours. It should begin with informed consent and end with debriefing. For the proposal, you will be telling me what you plan to have participants do.

3. Materials: This is the “how” section. Where in the previous section you might have said that participants’ self-esteem was measured, here you actually detail how you measured it. Typically this will include subheadings for each measure. You need to be detailed here. Describe each measure conceptually, provide an illustrative example item from a measure, report how participants responded, and explain how scores were calculated (if applicable). In the methods section of your final manuscript, you will also be expected to include descriptive and reliability statistics, but since you don’t have data yet, this won’t be necessary for your proposal.

As with the introduction section, we will be spending class time to go over this in more detail, and as before, you should consult the various references available to you for more detail. If you don’t recall what these references are, they are detailed at the end of the Step 2 assignment description.

Methods Proposal Rubric

Insufficient

or not present

Poor Needs Work Satisfactory Exceeds

Expectations

A clear description of the desired sample is provided, along with a plan for recruitment.

1 2 3 4 5 6 7 8 9 10

A clear and realistic plan for collecting data is stated.

1 2 3 4 5 6 7 8 9 10

All of the materials are described with enough detail for replication.

1 2 3 4 5 6 7 8 9 10

APA style guidelines are followed, and all required sections are included.

1 2 3 4 5 6 7 8 9 10

The prose is clear, precise, concise, and logically developed throughout

1 2 3 4 5 6 7 8 9 10

The paper is free of spelling, grammatical, and typographical errors.

1 2 3 4 5 6 7 8 9 10

Step 4 – Results Data collection should now be complete, and it’s time to figure out if you found support for your hypothesis or not. The results section is where you present the data to your reader. You don’t really get into interpretation or conclusions here. That’s in the next section. All you do is describe your data, how you tested your hypothesis, and what the outcome of those tests revealed. You use statistics in this section as the evidence to support your claims. Assignment Summary For this assignment, you will need to enter your data, analyze it using the appropriate tests, and report the results to the reader in a precise and parsimonious manner. This will likely be a fairly short section, as you are just describing your data. You should report all relevant analyses and their results (even if they go counter to the hypothesis). Tables and figures can be used to present a large number of statistics efficiently. You will be required to have at least one table or figure as a part of your results section. As with every step, you should follow APA formatting guidelines in the writing and in the creation of any tables or figures. Though it would not traditionally be included in a research manuscript, I am also asking that you attach the output for all the analyses that you report in your results. This is so that I can make sure that you are using the appropriate tests, and that you are interpreting the data correctly. Objectives Through this assignment, you are expected to develop your ability to

• Enter data into SPSS • Analyze data using the correct test for the hypothesis you are testing • Communicate results in a clear, accurate, and parsimonious fashion • Integrate appropriate statistics into the narrative of the text. • Follow APA guidelines for writing a results section, reporting statistics,

and creating a figure or table

Process A lot of the process for this paper will occur behind the scenes. Before you even begin your writing, you will need to create a data set in SPSS. For the most part, I’ve been giving you data to work with this term, so this might be new, but you should be familiar with what a data set should look like. Your

columns are your variables, and your rows are your participants. Use the variable view tab to create your variables and to add value labels anywhere that they might be necessary. If you are entering data from paper surveys, be sure that you are careful to avoid data entry errors. Once your data are entered, you will need to create any scales that you are using for your variables, and consider what statistical analyses you need to use to test your hypotheses. Print out the output for all the analyses you plan to report. This will be helpful to have on hand when writing the section, and I am asking you to turn it in with your draft, so it’s required. I suggest writing a first draft of the results section without including any statistics. You can put something like “(stat)” in as a place holder if you really want to, but avoid worrying about numbers and formatting. In many ways, statistics are to a Results section what internal citation is to an Introduction. They are the evidence that you are using to support the claims that you make, and though they are important because you must have evidence to support your claims, a reader should be able to run a black marker through all your stats and still be able to understand what analyses you ran, and what the outcomes were. Once you have written the section to your satisfaction, then go back and enter the numbers and finalize the formatting. What goes in a Results section? A results section is fairly straightforward. The purpose is to describe your data and the analyses that you performed. This means present some descriptive statistics on the variables that you are conducting analyses on (mean and standard deviation are typical for continuous variables, whereas frequencies and percentages are common for nominal variables), identify the analyses you are using, and present the results of these analyses. Statistics should either be presented in tables, or should be embedded within the text, using commas to separate them from the statements they follow. Letters from the English alphabet that are used to denote a statistic (e.g., R, R2, p, F, etc.) should be italicized. Greek letters (e.g., β & χ2) should not be italicized. Tables and figures can be used to facilitate the presentation of data or to visually illustrate a point. However, you should avoid redundancy in the presentation of data. That is, if you present the data in a table, don’t present it in the text and vice versa. All tables and figures should be reference in the text and presented on separate pages at the end of the document with appropriate formatting (i.e., according to APA guidelines).

Results Proposal Rubric

Insufficient

or not present

Poor Needs Work Satisfactory Exceeds

Expectations

Appropriate analyses are used to test the hypotheses, with the relevant statistics included.

1 2 3 4 5 6 7 8 9 10

Analyses and results are clearly and accurately described.

1 2 3 4 5 6 7 8 9 10

A correctly formatted table or figure is used to present information to the reader in a clear and easy to read manner.

1 2 3 4 5 6 7 8 9 10

APA style guidelines are followed.

1 2 3 4 5 6 7 8 9 10

The prose is clear, precise, concise, and logically developed.

1 2 3 4 5 6 7 8 9 10

The paper is free of spelling, grammatical, and typographical errors.

1 2 3 4 5 6 7 8 9 10

Step 5 – Discussion Now that you’ve analyzed your data, it’s time to interpret your findings for the reader. The discussion section begins with a short summary of your findings, along with their implications. It then goes on to talk about possible limitations in your research and future directions for research. When the results are simple and straightforward, the discussion section can be fairly short. In fact, sometimes it is so brief that the results and discussion sections are combined in some research reports. Assignment Summary For this assignment you will be writing the section that wraps up your paper. It talks about the findings and essentially tells the reader what the author thinks they should take away from the research. This requires you to suggest the implications of your research as well as to examine your research for issues of validity and reliability. Be sure that you are not including a lot of information that is redundant with other parts of your paper. That is, don’t rewrite the introduction. The discussion should be more focused on your research than on past research. Your paper should include a discussion of at least three limitations and provide at least two suggestions for future directions in research. Objectives Through this assignment, you are expected to develop your ability to

• Interpret results and possible implications of any findings • Critique research methodology and identify limitations of research • Discuss future directions for research

Process Since there will not be a lot of time between when you turn in the discussion section and when your final manuscript is due, I am going to have you complete an activity where you tell me the content that will go in your discussion section before you turn in your draft. What goes in a Discussion section? A discussion section should begin with a clear statement of support or no support for the hypothesis. Then you can talk about the implications of this research. These should include real world implications and a discussion of how the current work fits into the existing body of research on the topic. This is followed by a discussion of the limitations of the current research and ideas for future directions on this research topic. You should not just list

limitations. You must explain why you think something is a limitation as well. Future directions are ideas for studies that could expand upon the findings in this research. The discussion section typically concludes with a short paragraph on the importance or relevance of the current work.  

Discussion Rubric

Insufficient

or not present

Poor Needs Work Satisfactory Exceeds

Expectations

The section begins with a clear summary of the findings.

1 2 3 4 5 6 7 8 9 10

Provides a discussion of how the current research confirms, expands, or contradicts previous knowledge and what the real world implications are.

1 2 3 4 5 6 7 8 9 10

At least three reasonable limitations are discussed.

1 2 3 4 5 6 7 8 9 10

At least two specific future directions are identified.

1 2 3 4 5 6 7 8 9 10

APA style guidelines are followed.

1 2 3 4 5 6 7 8 9 10

The prose is clear, precise, concise, and logically developed.

1 2 3 4 5 6 7 8 9 10

The paper is free of spelling, grammatical, and typographical errors.

1 2 3 4 5 6 7 8 9 10

Step 6 – Putting Together Your Final Draft Congratulations, you are almost finished. With all the earlier steps completed, you shouldn’t have too much to do to get your final draft ready. Assignment Summary For this assignment, you will be turning in the completed final draft of your research manuscript. This will include a title page, an abstract, the paper itself, the references section, and at least one properly formatted table or figure. You will also be turning in at this time printed out copies of your meeting minutes, and the output for any analyses that were reported in your paper. Objectives This assignment should reinforce all of the previous learning objectives. Process The first thing that you might want to do is combine all your sections into one document. This document should begin with a title page, the second page should be an abstract, and then the paper itself will begin on the third page. If you haven’t done so already, this would be a good point to go through and make revisions to each section based on the comments that I gave you and any new thoughts you might have had while conducting the research and analyzing the data. The section that will likely need the most revision is the methods section. Remember, what you submitted was a methods proposal. Now, you are submitting the methods of a study that you have conducted. You will need to go back and include descriptive statistics on your participants and reliability analyses on any scales that you have constructed. Make sure that the materials that you are writing about match the variables that you ran analyses on, and delete any extraneous information. Once all of the sections have been revised, you will need to write an abstract. An abstract is a brief summary of your paper. The APA publication manual suggests that an abstract should be no more than 120 words. Shoot for about one sentence for each section of the paper. These are often challenging to write because they need to compress a lot of information into a small amount of space. I suggest starting by writing out your summary without worrying about word count. Then go back and start cutting and condensing.

After you have taken care of all of your content, you should do a final copy edit to look for typos and formatting errors. I recommend giving a copy to everyone in your group and separately proof reading. Then bring all the drafts together to compare edits. Hopefully, this will ensure that you catch all of the possible errors. The APA Publication Manual is your new best friend when it comes to issues of formatting or style. It contains all the guidelines for APA formatting, and I keep mine by my side for easy reference whenever I’m writing a paper. You should have access to one through the library, but if you can’t get to that, then your textbook and the Purdue Online Writing Lab are both great secondary sources.

Final Manuscript Rubric

Introduction

Significance of the research project

Excellent Clearly describes the research question and establishes the importance of the topic

Fair Describes the research question, but does not establish its importance

Poor Vague or inaccurate research question

Inadequate No introduction of a research question

Background Research

Excellent Establishes background and effectively guides the reader to the hypothesis through arguments backed by relevant scientific literature.

Fair Establishes some background by citing related literature, but fails to use the literature effectively to support the hypothesis.

Poor Uses literature to provide some background information that is tangentially related to the research question

Inadequate Scientific literature is not used, or is unrelated to the research question or hypothesis

Hypothesis

Excellent Establishes a clear hypothesis that describes the variables and the expected relationship, using language that is consistent with hypothesis testing

Fair States the hypothesis incompletely, vaguely, or imprecisely.

Poor States a hypothesis that shows confusion, misunderstanding, or little to no apparent link to the research question

Inadequate No hypothesis articulated

Methods

Participants

Excellent Provides details of how participants were sampled and recruited, and uses text and descriptive statistics to present relevant demographic features of the sample to the reader.

Fair Provides general information about the participants or the sampling and recruitment procedure. Omits some important information or may provide irrelevant or unnecessary details.

Poor Omits significant details about the sampling procedure, recruitment strategy or participants.

Inadequate Description is so vague, confusing, or incomplete that the reader could not reasonably discern who the participants are or how they were recruited and sampled.

Procedure

Excellent

Provides enough details about the procedure that another researcher could replicate the study. Provides details that convince the reader that the procedure is a valid one for testing the hypothesis, while omitting information about practices that are irrelevant or would not be question.

Fair Provides a general account of the procedure, but either omits important information or presents unnecessary details. May be in list rather than narrative form.

Poor Omits enough significant details that the reader would not be able to replicate.

Inadequate Description is so vague, confusing, or brief that the reader would not be able to discern what happened.

Materials

Excellent Provides descriptions of materials that would enable the reader to replicate the research. Avoids unnecessary detail for a reader with a scientific background.

Fair Provides a general description of materials that leaves the reader to guess about significant details, or includes unnecessary details

Poor Omits enough detail about the materials that the reader would be unable to completely replicate the research.

Inadequate Provides little to know detail about the materials

Results

Analysis

Excellent Conducts all of the appropriate analyses for the data collected. Presents all the relevant statistics for each analysis, using APA formatting guidelines

Fair Conducts most of the appropriate analyses, but reports incomplete or incorrectly formatted statistics.

Poor Conducts inappropriate analyses or presents incorrect statistics.

Inadequate It is unclear what analyses were run, or statistics are not reported.

Text

Excellent

Presents all the relevant results in paragraphs, using APA formatting guidelines. Describes the results fully and accurately. Statistics are incorporated smoothly into the text, and tables and figures are referenced appropriately.

Fair Presents some of the major results. The description of the results is incomplete or imprecise. Results may be presented as a list or series of stats.

Poor Fails to present major results. Fails to incorporate statistics, tables, or figures.

Inadequate Presents not results in text.

Tables and Figures

Excellent Uses a table or figure to present results in a clear and easy-to-read manner. The table or figure is appropriate for the presentation of the reported data, and can be understood without having to refer back to the text.

Fair Uses a table or figure, but it is difficult to read or understand. The table or figure presents the reader with little information. The reader may need to look at the text to interpret the table or figure.

Poor

Presents irrelevant or inaccurate information in the table or figure. The table or figure is constructed in such a way that it is confusing and/or difficult to read. The reader must read the text to understand the information in the table.

Inadequate No tables or figures are provided.

Discussion

Hypothesis

Excellent Restates the hypothesis early in the discussion and argues persuasively, using the results from the study, whether the hypothesis was supported or rejected.

Fair Refers to the hypothesis without restating it, or fails to indicate whether the results support or reject it. Makes only a loose connection to specific results.

Poor Makes vague reference to the hypothesis and does not correctly report whether the results support or reject it, or refers to a hypothesis not stated in the introduction.

Inadequate Provides no reference to a hypothesis and whether it was supported or rejected.

Implications and Relation to the field

Excellent Provides a discussion of how the current research confirms, expands, or contradicts previous knowledge and what the real world implications are.

Fair Provides a general or erroneous discussion of the relation between the current work and previous knowledge and of the real world implications

Poor Provides a discussion of the current work that fails to adequately relate it to existing knowledge and real world implications.

Inadequate Provides no discussion of the relation between the current work and previous knowledge or real world.

Limitations

Excellent Addresses any specific limitations in the study, detailing specifically the problems that these limitations could have posed and any alternative explanations that could result from them.

Fair Addresses limitations, but does not clearly indicate why the limitations were problematic or the alternative explanations that they could allow.

Poor Addresses general limitations, but does not make a compelling case for how or why they limit the research.

Inadequate Does not address any limitations in the existing work.

Future Directions

Excellent Suggests one or more specific studies that could explain or expand upon the study’s results.

Fair Suggests one or more general approaches that could explain or expand upon the study’s results.

Poor Suggests one or more general or specific approaches that would not succeed in explaining or expanding the study’s results.

Inadequate Provides no proposal for future research.

General

Title Page

Excellent The paper begins with an appropriately formatted title page.

Fair The paper begins with a title page with some formatting errors.

Poor There is a title page, but it is formatted incorrectly or is missing information.

Inadequate There is no title page.

Abstract

Excellent Clearly and accurately summarizes the manuscript in fewer than 120 words.

Fair Generally summarizes the manuscript, but goes over the word limit, omits important information, or is confusingly vague.

Poor Provides a vague or incomplete summary of the manuscript. May exceed word limit.

Inadequate No abstract or the abstract does not summarize the manuscript.

References

Excellent All cited works are referenced at the end of the document in the correct order and are appropriately formatted.

Fair The references section has formatting errors or is missing cited references

Poor The references section has formatting errors and is missing cited references

Inadequate There are significant errors in the content and formatting of the references section.

Formatting

Excellent Conforms to APA style guidelines throughout.

Fair Conforms to APA style guidelines with few errors.

Poor Contains several formatting errors.

Inadequate Does not appear to be written with any regard given to APA formatting guidelines.

Quality of Prose

Excellent Uses prose that is clear, precise, and logically developed throughout, with no grammatical, spelling, or other errors.

Fair Prose is generally clear, precise, and logically developed, with few grammatical, spelling, or other errors.

Poor Prose contains numerous grammatical, spelling, or other errors.

Inadequate Prose does not meet expectations for college level academic writing.

Stats Output

Excellent Printed output for all analyses are included at the end of the document.

Fair Output for most, but not all, analyses are present.

Poor Only a few of the analyses are included.

Inadequate No output is included.