Paper Discussion

• Why and how to read (or review) a paper?

• Case study

Compiled and modified by D. Hui (original authors are listed and original slides are posted on the website in the references

© Nick Feamster and Alex Gray 2006-2007

Why Read Research Papers?

• Keep current in your own field– First step for a new study (novel, not repeat)

• Start a new research area – Learn about a sub-field (e.g., invasive spp)

• New papers and old ones

– Learn about another discipline that may offer solutions to a problem

• Gap-filling; meta-analysis; Ragweed study

• Read for a conference or a class

© Nick Feamster and Alex Gray 2006-2007

Step 1: Deciding What to Read

• Purpose: Learn about “hot topics” of current research in an area. (searching for problems, etc.)

• Approach: Scan papers in latest conference proceedings, top journals, databases

• Purpose: Get up to speed on sub-field• Approach: Transitive closure of related work of

papers in a top conference, review papers

• Purpose: Learn about an area that is further afield• Approach: Ask expert colleagues

© Nick Feamster and Alex Gray 2006-2007

Step 2: Deciding How to Read

• Always “top down”

– First: Abstract, introduction, conclusion

– Rest of paper if necessary

• If you want to do follow-up research

• If you want to better understand the methods/conclusions

© Nick Feamster and Alex Gray 2006-2007

Invariant Comprehension Questions to consider while reading

• What is the problem?

• What are the contributions?

• What are the conclusions?

• What is the support for the conclusions?

© Nick Feamster and Alex Gray 2006-2007

Invariant Evaluation Questions

• What is the “intellectual contribution”?– Each paper should have a single key intellectual contribution– Remembering this key idea will also give your brain a way to

“index” the paper

• What is the main conclusion/contrubution?– New finding? Method? Perspective?

• (Why) is the conclusion important?• Does the content support the conclusion?

– If so, how? – Are the methods sound? In other words, do the main

conclusions appear to be correct?– Are the results likely to be affected by the method?

© Nick Feamster and Alex Gray 2006-2007

From Reading to Research

• A major reason to read research papers is to obtain new research ideas

(passive reading and active reading)

• How can we arrive at new research ideas by studying papers that describe “solved problems”? (further study, applied to other studies)

© Nick Feamster and Alex Gray 2006-2007

Some Questions to Ask

• Context switch: Does the solution or technique apply to other problem domains?

• Unfinished business: Does the paper describe future work or directions? Open problems?

Technique questions when reviewing

• Is the writing clear, concise, and intelligible?

• Is the paper logical? Does it make a significant and novel contribution to the field?

• Are there any fatal methodological flaws?

• Are all the figures and tables clear and necessary?

Benos et al. 2003. How to review a paper

Keep in mind when you read

1. Importance of research question

2. Originality of work

3. Delineation of strengths and weaknesses of methodology / experimental / statistical approach / interpretation of results

4. Writing style and figure/table presentation

5. Ethical concerns (animal/human)

Benos et al. 2003. How to review a paper

Specific criteria of read (review)

1. Scientific quality of the work

• Are the methods appropriate and presented in sufficient detail to allow the results to be repeated?

• Are the data adequate to support the conclusions?

Benos et al. 2003. How to review a paper

Criteria of read (review)2. Presentations• Writing: Is it clear, concise, and in good English?

• Title: Is it specific and does it reflect the content of the manuscript?

• Abstract: Is it brief and does it indicate the purpose of the work, what was done, what was found, and the significance?

• Figures: Are they justified? Are they sharp, with lettering proportionate to the size of the figure? Are there legends to explain the figures? Tables: Can they be simplified or condensed? Should any be omitted?

Benos et al. 2003. How to review a paper

Criteria of review

3. Rating• Assign a rating on the reviewer form; rank

the manuscript relative to other work in the same field.

4. Comments for authors• On the reviewer form, provide specific comments,

preferably numbered, on the design, presentation of data, results, and discussion.

• Please be certain that your comments to the author(s) are consistent with your rating recommendation.

Benos et al. 2003. How to review a paper

Abstract

Purpose of this study

Test the simulated results (hypothesis) as illustrated in Fig. 1

An overall goal of theexperiment is to investigate how disturbance caused bytimber harvesting influences vascular plant diversitythrough time and across spatial scales ranging from 2 hato 1 m2.

Methods (Experimental Design)

Methods (Measurements)

• Sampling time: pre-disturbance; 1 year post-disturbance and 10 years post-disturbance

• Measurements: presence of all herbaceous and woody plant species were recorded at each of the three scales (EU ¼ 2 ha; plot ¼ 576 m2; and subplot¼1 m2) and used to document species richness at each scale.

• Richness was averaged across plots and subplots within experimental units to generate mean richness sampled at 576 m2 or 1 m2, respectively.

• Consider the scale at which disturbance treatments were applied (2-ha experimental units) to represent the ‘‘regional’’ species pool, and plots and subplots to represent local scales.

Methods (Data analysis)

• ANCOVA (analysis of co-variance)– Test if regional species richness is related to local species

richness and sites, disturbance treatments, sampling time and their interaction.

– Independent variable: regional richness– Covariate: local specie richness – Treatment factors: sites, disturbance, sampling time

• PROC MIXED– A SAS procedure used to test the significance of the treatment effects

Results

Quadratic model is used to test for species saturation.

Results

Results

Results

support previous attempts to model local-regional richness relationships through time during community assembly (Mouquet etal. 2003). Mouquet et al. (2003) predicted that the slope of the LRR should increase through time until the community reaches equilibrium at which time the LRR slope would decrease to zero through competitive exclusion.

results differ slightly from Mouquet etal. (2003) in that the LRR slopes were steepestimmediately after disturbance, whereas their modeled LRR slopes became steepest at a relatively later stage after disturbance.

Questions?

• Do you think that their results support their conclusions?

• If you were asked to test the hypothesis, can you design an experiment? How?

References

• Dale J. Benos, Kevin L. Kirk and John E. Hall. 2002. How to review a paper. Advan Physiol Educ 27:47-52, 2003. doi:10.1152/advan.00057.

• Nick Feamster and Alex Gray, 2006. College of Computing, Georgia Institute of Technology, Reading and Reviewing Papers, website: http://xxxxxxxxxxxxxxxxx.

• Luc Teyton, How to Review Scientific manuscripts. Immunology and Microbial Science. Xxxx. Website: http:xxxxxxxx.