Post on 04-Jan-2016
1
Writing Writing Proposals, Proposals,
Getting Getting Reviews, Reviews,
and and PerseveringPerseveringMing Tai-Seale, PhD, MPHMing Tai-Seale, PhD, MPH
School of Rural Public HealthSchool of Rural Public Health
2
AcknowledgementAcknowledgement
Enola Proctor, PhDEnola Proctor, PhDWashington Univ at S.L.Washington Univ at S.L.
Kenneth Wells, MDKenneth Wells, MDUCLAUCLA
3
The AgendaThe Agenda
Life of a proposalLife of a proposal Scientific review: who, where, howScientific review: who, where, how Critical areas for improvementCritical areas for improvement Summary statementSummary statement Finding your agent: working with Finding your agent: working with
program staffprogram staff
4
5
3Overall StrategiesOverall Strategies
Think First, Second, and ThirdThink First, Second, and Third Get Feedback at all Stages (3 times) Get Feedback at all Stages (3 times) Develop and Follow Timeline for Develop and Follow Timeline for
SubmitSubmit Plan on 3 Major RewritesPlan on 3 Major Rewrites Talk to Agency Staff Early, 3 TimesTalk to Agency Staff Early, 3 Times After a Good Idea and Opportunity, After a Good Idea and Opportunity,
It’s Methods, Methods, MethodsIt’s Methods, Methods, Methods-From Ken Wells-From Ken Wells
6
Make a list of 5 project ideas that Make a list of 5 project ideas that you would want to work on for yearsyou would want to work on for years
Prioritize and sketch out 1-3 ideasPrioritize and sketch out 1-3 ideas Get feedback from senior Get feedback from senior
researchers, clinicians, or patients researchers, clinicians, or patients
and community membersand community members
Finding IdeasFinding Ideas
7
Have a Conceptual Have a Conceptual FrameworkFramework
Have a Conceptual Have a Conceptual FrameworkFramework
8
Make a list of 5 project ideas that Make a list of 5 project ideas that you would want to work on for yearsyou would want to work on for years
Prioritize and sketch out 1-3 ideasPrioritize and sketch out 1-3 ideas Get feedback from senior Get feedback from senior
researchers, clinicians, or patients researchers, clinicians, or patients and community membersand community members
Having IdeasHaving Ideas
9
Shaping the IdeaShaping the Idea Based on literature, agency Based on literature, agency
priorities, identify next steps suitable priorities, identify next steps suitable to your stage of development and to your stage of development and institution capabilitiesinstitution capabilities
Identify data sources neededIdentify data sources needed Identify special opportunitiesIdentify special opportunities SENIOR CONSULTSENIOR CONSULT
10
Clarify settings and subjectsClarify settings and subjects Develop Partnership – What Develop Partnership – What
model of collaboration?model of collaboration? Consider diversity goalsConsider diversity goals
CONSULT with a statisticianCONSULT with a statistician
Develop the DesignDevelop the Design
– HypothesesHypotheses– SamplingSampling– DesignDesign
11
Developing the Design Developing the Design (cont.)(cont.)
Develop rough ballpark for budgetDevelop rough ballpark for budget SENIOR HELP NEEDEDSENIOR HELP NEEDED
Discuss concept with funding Discuss concept with funding agency, based on aim, opportunity, agency, based on aim, opportunity,
design, and likely budgetdesign, and likely budget
12
Develop Conceptual Framework Develop Conceptual Framework Identify Main VariablesIdentify Main Variables
Draft Interventions, if applicable Draft Interventions, if applicable Develop Pilot DataDevelop Pilot Data
GET SENIOR CONSULT AND GET SENIOR CONSULT AND BEG FOR $BEG FOR $
Write Aims, Background, Design Write Aims, Background, Design (Pretend you’re almost done!)(Pretend you’re almost done!)
First DraftFirst Draft
13
First DraftFirst Draft
Statistician Consult to help outline Statistician Consult to help outline main analysis and develop main analysis and develop Power Power Calculations to determine:Calculations to determine:
Is the Study Affordable? Is the Study Affordable? If Yes, PROCEED TO PARK PLACEIf Yes, PROCEED TO PARK PLACE If Not, Go Back to Start (JAIL) If Not, Go Back to Start (JAIL)
14
Second DraftSecond Draft Develop Operations plansDevelop Operations plans
team organization, data collection, timelineteam organization, data collection, timeline Detailed Budget--GET HELPDetailed Budget--GET HELP
Statistical Consult: Detailed Analysis PlanStatistical Consult: Detailed Analysis Plan Review assumptionsReview assumptions Don’t delegate blindlyDon’t delegate blindly They are YOUR hypotheses; They are YOUR hypotheses; Modify design, scope, budgetModify design, scope, budget as neededas needed
Human Subjects Section: Human Subjects Section: Consult with IRB, MentorsConsult with IRB, Mentors Plop revised draft together Plop revised draft together
15
Third DraftThird Draft Meet all agency requirementsMeet all agency requirements Develop budget justificationDevelop budget justification Highlight “value added,” pilot data, Highlight “value added,” pilot data,
fit of aims, method, and analysis to fit of aims, method, and analysis to modelmodel
AGENCY & SENIORAGENCY & SENIORREVIEWREVIEW
16
It’s Not Over YetIt’s Not Over Yet Take the feedback seriouslyTake the feedback seriously Revise the whole proposal if neededRevise the whole proposal if needed
Common problems: Not feasible (budget too Common problems: Not feasible (budget too large or scope too broad ); No pilot data; large or scope too broad ); No pilot data; Aims not specific; Background has literature Aims not specific; Background has literature but no synthesis, framework, value-added but no synthesis, framework, value-added unclear; Methods are under-developed unclear; Methods are under-developed (alternatives not considered; analysis not tied (alternatives not considered; analysis not tied to hypotheses; design flaws: wrong sample for to hypotheses; design flaws: wrong sample for aim, aim,
causal inference poor; limitations causal inference poor; limitations unaddressedunaddressed
17
18
DisciplinesDisciplines Anthropology Anthropology BiostatisticsBiostatistics EconomicsEconomics Epidemiology Epidemiology Health services Health services
researchresearch Medicine Medicine NursingNursing Organizational TheoryOrganizational Theory SociologySociology
MethodologicaMethodological Orientationsl Orientations QuantitativeQuantitative QualitativeQualitative MixedMixed
Stages in Their Stages in Their Own CareersOwn Careers Senior ScholarsSenior Scholars Emergent Emergent
scientistsscientists
Who Serve on Study Sections?
19
The ReviewThe Review
Source: Enola Source: Enola ProctorProctor
20
Reviewer chargeReviewer charge
Assess likelihood that proposed Assess likelihood that proposed research will have substantial impact research will have substantial impact on pursuit of NIH research goals:on pursuit of NIH research goals:
Advance understanding ofAdvance understanding ofBiological systemsBiological systemsImprove control of diseaseImprove control of diseaseEnhance healthEnhance health
21
Review CriteriaReview Criteria SignificanceSignificance ApproachApproach InnovationInnovation InvestigatorInvestigator EnvironmentEnvironment Priority
Populations Budget Human Subject
Protections
SummarySummaryMajor Major
strengths and strengths and weaknessesweaknesses
RecommendatiRecommendation for or on for or against fundingagainst funding
22
Significance Is the dependent variable important to
NIH objectives? Does the study extend boundaries of
current knowledge? Does the study address funding agency
priorities? Bridging Science and Services Surgeon General Surgeon General’s report IOM: quality Agency partnership: “The Road Ahead”
Can the findings inform policy or practice?
23
Approach: Aims
Is the research question fully specified?
Aims are manageable, but reflects long term agenda
Incorporates relevant concepts, variables
24
Approach: Methods Setting and context
Collaborative arrangements feasible data collection plan (sufficient subjects,
reasonable procedures, plan (sufficient subjects, reasonable procedures,good measures)
Manageable: Sufficient time for methodology and products?
Person power: enough, and the right people Analytic model = fully specified
rival explanations controlled through design or measurement of relevant variables
Procedures protect data quality Appropriate methods of analysis Sufficient power
25
Methods: Critical Areas for Methods: Critical Areas for ImprovementImprovement
Design problemDesign problem MeasurementMeasurement Choice of variablesChoice of variables Intervention/comparisonIntervention/comparison
Analysis problemAnalysis problem Choice of approachChoice of approach TechniqueTechnique TestTest
Theoretical or conceptual model or framework Theoretical or conceptual model or framework Missing, deficient, or erroneousMissing, deficient, or erroneous
26
Innovation
Original and innovative aims Novel concepts, methods,
approaches Challenges existing thinking or
approaches Advances new methods or
technologies
27
Investigator Sufficient experience to direct the project
Well trained Productive
Proposed project builds on background work investigator publications preliminary studies
Proposed work will make “next logical next logical” contribution to knowledge Is surrounded by a team who can ensure success Range of substantive and methodological expertise Multidisciplinary
Critical areas for improvement Investigator expertise deficient Needs consultants or collaborators
28
Environment Environment contributes to project
success Project capitalizes on environment,
its people, and resources and resources
Collaborations Evidence of organizational support
29
Ks Review GuidelineKs Review Guideline CandidateCandidate Career development planCareer development plan Research planResearch plan Mentor/co-mentorMentor/co-mentor Environment and institutional commitmentEnvironment and institutional commitment BudgetBudget Human subjectsHuman subjects Women/minorities/childrenWomen/minorities/children SUMMARYSUMMARY
major strengths and weaknessesmajor strengths and weaknesses Recommendation for or against fundingRecommendation for or against funding
30
Overall Review All criteria are considered when
assigning overall score Application does not have to be
equally strong in all categories in all categories
Major issue: will proposed research have major scientific impact?
31
Rating Scale in NIH Rating Scale in NIH ReviewReview
1.0 virtually flawless1.0 virtually flawless, with negligible , with negligible weaknessesweaknesses
1.51.5 extremely strongextremely strong, with a few minor , with a few minor weaknessweakness
2.02.0 very strongvery strong, but with moderate weaknesses, but with moderate weaknesses
2.5 strong2.5 strong, but with some major weaknesses , but with some major weaknesses that must be addressedthat must be addressed
3.03.0 fairfair, neutral balance of strengths and , neutral balance of strengths and weaknessesweaknesses
3.5 weak3.5 weak, but with some major strengths, but with some major strengths
32
Priority ScorePriority Score
How is the summary priority score How is the summary priority score calculated?calculated? Group average Group average Average Score times 100Average Score times 100 Equal weightEqual weight
Some are “un-scored”Some are “un-scored” What is the fundable range?What is the fundable range?
Study sections can have different normsStudy sections can have different norms When in doubt, ask the project officerWhen in doubt, ask the project officer
33
The Physical The Physical SettingSetting
34
Critical Areas for Critical Areas for Improvement in Improvement in RsRs
Design problemDesign problem MeasurementMeasurement Choice of variablesChoice of variables Intervention/comparisonIntervention/comparison
Analysis problemAnalysis problem Choice of approachChoice of approach TechniqueTechnique TestTest
35
Critical Areas for Critical Areas for Improvement in Improvement in RsRs
Weak justification for studyWeak justification for study Background and significance Background and significance
unconvincingunconvincing Literature review incompleteLiterature review incomplete
Investigator expertise deficientInvestigator expertise deficient Needs consultants or collaboratorsNeeds consultants or collaborators Theoretical or conceptual model or Theoretical or conceptual model or
framework framework Missing, deficient, or erroneous Missing, deficient, or erroneous
36
Summary Summary StatementStatement
37
How to Read the Pink SheetHow to Read the Pink Sheet
Expect the language to be Expect the language to be Frank, and Frank, and Not overly enthusiastic Not overly enthusiastic
Be emotionally detached, after the Be emotionally detached, after the initial…initial…
Talk to an experienced grant-makerTalk to an experienced grant-maker Resubmit unless you see “Resubmit unless you see “fatally flawedfatally flawed”” Do NOT resubmit right awayDo NOT resubmit right away Recruit a “cold reviewer”Recruit a “cold reviewer”
38
Take a Vacation …Take a Vacation …
39
Program StaffProgram Staff Project officer – Your AgentProject officer – Your Agent
Read your concept paper and draft Read your concept paper and draft Send it in Send it in EARLY!, and EARLY!, and OftenOften
Interpret the fundability of your priority Interpret the fundability of your priority score score
40
ResourcesResources
Instructions on how to prepare your Instructions on how to prepare your applicationapplication http://grants.nih.gov/grants/funding/phs398/section_1.htmhttp://grants.nih.gov/grants/funding/phs398/section_1.htm
ll
Follow it VERY closelyFollow it VERY closely
Panel presentation at AcademyHealth Panel presentation at AcademyHealth 20062006 http://www.academyhealth.org/2006/demhttp://www.academyhealth.org/2006/dem
ystifying2006.pdfystifying2006.pdf