Packets never lie An in-depth overview of 802.11 frames small.pdf
The Data Never Lie
21
The Data Never Lie But, Do We? Los datos nunca mienten pero, ¿y nosotros? Eric N. Schreffler, ESTC 13 May 2009 ECOMM 2009, San Sebastian, Spain
description
The Data Never Lie. But, Do We? Los datos nunca mienten pero , ¿y nosotros ? Eric N. Schreffler , ESTC 13 May 2009 ECOMM 2009, San Sebastian, Spain. The Data Never Lie. But, Do We? The Politics and Policy Implications of Mobility Management Evaluation. Overview. - PowerPoint PPT Presentation
Transcript of The Data Never Lie
The Data Never LieBut, Do We?
Los datos nunca mientenpero, ¿y nosotros?
Eric N. Schreffler, ESTC13 May 2009
ECOMM 2009, San Sebastian, Spain
The Data Never LieBut, Do We?
The Politics and Policy Implications of Mobility Management Evaluation
OverviewWhy Evaluate?
Why Do I Care?
How Can Evaluation be Manipulated?
How Can We Avoid the “Dark Side?”
A Challenge
A Disclaimer… This is my opinion I am being a bit harsh to
make a point Most people here do very good
evaluations We will learn a lot more this
week I am an American… we know
it all
Why Evaluate?Satisfy funders?Satisfy policy-makers?Sound management practice?Measure progress against objectives?Refine program or project?
Don’t you really want to know?
Why are We Scared to Evaluate? It costs money It takes a lot of time Need to plan before
project starts Behavior change takes a
long time We are not researchers or
academics
Why Really Are We Scared to Evaluate? What if the results are
not favorable?
Will it make me look bad?
What if I FAIL?
Why Do I Care? I have been evaluating
Mobility and Demand Management programs for almost 30 years
I have seen the good, the bad, and the ugly
I know MM gets marginalized I believe in the overall
effectiveness and cost effectiveness of MM/TDM
Manipulating ResultsCan Evaluations be Manipulated?
How Can Evaluations be Manipulated?
How Are Evaluations Manipulated?
Manipulating Evaluation Focus only on “before” forecasts,
not “after” results;
assuming forecasts =results
Manipulating Evaluation Focus only on “before” forecasts, not “after”
results
Use “rules of thumb” or expected results
Manipulating Evaluation Focus only on “before” forecasts, not “after”
results Use “rules of thumb” or expected results Focus data collection only on the
“converted”
Manipulating Evaluation Focus only on “before” forecasts, not “after”
results Use “rules of thumb” or expected results Focus data collection only on the “converted” Use anecdotal
stories; qualitative findings
Manipulating Evaluation Focus only on “before” forecasts, not “after”
results Use “rules of thumb” or expected results Focus only on the “converted” Use anecdotal stories;
qualitative findings Spin results
Manipulating Evaluation Focus only on “before” forecasts, not “after”
results Use “rules of thumb” or expected results Focus data collection only on the “converted” Use anecdotal stories; qualitative findings Spin results Omit results
Avoiding the Dark Side
TrafficEngineer
Funding SourceMobility Manager
Avoiding the Dark SideBuild evaluation into funding processEstablish credibility through scientific rigorPool resources and resultsUse guidance offeredUse local academics
Advice: know your weaponKnow your evaluation,
love your evaluation,
for one day, your evaluation just might save your life
The Results May Surprise YouAccording to
Congressionally-mandated study of principle funding source used in US, TDM and MM are among the most cost effective strategies for reducing emissions
TRB Special Report 264
Just Do It!!!
contactEric N. SchrefferTransport ConsultantSan DiegoCalifornia
001.858.538.9430
muchas gracias para su atenciōn
