Jonathan D. Breul Evaluation as an instrument to improve the quality of public spending - a look at...

Jonathan D. BreulJonathan D. BreulEvaluation as an instrument to improve Evaluation as an instrument to improve the quality of public spending - a look at the quality of public spending - a look at

the global financial crisisthe global financial crisis

V Konferencja EwaluacyjnaV Konferencja EwaluacyjnaWarszawa, 23.10.2009Warszawa, 23.10.2009

© Copyright IBM Corporation 2009

Ministry of Regional Development &

Polish Agency for Enterprise Development 5th Evaluation Conference Warsaw, Poland October 23, 2009

Evaluation as an instrument to improve the quality of public spending - a look at the global financial crisis


Role of evaluation with the economic crisis

Imperative to measure the impact of investments and help make adjustments.

While it may be too late to shape how governments respond to this economic crisis, we have a responsibility – at a minimum – to capture lessons that can inform responses in the future.


Defining success

Confusion whether the programs represent spending or investment

In US, the Recovery Act has multiple purposes:

1. Government spending

2. Tax cuts

3. State fiscal relief

Not only intended to quickly create jobs – but also to make longer term investments the will benefit the growth of the economy in the longer term.


Measurable criteria

Achieve economic stimulus: Jobs created or job losses avoided

Achieve long-term public benefits by investments in transportation, environmental protection or technology

Foster energy independence or improved educational quality

Deliver program results

Satisfy transparency and accountability requirements


Many challenges

Measure outputs or outcomes?

It might be impossible to measure the impact or results – since impact often times lags years behind the actual investment

Often other factors will influence any particular impact

Hugely difficult and probably impossible to make a causal connection without a counterfactual.

We don’t know what would have happened in the absence of the Recovery Act

Compound effect of Recovery Act, and other measures


Stimulus job creation

The models are based on historical experience

But, we are outside anything we have experienced historically.

We are completely in a world we do not understand and know.

High level of uncertainty.

White House: plan will create 3.3 million to 4.1 million jobs by late 2010

Congressional Budget Office: estimates 1.2 million to 3.6 million additional jobs


Economists use a two-step process

First, they project how much the legislation will make the economy grow

Then they predict how many jobs that growth will create.

The second part is less complicated – because economists tend to rely on rules of thumb.

The White House assumed that each 1 percent increase in the economy’s size would produce 1 million jobs

The initial calculation is tougher.

To make it, economists rely on a mix of facts and assumptions.

Understandably, they come to different conclusions.


The White House method assumes that things were getting worse and that the stimulus is the sole factor responsible for stopping the bleeding.

Procedure is somewhat crude and does not take into account the obvious differences in wages and other costs across different types of projects and across different parts of the country

It does take into account key differences between tax changes or state fiscal relief, an direct government investment spending.

Council of Economic Advisors estimates


OMB reporting guidelines

Direct reporting requirements cover one-third ($271 billion) of stimulus funds.

Recipients and sub-recipients required to report how many jobs are created or saved.

Total hours of work created divided by the number of hours worked by a full-time worker, along with a brief description of the types of work involved.

This data will later be used to evaluate the quality of jobs created by stimulus funding.


Obama Administration program evaluation effort

October 7, 2009, the Office of Management and Budget has issued new guidance and funding to agencies that develop and perform high-priority evaluation activities

Step toward making program reviews an essential function of government

Most rigorous study designs appropriate for different programs, given size, stage of development, etc.

Focus on “impact evaluations” to determine causal effects

http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-01.pdf


We may never know for sure.

But - the need to enhance public sector performance has become more urgent as governments face mounting demands on public expenditure and calls for higher quality services.

The benefits of being clearer inside and outside government about purposes and results are undeniable.

Is stimulus stimulating jobs?


Contact information

Jonathan D. BreulExecutive DirectorIBM Center for The Business of Government1301 K Street NW, 4th floor, West TowerWashington, DC 200045(202) 515 -4382

[email protected]

www.businessofgovernment.org

Jonathan D. Breul Evaluation as an instrument to improve the quality of public spending - a look at...

Documents

Transcript of Jonathan D. Breul Evaluation as an instrument to improve the quality of public spending - a look at...