Main menu:

History of RPE Thought

Posts by Tag

RSS New from the CCPA

  • 2019 Federal Budget Analysis February 27, 2019
    Watch this space for response and analysis of the federal budget from CCPA staff and our Alternative Federal Budget partners. More information will be added as it is available. Commentary and Analysis  Aim high, spend low: Federal budget 2019 by David MacDonald (CCPA) Budget 2019 fiddles while climate crisis looms by Hadrian Mertins-Kirkwood (CCPA) Organizational Responses Canadian Centre for Policy […]
    Canadian Centre for Policy Alternatives
  • Boots Riley in Winnipeg May 11 February 22, 2019
    Founder of the political Hip-Hop group The Coup, Boots Riley is a musician, rapper, writer and activist, whose feature film directorial and screenwriting debut — 2018’s celebrated Sorry to Bother You — received the award for Best First Feature at the 2019 Independent Spirit Awards (amongst several other accolades and recognitions). "[A] reflection of the […]
    Canadian Centre for Policy Alternatives
  • CCPA-BC welcomes Emira Mears as new Associate Director February 11, 2019
    This week the Canadian Centre for Policy Alternatives – BC Office is pleased to welcome Emira Mears to our staff team as our newly appointed Associate Director. Emira is an accomplished communications professional, digital strategist and entrepreneur. Through her former company Raised Eyebrow, she has had the opportunity to work with many organizations in the […]
    Canadian Centre for Policy Alternatives
  • Study explores media coverage of pipeline controversies December 14, 2018
    Supporters of fossil fuel infrastructure projects position themselves as friends of working people, framing climate action as antithetical to the more immediately pressing need to protect oil and gas workers’ livelihoods. And as the latest report from the CCPA-BC and Corporate Mapping Project confirms, this framing has become dominant across the media landscape. Focusing on pipeline […]
    Canadian Centre for Policy Alternatives
  • Study highlights ‘uncomfortable truth’ about racism in the job market December 12, 2018
    "Racialized workers in Ontario are significantly more likely to be concentrated in low-wage jobs and face persistent unemployment and earnings gaps compared to white employees — pointing to the “uncomfortable truth” about racism in the job market, according to a new study." Read the Toronto Star's coverage of our updated colour-coded labour market report, released […]
    Canadian Centre for Policy Alternatives
Progressive Bloggers

Meta

Recent Blog Posts

Posts by Author

Recent Blog Comments

The Progressive Economics Forum

Why incentive pay won’t fix education or health care

It turns out — surprise! — that it’s really hard to measure quality in complex social systems and that employing simplistic quantitative measures can backfire.

That’s the take-home message from a recent talk by UC Berkley economist and public policy professor Jesse Rothstein who came to SFU to present his latest research on using standardized test scores to measure teacher effectiveness in the US.

Prof. Rothstein was involved in a 3-year pilot project in Tennessee designed as an experiment to check whether offering teachers bonus pay improves students’ test score performance. Teachers were randomly assigned into two groups, one was offered bonus pay if their students did well on standardized tests (experimental group) and the other one wasn’t (control group).

After 3 years, there weren’t significant differences in student achievement on standardized tests between the two groups, clearly showing that offering teachers bonus pay did not improve student achievement. Yet, the Obama administration (for which Professor Rothstein worked recently) is continuing to explore incentive pay as a way to improve the education system.

Come to think of it, the idea of incentive pay has become the holy grail in governments’ quest for improving the performance of complex social systems like health care and education. On the surface, there’s a certain intuitive appeal to the idea of paying more to those doing a better job. The “economic theory” behind it is that offering to pay people more for doing a good job will lead to increased work effort as rational individuals choose to maximize their pay.

That’s what’s driving US policy makers to test schemes of offering teachers incentive pay to improve school achievement. That’s what’s driving BC Health Minister Kevin Falcon to offer hospitals funding based on the number of surgeries they do (what he calls patient-focused funding).

But when used mechanically — by tying incentives to some quantitative measure of performance, like test scores or number of surgeries done — such schemes are likely to fail.

The devil — as usual — is in the details. And the details are that before policy-makers can give somebody a bonus for doing a good job, they need to be able to measure what a good job looks like. This is where standardized tests come in along with other quantitative measures such as the number of medical procedures performed, or the length of hospital stay.

But it turns out — surprise! — that it’s really hard to measure quality in complex social systems and employing simplistic quantitative measures can backfire. In fact, Prof Rothstein quoted an obscure scholar of methodology by the name of Donald Campbell, who coined a rather pessimistic “Campbell’s law” in the late 1970s:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

In other words, the perverse incentives that pay-for-performance schemes create in complex social systems may well outweigh any positive incentives for real improvement.

Campbell looked at education in particular (see his working paper here), and argued that

“achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.

Disappointing news for economists, and a reminder to acknowledge the limitations of the quantitative methods we, as a profession, so often peddle to policy-makers.

What does this tell us about BC’s new incentive-based funding model in health care?

Kudos to the SFU Centre for Education Research and Policy for organizing the public event.

Enjoy and share:

Comments

Comment from Louis Erlichman
Time: October 15, 2010, 6:37 am

Ironically, this mirrors the problems that used to be cited when I was in school (back in the middle ages) with incentives in soviet-style planning systems. The standard example was a factory producing nails. If the quota is based on weight, the factory will produce the largest nails possible; if volume, the smallest; if there is no constraint on inputs, you have high waste rates, etc. Incentives based on simple measures distort the process they are supposed to be facilitating.
Which still leaves open the difficult question of how you make quality judgements and practical decisions without the crutch of the all-powerful market.

Comment from Ken
Time: October 15, 2010, 8:17 am

This also reminded me about this TED talk about motivation – http://www.ted.com/talks/dan_pink_on_motivation.html

Comment from Marc Lee
Time: October 15, 2010, 12:07 pm

Below is a link to a great video on what motivates people, which seems very pertinent to incentive pay:

http://www.youtube.com/watch?v=u6XAPnuFjJc

Comment from Travis Fast
Time: October 16, 2010, 7:02 am

If we combine Iglika’s post with Marc’s video link and reflect back on Mankiw’s now infamous article in the NYT we are left with one of three conclusions.

A) Mankiw is wrong.

B) The type of intellectual work he does is akin to basic mechanical manipulation of say moving a mountain of manure from spot X to spot Y.

C) A & B are correct if, and only if, Mankiw does not generalize from what he does for work to what real professionals do for work.

Comment from Purple Library Guy
Time: October 17, 2010, 2:05 am

I notice those two video links are to very different-style presentations by the same guy. The first is of a stage presentation in a big hall, the second is talk accompanied by whiteboard drawing. I like the second better. Either way, I’ve heard this stuff from other sources as well. The crux–best motivation for most tasks is not reward, but autonomy, mastery, and purpose.
This suggests that the answer to Mr. Erlichman’s question is to instead ask the nail factory people to do a good job making nails, because there are people out there who will need nails. Give them autonomy, and give them purpose. They will then work to be the best nail-producers they can be, independent of quotas.

Also suggests that not only is the incentive thing for health care bad because of measurement problems, it’s also bad because it’s a terrible motivator. So it will fail at producing the right behaviour even as it succeeds in producing the wrong behaviour.

The deeper problem is something this Dan Pink guy doesn’t, probably can’t, realize. He notes that there’s a deep mismatch between the science and business practice, and wonders briefly why. He also notes that monetary reward does work for certain narrow, routine tasks and talks about how many “20th century” tasks fit that description while “21st century” tasks do not. He seems to believe that it’s just a matter of time before thinking catches up.
But it isn’t. There’s a reason those “20th century” tasks were narrow and routine, it’s called Taylorism and it’s all about control over workers. Those tasks were carefully arranged to be narrow, routine, controllable and measurable. And business gurus have been rediscovering the importance of employee autonomy and creativity for decades. It has never stuck and it can never stick. It is utterly important to capitalism (not markets, but capitalism in the sense of private ownership and control of firms) that production is ordered hierarchically and that there be strong reasons to give more capital to those at the top. If reality and science do not co-operate in providing claims on capital for those at the top, such as incentive bonuses, reality and science will be ignored in favour of just-so stories that will get the job done. If reality and science do not co-operate in amplifying the importance of management and control to production, again they will be ignored.

In the end, these studies of motivation amount to an argument for an essentially socialist or anarchist kind of production (again, doesn’t say anything about markets, but it does say something about organization of work). Not USSR top-down “socialism”, but socialism in the sense of people having control over the work they do. And indeed the record of worker-controlled co-ops, such as the factories taken over by workers in places like Argentina, is generally quite good.

Write a comment





Related articles