Evaluating Teachers, and Lessons for Evaluating Clinicians

My kids are in college and beyond now, but I read articles about public school teacher evaluations with interest.  The debate about how to rank and pay doctors seems a lot like the arguments about how to rank and pay teachers.

The Obama Administration’s Race to the Top has encouraged states to make a portion of teacher pay dependent upon “value added,” or how much more students improve on standardized tests than they would have been expected.   Sounds familiar to physicians – whose patient care is reviewed through claims analysis and who are assigned to higher or lower cost tiers, or who are granted higher or lower pay for performance payment.

The Los Angeles Times  broke ground late this summer by publishing the results of student test results of all that city’s school district’s teacher evaluations. 

Yesterday’s NY Times  had an article pointing out that while teacher evaluations on the average might be accurate, there is a very high likelihood that they are inaccurate regarding individual teachers. 

In the educational world, a teacher in the top quartile had about a one in three chance of being in the top quartile the following year.   A single year of data would misclassify 35% of all teachers!  

In health care, we have much the same problem.  Researchers report in last weeks’ New England Journal of Medicine that they sent Massachusetts hospital claims and mortality data to four different software vendors and asked them to rank hospitals by risk-adjusted mortality.  The results?   Enormous scatter – hospitals that appeared to have a low risk-adjusted mortality rate with one vendor had a high risk-adjusted mortality with another vendor. The Annals of Internal Medicine reported earlier this year that using different rules to attribute patients to physicians led to as many as 61% of physicians being misclassified individually. 

What does this mean? 

It’s not easy to distill the quality of a teacher (or a physician) into a single number.  The validity of evaluation seems to increase with more data; this is a big problem on the physician side where no single insurer has all claims data.  Risk adjustment of some sort is important – but risk adjustment is contentious and will sometimes just sow more confusion.  The validity of an evaluation is highest when you look at a large group – rather than an individual physician (or teacher).   But, parents want to know the evaluation of their kids’ teachers, just as patients want to know how their doctor scored.  There is a valid argument in both settings that physicians (and teachers) work as teams – so perhaps we should be satisfied with team ratings. 

Should we pay for performance?   A Finnish educator, basking in the glory of a recent OECD report extolling the success of the Finnish public schools, says that the real answer is to treat teachers with respect and honor, rather than grading teachers on a curve.  He advocates equity and cooperation, rather than choice and competition.   In education, as in health care, not everyone will be able to have the best teacher (or doctor).  So making raising the bar for the education and health care for all is exceptionally important.

We in health care should be carefully watching the debate on teacher evaluations and teacher pay. 

Read More

Saving Money and Saving Lives

Today’s Managing Health Care Costs Indicator is
$86 Billion

A brief article in yesterday's New York Times notes that California, by having among the highest tobacco taxes around, and by spending significantly on tobacco education and counter-marketing, has the lowest adult smoking rate of any state except Utah, and has seen rats of lung cancer decrease at a rate three times as fast as the rest of the country. All told, the state calculates that it has saved $86 billion in health care costs.  (I can't find the actual report, so I would discount this somewhat). There's more detail at this blog, and here's a link to a press release from September.

Read More

Hans Rosling's 200 Countries, 200 Years, 4 Minutes

This is a fascinating video from the BBC with a statistician showing increasing wealth and life expectancy by country from 1810 to 2009.  The visualization is excellent - and the impact of the WWII and the swine flu epidemic is impressive. More impressive (and not commented upon) is the fall in life expectancy in China associated with the Great Leap Forward in the  late 1950s.

What does this have to do with health care costs?  The places where health care is most expensive just so happen to be those with the highest GDP -and in general they have the longest life expectancy.   Increasing wealth is enormously important to improved health - all the more reason we should manage our economies for robust growth.  When health care costs end up being so high that they are a major impediment to growth, that can actually lower health!

Read More

Where Doctors Are Paid Too Little

Today’s Managing Health Care Costs Indicator is $900

That’s how much a Czech physician makes each month, NPR reported this weekend.  

 “Pay doctors more than a streetsweeper” reads the sign on a retired ambulance piloted by a doctor campaigning for better wages.  Czech Republic physicians can move to Germany or Austria and increase their wages by a factor of 5, while cutting their hours in half.

Physicians in the United States are on the average the best paid in the world, and that is part (but only part) of why health care costs here are so high.  See this post from last fall for more details. But paying doctors too little can lead to a terrible brain drain in a world where professional skills are transferrable across national boundaries.   The remaining physicians will have inadequate capacity to care for all Czech residents, which will lead to a decrease in the quality of care

Clearly, the Czech Republic pays its physicians too little.   That might look a way to save money, but it’s not likely to be effective at delivering high quality health care over the long run.

Read More

Rationing Health Around The World

The World, a public radio show cohosted by the BBC and the Boston NPR station, had a four part series on rationing health care  around the world last week.  The radio shows as well as the web material help demonstrate how hard it is to distribute scarce health resources. 

The series starts in South Africa, where committees in hospitals determine who gets dialysis and who doesn’t.  These committees started by considering “social worth,” but have moved to prioritizing those who would be good candidates for kidney transplants, and who could therefore get off dialysis quickly.  If you’re a drinker, or if you’re overweight – well, dialysis could save your life, but it won’t. 

The series moves on to the UK, lauded by health policy experts around the world, where the National Institute for Clinical Excellence (NICE) assesses comparative effectiveness and determines how much the National Health Service (NHS) will spend to save a quality adjusted life year (QALY).  Right now the number is south of $50,000.  If a new drug might help you but would cost more, it isn’t covered.  NICE has been successful at pressuring drug companies to lower prices in exchange for access to the NHS market – but its authority to do so expires in 2012.  It’s hard to feel good about denying a drug that could save someone’s life.

Next stop is Zambia, which can’t afford HIV drugs for all the HIV patients.  The country has long had poor governance, and has a shortage of health personnel and medicines.  The main method of rationing in Zambia is the queue.  People have to wait a half day for a brief clinician appointment, and another half day for the pharmacist to fill a prescription. Those who cannot wait – perhaps because they are employed or have young children – go without life saving medications.

The final installment is a visit to India, where a resourceful pediatrician faced a shortage of ventilators during an influenza epidemic.  She fashioned homemade continuous positive airway pressure machines from a few dollars of readily available supplies, and saved  many dozens of lives.  This is an example where disruptive innovation made an enormous difference.  The homemade machines weren’t nearly as good as ventilators for these critically ill children.  But the makeshift machines were better than nothing, and the physician improved them “on the fly” during the epidemic.

These stories point out that rationing is painful – and it’s not restricted to poor countries.   We already ration care in the US by rationing access.  It’s not easy for a Medicaid patient to find a dentist or a specialist in many states, as Medicaid pays very low rates.  We might give FDA approval to a $90 a month medicine for cancer, but few eligible patients who don’t have generous insurance coverage will benefit from these medicines.  In Arizona, Medicaid beneficiaries are dying because the state will not fund evidence-based transplantations. 

We’ll be having more distributive justice conversations in the US, because we won’t be able to afford uniform access to all health care innovations as they are currently priced.  The best way to minimize the number of times we refuse to offer useful therapy to patients is to lower the cost of interventions, but we’ll still face a demand for more health care than we can afford.

Read More

Orthopedist vs. Anesthesia and a Pair of Articles Putting the ACA in Historical Perspective

Extranormal produces great 'homemade' videos.  Some of the pronunciation of medical terms is rough (asystole = the heart has stopped for my nonmedical readers. Temperature of 29= 84 degrees F, and normal pH is 7.4)

By the way, a nicely paired set of articles by perceptive NY Times journalists today and tomorrow.

Today, David Leonhardt reminds us that this quote

“We are against forcing all citizens, regardless of need, into a compulsory government program,” said one prominent critic of the new health care law. It is socialized medicine, he argued. If it stands, he said, “one of these days, you and I are going to spend our sunset years telling our children, and our children’s children, what it once was like in America when men were free.”


is from Ronald Reagan talking about Medicare in the 1960s.   He goes on to discuss the tension between Americans who believe in individual responsibility (the laissez fair conservatives) and those who believe in a minimum standard of living (progressives.)  Both traditions have played a role in America's success; the tension remains


Tomorrow, Matt Bai reminds us that Social Security started collecting premiums in 1935 but didn't pay out pensions until 1941 - and was under siege for decades until it became a critical part of our social fabric (and until most families were getting some benefit from the program).  He suggests that the final analysis of the Affordable Care Act can't be written for a generation. 

Read More

URL Dump

So many good articles, and so little time to blog about them.

A few articles worth bookmarking:

12/11/10 Associated Press reports on efforts in Massachusetts to lower costs

12/10/10 WBUR reports on a large delivery system joining a Blue Cross Blue Shield global payment system  

12/12/10: NYT business section article pointing out that physicians right now offer less access to those in low-reimbursement health plans such as Medicaid.   We're rationing care - just not explicitly

12/13/10: New Yorker article ‘The Truth Wears Off’ explains why clinical trials initially report huge impact, and that impact is much lower in the real world.  This article is behind a paywall, and I will be writing a blog about implications of this article later this week. 

12/12/10: New York Times editorial on the deficit and health care 

12/12/10: New York Times report on spouses getting Medicaid coverage even though they have the means to pay for nursing home care 

Read More

Urologists Recommend IMRT for Prostate Cancer – And Double Their Income

Today’s Managing Health Care Costs Indicator is $40,000

The Wall Street Journal continues to review Medicare payment records and ask probing questions about where the money is going.  I’ve previously recommended the WSJ evaluation of some of the high-billing probably fraudulent metro New York primary care physicians

Last week, the WSJ turned its attention to intensity-modulated radiation therapy  (IMRT), which is used to treat prostate cancer.  Among the treatments for prostate cancer are watchful waiting (small cost), surgery ($16,000 by the WSJ – sounds low to me), radiation seed implant ($19,000), and IMRT ($40,000). IMRT uses a computer to do a 3-D reconstruction to limit the amount of radiation delivered to noncancerous tissue.

The WSJ posts estimates from vendor about how IMRT can boost the income of a urologist by $336,000 if s/he refers just two new cases a month.  IMRT became available in the mid-2000s, when the urologists were suffering from serious loss of income after Medicare clamped down on profiting from prostate cancer drug markups.

Self referral continues to drive health care costs higher.  I’ve blogged on this before – there are higher rates of rotator cuff surgery in the practices of surgeons  who own ambulatory surgery facilities, and orthopedists who own MRIs and CTs refer to their machines with apparently excessive frequency.   Here’s a link to an older review of the literature on self referral.      

I don’t think many urologists who give IMRT believe that they are overutilizing this procedure.   However, the potential to double income is highly likely to have a subconscious effect.   We need to get away from paying for each unit of service, and dangling almost irresistible incentives in front of those who we expect to make decisions in the best interests of their patients.

Read More

Medical Bills Lead to Refinancing Woes

Today’s Managing Health Care Costs Equation is $11+$11=$14,000

There have been many reports of medical bills contributing to personal bankruptcies in the United States.   In 2007 researchers suggested that medical problems were associated with 62% of personal bankruptcies (and almost four in five of those who had medical cost-associated bankruptcy had health insurance).   

The Wall Street Journal reported this week that small unresolved health care bills were taking a bite out of credit scores from some unaware consumers – leaving them unable to get inexpensive refinancing.    Consumers could pay decades of higher interest because they were turned over to collection agencies for tiny bills.   One Texas resident reported that refinancing would carry $14,000 in fees since she had two $11 physician charges which had been turned over to a collection agency.

Thanks to John Donahue for this suggestion.

Read More

Health Care is Robbing Our Children’s Education

Today’s Managing Health Care Costs Indicator is $1 Billion

All countries eventually spend 100% of their GDP – and why not spend more on health care, which offers us benefits we really value?

Well –because health care costs ‘crowd out’ other important services, including societal functions that lay the groundwork for our future economic prosperity.  I'm thinking again about education. 

This month, Massachusetts schools are singled out in The Atlantic Monthly  as being far better than those of other states, although far inferior to the schools in many foreign countries. 

This week, however, The Boston Foundation  released a report detailing how health care costs across the state are putting our children’s education at risk.

Health care costs have risen on average 13.6% per year from 2000-2007, while general cost of living has only risen 3.4%.   Health care costs went up a staggering $1 billion from 2000-2007 for Massachusetts school districts – gobbling up every cent of increased state aid during that time, and devouring an extra $300 million in addition that required cuts elsewhere in the budget.   High health care costs for state workers have also hobbled the state’s ability to provide full funding for poorer school districts. The report notes that “from 2000-2007, increased spending on health care consumed 2/3 of the entire increase in state spending”

This data complements a report from Peter Orszag in September which showed that as states spent more on health care, they underfunded public colleges and universities.

Here is how Massachusetts school systems are coping with the 144%  increase in the cost of health care from 2000-2007.

-        Purchases of textbooks  ↓ 57%

-        Teacher training ↓ 23%

-        Overall non-inflation adjusted budget excluding health care ↓ 2%

Click to enlarge 

During this time, school budgets would have had to increase by 26% just to keep up with inflation. The report also noted that school districts in poorer communities have fared most poorly, while wealthy districts were in a better position to address rising health care costs by increasing tax rates. 

If we want to have world class schools for our kids, we have to control the costs of health care.

Read More

Smoked Pig

Today’s Managing Health Care Costs Indicator is $2159

Today’s New York Times features another tale from the land of variation.

  

Cardiologist Mark Midei inserted 30 cardiac stents in patients in a single day in 2008. The stent manufacturer (Abbott) purchased a $2159 barbeque dinner including a slow-smoked whole pig for the cardiologists’ home two days later.

The pig is a great headline, and a savory metaphor for the problem of overuse of high-margin procedures, some of which are invasive and have clear associated dangers.  The hospital subsequently notified 585 patients that they might have received medically unnecessary stents.  585!  This is mind-boggling.  

This story, initially reported in the Baltimore Sun, is reminiscent of the Tenet hospital in Redding, California.  That hospital built an empire providing cardiac surgery with very low complication rates – but it turned out that many of the cardiac surgery patients had normal coronary arteries. See Shannon Brownlee's excellent 2007 book Overtreated for the details.  

It’s especially interesting how this physician’s behavior was revealed.  His cardiology group had agreed to be purchased by the local hospital, but the hospital turned around and made him a separate deal and decided not to purchase the entire group. At that point, his spurned group attacked him, and a former chief executive vowed to “spend the rest of my life trying to destroy him personally and professionally.”  The hospital has paid $22 million to settle federal charges of kickbacks to Dr. Midei's former group. 

Most variation in health care is not as extreme as the allegations at St Joseph’s in Baltimore.   Most physicians who practice at the 90^th percentile of resource use don’t realize that they are using more resources than their colleagues, and few believe that they are performing unnecessary care.   We need better reporting on resource utilization and continuing effective peer review.  It’s a pity when peers are only effective at reining in rampant overutilization when they no longer inure benefit from that utilization.

Read More

Death Panels are Here!

Today’s Managing Health Care Costs Indicator is $4.5 Million

Well, we have a death panel.  It’s the Arizona legislature.

Arizona was the last state in the union to implement a Medicaid program (jointly funded by the state and the federal government).  The program is aptly called the Arizona Health Care Cost Containment System.   

Arizona faces a yawning budget deficit –as do many states.  The bursting housing bubble hit Arizona hard, and tax collections are way down.  The state has already sold and leased back its capitol building, and new taxes are off the table.

The state determined that the success rate of heart, pancreas, lung and liver transplants is too low to justify the high costs of these procedures, and stopped paying for these as of October 1.  

Here’s what this means for a patient:

Francisco Felix, 32, a father of four who has hepatitis C and is in need of a liver, received news a few weeks ago that a family friend was dying and wanted to donate her liver to him. But the budget cuts meant he no longer qualified for a state-financed transplant.

He was prepared anyway at Banner Good Samaritan Medical Center as his relatives scrambled to raise the needed $200,000. When the money did not come through, the liver went to someone else on the transplant list.  NYTimes December 3  

There is no easy answer here.  The state has too many needs for its limited budget, and these transplants are indeed enormously expensive.   However, transplants have proven to be effective, and the alternative for patients is death.  It’s not clear that a state agency is the best party to evaluate success rates of procedures, and the Times reports that “national transplant groups call the figures misleading.”

We need to eliminate unnecessary medical expenditures to leave room in the budget for proven effective therapy, even when it’s expensive.  There are 100 people on transplant lists in Arizona who will be removed if the legislature doesn’t reconsider. The total projected savings are $4.5 million. 

Read More

Variation in Practice: Rosiglitazone Case Study

Rosiglitazone, a drug used for treating diabetes, has been shown in recent studies to be highly associated with increased risk of cardiovascular complications. While it's effective lowering blood sugar and helping patients attain a good hemoglobin AIC, which is how we judge diabetes control, it appears to increase the chances of a bad outcome.

The initial study showing that this medicine appeared to be 'trouble' was a metaanalysis published in 2007, and accompanied by an FDA "black box" warning.  The good news is that this new information and warning was associated with an impressive ~45% drop-off in drug utilization.  We often worry about the slow pace of incorporation into practice of new information.  In this instance the communication around the dangers of rosiglitazone looks like a big success.

click to enlarge 

These images come from an article in the New England Journal of Medicine on November 25. 

Here's a worry though.  Look at the geographic variation of use of this drug from 2005 to 2010.

Click image to enlarge 

The dropoff in rosiglitazone use is pretty uniform across the country- about 75% decrease in use for each quartile of previous utilization.  (The authors don't give state-specific data).  However, the northern plains and New England states had a low rate of use of this medicine in the first place -and maintained this over time.  Use of this (expensive) medicine represented an exceptionally large portion of diabetes medication costs in states including Idaho, Utah, Wyoming, Oklahoma, and Kentucky in 2005. Although these states used the highest amount of an exceptionally expensive diabetes medicine - they are not states known for differentially higher quality diabetes care.

This is another illustration of the Dartmouth Atlas contention.   Variations in overall cost usually don't purchase better quality - and sometimes purchase higher risk.

Read More