'Policy based evidence' criticised
Politicians push for evidence that supports whatever initiative they back, but formal evaluation often shows such initiatives to have failed, according to a former health tsar.
"We need evidence based policy, not policy based evidence," says David Oliver, former national clinical director for older people at the Department of Health, in an article in The BMJ this week.
In July this year commissioners throughout England published projections for reductions in urgent admissions to their local hospitals. "But the size and speed of these reductions were not informed by any credible peer reviewed evidence - they rarely are," writes Professor Oliver, visiting professor of medicine for older people, School of Health Sciences, City University, London.
Similar annual projections have been made for at least a decade, he says. "Yet although we have lost about one third of acute and emergency hospital beds in England in the past 25 years, emergency admissions have risen by 37% in the past decade."
He adds: "This farcical game represents a triumph of management consultancy over evidence, and of hope over experience."
The Department of Health is a serial offender of seeking "policy based evidence," argues Professor Oliver.
"A minister has a bright idea. He or she commissions an 'evaluation.' The minister and NHS leaders prematurely overclaim benefits before any research is published. The evaluation is rigorously scrutinised by independent researchers, using control data. The promised benefits are found to be illusory. The minister has moved on.
"The next minister with the next big idea starts the cycle anew. Researchers and frontline services receive no apology for the spin, wasteful service changes, or the hijacking of the research process."
Prof Oliver points to the Department of Health's partnerships for older people projects, which claimed big benefits and savings, but when formally evaluated "proved disappointing, despite the earlier spin."
Having been so critical, he offers some constructive solutions. He believes that services "should make three to five year plans for sustainable service change, based on evidence of what works and, crucially, what doesn't".
He adds: "We should stop wasting money on costly and often poorly evidenced consultancy reports and short term pilots leading to no sustainable services."
Locally, he suggests "we should consider the best effects, as identified by peer reviewed studies, and halve the projected effect, and double the time to achieve it" while nationally, "politicians must stop the cycle of eye catching short term initiatives and pilots".
Finally, he says politicians "should respect evidence, researchers, and due process and not use it for their own ends".
He adds: "It would be more honest to start an initiative because 'we think it makes sense' than to commission research, push for findings that justify the policy, and then exaggerate the benefits before it's even published.
"In short, we need evidence based policy, not policy based evidence."
A Department of Health spokesman said: "All policy is based on the best evidence available, and new schemes are often piloted or tested before they are considered for national roll out.
"For example, the full results from a large and peer-reviewed study showed that overall, telecare and telehealth can help people with long term conditions live independently and improve their quality of life. The study was evaluated thoroughly by independent researchers and published in the British Medical Journal."