We spend many billions of dollars each year on the discovery and development of new drugs, but almost none of it addresses two crucial questions: How do these new therapies compare with already known ones? What are the relative benefits and harms in a particular situation for a person like you?

Such questions can best be answered by comparative-effectiveness research.

To get approval from the Food and Drug Administration, drugs must be proved both effective and safe. The costs of doing this are significant and they are most often borne by the pharmaceutical industry.

But the FDA’s bar, while meaningful, often is not very useful for what physicians and patients really care about every day: how effective and safe drugs are compared with one another.

Real-world questions

Consider antibiotics. In my work as a pediatrician, questions about their use come up a lot. Which drug is the best first-line therapy for which common illnesses? We don’t know. How long should we treat for different infections? We don’t know. What are the relative trade-offs between benefits and side effects in different patients in different circumstances? We don’t know.

The questions we need answered are legion. All the guidelines and practices we have are best guesses.

Comparative-effectiveness research can take on many forms and involve more than drugs.

Because of a trial published in The American Journal of Preventive Medicine, we know that an intensive lifestyle intervention works better than metformin to promote weight loss.

A study published in JAMA Pediatrics last year showed that adding individual health coaching to enhanced primary care reduced pediatric obesity, but no more than enhanced primary care alone.

A comparison of different levels of insurance published in The New England Journal of Medicine showed that enhanced drug insurance coverage led to increased medication adherence, lower patient spending and lower rates for a first major vascular event (like a heart attack or stroke), with no increased overall health care spending.

A blood pressure study

We know that high blood pressure is both terribly prevalent and a significant risk factor for cardiovascular disease. We also know that there are a lot of drugs out there, all FDA-approved, that can help reduce this risk by better controlling blood pressure. But which is best?

This question is not new. In 2002, the results of the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial — a comparative-effectiveness trial — were published in JAMA.

Participants had to be at least 55, have hypertension and have at least one other risk factor for coronary heart disease. They were randomly assigned to take one of four drugs, each with an entirely different mechanism, representing a different class of drugs.

Chlorthalidone is a diuretic, or a drug that increases urine output. Amlodipine, a calcium channel blocker, causes blood vessels to relax and widen, and lowers the heart rate. Doxazosin does the same by blocking the effects of adrenaline on muscles throughout the body. Lisinopril blocks the enzyme angiotensin, which tightens blood vessels, leading to lower blood pressure. All the patients were tracked for four to eight years.

All of these drugs had been proved safe and effective; we just didn’t know what worked best as a first-line therapy for the many people with high blood pressure.

The main outcomes of interest were death from coronary heart disease, or a heart attack that did not lead to death. By those measures, there was no difference between any of the four drugs in that period. But chlorthalidone outperformed two of the others in lowering systolic blood pressure. That drug also performed better in preventing heart failure (a gradual weakening of the heart) and stroke, and lowering rates of cardiovascular disease.

The take-home message was that the diuretic was better in preventing at least one of the major types of cardiovascular disease. It was also the least expensive.

As you can imagine, this is immensely valuable information. It tells us what drug is best to start if you have someone over 55 with high blood pressure and at least one risk factor for coronary heart disease. That is exactly the kind of question that only a comparative-effectiveness trial can answer.

This study was enormous; it took place in 623 centers in Canada and the United States between 1994 and 1998, and included over 33,000 participants. It also cost more than $100 million and that was two decades ago.

So was the debate over? Far from it. A stunning number of papers have been written critiquing this study.

There are methodological concerns, in that the primary endpoint (death and heart attacks) was somewhat ignored in favor of secondary outcomes like strokes and blood pressure. Most of the patients had probably been on other therapies before starting the trial, so it is not clear if prior therapy could have changed results. Many patients received more than one drug and the stepwise addition (adding drugs one at a time) might have favored the diuretic. Significantly more people on the diuretic developed diabetes than those on other drugs. And in the many years since the trial, these drugs have all become cheaper and generic, and more drugs have appeared, making the answers somewhat murky once again.

Basic science research is necessary to make breakthroughs in how treatments might be created. Randomized controlled trials are necessary to determine if they have efficacy. Pragmatic trials can tell us if and how they are effective in real-world settings. Health services research can improve the ways in which we deliver them. But without comparative-effectiveness research, too many important questions that concern patients will remain unanswered.