Medical Research, Statistics, and Conclusions

 

An important part of my education at Case Western Reserve Medical School in the early 1970s was the interpretation of medical research and writing.  I’m concerned that this critical skill has become less of a focus for many physicians and in many of today’s medical schools.

 

Because most of today’s medical research is supported directly or indirectly by the medical industry, the conclusions should be suspect.  Stated conclusions are often poorly supported by the actual data.  Spin generally favors the benefits and understates the negative aspects of medications and devices that generate industry profit.  It is essential that informed physicians critically evaluate results and conclusions.

 

I’ll offer some (not entirely) hypothetical examples.

• A new anticoagulant used for atrial fibrillation lowers the risk of embolic stroke from about 2% to 1%.  The risk of an equally severe or fatal bleed, frequently into the brain, from this medication is about 3% compared to less than 1% without the drug.  This risk is prominent in older patients who are prone to falling.  It may be stated that the new medication decreases the risk of embolic stroke by half while causing only a 2% increase in the risk of serious cerebral or other hemorrhage.  It may appear that anyone with atrial fibrillation should take the new treatment.  Or does the treatment actually triple the risk of a serious problem?  And what about alternate and equally effective treatments that may be less risky?  How would you expect for this drug to be marketed?

• A chemotherapeutic drug called Mitomycin can be instilled into the bladder following removal of a small low-risk bladder tumor.  This treatment decreases the risk of having another small bladder tumor by about 25% (compared to no postoperative treatment) over the following 2 years.  The “need to treat” is about 8, meaning that 1 patient is spared a second procedure for every 8 patients who receive the Mitomycin.  The use of this treatment became an American Urological Association “Standard of Care”.  Yet, only about 25% of practicing urologists followed this recommendation.  It was rarely mentioned that about 2% of patients who received the Mitomycin developed severe and sometimes permanent complications.  Additionally, these potentially avoidable second tumors could be fulgurated as an outpatient, and the patient could immediately return to normal activities.  When I was a busy urologist, I did these subsequent procedures without sedation or anesthesia using a Nd:YAG laser.  The risk of progression or death from these recurrences was essentially nonexistent.  This recurrence and treatment was just an inconvenience. The benefit/risk was about 6 patients needing a minor procedure to avoid each life-changing complication.  Why would a patient or an accurately informed physician take this risk?  I commend the majority of urologists who refused to comply with this AUA guideline.

• A pharmaceutical representative came to my office to promote testosterone gel.  She advised us that the majority of older men had an abnormal testosterone level.  Her position was that these men should be treated with her product.  If most men have a condition, the condition is, by definition, normal.  Testosterone products have been heavily marketed, and these drugs have generated massive profits for the pharmaceutical industry and for the “testosterone mill” physician offices that have excessively prescribe these agents. I see many parallels to the opioid excesses.  The harm from testosterone supplements is rarely acknowledged.  Product representatives are basically minimally-informed individuals who are paid to repeat talking points as instructed by the industry.  They should never be trusted for medical advice.

 

 

The FDA has not been an accurate source of medical advice.  This is because of their process.  Some excellent medications have been proven to be highly safe and effective, based on well-conducted research, and they have become the standard of care among informed physicians.  Yet these same medications are not and never will be approved by the FDA for these indications.  These medications are not profitable for the pharmaceutical industry because they have been available for too many years to be patented.  FDA approval is expensive, and nobody wants to pay the tab unless they can recover their costs.

 

An example is tamsulosin (Flomax).  This drug has been FDA approved only for the treatment of voiding difficulties.  Yet, multiple good studies have shown benefit to assist the passage of ureteral stones.  American Urological Association guidelines advise this medication to assist in stone passage, yet FDA approval is not anticipated.  Tamsulosin is an inexpensive generic medication, so nobody can recover the cost of an FDA application.  Beyond that reality, tamsulosin use to assist stone passage is allowed and consistently used even without formal approval.

 

Other medications that are neither safe nor effective were approved decades ago based on poorly-conducted research.  Subsequent high-quality research has established that these drugs are useless and potentially dangerous, yet FDA approval has not been withdrawn.  Again, nobody is paying the tab for this effort.  Two good examples are Urecholine for urinary retention and Elmiron for interstitial cystitis.  Good studies show both drugs to be useless and potentially harmful.  Still, pharmaceutical product representatives continue to effectively market these drugs to intellectually-limited physicians, citing the FDA approval to support the use these medications.

 

In recent years, the FDA has been increasingly subject to political influence rather than scientific research.  Clearly, the FDA has failed the public.  Although it is important to know the FDA position on drugs and treatments, good physicians need to make decisions based on actual science.

 

 

I have been observing litigation over the opioid crisis with interest.  The pharmaceutical industry certainly shares much of the blame.  Their marketing and profit-driven dishonesty should be punished.  But mostly, I blame the physicians who prescribed the opioids.  Certainly, physicians were bombarded with misinformation from the industry encouraging these prescriptions, but competent physicians should not have based their decisions on marketing.  I have always known that opioids were addictive and that they should not be used for chronic pain.  This was never a secret.  Beyond incompetence, greed played a part in excessive and improper prescriptions.  This too should be punished.  While drug distributers and retailers could have recognized the issues, their roles were complacent rather than central.  Furthermore, the role of the government in encouraging excessive opioid use was central.  Physicians and hospitals were financially punished if patients perceived inadequate pain control.

 

Some types of stated medical conclusions are particularly troublesome for me:

• Certain cancer treatments are stated to “decrease the risk of death”.  We all know that the risk of death is 100%, so what do they mean by this?  If they mean decrease the risk of death within one year or within 2 years, what happens to the others?  Are they all dead 2 months later, or do the others live for another 5 years?  It matters.

• Other cancer treatments are stated to decrease the “cancer specific mortality”.  Do these patients live longer, or do they die sooner from a different cause, perhaps related to the treatment?  It matters.  The most certain way to keep a patient from dying from cancer is to quickly cause his death from another cause such as a toxic treatment.  We should really look at overall mortality and we should probably report quality-adjusted years rather than total years.  Some lives are hardly worth continuing following morbid treatments that ruin the quality of life.

• Another problematic metric is years of life following diagnosis.  Earlier diagnosis will almost always result in more years before mortality from the diagnosed disease.  Earlier diagnosis with ineffective treatment is likely to decrease total longevity while giving the appearance of benefit.

• I’m troubled by the often-stated fact that screening and early treatment of prostate cancer decreases the frequency of patients who are diagnosed with advanced prostate cancer.  The statement is accurate, but it implies a benefit that may not exist.  Just because the diagnosis is reported before advanced disease develops does not establish that the same patient does not progress to advanced disease at the same age that would have occurred without the early diagnosis.  Mammography for early diagnosis of breast cancer suffers from the same problem and appears to have the same outcome.  Early diagnosis and treatment has not greatly decreased the prevalence of eventual advanced disease.

 

 

Today’s physicians are inundated with medical articles and new research.  Considering the demands of direct patient care and administrative overload, it’s easy to understand why few physicians are able to dive deep into the actual methods and data underlying medical research.  But it has never been more important.

 

Here is how I look at a medical journal:

• First, I consider the journal.  Is this a “peer-reviewed” and official journal, or is the magazine purely sponsored by the medical industry?  Industry magazines are interesting, but they are not a good source of reliable conclusions.

• Next, I look at the topic.  Most articles have no value for me and are not worth my time.

• If the topic is worthwhile, and particularly if in a peer-reviewed journal, I look at the authors and their “conflict-of interests” disclosures.  These disclosures will not keep me from reading the article, but conflicts increase my skepticism and scrutiny.  There are some proliferative authors that I know from experience to be intellectually dishonest.  I may look over the material, but I discount their conclusions. I don’t even fully trust their data.  These individuals make an exorbitant income from the industry, and they generally promote the highest bidder.

• On the infrequent occasions that I find an article that could change my opinion or practice, I carefully scrutinize every detail of the method and data.  My first thought is always one of skepticism.   What confounding variables could have influenced the data?  How many patients were involved?  Was this a multi-center study?  Was this a randomized and controlled study or simply a retrospective association?  Have the findings been confirmed elsewhere?  In my experience, when a finding does not seem right, I can usually identify a problem with the method or conclusion.  Often, the problem is blatant.

 

 

Unfortunately, misinformation and even corruption have become common in medical writings.  Statistics can be misleading.  Conclusions are often not well-supported by the data.  There’s just too much money involved in the medical industry.  Honesty and ethics are no longer the rule.  Good physicians need to take direct responsibility for understanding the science behind their decisions and medical advice.

 

 

RDB