ASH, unscheduled trips to the ICU, Iowa City drama. A lot has happened since I ran the last installment of Danny Parker’s series about blood glucose and metformin.
Here’s a link to Danny’s last installment:
This time, Danny reminds us why it’s so hard for researchers to crack the multiple myeloma code:
Proving Anti-Myeloma Drugs Work: Why it’s so Hard
Nature does not give up its secrets easily.
This is particularly true with complex systems. What’s a complex system? That person in the mirror: a human being.
As many of you know, I have spent many hours looking at cohort, meta-analysis and case-control studies looking at all kinds of issues associated with diet and myeloma. These data are sparse enough and hardly rigorous. I know the dangers of these studies and at least try not to cherry pick them.
I’m not interested in pseudoscience, nor should you be. The anti-vaccination movement tills that territory. A big difference for me and my examination of influences of diet and myeloma: you maintain a flexible attitude towards evidence in lieu of definitive clinical trials. And you know that while you are trying to exploit beneficial associations in the existing data, you can’t be sure of whether the recommendations will help. “The science is soft,” as Dr. Jeff Wolf, my myeloma specialist says.
However, I still believe it is useful to try to exploit evidence on how we might help ourselves. Most often, such studies are all we have. And it is quite true that I tend to pay attention when there are multiple studies pointing to the same factors (cruciferous vegetables and fish appear helpful to the reducing chance of developing myeloma, while sugar and butter do not). Of course, we hardly know if the same factors might be useful for people that already have myeloma.
Today we hear the repeated warning to would be data miners: correlation does not imply causality. And that caution is pounded into the skulls of budding statisticians in graduate schools everywhere. I heard it plenty too– and it is fair warning.
But one also does well to listen to the sage advice of Edward Tufte:
“Correlation is not causation,” he counsels,” but it sure is a hint.”
In other words, more studies and independent reviews of the same phenomenon by multiple sources showing correlation is a powerful hint that there may be causality lurking about. But while correlation is necessary for causation, the opposite is not true. Still, consistent evidence of correlation with studied variables versus a parameter of interest compels us to look further. And we should look if maintaining a skeptical stance.
Just because two trends are associated and follow each other lock-step does not mean that one causes the other. Indeed, the statistical topography of data analysis is strewn with failed associations that were taken as indicating that one factor was causing another. Some of these, of course, are laughable. Could increased apple imports be responsible for higher divorce rates? Of course not!
The reasons for such non-causal associations are beyond what I will write here for the myeloma community, but span the gamut from time related variation in two factors that are completely unrelated, the Simpson paradox where the trends in two groups reverse when they are combined (!), opportunity sample bias and even chance correlation. But the most common and insidious reason for non-causal associations is that many real and hidden causal factors in systems hide in associations with other parameters more easily observed and measured.
These are confounding variables that are notorious villains in the statistical slag heap of premature conclusions.
I wanted to go ahead and run Danny’s eighth installment today, too. That’s because I can’t wait for you to read his final conclusions; the real pay off:
8) Why is Myeloma Drug Research So Hard? (Part Two)
Why do confounding factors make myeloma treatment research so difficult? An example:
After WWII, it became apparent that smoking and lung cancer were associated. Multiple researchers were able to show clear correlation in small cohort studies, but was smoking causative? There had been a 600% increase in the rate of lung cancer in the preceding two decades. While it was clear there was a correlation between lung cancer and smoking, to prove smoking caused cancer was a much more difficult. And similar to the arguments over carbon emissions and climate change today, well-funded lobbies and powerful interests sought to sow doubt about the certainty of findings.
Might there be a hidden confounding factor that was responsible for the correlation between smoking and lung cancer? Could air pollution from more cars on the roads that grew with the popularity of smoking be a factor? Perhaps people who were more genetically predisposed to develop lung cancer were also those who took up smoking? Or perhaps the less well educated and those of lower income who tended the smoke also had much more deficient medical care. Could these be the actual factors involved? While any of those explanations might be plausible, it would take a survey of over 40,000 doctors in Great Britain followed over twenty years to reveal powerful statistical proof in 1954:
Even today, we still find that education level appears strongly correlated with lung cancer rates. While this association is real, it hides the fact that the most important driving causal factor is the rate of tobacco smoking which happens to be correlated with education level. But make an earlier premature leap and you could conclude that we can cut lung cancer rates with more time spent in college—which might be true, but lung cancer’s real smoking gun is smoking itself. One can be easily fooled by co-founders and serial association.And even cohort studies are no substitute for randomized trials.
For instance, in the 1980s, epidemiologic evidence suggested that beta-carotene might decrease lung cancer risk.
Double-blind clinical trials were initiated, and in the 1990s, those trials showed that not only is beta-carotene not protective, it actually increased lung cancer risk—16% in one study and 28% in another—and so the studies were halted. A follow-up study in 2004 corroborated those results.
And even well-trained doctors and diagnostic experts can be swayed by large numbers of cohort studies and a consistent chain of evidence. Back in April when I began to work on this series, I spoke with Dr. Vincent Rajkumar, arguably one of the preeminent myeloma researchers today.
As a cautionary tale, the good doctor told me about work he did back in 1997 where he and colleagues reviewed many studies showing hormone replacement therapy looked to be helpful for women to reduce their chances of cardiac events. They then composed a paper where they recommended such therapy be routinely recommended for post-menopausalwomen. They were confident the protocol would help and potentially save lives.
But they were wrong. Completely wrong.
A follow-up randomized study by the Women’s Health Initiative not only showed hormone replacement therapy was not helpful to reduce cardiac events, but, in fact, the opposite was true. It was unhelpful. Only a true randomized trial was able to reveal that bias in the opportunity samples was hiding the truth of the matter.
I was impressed by Dr. Rajkumar’s candor and my enthusiasm for showing the link between blood glucose, metformin and myeloma slowed the composition of this series. Why? While the evidence on blood glucose, diabetes and metformin is intriguing, it is hardly definitive. There might be other hidden factors at work. What to do?
On this, statistician George Box was aptly paraphrased: “The only way to find out what will happen when a complex system is disturbed is to disturb the system, not merely to observe it.”
Randomized clinical trials where the treatment is disturbed relative to the group receiving it and lurking confounding variables are randomly distributed across groups.
So, how to establish causality in a sea of cohort and case control studies? While there are now several studies pointing to metformin having large potential impacts on MGUS or myeloma, although intriguing they are not definitive.
I think we can readily agree that the human body is a complex system, so the recommendation is to carefully design experiments and randomize subjects so that the influences are randomized in a particular way so that we can find out what effects are truly caused by the factors of interest. And then to also use some of the other means to combat statistical bias: randomized studies conducted by other researchers and on different populations and with good sample sizes.
Will metformin help with efficacy of our conventional treatments such as Dexamethasone and Velcade? And will metformin and ritonavir turn out to be another treatment in the growing pharmacological arsenal against myeloma? Or another failed drug combination of many to have not been proven in the difficult arena of rigorous clinical trials?
We can’t know. I just wanted those of us in the myeloma world to understand the difficult challenge our doctors and medical researchers face.
Thank them, would you?
I’ll run Danny’ final installment about what you and I can do to help ourselves battle myeloma, soon. Tomorrow more about how my great, mind numbing news that Team Killingsworth’s radical, salvage modified tandem transplant therapy gamble has paid off.
Feel good and keep smiling! Pat