Randomized control trials (RCTs) are considered the gold standard of evidence in the medical industry. This superiority has arisen because “randomized controlled trials are generally regarded as the most scientifically rigorous method of hypothesis testing available.”(Grossman and Mackenzie, 523) With an increasing obsession with unbiased evidence and objectivity over the last three quarters of a century, and RCTs abilities to reduce bias, they have received the privileged status of best method in every case. As noted by Marks, the use of randomization ensures that “neither our personal idiosyncrasies, consciously or unconsciously applied, nor our lack of judgement have entered into the construction of the two(or more) treatment groups and thus biased them in any way.” (Marks, 347)
The authors of this weeks articles argue that although RCTs may be a good method in some situations, there are methodological limitations to these studies and because of these limitations it is wrong to assume that these types of studies should be viewed as universally superior. This view creates an industry where what matters is whether or not the study is an RCT, when the focus should be on whether or not the study is producing useful and accurate results.
Harry M. Marks provides a history of RCT’s and looks at the social objects of mistrust. Marks discusses how RCTs emerged in the 1950’s through methodological reforms and the need to eliminate bias. “The obsession with “bias” Marks argues “owes something to the contemporary anxieties about the dark powers of subjectivity.” (Marks, 349) The problem is not that we sought after objectivity, the problem is that we settled with RCTs, believing that they brought the scientific method to clinical trials and alleviated all bias. Even though we have made progress since the 1950’s social mistrust continues and we are left with RCTs as the gold standard.
The acceptance of RCTs as the gold standard makes it such that well-designed observational studies are immediately dismissed without question, and assumes the flawlessness of RCTs. However, this is quite misleading as pointed out by Jason Grossman and Fiona J. Mackenzie who show that there are many things that can lead to poor quality RCT results including but not limited to; mistaken units of analysis, problems with blinding and use of suitable placebo, and false negatives. Rather than focus on answering the questions at hand and looking for studies that will provide the best, most useful answer, the industry creates the image that RCTs will always provide the best answer. Gorman and Mackenzie sum it up nicely when they argue that “the question of whether a study is an RCT or not trumps the question of whether it is a good study or not” (Grossman and Mackenzie, 524) and this is clearly highly problematic.
There is truth to the statement that RCTs and the techniques involved do reduce bias, but the word to be emphasized is ‘reduce’. Randomization and the other techniques utilized by RCTs are not enough to eliminate bias altogether. Furthermore, “the assumption that randomization always reduces bias creates what Horwitz (1987) terms ‘the illusion of homogeneity’ ” (Grossman and Mackenzie, 527) and this illusion leads to an ignoring of other factors that may cause a study to go astray. Moreover, many of the techniques RCTs deploy to rid the studies of bias such as randomization, control group, and blinding, are being utilized in other types of studies that may be more applicable to the project at hand. If the superiority of RCTs rests on their abilities to reduce bias, than it appears that RCTs should either be dethroned or many other studies, with similar abilities to reduce bias, should be brought up to the same level of prestige as RCTs.
There are many limitations to RCTs and one of the main limitations is with regards to sample size. A small sample size with many variables renders randomizing pointless because there is no way to balance all the variables. Another limitation arises from distributing subjects through randomization because some studies can attain more accurate results by distributing subjects in an systematic manner. Even further limitations to RCTs, such as bleeding off effects, arise in social contexts. Of course there are many instances where good, well orchestrated RCTs are beneficial but there are also many cases where their limitations cause them to provide worse evidence than other types of studies.
A further issue gets noted in the Maudsley debate and raised by Linsey McGoey, namely, that the recruitment of patients is “not representative of the vast majority of people who will be taking(the drug)”(McGoey, 61) because many individuals who suffer from more extreme cases of an illness are purposefully left out for ethical reasons. Although the ethical argument provides adequate reasoning to leave out particular subjects, it does not allow for a fair representation of the effects that drug will have on the the majority of individuals who will be taking it. Thus, the results derived from RCTs and applied in practice are misleading.
These are only a few of the limitations of RCTs and it may be thought that an increased awareness may change the view of RCTs as the gold standard. However, the opposite appears to be true. Linsey argues that despite our intuitions, awareness of our limitations actually further perpetuates the notion of RCTs as the gold standard. She argues that “the solution to failed RCTs is more RCTs” because they are accepted as the only credible means at which we can derive socially accepted evidence.
This paradox arises because the more we are aware of the deficiencies or limitations of a process, the more we rely on regulatory powers to provide us with information derived directly from the deficient source. McGoey shows that it is not our ignorance to the inadequacies of RCT’s that leads to an increase in the reliability of RCTs, on the contrary, the more we are aware of their limitations and “the more useless RCT’s are in practice, the more their strength is augmented” (McGoey, 74)
Because of this and the belief that RCTs are a form of “morally incontestable experimentation”(McGoey, 69) individuals within the industry who are aware of the limitations are still required to embrace RCTs in order to further their own interests. Thus, the ignoring of the limitations becomes useful to achieve one’s own ends. Furthermore, because of the universal recognition of RCT’s as the gold standard any issues with RCTs seems to be related back to the individual trial rather than the methodology itself. Thus, we either blame individual trials and leave the methodology alone and no change occurs, or attack the methodology and further increase it’s strength and change occurs but in the wrong direction. There seems no straightforward way to deal with the problems of viewing RCTs as the gold standard.
As pointed out in the Maudsley debate there is a further issues with how well “statistical significance from RCTs translate into clinical usefulness, uncertainty surrounding patient response, and uncertainty surrounding how well rating scales”.(McGoey, 68) The point is that although there may be a clear statistical benefit to a drug over a placebo this may not be the case in a practical, clinical setting. The case noted at the Maudsley debate was in regards to anti-depressants since the limitations of RCTs are “especially visible with psychiatric drugs” (McGoey,65) and as noted above applying the results of these RCTs in practice may be dangerous since the results are misleading when they are translated from statistic to practice.
Not only are there issues with how well statistical significance will translates over to a practical setting, there is also an issues with interpretation. This point is illuminated in the case where Kirsch and Turner and Rosenthal received almost identical results in regards to an anti-depressant over a placebo but Turner and Rosenthal “concluded antidepressants are clinically superior to placebo, while Kirsch concluded antidepressants are ineffective.”(McGoey, 67) These problems lead to greater uncertainties in trying to understand the clinical usefulness of drugs derived from RCTs. With the many limitations of RCTs including their misleading results it seems that the authors are right in questioning the universal superiority of these flawed studies.
RCTs do have many benefits and are useful studies in some areas, but to assume that they are the best option in every situation is mistaken, as highlighted in the case of antidepressants. Even if we accept that there has been “methodological progress” (Marks, 351) it certainly is not enough to warrant RCTs as the gold standard. What is important is trying to attain the best possible results for the issue at hand, but the perpetuation of the superiority of RCTs by companies such as MERGE further engrains them within our medical policies and clinical medicine. The authors this week clearly show that we need to give more credit to observational studies and remove RCTs status as the gold standard.
Prozac Time | Life O… on Prozac & Sarafe- Identical… shitangshuroy on What Constitutes ‘Genuin… shitangshuroy on Further Ethical Issues in… 14 to 40 percent of… on Publication bias shitangshuroy on Should Pharmaceutical Advertis…