The Grossman and Mackenzie piece brings forward an important discussion regarding the dominance of randomized controlled trials (RCTs). There is certainly enough recent evidence to suggest that academic and industry research overly emphasizes the reliability. But I was a bit surprised by the authors’ position in some instances. Their criticism of the dominance of RCTs, for instance, states that there will never be a universal study design that can be applied to any situation. This seems correct, but implies that proponents of RCTs are apt to employ RCTs even when inappropriate. Perhaps RCTs are not as dominant as the authors suggest. I’m unconvinced that one of the two major problems with the “dominance” of RCTs is the naive view that RCTs can be universally employed. There are too many examples of non-RCT research to counter this view (i.e. all phase I and most phase II studies).
When criticizing randomization, the authors unfortunately neglect to take into account stratification. The problem that their hypothetical example of randomizing a small group of males and females presents can be addressed by stratifying sex prior to randomization. Stratification is almost always used within RCTs. It’s very important for the integrity of the data. For example, a large phase III, placebo controlled, study is evaluating the affects a non-steroidal anti-inflammatory drug has on the current standard of care for surgically resected colon cancer. The data would be compromised if all of the subjects in one arm happened to have 1- 3 positive lymph nodes and all of the subjects in the other arm have 4 or more. So a stratification must occur prior to randomization to ensure a “statistically healthy” representation in both arms.