Economist Emily Oster argues that because randomized studies don't reproduce the results of observational studies, for the vast majority of people vitamin supplements probably aren't beneficial.
In these trials, participants were randomly assigned to take supplements. Because the assignment was random — and the trials were big — the demographic and health characteristics of the supplement group and the non-supplement group were similar before the study started. When researchers looked at participants’ health over the long term, they could therefore be confident that any differences they saw across groups were due to the supplements, and not some other factor.
When the results of these studies came out, they largely refuted the idea that these supplements offered benefits.
Which conclusion would be fine if we were confident of the randomized studies' results. I can, offhand, think of two potential problems.
1. Were the vitamin dosages in the treated groups sufficient? In a metastudy that Ms. Oster links to the treated groups were given, apparently, an average of 2000 IU of vitamin D per day. That dosage, at least for some people, may well not be enough. The right amount depends on maintaining a high enough blood level, a level which depends on a number of characteristics, including age and weight.
2. Another potential problem is whether the range of the data is large enough. For instance, Ms. Oster cites the Physician's Health Study, a study of "14,000 male physicians". What if physicians were really healthy? (I've seen my share of doctors and I've seen only one that might have been overweight, and he wasn't seriously so.) The marginal effect of treating one group might be very hard to detect.
An example of this problem from education: some people argue that SAT scores don't predict college grades very well. They cite studies of Ivy League students. And yes, the difference between 1400 and 1450 (on the old 1600 scale) might have little correlation with college grades. But the difference between 900--those folks should be relatively rare at Ivy League schools--and 1400 can, and seemingly does, matter a lot.
A final point: maybe supplements are a waste of money for most people, most of the time. But so are all forms of insurance.