Newmark's Door

Another excellent post by Arnold Kling.

W. Edwards Deming distinguished experiments from tampering. With an experiment, you change a process and explicitly compare the results to a baseline. With tampering, you change the process without rigorously examining the results.

For example, in education, most curriculum changes involve tampering. Schools rarely test to see whether a curriculum works.

I once sat next to a high official in the Department of Education, and he was horrified when I suggested experiments in education. “Would you want your child to be part of an experiment?” he asked, incredulously. “The schools do it all the time,” I responded. “They just don’t bother checking to see whether their experiments work.”