Gary N. Smith, Fletcher Jones Professor of Economics, Pomona College, addresses something I've wondered about but haven't yet taken the time to try to figure out: how do these machine learning algorithms establish causality?
Artificial intelligence (AI) algorithms are terrific at discovering statistical correlations but terrible at distinguishing between correlation and causation. A computer algorithm might find a correlation between how often a person has been in an automobile accident and the words they post on Facebook, being a good software engineer and visiting certain websites, and making loan payments on time and keeping one’s phone fully charged. However, computer algorithms do not know what any of these things are and consequently have no way of determining whether these are causal relationships (and therefore useful predictors) or fleeting coincidences (that are useless predictors).
And here he proposes the "Smith Test" to determine if computer algorithms can make reliable recommendations.