A Case Against Mission-Critical Applications of Machine Learning

IN THEIR COLUMN “Learning

Machine Learning” (Dec. 2018),

Ted G. Lewis and Peter J. Denning

raised a crucial question about

machine learning systems: “These

[neural] networks are now used for critical

functions such as medical diagnosis …

fire-control systems. How can we trust the

networks?” They answered: “We know

that a network is quite reliable when its

inputs come from its training set. But

these critical systems will have inputs

corresponding to new, often unanticipated

situations. There are numerous

examples where a network gives poor

responses for untrained inputs.”