Introduction to ML Strategy

Why ML Strategy?

Orthogonalization

Setting up your goal

Single number evaluation metric

Satisficing and optimizing metrics

Train/dev/test distributions

When to change dev/test sets and metrics

Comparing to human level performance

Why human-level perfomance

Avoidable bias

Understanding human-level performance

Surpassing human-level performance

Improving your model perfomance