Vladimir Haltakov, Self-Driving Car Engineer at BMW Group, presents the “Is My Model Performing Well? It Depends…” tutorial at the May 2021 Embedded Vision Summit.
There are many statistical metrics used to measure the performance of machine learning models. While they work well when the model itself is the final product, classical measures are often not enough when the model is a part of a more complex system.
In this presentation, Haltakov focuses on products where machine learning models do not generate the final output. The challenge in these cases is to define a metric that accurately captures the effect of different failures of the model on the final system performance. This becomes even more difficult when there are conflicting requirements within the system. He discusses strategies to deal with these problems and shares insights from real-world examples.
See here for a PDF of the slides.
