CODA: Consensus-Driven Active Model Selection

Wildlife Photo Classification Challenge

You are a wildlife ecologist who has just collected a season's worth of imagery from cameras deployed in Africa and Central and South America. You want to know what species occur in this imagery, and you hope to use a pre-trained classifier to give you answers quickly. But which one should you use?

Instead of labeling a large validation set, our new method, CODA, enables you to perform active model selection. That is, CODA uses predictions from candidate models to guide the labeling process, querying you (a species identification expert) for labels on a select few images that will most efficiently differentiate between your candidate machine learning models.

This demo lets you try CODA yourself! First, become a species identification expert by reading our classification guide so that you will be equipped to provide ground truth labels. Then, watch as CODA narrows down the best model over time as you provide labels for the query images. You will see that with your input CODA is able to identify the best model candidate with as few as ten (correctly) labeled images.