AI-Model Weakness Detection

Project Description

In this project we researched weaknesses of AI-Models in Computer Vision, specifically for pedestrian recognition in autonomous driving. Computer Vision Models never work perfectly on all data. Singular images or even whole sets of classes get classified wrongly sometimes. Our goal was to detect image clusters in the dataset with poor performance and analyze the similarities between them in order to understand the weaknesses of the applied model. AI Models will generalize to what data they are trained on and this approach uncovers weaknesses in the model, as well as in the data it is trained on. A simple example of a weak cluster could be a dataset that does not contain images of pedestrians with both legs occluded by an object. In this case we might find many images with this specific situation with poor inference performance.

Approach

We employed a vector search based approach in which we calculated vector embeddings for each image. A vector embedding is a semantic representation of an image in high dimensional space. We utilize different Computer Vision Models for classification and embeddings to calculate those embeddings. The idea follows the principle that images with similar features will have similar vector representations and thus will cluster together in the vector space.

We can use the cosine distance between two vectors to identify their similarity where a small distance means high similarity. Given an image with poor performance we can then easily search the database for images with small distances in the vector space and identify images that have similar situations depicted.