Introduction
In our pursuit of understanding image classification models, our goal was to determine the tipping point at which a classification model begins to struggle when faced with increasingly distorted images. We embarked on this journey by exploring two different approaches to image classification and observing how they react as images become progressively more challenging to classify due to noise or other disturbances.
Through our exploration, we sought to uncover insights into the behavior of classification models under adverse conditions. By pinpointing the moment at which a model’s accuracy starts to decline, we aimed to gain a deeper understanding of the factors influencing its performance. This understanding could provide valuable insights for improving the robustness and reliability of image classification models in real-world scenarios, where images may often be corrupted or distorted.
Our journey aimed to shed light on the nuanced interplay between image quality and classification accuracy, ultimately contributing to advancements in the development of more resilient and effective image classification systems.
We started by diving into Teachable Machine and ResNet50, two popular methods for image classification. We wanted to see how they would react when faced with images that were becoming harder and harder to classify.
Approach 1: Teachable Machine
Teachable Machine’s image classification model utilizes a CNN architecture to effectively learn and recognize patterns within images, enabling it to classify them accurately based on the features it has learned during training.
Teachable Machine offers an intuitive platform for training image classification models, making it accessible to users with varying levels of expertise. In our endeavor, we meticulously curated a dataset by recording 500 unique image samples of both bananas and apples. Here are a sample of recorded input data.
Apple Group
Armed with this dataset, we embarked on a binary classification task, aiming to distinguish between these two fruits. The model showcased commendable performance in Classifying bananas across various levels of image corruption, encompassing scenarios like blurring, alterations mimicking apple markings, and changes in color and background. Here are some test data samples.
Sample Test Data
This model performed very well for these test samples, as the disturbances are not very high, even though it gave 100% accuracy in classifying bananas for the above samples.
Results of Sample Test Data
However, as our exploration progressed, we noticed a decline in accuracy when the model encountered unconventional inputs, such as images depicting partially eaten bananas. This decline hinted at the model’s limitations in generalizing beyond the confines of its training data.
Starting from 100%, it declined to 53% and then to 29% and then to 0%.
Unconventional Sample Data
Analyzing Teachable Machine’s Behavior
we discerned several contributing factors to this decline in accuracy. Primarily, the lack of diversity in the model’s training data may have hindered its ability to adequately represent the myriad of potential inputs encountered in real-world scenarios. Consequently, the model struggled to generalize to novel or unconventional images, leading to inaccuracies in classification. Additionally, Teachable Machine’s feature extraction capabilities may not have been robust enough to handle variations in image characteristics that deviated significantly from the norm, further exacerbating its performance limitations.
Approach 2: Resnet-50 from Hugging Face
ResNet-50, a deep convolutional neural network that has been pre-trained on the extensive ImageNet dataset, stands out for its exceptional performance in handling image classification tasks.
For our investigation, we utilized an online Hugging face Microsoft/resnet-50 Inference API subjected to a series of progressively corrupted banana images. These corruptions included various levels of blurring and the overlaying of apple markings at different opacities.
Overlaying Apple image on Banana
we subjected the classification model to a unique challenge involving overlaying apple images onto banana images at varying opacities (25%, 40%, 50%, 60%, 75%). The results were intriguing: for the 25% opacity, ResNet-50 confidently classified the image as a banana. However, as the opacity of the overlaid apple increased, ResNet-50’s classification shifted towards identifying the image predominantly as an apple, with probabilities reaching as high as 90% for 50%, 60%, and 75% opacities. Notably, at 75% opacity, an unexpected classification of 10% as pomegranate emerged, adding a surprising twist to the analysis.
In addition to the overlaying experiment, we also explored the impact of blurring on ResNet50’s classification accuracy. For images subjected to 50% blur, ResNet50 displayed uncertainty, assigning only a small probability (0.228) to the banana class, with minor probabilities distributed across various other classes. However, the most astonishing result came with 100% blur, where ResNet50 failed to classify the image as a banana at all. Instead, it produced perplexing classifications such as match sticks, sunscreen, and stethoscope, highlighting the model’s inability to recognize the blurred banana image.
Analyzing ResNet-50 model’s Behavior
These findings illuminate the intricacies of ResNet-50’s behavior when confronted with challenging image distortions. While the model demonstrates impressive resilience to certain types of corruption, such as overlaying images, its performance may falter in scenarios involving extreme blurring, underscoring the importance of understanding the limitations of image classification models in real-world applications.
Summary
In our exploration of image classification using Teachable Machine and ResNet-50, we uncovered fascinating insights into the behavior of these models when faced with progressively challenging scenarios. Teachable Machine, with its intuitive platform, showed commendable performance in classifying bananas across various levels of image corruption. However, it struggled with unconventional inputs, indicating limitations in its ability to generalize beyond the scope of its training data. Conversely, ResNet-50, a deep convolutional neural network pre-trained on the ImageNet dataset, demonstrated remarkable resilience to image corruption. It accurately classified images even when overlaid with apple markings at varying opacities, showcasing its robust feature extraction capabilities. However, ResNet-50’s performance faltered in extreme blurring scenarios, highlighting the importance of understanding model limitations. Overall, our exploration sheds light on the complexities of image classification and underscores the need for further research to develop more robust and reliable models for real-world applications.
Author Of article : Nikhil Reddy Read full article
