Comparing a large/general CNN to a smaller more specialized one? I am still somewhat a novice in the ML world, but I had a strange idea about CNNs and wanted to ask if this would be a valid way to check the robustness of a general CNN that classifies certain images.
Let's say that I make a CNN that takes in many different images of sports players performing a certain action (basketball shot, football kick, freestyle in swimming, flip in gymnastics, etc). Firstly, would it be possible for such a CNN to distinguish between such varied images and classify them accurately? And if so, can it be a good idea to compare this &quot;larger&quot; CNN to multiple &quot;smaller&quot; more specialized ones that take in images from one particular sport?
In other words, I want to know that if I have a &quot;larger&quot; CNN that gives me an output like &quot;football being kicked&quot;, is there a way to then double-check that output with a smaller CNN that only focuses on football moves? In essence, could we create a  system where once you obtain an output from a general CNN, it automatically classifies the same image through a more specialized CNN, and then if the results are of similar accuracy, you know for sure that CNN works?
Kind of like having a smaller CNN as a &quot;ground-truth&quot; for the bigger one? In my head it kind of goes like this:
large_net_output = 'Football kick identified with 95.56% confidence' 

for sport in large_net:
    if sport == 'football':
        access = small_net_for_football
        return small_net_for_football_output

    elif sport == 'swimming':
        access = small_net_for_swimming
        return small_net_for_swimming_output

    elif sport == 'baseball':
        access = small_net_for_baseball
        return small_net_for_baseball_output

# and so on....
&gt;&gt;&gt; small_net_for_football_output = 'Football kick identified with 97.32% confidence'

robustness_check = large_net_output - small_net_for_football_output
print(robustness_check)

&gt;&gt;&gt; 'Your system is accurate within a good range of 1.76%'
     

I hope this makes sense, and that this question does not cause any of your to cringe. Would appreciate any feedback on this!
After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a &quot;Big Task agnostic CNN&quot; to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read Big Self-Supervised Models are Strong Semi-Supervised Learners by Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi and Geoffrey Hinton. The code for the following can be found here. But I feel that training such a model would take up a lot of computational resources.