Can I (/does it make sense to) create a pandas dataframe to hold custom class instances?
A DataFrame's columns (Series) can have any of the NumPy dtypes, including object, which can hold arbitrary Python objects.
Doing so gives up most of the speed and space benefits of using NumPy/Pandas in the first place. And also the type checking—if you accidentally insert an object that isn't an instance of a match subclass, it will just work. And many convenience features. 
But you do still get some convenience features, and sometimes that's a more than good enough reason to use Pandas.
If the performance isn't acceptable, though, you will have to rethink things. For example, if you flatten the object out into a set of attributes (maybe some of them NaN or N/A for some of the subclasses) that you can store as a row, especially if some of those attributes are things like floats or ints that you want to do a lot of computation on, you'll get a lot more out of Pandas—but at the cost of losing the OO benefits of your classes, of course.
Occasionally, it's worth building a hybrid to get the best of both worlds: a DataFrame whose rows hold the storage for your match objects, but then also a match class hierarchy that holds an index or even a single-row DataFrame (a slice from the main one) and provides an OO view onto the same information. But more often, it isn't worth the work to do this, as either almost all of your code ends up being Pandas or almost all of it ends up being OO.
One last possibility, if there are huge numbers of these things, would be an ORM that uses a relational database, or an old-school hierarchical database,1 for storage and indexing.

1. In fact, building a hierarchical database out of a bunch of DataFrames and then wrapping it in an object model is basically the same thing as the previous paragraph, but the idea here is that you use stuff that's already built to do all the hard and/or tedious stuff, not that you build a database just to hide it from yourself.
I am working with tracking data that tracks tennis players in a local match. The data is provided to me in a json file and every 100 milliseconds (so 10 times a second), it knows where the players on both sides of the court are and where the ball is and provides other match data as well. Using this tracking data, I created a bunch of custom python classes and subclasses to create a "match" object and I am loading all of the tracking data into this "match object." Can I create a pandas df holding instances of custom classes/does this make sense? 

Backstory/thought process (in case my question doesn't make sense which is highly possible...)
The first time I implemented this I basically created a "match" object where I used python OOP and had the match broken into games, sets, points, players, etc. The players part is a little confusing because due to calculations on my end we created a player object and a new instance of a player every 100 milliseconds (it's hard to wrap your head around because one player is the same throughout the game, but think of it as that player at that exact moment in time). I'm not sure if it makes more sense to change these "player" objects instead into rows in a pandas dataframe (they're are a ton of them, think about a 3 hour match) or instead if I can just create a pandas df and have a player be a column. Players make up points and then points make up frames, so if I did change the player objects to a pandas df it would be hard because than I would have a bunch of rows in the dataframe making up a point and then a bunch of points making up a game.. and whatnot
Because there is so much tracking data, efficiency considerations are important to me (although I prefer to do things something somewhat slower but not drastically but it helps me ensure/check all data)