Machine Learning library in Python, list or numpy or pandas Background: We are trying to build a customized ML library in Python 3 to tackle analysis we often repeat, in a general fashion. But it would not be nearly as general as sklearn. In fact, we are prepared to break some interfaces if that give us enough performance boost in return. 

The basic starting point would be constructing a Learner by feeding it X and y, and predict on new input X0

learner = Learner(X, y)
y0 = learner.predict(X0)


One of the design decision is what data type to use for X and y, here are 3 choices, along with some rudimentary brainstorm advantages for each respectively:


native Python list: X being a list of lists, y being a list. Would this have better performance for being 'closer to metal'?
numpy: X being an ndarray of (n, p), y being an ndarray of (n, ). This would benefit from the richer functionalities in numpy/scipy. This is also the data type choice of sklearn.
pandas: X being a DataFrame, y being a Series. This can utilize more Data Analysis (read: dirty work) functionalities from pandas. This way we can also refer to variables with their names instead of just integer indices. But performance would be the worst? 


Please share your thinking of pros and cons for each choice from both tech and math perspectives. Thanks in advance!

PS: I thought about whether this should be a StackOverflow question, but still feel this is more Data Science.
Pandas does normally a decent job allowing dataframes to behave as numpy arrays. 

My recommendation is to use numpy types, the reason is that, for consistency with pretty much what the industry is doing, you are much safer with numpy.

I love pandas, and I love the dataframes, but they provide extra functionality that the model does NOT need, the same way in general programming you will not use a String to represent a boolean (even though tou could do it with a String), simply because you should use whatever data types provides you the functionality you need... and nothing else.

So, numpy is the way to go. As for python lists, you do not get the mathematical operations that you get with numpy, so do not consider them.