Tool for analyzing a Python matrix and generating a report on the contents (column types, NaN counts, means, etc.) I'm looking for a tool/library that will take a numpy or pandas matrix and generate a list of statistics for the matrix and columns. Specifically, for each column, I'd want info like the following:


Assumed data type (numerical vs string)


Assuming it's determined to be a numerical column:


Mean and Std Deviation
Max and min values
Number of NaN's
% of NaN's


Assuming it's determined to be a string column:


Number of distinct string values


So the tool takes the matrix and outputs some kind of report along these lines. The goal is to do some sanity checking on my input data set as well as checking data transformations along the way. For example, if I'm doing some kind of transformation and values in column A go from 1% NaN to 60% NaN, then I did something really bad.

Does such a tool exist?
You could use pandas.describe.

import pandas as pd
import numpy as np
df  = pd.DataFrame({'first': [1,2,3,4,np.NaN], 'second': [100,200,np.NaN, np.NaN,4]})
df.describe(include='all')


Regarding the percentage of NAs, you could use just a small formula for that:

count_nan = len(df) - df.count()
count_nan / df.shape[1]