How to replicate the "Show empty cells as" functionality of Excel graphs
Asked the dev about it - 

There is a dispBlanksAs property of the ChartContainer but this currently isn't accessible to client code.

I looked through the source some more using that answer to guide me. The option is definitely in there, but you'd have to modify source and build locally to get at it.
So no, it's not accessible at this time.
When selecting a data source for a graph in Excel, you can specify how the graph should treat empty cells in your data set (treat as zero, connect with next data point, leave gap). 
The option to set this behavior is available in xlsxwriter with chart.show_blanks_as(), but I can't find it in openpyxl. If anyone knows where to find it or can confirm that it's not present, I'd appreciate it.