Commerce Product Display or just Views Display of line items? I will be presenting the products in one view and don't need the display nodes as they wont be seen.

Do I have to create display nodes for each product or can I just use views to display the line items in a table for example?

Is there any catch with doing this?
You can't see products because the permission to see products is disabled by default.
Just enable it at /admin/people/permissions. E.g. enable View any product of any type if you want to permit users to see directly entities.

But also have a look at the module Inline entity form. you can edit both node display and attached product entities in one form.