Add a custom line item type with custom fields when my module is active
As stated by @pcambra, the hook to define custom line items is hook_commerce_line_item_type_info.

The base key is used to build the name of some callback functions. If not specified, it defaults to the line item type name.

One of the possible callbacks is "configuration", and it is invoked when the line item is first detected by Commerce. In this callback is where you should create the fields/instances related to your line item type. Commerce already does it in commerce_product_reference module. See commerce_product_reference_commerce_line_item_type_info and commerce_product_line_item_configuration for reference.

Also, you can consult this blog post for some tricks to ease the creation of fields/instances.