Override formatting for paragraph edit view
Most edit forms utilize your current admin theme and form render arrays with according widget subforms.

So - in theory - you have several options:

(1) Provide your own admin theme with templates for your single forms. Using a contrib admin theme as base theme would make up for an easy start.

(2) Create a custom module and implement one of the many hooks that allow to alter your forms (e.g., hook_form_FORM_ID_alter, hook_field_widget_WIDGET_TYPE_form_alter, ...).

(3) If suitable, choose your front-end theme for editing your content. Then you can implement your templates there. For this, you'd want to implement a custom theme negotiator in a custom module.

In most cases however, the above methods are overly time-consuming.

To improve usability of back-end forms for content authors, have a look into the field_group module. It allows you to structure your back-end editable forms and displays. You can add groups like detail wrappers, horizontal/vertical tabs, and/or containers with custom HTML-elements and classes.