Allow module to overwrite multiple templates from module
I was able to figure it out. I only used hook_theme_registry_alter().  You have the copy $theme_registry['paragraphs'] to all items you want to use paragraphs.

/**
 * Implements hook_theme_registry_alter().
 */
function my_module_theme_registry_alter(&amp;$theme_registry) {
  $module_path = drupal_get_path('module', 'my_module');

  // Use the templates in my module's template folder.
  $theme_registry['paragraph__accordion'] = $theme_registry['paragraph'];
  $theme_registry['paragraph__accordion']['path'] = $module_path . '/templates/paragraph';
  $theme_registry['paragraph__accordion']['template'] = 'paragraph--accordion';
  $theme_registry['paragraph__carousel'] = $theme_registry['paragraph__accordion'];
  $theme_registry['paragraph__carousel']['template'] = 'paragraph--carousel';
}