Why is accessing paragraphs slow? Is there a better approach?
Well after fiddling around with it some more, I suppose the best answer here is to just cache the variable  $para_ids using \Drupal::cache() and then update it whenever there is a node/entity update or insert. I could look into database optimization to see what is actually happening with the $item->entity reference, and put an index on the joins/where statements but I am not very confident that is the best way.