Output Paragraph values instead of the Paragraph IDs in a View REST export?
I thought about using a custom field formatter, but that seems like a
  lot of work just to output JSON data


No Code Solution:

Install Views Field View

Create a new View, "User view" in this case (does not to be a REST view).


Add user fields you want to show.
Add Contextual filter of ID


When adding, scroll down to the "More" section. Checkmark allow multiple values.



Go to your Rest View


Add field of "View", select the view you added above. 


For contextual filter, use the replacement pattern for your author's field.