Closet door minimums
The Code only stipulates the size of one door in a residence. It requires one egress door that...

has a 32” clear opening with the door in the 90 degree position (which requires a 36” wide door)
is located at a exterior entrance into the residence opening into a public way or open side yard that opens to a public way
does not open into a garage or carport
is a minimum of 78” high measured from top of threshold to bottom of stop
is side-hinged (no sliding doors allowed)

See 2017 ICC R311.1 and R311.2.
The only other requirement is for sleeping room egress windows, but that’s not your question.