If operating an ultralight with private pilot certificate, which rules apply?
If the aircraft is not registered, it can only be operated under part 103.
FAR 103.1 states that an ultralight may not &quot;have any U.S. or foreign airworthiness certificate&quot;.  So a registered aircraft could not be operated under Part 103.
Your question mentioned transponders.  Bear in mind that FAR 91.1(e) exempts ultralight vehicles from all of Part 91.  Therefore, if operating under Part 103, there is no particular airspace that would become available if a transponder were added to the ultralight, that would not be available otherwise, unless a particular ATC facility chose to grant the &quot;prior authorization&quot; required for Part 103 operations in certain controlled airspace, specifically on that basis.  This seems unlikely.  In fact, it is probably safe to assume that the FAA would attempt to prohibit a transponder from being added to an aircraft operating under Part 103, although it's not clear what regulation this would be based on.
Related:
In the United States, are ultralights legally considered aircraft?
Would FAR 91.227, or any other regulation, forbid use of ADS-B-out in an unregistered ultralight?