Remove support under interior stairs
The post that I point out looks like it is notched around the angled stair carriage, an important detail. The wall sounds like you want to keep in place anyway....

Other parts could be removed but the stair would be bouncy, and would need reinforcement to make it act right.