Minimum setback for retaining wall
Are you talking about a retaining wall above the house's ground level, or below?

If above, you will need to provide enough clearance for any emergency exits but other than that I can't think of any limit. As for being built into the house, you could think of a walk-out basement as having a retaining wall on the uphill (non-walkout) side.

If you are talking about a wall below the house, you need to make sure you don't undermine the house's foundation. The distance you need to be away depends partly on the soil so if you are wanting to get close to the house (say, closer than 2 or 3 times the wall height) you would definitely want to talk to an engineer.