What is the easiest way to remove this stump (under wall)?
Easiest? Saltpeter. If you drill holes with a spade bit, fill them with a liquid stump remover, then wait for a week or so it should convert the wood to a punky black consistency that's much easier to dig out. If the chemical you used is saltpeter you will have converted the wood to an oxidizer. Should the stump somehow become saturated with kerosene that was left in the holes for a day or two and set alight it would burn even underground, destroying all of the roots. 

Your best bet is probably a reciprocating saw. It won't take a week and a half or attract the fire marshal. It's also safer, in my opinion, than using a chainsaw in very tight quarters.

There are wheels that would allow you to grind the rest with an angle grinder. I'm not too sold on them from a safety perspective, so it'd be a last resort for me personally. 

On the professional side of the house, makers of large oscillating saws for masons offer general purpose blades that cut wood. (ArborTech's AS170, for example) Unfortunately it's probably not a common enough tool to be available at most rental places. I'd call and ask, but not get my hopes up.