Smart UV Project: the sequence of the faces is wrong and has inconsistent width
I think you can work a bit on this: I tried with your file and got some decent result, imho.

The first thing is: I isolated just the exterior walls.



then I set the front view and unwrapped cylindrical



I got an (extremely high) quite nice square unwrap



but helping with 2d cursor and the pivot choice, it's easy to scale it dow to make it a bit more manageable







then I loaded a brick wall image similar to yours



and aligned and scaled the uv map to the image

what about the face sequence? it seems ok, look them while I select (in top view) all faces...



Hth..