shadows on wrong side of normals
You are right about your normal map. Somehow all the pixels that are white in the brick diffuse texture appear to be Transparent in your normal map.



I have no idea what was going on with the end part of your wall that wasn't showing shadow in your original image but after remodelling it from a basic cube it seemed to be working as expected.