Join duplicated side back on to side duplicated from
Thanks Mr Zak - it worked. 

In my particular version of Blender (2.77) I first had to select the side - select the Magnet symbol, select the Cube symbol right beside it to select the "Snap Element" (Volume, Face, Edge, Vertex or Increment) and then position it they way it needed it.