How to find a symmetry group of a system if all the symmetry transformations do not obey closure and don't form a group?
Hmm, so I'm not entirely sure what notation you're using ($D_4$ sounded like Schönflies to me), but I'm sure you can figure out the equivalent.
In general I agree with Andrew's point that molecular symmetry is determined by atomic positions and not orbitals. But at least to me as of now, this seems to be a thought exercise more than anything (after all, no real atom has only two p-orbitals), so I'll play along. In Schönflies notation your system would be labelled $D_\mathrm{2h}$. The three $C_2$ axes and three mirror planes are illustrated below. I left out the identity operation $E$ and the centre of inversion $i$.