Magical Modulo Squares
Jelly, 13 10 bytes

-3 thanks to Nick Kennedy

Feels like the repeated code should be is golf-able, but I have did not managed it...

*€Ṗ%µQƑƇḢị


Try it online! (footer pretty-formats as a grid)

How?

*€Ṗ%µQƑƇḢị - Link: integer, p
 €         - for each n in [1..p]
*          -   exponentiate with:
  Ṗ        -     pop = [1..p-1]
           - ...i.e [[1^1,1^2,...,1^(p-1)],[2^1,2^2,...,2^(p-1)],...,[....,p^(p-1)]]
   %       - modulo p
    µ      - start a new monadic chain (call that list of lists X)
       Ƈ   - keep those which:
      Ƒ    -   are invariant under:
     Q     -     de-duplicate
        Ḣ  - head
         ị - index into the list of lists X