Draw diagonal positions of me squared
Jelly, 14 bytes
⁼þµḤ+Ṛị“/\X#”Y

A monadic link taking a number and returning a list of characters; or a full program printing the result.
Try it online! or see [1-12] in one go.
How?
⁼þµḤ+Ṛị“/\X#”Y - Link: number, n
 þ             - outer product (implicit range build on left AND right from n) with:
⁼              -   is equal (yields a table of 0s except the main diagonal is 1s)
  µ            - monadic chain separation, call that I
   Ḥ           - double (change all the 1s to 2s)
     Ṛ         - reverse I (a table of 0s with 1s as the anti-diagonal)
    +          - add (vectorises to make a table of zeros with 2s on the diagonal,
               -      1s on the anti-diagonal and, if the meet, a 3 at the centre)
       “/\X#”  - literal list of characters ['/','\','X','#']
      ị        - index into (replaces... 1:'/'; 2:'\'; 3:'X'; and 0:'#')
             Y - join with newline characters
               - as a full program: implicit print