Return neighbors index in an 3x3 grid
MATL, 17 16 bytes
9:qWIe1Y6Z+i)BPf

The array is 1-based, that is, contains numbers from 1 to 9.
Try it online! Or verify all test cases.
Explanation
Consider input 2 as an example.
9:q  % Push [0 1 2 ... 8]
     % STACK: [0 1 2 ... 8]
W    % Rise to 2, element-wise
     % STACK: [1 2 4 ... 256]
Ie   % Reshape as 3-row matrix (column-major order)
     % STACK: [1   8  64;
               2  16 128;
               4  32 256]
1Y6  % Push [0 1 0; 1 0 1; 0 1 0]
     % STACK: [1   8  64;
               2  16 128;
               4  32 256],
              [0   1   0;
               1   0   1;
               0   1   0]
Z+   % Convolution, maintaining size
     % STACK: [10  81 136;
               21 170 336;
               34 276 160]
i    % Take input, n
     % STACK: [10  81 136;
               21 170 336;
               34 276 160],
               2
 )   % Get n-th entry (1-based; column-major order)
     % STACK: 21
B    % Convert to binary
     % STACK: [1 0 1 0 1]
P    % Flip
     % STACK: [1 0 1 0 1]
f    % Find: gives indices of nonzeros. Implicitly display
     % STACK: [1 3 5]