title: Permutation-invariant neural network
body: <p>I have a dataset from Multiple Object Tracking trials, where a participant follows 8 points on a display, 4 of which are targets (marked briefly at the beginning of the trial) and 4 are distractors. At the end of the trial, the person marks the 4 targets. My dataset only includes trials where the participant's reply was correct. I have 10 frames per second, each frame includes the positions of the points and the position of the eye gaze, so 18 numbers in total. A trial lasts 8 seconds.
  There are 40 possible trajectories for the points.</p>
  
  <p><a href="https://i.stack.imgur.com/Nteoh.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Nteoh.png" alt="An example of what a frame in a trial looks like, the red points are targets, the blue cross is the gaze."></a></p>
  
  <p>I'm trying to train a neural net to mark the 4 targets solely based on the positions of gaze and the points. The problem is, in the data set, the answer is always the first 4 points in the vector. If I used these outputs for training, the net would just learn to always say [1,1,1,1,0,0,0,0].
  Is there a way I can alter the input or output(or both) — by computing different features for example — so that it doesn't matter to the net in what order it recieved the points? The fact that a point's coordinates are the first(second, third…) in the input vector conveys no meaning in this task.</p>
  
  <p>What I tried so far:</p>
  
  <ul>
  <li>during training, permute each input randomly (and output
  correspodningly) and iterate over all 70 possible permutations of the
  output vector [1,1,1,1,0,0,0,0] so that the permutations are
  represented equally in the training. Didn't work (success rate was
  1/70, which is equal to chance)</li>
  <li>sort the points from left to right (by x-coordinate) - results improved, but the net basically memorized the trajectory and worked equally well even when I removed the eye gaze position. Of course I want the net to reply correctly even for new trajectories that it wasn't trained on</li>
  </ul>
  
  <p>I have an idea for input features, I could partition the display into a discrete mesh and put 1 where there is a point, some other number where the position of the gaze is and 0 elsewhere. I don't know however, what would the output look like, any ideas?</p>
  
  <p>I know I can't find an answer about a whole trial from one frame, so I'm hoping to combine the outputs of the net for all 80 frames of a trial and find the answer from that.</p>
  
  <p>I'm not even sure there is any hope an NN would manage to learn this. Are there any machine-learning models that are permutation-invariant? I have searched for a long time and found nothing.</p>