Deciding what type of model to use for predicting the bottom decile of student grades
My first attempt was a neural network with binary classification,
  forcing the model to predict that the student is either in the bottom
  2%ile, or not. Having read a few posts on FHarrell.com, I was thinking
  that a logistic regression that outputs the probability of being in
  the bottom percentiles is better?


I would recommend you start with logistic regression as a baseline. Then when you have your baseline I would compare the LR-model with a tree-based ensemble since they tend to work really well on tabular data. I would recommend random forest, it is easy to work with and is not that prone to overfitting. Then if you are feeling fancy you could try gradient boosting with XGBoost or LGBM. They tend to give marginally better results, but takes some more effort to work with.


  Or alternatively, should I just do a non-binary linear regression to
  directly predict the students %ile? The reason I thought that this
  wouldn't be as useful for my purposes is because I don't care about
  making predictions between any of the other percentiles. For example,
  I don't care about distinguishing between an A and a B student, I am
  just looking to identify F students. Therefore I would be losing
  accuracy in the bottom 2%ile and gaining it in the other irrelevant
  %iles?


I agree.


  Finally, I think that the neural network makes sense given that there
  is interaction between the variables.


Neural networks are generally not that good on tabular data. Unbalanced, which your have with a ration of 1:50, is especially hard for NNs. Then they tend to classify everything as the majority class.