Are there any image classification algorithms which are not neural networks?
Part of the problem with answering this question is there are actually two questions.  The first:

Are there any image classification algorithms which are not neural networks?

Yes, lots. But now the actually question:

Is there any paper which tries an approach which does not use neural networks in LSVRC?

In your question, you rule out methods such as kNN and SVM because of speed. Bag of Words is one method used to solve this problem.  MATLAB has a good demonstration (http://www.mathworks.com/help/vision/examples/image-category-classification-using-bag-of-features.html).  But BoW incorporates k-means clustering, so that may not fit your needs.

There are some other interesting image classification methods such as texture analysis. TA is being researched as a way to classify malevolence of disease in medical images (such as tumors).  Here is a commonly referenced paper:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2701316/

Here's an overview of image classification:
http://www.tandfonline.com/doi/full/10.1080/01431160600746456