People detection methods
If you don't need it to work in real time you should not worry about your CPU that much. 

There are a few models for face detection using Res Net 10, with portability to OpenCV, those might be enough if you the people you are trying to count are facing forward.

Else, you can use a Res Net 10, it runs up to 100 FPS on a Intel i5 7200u, which is not that big deal.

Update

Oh, this Face Detection library has optimization form ARM processors  using Tengine which may help your case. About face/person I've seen a detection algorithm using OpenCV that detects human heads in any position, including the back of the head. If you want to do person count I think that should suffice. I will try to find it and post here.