Why PyTorch is faster than sklearn models?
It is difficult to answer your question without the access to your code. The best way to understand the difference is to profile the code and see where the bottlenecks are for your specific problem.
For this, you can use different profiling modules in python:

cProfile
python line profiler