How to speed up phone without Factory Reset?
Well it's possible only if you are deleting unnecessary apps and free us the storage. Another thing that you can do is root your phone and delete even those system apps which you don't need. 

But I prefer to factory reset the phone to speed it up.