Something I like to do on Deck for some games is set the resolution to be 1600x1000 rather than 1280x800. Upscalers always looks rough to me at lower resolutions, and FSR quality for 1280x800 is upscaling from a bit lower res than 1600x1000 with FSR Performance.