Change quality of built-in recording script
I downloaded a trial version of ScreenFlow, and it looks like there are a few ways to improve the video quality.

YouTube Export

If you're using the YouTube publishing option, be sure to check the Publish in 720p/1080p option:


If you don't have that selected, the video will be downscaled by YouTube to 360p. You may also want to check the Letterbox option to maintain the aspect ratio on YouTube.

Custom Export

You can gain more control over the export options by using the Export option, in the File menu (or ⌘E). There are various presets, I would suggest trying the Web - High option first, and see if that's to your taste. You'll also want to be sure to change the Dimensions settings, which by default are set to scale to 50%.

If you're wanting to put stuff on YouTube, I suggest exporting at your default resolution (i.e. 100% scaling) — YouTube will re-encode it anyway, but uploading the best source material is the best way to maintain quality. If you find the quality of the Web - High isn't sufficient, you can click the Customize… button, and increase the Datarate setting.

Lossless Export w/ 3rd Party Tools

Finally, if you're still not satisfied, you could ScreenFlow's lossless recording, and use an external tool to compress it. To enable lossless recording, open Preferences, click Advanced and change the Screen Recording Compression from Adaptive to Lossless. Then when you export, use the Lossless preset (or Lossless - ProRes, although this format is not supported by as many tools). Then use another tool, such as Handbrake to tweak the settings as you desire.

Hopefully you can find something amongst these options that will get you the output you desire.