Showing a message while downloading a file from hook_form_submit
Your workflow lacks one step. What you want it to be, is:


Send user to "Your download should start shortly" page. On that page, place a script that will execute file download in new window after some arguably small timeout, and a target="_blank" link to the file, for cases when auto-downloading is disabled client side for security reasons.
When the script executes successfully and starts to download file for user, or when user clicks the link to direct download, use JS to send him to "thank you" page. Of course if JS fails, it might not get executed, so be sure all courtesy you really need is also displayed on the page mentioned in point one.