Fake security message playing continuously on Galaxy 7
Find the app in Settings -> Applications, and force stop it. That should stop the app from doing anything. Restarting the app should hopefully allow you to continue using it without issue.

Sometimes apps will continue playing audio in the background, and unfortunately this applies to ads as well.