What Macs do when verifying applications
Apple provides digital 'certificates' to registered developers, who pay $99 per year to register. Any applications signed with those certificates will be 'verified'. 

It's nothing to do with the App Store as such. Registered developers can make certified apps available on their own websites.

Apple can remove the certificate from developers who are found to be shipping malware or being naughty in some other way.

When you first open an application, MacOS checks that the certificate is valid. If the app was made by a developer who is not registered with Apple, then you will get the message that the app is unknown to Apple, and therefore potentially risky. Of course, lots of very good software is written by developers who don't register with Apple.