How do Antivirus for Android detects a Malware application?
Your guess is wrong. Anti-virus applications on Android are completely unrelated. There are two ways that Google finds and removes dodgy apps and malware from Google Play.

Reports by users

Every Play Store page has a "report" button. Google investigates apps that are reported, whether that's for copyright or trademark issues, violating their policy about content (e.g. pornography), for breaking their other policies (e.g. invasive ads, interfering with other apps), or for being malware or scams.

Automatic scans

Google Play now automatically scans apps that are added by developers, hoping to detect dodgy apps like these before they can hurt users or cause problems. For obvious reasons, Google doesn't disclose details of what kind of scans they perform or how the technology works.

If a user has a malware app and it's detected by an anti-virus program, the user might choose to report it to Google Play themselves (if the malware came from there), but that's the only part that anti-virus apps on smartphones play in the process.