How do I search for the exact string match using CiviCRM API
The Contact API is non-standard in APIv3; a major goal of APIv4 is to remove these quirks.  The behavior you're seeing on Display Name is non-standard.  It also seems that a lot of the other operators (e.g. "LIKE") don't work on Display Name.

If you can, you can search on First Name and/or Last Name, neither of which display the problem described here.