What tools should I deploy to support AWS for a Managed Services company?
Installing a box is probably not the best solution, mostly because there's no real need for it.

You should ask for an access to AWS console in customer account with proper rights fitting your responsibilities and maybe setup AWS config, Cloudwatch for the monitoring and alerting part, Cloudtrail and maybe Amazon Inspector for the audit part.

To ensure machines and AWS services themselves are properly configured you may rely on AWS Config/Amazon Inspector or tools like inspec (and here you may or may not need a bastion host on your customer VPC to check other machines, this mostly depends on the agreement you have with customer about direct links/VPN)