1. A computer-implemented method, comprising: obtaining, at a policy management service, an access control policy associated with a user of a computing resource service provider, the access control policy having a corresponding set of effective permissions associated with one or more resources of the computing resource service provider; obtaining usage history information generated as a result of a plurality of previously submitted requests for access to the one or more resources that implicate the access control policy, the usage history information comprising an indication of one or more outcomes of the plurality of previously submitted requests; producing a set of policy modification recommendations based at least in part on a comparison of the usage history information to the access control policy, each policy modification recommendation of the set of policy modification recommendations specifying an alteration to the set of effective permissions; selecting a policy modification recommendation from the set of policy modification recommendations; altering the access control policy in accordance with the selected policy modification recommendation by at least removing an unnecessary permission or adding a missing permission to the access control policy to produce a modified access control policy; presenting the modified access control policy to the policy management service to cause the access control policy to be replaced by the modified access control policy; and controlling access to at least one of the one or more resources using the modified access control policy.