Select rows with the highest value for one ID, after grouping by two IDs
Simply adding DISTINCT ON (PersonID) will do the job in PostgreSQL:
SELECT DISTINCT ON (PersonID)
       AgentID, PersonID, Count(*) AS BustCount
FROM   InvolvedIn i
WHERE  AgentID IS NOT NULL 
AND    isCulprit
GROUP  BY AgentID, PersonID
HAVING COUNT(*) &gt; 1
ORDER  BY PersonID, BustCount DESC, AgentID;

I added AgentID as (optional) third ORDER BY expression and tiebreaker to get a deterministic result in case of multiple agents with the same amount of busts. Adapt to your requirements.
Detailed explanation:

Select first row in each GROUP BY group?

We can do all of it in a single SELECT without subquery, because DISTINCT ON is applied after aggregation. About the sequence of events in an SELECT query:

Best way to get result count before LIMIT was applied

Oh, and WHERE isCulprit = TRUE is just a more noisy way of saying WHERE isCulprit.