It stands in the site of an old Kan ` eiji , a temple very related with the Tokugawa shoguns , who had built the temple to guard Edo Castle against the Northeast . It occupies the site of the former Kan ` ei-ji , a temple closely associated with the Tokugawa shoguns , who had built the temple to guard Edo Castle against the north-east , then considered an unlucky direction .