Using MySQL 8.0 "Upgrade Checker" on RDS
You can run util.check_for_server_upgrade() with a Mysql user who has the ALL PRIVILEGES. As you are using AWS RDS your master MySQL user don't have SUPER user permission, that's why your are getting this error, you can check your user permission with SHOW GRANTS FOR 'user'@'host'. 
Only rdsadmin user have the ALL PRIVILEGES in AWS RDS MySQL instance. rdsadmin is amazon's own user that is used by Amazon to perform release or version update, taking backups and other maintenance tasks. Amazon's support should be able to advise you better in this topics.