Grant reference on type
According to BOL, the REFERENCES permission allows the following:


  The REFERENCES permission on a table is needed to create a FOREIGN KEY
  constraint that references that table.
  
  The REFERENCES permission is needed on an object to create a FUNCTION
  or VIEW with the WITH SCHEMABINDING clause that references that
  object.


There is no reason for a public user to have this permission when they are unable to CREATE or ALTER any other objects.