Does an empty entity make any sense in an entity-relationship diagram?
This is fine as long as all you need is an index for the class, though it may be very difficult to troubleshoot exactly what is meant by a particular record in either future develompent or by a user for any reason at all.

If you want to be able to define anything about the class, say a name or other defining attribute, you'd just want to add this to the table as well.

For example:

TABLE SchoolClass (classNumber INTEGER PRIMARY KEY, classDescription TEXT)