What system information is stored in the primary file in SQL Server?
The system objects are the objects which store the schema for all your objects.  The stored procedures, table schemas, etc. are all stored in these system objects.

If the system objects become corrupt you have to restore the primary filegroup.

You can view the data within the system objects by querying the system catalog views such as sys.tables, sys.procedures, etc.


  Do they appear in the 'System' Database?


Not sure what you are talking about, there's no database called "System".  The system databases are master, msdb, model (to some extent) and tempdb.  There is no user database data stored within the system databases.  (Things like logins, jobs, SSIS packages not withstanding.)