Setting up a simple "permissions architecture" in PostgreSQL
To answer your direct question "if I revoke the CREATE privilege from the PUBLIC role, how do I see that ? "...

Use the following SQL (from this stackoverflow question) to get database-specific privileges for PUBLIC role:

   SELECT nspname,
       coalesce(nullif(role.name,''), 'PUBLIC') AS name,
       substring(
          CASE WHEN position('U' in split_part(split_part((','||array_to_string(nspacl,',')), ','||role.name||'=',2 ) ,'/',1)) &gt; 0 THEN ', USAGE' ELSE '' END 
          || CASE WHEN position('C' in split_part(split_part((','||array_to_string(nspacl,',')), ','||role.name||'=',2 ) ,'/',1)) &gt; 0 THEN ', CREATE' ELSE '' END 
       , 3,10000) AS privileges
FROM pg_namespace pn, (SELECT pg_roles.rolname AS name
   FROM pg_roles UNION ALL SELECT '' AS name) AS role
 WHERE (','||array_to_string(nspacl,',')) LIKE '%,'||role.name||'=%'
 AND nspowner &gt; 1;


Result:

      nspname       |   name   |  privileges   
--------------------+----------+---------------
 pg_catalog         | postgres | USAGE, CREATE
 pg_catalog         | PUBLIC   | USAGE
 information_schema | postgres | USAGE, CREATE
 information_schema | PUBLIC   | USAGE
 test               | postgres | USAGE, CREATE
 public             | postgres | USAGE, CREATE
 **public             | PUBLIC   | USAGE, CREATE**


Now, run the following:

REVOKE CREATE ON SCHEMA public FROM PUBLIC;


Re-run the first SQL, and we get the following result-set:

   nspname       |   name   |  privileges   
--------------------+----------+---------------
 pg_catalog         | postgres | USAGE, CREATE
 pg_catalog         | PUBLIC   | USAGE
 information_schema | postgres | USAGE, CREATE
 information_schema | PUBLIC   | USAGE
 test               | postgres | USAGE, CREATE
 public             | postgres | USAGE, CREATE
 **public             | PUBLIC   | USAGE**


Compare the last lines from the 2 result-sets. You will see the missing CREATE privilege. 

Don't forget to add the following back:

GRANT  CREATE ON SCHEMA public TO  PUBLIC;


To answer your other direct question : "Does that mean that the public role has all privileges granted (by default) ? " 

'PUBLIC' is not an explicit role. This is what tripped me when I started Postgres.

Run the following command to confirm this - PUBLIC is absent from this result set:

select * from pg_roles;


Erwin Brandstetter has given a great explanation of PUBLIC role and another even more detailed explanation of PUBLIC role here . 

My own notes - just paraphrasing the Postgres documentation on GRANT...

Any particular role will have the sum of following privileges:


privileges granted directly to it + 
privileges granted to any role it is presently a member of +
privileges granted to PUBLIC.


This official Postgres post does a great job of explaining how to setup rights:
Managing Rights in Postgres