Error 42P01: relation does not exist
What you had originally was a correct syntax - for tables, not for schemas.  As you did not have a table (dubbed 'relation' in the error message), it threw the not-found error.

I see you've already noticed this - I believe there is no better way of learning than to fix our own mistakes ;)

But there is something more.  What you are doing above is too much on one hand, and not enough on the other.

Running the script, you


create a schema
create a role
grant SELECT on all tables in the schema created in (1.) to this new role_
and, finally, grant all privileges (CREATE and USAGE) on the new schema to the new role


The problem lies within point (3.)  You granted privileges on tables in replays - but there are no tables in there!  There might be some in the future, but at this point the schema is completely empty.  This way, the GRANT in (3.) does nothing - this way you are doing too much.

But what about the future tables?

There is a command for covering them: ALTER DEFAULT PRIVILEGES.  It applies not only to tables, but: 


  Currently [as of 9.4], only the privileges for tables (including views and foreign tables), sequences, functions, and types (including domains) can be altered.


There is one important limitation, too:


  You can change default privileges only for objects that will be created by yourself or by roles that you are a member of.


This means that a table created by alice, who is neither you nor a role than you are a member of (can be checked, for example, by using \du in psql), will not take the prescribed access rights.  The optional FOR ROLE clause is used for specifying the 'table creator' role you are a member of.  In many cases, this implies it is a good idea to create all database objects using the same role - like mydatabase_owner.  

A small example to show this at work:

CREATE ROLE test_owner; -- cannot log in
CREATE SCHEMA replays AUTHORIZATION test_owner;
GRANT ALL ON SCHEMA replays TO test_owner;

SET ROLE TO test_owner; -- here we change the context, 
                        -- so that the next statement is issued as the owner role

ALTER DEFAULT PRIVILEGES IN SCHEMA replays GRANT SELECT ON TABLES TO alice;

CREATE TABLE replays.replayer (r_id serial PRIMARY KEY);

RESET ROLE; -- changing the context back to the original role

CREATE TABLE replays.replay_event (re_id serial PRIMARY KEY);

-- and now compare the two

\dp replays.replayer
                                   Access privileges
 Schema  │   Name   │ Type  │       Access privileges       │ Column access privileges 
─────────┼──────────┼───────┼───────────────────────────────┼──────────────────────────
 replays │ replayer │ table │ alice=r/test_owner           ↵│ 
         │          │       │ test_owner=arwdDxt/test_owner │ 

\dp replays.replay_event
                               Access privileges
 Schema  │     Name     │ Type  │ Access privileges │ Column access privileges 
─────────┼──────────────┼───────┼───────────────────┼──────────────────────────
 replays │ replay_event │ table │                   │ 


As you can see, alice has no explicit rights on the latter table.  (In this case, she can still SELECT from the table, being a member of the public pseudorole, but I didn't want to clutter the example by revoking the rights from public.)