SP2-0584: EXIT variable was non-numeric error
It should be a simple boolean.
This should work and be more efficient.
The SQL%ROWCOUNT attribute will be populated with the number of rows deleted. If you didn't delete any then just exit.

FOR i IN 1..100
LOOP
  DELETE
  FROM mytable
  WHERE date &lt;= (SYSDATE - 30)
  AND ROWNUM &lt; 5000;

  EXIT WHEN sql%rowcount = 0;

  COMMIT;

END LOOP;


If you aren't familiar with the implicit cursor attributes take a look at this in the Oracle docs - https://docs.oracle.com/cd/B28359_01/appdev.111/b28370/sql_cursor.htm

EDIT: Just noticed the title of the post. The exit that the message is referring to is the exit from SQL*Plus and as such I'm not sure you are running this the right way. Please post all the code and tell us how this is being run.