How can I cap the total space that mysql / mariadb binary logs take up? I recently had my disk space completely filled with mysql binary log files (mysql-bin.000123, etc).

I was able to resolve short term by with PURGE BINARY LOGS TO (via this answer). 

To try and keep it from happening going forward I set lower limits on expire_logs_days and max_binlog_size. Yet, I'm still worried that a huge number of log files could be generated in a short period without a limit on the total space used by logs or the total number of logs allowed.

Is there something I'm missing? Or do I just need to keep setting expire_logs_days lower or turn off binary logging all together if it keeps being a problem?

EDIT:

I understand it's important to diagnose what's causing the logs to fill up quickly and solve that issue. This question is not about how to diagnose what's making them fill quickly, but rather - how do I safeguard against the disk filling to 100% and crashing the server in the case that something is setup wrong or an unexpected circumstance starts to fill these logs.
max_binlog_size has little effect on keeping disk space down; expire_logs_days is the one to worry about.

How fast are the logs filling up?

One way binlogs can fill up extremely fast is when you accidentally have the same server_id on multiple servers in a Replication setup.

Which version?  Which binlog_format?  Lots of multi-row UPDATEs/DELETEs?

More

expire_logs_days is the best available builtin tool.  There may be 3rd party monitoring tools (or you could write one) that watch for disk space.

In any system, you should have some kind of monitoring on disk space -- it is not just the binlog that can go berserk.  The "general log" and the slowlog can, also.  And your application can go berserk.  And so forth.