I need help understanding Postgres's archive cleanup functionality
You're correct. If you want PITR capability, you need to keep all archives between when the base backup was taken and when you want to be able to recover to.

For this reason it's often a good idea to do a fresh base backup (say) weekly, and rotate them out, so you keep a base backup plus a week of archives.

PgBarman should be able to help you with this.

Archive cleanup would not be desirable if you want PITR.