Is it counter-productive to study while tired? I've noticed I retain a much smaller fraction of the information I consume when I'm tired compared to when I'm well slept. Not only that, but I typically have to revisit the same material more times due to this issue, and at that point the information no longer seems new and exciting.

Outside the context of academia and deadlines, if one is self-studying new material is it more beneficial to wait till they're better slept or should they power through the fatigue?
Yes. When studying sleep is crucial in consolidating memories. Recently a wave of research and opinion has hit around optimizing learning, sleep is always mentioned as key. To prominent authors are Benedict Carey and Barbara Oakley. There work is much easier to process than the below research paper, but we like well researched answers!

So here we go, why studying while fatigued is counter-productive and the underlying mechanism of memory consolidation.


  Whereas older research concentrated on the role of rapid-eye-movement (REM) sleep, recent work has revealed the importance of slow-wave sleep (SWS) for memory consolidation and also enlightened some of the underlying electrophysiological, neurochemical, and genetic mechanisms, as well as developmental aspects in these processes. Specifically, newer findings characterize sleep as a brain state optimizing memory consolidation, in opposition to the waking brain being optimized for encoding of memories


Further


  Consolidation originates from reactivation of recently encoded neuronal memory representations, which occur during SWS and transform respective representations for integration into long-term memory. 


This work is available for free from the NCBI here