What are the advantages of forgetting?
Short answer
It has been shown that loss of long-term memories may enhance the retrieval of others. Short-term working memory is explicitly designed to be volatile and non-lasting. However, there are many other types of memories where memory loss may not be explicitly beneficial, or even outright debilitating such as in the case of Alzheimer's or stroke.

Background
First off all, there are many types of memories, including sensory memory, motor memory, short-term (working) memory, long-term memory, explicit &amp; implicit memory, declarative &amp; procedural memory and so on. Hence, because the question is quite broad, I will focus on long-term memory, short term-memory and sensory memory to discuss that memory loss can be beneficial, neutral, or detrimental.  


Beneficial effects of loosing memories
Long-term memory is probably what you are after and there are studies in that field that have linked the loss of memories to enhanced processing of other memories. More specifically, there are adaptive benefits of forgetting, namely a reduced demand on cognitive controls during future acts of remembering of other stored information. Even more specific: retrieval of memories after forgetting others are thought to reduce the necessary engagement of functionally coupled cognitive control mechanisms that detect (anterior cingulate cortex) and resolve (dorsolateral and ventrolateral prefrontal cortex) mnemonic competition (Kuhl et al., 2007). The improvement of particular memory processes by forgetting others may be linked to them being closely related. Indeed, motor tasks more remote do not benefit much from forgetting unrelated ones (Shea &amp; Right, 1991).   
Inherently volatile memory
Short-term working memory is explicitly designed to aid in on-demand task performance. Short term memory is for example used to remember a set of components (e.g., colors) and use that information to deal with a certain task at hand (which objects depicted here match the colors you just saw?). If all these memories would be retained, tasks dependent on working memory would not be possible. Sensory memory an ultra-short term memory that is kept only for very short amounts of time, allowing people to, e.g., track a light and make a symbol or letter out of it before the information is funneled to the short term-memory. 
Neutral effects of loss of neural function: a side track off memory lane
However, forgetting may be simply another example of the use it or lose it principle that applies to pretty much everything in the human body; when you don't walk, the bones in the legs will weaken along with the musculature used for locomotion. Similarly, when the inner ear or the retina becomes dysfunctional and degenerate, the deafferented auditory nerve and optic nerve start to degenerate, respectively. The associated deafferented sensory cortices will slowly be taken over by other adjacent cortical areas due to the plasticity of the cortex. In blind folks, for example, the tactile and auditory cortices have been shown to take over the primary visual cortex. Given that the visual cortex is huge compared to the tactile and auditory cortex, one would expect substantial increase in performance on tactile and acoustic tasks in blind folks. Yet, this is debated (Stronks et al, 2015). In fact, normally sighted folks can learn braille as well as their blind peers, suffice they get an equal amount of practice. In other words, practice is the key, not enhanced areas of cortex being available per se. Hence, 'forgetting' to see or 'forgetting' to hear is, as far as my knowledge goes, not associated with any benefits whatsoever, barred a minority of studies that showed a slight benefit of being blind in auditory tasks (Stronks et al., 2015).
Pathological memory loss - not so good:
However, forgetting of memories may also be pathological; think of the impaired short-term memory of Alzheimer's patients, or amnesia due to stroke. Forgetting is not always beneficial.


References
- Kuhl et al., Nature Neurosci (2007); 10: 908-14
- Shea &amp; Right, Res Quarterly Exercise Sport (1991); 62(3)
- Stronks et al., Brain Res (2015); 1624: 140–52