Frequency dependency vs. memory effects
First of all you should separate concerns: physical reality is one thing, mathematical models we use to describe that reality are a different thing, albeit related. 

As you point out (admittedly with clumsy terminology) reactive circuit elements "have memory" because they store energy. What does it mean? It means that their current state (and hence behavior) depends on the past evolution of the relevant electrical quantities (currents, voltages, charges, etc.) and not only on the values those quantities have in a single instant of time (the current time, i.e. "now"). 


Note (Just to get the terminology straight): it is common to say that a system has memory, but that is not called memory effect. If you say memory effect you could confuse people about what you mean.


Why this happens? Because energy obeys one of the most basic principle of nature, i.e. the principle of conservation of energy.
Therefore the energy a component has adsorbed from the outside cannot disappear, but must be either be stored inside the component or converted into another type of energy.
Storing energy alters the structure or the physical state of the object, hence it "leaves some traces of the past history", i.e. "memory".

That's for the physical part. Now let's tackle the math!

If we want to describe quantitatively a physical object, a component, we must develop suitable mathematical models that allow us to analyze the component in the contest of a circuit and, hopefully, to aid us in designing a circuit. Therefore there must be ways to model the "memory" some components present in practice.

Since you seem aware of the math involved in modeling a linear time-invariant (LTI) system using impulse responses and convolution integrals, so I will skip some details. The point is that a system with no memory, like a resistor, will be described with a relationship which relates input and output at the same instant of time: \$v(t) = R \cdot i(t)\$. This is an example of so-called (purely) algebraic system or memoryless system , in system theory jargon.

If you translate it into an impulse response model, you see that such a system has a purely impulsive impulse response, namely: \$ h(t) = R \delta(t) \$. In fact:

$$
v(t) = h(t) \otimes i(t) = R \delta(t) \otimes i(t) = R i(t)
$$

where I indicated convolution by the symbol \$\otimes\$.

Take the Fourier transform (or Laplace transform) of \$h\$ and you'll get the so-called frequency response function (or transfer function) of the system \$H(j2\pi f)\$ (or \$H(s)\$).

As you can see, in the example of the resistor, \$H(s) = R\$, hence the component has infinite bandwidth, one hallmark of a memoryless system.

A system that can store energy, i.e. a system with "memory" (e.g. a reactive component), will have a more complicated representation. If it is a linear system (impulse response is useful only in that context) the corresponding impulse response will not be a simple Dirac function, but will be non-zero even after the initial instant when the Dirac delta is applied at the input (that's the point of the impulse response, it is a response to a \$\delta(t)\$ input). The fact that the response of the system to an excitation at time 0 extends further in time it's a sign that the system "remembers" what happened at time 0 and "keeps on doing something" after "the fact".

If we translate this thing in the frequency domain or the s-domain we discover that the  system won't have unlimited bandwidth, but its frequency response will decay outside some limited frequency interval (its pass-band). So, definitely the "smearing" of a signal in the time-domain, which corresponds to a bandwidth limitation in frequency domain, is a consequence of the system having "memory", i.e. responding to excitations by remembering what happened in its past. Another point to consider: a (physically-realizable) band-limited system is a system that introduces a delay from input to output. This again call forth the idea of "memory": the system reacts after a while to the input because "it remembers something happened in the past".

Of course that is only the limited perspective of LTI system. You can have memory also in non-linear systems, for example transformers core. You may know that ferromagnetic materials used to fabricate cores are non-linear materials that present a phenomenon called hysteresis. I won't delve in the details, but the core "remembers" its magnetization state (that is how magnetic core computer memories worked in the past). Again, physically this reflects a change in the state of the object due to some energy transfer in the past: some energy must have been provided to the core to realign its magnetic domains. The residual magnetization is a clue that "something happened" to the core in its past.