1. A memory element, comprising: a first electrode, a memory layer, and a second electrode in this order, wherein the memory layer includes a resistance change layer provided on the first electrode side, an ion source layer provided on the second electrode side, an intermediate layer provided between the resistance change layer and the ion source layer, and a barrier layer provided at least either between the ion source layer and the intermediate layer, or between the intermediate layer and the resistance change layer, and the barrier layer containing a transition metal or a nitride thereof.