Is short-term memory theoretically possible to with spiking neurons?
While we don't entirely know how all aspects of working memory operate, experiments show that a subset of cells in the brain exhibit persistent activity during working memory tasks.
Here's an example from a review by Curtis &amp; D'Esposito:

This is a task in panel (a) where there is a cue presented briefly (marked by &quot;C&quot;), and then the monkey needs to remember the cue (during the time marked by &quot;D&quot;), and finally at time R the targets appear and the monkey can make a response.
We see in spiking recordings from monkeys that there are cells in dorsolateral prefrontal cortex that are activated not only by the cue and during the response, but they have persistent activity during the delay period. In human studies we can't ordinarily place electrodes but part (b) shows a similar study using fMRI, and the colored areas on the brain indicate brain areas that demonstrated more blood flow (marking metabolism) during the delay period, which includes areas of the same brain region as the monkey experiments.
The explanations are recurrent network activity and bistability: a population of cells that is excited can continue to excite each other and remain active until stopped (see for example attractor networks for a conceptual idea), as well as bistability in individual neurons where a neuron that reaches threshold continues to fire tonically rather than firing a single action potential, until firing is ceased via inhibition. Camperi and Wang 1998 is an excellent, clear review of these concepts with a straightforward mathematical approach to how you can model this in silico. It's probably more efficient to start from here rather than reinventing it on your own.
I also recommend other papers by XJ Wang who has done extensive work in this area. I also recommend you look at phase-space diagrams which can help conceptualize dynamic systems with stable properties (and which network parameters are necessary for stability), they come up a lot in theoretical neuroscience. Dayan and Abbott's Theoretical Neuroscience is also a good starting point if you can find a copy.

Camperi, M., &amp; Wang, X. J. (1998). A model of visuospatial working memory in prefrontal cortex: recurrent network and cellular bistability. Journal of computational neuroscience, 5(4), 383-405.
Curtis, C. E., &amp; D'Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends in cognitive sciences, 7(9), 415-423.