How does one characterize a neural network with threshold-based activation functions?
The model you describe is a kind of a leaky integrate-and-fire (LIF) neuron (see p. 7). It is leaky because the membrane potential decreases steadily in the absence of input. In contrast, in the simple integrate-and-fire (IF) model the membrane potential is retained indefinitely until the neuron spikes, at which point it is reset to 0. However, LIF neurons are usually modelled with exponential decay of the membrane potential, where you have a time constant $\tau$ and you compute the potential $P_{t}$ at time $t$ based on the potential $P_{t_{last}}$ at time when the last input arrived as 

$P_{t} = P_{t_{last}} exp(- \frac{t - t_{last}}{\tau})$

This is the same formula as radioactive decay (see here for more details). The idea is that this model is inherently 'aware' of time, whereas the IF model (and your design above) do not factor in the timing of the spikes, so they act like a classical neural network activation. In any case, whether or not a neuron would fire does depend on the firing threshold, so I think that treating the threshold as a learnable parameter is justified - you just have to decide what rules to use for updating it.

Based on what you describe as your understanding of spiking neural networks, it seems that you have been reading about the Hodgkin-Huxley (HH) model (also in that paper I linked to). (Please correct me if I'm wrong.) You are correct in thinking that spikes in the brain are not infinitely narrow like a delta function but more like a very sharp sinusoidal signal, and the HH model faithfully reproduces that. However, the reason why the HH model is not actually used for simulations is that it is computationally very taxing. In practice, in most cases we do not actually care about the state of the neuron between inputs, as long as your model accurately describes the neuron state and what happens to it when an input arrives. 

There are other models that approximate the HH model very closely but are much faster to simulate (like the Izhikevich model). However, the LIF model is very fast and sufficient in most cases.

Hope this helps!