Details on soft decision trees In Distilling a Neural Network Into a Soft Decision Tree the authors mention 


  exponentially decaying running average of the actual probabilities with a time window


but provide no formula for it.

What does it mean speaking formal?
This is likely the Exponential Moving Average function.
In simplest form it is
ma_new = alpha * new_sample + (1-alpha) * ma_old

Where alpha is the parameter that controls rate of decay, between 0.0-1.0.