Should I update my regularisation L1 and L2 regularisation parameters in online setting?
As you say @Alexis, cross-validation is the standard way of selecting a model (or its parameters) in offline learning (especially when previous knowledge in not available). For online training, one usually incorporates some sort of (automatic) compensation for the model parameters that adjust them at the same time that training takes place (the alternative being to cross-validate on some gather data or equivalently to pre-optimize your parameters). Check also this answer as well as, for example, this paper (it shows that this issue is a very active research area)!