Title: On Bayesian Exponentially Embedded Family for Model Order Selection,
Abstract: In this paper, we derive a Bayesian model order selection rule by using the
exponentially embedded family method, termed Bayesian EEF. Unlike many other
Bayesian model selection methods, the Bayesian EEF can use vague proper priors
and improper noninformative priors to be objective in the elicitation of
parameter priors. Moreover, the penalty term of the rule is shown to be the sum
of half of the parameter dimension and the estimated mutual information between
parameter and observed data. This helps to reveal the EEF mechanism in
selecting model orders and may provide new insights into the open problems of
choosing an optimal penalty term for model order selection and choosing a good
prior from information theoretic viewpoints. The important example of linear
model order selection is given to illustrate the algorithms and arguments.
Lastly, the Bayesian EEF that uses Jeffreys prior coincides with the EEF rule
derived by frequentist strategies. This shows another interesting relationship
between the frequentist and Bayesian philosophies for model selection.