Time series of categorical data is not a widely studied research topic. Particularly, there is no available work on the Bayesian analysis of categorical time series processes. With the objective of filling that gap, in the present paper we consider the problem of Bayesian analysis including Bayesian forecasting for time series of categorical data, which is modelled by Pegram's mixing operator, applicable for both ordinal and nominal data structures. In particular, we consider Pegram's operator-based autoregressive process for the analysis. Real datasets on infant sleep status are analysed for illustrations. We also illustrate that the Bayesian forecasting is more accurate than the corresponding frequentist's approach when we intend to forecast a large time gap ahead. Copyright (c) 2016 John Wiley & Sons, Ltd.