What is the difference between conditional and unconditional risk premia?
In case someone finds this helpful, I'll answer my own question.

Risk premium, say for example on some risky assets such as stocks, is the expected return on the risky asset less the risk-free rate.

However, empirically we have that this risk premium on stocks is much larger than the riskfree rate and also that it varies quite a bit more. So we introduce the idea of conditional vs. unconditional, which speaks to this time-varying aspect.

Conditional risk premia means risk premia conditioning on time-$t$ information (so with a time-$t$ subscript as in conditional moments, conditional variance, etc.). So it is the conditional expectation of the return of a risky asset over the risk free rate, where the conditioning is done on the agent's available information at time-$t$. Although it depends on the model, this is an object that is in general time-varying.

Unconditional risk premia just means risk premia without conditioning on time-$t$ information. Generally, this is not time varying.

A good reference for learning about conditioning information on asset pricing models is Chapter 8 in John Cochrane's book "Asset Pricing".