Determining pulse shape for Viterbi Equalization I believe I understand well enough how the Viterbi algorithm can reverse the effects of ISI when the combined impulse response (transmit filter, multipath, receive filter) is known. I've read How I Learned to Love the Trellis by Bernard Sklar, and I also asked about it in a previous question: Applying Viterbi algorithm to compensate for ISI in PSK31.

But how can this technique be applied when the channel distortion isn't known? From my superficial understanding of equalizing algorithms like CMA, they attempt to infer the channel distortion and compensate for it dynamically. My particular interest is HF channels, which would certainly seem to benefit from some dynamic adjustment.

However, in my study of Viterbi equalization so far, I've not found an explanation of how the unknown impulse response of the channel is found. Without that information, the best we can accomplish is to compensate for known elements like the receive and transmit filters. And in a case with no inherent ISI (like RRC filters on each end), then does Viterbi equalization have any value at all?
I'll answer my own question with what I've learned in the past 28 days. Hopefully I've got it right!

Viterbi equalization is just one way to equalize the channel, as an alternative to some other method such as convolving the signal with an equalizing filter. Estimating the channel is an entirely separate problem which must be solved independently of the equalization method.

The trouble with an equalizing filter is it compensates for dips in the channel frequency response by amplifying those frequencies. Unavoidably, the noise is also amplified.

There's a tradeoff to be made: the flatter the frequency response is made, the lower the error due to ISI, but the higher the error due to noise. There's an optimal amount of equalization which minimizes the error due to noise and ISI. An equalizer that tries to hit this sweet spot is a minimum mean square error (MMSE) equalizer.

Viterbi equalization offers an alternative solution. The channel estimate will produce an estimated channel impulse response. The coefficients in this impulse response can be used to build the trellis, and the Viterbi algorithm can then find the most likely sequence of transmitted bits, taking into account the estimated distortion from the channel. The receive filter remains a simple matched filter, with no equalization. The Viterbi equalizer is not linear (unlike the equalizing filter alternative), so it's able to compensate for deep spectral nulls in the channel response in a way the equalizing filter can't. The cost is increased complexity.