STM32: SPI hardware to implement custom serial interface
assume I can pick any one up to M7


Then pick one from STM32H7 series, it has direct hardware support for all the features you need for this, independent fractional PLL to generate arbitrary SPI clock rates, and variable frame length if you need bit stuffing, i.e. frame length is not necessarily a multiple of 8 bits.

Any other STM32 can do it as well, even the smallest ones, but some tricks would be needed, like looping back a timer output externally to the SPI clock input, as they have only a simple power-of-two prescaler for SPI clock generation.


  Any better way to achieve this with only an STM32?


I can't think of any.


  How can I generate 153.6 and 614.4 kHz clocks for SPI Tx?


Experiment with the clock tree configurator in STM32CubeMX to understand how it works (faster than by reading the reference manual), get PLL2 or PLL3 to output 0.1536*256=39.3216 MHz and map it to the SPI clock. Then use the prescaler in the SPI peripheral to get 153.6 kHz (/256) or 614.4 kHz (/64).



I'm quite certain that a better match is possible by fiddling with the divisor and multiplier values.


  Can the SPI slave receive any clock? Like 614.4 kHz?


Any clock up to the limit in the datasheet, which is usually in the tens of MHz range. There is no low limit.