use ffmpeg to convert interleaved raw yuv/pcm to compressed
Here's a method that should, in principle, work. It assumes VAVA interleaving and for somewhat efficient processing requires the packet sizes for both video and audio having a reasonably large greatest common divisor.

The examples below assume a video packet size of 1382400 bytes and audio of 7680 bytes. As it happens, the former is exactly 180 times the latter. The idea is we can ingest the raw stream as a raw "audio" stream and deinterleave using a selection expression, which routes 180 "frames" to one output and the 181st frame to another. Then the separated outputs are dumped into regular or fifo files which can then be read by another ffmpeg process with a sane interpretation of the input. 

Command 1:

ffmpeg -f u8 -channels 1 -i raw-input-url -filter_complex "asetnsamples=7680,aselect='1+not(mod(n+1,181))':n=2[vid][aud]" -map [vid] -f u8 file-vid -map [aud] -f u8 file-aud

Command 1 with input offset:

ffmpeg -f u8 -channels 1 -i "subfile,,start,3248615,end,0,,:raw-input-url" -filter_complex "asetnsamples=7680,aselect='1+not(mod(n+1,181))':n=2[vid][aud]" -map [vid] -f u8 file-vid -map [aud] -f u8 file-aud

where 3248615 is in bytes and should point to the start of a video frame for this to work.

Command 2:

ffmpeg -f rawvideo -pixel_format yuv420p -framerate F -video_size WxH -i file-vid -f f32le -channels N -sample_rate R -i file-aud { -encoding parameters } output