Can I just rinse a shaker, solely used for immediate consumption of whey with milk?
So let’s be clear - rinsing with just water won’t remove all of the fats and proteins in your container. It’s definitely not “clean” afterwards.
But even for the bits left in the container food safety guidelines apply, so if you immediately freeze the shaker in between uses, you just have to add up the time the shaker spends in the danger zone and/or above freezing. Whatever value reaches the „no longer safe“ threshold first (two hours for the danger zone above 40°F/4°C, three days in the fridge) gives you the “now it’s really time for a real cleanup” trigger.
Off the cuff, I‘d say that your three day cycle would still be technically safe. Whether it’s really a good idea is up to you. I like my food prep equipment fresh and clean for each use. More frequent cleaning will also serve as extra safety net in case some cleaning cycle was not perfect. And of course in a commercial setting all food handling and cleaning regulations have to be observed independently of what I outlined in this answer.