Safe to rely on individual 18650 cell protection in a 4S4P configuration?
From the end:    

I DO think that recycling cells from old packs is potentially viable. Often one or a few cells fail and 'take the whole pack down' and people do reuse cells on an amateur basis and are planning to do so commercially at large scale. But for good results a reasonable degree of cell matching or of individual cell management seems almost mandatory.

_________________________________

A 4S4P hot swappable 18650 battery bank is doable with sufficient* design.
BUT I'd expect the chances of it working safely and reliably with multiple swaps over an extended period, using just original individual cell protection, would be minimal. 
A system with tailored individual cell protection would have greater chance.
Best of all would be a system that worked on nested single cell, per individual string and multiple string protection. Whether that would be economic or viable is TBD. I know that there are people attempting to do this commercially - whether they can succeed is also TBD. 

*The value of "sufficient" depends on what the end product is to be used for and in what environment and how well you are willing or able to characterise cells before use. Original individual cell protectors  ("ICP")  aim at fairly coarse "don't let it die spectacularly" protection. So while "vent with flame is a failure, "be treated roughly and lose capacity after too few cycles" is beyond its purview. 

Individual cells should NEVER be charged to above 4.2V/cell. How accurate the individual 4.2V of a given ICP is is unknown and may vary with quality / brand/temperature/POSSIBLY aging and Murphy. Charging a 4S string to 16.8V MAY allow no individual cell to charge beyond 4.2V and MAY allow all cells to charge to within say 0.25 to 0.5V of 4.2V, and may not. If cells are mixed indiscriminately then worst case may conceivably see 3 x conservative voltage limiters and one no longer really working one mixed in a 4S string so you get say 3 x 4.1V and 1 x 4.5V. You MAY survive that. You may not.  

"Real people" seem to think you need to balance  4S strings in normal use, so there seems little chance that semi randomly selected strings of 4S unbalanced would work well. Best case the individual string voltage limiters would work, but if termination is based on I chg at CV tailing down to some % of CC max then overall string capacity could be significantly reduced by one low capacity cell. This then tends to "pump" the bad cell down over multiple cycles as it is as say 90% when others are at 100% on charge, so that on the following discharge it reaches Vminimum before the others and as Vstring is not yet at Vininun_string the low cell gets driven into over-discharge, has its capacity further reduced thereby and then trails the others on the next charging cycle.

I could keep on making up various realistic but also semi-contrived "could happen" scenarios but it should be enough to say that the variability in normal use with same brand and model cells in similar condition is enough that active balancing is deemed necessary for commercial use SO in this case with mismatch of just about everything it seems an utter necessity.

I DO think that recycling cells from old packs is potentially viable. Often one or a few cells fail and 'take the whole pack down' and people do reuse cells on an amateur basis and are planning to do so commercially at large scale. But for good results a reasonable degree of cell matching or of individual cell management seems almost mandatory.