Fair assignment of storage for multiple writers (load balance) I'm trying to optimize a system with a better algorithm for allocating storage.  The system has 'N' writer processes and 'M' disks. (N &lt; 30, M &lt; 10. N can change, M is constant).
Any process can write to any disk and each process regularly writes a file every, say, 5 minutes, but the size of the file can be quite different. Each writer independently &quot;chooses&quot; which disk it writes to on a file-by-file basis; so it might write one file to Disk 2 and the next file a few minutes later to disk 8. There is no communication between writers.
This goes on forever.

In the steady state, all disks are &quot;full&quot; and a &quot;deleter&quot; process continuously deletes the oldest files until all disks are no longer &quot;really full&quot;.  (there are two water marks: &quot;almost full&quot; and &quot;really full&quot;, if that helps), so the &quot;deleter&quot; wants:
        Almost Full    Really Full
Disk 1       Y             N
Disk 2       Y             N
Disk 3       Y             N

I need to optimize how the writer chooses which disk to write to such that, as much as possible, different writers are writing to different disks. This is important to balance I/O load.
I had thought this wouldn't be too hard, but the more I think of it, the more problems arise. I've tried a number of things:

Randomly choose disk from those with free space. However since there aren't many disks (M is small), there is a good chance all writers will choose the same destination disk and overload the I/O system.  Bad.

Index/hash the writer number across the list of disks with free space. So if N == M, Writer 1 always ends up writing to Disk 1. However this fights with the deleter working to maintain free space by deleting the oldest files: eg, the first disk gets full but the oldest file is on a different disk.

Use first free disk. However this ends up with all writers writing to the same disk.
There are multiple solutions to this problem:

Consistent hashing with bounded loads
Generic Bin packing algorithm(Knapsack or other)

Both require that the writer know the free storage amount of the drives.
Consistent hashing with bounded loads is based on the consistent hashing algorithm published by akamai and fixed the issue of load inbalancing by overflowing across the ring to the next free &quot;bin&quot;.