Multi User limited Time Encryption of big Datasets Updated Question: New Thread



Original Question: (Thanks cypherfox for answering that one)

I am in need to store data > 100 TB on a local network share (NAS/SAN) and grant individuals access to the encrypted data. For all users, besides me, the data can be read only, and sequential access to the data (only one user can access the data at one time) is fine.  

The biggest problem is that I need to revoke individual users access from time to time (in addition to the recall of the network share access). I could encrypt the master key with user individual keys but this would not prevent technical experienced users from stealing the master key right? So I would need to re-encrypt the data every-time a user is recalled. Is there a way to do this more efficient than copying all data to a new encrypted image/disk? For 100 TB this would need more than 9 days. 

Maybe someone can give me an advise which system/tools/techniques I should use and where the decryption should take place (client/server). Here the summarized key facts:


~ 100 TB Data
max 5 user at the same time or 20 over the whole time span
data stored an a local NAS/SAN connected to an 'offline' network
the files have easily multiple GB, so some kind of transparent decryption to avoid copying the data would be necessary
even a user could copy the data while he is a regular user, and even if he has no network share access anymore, we need to depreciate his decryption access
I'd start by dividing the read, write, and replicate roles into 3 keys. Tahoe-LAFS does this using "semiprivate" keys, each no larger than an EC curve point. I.e. Using ed25519, these keys would all be 32-bytes. The write key implies read, and the read key implies the replicate/verification key.

Once these keys have been revealed to someone, they cannot be revoked. To implement revocation, we need an additional authority that may be revoked, such as Macaroons. Macaroons support both delegation (sharing) and provisioning (with additional restrictions). These restrictions may reference clocks, or limit the functions the user may call (I.e. to read, write, append, truncate, delete, ..), or query out to external services to either check if the credential is still valid (revocation) or to check if the message is approved by the authority (configurable filters).

You cannot force a user to delete any ciphertexts, keys/credentials, nor plaintexts. You can revoke credentials to limit the distribution of ciphertexts. You don't need to rotate the encryption key so long that your adversary cannot also obtain the ciphertext (a revoked user cannot).

If your trusted server is allowed to read the user's data, then you don't need encryption at all (aside from session transports) as you only care about the authority to access. If the users can create new documents, then you'll likely prefer to lock the server (and you, the administrator) out of the user' personal data.