Ask HN: Deploying AI service on client's server without exposing model and code?
I have a deep learning inference application written in PyTorch which I currently serve over API. The application is deployed through docker containers.Recently we gained a new client who wants to trial the service, but for privacy reasons they would like the application deployed on their own servers. Just for context the client is a mature online company with decent size.<p>Is there a way to deploy my application without allowing the client to simply copy my model from the container? I assume this can&#x27;t be an unique situation because a lot of AI companies have clients with sensitive data (e.g. hospitals, banks etc.)<p>What I&#x27;ve thought of now is to first embed a kill-switch to delete files from container (since the customer is trialing our product for a limited time). As for how to prevent them from directly attaching to the container and copying everything:<p>1) Restrict access to the docker container somehow (From further research it seems this is not possible with docker)<p>2) Embed some sort of script into the container such that every time user tries to attach the container, the model and source code files get deleted. We will have removed the image from the server after container creation so they won&#x27;t be able to run directly run commands from CLI of the host. (is this feasible?)<p>3) Compile script into C&#x2F;C++ code and obfuscate (this is not ideal since it will take a lot more work and if they can still copy the contents of the container and they may still be able to access the model).<p>4) Other options (any recommendations? maybe a more suitable technology than docker?)<p>I feel like this shouldn&#x27;t be an isolated situation since a lot of deep learning applications have data privacy concerns. There is always the traditional way of compiling the code, obfuscating, and using product key but most deep learning models still run on python.<p>I know there is no fool-proof way but just anything to make it harder for them to simply rip us off.