Clearing cache breaks site
Are you sure there are no differences in the codebase? This would explain the issue. The plugins/routes are cached in the database until you clear the cache. Then they are rediscovered from the codebase and the missing plugins or routes become visible. It's not necessarily that the code is missing physically, it can also be a problem with autoloading. 

If there is a problem with syncing the codebase by git/composer, use another tool, for example rsync. Make sure to sync the complete codebase, also the files in the root folder.