Private and public contacts
You're asking two different (but related) questions:


How can I hide a group of contacts from a set of users?
How can I show a group of contacts but hide their activities?


For the first question, you should:


Make sure the non-privileged staff have a CMS role that does NOT have the "View All Contacts" or "Edit All Contacts" permissions.
Add all the contacts they SHOULD have access to into one or more groups.
Use ACLs to grant access ONLY to the contacts the staff should see.


If you want to allow a set of users to see contacts but not their activities:


Install the Activity Type ACL extension.
It's called an "ACL extension", but really this adds CMS permissions to roles so that you can deny contacts access to certain activity types.