How do I set up HTTPS on a multi-site?
Are you using a module to handle the multisite configuration? A place where you can associate each site to a domain, theme and other configuration settings. Also you should be able to add https protocol to the desired domain.

If you don't use any module, check settings.php of each site looking for $base_url where you can change it from http to https.