How to transition from macOS Server deprecated services to the open source alternatives?
In June 2018, Apple released official documentation outlining the step by step directions to move most services to alternatives.
The services included in this are

DNS
DHCP
VPN
RADIUS
NetInstall
Websites
Wiki
Calendar and Contacts
FTP
Software Update


The goal of each section is to have

The identical configuration as the macOS Server [service name] service.

Note: Some services are unable to be replicated completely

Major elements of the macOS Server Wiki structure, user model, and semantics—that is, separate wikis with separate ACLs, pages and files associated with specific wikis, user pages and blogs, page histories, directory- based users and groups, themes, avatars, custom banners won’t fully map to any current open source project.
APNs requires special Transport Layer Security (TLS) certificates in order for the server to authenticate to the Apple-hosted service. Those certificates are generated by macOS Server. There is currently no alternative means of generating the required APNs certificates, so push notifications can’t be used when macOS Server is no longer available.