OS X Server leaves some unwanted open ports after removal
Just removing the Server.app (3.2.2) for 10.9 Mavericks isn't enough. You also have to unload several launch daemons.

By comparing all "System LaunchDaemons" in a basic Server.app install and an LDAP/VPN environment in two different VMs I assume that the following system launch daemons have to be unloaded permanently:

org.openldap.slapd
com.apple.xscertd-helper
com.apple.xscertd
com.apple.xscertadmin
com.apple.PasswordService
com.apple.odproxyd
org.apache.httpd


The most comfortable tool to do this is probably LaunchControl.

To completely remove all files and folders created by installing, launching and configuring Server.app I have to compile a list of files and folders to delete first (which will take some time).

Here is a list of newly created files and folders after installing and configuring OpenLDAP and VPN in Server.app (3.2.2):

/System/Applications/Server.app/
/System/Applications/Workgroup Manager.app/
/System/Library/LaunchDaemons/com.apple.serverd.plist
/System/Library/Logs/EventMonitor/
/System/Library/Logs/EventMonitor/EventMonitor.error.log
/System/Library/Logs/Mail/
/System/Library/Logs/PasswordService/
/System/Library/Logs/PostgreSQL/
/System/Library/Logs/ProfileManager/
/System/Library/Logs/radiusconfig.log
/System/Library/Logs/Server
/System/Library/Logs/ServerSetup.log
/System/Library/Logs/Setup.log
/System/Library/Logs/slapconfig.log
/System/Library/Logs/WebDAVSharing.log
/System/Library/Logs/WebServer
/System/Library/Preferences/com.apple.AccountsConfigServer.plist
/System/Library/Preferences/com.apple.AppleFileServer.plist
/System/Library/Preferences/com.apple.openldap.plist
/System/Library/Preferences/com.apple.serverd.plist
/System/Library/Preferences/com.apple.servermgr_accounts.plist
/System/Library/Preferences/com.apple.servermgr_info.plist
/System/Library/Preferences/com.apple.servermgrd.plist
/System/Library/Preferences/edu.mit.Kerberos.kadmind.launchd
/System/Library/Preferences/edu.mit.Kerberos.krb5kdc.launchd
/System/Library/Preferences/OpenDirectory/Configurations/LDAPv3/
/System/Library/Preferences/OpenDirectory/DynamicData/…………….plist
/System/Library/Preferences/OpenDirectory/DynamicData/LDAPv3/
/System/Library/Preferences/SystemConfiguration/autodiskmount.plist
/System/Library/Preferences/SystemConfiguration/com.apple.RemoteAccessServers.plist
/System/Library/PrivilegedHelperTools/com.apple.serverd
/System/Library/Security/Trust Settings/
/System/Library/Server/
/System/private/etc/af.plist
/System/private/etc/asl/com.apple.server.asl
/System/private/etc/bootpd.plist
/System/private/etc/certificates/Server Fallback SSL Certificate… #several
/System/private/etc/certificates/Code Signing Certificate… #several
/System/private/etc/certificates/fqdn.…………….cert.pem
/System/private/etc/certificates/fqdn.…………….chain.pem
/System/private/etc/certificates/fqdn.…………….concat.pem
/System/private/etc/certificates/fqdn.…………….key.pem
/System/private/etc/emond.d/state/
/System/private/etc/krb5.conf
/System/private/etc/newsyslog.d/com.apple.devicemgr.conf
/System/private/etc/newsyslog.d/com.apple.mailservices.conf
/System/private/etc/newsyslog.d/servermgr_calendar_log.conf
/System/private/etc/openldap/rootDSE.ldif
/System/private/etc/openldap/slapd_macosxserver.conf
/System/private/etc/openldap/slapd.conf
/System/private/etc/openldap/slapd.d.backup/
/System/private/etc/openldap/slapd.d/
/System/private/etc/paths.d/com.apple.server
/System/private/etc/pear.conf
/System/private/etc/php.ini
/System/private/etc/php.ini-5.4-previous
/System/private/etc/postfix/aliases.desktop
/System/private/etc/rc.server
/System/private/etc/rc.server.firewall
/System/private/tftpboot/NetBoot/
/System/private/var/db/.ServerSetupDone
/System/private/var/db/af/
/System/private/var/db/BootCaches/……………/app.com.apple.Server.v3.playlist
/System/private/var/db/dovecot.fts.update/
/System/private/var/db/emondClients/com.apple.server
/System/private/var/db/krb5kdc/
/System/private/var/db/launchd.db/com.apple.launchd.peruser.220/
/System/private/var/db/launchd.db/com.apple.launchd.peruser.70/
/System/private/var/db/launchd.db/com.apple.launchd.peruser.93/
/System/private/var/db/launchd.db/com.apple.launchd.peruser.94/
/System/private/var/db/net-snmp/
/System/private/var/db/ntp-kod
/System/private/var/db/openldap/authdata/
/System/private/var/db/openldap/replication/
/System/private/var/db/receipts/com.apple.WorkgroupManager.bom
/System/private/var/db/receipts/com.apple.WorkgroupManager.plist
/System/private/var/db/ServerPerfLogClients/com.apple.server
/System/private/var/db/systemstats/…………….boot.events.….stats
/System/private/var/db/systemstats/…………….powerd.events.….stats
/System/private/var/db/systemstats/…………….system_events.events.….stats
/System/private/var/db/systemstats/…………….system_events.periodic.….stats
/System/private/var/dovecot/
/System/private/var/folders/zz/
/System/private/var/log/apache2/accept.lock.710
/System/private/var/log/apache2/access_log
/System/private/var/log/apache2/error_log
/System/private/var/log/apache2/rewrite.lock
/System/private/var/log/caldavd/
/System/private/var/log/caldavd/agent.log
/System/private/var/log/caldavd/certupdate.log
/System/private/var/log/caldavd/migration.log
/System/private/var/log/caldavd/postgresql/
/System/private/var/log/caldavd/postgresql/postgresql_1.log
/System/private/var/log/caldavd/servermgr_calendar.log
/System/private/var/log/caldavd/xpg_ctl.log
/System/private/var/log/com.apple.launchd.peruser.220/
/System/private/var/log/com.apple.launchd.peruser.70/
/System/private/var/log/com.apple.launchd.peruser.93/
/System/private/var/log/com.apple.launchd.peruser.94/
/System/private/var/log/com.apple.launchd/…………….launchd.events.….stats
/System/private/var/log/devicemgr
/System/private/var/log/eventmonitor/
/System/private/var/log/eventmonitor/StoreData
/System/private/var/log/getsslpassphrase.log
/System/private/var/log/krb5kdc/
/System/private/var/log/localemanager.log
/System/private/var/log/mail.log
/System/private/var/log/ppp/vpnd.log
/System/private/var/log/radius/
/System/private/var/log/servermgrd.log
/System/private/var/log/slapd.log
/System/private/var/log/swupd/
/System/private/var/log/xscertd.log
/System/private/var/pgsql_socket/
/System/private/var/run/caldavd/
/System/private/var/run/httpd.pid
/System/private/var/run/jabberd/
/System/private/var/run/kadmind.pid
/System/private/var/run/kdc.pid
/System/private/var/run/kpasswdd.pid
/System/private/var/run/ldapi
/System/private/var/run/passwordserver
/System/private/var/run/racoon.pid
/System/private/var/run/racoon/
/System/private/var/run/servermgrd.pid
/System/private/var/run/slapd.args
/System/private/var/run/slapd.pid
/System/private/var/run/vpnd-L2TP.pid
/System/private/var/servermgrd/
/System/Users/username/Library/Caches/com.apple.Server.v3/
/System/Users/username/Library/LaunchAgents/
/System/Users/username/Library/LaunchAgents/com.apple.serveralertproxy.plist
/System/Users/username/Library/Preferences/com.apple.Server.v3.plist
/System/Users/username/Library/Preferences/com.apple.ServerAssistant.plist
/System/usr/bin/pear
/System/usr/bin/peardev
/System/usr/bin/pecl
/System/usr/lib/php/.channels/
/System/usr/lib/php/.depdb
/System/usr/lib/php/.depdblock
/System/usr/lib/php/.filemap
/System/usr/lib/php/.lock
/System/usr/lib/php/.registry/
/System/usr/lib/php/Archive/
/System/usr/lib/php/Console/
/System/usr/lib/php/data/
/System/usr/lib/php/doc/
/System/usr/lib/php/OS/
/System/usr/lib/php/PEAR.php
/System/usr/lib/php/PEAR/
/System/usr/lib/php/pearcmd.php
/System/usr/lib/php/peclcmd.php
/System/usr/lib/php/Structures/
/System/usr/lib/php/System.php
/System/usr/lib/php/test/
/System/usr/lib/php/XML/
/System/usr/lib/sasl2/openldap/digestmd5WebDAV.la
/System/usr/lib/sasl2/openldap/digestmd5WebDAV.so
/System/usr/lib/sasl2/openldap/libcrammd5.2.so
/System/usr/lib/sasl2/openldap/libcrammd5.la
/System/usr/lib/sasl2/openldap/libdigestmd5.2.so
/System/usr/lib/sasl2/openldap/libdigestmd5.la


keys:

/: if a line ends with a / the whole folder or app and its content is newly created  
username: your username  
……………: some arbitrary UUID or digits/letters &gt; 8
…: some arbitrary digits/letters &lt; 8
fqdn: full qualified domain name


Despite the greatest care taken in the creation of this list I cannot guarantee its accuracy. Delete those files at your own risk.