How should I copy OS X server environment to another machine for update?
There is no silver bullet to migrate your Mavericks server installation to a brand new machine.

It really depends on the acceptable downtime and on the hardware.

First some facts:


Migrating a server installation or a Time Machine backup of it over a network is not possible
Any migration either with the System Assistant or the Migration Assistant has to be completed before starting the new Server.app
Copying a Time Machine backup (Backups.backupdb) can't be done simply with the Finder by drag and drop. Time Machine has to be stopped while copying - it is possible to copy it with the Finder but not by simply dragging and dropping!
A backup to a second partition on the same disk is - in the proper sense - no backup.


Prerequesites:

OS X El Capitan bootable installer thumb drive




dd or rsync method


Start the new Mac in Target Disk Mode and attach it to the old Mac with a Thunderbolt cable
Prepare the additional disk - i.e. remove all partitions except the EFI partition (and the Recovery HD if necessary - but it will be recreated later if not present)
Now either create a new partition using gpt with the same size as the server volume on the target disk and dd the source partition to the target partition in Recovery Mode or booted to the installer thumb drive.
or install Carbon Copy Cloner (which uses rsync), create a new partition on the target with Disk Utility and clone the source volume to the target volume
unmount the new Mac and detach the Thunderbolt cable
boot to the OS X El Capitan bootable installer thumb drive and "upgrade" the old server installation
Download and start Server.app  

Time needed: 
dd: ~85 min (all blocks are copied)
rsync: depends on the size of all files and folders (~50-70 MB/s)
OS X install: ~30-40 min

System Assistant


Start the new Mac in Target Disk Mode and attach it to the old Mac with a Thunderbolt cable
Now create two volumes with Disk Utility - one for the future main volume and one for the Time Machine backup volume.
Disable "Ignore ownership of this volume" for the new "backup volume"
Stop Time Machine in the System Preferences
In Finder open your current backup volume
Choose the backup folder and hit cmdC
open the new Time Machine backup volume and hit shiftalt cmdV. After some time you have to enter an admin name and its password. Alternatively use this method: Time Machine: How to transfer backups from a current backup drive to a new backup drive.
After copying is done unmount the volume and detach the new Mac mini.
Reenable Time Machine in the System Preferences
boot the new Mac mini to the OS X El Capitan bootable installer thumb drive and install OS X to the system volume
While configuring the new Mac use the System Assistant to migrate users and data from the Time Machine backup volume.
Download and start Server.app

Migration Assistant (keep already installed OS X El Capitan)


Split your main volume in two on your new Mac mini
Disable "Ignore ownership of this volume" for the new "backup volume"
Stop Time Machine in the System Preferences on the old Mac mini
Share the Time Machine backup volume on the old Mac mini
Mount the shared Time Machine backup volume on the new Mac mini
Choose the backup folder and hit cmdC
open the new Time Machine backup volume and hit shiftalt cmdV. After some time you have to enter an admin name and its password.
Open Migration Assistant on the new Mac mini and migrate the server installation from the local backup volume
Download and start Server.app

Just bluntly try to update the old server to El Capitan and install the new version of Server.app. Try to recover from Time Machine backup when failed.

If the upgrade is successful that's by far the fastest method (dding to the new server not included). Disable Time machine before upgrading.
(Experimental) Migrate everything to a hypervisor - but that's probably an extra answer.


Get a hypervisor like VMware ESXi 6.0 and install it on a Mac mini
Install the vSphere client on a VM Windows box.
Create an image of the old Mac mini
Probably some hacking (of the hypervisor and the OS X image) is necessary to get that running
In the future you may easily move guest VMs to new hardware by simply firing up vSphere client and choosing a new host also equipped with an hypervisor.
link1 link2 google-search-link