Replace a failed disk in a Disk Utility RAID 1
I was unable to find how to rebuild the RAID from disk utility after a couple of hours of research. However, I managed to do it from the command line.

sudo diskutil list  // to get a list of disks. Note the disk number of your new disk (e.g. disk4)
sudo diskutil appleraid list // note the uuid of the raid in question
sudo diskutil appleraid add member disk4 UUID-OF-YOUR-RAID


It will then kick you back to disk utility and start rebuilding the raid :)

Also, if the failed disk remains on the list, you can remove it:

diskutil appleraid remove UUID-OF-FAILED-DISK UUID-OF-RAID