Apple Software RAID: Add disk to array online
So you currently have Disk1 + Disk2 = RAID Set1(striped)
You want to know if you can RAID Set1 + Disk3 + Disk4 = RAID Set(striped).  Disk Utility doesn't allows this.
You can however have Disk3 + Disk4 = RAID Set2(striped) and then use a symbolic link or mounting to move the set to a location that is convenient for you.
You would think that you could Disk1 + Disk2 = RAID Set1(striped), Disk3 + Disk4 = RAID Set2(striped), RAID Set1 + RAID Set2 = RAID Set3(concatenated). Unfortunately it doesn't look like Disk Utility allows this.
Or you do what you mentioned earlier, and backup everything then blow away your existing RAID Set to create a new one with all 4 disks.