Calculate total apparent, average and reactive power in 3-phase
I am confused on the difference between apparent, average and
  reactive power and how each is calculated


Average power is the real power you get billed for as kWh and, is the numerical multiplication of voltage and current waveforms, often done in an MCU these days but back in the old days a thing called a wattmeter did things magnetically then, along came analogue multiplier chips and solid state magnetic multipliers as patented by Landis &amp; Gyr. This is all about four quadrant multiplication of the instantaneous signals and not to be confused with either of the following: -

Apparent power is RMS voltage x RMS current and only equals average power (above) on a purely resistive load. For pure sinewaves: -

\$Power_{Average} = Power_{Apparent}\cdot Power\space Factor\$ 

i.e. \$P=V\cdot I\cdot Cos{\phi}\$

Where power factor is the cosine of the phase angle between voltage and current - clearly when that angle is zero, power factor = 1.

Reactive power is \$\sqrt{Apparent^2 - Average^2}\$: -



See also this site for further details.