How can I size the DC link capacitor of a 3 phase inverter?
If the inverter is required to supply an inductive load, the DC link capacitor needs to be sized to carry the reactive component of the load. The reactive load current will produce a high ripple current in the link capacitor. That will require a higher link capacitance than would be required to smooth the ripple voltage of the rectifier. Look for papers or a text on PWM variable frequency drive (VFD) design.

The diagram below shows the basic power circuit for rectifier-inverter or AC to AC converter with a DC link. The load is shown as single motor as for a VFD, probably the most common use of this circuit.



For an inductive load, the load current lags the inverter output voltage. That causes current to flow through the inverse parallel diodes back to the DC link during a part of each cycle. The inverter load consists of real power, mostly converted to mechanical output power, and reactive volt-amperes required to maintain the magnetic fields. The real power is seen at the DC input I(InvDC) as the average DC current multiplied by the average DC voltage. The reactive volt-amperes are seen at the same point as a ripple current component. Since reactive energy can not pass through the rectifier, it is seen as ripple current in the DC link capacitor. The DC link capacitor must, in effect, act as a power factor correction capacitor for the motor.