My hybrid motor starter keeps tripping
Pardon my ignorance on this application, but I do not know the nature of your load current if that was measured or rated. 


My understanding is the Load Transfer Switch(LTC) &amp; motor operate in a discontinuous mode of AC start-surge then DC-rapid stop, then idle for &lt;30 seconds in order to regulate the primary grid voltage into the transformer. The  Hybrid Relay responds to Control Left(Reverse) and Right(Fwd) motor commands in a rotary LTC high-voltage high current-switching Tap Changer for either a Reactive or resistive, On or OFF Load Diverter.

The start/FLA (Full load Amp) current ratio of 3.5 (=12.1A/3.5A) is common for 3ph induction motors. The intervals of start/stop surge-current and the **unknown dwell time*  for start/stop, are a major factor in choosing the right protection levels.

This must be a special application for some Load Transfer Switch, either reactive or resistive Load Transfer Changer (LTC) for a large power transformer to regulate the primary coil voltage every 30 seconds.  You must be a designer for this power transformer to understand the nature of this load.

Depending on the condition of rotary contacts this motor moves, there may be some characteristic loads I am not familiar with, during the arc-current bypass operation in the load during this motorized transfer-switch operation, during both the motor start and rectified AC to DC winding fast stop operations.  The timing and design of these characteristics must be controlled such that LTC motor-drive starts and stops quickly to minimize LTC wear. 

If the start current is 12A then a full speed reverse will be 24A and if reversing every 30s your average power will exceed the breaker 9A rating. 


It is unknown what the Relay current is during a rotate start-stop and if motor uses the DC brake option.


The Hybrid Breaker estimates the heat rise in the motor using some I^2 vs time delay algorithm. 
  -  This Current-time profile is also unknown.   


Since the motor is not too hot when tripped, then the protection level is too low or something has changed in the mechanical loads of LTC contacts that have changed the performance.


It is not a good idea to reverse at full speed as you should stop first then reverse.  

Advice

I recommend;
- contact the OEM's for motor &amp; LTC load current profiles
- get Phoenix Contact's Amp-second breaker curves
- monitor breaker current characteristics for conditioned maintenance to LTC.
- join / contact Linked-In for a  wealth of wisdom in the Transformers group
- consider Consultant: Mr Csanyi for expert advice.