BLDC Motors: Calculating the offset between rotor flux angle and rotary encoder Measurement
Your initial idea is correct. I will make the assumption that you have a function that runs the Space Vector Machine and that takes 2 input arguments: the stator angle and the magnitude of the stator vector you want to apply. Here is what I suggest to you to solve your problems :
Motor phase sequence not always the same :
Main idea: Feed the SVM with an angle ramp and a positive amplitude to see if rotor turns in the right direction.

Connect your phases the way you want on your motor controller;
Remove any load on the rotor;
Launch a first procedure that manually ramp the SVM's angle at a slow and desired speed, with a constant positive vector amplitude. It will generates a rotary stator vector and the rotor magnetic field will align with it (some friction could make them not perfectly aligned, but you do not care here). If speed is low enough, magnitude high enough (but not too high to avoid excessive current) you should see the rotor spinning;
According to the direction of rotation, you can determine if the phase sequence is the good one, or if 2 phases have been switched. Keep in mind that a circular permutation of all the phases will not reverse the speed, but it will &quot;add&quot; 120 deg to your offset. But you don't mind because you will automatically compute the offset.
If the direction is not the good one, switch duty cycles of 2 phases (let's say A and B) just before the SVM will apply them.
Save this information in some Flash memory to use it after a reboot.

Automatically compute the offset (1st method)
Main idea: Feed the SVM with a 0 angle and look at the AS5147 measured angle to find the offset


Feed the SVM with a 0 angle and a positive constant amplitude. To find the right amplitude value, do some tests. Increase it until the rotor is locked and hard to turn by hand. Be careful not to drive too much current;
The rotor field will align with the stator field along the a axis of the (a,b,c) reference and the AS5147 value will be your offset.

This method will not be extremely accurate because of friction but it will give you a good estimation of the offset.
Automatically compute the offset (2nd method)
Main idea: Feed the SVM with an angle ramp and look at the AS5147 measured angle to find the offset


Manually ramp the SVM angle in one direction (the same ramp as for phase sequence detection will be ok);
Each time the SVM angle is 0, add the AS5147 output to a variable. Do it a couple of motor turns and compute the mean of this value at the end of the sequence;
Do it again but in the opposite direction;
With the assumption that friction has the same effect in both direction, one value will be your offset+friction and the other will be your offset-friction. Sum them up and take the half of it and you will have a quite good offset value.

Bonus to improve the offset calculation
Here is how you should use the previous offset:

Get AS5147 value;
Subtract to it the offset previously computed and then you have the rotor position in the (a,b,c) reference. (Example: a raw value of &quot;Offset&quot; degrees given by the AS5147 will tell you that you are on the A-axis of (a,b,c) reference (0 deg));
Without field weakening, and with no phase compensation due to torque and speed (only a simple open loop control with no Vd), you now feed your SVM with this angle +/- 90deg (Vq) to make the motor turn in one direction or the other.
Apply a pre-defined vector magnitude to the SVM and measure the speed;
Apply the opposite pre-defined magnitude and measure the speed;
Decrease or increase the offset on the fly to make both speeds match.