What could be the cause of high leakage inductance on my custom flyback transformer?
the main cause of high leakage inductance is too much separation between the primary and secondary windings.  But please provide more details, such as a picture of the cross section, what's the core material, is it gapped, etc.