Creating an Arbitrary Custom Transform Orientation in Python
Test code to create orientation and use in all 3d views in the current (context) screen.

import bpy
from math import radians
from mathutils import Matrix
context = bpy.context
scene = context.scene

# create view

bpy.ops.transform.create_orientation(name="Frank", overwrite=True)
orientation = scene.orientations.get("Frank")
print(orientation)

# make up some matrix
# rotated 45 degrees on (1, 1, 1) axis
m = Matrix.Rotation(radians(45), 3, (1, 1, 1))
orientation.matrix = m

# find 3d views to set to "Frank"
screen = context.screen
views = [area.spaces.active for area in screen.areas if area.type == 'VIEW_3D']
for view in views:
    view.transform_orientation = orientation.name


Note you can override operators to "trick" them into thinking they are running in another area.