Can one have a condition like this in semidefinite programming?
Assuming that a, b, and u are real variables in your semidefinite program, the answer is negative.

The important fact here is that each constraint in a semidefinite program defines a convex set.  Your condition cannot be written as a constraint in a semidefinite program because the set of points (a, b, u) ∈ ℝ3 that satisfy the condition is not convex.  I suggest that you try to prove that this set is indeed not convex, but here is a proof (put the mouse cursor in the box below to show the proof):


   Proof: (a, b, u) = (1, 1, 1) and (a, b, u) = (3, 3, 9) satisfy the condition, but their midpoint (a, b, u) = (2, 2, 5) does not satisfy the condition.