Connection between castability and convexity
This is a proper answer but feel free to correct me I think I did not get the right definitions. This is why I begin with simple facts that should be checked first.

I suppose your are talking about $\vec v$-castability of a "open" polyhedron.


Indeed it seems that no "closed" polyhedron is $\vec v$-castable.
For every $\vec v$, a convex "closed" polyhedron can always be cut into two "open" parts among a plane of normal $\vec v$ that are $\vec v$-castable and $(-\vec v)$-castable.
The test of $\vec v$-castability is in $O(n)$ (even if is not convex)
The problem (Is there a $\vec v$ s.t. $P$ is $\vec v$-castable?) seems to be linearly reducible to the convex hull, which is in $O(n\log n)$:


First consider each outward normal $\vec{n_i}$ into a point of the unit sphere.
Compute $H$ the convex hull of these points.
If the origin $0$ is in $H$ then for all $\vec v$, $P$ is not $\vec v$-castable.
If the origin $0$ is not in $H$ then let $\vec v$ be the vector starting at the projection of $0$ on $H$ and ending at $0$. The vector $\vec v$ defines a half-space containing none of the $\vec{n_i}$ meaning that $(\vec v, \vec{n_i})&gt;90&#176;$.
If the origin $0$ is on the surface of $H$, just take the normal of $H$ in $0$ for $\vec v$.



not in the convex hull iff there exists $\vec v$ such that 


If $P$ is convex and "open" (whatever it means), then you only need its "frontier" and the corresponding orientation. You apply the same algorithm as above on the frontier (plus the orientation vector) reducing the complexity. For a polygon it becomes in $O(1)$ if you already know the two segments on the frontier.


Hope this helps.