Why I can't parse factorial?
I have no comment on the precedence of the factorial operator; my inclination is to consider it to have higher precedence than other operators because of the almost universal rule that postfix operators bind more tightly than other operators.

When implementing that, you need to create a pseudotoken to modify the default precedence of the prefix operator:

%precedence PREFIX_OP
%right      '^'
%precedence '!'
%%
expr : '!' expr %prec PREFIX_OP
     | expr '^' expr
     | expr '!'
     | '(' expr ')'
     | T_NUMBER


(Using actual character constants in your grammar really makes it more readable, and it also simplifies writing the lexer. So I did that.)

To understand this, remember that precedence relations always compare the precedence of a reduction with the precedence of a shift. These are two distinct things; they only seem to be the same because (f)lex sets the precedence of a reduction (i.e. the right-hand side of a rule) to be equal to the precedence of the last non-terminal in the rule (unless it is specified explicitly with %prec). That's often the "right thing" but it leads to a lot of confusion because it makes it appear that the precedence comparison is between two objects of the same type.

In the resolution of !3!, the parser must decide what to do after seeing !3. The only thing it can do with the 3 (a T_NUMBER) is reduce it to an expr; there is no ambiguity and thus no need to consult precedence relations. (Consequently, %nonassoc T_NUMBER has absolutely no effect, which is which I removed it from the grammar.) It then has two options:


reduce '!' expr to expr
shift the second !


To decide, it precedence-compares the reduction expr: '!' expr with the terminal !. Since the precedence of the reduction was explicitly set with %prec PREFIX_OP this results in a comparison between reduction PREFIX_OP and shift !; the shift wins. The same would happen with 2^3! because the comparison will involve the reduction expr: expr '^' expr whose last terminal is '^' with the shift !.

In your original grammar, the precedence declaration 

expr: expr '!' %prec T_FAC


had no effect whatsoever, because that reduction is never ambiguous (3 ! 2 is a syntax error; no grammar rule applies to it.) Furthermore, the ambiguity between the reduction expr '^' expr and the shift !  was incorrectly resolved because the terminal was declared as having lower precedence than the reduction. Just changing the order of the precedence list wouldn't have the desired result because that would also change the precedence of the reduction expr: '!' expr.