Why do ASC and DESC come *after* a CASE clause?
Why do ASC and DESC come after a CASE clause?  
Why doesn't the CASE statement contain the ASC or DESC (if you try to do that you get a syntax error)?


Because a CASE expression (it's neither a clause nor a statement) can have a value/expression as a result, not code words like ASC or FROM. The syntax provided in CASE expression documentation is:

Searched CASE expression:
CASE
     WHEN Boolean_expression 
       THEN result_expression [ ...n ] 
     [ ELSE else_result_expression ] 
END



It might resolve to:

ROW_NUMBER() OVER (ORDER BY
    DESC,
    AggregateId ASC
) AS RowNumber



No, it cannot. Due to the implied ELSE NULL part that all CASE expressions have as a default, if you don't provide it.