Student caught cheating when leaving class after handing me the exam
Exact rules will depend on where you are.  But normally the student has due process rights: to be confronted with the evidence of her cheating, and to appeal to some higher administrator or body if she desires.    Two of your options (grade more harshly / secretly lower grade) are things that you would do unilaterally, without the student's knowledge, and without giving her the right to challenge.  In my view, these options are therefore totally inappropriate and unethical.

The evidence against her is your testimony about what you observed.  If you think this would be sufficient evidence under your university's rules, then it is appropriate to pursue formal punishment as the rules dictate.  If you don't think it's sufficient, then do nothing, and let the student complete the rest of the course without prejudice. (Though of course you can watch her more closely in the future, and perhaps institute procedures to deter this kind of behavior in general.)