Safe Cast in Crystal Reports Comparison
Thanks to the comment by beccasue, I found the solution:

Using a Nested If with IsNumeric and ToNumber worked.

Example of nested Crystal Reports if statement copied from a SO question:
Nested if else in Crystal Reports


if {table1.id} &lt;&gt; 1 then
   (if {table1.name} &lt;&gt; "a" then
      var1 := "Hello"
   else
      var1 := "Hi";)
else
   var1 := "Bye";