Classifying Java exceptions
First of all, some basics of classification (and in general any supervised ML tasks), just to make sure we have same set of concepts in mind. 

Any supervised ML algorithm consists of at least 2 components: 


Dataset to train and test on.
Algorithm(s) to handle these data.


Training dataset consists of a set of pairs (x, y), where x is a vector of features and y is predicted variable. Predicted variable is just what you want to know, i.e. in your case it is exception type. Features are more tricky. You cannot just throw raw text into an algorithm, you need to extract meaningful parts of it and organize them as feature vectors first. You've already mentioned a couple of useful features - exception class name (e.g. com.acme.PrintException) and contained words ("Timeout"). All you need is to translate your row exceptions (and human-categorized exception types) into suitable dataset, e.g.: 

ex_class                  contains_timeout  ...   | ex_type
-----------------------------------------------------------
[com.acme.PrintException, 1                , ...] | Availability
[java.lang.Exception    , 0                , ...] | Network
 ...


This representation is already much better for ML algorithms. But which one to take? 

Taking into account nature of the task and your current approach natural choice is to use decision trees. This class of algorithms will compute optimal decision criteria for all your exception types and print out resulting tree. This is especially useful, because you will have possibility to manually inspect how decision is made and see how much it corresponds to your manually-crafted rules. 

There's, however, possibility that some exceptions with exactly the same features will belong to different exception types. In this case probabilistic approach may work well. Despite its name, Naive Bayes classifier works pretty well in most cases. There's one issue with NB and our dataset representation, though: dataset contains categorical variables, and Naive Bayes can work with numerical attributes only*. Standard way to overcome this problem is to use dummy variables. In short, dummy variables are binary variables that simply indicate whether specific category presents or not. For example, single variable ex_class with values {com.acme.PrintException, java.lang.Exception, ...}, etc. may be split into several variables ex_class_printexception, ex_class_exception, etc. with values {0, 1}:

ex_class_printexception  ex_class_exception  contains_timeout | ex_type
-----------------------------------------------------------------------
[1,                    , 0                 , 1              ] | Availability
[0,                    , 1                 , 0              ] | Network


One last algorithm to try is Support Vector Machines (SVM). It neither provides helpful visualisation, nor is probabilistic, but often gives superior results. 



* - in fact, neither Bayes theorem, nor Naive Bayes itself state anything about variable type, but most software packages that come to mind rely on numerical features.