Checking if file is within a directory for security
Nit-Picks

This exception-handling code should include the IOException as part of the SecurityException re-throw:

try {
    tested = new File(file).getCanonicalFile();
} catch (IOException e1) {
    // set the cause of the SecurityException so that users can potentially fix things
    throw new SecurityException(
            "The basedir resolution failed to resolve.", e1);
}


General

I believe the rest of the code supplies the required functionality. There are odd cases where, for example, on Linux you can remount a sub-directory at a different mount point. These two distinct file-systems may mirror each other, and a file/directory in one mount-point is identical to the other mount point.

This may produce false-positives in your code (where your code will claim the directory is not allowed, but it actually is...). False-positives in 'edge cases' in a security situation are better than false-negatives.

The intricacies of the new-in-Java-7 NIO2 Path class/interface may help with this. You could translate your File instances in to Path instances, and then use the isSameFile(...) method... which may, or may not help (reading the documentation it is unclear, and I do not have direct experience with that).

Update .... more Paths:

Using the Path and other NIO2 features, your code could probably be reduced to:

// basePath is already resolved as a Path.realPath from basedir
// classPath is already resolved as a Path.realPath from classDir
Path filePath = file.topath().realpath();
if (!filePath.startsWith(basePath) || !filePath.startsWith(classPath) {
    // illegal file access....
    throw new SecurityException(.....);
}