Function to validate a file name
I'm not sure what you're using this for, but my advice: Don't bother validating the filename.  Just open the file.  The system knows the rules better than you do, and by adding too much validation you can cause problems down the line.

On Windows these rules are especially complex.  Frankly you will never find them all.  Here's a few you missed, off the top of my head:


Filenames may contain unicode characters.  You are using char*.  Unless you're using UTF-8 and passing all your filenames to MultiByteToWideChar before they reach a file API, you're ruling out a lot of potential valid paths based on that.  (Not to mention the way these filenames get mapped to Unicode is now subject to the user's language settings, which one can argue is a bad thing in its own right...)
MAX_PATH.  You're assuming that all paths will fit in a buffer of that size.  On Windows you can actually have a path longer than MAX_PATH if it starts with \\?\; also things like \\?\C:\foo are valid paths.
Names containing trailing spaces or dots, which you seem to be filtering on, are also valid and distinct if the path begins with \\?.  Without that prefix, trailing dots and spaces get truncated.
Paths under \\.\, \\?\GLOBALROOT.


Not to mention, these rules could potentially be subject to change.  Note that Win9X had different rules than NT.  What if such a change occurs again and breaks your logic?  Don't bother duplicating these rules in your code; don't second-guess the filesystem either.  Just pass the name to CreateFile.

(Aside: If your filename happens to come from an untrusted source, then it probably is fair to be overly restrictive...)