How do I delete a file from my Xoom?
You'll need a file manager, and just browse to the same path that you do when you attach to email.

If the Xoom doesn't have a file manager, I recommend Root Explorer.  If you don't have root you can try another, like Astro File Manager.