Blender writes render even when there is no composite node?
As gandalf3 said, I think the best way for now is to delete the files automatically with some python script. Just make sure that the scene output goes to it's own directory, so you don't delete any other files. Run the script and render.

import os
import glob
import bpy

def delete_garbage_files(scene):
    out_path = bpy.path.abspath(scene.render.filepath)
    out_dir, _ = os.path.split(out_path)

    # Remove all files of the currently selected output format.
    ext = scene.render.file_extension
    for f in glob.glob(os.path.join(out_dir, "".join(("*", ext)))):
        os.remove(f)
        print("Deleted: {}".format(f))

    # Also delete the directory.
    try:
        os.rmdir(out_dir)
        print("Deleted: {}".format(out_dir))
    except OSError:
        print("Didn't delete directory {}, probably not empty".format(out_dir))

# Post render handler, to delete the files after every rendered frame.
bpy.app.handlers.render_post.append(delete_garbage_files)


To delete the files after the whole render job finishes, you can use render_complete instead of render_post But if you cancel a render, the files won't be deleted.

# Complete render handler, to delete the files when the whole render job
# is finished.
bpy.app.handlers.render_complete.append(delete_garbage_files)


Probably the best solution is to use a render_cancel and render_complete handler. Then the files will only be deleted when the render is finished or canceled and not after every rendered frame.

# Use both cancel and complete render handlers.
bpy.app.handlers.render_cancel.append(delete_garbage_files)
bpy.app.handlers.render_complete.append(delete_garbage_files)


If you want to automatically delete the 'garbage' files the next time you open the file and render, you should have the name of the script end in '.py' and check the 'register' option: