Unit tests for directory-entry counter
Why do you create named temporary files within your named temporary directory? The whole directory (including its contents) will be deleted as soon as the context ends.

So, just create a fixed (or random if you want to) number of zero byte files in that directory:

import os
import tempfile

...

with tempfile.TemporaryDirectory() as tmpdirname:
    print('created temporary directory\n', tmpdirname)
    n = 34 # create 34 files
    for i in range(n):
        open(os.path.join(tmpdirname, str(i)), "w").close() # create the empty file
    print('created', n, 'files')
    self.assertEqual(n, f.count_files(tmpdirname))