Is it possible to auto generate and superimpose Frame# over images imported into video edit?
Yes it's possible.

You just have to open the Metadata section in Render Tab of Properties panel, check Stamp output, adjust the size and colors, and check the Seq. strip box (uncheck all other if not needed), and it will print the filename of the current rendered image on the top right corner of the output.