How do I get rid of the "Configure Blocks" link above my menus?
I think those links come from the core "Contextual links" module; you should be able to pop to http://mysite.com/admin/modules, disable the module and see the links disappear :)

A bit of extra info on Contextual Links:

http://drupal7ish.blogspot.com/2011/06/contextual-links.html