Best way to profile/optimize a website on google's appengine
Question: I'm currently trying to optimize my website, which run on the google's
appengine. It's not an easy task, because I'm not using any powerful tool.

Does anyone have experience in optimizing python code for this purpose? Have
you find a good python profiler?


Answer: I have found [Gprof2Dot](http://code.google.com/p/jrfonseca/wiki/Gprof2Dot)
extremely useful. The output of the profiling modules I've tried as pretty
unintuitive to interpret.

Gprof2Dot turns the cProfile output into a pretty looking graph, with the
slowest chain(?) highlighted, and a bit of information on each function
(function name, percentage of time spend on this function, and number of
calls).

[An example graph
(1429x1896px)](http://jrfonseca.googlecode.com/svn/wiki/gprof2dot.png)

I've not done much with the App Engine, but when profiling non-webapp scripts,
I tend to profile the script that runs all the unittests, which may not be
very accurate to real-world situations

One (better?) method would be to have a script that does a fake WSGI request,
then profile that.

WSGI is really simple protocol, it's basically a function that takes two
arguments, one with request info and the second with a callback function
(which is used for setting headers, among other things). Perhaps something
like the following (which is possible-working pseudo code)...

    
    
    class IndexHandler(webapp.RequestHandler):
        """Your site"""
        def get(self):
            self.response.out.write("hi")
    
    if __name__ == '__main__':
        application = webapp.WSGIApplication([
            ('.*', IndexHandler),
        ], debug=True)
    
        # Start fake-request/profiling bit
        urls = [
            "/",
            "/blog/view/hello",
            "/admin/post/edit/hello",
            "/makeanerror404",
            "/makeanerror500"
        ]
    
        def fake_wsgi_callback(response, headers):
            """Prints heads to stdout"""
            print("\n".join(["%s: %s" % (n, v) for n, v in headers]))
            print("\n")
    
        for request_url in urls:
            html = application({
            'REQUEST_METHOD': 'GET',
            'PATH_INFO': request_url},
            fake_wsgi_callback
            )
            print html
    

Actually, the App Engine documentation explains a better way of profiling your
application:

From <http://code.google.com/appengine/kb/commontasks.html#profiling>:

> To profile your application's performance, first rename your application's
> `main()` function to `real_main()`. Then, add a new main function to your
> application, named `profile_main()` such as the one below:
>  
>  
>     def profile_main():
>         # This is the main function for profiling
>         # We've renamed our original main() above to real_main()
>         import cProfile, pstats
>         prof = cProfile.Profile()
>         prof = prof.runctx("real_main()", globals(), locals())
>         print "<pre>"
>         stats = pstats.Stats(prof)
>         stats.sort_stats("time")  # Or cumulative
>         stats.print_stats(80)  # 80 = how many to print
>         # The rest is optional.
>         # stats.print_callees()
>         # stats.print_callers()
>         print "</pre>"
>  
>
> [...]
>
> To enable the profiling with your application, set `main = profile_main`. To
> run your application as normal, simply set `main = real_main`.

