How do I activate Google Custom Search?
As you can read in README.txt file:

*** About Google Site Search module
---------------------------------------
This requires "Site Search Edition" (paid version of Google's Custom Search).


You need to have a paid Google Site Search engine, so you can receive results in custom format (e.g. XML).

If you've free version, check the alternatives modules such as: Google Search Appliance or Google Custom Search Engine. Or use alternative services such as Yahoo YQL (e.g. YQL Views).