How to restrict content access within a site? In a community site. I want to restrict users to have some private parts of the site like forum, groups and discussion board with their scope (should not be visible to others).

That is a specific forum should be available to a specific set of users who belongs to a specific group.

Eg: Let's say there are multiple groups of users like cricket players,  football players. I want to restrict the visibility and access to cricket forum only to cricket players and Football forum respectively. And there is a third forum called sports which should be visible to both.

The question can split into two:


How to create a specific set of users?
How can I restrict the specific set of users to the forums, groups, etc...
1) How to create a specific set of users?


That's what you typically do by creating appropriate "roles" in your site, and then for each user, you assign appropriate "roles".


  2) How can I restrict the specific set of users to the forums, groups, etc...


Combine this with granting "Permissions" to "roles" (which are then inherited by all users with the appropriate role). E.g. you grant access to (selected) forums.

When it comes to "groups", there are typically these modules that get used for that:


The most commonly used is Organic Groups, for which there is a set of amazing video tutorials to Learn Organic Groups. Especially the video about "Basic Organic group setup" should help to get you going with it. Here is what you'll learn from it (quote from that link):


  
  How to create a content type that is used as groups – meaning containers for other stuff (content and other entities)
  How to modify a content type (Articles) to allow it to be posted to selected groups
  That the user creating a group by default will be the group manager
  How to add more members to a group
  That there is an example feature shipped with OG
  That groupness and group contentness is actually managed by special fields
  That group membership is actually also managed with a field (of the same type that tells what group a content piece belogs to)
  

For anybody who considers the Organic Groups module as rather complex (like me ...), have a look at the (fairly new) Group module as a possible alternative. Some more details about it (from its project page):


  Organic Groups allows content itself to be groups, which isn't always what people want. It relies on an entity reference field to keep track of the ties between a group (node, term, ...) and its content (node, term, user, ...).
  
  Groups instead creates groups as entities, making them fully fieldable, extensible and exportable. Every group can have users, roles and permissions attached to it. Groups can also act as a parent of any type of entity. Seeing as a group itself is also an entity, creating subgroups is very easy.


Even though it only has a beta release for D7 so far, its Usage statistics seem to indicate it is like a "rising star". And I've heard it mentioned recently in various occasions as a valid alternative for the "heavy" Organic groups module.

The Group module also integrate nicely with the Rules module, as explained in comment #2 of issue 2603136, which states the following:


  ... you can already use Rules to:
  
  
  Add a new Group
  Create a new GroupMembership and save it (the equivalent of $group->addMember())
  Add or remove a GroupRole from a GroupType
  React on new GroupMembership or Group entities
  …
  
  
  What hasn't been done yet is custom Rules actions or conditions. Seeing as 90%+ of Group is pure Entity API CRUD-operations, there has been no custom Rules code yet given how much you can already accomplish out-of-the-box.
  
  Useful Rules we may add could be:
  
  
  E-mail all members of a Group, optionally filtered by GroupRole
  Easier-to-understand labels for the above list: "Member joined group" sounds easier than "GroupMembership entity is created"
  
  
  ...


Refer to Integrations with other modules and its "Related issues" about other modules for which integrations already exist, or are in the pipeline.


Note: 
It seems to me that "a particular school" (as in your comments below) would be a perfect with a "particular group" (no matter which of the 2 modules above you would use).