How to configure access to courses for teachers and students? In my site I have teachers and students. The student can subscribe to max 3 courses.

I created cTypes:


Modules (which contains the actual information itself) and  
Course (which contains a reference to the module)


I create rules:


The moment the student is subscribing to a module I create a instance
of Course. I set a field course.status to active. (list integer)
When the student is ending the course the field course.status is going to deactive.


Questions:


Which access module helps me to organize the student to denied access to the module while course.status is deactive?
Which access module helps me to organize teachers to see information only from students who followed or is following his course?


Everything should be organized by itself or  by rules.

I looked into a custom ruleset with the help of vbo's. I looked into Organic groups and I looked into workbench access and I do not know what to choose. How to keep it simple and effective...
You might want to give it a try to use the Group module, which allows for creating arbitrary collections of your content and users on your site, and grant access control permissions on those collections.

It is available as of D7, and has a D8 version also. It creates groups as entities, making them fully fieldable, extensible and exportable. Every group can have users, roles and permissions attached to it (refer to "What are the various roles supported by the Group module?" for more details about that).

Sample configuration

Below is an example of a possible configuration, which may help to get you started with possible configurations (using the Group module):

Group type ID ! Group type ! Group role ID ! Role
--------------+------------+---------------+------------
 1            ! Class      ! 3             ! Instructor
 1            ! Class      ! 4             ! Student
 1            ! Class      ! 5             ! Alumni
 2            ! Customers  ! 6             ! Lead
 2            ! Customers  ! 7             ! Prospect
 2            ! Customers  ! 8             ! Licensed Customer
 2            ! Customers  ! 9             ! Sales
 2            ! Customers  ! 10            ! Support
 2            ! Customers  ! 11            ! Accounting
 2            ! Customers  ! 12            ! R&amp;D
 3            ! Project    ! 13            ! Project Manager
 3            ! Project    ! 14            ! Developer
 3            ! Project    ! 15            ! Customer 


Group Types

The above example consists of 3 group types:


Class.
Customers.
Project.


Group Roles

For each of the 3 group types, there are specific Roles defined, some more details about some of them:


for the Class group, these are the 3 roles that are configured:


Instructor.
Student.
Alumni.

for the Project group, these are the 3 roles that are configured:


Project Manager.
Developer.
Customer.



These roles can be configured per Group type and will only be available on groups of that group type. As an example, for group type "Class", the defined roles are Instructor, Student and Alumni (which are not available for the other group types).

On top of the group specific roles, there are also Special Global Roles: These are the special (hardcoded) roles named Outsider (= a signed in user that did not join a group) and Member  (= a signed in user that did join a group). These cannot be defined by the Drupal administrator, and they are always available for any group. These roles can have different permissions sets for each group type. Special case: the role Anonymous is related to a user that is not signed.

Groups

Each of the configured group types can have any number of Groups.

Examples of groups:


for the Class group type, there could be these Groups:


Getting started with Drupal.
Drupal for site builders.
Become an expert in using the Rules module.

for the Customers group type, there could be these Groups:


Service Requests.
Order Processing.
Sales Orders.
Relationship Management.



Permissions

For each of the configured group types, you can then also configure for each Content Type which Role has what kind of access (= none, view, create, edit, delete). Be aware: even though this looks similar to Drupal's permission configuration, these permissions are not the same permissions (they are specific to the Group module).

Comparison with the Organic groups module

Have a look at my answer to "What are the features of the Group module versus Organic Group module?" for details about:


How both modules (Group versus Organic groups) compare to each other.
Integration with various modules that you might want to take advantage of also.


Resources


Community documentation for the Group module.
Video tutorial "How to use the Group module in Drupal 8 to manage editorial access control to different areas of a web site".


Integration with various modules

The Group module integrates with various other (popular) modules, including:


Rules, as illustrated in the answer to the question "How to bulk assign existing content to a group created with the Group module?".
Views, as illustrated in the answer to the question "How to show the content of a group on the group page when using the Group module?".


The beauty, IMO, of these integrations are that they are more or less out-of-the box (not dozens of extra modules needed to make those integrations work).

Addressing your specific questions


  Which access module helps me to organize the student to denied access to the module while course.status is deactive?


Probably the easiest implementation of this, is to use the Rules module to create an appropriate "Group Membership" entity, which is what it takes to grant a user access to a Group. And assuming you have something in place that allows you to create a view of all students (users) that at some point should receive access to some course, you could use the Views Rules to create such entity for each user in such views result, and thereby using some Rules Event(s) / Condition(s) related to the update of course.status from deactive to something like active.


  Which access module helps me to organize teachers to see information only from students who followed or is following his course?


This is by design of the Group module (just a matter of appropriate configuration of the Group module).

PS: this answer applies to either D7 or D8.