What approaches are there to not display a search result that a user has no permission to see? Question

What approaches are there to not display a search result that a user is not supposed to see?

Example

Suppose we have the following situation.

High-level view

In an enterprise, there are different repositories like websites, databases, filesystem folder etc. And there is a search engine that crawls all those repositories and creates an index. The user can then use the search engine UI to perform searches.

Let's say the search engine was Apache Lucene and the encompassing indexing and retrieval system Apache Solr.

User permissions are managed via Microsoft Active Directory.

The content in the different repositories are managed with all kinds of applications like MS Word, custom-made applications, database management tools etc.

Search-index and user permissions


The index contains the documents A1, A2, B1 and B2.
User A has only permission to see documents of type A*.


Performing a search


Now, user A performs a search for A* AND B*.


Displaying the results

A system administrator with permission to see everything should see all four documents in the search results: A1, A2, B1 and B2.

User A however, who only has permissions to see documents of type A*, should/will only see two documents in the search results: A1 and A2. So user A shouldn't even know that documents B1 and B2 exist.

Elaboration on the question

What ways are there to implement that requirement that users only see documents they are allowed to see?

I suppose it would be a bad idea to add the information to the index whether a user may see a document. I think that because of this use case: suppose all of a sudden user A may only see document A1 and not A2 anymore; those permissions are now set for example on the filesystem. Now it takes a while until the permissions in the index are updated and during that time user A would still be able to see document A2 in the search results even if he can't click on the result and access it anymore.

Also, the above approach probably would break down when there are hundreds of millions of documents and thousands of employees.

Where and how would the security aspect be implemented in a content indexing and retrieval system in the above scenario?
Since you aren't sure which software you would like to use, I can explain this concept only in general, by example. The clearest example is a content management system like Drupal or Wordpress (not necessarily a data science tool but the idea is the same).

In a content management system, there are different types of content which can be accessed. In your case, there would be two types - document type A and document type B. There can be an unlimited number of documents, each assigned one or more types. 

The software has a permission system that gives access to each document type. This is built with roles. In your example, there would be a system administrator role, and a general user role.

Each role has permissions associated with it. You would configure system administrator to be allowed to view document types A and B, but you would configure general users to only be able to see document type A.

Each user registers an account in the software system. A site administrator assigns each account one or more roles. So a user with the system administrator role would be able to see all documents of type A and B. However, if a user only has the general user role, they would only be able to see documents of type A. Also noteworthy is the fact that each user can have multiple roles and will get all permissions associated with every role they have.

USER -has-> ROLE(s)

ROLE -has-> PERMISSION(s)

CONTENT TYPE -requires-> PERMISSION(s)

Every time the content is requested, the software system ensures that the user making the request has the permission to access the content.

When a search is run, the software system has to validate every potential result individually before displaying the search results to the user. The user must have the permission to see each piece of content that is be delivered back from the search.