How do I grant Read Access to FILESTREAM data in a table but not grant SELECT access?
There's no way to restrict SELECT access to a set number of rows.

If the users don't need access to the entire table, but just their "own" rows, you could configure row level security (SQL 2016 and up) to restrict access to only what they need. This will not prevent the user from performing a SELECT *, but it will restrict the volume of data returned if they do run one.

Ultimately, this is the kind of situation which would be best solved by not allowing users to have any direct access to the table, and instead have the data returned via app code which can be controlled to not run a SELECT *.