Selecting from a column in a joined table in Views
I solved my own issue.

I needed to expose the joined table to views as well. So on top of setting up $data['students'], I also needed to set up $data['student_names'], with a join back to the original table. Here is my hook_views_data() in its entirity:

$data['student_names']['table'] = [
  'group' =&gt; t('Student'),
  'base' =&gt; [
    'field' =&gt; 'sid',
    'title' =&gt; t('Student'),
  ],
  'join' =&gt; [
    'students' =&gt; [
      'left_field' =&gt; 'sid',
      'field' =&gt; 'sid',
      'type' =&gt; 'inner',
    ],
  ],
];

$data['student_names']['given_name'] = [
  'title' =&gt; t('Given name'),
  'help' =&gt; t('The given (first) name of the student'),
  'field' =&gt; [
    'handler' =&gt; 'views_handler_field',
    'click sortable' =&gt; TRUE,
  ],
];

return $data;