Is there any way to adjust the items per block options? In Drupal 8, whenever you create a view block, you have the option to specify the number of items displayed in that view block which is awesome.

Currently, you can choose between 1, 5, 10, and 20. Is there any way to change this to include 1, 2, 3, 4, 5" or just be able to manually enter a number?

I was really excited about this feature and need to use it, but I need to really choose between 1, 2 or 3 items per block.

Edit: I'm referring to the block management screen, not the views configuration screen.
Thanks to the answer @Berdir provided, I've been able to replace the restrictive select field with a more flexible number field.
In mymodule.module
The form_id is block_form (andpanels_edit_block_form and panels_add_block_form for Panelizer). Change the #type to number to make it so that you can input a number. Unset the #options to avoid validation errors (Invalid choice).
/**
 * Implements hook_form_alter.
 *
 * Changes the input type of the &quot;Items per block&quot; setting of
 * a view-block in Blocks/Panelizer from 'select' to 'number',
 * so that amounts other than 5, 10, 20 and 40 can be chosen.
 */
function MYMODULE_form_alter(&amp;$form, &amp;$form_state, $form_id) {
  if (in_array($form_id, ['block_form', 'panels_edit_block_form', 'panels_add_block_form'])) {
    if (isset($form['settings']['override']['items_per_page'])) {
      $form['settings']['override']['items_per_page']['#type'] = 'number';
      unset($form['settings']['override']['items_per_page']['#options']);
    }
  }
}