Can you use basic FAPI without drupal_build_form?
From https://api.drupal.org/api/drupal/includes%21form.inc/function/drupal_get_form/7.x :


  This function should be used instead of drupal_build_form() when $form_state is not needed (i.e., when initially rendering the form) and is often used as a menu callback.


It still does a drupal_build_form under the hood, though, but at least you don't have to do it. Mind you, it may or may not work in your specific case, depending on where / how you create and call your form.