Custom field widget that displays a button with JS action and a hidden textbox
Welcome here. You're question is complicated with many steps some involving the Drupal API and others involving Javascript and Ajax. Your question is specific to your needs, so you'll likely not find a module that does exactly what you're asking for out of the box.

You're correct that the Drupal API should be able to do this -- but by asking people to show you how to do like your 5; is basically asking someone to do your homework for you. I can outline the tools you'll need. It's up to you to come to a working solution.

1st off -- outline your code outside of the Drupal API and simply thing "how would this form work in psuedocode" using say PHP and JS; building those steps will take you a long way.

Drupal Resources to help you:


The Field API Documentation pages.
The Official Field API Tutorial, which links some 3rd party tutorials as well.
If the above 2 resources aren't enough to understand what the Field API does look at the Examples module for Developers.
It sounded like you wanted AJAX in your form. Read up on the Drupal Javascript API Documentation, specifically the Ajax Forms section, and again if that's not enough look at the Examples module, or google "drupal jquery form" as while Drupal has it's own JS API, Drupal bundles jQuery so you could try to just use jQuery to fulfill your needs mostly without the field api explicitly.

what do I mean by this? Here is a jquery autocomplete tutorial which uses a text field "in" drupal but is simply using jquery's native autocomplete functions. I used Google Translate for that page and View Source in my browser to fill in the tutorial gaps -- to re-engineer this on a site I have.



Lastly, either route you'll take you will likely be integrating with an existing Node form. You probably want to use hook_form_alter to inject your custom business logic into the existing Drupal functionality.