Form storage is cleared after ajax submission
You say this is about building the form. If you display a form on a page, then this form is rendered, but the form object is not stored yet. This happens later when the form is submitted, then you can use $form_state.

Generally when you build a form this is the same like rendering of non form content. Put the initial values in HTML (in the form elements). Similar to default values for visible fields use hidden fields for other values. You also have to add the correct cache metadata to the form so that the render cache varies by the data and is invalidated when the data is modified.

If you don't want to transport data in a rendered form then use a session or tempstore. This kind of storage is not restricted to form requests, so strictly not connected to this issue, but for logged in users these are easily available and for anonymous you can start a session, which you need to do anyway if you want to personalize a form.