Alter existing form and add element with an ajax "add more" button
Answering my own question after resolving the issue. 
Removing $form_state-&gt;setCached(FALSE); fixed the issue.   

However, I am not sure why it did or whether this have negative consequences. Hope this helps others.