Modal API not displaying ajax response
After some more research, I found the solution in a comment on this article - basically, every article I could find on the subject seems to have been written prior to some API changes being made, in how a dialog or modal is invoked. 

An example link would be:

&lt;a href="/blah" class="use-ajax" data-dialog-type="modal" data-dialog-options="{"width":500}"&gt;Show me a 500px wide modal&lt;/a&gt;


So, the key thing is that data-accepts attribute is replaced with data-dialog-type.