1. A method implemented in a data processing apparatus, the method comprising: receiving, at a data processing apparatus including one or more hardware processors, a query from a user device, the query including a plurality of terms input by a user, the user device displaying plurality of active non-textual data elements in an active window of an application environment when the user input the query; determining, by the data processing apparatus, that the query is an ambiguous query and that the user device is displaying the active non-textual data elements that were displayed when the query was input and in response: identifying, by the data processing apparatus, the active non-textual data displayed in the application environment on the user device; automatically determining, by the data processing apparatus, for each of the active non-textual data elements, modification data from the active non-textual data element for the query; automatically generating, by the data processing apparatus, for each modification data for each active non-textual data element, a set of modified queries based on the query and the modification data; automatically scoring, by the data processing apparatus, the modified queries according to one or more scoring criteria; automatically selecting, by the data processing apparatus, one of the modified queries based on the scoring; and providing, by the data processing apparatus, to the user device, search results responsive to the selected modified query.