10. A non-transitory computer readable storage medium having instructions for causing a computer to execute a method, comprising: displaying a first page of an application being developed that includes an object; receiving a text instruction in a natural language that semantically describes at least two of a location, type, and allowed action of the object on the first page and provides an instruction to execute on the object; executing the instruction on the object to cause the application to display a second page, and testing primitives in the natural language to verify automation steps produce particular results using a natural language test.
15. The non-transitory computer readable storage medium of claim 10 further comprising, inferring a hierarchy of objects displayed on the first page by implementing one of image segmentation to locate components of a graphical user interface (GUI), object recognition to classify GUI components of interest, and hierarchical analysis using graph grammars to construct a hierarchy with correct semantics.