1. A computer-implemented method, comprising: determining an object in a graphical user interface (GUI) rendered by a remote server and displayed on a client device based on display information transmitted from the remote server to the client device over a network, wherein the object is manipulated by the server in response to one or more input events, wherein the object is a three dimensional object that is animated by the remote server; selecting a portion of the GUI displayed on the client device that includes at least some background around the object, wherein the portion of the GUI that includes at least some background of the object is selected based on dimensions of the object at various view angles; sampling colors of a set of points in the selected portion of GUI by the client device; determining whether the object has been updated by a frame update based at least in part on the sampled colors of the set of points in the selected portion of the GUI.