of the algorithm: Q(s, a) ←− r + maxaQ(s', a') Given a state s and action a, r and s are the reward and state after performing (s,a), and a' is the range over all the actions. Crowd rendering and animation Rendering and animating a large number of agents realistically, especially in real time, is challenging. To reduce the complexity of 3D rendering of large-scale crowds, techniques like culling (discarding unimportant objects), impostors (image-based rendering) and decreasing levels of detail have been used. Variations in appearance, body shape and size, accessories and behavior (social or cultural) exist in real crowds, and lack of variety affects the realism of visual simulations. Existing systems can create virtual crowds with varying texture, color, size, shape and animation. Real world applications Virtual cinematography Crowd simulations have been used widely across films as a cost-effective and realistic alternative from hiring actors and capturing shots that would otherwise be unrealistic. A significant example of its use lies in The Lord of the Rings (film series). One of the most glaring problems for the production team in the initial stages were large-scale battles, as the author of the novels, J. R. R. Tolkien, envisioned them to have at least 50,000 participants. Such a number was unrealistic had they decided to only attempt to hire real actors and actresses. Instead they decided to use CG to simulate these scenes through the use of the Multiple Agent Simulation System in a Virtual Environment, otherwise known as MASSIVE. The Human Logic Engine based Maya plugin for crowd simulation, Miarmy, was used for the development of these sequences. The software allowed the filmmakers to provide each character model an agent based A.I. that could utilize a library of 350 animations. Based on sight, hearing, and touch parameters generated from the simulation, agents would react uniquely to each situation. Thus each simulation of the scene was unpredictable. The final product clearly displayed the advantages to using crowd simulation software. Urban planning The development of crowd simulation software has become a modern and useful tool in designing urban environments. Whereas the traditional method of urban planning relies on maps and abstract sketches, a digital simulation is more capable of conveying both form and intent of design from architect to pedestrian. For example, street signs and traffic lights are localized visual cues that influence pedestrians to move and behave accordingly. Following this logic, a