Urban environments are increasingly pervaded by embedded networks and mobile digital components capable of sensing, computing, and acting in real-time. These augmented environments have the capability to dynamically respond to and adapt to their context and to behavioral patterns of their human occupants. In doing so their forms of interaction can be fluid and ad-hoc and this paper proposes a framework based on improvisation for the analysis and the design of such environments. Following a study of the nature of the improvisation process, a systems view of improvisation is adopted. The structural elements of the Viewpoints improvisation technique are then mapped to design stages of second-order systems to develop a framework for the design of interactive urban environments.