1. A method for generating data to provide situational awareness or decision-making assistance to a user in relation to a physical environment, the method comprising, with a computer system: processing input data comprising at least data associated with the physical environment; and when a need for situational awareness or decision-making assistance is detected based on the input data, generating response data, the response data derived from multimodal data from a plurality of electronic data streams comprising audio, visual and textual information, the data streams received from a plurality of data sources, wherein generating the response data comprises: determining a characteristic of the need for situational awareness or decision-making assistance; extracting semantic information from the audio, visual and textual information; correlating the extracted semantic information in accordance with the characteristic; selecting a subset of the audio, visual and textual information based on the correlation of the extracted semantic information with the characteristic; and outputting at least a portion of the selected subset as the response data.
1. A method for generating data to provide situational awareness or decision-making assistance to a user in relation to a physical environment, the method comprising, with a computer system: processing input data comprising at least data associated with the physical environment; and when a need for situational awareness or decision-making assistance is detected based on the input data, generating response data, the response data derived from multimodal data from a plurality of electronic data streams comprising audio, visual and textual information, the data streams received from a plurality of data sources, wherein generating the response data comprises: determining a characteristic of the need for situational awareness or decision-making assistance; extracting semantic information from the audio, visual and textual information; correlating the extracted semantic information in accordance with the characteristic; selecting a subset of the audio, visual and textual information based on the correlation of the extracted semantic information with the characteristic; and outputting at least a portion of the selected subset as the response data. 11. The method of claim 1 , comprising: extracting one or more concepts from the audio, visual and textual information, the one or more concepts comprising at least one of objects, actions, scenes and emotions; indexing the extracted concepts; and generating the response data using the indexed concepts.