This paper presents an approach to designing a file manager for speech interfaces. Speech interfaces are those which primarily use speech recognition and synthesis for interacting with the user for input and output. The present system used in various graphical user interface (GUI) operating systems for file managers is based on the 'What you see is what you get' model providing a 'point and click' approach to support a hierarchical folder structure. While the 'point and click' is suitable for GUI, it becomes inefficient for speech interfaces. The speech interfaces implemented currently in systems like Microsoft speech recognition navigate through hierarchical file structure, having to navigate sequentially through various sub-folders and necessitates the need for a display to view the folders and files in the directory being traversed. In our approach the user is freed from the responsibility of traversing through the folders and provides a method to retrieve files based on the various file attributes and relevance. The paper focus on the design and architecture of the file manager. The paper also analysis the results based on the usability survey of the system and presents a comparison with existing file managers. The advantage of using this system is that it is designed specifically for speech interfaces and allows the user to use the system with minimal or no displays.