It allows users to control games on the touchscreen devices and can show extra information about progress.
It also lets the handhelds act as controllers for the firm's Internet Explorer web browser, which will launch on the console later this year.
The move poses a challenge to Nintendo's Wii U Game Pad.
While the Japanese company's product is specifically designed to enhance the experience of using its next-generation console, Microsoft is offering users a way to extend the functionality of its existing games machines via third-party devices they may already have.
"Xbox SmartGlass works with all the devices that you already own: the TV you already own, the phone you already own and the tablet you already own," said Marc Whitten, head of Microsoft's Xbox Live division.
"Xbox SmartGlass turns any TV into a smart TV."
A demo showed the highly anticipated first-person shooter Halo 4 using the facility to trigger maps and other data about its in-game environment.
The announcement
 was made in Microsoft's keynote presentation at the E3 video games trade show in Los Angeles.
Mr Whitten added that SmartGlass would also enhance television programmes and films streamed through the Xbox.
He gave the example of a viewer watching Game of Thrones on their television and being able to look down at their tablets to see where on-screen characters were located on a map of the show's fantasy world.
The demonstration also showed how the linked device could bring up details about the cast and characters while the user was watching a film.
"It lets Microsoft say that you don't need to buy new hardware to have a two-screen experience," Stephen Totilo, editor of gaming website Kotaku, told the BBC.
"Nintendo looked to be ahead of everyone else on this, but now Microsoft is breathing down its neck. But there may be a little bit of smoke and mirrors here as I'm not convinced SmartGlass will have as much functionality as the Wii U offers."
The app was also mocked by South Park's creator Trey Parker who was at the event to promote a new video game featuring his characters.
"How many times have you been watching an episode of South Park and thought I'd like to be able to watch this on my television while hooked into my mobile device which is being controlled by my tablet device which is hooked into my oven all while sitting in the refrigerator," he joked.
New uses for Microsoft's Kinect motion and voice sensor also featured heavily in the E3 presentation.
A tie-up with Nike
 will deliver a title towards the end of the year in which a digital personal trainer offers feedback on owners' workout techniques based on information gathered by the Kinect.
Voice commands were shown being used to change team formations in the next edition of the American Football simulation John Madden NFL, and to call in air strikes in the action game Splinter Cell Blacklist.
A new game called Wrecketer also allows players to use gestures to launch and guide missiles to destroy castles in a title based around a similar concept to Angry Birds.
Microsoft also promoted its device as an entertainment hub.
A new service called Xbox Music will offer "over 30 million music tracks" to the Xbox as well as Windows 8 and Windows Phone devices.
A video suggested the service would rival similar offerings from Spotify, Rdio and others. However, details were scant with no launch date or countries mentioned, and no comment about how it would affect the firm's existing Zune product.
US users were promised new content including NBA basketball games, Paramount Pictures movies and Nickelodeon children's programmes.
Much of the rest of the presentation was dedicated to trailers showcasing new titles in existing video game franchises such as Tomb Raider, Fable, Call of Duty and Resident Evil.
"We're probably getting a new Xbox console in 2013 so that will be what the next E3 is about, so they had to spend this year talking about the stuff that precedes it," said Mr Totilo.
"But people who get upset about sequels forget they tend to be some of the best games ever made as developers don't have to worry about working with new hardware, and can instead hone their craft and be creative."
Microsoft has unveiled Xbox SmartGlass: a service to allow tablet computers and smartphones to communicate with its video games consoles.