After his playing career was over, O'Brien jumped into coaching in 1977 as an assistant at the University of Connecticut under Dom Perno. He spent five years there, during which time the Huskies were 91-50 and made three post season appearances. St. Bonaventure University hired O'Brien in 1982. He stayed four years as head coach before returning to his alma mater in 1986, succeeding Gary Williams who had been named head coach at Ohio State University. He was Atlantic 10 Conference Co-Coach of the Year in 1983, when he led St. Bonaventure to a 20-10 record and a berth in the National Invitation Tournament. Basketball Times named him as its National Rookie Coach of the Year in 1983. Head coach at Boston College O'Brien coached for eleven seasons at Boston College. This tenure included NCAA tournament appearances in three of his last four seasons and postseason appearances in five of his last six years. O'Brien was named Big East Co-Coach of the Year honor in 1995–96, Kodak District I Coach of the Year in both 1992 and 1993, and Eastern Basketball Coach of the Year in 1994. In 1994, O'Brien led Boston College to the Elite Eight in the NCAA tournament and scored back-to-back wins over Indiana and North Carolina. Academics were a strong priority throughout O'Brien's coaching career – in his first 10 years at Boston College, all 25 players who completed their eligibility also earned their degrees. During his final season at Boston College, O'Brien led the team to the 1997 Big East regular-season and tournament titles. However, O'Brien began running into problems getting his recruits past the admissions office after star recruit Chris Herren was kicked out of school in 1994 for drug and discipline problems; he eventually transferred to Fresno State. During 1995-96, at least three prospective recruits were denied admission despite easily meeting NCAA standards. When two Boston-area recruits were turned down in November 1996, O'Brien nearly resigned. He ultimately left for Ohio State after the 1996–97 season and took nearly all of his players with him, including star freshman point guard Scoonie Penn. O'Brien subsequently sued Boston College for slander and breach of contract, suggesting racial bias may have played a role in why one of his recruits was denied admission. Head coach at Ohio State With Penn and most of the new arrivals forced to sit out the season, O'Brien's first year in