The communities and local government committee said the eight biggest firms built more than half of all new homes.
MPs said the government should do more for smaller builders who do not have the scale to bid for big projects.
But the Home Builders Federation, which represents large and small businesses, said only big firms could spread the risks large-scale projects pose.
The committee also said local authorities should prepare land for home building.
That would include providing the infrastructure needed, such as roads and public transport.
"The housing market is broken, we are simply not building enough homes," said Clive Betts MP, chair of the committee.
"Smaller builders are in decline and the sector is over-reliant on an alarmingly small number of high-volume developers, driven by commercial self-interest and with little incentive to build any quicker.
"If we are to build the homes that the country so desperately needs, for sale and for rent, then this dominance must end."
Government promises more affordable homes
Labour promises to build one million new homes
The committee found that smaller builders struggled to obtained land for development, as local authorities focused on large sites which only big companies could afford to take on.
The Homebuilders Federation said: "We fully support the committee's call for measures to assist smaller builders, encourage new entrants and scale up specialist housing sectors, such as the retirement housing market.
"The vast majority of the big increases in housing supply in recent years have come from the larger, mainstream builders - but we need more builders of all sizes and specialisms if we are to tackle our acute housing shortage."
In February the government promised to build more affordable houses and help people buy and rent, after admitting the current market was broken.
The housing strategy for England included giving councils powers to pressure developers into starting building on land they own.
Ministers also pledged to make renting more "family-friendly" with longer tenancies offered.
However, Labour called the measures announced "feeble beyond belief".
MPs have called for an end to the dominance of big home-building firms to fix the "broken" housing market.