The Markit/CIPS manufacturing Purchasing Managers' Index fell to 49.2 from 50.7 in March. A reading below 50 indicates falling output.
It is the first time that activity in the sector has fallen since March 2013.
Firms blamed soft domestic demand, a fall in new business from overseas and uncertainty ahead of the EU referendum.
A slowdown in the oil and gas industry, a major customer for UK companies, is also hitting production.
The index for new orders fell to 50.4 in April, from 51.9 the month before, matching February's three-year low.
Rob Dobson, senior economist at Markit, said: "On this evidence manufacturing production is now falling at a quarterly pace of around 1%, and will likely act as a drag on the economy again during the second quarter and putting greater pressure on the service sector to sustain GDP growth.
"The manufacturing labour market is also being impacted, with the data signalling close to 20,000 job losses over the past three months."
Last week, official figures showed UK economic growth slowed to 0.4% in the first quarter of the year from 0.6% in late 2015, propped up by the services sector.
David Noble, group chief executive at the Chartered Institute of Procurement and Supply (CIPS), said: "Recent fears over a stall in the UK's manufacturing sector have now become a reality.
"An atmosphere of deep unease is building throughout the manufacturing supply chain, eating away at new orders, reducing British exports and putting more jobs at risk.
"A sense of apprehension across the sector is being caused by enduring volatility in the oil and gas industry, falling retailer confidence and the uncertainty created by the EU referendum."
The Markit/CIPS survey found new export orders contracted for the fourth straight month in April as the global economy continued to slow. A measure of employment in the manufacturing sector was also below the 50 mark for its fourth straight month.
Lee Hopley, chief economist at the manufacturers' organisation, EEF, said: "The sharp drop to a three-year low and another month of reported job cuts could be the clearest sign yet that referendum uncertainty is starting to weigh on the real economy.
"However, this is just another straw on the back of a sector already grappling with the struggling oil and gas sector, softening domestic demand and weak order outlook from other parts of the world, all of which are failing to provide any counterbalance to the political uncertainty at home."
UK manufacturing activity contracted in April for the first time in three years, a survey has indicated, adding to fears over the economy's strength.