Despite the increasing levels of pollution in many tropical African countries, not much is known about the strength and weaknesses of policy and institutional frameworks to tackle pollution and ecological status of rivers and their impacts on the biota. We investigated the ecological status of four large river basins using physicochemical water quality parameters and bioindicators by collecting samples from forest, agriculture, and urban landscapes of the Nile, Omo-Gibe, Tekeze, and Awash River basins in Ethiopia. We also assessed the water policy scenario to evaluate its appropriateness to prevent and control pollution. To investigate the level of understanding and implementation of regulatory frameworks and policies related to water resources, we reviewed the policy documents and conducted in-depth interviews of the stakeholders. Physicochemical and biological data revealed that there is significant water quality deterioration at the impacted sites (agriculture, coffee processing, and urban landscapes) compared to reference sites (forested landscapes) in all four basins. The analysis of legal, policy, and institutional framework showed a lack of cooperation between stakeholders, lack of knowledge of the policy documents, absence of enforcement strategies, unavailability of appropriate working guidelines, and disconnected institutional setup at the grass root level to implement the set strategies as the major problems. In conclusion, river water pollution is a growing challenge and needs urgent action to implement intersectoral collaboration for water resource management that will eventually lead toward integrated watershed management. Revision of policy and increasing the awareness and participation of implementers are vital to improve ecological quality of rivers.