How to proceed when you are coordinating a project among several research groups and they do not contribute to the progress of the project
The path to proceed likely depends strongly on four factors:


Does the project have clear and specific milestones in the contract that the collaborators are failing to meet?
Are the others subcontractors to your institution or independently contracted?
What type of oversight is being exerted by the funding agency, and how is the relationship with the responsible program manager?
Are you willing to burn a bridge?


Typically, as lead PI you are responsible for the execution of the entire project, not just your portion, and therefore have a responsibility to your funding agency to appropriately manage your subcontracting collaborators.  If the funding agency has instead decided to execute independent contracts to each of the PIs in the grant, then it is largely out of your hands and there is little that you can (or should) do.

Assuming you are lead PI, then if the contract (not the proposal) has requirements in its statement of work that they are not meeting by the given milestone deadlines, then you have much more leverage and a clear case for a "shape up or get defunded" conversation.  Similarly, if the funding agency is exerting strong oversight (e.g., in the USA, DARPA often threatens de-funding unless you meet strong milestones by particular frequent deadlines or regularly outperform other research teams; NSF has a much looser oversight process that often just wants to see that something interesting is being accomplished) then you have a strong case that their failure to perform is directly imperiling the whole research project.  In either case, as lead PI you have both the right and the responsibility to cancel their funding and redirect it to better accomplish the research tasks that you are being held accountable for.  

If the contract doesn't have accountable goals and you aren't being held accountable for their behavior, however, then it is more just a moral offense to you, and you have much less basis on which to legitimately defund your colleagues.  Instead, you might look at it as a lesson for the future on writing more specific contracts that can give you the leverage you need to deal with non-performers.

Now if you do defund your colleagues, this has a high likelihood of burning bridges.  If you didn't trust them to shape up, why would you want to work with them again?  If they feel you defunded them unfairly, why would they want to work with you again?
If the mandate comes from the funding agency program manager rather than you, that may make the defunding somewhat more acceptable by making it "not your fault," but your colleagues may still see it that way.  Thus, if the relationship is more important than the money being lost, you may then want to just write this off as a lesson learned and plan more carefully for the future.

In short: there are no easy paths, but which one to walk depends on the balance of relationship value vs. accountability in your contract with your funder and your colleagues.