How much maximum force can be applied by fingers on the brake lever?
If your question is about the maximum hand grip can applied to brake so that the braking system will not be broken apart, then it is as another answer.

However it has little value on designing a braking system.

How I interpret the question is: how much force is typically applied to the brake lever to stop the bike. 

There are many designing considerations and I only list a few main one:


Maximum force on to lever that a person feels comfortable at gripping for a long duration (for me it's about 5-7 kg or 50-70N)
Modulation how well the spread between the the force at which the bike starts to decelerate, and the Maximum force. This depends on the braking technology, the pad/rim surface/rotor. But in general, disc brake is superior because of the (more) linearity in braking. 
Linearity: how linear the stopping power versus the force applied by brakes? This is interesting and thus somehow explains why road bike is equipped with caliper, hybrid with cantilever, and mountain bike with disc brake. Linearity improves as we go from caliper-cantilever-disc. The discrepancy is large when higher stopping power is required. But at smaller (required) stopping power, a caliper is not that much different front disc brake for a given braking range.
Maximum stopping power This is safety feature, but it should not be easy to flip the bike with too little gripping onto the brake lever. This has to take into account the bicycle geometry, and that is why road bike is normally equipped with caliper brake. Even some (supposed-to-be) road bike has disc brake, the rotor/leverage/cylinder was designed so that it has much less stopping power than a mountain bike disc brake.


Well, it depends on type of brake and how fast you would like to stop the bike, at what speed, and at what momentum.

If you just want a number to work with (for your home work?), I can give you.
A normal cable brake on flat handle bar would need about 40-50 N (3-4 kg) for hard stop, about 20-40N (1-2 kg) for normal deceleration.

Same stopping power with drop handlebar and caliper brake would need about 1.5 to 2 times as much (comparing Figure 1 and 2), depends on hand position (leverage). Typically hands at hood position requires more force for same stopping power in comparison to hands at drop position.

Same stopping power with hydraulic disc brake needs about 0.5-0.7 fold. This depends on the rotor size/pattern and type of ceramic and thus it is hard to quantify. But the general still holds, if you are designing a disc brake that has an enormous stopping power for normal stopping, you actually degrades the modulation ability of said brake.



Figure 1a. Normal barking force on drop position of drop handlebar


Figure 1b. Braking force required to lever on emergency stop on drop position of drop handlebar



Figure 2a Braking force required onto lever in cantilever brake for normal stop


Figure 2b Braking force required onto lever in cantilever brake for hard stop