Why the brain region with less activity is not more responsible for some action? From my naive understanding, it seems that when people try to find correlations between a human action and a certain pattern of activity in the brain, they measure the brain activity with some device  designed for that while a group of people perform that action. It seems that it is usually deduced that the brain region that shows more activity is more responsible for the action under investigation. 

What is the proof that the relation is not inversed? That is: Why the region with less activity is not actually more responsible for the action under investigation?
The thing neuroscientists do is look at the differences in brain activity. They thus don't see "a high activity in brain area X". Instead, they see, between condition A and B, a difference in activity in brain area X. These differences can be both positive and negative, and each may receive it's own explanation. 

There must thus always be some kind of reference. This references can be done in different ways:


Brain activity during a task is compared to some resting state activity. Before or after the experiment the subject is asked to sit still, often look at a fixation cross and do nothing. The assumption is that, while doing nothing, the brain is in some sort of neutral state. Every difference in activity you then find, could be attributed to the brain processes necessary to perform the experimental task. 
A control task is performed. In a study to emotional responses (Moser et al., 2006), for example, they show pictures. Some pictures contain emotion eliciting content, other pictures are perceived as emotional neutral. The reason a control task would be performed is to eliminate unknown variance in the data. If you are interested in an emotional response to pictures, and would compare that to a resting activity, as described above, you do not know for sure whether the difference in brain activity is caused by an emotional response, or by the fact that you presented a picture. By presenting emotion-full pictures and comparing brain activity with activity during presenting neutral pictures, the only difference is the emotional content in the picture. Not the type of task. 
You could perform one task and make a distinction based on behavioral results you obtained. In the following fMRI study (Drummond et al., 2005), subjects performed a psychomotor vigilance task (a simple reaction time (RT) task). The long RT-trials were seperated from the short RT-trials, and then brain activity is compared. This way, everything is the same in the experiment, but the behavior of the subjects. 




References

Drummond, S., Bischoff-Grethe, A., Dinges, D. F., Ayalon, L., Mednick, S. C., &amp; Meloy, M. J. (2005). The neural basis of the psychomotor vigilance task. Sleep, 28(9), 1059-1068.

Moser, J. S., Hajcak, G., Bukay, E., &amp; Simons, R. F. (2006). Intentional modulation of emotional responding to unpleasant pictures: an ERP study. Psychophysiology, 43(3), 292-296.