Video Compression - When would an average of the previous and next I or P frame be useful
Before elaborating anything, i would urge you to go through my answer here that simplifies some of the common confusions prevailing about motion vectors and estimations.



Your essential question is - when or how does encoder decides to put B frame or bidirectional motion vectors. I will break your questions into few as follows: 

1. When does encoder decides to use B frames?

Almost always the number of B frames between I or P is always fixed. This is usually 2 or 3 pictures. The distance between I frame (called GoP size) may keep varying, but number of B frames will always remain fixed. From the point of view of MPEG, this is not really a restriction, but as one of your other question discusses, if you keep changing B frames, there will be serious problems of rhythms between the transmission and receptions. 

Also, even before estimating whether a given frame will be a good choice to act as a B frame or a new I or P frame, you have to actually decide the forward reference and past reference for from where the prediction. Iterating over whether given frame will be perfect to work as P against frame already gone or should you choose some distance frame as P and current as B is too much overhead to get encoding done, and in the process you are fluctuating delay heavily. 

So in essence, typically encoders decides number of B frames as 1,2 or maximum 3 and corresponding pictures are always fixed to be B frames. 

2. So then what if i have a huge black/uncorrelated picture between last I and future P?

Ok - now that you have chosen to see that given frame is B, but almost nothing matches of that frame between either past or future; or sometimes only either side of the frame? This doesn't really put encoder in trouble though. 

In MPEG, every frame is divided in terms of macro blocks. Each macroblcok has motion prediction from forward or reference blocks from respective past or future pictures. 


I frame can ONLY have Intra blocks - which has no motion compensation, so it doesn't need any other frame ready for decoding itself. 
P frame can have INTRA as well as FORWARD-PREDICTED macro-blocks. So even in case of P frame, when current block doesn't match with anyone in the past frame, P frame can simply use INTRA block. 
B frame allows all type of macroblock including INTRA (as in I), FORWARD-PREDICTED mb (where motion vector depends only on past picture) , BACKWARD-PREDICTED mb (where motion vector depends only on future picture) and BIDIRECTIONAL macroblock which has two motion vectors from past and future and they are averaged out to form current block.


Hence, in case, the B picture or part of it happens to be very independent it is quite possible to put INTRA blocks for those MBs, and if it matches only either side of the picture encoder can use only FORWARD or BACKWARD prediction either way. 

3. So does all MB's of a given frame have to be of same type?  

Not at all, suppose a new small object just entered in a frame which is encoded as B frames, only part of the frame needs INTRA Macroblocks but majority of others can predict their motion from other blocks. 

Understand, that all MPEG cares about is, if encoder can find a block which is almost as good as a current one, use to minimize the error, if not bad luck and encode this as a fresh new information. Even this ability to find itself varies from encoder to encoder. 

4. So how does encoder make so many choices?  

Usually, for the B pictures, encoder starts a usual business by searching the most appropriate blocks from past and followed by future picture. 

a. if both frame produce some reasonable motion vector, this is most preferred to minimize the energy and book the block as BIDIRECTIONAL MB. 

b. if either past frame or future frame doesn't produce motion vector - i.e. the minimum energy (SAD) is higher than a useful number, then correspondingly only 1 motion vector is encoded making MB FORWARD or BACKWARD prediction only. 

c. if none of the reference frames have good enough motion vector - the MB is encoded as INTRA.

5. So why don't I simply encode everything as INTRA block and save from all the hassel? 

You can absolutely do that. It will still be perfectly legal MPEG stream. Just that INTRA coding will take more symbols to encode and hence it will have to be quantized badly to fit under a given bit budget. Putting everything as INTRA has significant loss of efficiency because typically video always has huge temporal redundancy that INTRA blocks don't exploit; but they do give random access freedom.