The efficiency of looping over nested dictionaries in Python
One obvious improvement:

for word_pos, word_pos_prob in word_pos_dict.items():
    marginal_prob = 0        
    for pos_prob in pos_dict.values():
        marginal_prob += word_pos_prob * pos_prob 


Simplifies to:

multiplier = 0
for pos_prob in pos_dict.values():
    multiplier += pos_prob
# or just 'multiplier = sum(pos_dict.values())'

for word_pos, word_pos_prob in word_pos_dict.items():
    marginal_prob = word_pos_prob * multiplier        


Now the inner loop only runs once.