Estimating probability from generative model
Think of this as estimating a probability of some outcome, based on observations of many trials.  Suppose we flip a coin 1000 times, and it comes up heads 327 times.  What is your best estimate for the probability that the next flip will come up heads?  $0.327$, right?  You get that by dividing the number of heads by the number of trials.

What if you rolled a six-sized die 1000 times, and it came up a "one" 327 times?  What is your best estimate for the probability that it comes up "one"?  The same thing -- $0.327$, right?

It's the same here.  Each time that you generate $(s',o)$ and find that $s'=s'_0$, think of that as a trial -- the outcome of the trial is the observed value of $o$.  (Ignore all iterations where $s' \ne s'_0$; they're not even a trial.)  So, if you have a particular bin, say the bin corresponding to $o=1$, then you count the number of trials where you observed that $o$ was $1$, and divide by the total number of trials.  That's your estimate for the desired probability.