Is there some model-based variation of the Q-Learning algorithm which learns on a 3D SxAxS' table instead of a 2D SxA table?
The concept of state-action values $Q$ is to denote how good is to be in a particular state and perform a particular action in terms of expected future reward. From what I understand from your question, you are interested in the problem of model uncertainty (uncertainty on the dynamics of the system). In other words, our artificial agent interacts within an unknown environment (transition dynamics $T(s,a,s')$ and reward dynamics $R(s,a)$ or $R(s,a,s')$ are unknown).

The framework you should take a look at is Bayesian Model-based RL. I outline an approach so you can have an idea:

Modelling Transitions

First assume that we have uncertainty on the transitions of the environment $T(s,a,s')$.
To tackle this we will assume that our agent maintains a distribution over possible transitions. Without getting into the theoretical math, I will illustrate this by using a simple Dirchlet-Multinomial model:

The states are sampled from a Multinomial likelihood $s'\sim Mult(p_{ss'}^{a})$ and we assume a prior over the transitions $p_{ss'}\sim Dir(\alpha)$, where $\alpha$ is set to $1/|\cal{S}|$, where is $\cal{S}$ is the state space.
The posterior over transitions will be also a Dirichlet because of the conjugacy of the likelihood and prior distributions. To update such a posterior you need to perform simple algebraic calculations and maintaining the counts of each transition.

The Algorithm

The agent does two processes:


(Simulation or Planning phase) Samples an MDP from the posterior over transitions (in the first iteration you sample from the prior distribution). Having a 'fixed table of tranistions' solves the sampled MDP (e.g with Value iteration) and selects the optimum action.
(Real-world interaction phase) In the real world, you observe the reward of your action and the new state of the environment. You update the posterior counts.


Eventually, and if you have chosen an appropriate distribution for your domain the agent will adapt to the unknown environment. Of course richer priors with non conjugacy will lead to MCMC sampling methods. I refer you to this paper to get an overview of the problem: Model-based Bayesian Exploration and the quite advanced: Bayes-Adaptive MDPs for further research and exploration.