we propose a method 
 for inferring the kinetic rate matrix for stochastic 
 systems for which the steady state popuations are known . the types 
 of systems of interest include protein folding , ion channels , molecular motors , the evolutionary dynamics of protein sequences , the collective firing patterns of neurons , or noisy gene expression . in 
 such studies , stationary probabilities { pi } of n stable states { i } are often known or estimated from experimental data , detailed molecular mechanics calculations , coarse grained theories , or from maximum entropy . moreover , suppose further that 
 we know some average global dynamical quantity , such as the average 
 mutation rate ( number of amino acid changes per unit time ) , the average 
 current through an ion channel , or the average number of neurons that 
 change their state from spiking to resting ( and vice versa ) per unit 
 time step . how can we infer the full distribution of microscopic state - to - state 
 rates , given just the stationary - state populations and one or more 
 average overall rate quantities ? more specifically , we are interested 
 in a principled way to solve 
 the following under - determined  inverse  kinetics problem . 
 consider a stationary and irreducible markov process among i = 1,2,3, ... ,n states . suppose you know 
 the following : ( a ) the stationary state probability distribution , 
 { pi } of the occupancies 
 of those states , and ( b ) the value w 
 of some dynamical observable w averaged over the 
 ensemble of stationary state trajectories . from these n + 1 quantities , we want to infer the n  n microscopic transition probabilities,1between those states . while the stationary distribution indeed 
 constrains the transition 
 probability matrix , it does not uniquely determine it . here , we develop 
 a procedure based on the principle of maximum caliber , a variant of 
 the principle of maximum entropy , that is applicable to dynamical 
 processes . this variational principle allows 
 us to uniquely infer n transition 
 probabilities that are consistent with n imposed 
 constraints . first , we define the path entropy , s , over a given ensemble {  } of trajectories 
   
 ...  i  j  k  l ... as2maximum caliber is a variational 
 principle 
 that chooses a unique probability distribution { p( ) } over trajectories from all possible candidate distributions 
 as the one that maximizes the path entropy while otherwise satisfying 
 the relevant stationary and dynamical constraints . consider an ensemble of 
 stationary state trajectories {  } 
 ( see above ) having a total time duration t. restricting 
 our attention to first - order markov trajectories allows us to greatly 
 simplify the path entropy and carry out our analysis in terms of trajectories 
 of single steps , i  j. the markov property implies that the probability 
 of any particular trajectory  can be expressed in terms of 
 the transition probabilities { kij},3the path entropy of the above ensemble is 
 directly proportional to the total duration t of 
 the trajectory . the path entropy per unit time is given by4 the microscopic transition probabilities 
 of 
 any markov process 
 are subject to two types of constraints . first , from state i at time t , the system must land 
 somewhere at time t + dt . second , a system in state j at time t + dt must arrive from somewhere , so5 third , we require one additional constraint 
 that is global , that 
 is , averaged over the entire ensemble of trajectories . we fix the 
 path ensemble average of some dynamical quantity w. the average w over any given stationary trajectory  is given by6 the path ensemble average w is7since  is a stationary state trajectory , 
 the path ensemble average w of eq 7 simplifies to8 maximization of the path entropy subject to these three constraints 
 can be expressed equivalently in terms of maximization of a quantity 
 called the caliber : 9where  
 is the lagrange multiplier associated 
 with the constraint w and { ai } and { li } enforce the to - somewhere constraint and the from - somewhere constraint , respectively . to solve for the matrix kij of transition probabilities , we take the derivative of the 
 caliber   with respect 
 to kij and equate it 
 to zero . this gives10where we have made the substitutions : e = 
 i / pi and e = j . the values kij * are the transition probabilities that satisfy 
 the constraints and otherwise maximize the caliber . for simplicity 
 of notation , we drop the superscript * in the remainder of this paper , 
 that is , kij *  kij . in this problem , to compute the kij values , we first must determine 
 the values of the lagrange 
 multipliers i , j , and . we do so by substituting the constraint 
 relations mentioned above . for a given value 
 of  , the modified lagrange multipliers i and j are determined by satisfying the to - somewhere and from - somewhere conditions indicated above . from eqs 511where e = wij . equation 11 can be simplified 
 if we define a nonlinear operator d over column vectors x = [ x1,x2 , ... ] as d[x]i = pi / xi . we have12where  
 = [ 1,2 , ... ] and  
 = [ 1,2 , ... ] are the 
 column vectors 
 of lagrange multipliers . for a particular value of the lagrange 
 multiplier , eqs 12 can be numerically 
 and self - consistently solved for { i } and { i}. in practice , we choose 
 an appropriate  by first constructing transition probabilities 
 { kij } for multiple values 
 of  ( see eq 10 ) and choosing the value 
 of  that satisfies13where w is 
 the prescribed value of the ensemble average of the dynamical quantity w. so far , the treatment 
 above is generally 
 applicable to nonequilibrium systems . however , if we are interested 
 in restricting our attention to systems in thermodynamic equilbrium , 
 we can impose an additional constraint that the system must satisfy 
 detailed balance , pikij = pjkji . in this case , the caliber can be expressed as14here , dij are lagrange 
 multipliers that impose the detailed balance 
 condition . differentiating with respect to kij and setting the derivative to zero , we 
 find ( see eq 10)15where 
 ij = e = 1/ji . now 
 we have to determine the modified lagrange 
 multipliers i , j , ij , and  
 from the imposed constraints . we have16since ij = 1/ji , we have17 in eq 17 , identifying 
 i = ( ii),18where 
 e = wijsym is the symmetrized form of wij = e ( see above ) . it is 
 easy to see that kij in 
 eq 18 satisfies 
 detailed balance . the lagrange multipliers  = [ 1,2, ... ,n ] and  are determined by the same procedure as described above . we have19 in other words ,  is the solution 
 of the nonlinear equation20and  
 is determined by adjusting the 
 path ensemble average21 we now illustrate how the present 
 maxcal method can be used to 
 take a stationary - state distribution and a global dynamical constraint 
 and to infer microscopic kinetics . the number , n(t ) , of water molecules in the hydration 
 shell is a quantity that fluctuates with time t ( see 
 figure figure 1 ) . we want to compute how fast 
 the water molecules enter or exit the solvation shell . if the time 
 interval dt is small , n(t ) and n(t + dt ) will be statistically correlated . here , we construct a markov process 
 to model the time series { n(t)}. 
 we will require the markov process to reproduce ( a ) the stationary 
 distribution p(n ) that is observed 
 in molecular dynamics simulations , and ( b ) the average change in occupancy 
  per time step of duration dt , a path ensemble 
 average ( see appendix for details of molecular 
 simulation ) . while the choice of dynamical constraint(s ) remains somewhat 
 arbitrary , it is validated only a posteriori . in 
 other words , the constrain(s ) is a modeling aspect of any maximum 
 entropy method . we have22where n(t ) = i and n(t + 
 dt ) = j. since the system 
 should satisfy detailed balance , the transition 
 probability kij for a 
 transition n(t ) = i  n(t + dt ) = j is given by ( see eq 18),23 hydration shell ( black 
 circle ) around a central water 
 molecule 
 ( blue disc ) is dynamically populated by other water molecules in the 
 bulk solvent medium ( red discs ) . the probability , p(n ) that the hydration shell has exactly n water molecules is a key quantity in determining the solvation 
 free energy of liquid water . for a given value of , we determine the lagrange multipliers 
 i from eqs 20 above . in order to determine the lagrange multiplier  which 
 dictates the rate of transition between states , we first construct 
 markov processes for different values of . panel b of figure 
 figure 2 shows that the path ensemble average 
 of the change in occupation number per unit time step  is exponentially 
 decreasing with . from trajectories sampled at every 5 fs from 
 the md simulation , we find that experimental trajectory average expt =    0.0629 which corresponds 
 to  
  3.29 . from here onward , we use  = 3.29 and construct 
 the transition probabilities { kij } ( see eq 23 ) . note that the path ensemble 
 average  and consequently the lagrange multiplier  , 
 depend on the time interval dt between two observation 
 ( dt = 5 fs here ) . from the markov process constructed 
 with  = 3.29 ( see above ) , 
 we now compute various dynamical quantities : ( a ) the probability pd of jump size d , ( b ) the occupancy autocorrelation n(0),n() , and ( c ) the transition 
 probabilities kij , and 
 we compare to those obtained directly from the md simulation trajectory . in general , the maxcal method will be of value when transitions are 
 hard to simulate , such as for large kinetic barriers . here , we are 
 just illustrating with a toy problem for which we can determine the 
 transition probabilities independently from the simulations . panel a : the 
 stationary distribution p(n ) of 
 the number of water molecules in the hydration shell 
 of radius r = 3.2  of a reference water molecule . panel b : the dependence of the ensemble average of change in water 
 occupancy number  = |n(t + dt ) n(t)| on the lagrange multiplier . we see that  
 depends exponentially on . a higher  implies slower 
 dynamics and vice versa . we choose  = 3.29 to match the observed 
   0.0629 in the molecular dynamics simulation . panel a : the probability pd of jump size estimated from the trajectory 
 derived from md 
 simulation ( red squares ) is compared to the one predicted using the 
 transition probabilities of the markov process ( dashed black line ) . panel b : the normalized occupancy autocorrelation n(0),n() as 
 estimated from the md trajectory and as predicted from the transition 
 probabilities of the markov process . panel c : we directly compare 
 the transition probabilities kij for the probability of transition i  j empirically obtained from the md trajectory to the ones 
 predicted by using eq 23 . from the long simulation trajectory , the probability pd of jump size is estimated 
 as the histogram 
 of d = n(t + dt ) it is given by25where k( ) = k is the 
  power of 
 the matrix of transition probabilities { kij}. in figure 3 we plot pd , n(0),n() , and { kij } estimated from the molecular 
 dynamics trajectory 
 and compare them to our predictions from markov modeling . even though 
 we constrained only the mean value  = |d| of pd , the 
 modeled markov process captures the entire distribution pd with high accuracy . similarly the occupancy 
 correlation n(0),n() is also reproduced with high accuracy even 
 though we did not utilize any information about it 
 when inferring the transition probabilities of the markov process . moreover , our modeling also accurately captures the individual transition 
 probabilities { kij } over 
 4 orders of magnitude . we have presented 
 here a variational approach that computes n  n microscopic transition probabilities 
 of a markov process , given only knowledge of a stationary state population 
 distribution and one trajectory - averaged dynamical property . in this 
 approach , we maximize the path entropy subject to constraints ; that 
 is , we maximize the caliber . we show that this method gives correct 
 values of dynamical quantities in an example of molecular dynamics 
 simulations of a water solvation shell around a single water molecule . 
 this method may be useful for analyzing single - molecule experiments 
 such as on ion channels , dynamics of neuron 
 firing , and the dynamics of protein - sequence 
 evolution , for example .
we 
 present a principled approach for estimating the matrix of microscopic 
 transition probabilities among states of a markov process , given only 
 its stationary state population distribution and a single average 
 global kinetic observable . 
 we adapt maximum caliber , a variational 
 principle in which the path entropy is maximized over the distribution 
 of all possible trajectories , subject to basic kinetic constraints 
 and some average dynamical observables . 
 we illustrate the method by 
 computing the solvation dynamics of water molecules from molecular 
 dynamics trajectories .