On	
  the	
  average	
  uncertainty	
  for	
  systems	
  
with	
  nonlinear	
  coupling	
  
	
  

Kenric	
  P.	
  Nelson 1, 2,	
  Sabir	
  R.	
  Umarov 3,	
  and	
  Mark	
  A.	
  Kon 1	
  

Abstract	
  
	
  
The	
  increased	
  uncertainty	
  and	
  complexity	
  of	
  nonlinear	
  systems	
  have	
  motivated	
  
investigators	
  to	
  consider	
  generalized	
  approaches	
  to	
  defining	
  an	
  entropy	
  function.	
  	
  New	
  insights	
  are	
  
achieved	
  by	
  defining	
  the	
  average	
  uncertainty	
  in	
  the	
  probability	
  domain	
  as	
  a	
  transformation	
  of	
  
entropy	
  functions.	
  	
  The	
  Shannon	
  entropy	
  when	
  transformed	
  to	
  the	
  probability	
  domain	
  is	
  the	
  
weighted	
  geometric	
  mean	
  of	
  the	
  probabilities.	
  	
  For	
  the	
  exponential	
  and	
  Gaussian	
  distributions,	
  we	
  
show	
  that	
  the	
  weighted	
  geometric	
  mean	
  of	
  the	
  distribution	
  is	
  equal	
  to	
  the	
  density	
  of	
  the	
  distribution	
  
at	
  the	
  location	
  plus	
  the	
  scale	
  (i.e.	
  at	
  the	
  width	
  of	
  the	
  distribution).	
  	
  The	
  average	
  uncertainty	
  is	
  
generalized	
  via	
  the	
  weighted	
  generalized	
  mean,	
  in	
  which	
  the	
  moment	
  is	
  a	
  function	
  of	
  the	
  nonlinear	
  
source.	
  	
  Both	
  the	
  Rényi	
  and	
  Tsallis	
  entropies	
  transform	
  to	
  this	
  definition	
  of	
  the	
  generalized	
  average	
  
uncertainty	
  in	
  the	
  probability	
  domain.	
  	
  For	
  the	
  generalized	
  Pareto	
  and	
  Student’s	
  t-­‐distributions,	
  
which	
  are	
  the	
  maximum	
  entropy	
  distributions	
  for	
  these	
  generalized	
  entropies,	
  the	
  appropriate	
  
weighted	
  generalized	
  mean	
  also	
  equals	
  the	
  density	
  of	
  the	
  distribution	
  at	
  the	
  location	
  plus	
  scale.	
  	
  A	
  
coupled	
  entropy	
  function	
  is	
  proposed,	
  closely	
  related	
  to	
  the	
  normalized	
  Tsallis	
  entropy,	
  but	
  
incorporating	
  a	
  distinction	
  between	
  the	
  additive	
  coupling	
  and	
  multiplicative	
  coupling.	
  	
  
	
  
Keywords:	
  	
  complex	
  systems,	
  nonlinear	
  coupling,	
  non-­‐additive	
  entropy,	
  nonextensive	
  statistical	
  
mechanics	
  

1 Introduction	
  
	
  
	
  
Entropy	
  is	
  the	
  standard-­‐bearer	
  for	
  defining	
  the	
  average	
  uncertainty	
  of	
  a	
  
probability	
  distribution	
  or	
  density	
  function	
  [1–3].	
  	
  Boltzmann,	
  Gibbs,	
  and	
  Shannon	
  
(BGS)	
  demonstrated	
  that	
  the	
  kernel − ln p 	
  is	
  necessary	
  to	
  form	
  a	
  weighted	
  arithmetic	
  
average	
  of	
  the	
  uncertainty	
  of	
  a	
  probability	
  distribution.	
  	
  Khinchin	
  showed	
  that	
  the	
  
N

entropy	
  function H = −∑ pi ln pi is	
  unique	
  given	
  the	
  axioms	
  that	
  it	
  is	
  	
  
i =1

	
  
a)	
  continuous	
  in	
  pi,	
  	
  
	
  
b)	
  maximized	
  at	
  the	
  uniform	
  distribution,	
  	
  
	
  
c)	
  not	
  changed	
  by	
  a	
  state	
  with	
  zero	
  probability,	
  and	
  	
  
	
  
d)	
  additive	
  for	
  conditionally	
  independent	
  probabilities.	
  	
  	
  
	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  
Correspondence:	
  	
  Kenric	
  Nelson,	
  kenricpn@bu.edu,	
  781-­‐645-­‐8564	
  
1	
  Boston	
  University,	
  Boston,	
  MA	
  
2	
  Raytheon	
  Company,	
  Woburn,	
  MA	
  
3	
  University	
  of	
  New	
  Haven,	
  New	
  Haven,	
  CT	
  	
  

Rényi	
  ,	
  Tsallis	
  and	
  Amari	
  [4–6]	
  sought	
  to	
  broaden	
  the	
  definition	
  of	
  average	
  
uncertainty	
  to	
  account	
  for	
  the	
  influence	
  of	
  nonlinear	
  dynamics	
  in	
  complex	
  systems.	
  	
  
Generalized	
  entropy	
  measures	
  have	
  been	
  applied	
  to	
  a	
  variety	
  of	
  nonlinear	
  systems	
  
such	
  as	
  decision	
  making	
  under	
  risk	
  [7,8],	
  communication	
  channel	
  equalization	
  [9],	
  
compressive	
  sensing	
  [10],	
  the	
  edge	
  of	
  chaos	
  [11],	
  space	
  plasma	
  [12,13],	
  high	
  energy	
  
physics	
  [14,15]	
  and	
  quantum	
  entanglement	
  [16].	
  	
  	
  Hanel	
  and	
  Thurner	
  [17,18]	
  have	
  
shown	
  that	
  requiring	
  just	
  the	
  first	
  three	
  of	
  Khinchin’s	
  axioms	
  leads	
  to	
  a	
  two-­‐
parameter	
  generalization	
  of	
  entropy,	
  one	
  of	
  which	
  is	
  the	
  Tsallis	
  generalization	
  and	
  
the	
  focus	
  of	
  this	
  communication.	
  
The	
  objective	
  of	
  the	
  paper	
  is	
  to	
  show	
  the	
  generalized	
  average	
  uncertainty	
  for	
  
a	
  nonlinear	
  system	
  can	
  be	
  defined	
  in	
  the	
  probability	
  domain	
  as	
  the	
  weighted	
  
generalized	
  mean.	
  	
  This	
  is	
  derived	
  as	
  a	
  transformation	
  of	
  the	
  generalized	
  entropy	
  
functions.	
  	
  New	
  evidence	
  for	
  the	
  importance	
  of	
  generalizing	
  the	
  average	
  uncertainty	
  
is	
  provided.	
  	
  For	
  the	
  distributions	
  that	
  minimize	
  the	
  weighted	
  generalized	
  mean	
  
(maximum	
  generalized	
  entropy)	
  constrained	
  by	
  a	
  location	
  and	
  scale,	
  the	
  density	
  at	
  
the	
  location	
  plus	
  the	
  scale	
  is	
  the	
  generalized	
  average	
  uncertainty.	
  	
  	
  
	
  
A	
  review	
  of	
  the	
  average	
  uncertainty	
  for	
  important	
  members	
  of	
  the	
  
exponential	
  family	
  provides	
  a	
  helpful	
  framework	
  prior	
  to	
  introduction	
  of	
  the	
  effects	
  
of	
  nonlinearity.	
  	
  The	
  Boltzmann-­‐Gibbs-­‐Shannon	
  entropy	
  transformed	
  to	
  the	
  
probability	
  domain,	
  is	
  the	
  weighted	
  geometric	
  mean	
  of	
  the	
  distribution,	
  that	
  is	
  
⎛ N
⎞ N
	
  
(1)	
  
Pavg ≡ exp ⎜ +∑ pi ln pi ⎟ = ∏ pipi , 	
  	
  
⎝ i =1
⎠ i =1
where	
  the	
  weights,	
  now	
  as	
  powers,	
  are	
  also	
  probabilities.	
  	
  The	
  average	
  uncertainty	
  
Pavg	
  is	
  the	
  average	
  probability	
  of	
  being	
  able	
  to	
  determine	
  the	
  state	
  of	
  the	
  system.	
  	
  
The	
  average	
  uncertainty	
  ranges	
  from	
  certainty	
  (Pavg	
  =1)	
  when	
  one	
  of	
  the	
  states	
  is	
  
certain,	
  to	
  (Pavg	
  =1/N)	
  when	
  all	
  states	
  are	
  equiprobable.	
  	
  The	
  intuition	
  is	
  that	
  the	
  
total	
  probability	
  of	
  the	
  distribution	
  is	
  the	
  product	
  of	
  the	
  probabilities	
  and	
  the	
  
average	
  is	
  determined	
  by	
  weighting	
  each	
  term	
  by	
  the	
  probability.	
  	
  Assuming	
  a	
  
continuous	
  distribution,	
  the	
  average	
  density	
  for	
  the	
  exponential	
  distribution	
  is	
  then	
  
⎛ ∞ −( x− µ ) ⎛ −( x− µ ) ⎞ ⎞
⎛ −( x− µ ) ⎞
1
1
1
σ
f avg ⎜ σ e
⎟ = exp ⎜ ∫µ σ e σ ln ⎜ σ e σ ⎟ dx ⎟
⎜
⎝
⎠
⎝
⎠ ⎟⎠ 	
  	
  
⎝
	
  
(2)	
  
1
=
σe
!
and	
  for	
  the	
  Gaussian	
  distribution	
  the	
  average	
  density	
  is	
  	
  
2
2
2
−1⎛ x − µ ⎞ ⎞
−1⎛ x − µ ⎞
−1⎛ x −µ ⎞ ⎞
⎛ ∞
⎞
⎛
⎛
⎜
⎟
⎜
⎟
⎜
⎟
2⎝ σ ⎠ ⎟
2⎝ σ ⎠
2⎝ σ ⎠ ⎟
1
1
1
⎜
⎜
⎜
favg 2πσ e
= exp ∫ 2πσ e
ln 2πσ e
dx ⎟
⎜ −∞
⎜
⎟
⎜
⎟ ⎟
⎝
⎠
⎝
⎠ ⎠ 	
  	
  
	
  
(3)	
  
⎝
1
=
.
2π eσ
In	
  both	
  cases	
  the	
  average	
  density	
  is	
  equal	
  to f ( µ + σ ) .	
  	
  	
  
	
  
Rényi	
  showed	
  that	
  information	
  theory	
  could	
  be	
  broadened	
  by	
  considering	
  
the	
  weighted	
  generalized	
  mean	
  of	
  probabilities	
  [19]	
  as	
  the	
  kernel	
  prior	
  to	
  applying	
  
2	
  	
  

the	
  logarithm	
  function.	
  	
  Through	
  consideration	
  of	
  the	
  statistics	
  of	
  a	
  weighted	
  
distribution	
   piq ,	
  Tsallis	
  showed	
  that	
  use	
  of	
  both	
  the	
  weighted	
  generalized	
  mean	
  and	
  
a	
  generalization	
  of	
  the	
  logarithm	
  function	
  provided	
  a	
  model	
  of	
  non-­‐additive	
  entropy.	
  	
  
Because	
  of	
  its	
  role	
  in	
  raising	
  probabilities	
  to	
  a	
  power,	
  the	
  parameter	
  q	
  can	
  be	
  
interpreted	
  as	
  the	
  number	
  of	
  independent	
  random	
  variables	
  whose	
  combined	
  
distribution	
  provides	
  a	
  basis	
  for	
  determining	
  generalized	
  statistical	
  properties.	
  	
  This	
  
analysis,	
  broadly	
  referred	
  to	
  as	
  nonextensive	
  statistical	
  mechanics	
  [20],	
  has	
  been	
  
shown	
  to	
  model	
  a	
  wide	
  variety	
  of	
  complex	
  systems.	
  	
  	
  
	
  
Section	
  2	
  introduces	
  the	
  concept	
  of	
  nonlinear	
  statistical	
  coupling	
  which	
  is	
  an	
  
interpretation	
  of	
  nonextensive	
  statistical	
  mechanics	
  focused	
  on	
  the	
  role	
  of	
  
nonlinearity	
  in	
  deforming	
  statistical	
  analysis.	
  	
  From	
  examination	
  of	
  multivariate	
  
distributions,	
  it	
  was	
  shown	
  that	
  the	
  coupling	
  parameter	
   κ is	
  related	
  to	
  q	
  via	
  the	
  
ακ
. 	
  	
  	
  	
  
influence	
  of	
  the	
  dimensions	
  d	
  and	
  power α of	
  the	
  state	
  variable	
  by	
   q = 1+
1+ dκ
!
Other	
  approaches	
  have	
  been	
  taken	
  to	
  parameterizing	
  the	
  Tsallis	
  statistics,	
  including	
  
κ ' = 1− q ,	
  which	
  was	
  originally	
  proposed	
  as	
  a	
  definition	
  of	
  nonlinear	
  statistical	
  
1
	
  	
  
1− q
which	
  is	
  the	
  translation	
  to	
  the	
  kappa-­‐distribution	
  used	
  in	
  space	
  physics	
  [12].	
  	
  There	
  
are	
  also	
  generalizations	
  of	
  entropy,	
  which	
  use	
  the	
  kappa	
  symbol,	
  such	
  as	
  the	
  work	
  
by	
  Kaniadakis[24].	
  	
  	
  
	
  
In	
  Section	
  3,	
  the	
  main	
  objective	
  relating	
  average	
  uncertainty	
  and	
  the	
  width	
  of	
  
coupled	
  exponential	
  distributions	
  is	
  established.	
  	
  In	
  Section	
  4	
  we	
  show	
  that	
  the	
  
generalized	
  entropies	
  of	
  Renyi	
  and	
  Tsallis	
  along	
  with	
  a	
  modified	
  normalization	
  of	
  
the	
  Tsallis	
  entropy	
  denoted	
  as	
  the	
  coupled	
  entropy,	
  can	
  each	
  be	
  expressed	
  as	
  the	
  
weighted	
  generalized	
  mean	
  of	
  the	
  distribution	
  and	
  a	
  transformation	
  to	
  the	
  entropy	
  
scale	
  using	
  a	
  generalized	
  logarithm	
  function.	
  

coupling	
  [21]	
  and	
  has	
  been	
  utilized	
  by	
  other	
  investigators	
  [22,23];	
  and κ '' =

2 Nonlinear	
  Statistical	
  Coupling	
  
	
  

Fundamental	
  to	
  complex	
  systems	
  is	
  the	
  influence	
  of	
  nonlinearity	
  on	
  
dynamics	
  [25,26].	
  	
  As	
  such	
  in	
  modeling	
  the	
  statistical	
  mechanics	
  of	
  a	
  complex	
  
system,	
  we	
  choose	
  the	
  source	
  of	
  nonlinearity	
  to	
  define	
  the	
  nonlinear	
  statistical	
  
coupling	
  (NSC)	
  κ	
  [21].	
  	
  NSC	
  has	
  two	
  contexts,	
  external	
  coupling	
  between	
  random	
  
variables	
  and	
  internal	
  coupling	
  between	
  the	
  states	
  of	
  a	
  random	
  variable	
  [27].	
  	
  The	
  
role	
  of	
  the	
  coupled	
  product	
  function	
  in	
  modeling	
  external	
  coupling	
  was	
  developed	
  in	
  
[28],	
  building	
  upon	
  the	
  q-­‐algebra	
  of	
  nonextensive	
  statistical	
  mechanics	
  [29].	
  	
  In	
  this	
  
paper,	
  the	
  focus	
  is	
  on	
  internal	
  coupling.	
  	
  The	
  probability	
  average	
  translated	
  from	
  
BGS	
  entropy	
  in	
  (1)	
  treats	
  the	
  probability	
  of	
  each	
  state	
  as	
  independent,	
  thus	
  
multiplying	
  the	
  probabilities	
  of	
  the	
  distribution.	
  	
  The	
  dependence	
  due	
  to	
  the	
  
probabilities	
  summing	
  to	
  one	
  is	
  incorporated	
  via	
  the	
  power	
  term.	
  	
  The	
  generalized	
  
average	
  uncertainty	
  introduced	
  in	
  Section	
  3	
  will	
  make	
  use	
  of	
  the	
  coupled	
  product	
  as	
  

3	
  	
  

a	
  model	
  of	
  dependency	
  between	
  the	
  states.	
  	
  First	
  we	
  introduce	
  the	
  basic	
  
relationships	
  of	
  nonlinear	
  statistical	
  coupling.	
  
Positive	
  coupling	
   (0 < κ < ∞ ) 	
  is	
  the	
  domain	
  of	
  heavy-­‐tail	
  statistics	
  in	
  which	
  
the	
  nonlinearity	
  causes	
  increased	
  variation.	
  	
  Examples	
  include	
  multiplicative	
  noise	
  
[30],	
  superstatistics	
  in	
  which	
  the	
  variance	
  fluctuates	
  [31,32],	
  and	
  the	
  inverse	
  of	
  the	
  
degree	
  of	
  freedom	
  [33–35].	
  	
  Negative	
  coupling	
   ( −1 < κ < 0) 	
  is	
  the	
  domain	
  of	
  
compact-­‐support	
  statistics	
  in	
  which	
  the	
  nonlinearity	
  causes	
  reduction	
  in	
  variation.	
  	
  
The	
  two	
  domains	
  are	
  related	
  by	
  a	
  conjugate	
  dual	
   1−+κκ . 	
  	
  Linear	
  systems	
   (κ = 0) 	
  are	
  
the	
  domain	
  of	
  exponential	
  statistics	
  and	
  logarithmic	
  measures	
  of	
  information.	
  	
  	
  
The	
  fluctuation	
  of	
  the	
  variance,	
  studied	
  as	
  superstatistics	
  [31,32],	
  is	
  a	
  helpful	
  
example.	
  	
  For	
  a	
  system	
  governed	
  by	
  an	
  exponential	
  distribution,	
  	
  
1 −σx '
e ,!!x ≥ 0, 	
  	
  
	
  
(4)	
  
!σ '
but	
  with	
  the	
  inverse	
  scale	
   β ' = 1 	
  fluctuating	
  according	
  to	
  a	
  gamma	
  distribution	
  
σ'
!
the	
  modified	
  statistics	
  are	
  governed	
  by	
  a	
  deformed	
  exponential	
  distribution,	
  which	
  
we	
  will	
  call	
  a	
  coupled-­‐exponential.	
  	
  The	
  scale	
  and	
  shape	
  of	
  the	
  coupled	
  exponential	
  
are	
  defined	
  by	
  the	
  mean	
  and	
  relative	
  variance	
  of	
  the	
  inverse	
  scale	
  

σ = β'
	
  

κ=

−1

,
2

β '2 − β '
β'

.

2

	
  	
  

(5)	
  

!
For	
  this	
  mean	
  and	
  relative	
  variance,	
  the	
  gamma	
  distribution	
  is	
  	
  

( ) Γ ( )( ) ( )

f β' =

	
  

1

κ
σ

1
σ

1

−1

β' κ e

−

σβ '
κ

, 	
  	
  

(6)	
  

!
and	
  the	
  coupled	
  exponential	
  distribution	
  is	
  	
  
∞
⎛1 ⎞
−⎜ +1⎟
1
	
  
(7)	
  
1+ κ x ⎝ κ ⎠ = ∫ f β ' β 'e − β 'x d β ' .	
  	
  
σ
0
!
	
  
The	
  coupled	
  exponential	
  function	
  is	
  notated	
  as	
  an	
  analog	
  of	
  the	
  exponential	
  
function	
  

(

( )(

)

() (

1
+1
κ
+

)

)

()

( )

(8)	
  
exp x ≡ 1+ κ x
,!! a ≡ max 0,a .	
  	
  
+
! κ
The	
  inverse	
  of	
  the	
  exponent	
  is	
  a	
  modified	
  coupling,	
  which	
  will	
  be	
  referred	
  to	
  as	
  the	
  
multiplicative	
  coupling.	
  	
  For	
  d	
  dimensions	
  [28,36]	
  the	
  exponent	
  generalizes	
  to	
  
κ −1 ≡ 1 + d ,	
  but	
  only	
  the	
  one-­‐dimensional	
  function	
  will	
  be	
  used	
  in	
  this	
  paper.	
  	
   κ d will	
  
! d κ
!
be	
  used	
  to	
  modify	
  the	
  product	
  function.	
  	
  The	
  inverse	
  of	
  the	
  multiplicative	
  coupling	
  is	
  
	
  

(

)

−κ d−1

( )
−κ −1

equal	
  to	
  the	
  asymptotic	
  fractal	
  dimension	
   lim 1+ κ x
= O x d 	
  of	
  the	
  coupled	
  
!x→∞
exponential	
  distribution	
  (7);	
  thus	
  the	
  inverse	
  of	
  the	
  coupling	
  terms	
  are	
  summarized	
  
by	
  
4	
  	
  

κ −1 = ν !!!Degree!of!Freedom,

	
  

	
  	
  
κ d−1 = D!!!Asymptotic!Fractal!Dimension.
!
Raising	
  to	
  a	
  power	
  is	
  not	
  identical	
  to	
  multiplying	
  the	
  argument,	
  so	
  an	
  
additional	
  parameter	
  definition	
  is	
  required	
  to	
  show	
  the	
  relationships.	
  	
  In	
  
anticipation	
  of	
  its	
  use	
  with	
  distributions,	
  the	
  power	
  used	
  is	
   1α 	
  	
  
1/α
κ

exp

	
  

1⎛ 1 ⎞
+1
α ⎜⎝ κ ⎟⎠
+

( x ) ≡ (1+κ x )
( x ) ≡ (1+ ακ x )

,

1⎛ 1 ⎞
+1
α ⎜⎝ κ ⎟⎠
+

	
  	
  

(9)	
  

(10)	
  

exp
.
! α ,κ
These	
  notational	
  definitions	
  are	
  needed	
  to	
  clarify	
  that	
  raising	
  to	
  a	
  power	
  is	
  
equivalent	
  to	
  both	
  multiplying	
  the	
  argument	
  and	
  dividing	
  the	
  nonlinear	
  coupling	
  
terms	
  
exp1/α x = expα ,κ x / α . 	
  	
  
	
  
(11)	
  
! κ
	
  
	
  
The	
  nonlinear	
  statistical	
  coupling	
  has	
  the	
  effect	
  of	
  changing	
  the	
  relationship	
  
between	
  variables	
  in	
  the	
  state	
  space.	
  	
  Multiplication	
  of	
  independent	
  coupled	
  
exponential	
  distributions	
  results	
  in	
  a	
  combined	
  variable	
  of	
  

()

(

(

) (

)(

(

)

))

1+ κ x 1+ κ y = 1+ κ x + y + κ xy .	
  
!
The	
  nonlinear	
  term	
  is	
  viewed	
  as	
  forming	
  the	
  coupled	
  addition	
  [21,29]	
  
	
  
x ⊕ y = x + y + κ xy ,	
  	
  
! κ
which	
  via	
  the	
  coupled	
  subtraction	
  is	
  seen	
  as	
  a	
  dilation	
  of	
  the	
  state	
  space	
  
x− y
	
  
.	
  	
  
x ⊖κ y ≡
1+ κ y
!
The	
  inverse	
  of	
  the	
  coupled	
  exponential	
  for	
  the	
  domain	
   1+ κ x > 0 	
  is	
  the	
  
coupled	
  logarithm	
  
⎞
1⎛ κ
	
  
lnκ x ≡ ⎜ x 1+κ −1⎟ ,!x > 0 .	
  	
  
κ⎝
⎠
!
Importantly,	
  the	
  integral	
  over	
  the	
  unit	
  interval	
  is	
  invariant	
  to	
  the	
  deformation	
  
	
  

()

1

∫ ln

κ

(12)	
  
(13)	
  
(14)	
  	
  

(15)	
  

( x )dx = 1 .	
  	
  This	
  insures	
  that	
  the	
  coupled	
  logarithm	
  deforms	
  the	
  measure	
  of	
  
−1

!0
information	
  without	
  modifying	
  the	
  “total	
  information”	
  over	
  the	
  unit	
  interval.	
  
Incorporating,	
  the	
  power	
   α 	
  the	
  definition	
  is	
  
ακ
⎞
1
1 ⎛ 1+
κ
	
  
(16)	
  
lnα ,κ x ≡ lnκ x α ≡
x
−1⎟ ,!x > 0 	
  	
  
⎜
α
ακ ⎝
⎠
!
	
  
The	
  role	
  of	
  the	
  coupled	
  addition	
  in	
  modifying	
  the	
  state	
  space	
  can	
  now	
  be	
  
expressed	
  in	
  terms	
  of	
  the	
  coupled	
  exponential	
  and	
  logarithm	
  functions	
  

()

( )

5	
  	
  

⎛ N
⎞ N
exp
x
=
exp
x
∏
κ
i
⎜⎝ ∑ ⊕κ i ⎟⎠ ,!! ∑ ⊕κ x i ≡ x1 ⊕κ ....⊕κ x N ,
i=1
i=1
i=1

( )

N

	
  

	
  	
  
(17)	
  
⎛ N ⎞ N
lnκ ⎜ ∏ x i ⎟ = ∑ ⊕ lnκ x i .
κ
! ⎝ i=1 ⎠ i=1
The	
  coupled	
  product	
  has	
  a	
  complimentary	
  role,	
  but	
  operates	
  on	
  the	
  exponent	
  of	
  the	
  
variable.	
  	
  Thus	
  not	
  only	
  do	
  the	
  coupled	
  sum	
  and	
  product	
  not	
  form	
  an	
  algebra	
  with	
  a	
  
distribution	
  property	
  [29],	
  but	
  are	
  more	
  accurately	
  two	
  complementary	
  algebras	
  
[22,37].	
  	
  The	
  coupled	
  product	
  of	
  positive-­‐valued	
  variables	
  ! f > 0!and!g > 0 	
  is	
  

(

κ1

κ

)

1
κ1

, 	
  where	
  the	
  multiplicative	
  coupling	
   κ 1 = 1+κκ 	
  is	
  the	
  inverse	
  of	
  
!
+
!
the	
  exponent	
  of	
  the	
  coupled	
  exponential	
  function.	
  	
  	
  This	
  definition	
  extends	
  to	
  
multiple	
  variables	
  and	
  has	
  the	
  following	
  properties	
  
f ⊗κ g ≡ f
1

+ g 1 −1

κ
⎛ N
⎞
1+κ
exp
x
≡
exp
x i − N −1 ⎟
∏
⎜∑
⊗κ
κ
i
κ
⎝ i=1
⎠
i=1 1+κ

( )

N

( ) (

)

1+κ
κ

⎛ N ⎞
= expκ ⎜ ∑ x i ⎟ ,!for!expκ x i > 0 	
  	
  
⎝ i=1 ⎠

( )

	
  

(18)	
  

⎛ N
⎞ N
lnκ ⎜ ∏ ⊗ x i ⎟ = ∑ lnκ x i ,!for!x i > 0.
⎝ i=1 1+κκ ⎠ i=1
!
In	
  [28]	
  the	
  formation	
  of	
  multivariate	
  coupled	
  exponentials	
  required	
  the	
  coupled	
  
product	
  to	
  be	
  defined	
  such	
  that	
  the	
  output	
  dimension	
  was	
  the	
  sum	
  of	
  the	
  input	
  
dimensions.	
  	
  This	
  paper	
  is	
  focused	
  on	
  the	
  coupling	
  between	
  states	
  for	
  a	
  single	
  
random	
  variable	
  and	
  as	
  such	
  the	
  dimension	
  does	
  not	
  change.	
  	
  The	
  coupled	
  power	
  
will	
  also	
  be	
  needed,	
  	
  

x
!

	
  

⊗κa

1

(

κ1

(

≡ ax − a −1

))

1
κ1
+

,!x > 0. 	
  	
  

(19)	
  

⎛ −x α ⎞
Generalization	
  of	
  the	
  stretched	
  exponential	
  distributions	
   f x ∝exp ⎜
	
  
α ⎟
⎝ ασ ⎠
!
utilizes	
  the	
  two-­‐parameter	
  definitions	
  from	
  Equation	
  (10).	
  	
  Thus	
  the	
  coupled	
  
stretched	
  exponential	
  is	
  expressed	
  as	
  

()

1+κ

⎛ xα ⎞
⎛ −x α ⎞ ⎛
x α ⎞ −ακ
	
  
(20)	
  
expκ α ⎜ α ⎟ = exp−α ,κ ⎜
=
1+
κ
. 	
  	
  
⎜
α ⎟
σ α ⎟⎠ +
⎝σ ⎠
⎝ ασ ⎠ ⎝
!
The	
  coupled	
  exponential (α = 1) and	
  coupled	
  Gaussian (α = 2) distributions	
  are	
  of	
  
−1

particular	
  interest	
  given	
  their	
  role	
  as	
  maximum	
  generalized	
  entropy	
  distributions	
  	
  
	
  
	
  
	
  

6	
  	
  

with	
  constraints	
  on	
  the	
  scale	
  of	
  the	
  distribution.	
  	
  These	
  distributions	
  [38]	
  are	
  
defined	
  as	
  	
  
α
−1 ⎛ ⎛ x − µ ⎞ ⎞
1
α
	
  
(21)	
  
!
expκ ⎜ ⎜
⎟ ,!κ ≥ −1. 	
  	
  
⎜⎝ ⎝ σ ⎟⎠ ⎟⎠
Z κ ,σ ,α
!
For	
   κ ≥ 0 	
  and	
   α = 1 	
  the	
  domain	
  of	
  the	
  random	
  variable	
  is	
   x ≥ µ 	
  and	
   Z (κ ,σ ,α ) = σ . 	
  	
  

(

)

For	
   κ ≥ 0 	
  and!α = 2,! − ∞ < x < ∞ 	
  and	
   Z (κ ,σ ,α ) =

κ ( 12+κκ )!

πσ (1 + κ ) ( 21κ )!

, 	
  where	
  

x ! ≡ Γ ( x + 1) 	
  is	
  Euler’s	
  gamma	
  function.	
  	
  The	
  parameters	
  of	
  the	
  distributions	
  are	
  the	
  
location	
  μ,	
  the	
  scale	
  σ,	
  and	
  the	
  normalization	
  Z.	
  	
  One	
  of	
  the	
  benefits	
  of	
  this	
  approach	
  
to	
  the	
  definition	
  is	
  that	
  the	
  source	
  of	
  coupling	
  κ	
  corresponds	
  exactly	
  to	
  the	
  shape	
  
parameter	
  for	
  the	
  generalized	
  Pareto	
  distribution	
  and	
  the	
  inverse	
  of	
  the	
  degree	
  of	
  
freedom	
  for	
  the	
  Student’s	
  t-­‐distribution.	
  	
  For	
  the	
  compact-­‐support	
  domain,	
  the	
  
degree	
  of	
  freedom	
  is	
  negative	
  and	
  less	
  than	
  -­‐1.	
  
	
  
	
  
	
  
Definition	
  1	
  Coupled	
  Probability/Density	
  	
  Given	
  a	
  discrete	
  probability	
  
distribution	
   p = pi ,!i = 1,...,N 	
  the	
  following	
  distribution	
  is	
  called	
  the	
  coupled	
  
!
probability	
  distribution	
  	
  

{

}

α ,κ
Pi( ) =

	
  

1+

pi
N

∑p
j=1

ακ
1+κ

1+
j

ακ
1+κ

=

ακ

N

∏ p1+j κ

pi

j=1
j≠i

N ⎛
∑ ⎜⎜ pi
i=1
⎝

N

∏p
j=1

ακ
1+κ
j

⎞
⎟
⎟
⎠

. 	
  

(22)	
  

j≠i
!
Similarly,	
  for	
  a	
  given	
  continuous	
  density	
   f ( x ) 	
  	
  the	
  coupled	
  density	
  function	
  is	
  
defined	
  by	
  

()

α ,κ
f( ) x ≡

	
  

f
∞

∫f

1+

1+

ακ
1+κ

ακ
1+κ

( x ) . 	
  	
  
( x )dx

(23)	
  

−∞
!
The	
  integral	
  in	
  (23)	
  is	
  assumed	
  finite.	
  	
  The	
  expression	
  on	
  the	
  right	
  of	
  Equation	
  (22)	
  
is	
  shown	
  to	
  make	
  evident	
  the	
  modeling	
  of	
  a	
  coupled	
  state	
  of	
  the	
  system.	
  	
  The	
  Cauchy	
  
ακ
distribution	
   α = 2,!κ = 1 	
  is	
  an	
  illustrative	
  example.	
  	
  In	
  this	
  case	
  !1+
= 1 	
  and	
  the	
  
κ
!
coupled-­‐probability	
  is	
  formed	
  by	
  dividing	
  the	
  probability	
   pi 	
  of	
  state	
  i	
  by	
  the	
  product	
  
of	
  all	
  the	
  probabilities	
  of	
  all	
  the	
  other	
  states	
  and	
  then	
  renormalizing	
  them.	
  	
  This	
  
procedure	
  has	
  a	
  direct	
  connection	
  with	
  the	
  origin	
  of	
  the	
  Cauchy	
  distribution	
  as	
  the	
  
division	
  of	
  two	
  random	
  variables.	
  

(

)

7	
  	
  

An	
  important	
  result	
  of	
  nonextensive	
  statistical	
  mechanics	
  is	
  the	
  relationship	
  
between	
  coupled-­‐moments	
  and	
  the	
  parameters	
  of	
  the	
  coupled-­‐exponential	
  and	
  
coupled-­‐Gaussian	
  distributions	
  [21,39].	
  	
  Using	
  the	
  coupled	
  algebra	
  the	
  
generalization	
  of	
  the	
  nth-­‐moment	
  takes	
  the	
  form	
  	
  
N

x in

	
  

N

κ

n,κ
≡ ∑ x inPi( ) =
i=1

∑ xinpi
i=1
N

1+n

∑p

!

i=1

1+n

i

κ
1+κ

κ
1+κ

. 	
  	
  

(24)	
  

The	
  α	
  parameter	
  is	
  not	
  required	
  because	
  it	
  drops	
  out	
  from	
  the	
  expression

n ακ
.	
  	
  
!α 1+ κ

Lemma	
  1	
  	
  Given	
  either	
  the	
  coupled-­‐exponential	
  or	
  the	
  coupled-­‐Gaussian	
  
distribution,	
  the	
  generalized	
  scale	
  parameter	
  σ	
  is	
  determined	
  by	
  the	
  generalizations	
  
of	
  the	
  1st	
  and	
  2nd	
  moments,	
  respectively:	
  	
  	
  
⎛1
⎛ x ⎞⎞
= ∫ x ⎜ expκ−1 ⎜ ⎟ ⎟
σ
⎝σ ⎠⎠
−∞ ⎝
∞

	
  

x
!

κ

1+2κ
1+κ

⎛1
⎞
−1 ⎛ x ⎞
∫−∞ ⎜⎝ σ expκ ⎜⎝ σ ⎟⎠ ⎟⎠
∞

dx

1+3κ

1+2κ
1+κ

∞
−1 ⎛ x 2 ⎞ ⎞ 1+κ
−1 ⎛ x 2 ⎞ ⎞
⎛
⎛1
2
2 1
2
2
x = ∫ x ⎜ expκ ⎜ 2 ⎟ ⎟
dx
	
  
⎜ Z expκ ⎜ 2 ⎟ ⎟
∫
κ
Z
σ
⎝
⎠
⎝σ ⎠⎠
⎝
⎠
−∞
−∞ ⎝
!
See	
  [39]	
  for	
  the	
  proof	
  and	
  [21]	
  for	
  the	
  notation.	
  
∞

dx = σ , 	
  	
  

1+3κ
1+κ

dx = σ 2 . 	
  	
  

(25)	
  

(26)	
  

3 The	
  Coupled	
  Average	
  Uncertainty	
  

The	
  coupled	
  algebra	
  provides	
  a	
  foundation	
  for	
  defining	
  a	
  generalization	
  of	
  
the	
  average	
  uncertainty	
  given	
  coupling	
  of	
  statistical	
  states	
  which	
  is	
  equal	
  to	
  the	
  
weighted	
  generalized	
  mean	
  (or	
  weighted	
  p-­‐norm)	
  [22,27,40].	
  	
  Equation	
  (1)	
  is	
  
generalized	
  using	
  the	
  coupled	
  exponential	
  and	
  coupled	
  logarithm	
  functions.	
  
	
  
Definition	
  2	
  Coupled	
  Log	
  Average	
  	
  Given	
  a	
  set	
  of	
  weights	
   {wi , 	
  i = 1,..N } such	
  that	
  
N

∑w
i =1

i

= 1 	
  and	
  a	
  set	
  of	
  variables	
   { xi , 	
  i = 1,..N } ,	
  the	
  coupled-­‐log	
  average	
  of	
  the	
  

variables	
  is	
  
−1 ⎧ N
⎫
expκ α ⎨∑ wi lnκ x i−α ⎬. 	
  	
  
⎩ i=1
⎭
!
Lemma	
  2	
  The	
  coupled-­‐log	
  average	
  	
  

(

	
  

)

a)	
  can	
  be	
  expressed	
  equivalently	
  using	
  the	
  coupled	
  product,	
  and	
  	
  
b)	
  is	
  equal	
  to	
  the	
  weighted	
  generalized	
  mean	
  (or	
  weighted	
  p-­‐norm).	
  
	
  

8	
  	
  

(27)	
  

	
  
Proof	
  a)	
  We	
  have	
  	
  
wi

⊗ − ακ
⎧N
⎫ N
−α
α
	
  
expκ ⎨∑ wi lnκ x i ⎬ = ∏ ⊗ x i 1+κ ,	
  
− ακ
⎩ i=1
⎭ i=1 1+κ
!
where	
  we	
  used	
  the	
  properties	
  in	
  Equations	
  (18)	
  and	
  (19).	
  	
  
	
  
b)	
  We	
  have	
  
−1 ⎧ N
⎫
expκ α ⎨∑ wi lnκ x i−α ⎬
⎩ i=1
⎭

(

−1

)

(

)

⎛
⎛ w ⎛
= ⎜ 1+ κ ⎜ ∑ i ⎜ x
⎝ i=1 κ ⎝
⎝
N

	
  

⎛
= ⎜ ∑ wi x
⎝
! i=1
N

− ακ
1+κ
i

⎞
⎟
⎠

(28)	
  

− ακ
1+κ
i

⎞⎞⎞
−1⎟ ⎟ ⎟
⎠⎠⎠

1+κ
− ακ

	
  	
  

(29)	
  

1+κ
− ακ

.

ακ
Given	
  the	
  central	
  role	
  of	
  the	
  parameter	
   m =
for	
  the	
  weighted	
  generalized	
  mean,	
  
1+ κ
!
the	
  coupled	
  log	
  average	
  is	
  summarized	
  as	
  
−1

⎧N
⎫ ⎛ N
⎞m
ακ
	
  
	
  .	
  
expκ α ⎨∑ wi lnκ x i−α ⎬ = ⎜ ∑ wi x i−m ⎟ ,!m =
1+ κ
⎝
⎠
i=1
i=1
⎩
⎭
!
The	
  coupled-­‐log	
  average	
  extends	
  to	
  continuous	
  variables	
  by	
  the	
  relationship	
  
−1

(

)

(30)	
  

−1

⎧∞
⎫⎪ ⎛ ∞
⎞m
−α
−m
α ⎪
expκ ⎨ ∫ w x lnκ f
x dx ⎬ = ⎜ ∫ w x f
x dx ⎟ . 	
  	
  
	
  
(31)	
  
⎝ −∞
⎠
⎪
⎪
−∞
⎩
⎭
!
Using	
  the	
  coupled	
  probability	
  as	
  the	
  weight	
  of	
  the	
  generalized	
  mean	
  we	
  can	
  
now	
  define	
  the	
  average	
  uncertainty	
  of	
  a	
  distribution	
  that	
  originates	
  from	
  a	
  system	
  
with	
  nonlinear	
  coupling.	
  
	
  
Definition	
  3	
  Coupled	
  Average	
  Uncertainty	
  	
  Given	
  a	
  distribution	
   p = pi ,!i = 1,...,N 	
  
!
its	
  coupled	
  average	
  uncertainty	
  is	
  defined	
  by	
  the	
  coupled	
  log	
  average	
  with	
   x i = pi 	
  
!
α ,κ )
(
and	
   wi = Pi
	
  
!
	
  
−1

()

()

() ()

{

}

1+κ

	
  

⎛ N (α ,κ ) −1+ακκ ⎞ −ακ
Pκ avg ≡ ⎜ ∑ Pi pi ⎟ . 	
  	
  
⎝ i=1
⎠
!

9	
  	
  

(32)	
  

Furthermore,	
  the	
  coupled	
  average	
  uncertainty	
  simplifies	
  to	
  the	
  weighted	
  
generalized	
  mean	
  with	
  the	
  distribution	
  as	
  the	
  weight;	
  thus	
  
−1

⎛ N 1+m −m ⎞ m
1
⎜ ∑ pi pi ⎟
N
m
⎛
⎞
ακ
⎟ = ⎜ ∑ pi1+m ⎟ ,!m =
	
  
(33)	
  
Pκ avg ≡ ⎜ i=1 N
, 	
  	
  
1+ κ
⎝ i=1
⎠
⎜
⎟
1+m
⎜⎝ ∑ p j
⎟⎠
j=1
!
and	
  for	
  a	
  continuous	
  distribution	
  
⎧ ∞ 1+ ακ
⎫
1
f 1+κ x lnκ f −α x dx ⎪
⎪
∞
∫
m
⎛
⎞
−1 ⎪
⎪
1+m
	
  
(34)	
  
fκ avg = expκ α ⎨ −∞
x dx ⎟ . 	
  	
  
⎬=⎜ ∫ f
∞
ακ
1+
⎝
⎠
⎪
⎪ −∞
∫−∞ f 1+κ x dx
⎪
⎪
⎩
⎭
!
	
  
	
  	
  
	
  
Applying	
  the	
  coupled	
  average	
  uncertainty	
  to	
  the	
  coupled	
  exponential	
  and	
  
coupled	
  Gaussian	
  distributions,	
  shown	
  in	
  Figure	
  1,	
  produces	
  the	
  main	
  result	
  
regarding	
  its	
  relationship	
  with	
  the	
  location	
  and	
  scale	
  of	
  the	
  distribution.	
  

()

()

()

()

Theorem	
  1	
  	
  The	
  coupled	
  average	
   fκ avg 	
  of	
  either	
  the	
  coupled	
  exponential	
  
distribution	
  or	
  the	
  coupled	
  Gaussian	
  distribution	
  with	
  a	
  weight	
  defined	
  by	
  the	
  
coupled	
  probability	
  is	
  equal	
  to	
  the	
  density	
  of	
  the	
  distribution	
  evaluated	
  at	
   x = µ + σ . 	
  
Proof:	
  	
  Without	
  loss	
  of	
  generality	
  we	
  set	
  !µ = 0. 	
  	
  For	
  the	
  coupled	
  exponential	
  
distribution,	
  which	
  corresponds	
  to	
  !α = 1 	
  in	
  Equation	
  (21),	
  that	
  is	
  
⎛ x⎞
1
f x = expκ−1 ⎜ ⎟ ,	
  	
  the	
  coupled	
  average	
  uncertainty	
  is	
  
σ
⎝σ ⎠
!

()

1

⎛∞
⎞m
fκ avg = ⎜ ∫ f 1+m x dx ⎟ !
⎝0
⎠

()

1+2κ
⎛
⎞
1+2κ ∞
−κ
⎛
⎞
x
= σ κ ⎜ ∫ ⎜ 1+ κ ⎟
dx ⎟
⎜ 0⎝
⎟
σ⎠
⎝
⎠

	
  

=σ

	
  

!

=

1+2κ
κ

(

⎛ 1+ κ
⎝

)

() (

−1

σ⎞
⎠

1+κ
κ

)

1 −1
e 1 = f µ +σ .
σ κ

	
  
10	
  

1+κ
κ

	
  	
  

(35)	
  

Similarly,	
  for	
  the	
  coupled	
  Gaussian	
  distribution,	
  which	
  corresponds	
  to	
  !α = 2 	
  in	
  
⎛ x2 ⎞
1
expκ−1/2 ⎜ 2 ⎟ ,	
  the	
  coupled	
  average	
  uncertainty	
  
Equation	
  (21),	
  that	
  is	
   f x =
Z κ ,σ ,2
⎝σ ⎠
!
is	
  	
  
	
  

() (

)

1

⎛∞
⎞m
fκ avg = ⎜ ∫ f 1+m x dx ⎟ !
⎝ −∞
⎠

()

( ) ⎞⎟
( )( )!⎟⎠

⎛
κ 1+2κκ !
=⎜
⎜⎝ πσ 1+ κ 21κ

1+3κ
2κ

1+3κ
⎛∞
⎞
2
−2κ
⎛
⎞
⎜ 1+ κ x
∫ ⎜⎝ σ 2 ⎟⎠ dx ⎟⎟
⎜ −∞
⎜⎝
⎟⎠

( )
( )
( )( )
( )
⎛
⎞
κ ( )!
⎟ (1+ κ )
=⎜
⎜⎝ πσ (1+ κ )( )! ⎟⎠
1
=
e (1) = f ( µ + σ )
Z (κ ,σ ,2)
⎛
κ 1+2κκ !
=⎜
⎜⎝ πσ 1+ κ 21κ

	
  

1+3κ
2κ

⎞
⎟
! ⎟⎠

1+κ
2κ

⎛ πσ 1
2κ
⎜
1+
⎜⎝ κ 2κκ
−

!⎞
⎟
! ⎟⎠

1+κ
2κ

1+κ
2κ

	
  	
  

(36)	
  

1+κ
2κ

1
2κ

−1/2
κ

!

	
  
Figure	
  1	
  	
  The	
  a)	
  coupled	
  exponential	
  and	
  b)	
  coupled	
  Gaussian	
  distributions	
  have	
  the	
  property	
  that	
  the	
  coupled	
  
average	
  uncertainty,	
  equation	
  (34),	
  is	
  equal	
  to	
  the	
  density	
  at	
  the	
  location	
  plus	
  the	
  scale	
   f ( µ + σ ) .	
  	
  Several	
  
examples	
  of	
  the	
  distributions	
  with	
  coupling	
  ranging	
  from	
   −2 3 	
  to	
  2	
  are	
  shown.	
  	
  

Theorem	
  1	
  has	
  important	
  implications	
  for	
  defining	
  the	
  generalized	
  average	
  
uncertainty.	
  	
  For	
   κ → 0 	
  the	
  weighted	
  generalized	
  mean	
  is	
  the	
  weighted	
  geometric	
  
mean.	
  	
  Just	
  as	
  the	
  standard	
  deviation	
  is	
  an	
  expression	
  of	
  the	
  variation	
  of	
  the	
  random	
  
variable,	
  the	
  geometric	
  mean	
  of	
  a	
  distribution	
  weighted	
  by	
  the	
  distribution	
  is	
  an	
  
	
  
11	
  

expression	
  of	
  average	
  uncertainty.	
  	
  For	
  the	
  Gaussian	
  distribution,	
  one	
  can	
  verify	
  that	
  
the	
  weighted	
  geometric	
  mean	
  of	
  the	
  distribution	
  is	
  equal	
  to	
  the	
  density	
  evaluated	
  at	
  
x = µ + σ . 	
  
The	
  average	
  uncertainty	
  is	
  generalized	
  by	
  using	
  the	
  coupled	
  probability	
  
distribution	
  for	
  the	
  weight	
  and	
  taking	
  the	
   −ακ (1+κ ) moment.	
  And	
  when	
   f ( x ) 	
  is	
  a	
  
!
member	
  of	
  the	
  coupled	
  distributions	
   f µ + σ = fκ avg . 	
  This	
  relationship	
  is	
  shown	
  in	
  
	
  
!
	
  
	
  
Figure	
  2	
  for	
  the	
  heavy-­‐tail	
  domain	
  of	
  the	
  coupled-­‐Gaussian	
  distribution.	
  	
  The	
  
coupled	
  average	
  uncertainty	
  versus	
  the	
  source	
  of	
  coupling	
  parameter	
  for	
  the	
  
distribution	
  is	
  plotted	
  over	
  the	
  range	
   0 ≤ κ ≤ 1 ,	
  which	
  includes	
  the	
  Gaussian	
   (κ = 0) ,	
  

(

)

the	
  boundary	
  with	
  infinite	
  variance	
   (κ = 0.5) 	
  and	
  the	
  Cauchy	
  distribution	
   (κ = 1) . 	
  	
  
,
Four	
  values	
  for	
  the	
  coupled	
  average	
  uncertainty,	
  called	
  the	
  metric	
  coupling	
  
κ = 1 , 2 , 3 , 4 are	
  shown.	
  	
  In	
  each	
  case	
  the	
  maximum	
  uncertainty	
  
5 5 5 5 	
  
1 −1
corresponds	
  to	
   eκ 2 (1 ) .
Z
	
  
	
  

{

}

AverageUncertainty

0.5

Metric Coupling
one fifth
two fifths
three fifths
four fifths

0.4
0.3

g	
   Z1 eκ−1/2 (1)

0.2
0.1
0.0
0.0

0.2

0.4
0.6
Coupling of Distribution

0.8

1.0

Figure	
  2	
  	
  The	
  coupled	
  average	
  uncertainty	
  as	
  a	
  function	
  of	
  the	
  source	
  of	
  coupling	
  is	
  shown.	
  	
  The	
  x-­‐axis	
  varies	
  the	
  
coupling	
  of	
  a	
  coupled	
  Gaussian	
  distribution	
  with	
   σ = 1. 	
  The	
  four	
  lines	
  are	
  for	
  metric	
  coupling	
  of	
  

{

κ = 1 ,2 , 3 ,4
5

5

5

} , 	
  which	
  translates	
  into	
  weighted	
  generalized	
  mean	
  with	
  !1+2κκ . 	
  	
  When	
  the	
  metric	
  coupling	
  

5

equals	
  the	
  distribution	
  coupling	
  the	
  average	
  uncertainty	
  is	
  minimized	
  and	
  equals	
  

	
  
12	
  

1 −12
eκ (1) .	
  	
  	
  
Z

	
  

4 The	
  coupled-­‐entropy	
  function	
  

	
  
In	
  defining	
  the	
  coupled	
  log	
  average	
  in	
  Equation	
  (27)	
  the	
  role	
  of	
  the	
  
generalized	
  entropy	
  was	
  implied	
  but	
  not	
  explicitly	
  expressed.	
  	
  In	
  this	
  section	
  we	
  
show	
  that	
  the	
  Renyi,	
  Tsallis,	
  and	
  normalized	
  Tsallis	
  entropies	
  can	
  each	
  be	
  
interpreted	
  as	
  the	
  weighted	
  generalized	
  mean	
  of	
  a	
  distribution	
  translated	
  to	
  an	
  
entropy	
  scale	
  by	
  generalized	
  logarithms.	
  	
  Thus	
  translated	
  to	
  the	
  probability	
  domain	
  
they	
  are	
  each	
  identical	
  expressions	
  of	
  average	
  uncertainty.	
  	
  Use	
  of	
  the	
  coupled	
  
logarithm	
  leads	
  to	
  a	
  new	
  definition,	
  called	
  the	
  coupled-­‐entropy.	
  	
  Its	
  properties	
  with	
  
respect	
  to	
  the	
  nonlinear	
  coupling	
  are	
  compared	
  with	
  the	
  Renyi	
  and	
  Tsallis	
  
generalizations.	
  
	
  
In	
  information	
  theory,	
  the	
  negative	
  logarithm	
  of	
  a	
  probability	
  is	
  referred	
  to	
  
as	
  the	
  surprisal.	
  	
  From	
  Equation	
  (16),	
  the	
  coupled	
  surprisal	
  (or	
  generalized	
  measure	
  
of	
  information)	
  is	
  then	
  
⎞
1
1 ⎛ −1+ακκ
−α
	
  
(37)	
  
sα ,κ ≡ −ln −α ,κ x ≡ lnκ x =
x
−1
⎟ ,!for!x > 0 	
  	
  
α
ακ ⎜⎝
⎠
!
An	
  alternative	
  definition	
  is	
  possible	
  based	
  on	
  the	
  inverse	
  of	
  the	
  coupled	
  stretched	
  
exponential	
  (20)	
  taking x 	
  as	
  the	
  argument	
  	
  
! σ
1

⎛ 1 ⎛ −ακ
⎞⎞α
1+κ
lnκ x
≡ −α ln −α ,κ x ≡ ⎜ ⎜ x
−1⎟ ⎟ ,!for!x > 0 .	
  	
  
	
  
(38)	
  
⎠⎠
⎝κ ⎝
!
While	
  not	
  investigated	
  here,	
  the	
  alternative	
  definition	
  is	
  of	
  potential	
  significance	
  as	
  
an	
  approach	
  to	
  the	
  two-­‐parameter	
  generalized	
  entropy	
  function	
  proposed	
  by	
  Hanel,	
  
et.	
  al.	
  [17,18]	
  and	
  related	
  generalizations	
  investigated	
  in	
  [37].	
  	
  
	
  
Definition	
  4:	
  Coupled	
  Entropy	
  	
  Given	
  a	
  discrete	
  probability	
  distribution	
  
p = pi ,!i = 1,...,N 	
  or	
  a	
  continuous	
  density f x 	
  and	
  setting	
  the	
  Boltzmann	
  constant	
  
!
!
to	
  one,	
  the	
  coupled	
  entropy	
  is	
  defined	
  as	
  
1
−α α

(

{

) (

)

1
α

()

}

1+κ

	
  

	
  
	
  

Sα ,κ
!

Sα ,κ
!

N
⎛ N 1+ ακ ⎞ ακ
α ,κ
p ≡ −ln −α ,κ ⎜ ∑ pi 1+κ ⎟ = − ∑ Pi( ) ln −α ,κ pi , 	
  
⎝ i=1
⎠
i=1

()

(39)	
  

1+κ

∞
⎛ ∞ 1+ ακ
⎞ ακ
α ,κ
1+κ
f x ≡ −ln −α ,κ ⎜ ∫ f
x dx ⎟ = − ∫ f ( ) x ln −α ,κ f x dx . 	
  
⎝ −∞
⎠
−∞
	
  

( ( ))

()

	
  
13	
  

()

()

(40)	
  

The	
  coupled-­‐entropy	
  expands	
  to	
  the	
  following	
  expression	
  
1+κ

Sα ,κ

ακ
⎛ N 1+1+
⎞ ακ
p = −ln −α ,κ ⎜ ∑ pi κ ⎟
⎝ i=1
⎠

()

1+κ − ακ
⎛ N
⎞
ακ
ακ 1+κ
⎛
⎞
1+
1 ⎜
=
−1⎟ 	
  	
  
∑ p 1+κ ⎟⎠
⎟
ακ ⎜⎜ ⎜⎝ i=1 i
⎟⎠
⎝

	
  

(41)	
  

−1
⎛
⎞
ακ
⎞
1 ⎛ N 1+1+
κ
⎜ ∑p
=
⎟ −1⎟⎟ .
ακ ⎜⎝ ⎜⎝ i=1 i
⎠
⎠

!
For	
  the	
  two	
  principal	
  cases	
  of	
  interest,	
  namely	
  for	
   α = 1 	
  and	
   α = 2, the	
  coupled-­‐
entropy	
  becomes	
  	
  
−1
⎛ ⎛ N (1+2κ )
⎞
⎞
1⎜
1+κ )
(
⎜ ∑ pi
⎟ −1⎟ ,
S1,κ p =
⎜
⎟
κ ⎜ ⎜ i=1
⎟⎠
⎟⎠
⎝
⎝
	
  
	
  	
  
(42)	
  
−1
⎛ ⎛ N (1+3κ )
⎞
⎞
1
(1+κ ) ⎟ −1⎟ .
S2,κ p = ⎜ ⎜ ∑ pi
⎟
2κ ⎜⎜ ⎜ i=1
⎟⎠
⎟⎠
⎝
⎝
!
Both	
  the	
  Rényi	
  and	
  Tsallis	
  entropies	
  can	
  be	
  expressed	
  in	
  terms	
  of	
  the	
  coupled	
  
average	
  uncertainty	
  (32).	
  	
  To	
  facilitate	
  the	
  comparison	
  between	
  the	
  entropy	
  
ακ
functions	
  a	
  substitution	
  is	
  made	
  for	
  the	
  Rényi	
  order	
  and	
  Tsallis	
  index	
   q → 1+ 1+
.	
  	
  
κ
!
Table	
  1	
  summarizes	
  the	
  properties	
  of	
  the	
  transformation	
  from	
  an	
  average	
  
uncertainty	
  to	
  an	
  entropy	
  in	
  terms	
  of	
  the	
  moment	
  of	
  the	
  average,	
  and	
  the	
  power	
  and	
  
normalization	
  associated	
  with	
  the	
  generalized	
  logarithm,	
  	
  

()
()

1

	
  

⎛ N
⎞ Moment
−1
S ( p, Norm, Power, Moment ) =
GM Power − 1 , GM = ⎜ ∑ piMoment+1 ⎟
. 	
  (43)	
  
Norm
⎝ i=1
⎠

(

)

For	
  the	
  Rényi	
  entropy	
  (R)	
  the	
  relationship	
  with	
  the	
  coupled	
  average	
  uncertainty	
  is	
  
evident	
  given	
  its	
  use	
  of	
  the	
  natural	
  logarithm	
  
1+κ

	
  
	
  

SαR ,κ
!

⎛ N 1+ ακ ⎞ ακ
= −ln ⎜ ∑ pi 1+κ ⎟ .	
  	
  
⎝ i=1
⎠
	
  

	
  
14	
  

(44)	
  

Table	
  1	
  	
  Comparison	
  of	
  Generalized	
  Entropies	
  	
  The	
  generalized	
  entropy	
  
functions	
  are	
  expressed	
  in	
  terms	
  of	
  three	
  components,	
  the	
  moment	
  of	
  the	
  
generalized	
  mean,	
  the	
  power	
  and	
  the	
  normalization	
  (norm)of	
  the	
  generalized	
  
logarithm.	
  	
  

Entropy

Moment

Power

ακ
1+ κ
ακ
1+ κ
ακ
1+ κ

−ακ
1+ κ
ακ
1+ κ
−ακ
1+ κ

ακ
1+ κ
lim

power→0

Coupled
Tsallis
Normalized!
Tsallis
Renyi
Shannon

−ακ

ακ
1+ κ
−ακ
1+ κ

lim
lim

moment→0

Norm.

power→0

lim

norm→0

lim

norm→0

	
  
!
	
  
The	
  coupled	
  entropy	
  is	
  most	
  closely	
  related	
  to	
  the	
  normalized	
  Tsallis	
  entropy	
  (NT).	
  	
  
Expressed	
  in	
  terms	
  of	
  the	
  coupled	
  surprisal	
  (37)	
  the	
  normalized	
  Tsallis	
  entropy	
  is	
  
seen	
  to	
  be	
  the	
  coupled	
  entropy	
  (39)	
  multiplied	
  by	
   1+ κ 	
  	
  
!

(

	
  

()

SαNT,κ p ≡

(

− 1+ κ

)

N

∑ pi

1+

i=1
N

ακ
1+κ

ln −α ,κ pi

ακ
1+κ

)

1+κ

⎛ N 1+ ακ ⎞ ακ
= − 1+ κ ln −α ,κ ⎜ ∑ pi 1+κ ⎟ .	
  	
  
⎝ i=1
⎠

(

)

(45)	
  

∑ pi
i=1
!
Using	
  the	
  unnormalized	
  form	
  of	
  Equation	
  (45)	
  for	
  the	
  Tsallis	
  entropy	
  
definition,	
  the	
  lack	
  of	
  normalization	
  is	
  shown	
  to	
  be	
  equivalent	
  to	
  using	
  the	
  opposite	
  
sign	
  of	
  the	
  multiplier	
  for	
  the	
  transformation	
  
1+

1+κ

ακ
⎛ N 1+1+
⎞ ακ
T
κ
	
  
(46)	
  
Sα ,κ p ≡ − 1+ κ ∑ pi
ln −α ,κ pi = 1+ κ lnα ,κ ⎜ ∑ pi
⎟ . 	
  
⎝ i=1
⎠
i=1
!
Derivation	
  of	
  the	
  right-­‐hand	
  expression	
  is	
  shown	
  in	
  Appendix	
  A.	
  	
  The	
  coupled	
  
entropy	
  (C),	
  normalized	
  Tsallis,	
  and	
  Tsallis	
  entropy	
  have	
  then	
  the	
  following	
  
relationship	
  
SαNT,κ
SαT ,κ
C
S
=
=
. 	
  
	
  
(47)	
  
α ,κ
ακ
N
1+
1+ κ
1+κ
1+ κ ∑ pi
i=1
!
Figure	
  3	
  and	
  Figure	
  4	
  show	
  comparisons	
  of	
  the	
  entropy	
  functions	
  for	
  the	
  coupled	
  
exponential	
  distribution	
  and	
  the	
  coupled	
  Gaussian	
  distribution	
  respectively.	
  	
  The	
  
computations	
  for	
  the	
  two	
  figures	
  are	
  summarized	
  in	
  Appendix	
  B.	
  	
  For	
  the	
  coupled	
  
exponential	
  distribution	
  (Equation	
  (21)	
  with	
  !α = 1 )	
  with	
  the	
  scale	
   σ 	
  equal	
  to	
  one,	
  
both	
  the	
  coupled	
  and	
  Tsallis	
  entropy	
  are	
  equal	
  to	
  one.	
  	
  In	
  this	
  case,	
  the	
  normalized	
  
Tsallis	
  entropy	
  is	
  linear	
  and	
  similar	
  to	
  the	
  Shannon	
  entropy.	
  	
  As	
  shown	
  on	
  the	
  right	
  

() (

)

N

1+

ακ
1+κ

(

(

	
  
15	
  

)

)

of	
  Figure	
  3,	
  even	
  as	
  the	
  scale	
  varies	
  the	
  coupled	
  entropy	
  does	
  not	
  vary	
  significantly	
  
with	
  the	
  coupling.	
  	
  This	
  is	
  suggestive	
  that	
  the	
  coupled	
  entropy	
  represents	
  a	
  
stabilization	
  of	
  the	
  entropy	
  as	
  a	
  function	
  of	
  the	
  coupling,	
  in	
  contrast	
  to	
  the	
  Shannon	
  
and	
  Renyi	
  entropies	
  which	
  have	
  close	
  to	
  linear	
  growth	
  with	
  the	
  coupling.	
  	
  However,	
  
this	
  property	
  does	
  not	
  hold	
  for	
  the	
  coupled	
  Gaussian	
  distribution	
  (Equation	
  (21)	
  
with	
  !α = 2 )	
  shown	
  in	
  Figure	
  4.	
  	
  Instead,	
  the	
  coupled	
  entropy	
  is	
  quite	
  sensitive	
  to	
  
changes	
  in	
  both	
  the	
  coupling	
  and	
  the	
  scale.	
  	
  With	
  scale	
  equal	
  to	
  one,	
  the	
  coupled	
  
entropy	
  is	
  similar	
  to	
  the	
  Shannon	
  entropy,	
  particularly	
  when	
  !0 < κ < 0.5 .	
  	
  For	
  !κ > 0.5 ,	
  
the	
  difference	
  is	
  more	
  evident	
  as	
  the	
  coupled	
  entropy	
  is	
  slightly	
  super-­‐linear	
  and	
  the	
  
Shannon	
  entropy	
  is	
  slightly	
  sub-­‐linear.	
  	
  For	
  this	
  case,	
  the	
  normalized	
  Tsallis	
  entropy	
  
is	
  close	
  to	
  quadratic	
  with	
  the	
  coupling.	
  	
  The	
  Renyi	
  entropy	
  has	
  close	
  to	
  linear	
  growth	
  
for	
  both	
  distributions,	
  but	
  with	
  a	
  slope	
  that	
  is	
  less	
  than	
  the	
  Shannon	
  entropy.	
  	
  The	
  
sensitivity	
  of	
  the	
  coupled	
  entropy	
  with	
  !α ≠ 1 	
  would	
  be	
  reduced	
  if	
  Equation	
  (38)	
  is	
  
used	
  for	
  the	
  generalized	
  logarithm,	
  since	
  the	
  root	
  with	
  respect	
  to	
   α 	
  is	
  taken.	
  

5 Conclusion	
  
	
  

The	
  nonlinear	
  statistical	
  coupling	
  of	
  states	
  is	
  defined	
  in	
  terms	
  of	
  a	
  source	
  of	
  
nonlinear	
  coupling	
  κ	
  in	
  the	
  system.	
  	
  Positive	
  coupling	
  models	
  non-­‐stationary	
  
systems	
  in	
  which	
  the	
  variance	
  fluctuates.	
  	
  The	
  negative	
  domain	
  of	
  κ,	
  which	
  has	
  
decreased	
  variability,	
  is	
  limited	
  to	
   ( −1,0) ;	
  and	
   κ = 0 	
  is	
  the	
  linear	
  domain	
  with	
  
exponential	
  distributions.	
  	
  The	
  maximum	
  entropy	
  distribution	
  for	
  these	
  systems	
  
given	
  a	
  constrained	
  location	
  and	
  scale	
  is	
  the	
  coupled-­‐exponential	
  distributions,	
  
defined	
  in	
  Equation	
  (21).	
  	
  In	
  addition	
  to	
  the	
  nonlinear	
  coupling	
  these	
  distributions	
  
− xα

depend	
  on	
  the	
  stability	
  index	
  of	
  the	
  stretched	
  exponential	
  !e α .	
  	
  The	
  coupled	
  sum	
  
is	
  defined	
  by	
  the	
  nonlinear	
  coupling,	
  while	
  the	
  coupled	
  product	
  is	
  a	
  function	
  of	
  the	
  
−ακ
coupling,	
  the	
  stability	
  index	
  and	
  the	
  dimensions	
  
.	
  	
  In	
  this	
  paper	
  the	
  one-­‐
!1+ dκ
dimensional	
  case	
   d = 1 	
  was	
  examined.	
  	
  	
  
In	
  this	
  paper	
  the	
  coupled	
  average	
  uncertainty	
  in	
  the	
  probability/density	
  
ακ
space	
  is	
  introduced	
  as	
  the	
  weighted	
  generalized	
  mean	
  with	
  a	
  power	
  equal	
  to	
   1+
.	
  	
  
! κ
The	
  expression	
  is	
  derived	
  from	
  and	
  is	
  expressible	
  in	
  terms	
  of	
  deformation	
  of	
  the	
  
coupled	
  product	
  (18)	
  and	
  the	
  coupled	
  power	
  (19)	
  functions	
  
	
  
	
  
	
  

N

Pκ avg ≡ ∏ ⊗ pi
ακ 1
i=1
!

(− ακ 1 )

P
⊗Ki

	
  

	
  
16	
  

1+κ

⎛ N 1+ ακ ⎞ ακ
= ⎜ ∑ pi 1+κ ⎟ . 	
  	
  
⎝ i=q
⎠

(48)	
  

	
  
3.0

Normalized	
  Tsallis

Entropy

2.5

-­‐-­‐	
  	
  Shannon
−	
  	
  Coupled

3

Shannon

2.0

2

1.5

Renyi

1.0

1

Coupled	
  and	
  Tsallis

0.5
0.0
0.0

0
0.5

1.0

1.5

0.5

2.0

Coupling

1.0

1.5

2.0

Coupling

Figure	
  3	
  	
  Comparison	
  of	
  Entropy	
  Functions	
  for	
  Coupled	
  Exponential	
  Distribution	
  Left)	
  	
  The	
  entropy	
  versus	
  
coupling	
  with	
  the	
  scale	
  equal	
  to	
  one.	
  	
  The	
  Coupled	
  and	
  Tsallis	
  entropies	
  are	
  both	
  equal	
  to	
  1.	
  	
  The	
  Shannon	
  and	
  
Renyi	
  entropy	
  have	
  close	
  to	
  linear	
  growth.	
  	
  The	
  Normalized	
  Tsallis	
  entropy	
  is	
  similar	
  to	
  Shannon	
  entropy.	
  	
  
Right)	
  	
  Comparison	
  of	
  Shannon	
  (dashed)	
  and	
  Coupled	
  (solid)	
  entropy	
  for	
  !σ = {0.25,0.5,0.75,1,1.25,1.5,1.75,2} .	
  	
  
Both	
  entropies	
  increase	
  with	
  the	
  scale	
  !σ . 	
  The	
  Coupled	
  entropy	
  does	
  not	
  vary	
  significantly	
  with	
  the	
  coupling,	
  
while	
  the	
  Shannon	
  entropy	
  has	
  close	
  to	
  linear	
  growth.	
  

	
  

	
  

Entropy

4

4

Normalized	
   Coupled
Tsallis

3

3

Shannon

2

-­‐-­‐	
  	
  Shannon
−	
  	
  Coupled

2

Renyi

1

1
Tsallis

0
0.0

0.5

1.0

1.5

Coupling

0
0.0

2.0

0.5

1.0

1.5

Coupling

2.0

Figure	
  4	
  Comparison	
  of	
  Entropy	
  Functions	
  for	
  Coupled	
  Gaussian	
  Distribution	
  	
  Left)	
  	
  The	
  entropy	
  versus	
  
coupling	
  with	
  the	
  scale	
  equal	
  to	
  one.	
  	
  The	
  Coupled	
  and	
  Shannon	
  entropy	
  are	
  similar	
  with	
  close	
  to	
  linear	
  growth.	
  	
  
The	
  Renyi	
  entropy	
  also	
  has	
  close	
  linear	
  growth	
  but	
  with	
  a	
  smaller	
  slope.	
  	
  The	
  Normalized	
  Tsallis	
  entropy	
  has	
  
grow	
  close	
  to	
  the	
  square	
  of	
  the	
  coupling,	
  while	
  the	
  Tsallis	
  entropy	
  decays.	
  	
  	
  Right)	
  	
  Comparison	
  of	
  Shannon	
  
(dashed)	
  and	
  Coupled	
  (solid)	
  entropy	
  for	
  !σ = {0.25,0.5,0.75,1,1.25,1.5,1.75,2} .	
  	
  Both	
  entropies	
  increase	
  with	
  the	
  
scale	
  !σ . 	
  The	
  Coupled	
  entropy	
  has	
  increased	
  sensitivity	
  with	
   σ as	
  the	
  coupling	
  increase,	
  while	
  the	
  Shannon	
  
entropy	
  has	
  close	
  to	
  linear	
  growth.	
  

	
  

	
  

	
  
17	
  

	
  

For κ = 0 	
  this	
  is	
  the	
  weighted	
  geometric	
  mean	
  in	
  which	
  each	
  state	
  is	
  treated	
  as	
  
independent	
  and	
  thus	
  multiplied,	
  and	
  the	
  dependency	
  from	
  the	
  probabilities	
  
summing	
  to	
  one	
  is	
  contained	
  in	
  the	
  weight	
  of	
  each	
  probability.	
  	
  When κ ≠ 0 	
  the	
  
coupled	
  product	
  models	
  the	
  nonlinear	
  dependency	
  between	
  the	
  probabilities	
  of	
  the	
  
distribution	
  and	
  the	
  weights	
  are	
  now	
  the	
  deformed	
  coupled	
  probability.	
  	
  
Importantly,	
  for	
  the	
  coupled	
  exponential	
  distributions	
  the	
  coupled	
  average	
  
−1

()

probability	
  is	
  equal	
  to	
  the	
  density	
  at	
  the	
  location	
  plus	
  the	
  scale Pκ avg = 1Z eκ α 1 .	
  	
  The	
  
!
coupled	
  average	
  probability	
  has	
  been	
  successfully	
  utilized	
  in	
  the	
  analysis	
  of	
  
information	
  fusion	
  algorithms	
  [8,41]	
  and	
  is	
  recommended	
  as	
  a	
  simple	
  approach	
  to	
  
understanding	
  the	
  average	
  uncertainty	
  for	
  nonlinear	
  systems.	
  
Upon	
  transformation	
  to	
  the	
  probability	
  domain	
  the	
  Tsallis,	
  normalized	
  Tsallis	
  
and	
  Rényi	
  entropies	
  each	
  fulfill	
  the	
  relationship	
  between	
  the	
  coupled	
  average	
  
uncertainty	
  and	
  the	
  density	
  at	
  the	
  width	
  of	
  the	
  coupled	
  distributions.	
  	
  The	
  
potentially	
  improved	
  properties	
  of	
  the	
  coupled-­‐entropy	
  are	
  achieved	
  by	
  requiring	
  
the	
  coupled	
  logarithm	
  with	
   α = 1 	
  to	
  have	
  a	
  unit	
  integral.	
  	
  The	
  coupled	
  entropy	
  is	
  
defined	
  as	
  the	
  transformation	
  of	
   Pκ avg 	
  to	
  the	
  entropy	
  scale	
  by	
  the	
  coupled	
  logarithm,	
  	
  
!
−1
1+κ
⎛ ⎛ N (1+ακ )
⎞
⎞
ακ
− ακ
⎛ N 1+1+
⎞
1
1+
κ
(
)
⎜⎜ p
⎟ −1⎟ . 	
  	
  
Sα ,κ p ≡ −ln −α ,κ ⎜ ∑ pi κ ⎟
=
	
  
(49)	
  
∑
⎟
ακ ⎜⎜ ⎜ i=1 i
⎟
⎝ i=1
⎠
⎟
⎠
⎝⎝
⎠
!
For	
  the	
  coupled	
  exponential	
  distribution,	
  the	
  matching	
  coupled	
  entropy	
  does	
  not	
  
vary	
  significantly	
  with	
  the	
  degree	
  of	
  coupling.	
  	
  In	
  contrast,	
  for	
  the	
  Gaussian	
  
distribution,	
  the	
  matching	
  coupled	
  entropy	
  is	
  sensitive	
  to	
  both	
  the	
  coupling	
  and	
  
scale	
  of	
  the	
  distribution.	
  	
  Further	
  analysis	
  and	
  experimentation	
  is	
  recommended	
  to	
  
determine	
  how	
  the	
  coupled	
  entropy	
  can	
  be	
  used	
  to	
  model	
  the	
  effects	
  of	
  non-­‐
stationary	
  uncertainty	
  in	
  complex	
  thermodynamic	
  and	
  information	
  systems.	
  

()

	
  
18	
  

Appendices	
  
A. Derivation	
  of	
  coupled	
  entropy	
  from	
  average	
  uncertainty	
  
The	
  coupled	
  algebra	
  can	
  be	
  used	
  to	
  derive	
  the	
  equivalency	
  between	
  the	
  two	
  
expressions	
  for	
  the	
  coupled	
  entropy	
  
1+κ
ακ

⎛
⎞
Sα ,κ p ≡ −ln −α ,κ ⎜ ∑ p
⎟
⎝ i=1
⎠
!
(α ,κ )
where Pi is	
  the	
  coupled	
  probability	
  (22)	
  
!
N

()

	
  

ακ
1+
1+κ
i

1+

p
α ,κ
P( ) = i

	
  

N

α ,κ
= − ∑ Pi( ) ln −α ,κ pi 	
  	
  
i=1

ακ
1+κ

N

i

∑p

1+
j

	
  	
  

ακ
1+κ

j=1
!
Starting	
  with	
  the	
  expression	
  on	
  the	
  right,	
  the	
  coupled	
  power	
  is	
  used	
  to	
  move	
  the	
  
coupled	
  probability	
  from	
  a	
  multiplier	
  of	
  the	
  generalized	
  logarithm	
  to	
  a	
  power	
  of	
  the	
  
⊗b− ακ

argument	
  using	
  the	
  property	
   bln −α ,κ x = ln −α ,κ x
!
N

()

⊗

Sα ,κ p = − ∑ ln −α ,κ pi
i=1
!

	
  

(1+κ )

(

	
  

(

− ακ / 1+κ
Pi
− ακ 1+κ

(

)

))

.
N

N

Expanding	
  the	
  coupled	
  power	
  and	
  using	
  the	
  property	
   ∑ ln −α ,κ x = ln −α ,κ ∏ ⊗ x ,	
  
− ακ
i=1
1+κ
! i=1
⎛ α ,κ
Sα ,κ p = −ln −α ,κ ∏ ⊗ ⎜ Pi( ) p
− ακ
⎝
i=1
1+κ
!

()

	
  

− ακ
1+κ
i

N

N

((

Expanding	
  the	
  coupled	
  product	
  and	
  reducing	
   ∑ Pi
! i=1
⎛ N α ,κ
Sα ,κ p = −ln −α ,κ ⎜ ∑ Pi( ) p
⎝ i=1

− ακ
1+κ
i

()

	
  

⎛
= −ln −α ,κ ⎜ ∑ p
⎝ i=1
N

	
  

!

ακ
1+
1+κ
i

⎞
⎟
⎠

⎞
⎟
⎠

1+κ
ακ

.

	
  
19	
  

1+κ
− ακ

(

)

⎞
α ,κ
− Pi( ) −1 ⎟
⎠
α ,κ

1+κ
− ακ

. 	
  	
  

)

) −1 = 1− N
	
  
,	
  	
  

⎛ N
⎜ ∑ pi
= −ln −α ,κ ⎜ N i=1 ακ
1+
⎜
1+κ
⎜⎝ ∑ pi
i=1

⎞
⎟
⎟
⎟
⎟⎠

1+κ
− ακ

	
  	
  	
  

	
  
Since	
  the	
  normalized	
  Tsallis	
  entropy	
  is	
  just	
  a	
  multiple	
  of	
  the	
  coupled	
  entropy,	
  
the	
  derivation	
  is	
  not	
  repeated.	
  	
  The	
  derivation	
  of	
  the	
  Tsallis	
  entropy	
  starts	
  with	
  the	
  
middle	
  expression	
  of	
  equation	
  (46)	
  	
  

() (

)

N

ακ
1+κ

S p ≡ − 1+ κ ∑ pi
ln −α ,κ pi .	
  
i=1
!
The	
  relationship	
  is	
  expressed	
  in	
  terms	
  of	
  the	
  coupled	
  product	
  and	
  coupled	
  power	
  	
  
T
α ,κ

	
  

() (

)

1+

N

⊗

SαT ,κ p = − 1+ κ ln −α ,κ ∏ ⊗ pi
− ακ
i=1
1+κ
!
Expanding	
  the	
  coupled	
  power	
  and	
  the	
  coupled	
  power	
  

	
  

ακ
1+
pi 1+κ
− ακ 1+κ

(

)

. 	
  	
  

1+κ

SαT ,κ

	
  

⎛ N ⎛
⎞ −ακ
⎛ 1+ ακ
⎞⎞
p = − 1+ κ ln −α ,κ ⎜ ∑ ⎜ pi − ⎜ pi 1+κ −1⎟ ⎟ − N −1 ⎟ . 	
  
⎝
⎠⎠
⎝ i=1 ⎝
⎠

() (

)

(

)

!
Simplifying	
  and	
  using	
  the	
  relationship	
   − ln −α ,κ x − b = lnα ,κ x b 	
  	
  
1+κ

SαT ,κ
!

	
  

⎛ N 1+ ακ ⎞ ακ
p = 1+ κ lnα ,κ ⎜ ∑ pi 1+κ ⎟ . 	
  	
  
⎝ i=1
⎠

() (

)

B. Comparison	
  of	
  generalized	
  entropy	
  functions	
  

The	
  comparison	
  of	
  entropy	
  functions	
  for	
  the	
  Coupled	
  Exponential	
  and	
  Coupled	
  
Gaussian	
  distributions	
  shown	
  in	
  Figure	
  3	
  and	
  Figure	
  4	
  includes	
  the	
  following	
  
computations.	
  	
  The	
  distributions	
  analyzed	
  are	
  

) (1+κ )

(

expdistκ x; µ ,σ ≡
!

	
  
	
  	
  

1
σ

x− µ
σ

⎛1 ⎞
−⎜ +1⎟
⎝κ ⎠

	
  	
  

(50)	
  

1⎛ 1 ⎞
− ⎜ +1⎟

2
2⎝ κ ⎠
κ Γ ⎡⎣ 1+2κκ ⎤⎦ ⎛
⎛ x −µ⎞ ⎞
Gκ x; µ ,σ ≡
. 	
  	
  
	
  
(51)	
  
⎜ 1+ κ ⎜
⎟
⎝ σ ⎟⎠ ⎟⎠
π Γ ⎡⎣ 21κ ⎤⎦ ⎜⎝
!
The	
  computations	
  were	
  completed	
  using	
  MathematicaTM.	
  	
  For	
  the	
  Coupled,	
  Tsallis,	
  
and	
  Normalized	
  Tsallis	
  entropy	
  with	
  matching	
  coupling,	
  closed	
  form	
  solutions	
  exist.	
  	
  
The	
  entropy	
  functions	
  of	
  the	
  coupled	
  exponential	
  with	
  !σ = 1 	
  reduce	
  to	
  a	
  linear	
  
expression.	
  
	
  
a) Coupled	
  Entropy	
  

(

	
  

)

( )) = (
(
(expdist ( x;µ ,1)) = 1

S1,Cκ expdistκ x; µ ,σ
!S1,Cκ
!

κ

	
  
20	
  

)

κ

−1+ 1+ κ σ 1+κ

κ

	
  	
  

(52)	
  

	
  
	
  

( (

S2,Cκ Gκ x; µ ,σ
!

(
(

T
1,κ

S

S1,Tκ
!

	
  

)( )

(

−κ + 1+ κ κπ
=

⎛ Γ
⎜σ
⎝ Γ

( )
( )
1
2κ

1+κ
2κ

2κ

⎞ 1+κ
⎟
⎠

2κ 2

	
  	
  

(53)	
  

b) Tsallis	
  Entropy	
  	
  

	
  

	
  

))

1
κ 1+κ

( (

−κ

))
))

1+ κ − σ 1+κ
expdistκ x; µ ,σ =
	
  	
  
κ
expdistκ x; µ ,1 = 1

S2,Tκ Gκ x; µ ,σ

(
(

))

!

⎛
1⎜
1
= ⎜ 1+ − κπ κ
2
κ
⎜⎝

( )

−1
1+κ

(54)	
  

⎛ σΓ ⎡ ⎤ ⎞
⎣ ⎦
⎜
⎟
1+κ
⎝ Γ ⎡⎣ 2κ ⎤⎦ ⎠
1
2κ

−

2κ
1+κ

⎞
⎟
⎟ 	
  	
  
⎟⎠

(55)	
  

c) Normalized	
  Tsallis	
  	
  

	
  

NT
1,κ

S

S1,NTκ
!

(
(

κ
⎛
⎞
1+ κ ⎜ −1+ 1+ κ σ 1+κ ⎟
⎝
⎠
expdistκ x; µ ,σ =
	
  	
  
κ
expdistκ x; µ ,1 = 1+ κ

	
  

( (

S2,NTκ Gκ x; µ ,σ
!

))

)

(

⎛
⎜
1+ κ ⎜ −κ + 1+ κ κπ κ
⎜⎝
=
2κ 2

(

	
  

))
))

(
(

(

)

(

)( )

)

1
1+κ

(56)	
  

2κ
⎞
⎛ σΓ 21κ ⎞ 1+κ ⎟
⎜
⎟ ⎟
1+κ
⎝ Γ 2κ ⎠ ⎟
⎠

( )
( )

	
  	
  

(57)	
  

Closed	
  form	
  solutions	
  do	
  not	
  exist	
  for	
  the	
  Shannon	
  and	
  Renyi	
  entropies.	
  	
  The	
  
numerical	
  integration	
  of	
  the	
  Shannon	
  entropy	
  has	
  potential	
  errors	
  for	
  a)	
  small	
  
values	
  of	
  κ	
  when	
  the	
  integration	
  extends	
  to	
  regions	
  where	
   − ln f ( x )

f (x )

	
  is	
  undefined	
  

or	
  b)	
  large	
  values	
  of	
  κ	
  when	
  the	
  integration	
  is	
  truncated	
  before	
   − ln f ( x )

f (x )

has	
  

decayed	
  sufficiently	
  to	
  be	
  excluded.	
  	
  For	
  this	
  reason	
  the	
  Shannon	
  entropy	
  was	
  
determined	
  as	
  a	
  piecewise	
  numerical	
  integration	
  with	
  the	
  following	
  limits	
  

x !Limit
	
  

	
  

	
  

!

Coupling

100
0.00 < κ < 0.09
1000 0.09 ≤ κ < 0.74 	
  	
  
10,000 0.74 ≤ κ < 1.50
15,000 1.50 ≤ κ ≤ 2.00.

	
  
21	
  

(58)	
  

References	
  
[1]	
  

T.M.	
  Cover,	
  J.A.	
  Thomas,	
  Elements	
  of	
  information	
  theory,	
  John	
  Wiley	
  &	
  Sons,	
  
2012.	
  

[2]	
  

A.I.	
  Khinchin,	
  Mathematical	
  Foundations	
  of	
  Statistical	
  Mechanics,	
  Courier	
  
Corporation,	
  1949.	
  

[3]	
  

A.I.	
  Khinchin,	
  Mathematical	
  Foundations	
  of	
  Information	
  Theory,	
  Courier	
  
Corporation,	
  1957.	
  

[4]	
  

A.	
  Rényi,	
  On	
  measures	
  of	
  entropy	
  and	
  information,	
  Fourth	
  Berkeley	
  Symp.	
  
Math.	
  Stat.	
  Probab.	
  1	
  (1961)	
  547–561.	
  

[5]	
  

C.	
  Tsallis,	
  Possible	
  generalization	
  of	
  Boltzmann-­‐Gibbs	
  statistics,	
  J.	
  Stat.	
  Phys.	
  
52	
  (1988)	
  479–487.	
  

[6]	
  

S.I.	
  Amari,	
  H.	
  Nagaoka,	
  Methods	
  of	
  Information	
  Geometry,	
  Oxford	
  University	
  
Press,	
  2000.	
  

[7]	
  

C.	
  Anteneodo,	
  C.	
  Tsallis,	
  A.S.	
  Martinez,	
  Risk	
  aversion	
  in	
  trade	
  transactions,	
  Eur.	
  
Lett.	
  59	
  (2002)	
  635–641.	
  

[8]	
  

K.P.	
  Nelson,	
  B.J.	
  Scannell,	
  H.	
  Landau,	
  A	
  Risk	
  Profile	
  for	
  Information	
  Fusion	
  
Algorithms,	
  Entropy.	
  13	
  (2011)	
  1518–1532.	
  

[9]	
  

I.	
  Santamaria,	
  D.	
  Erdogmus,	
  J.C.	
  Principe,	
  Entropy	
  minimization	
  for	
  supervised	
  
digital	
  communications	
  channel	
  equalization,	
  IEEE	
  Trans.	
  Signal	
  Process.	
  50	
  
(2002)	
  1184–1192.	
  doi:10.1109/78.995074.	
  

[10]	
   W.R.	
  Carson,	
  M.	
  Chen,	
  M.R.D.	
  Rodrigues,	
  R.	
  Calderbank,	
  L.	
  Carin,	
  
Communications-­‐Inspired	
  Projection	
  Design	
  with	
  Application	
  to	
  Compressive	
  
Sensing,	
  SIAM	
  J.	
  Imaging	
  Sci.	
  5	
  (2012)	
  1185–1212.	
  doi:10.1137/120878380.	
  
[11]	
   E.P.	
  Borges,	
  C.	
  Tsallis,	
  G.F.J.	
  Ananos,	
  P.M.C.	
  de	
  Oliveira,	
  Nonequilibrium	
  
probabilistic	
  dynamics	
  of	
  the	
  logistic	
  map	
  at	
  the	
  edge	
  of	
  chaos,	
  Phys.	
  Rev.	
  Lett.	
  
89	
  (2002)	
  254103.	
  
[12]	
   G.	
  Livadiotis,	
  D.J.	
  McComas,	
  Beyond	
  kappa	
  distributions:	
  Exploiting	
  Tsallis	
  
statistical	
  mechanics	
  in	
  space	
  plasmas,	
  J.	
  Geophys.	
  Res.	
  114	
  (2009)	
  A11105.	
  
doi:10.1029/2009JA014352.	
  
[13]	
   L.F.	
  Burlaga,	
  A.F.	
  Vinas,	
  Triangle	
  for	
  the	
  entropic	
  index	
  q	
  of	
  non-­‐extensive	
  
statistical	
  mechanics	
  observed	
  by	
  Voyager	
  1	
  in	
  the	
  distant	
  heliosphere,	
  Phys.	
  
A	
  Stat.	
  Mech.	
  Its	
  Appl.	
  356	
  (2005)	
  375–384.	
  
[14]	
   S.	
  Chatrchyan,	
  V.	
  Khachatryan,	
  A.M.	
  Sirunyan,	
  A.	
  Tumasyan,	
  W.	
  Adam,	
  T.	
  
Bergauer,	
  et	
  al.,	
  Charged	
  particle	
  transverse	
  momentum	
  spectra	
  in	
  pp	
  
collisions	
  at	
   s = 0.9 	
  and	
  7	
  TeV,	
  J.	
  High	
  Energy	
  Phys.	
  2011	
  (2011)	
  86.	
  
doi:10.1007/JHEP08(2011)086.	
  
	
  

	
  
22	
  

[15]	
   V.	
  Khachatryan,	
  A.M.	
  Sirunyan,	
  A.	
  Tumasyan,	
  W.	
  Adam,	
  T.	
  Bergauer,	
  M.	
  
Dragicevic,	
  et	
  al.,	
  Transverse-­‐momentum	
  and	
  pseudorapidity	
  distributions	
  of	
  
charged	
  hadrons	
  in	
  pp	
  collisions	
  at	
  square	
  root	
  of	
  s	
  =	
  7	
  TeV.,	
  Phys.	
  Rev.	
  Lett.	
  
105	
  (2010)	
  022002.	
  doi:10.1103/PhysRevLett.105.022002.	
  
[16]	
   N.	
  Canosa,	
  R.	
  Rossignoli,	
  Generalized	
  nonadditive	
  entropies	
  and	
  quantum	
  
entanglement,	
  Phys.	
  Rev.	
  Lett.	
  88	
  (2002)	
  170401.	
  
[17]	
   R.	
  Hanel,	
  S.	
  Thurner,	
  M.	
  Gell-­‐Mann,	
  Generalized	
  entropies	
  and	
  the	
  
transformation	
  group	
  of	
  superstatistics,	
  Proc.	
  Natl.	
  Acad.	
  Sci.	
  108	
  (2011)	
  
6390.	
  
[18]	
   R.	
  Hanel,	
  S.	
  Thurner,	
  A	
  comprehensive	
  classification	
  of	
  complex	
  statistical	
  
systems	
  and	
  an	
  axiomatic	
  derivation	
  of	
  their	
  entropy	
  and	
  distribution	
  
functions,	
  EPL	
  (Europhysics	
  Lett.	
  93	
  (2011)	
  20006.	
  
[19]	
   A.	
  Rényi,	
  On	
  the	
  Foundations	
  of	
  Information	
  Theory,	
  Rev.	
  Int.	
  Stat.	
  Inst.	
  33	
  
(1965)	
  1–14.	
  
[20]	
   C.	
  Tsallis,	
  Introduction	
  to	
  Nonextensive	
  Statistical	
  Mechanics:	
  Approaching	
  a	
  
Complex	
  World,	
  Springer	
  Verlag,	
  2009.	
  
[21]	
   K.P.	
  Nelson,	
  S.	
  Umarov,	
  Nonlinear	
  statistical	
  coupling,	
  Phys.	
  A	
  Stat.	
  Mech.	
  Its	
  
Appl.	
  389	
  (2010)	
  2157–2163.	
  
[22]	
   T.J.	
  Arruda,	
  R.S.	
  González,	
  C.A.S.	
  Terçariol,	
  A.S.	
  Martinez,	
  Arithmetical	
  and	
  
geometrical	
  means	
  of	
  generalized	
  logarithmic	
  and	
  exponential	
  functions:	
  
generalized	
  sum	
  and	
  product	
  operators,	
  Phys.	
  Lett.	
  A.	
  372	
  (2008)	
  2578–2582.	
  
[23]	
   Q.A.	
  Wang,	
  L.	
  Nivanen,	
  A.	
  Le	
  Mehaute,	
  M.	
  Pezeril,	
  On	
  the	
  generalized	
  entropy	
  
pseudoadditivity	
  for	
  complex	
  systems,	
  J.	
  Phys.	
  A.	
  Math.	
  Gen.	
  35	
  (2002)	
  7003–
7007.	
  
[24]	
   G.	
  Kaniadakis,	
  M.	
  Lissia,	
  A.M.	
  Scarfone,	
  Two-­‐parameter	
  deformations	
  of	
  
logarithm,	
  exponential,	
  and	
  entropy:	
  A	
  consistent	
  framework	
  for	
  generalized	
  
statistical	
  mechanics,	
  Phys.	
  Rev.	
  E.	
  71	
  (2005)	
  046128.	
  
doi:10.1103/PhysRevE.71.046128.	
  
[25]	
   K.S.	
  Narendra,	
  K.	
  Parthasarathy,	
  Identification	
  and	
  control	
  of	
  dynamical	
  
systems	
  using	
  neural	
  networks.,	
  IEEE	
  Trans.	
  Neural	
  Netw.	
  1	
  (1990)	
  4–27.	
  
doi:10.1109/72.80202.	
  
[26]	
   S.H.	
  Strogatz,	
  Exploring	
  complex	
  networks.,	
  Nature.	
  410	
  (2001)	
  268–76.	
  
doi:10.1038/35065725.	
  
[27]	
   T.	
  Oikonomou,	
  Tsallis,	
  Renyi	
  and	
  nonextensive	
  Gaussian	
  entropy	
  derived	
  from	
  
the	
  respective	
  multinomial	
  coefficients,	
  Phys.	
  A	
  Stat.	
  Mech.	
  Its	
  Appl.	
  386	
  
(2007)	
  119–134.	
  
[28]	
   K.P.	
  Nelson,	
  A	
  definition	
  of	
  the	
  coupled-­‐product	
  for	
  multivariate	
  coupled-­‐
exponentials,	
  Phys.	
  A	
  Stat.	
  Mech.	
  Its	
  Appl.	
  422	
  (2015)	
  187–192.	
  
doi:10.1016/j.physa.2014.12.023.	
  

	
  
23	
  

[29]	
   E.P.	
  Borges,	
  A	
  possible	
  deformed	
  algebra	
  and	
  calculus	
  inspired	
  in	
  
nonextensive	
  thermostatistics,	
  Phys.	
  A	
  Stat.	
  Mech.	
  Its	
  Appl.	
  340	
  (2004)	
  95–
101.	
  
[30]	
   C.	
  Anteneodo,	
  C.	
  Tsallis,	
  Multiplicative	
  noise:	
  A	
  mechanism	
  leading	
  to	
  
nonextensive	
  statistical	
  mechanics,	
  J.	
  Math.	
  Phys.	
  44	
  (2003)	
  5194.	
  
[31]	
   C.	
  Beck,	
  E.G.D.	
  Cohen,	
  Superstatistics,	
  Phys.	
  A	
  Stat.	
  Mech.	
  Its	
  Appl.	
  322	
  (2003)	
  
267–275.	
  
[32]	
   G.	
  Wilk,	
  Z.	
  Włodarczyk,	
  Interpretation	
  of	
  the	
  Nonextensivity	
  Parameter	
  q	
  in	
  
Some	
  Applications	
  of	
  Tsallis	
  Statistics	
  and	
  Lévy	
  Distributions,	
  Phys.	
  Rev.	
  Lett.	
  
84	
  (2000)	
  2770–2773.	
  doi:10.1103/PhysRevLett.84.2770.	
  
[33]	
   W.	
  Gossett,	
  The	
  Application	
  of	
  the	
  “Law	
  of	
  Error”	
  to	
  the	
  Work	
  of	
  the	
  Brewery,	
  
Dublin,	
  1904.	
  
[34]	
   A.	
  Shah,	
  A.	
  Wilson,	
  Z.	
  Ghahramani,	
  Student-­‐t	
  processes	
  as	
  alternatives	
  to	
  
Gaussian	
  processes,	
  arXiv	
  Prepr.	
  arXiv1402.4306.	
  (2014).	
  
[35]	
   A.C.	
  de	
  Souza,	
  C.	
  Tsallis,	
  Student’s	
  t-­‐	
  and	
  r-­‐distributions:	
  Unified	
  derivation	
  
from	
  an	
  entropic	
  variational	
  principle,	
  Phys.	
  A	
  Stat.	
  Mech.	
  Its	
  Appl.	
  236	
  (1997)	
  
52–57.	
  doi:10.1016/S0378-­‐4371(96)00395-­‐0.	
  
[36]	
   S.	
  Umarov,	
  C.	
  Tsallis,	
  On	
  multivariate	
  generalizations	
  of	
  the	
  q-­‐central	
  limit	
  
theorem	
  consistent	
  with	
  nonextensive	
  statistical	
  mechanics,	
  AIP	
  Conf.	
  Proc.	
  
965	
  (2007)	
  34.	
  
[37]	
   G.	
  Kaniadakis,	
  M.	
  Lissia,	
  A.M.	
  Scarfone,	
  Two-­‐parameter	
  deformations	
  of	
  
logarithm,	
  exponential,	
  and	
  entropy:	
  A	
  consistent	
  framework	
  for	
  generalized	
  
statistical	
  mechanics,	
  Phys.	
  Rev.	
  E.	
  71	
  (2005)	
  46128.	
  
[38]	
   S.	
  Umarov,	
  C.	
  Tsallis,	
  M.	
  Gell-­‐Mann,	
  S.	
  Steinberg,	
  Generalization	
  of	
  symmetric	
  
alpha-­‐stable	
  Lévy	
  distributions	
  for	
  q>1.,	
  J.	
  Math.	
  Phys.	
  51	
  (2010)	
  33502.	
  
doi:10.1063/1.3305292.	
  
[39]	
   C.	
  Tsallis,	
  A.R.	
  Plastino,	
  R.F.	
  Alvarez-­‐Estrada,	
  Escort	
  mean	
  values	
  and	
  the	
  
characterization	
  of	
  power-­‐law-­‐decaying	
  probability	
  densities,	
  J.	
  Math.	
  Phys.	
  
50	
  (2009)	
  43303.	
  
[40]	
   C.	
  Lassner,	
  R.	
  Lienhart,	
  Norm-­‐Induced	
  Entropies	
  for	
  Decision	
  Forests,	
  in:	
  2015	
  
IEEE	
  Winter	
  Conf.	
  Appl.	
  Comput.	
  Vis.,	
  IEEE,	
  2015:	
  pp.	
  968–975.	
  
doi:10.1109/WACV.2015.134.	
  
[41]	
   K.P.	
  Nelson,	
  Reduced	
  perplexity:	
  Uncertainty	
  measures	
  without	
  entropy,	
  in:	
  
Recent	
  Innov.	
  Info-­‐Metrics,	
  Washington,	
  D.C.,	
  2014.	
  
	
  

	
  
24	
  

