Calculate the block entropy
Mathematica - 81 78 75 72 67 65 62 56 bytes

I haven't golfed anything in Mathematica before, so I suppose there's room for improvement. This one doesn't quite conform to the rules due to the Partition and Tally functions, but it's quite neat so I thought I'd post it anyway.

f=N@Tr[-Log[p=#2/Length@b&amp;@@@Tally[b=##~Partition~1]]p]&amp;


This works with any set of symbols, and can be used like

sequence = {2, 3, 4, 1, 2, 3, 0, 0, 3, 2, 3, 0, 2, 2, 4, 4, 4, 1, 1, 
   1, 0, 4, 1, 2, 2, 4, 0, 1, 2, 3, 0, 2, 3, 2, 3, 2, 0, 1, 3, 4, 4, 
   0, 2, 1, 4, 3, 0, 2, 4, 1, 0, 4, 0, 0, 2, 2, 0, 2, 3, 0, 0, 4, 4, 
   2, 3, 1, 3, 1, 1, 3, 1, 3, 1, 0, 0, 2, 2, 4, 0, 3, 2, 2, 3, 0, 3, 
   3, 0, 0, 4, 4, 1, 0, 2, 3, 0, 0, 1, 4, 4, 3};
f[sequence, 3]

&gt; 4.06663


Here is a somewhat ungolfed version:

f[sequence_, m_] := (
    blocks = Partition[sequence, m, 1];
    probabilities = Apply[#2/Length[blocks] &amp;, Tally[blocks], {1}];
    N[Tr[-Log[probabilities]*probabilities]]
)


It will probably run faster if I apply N directly to the result of Tally.

By the way, Mathematica does actually have an Entropy function, that reduces this to 28 bytes, but that's definitely against the rules.

f=N@Entropy@Partition[##,1]&amp;


On the other hand, here is a 128 byte version that reimplements Partition and Tally:

f=N@Tr[-Log[p=#2/n&amp;@@@({#[[i;;i+#2-1]],1}~Table~{i,1,(n=Length@#-#2+1)}//.{p___,{s_,x_},q___,{s_,y_},r___}:&gt;{p,{s,x+y},q,r})]p]&amp;


Ungolfed:

f[sequence_, m_] := (
    n = Length[sequence]-m+1; (*number of blocks*)
    blocks = Table[{Take[sequence, {i, i+m-1}], 1},
                   {i, 1, n}];
    blocks = b //. {p___, {s_, x_}, q___, {s_, y_}, r___} :&gt; {p,{s,x+y},q,r};
    probabilities = Apply[#2/n &amp;, blocks, {1}];
    N[Tr[-Log[probabilities]*probabilities]]
)