arXiv:1711.03580v1 [] 7 Nov 2017

First Results from Using Game Refinement
Measure and Learning Coefficient in Scrabble∗
Suwanviwatana Kananat
s.kananat@jaist.ac.jp

Hiroyuki Iida
iida@jaist.ac.jp

November 13, 2017

Abstract
This paper explores the entertainment experience and learning experience in Scrabble. It proposes a new measure from the educational
point of view, which we call learning coefficient, based on the balance
between the learner’s skill and the challenge in Scrabble. Scrabble
variants, generated using different size of board and dictionary, are analyzed with two measures of game refinement and learning coefficient. The
results show that 13x13 Scrabble yields the best entertainment experience and 15x15 (standard) Scrabble with 4% of original dictionary size
yields the most effective environment for language learners. Moreover,
15x15 Scrabble with 10% of original dictionary size has a good balance
between entertainment and learning experience.

Keywords— Game refinement, learning coefficient, Scrabble

1

Introduction

Scrabble has been played for a long while in various settings, e.g. as a friendly
game among friends or household members, in competitive matches and also as
a language learning tool. Using game refinement theory [4], we have discovered
that Scrabble has fun game aspect over educational aspect [2]. This paper is
an attempt to enhance the Scrabble with learning aspect. Emotional excitement or mental engagement in games is the subject of game refinement theory.
Early work in this direction has been carried out by Iida et al. [3], while constructing a logistic model based on game outcome uncertainty to measure the
attractiveness and sophistication of games, known as game refinement theory.
Although many efforts have been devoted to the study of scoring sports and
∗ Scrabble R is a registered trademark. All intellectual property rights in and to the game
are owned in the USA by Hasbro Inc., in Canada by Hasbro Canada Corporation, and throughout the rest of the world by J.W. Spear & Sons Limited of Maidenhead, Berkshire, England,
a subsidiary of Mattel Inc.

1

boardgames, Scrabble also has an educational aspect which requires extra
dimension to explore.
The structure of the paper is as follows. Section 2 presents the basic rules
of Scrabble. Section 3 and Section 4 describe Scrabble in two distinct dimensions of measurement from the perspective of entertainment and education,
respectively. Section 5 presents the assessment using the swing model, thus discusses the results of the analysis, and concluding remarks are given in Section 6.

2

Scrabble

Scrabble is a word anagram game in which 2 to 4 players competitively score
points by placing tiles, each bearing a single letter, onto a 15x15 board. The
standard board is shown in Figure 1. The tiles must form words that are
accepted by the dictionary, in either the vertical or horizontal direction in a
crossword style. There are 2 general sets of acceptable words, named OCTWL
3W

2L

3W

2W

2L

3L
2W

3L
2L

2W

2W

2L

2W
2W

3L
2L

3W

3L
2L

2L
2L

3L

2L
2L

2L

3L

3L

3L
2W

2W

2L

2W

2L

2W
2L

3L
2L

3W
2L

2W

3W

3L

2L
2W

2L

2W

2W

2L

2W
3L

3W

2W
3L

3W

2W
2L

3W

Figure 1: Standard Scrabble board
and SOWPODS. These 2 sets were developed specially for Scrabble so that
there are only words of 2-15 characters. OCTWL is generally used in the USA,
Canada, and Thailand while other countries are using SOWPODS. There are
differences in the number of words, as shown in Table 1.
Table 1: Acceptable words distribution in Scrabble
Set
OCTWL
SOWPODS
Usage
USA, Canada, Thailand
Others
Total Word
187632
267751

2

Table 2 shows the population distribution of players from cross-tables [10],
the unofficial online Scrabble resource. Obviously, there is a large difference
between those who are the native speaker and those who are not. Then, we
hypothesize that current setting of Scrabble is more attractive for players
that have sufficient English knowledge, than most language learners.
Table 2: Population distribution of Scrabble players in cross-tables
Country
Official language Players
%
Barbados
English, Bajan
2
0.149
Canada
English, French
293
21.8331
Israel
Hebrew, Arabic
1
0.0745
Thailand
Thai
3
0.2235
USA
English
1041
77.5708
Unknown
Unknown
2
0.149

3

Game Refinement Measure

This section gives a short description of game refinement theory. A general
model of game refinement was proposed based on the concept of game progress
and game information progress [4]. It bridges a gap between boardgames and
sports games.

3.1

Game Progress Model

The ’game progress’ is twofold. One is game speed or scoring rate, while another
one is game information progress which focuses on the game outcome. Game
information progress presents the degree of certainty of the game’s results in
time or in steps. Having full information of the game progress, i.e. after its
conclusion, game progress x(t) will be given as a linear function of time t with
0 ≤ t ≤ tk and 0 ≤ x(t) ≤ x(tk ), as shown in Eq. (1).
x(t) =

x(tk )
t
tk

(1)

However, the game information progress given by Eq. (1) is unknown during the
in-game period. The presence of uncertainty during the game, often until the
final moments of a game, reasonably renders game progress exponential. Hence,
a realistic model of game information progress is given by Eq. (2).
x(t) = x(tk )(

t n
)
tk

(2)

Here n stands for a constant parameter which is given based on the perspective
of an observer of the game that is considered. Then the acceleration of the game

3

information progress is obtained by deriving Eq. (2) twice. Solving it at t = tk ,
we have Eq. (3).
x′′ (tk ) =

x(tk ) n−2
x(tk )
t
n(n − 1) =
n(n − 1)
n
(tk )
(tk )2

(3)

It is assumed in the current model that game information progress in any type
of game is encoded and transported in our brains. We do not yet know about
the physics of information in the brain, but it is likely that the acceleration of
information progress is subject to the forces and laws of physics. Therefore, we
k)
expect that the larger the value x(t
(tk )2 , the more exciting the game becomes, due
in
√ part to the uncertainty of the game outcome. Thus, we use its root square,
x(tk )
,
tk

as a game refinement measure for the game under consideration. We
call it GR value for short, also call x(tk ) and tk as G and T respectively, as
shown in Eq. (4).
√
G
(4)
GR =
T
In the previous works, the game progress model has been applied to various
sports games [5] to verify its effectiveness. The appropriate zone of game refinement measure range from 0.07 to 0.08. The game progress model has been
expanded to other domains such as multiplayer card games [6] and video games
[7]. We show, in Table 3, the results of measures of game refinement for some
games.
Table 3: Comparison of game refinement values for some games
Successful shoot (G) Attempt (T)
GR
Soccer
2.64
22
0.073
Basketball
36.38
82.01
0.073
UNO
0.976
12.684
0.078
Badminton
46.336
79.344
0.086
Table Tennis
54.863
96.465
0.077
DotA
68.6
106.2
0.078

3.2

Swing Model

In scoring boardgames like Scrabble, swing, a state transition of advantage
during the game progress is considered as successful shoot, and game length as
attempt respectively. Let S and N be the average number of swings and the
game length, respectively. Then the refinement measure in the swing model is
given by Eq. (5).
√
S
GR =
(5)
N

4

4

Another Measure from Educational Perspective

This section gives a new measure from the educational perspective given by
focusing on a balance between complexity and learning efficiency.

4.1

Complexity

The measure of search-space complexity or complexity [1] indicates the total
possible in the game represented on the natural logarithm scale. Let B and
D be the average branching factor and average game length respectively. The
complexity is obtained by Eq. (6).
C = D ln B

(6)

Complexity measure can express the complexity of the game from the viewpoint of players. The player who has the ability to handle problems with higher
complexity would think wider and deeper, thus have a better solution and understanding the nature of the game. The complexity of some existing games
from the viewpoint of experts are shown in Table 4. According to the history of
game artificial intelligence development, chess computer Deep Blue won a world
champion Garry Kasparov in May 1997 [8]. Then, Go computer AlphaGo won
a world champion Ke Jie in May 2017 [9]. The difficulty in artificial intelligence
development obviously shows that complexity of Go is much more than that of
chess. Similarly, the complexity of chess is much more than that of Tic-tac-toe.

Table 4: Comparison of complexity for some board games
Branching factor (B) Game length (D) Complexity (C)
Tic-tac-toe
≤9
≤9
≤19.775
Chess
35
80
284.428
Go
250
208
1148.464

4.2

Learning Coefficient

From the experiments performed with different size of dictionary and different
player’s model, we found that the complexity measure has the linear relation
with AI knowledge base. It enables to calculate the slope per each dictionary
size. Let d and t be the custom dictionary size and total words in dictionary
respectively. The total words in custom dictionary d′ can be obtained by Eq. (7).
d′ = td

5

(7)

Let p and x be the current player knowledge base and the newly learned words
respectively. The new player knowledge base p′ can be obtained by Eq. (8).
p′ = p +

x
x
=p+
d′
td

(8)

By the definition of slope m, it can be obtained by the difference of complexity
in proportional to the difference of player knowledge base, as shown by Eq. (9).
m=

∆c
∆c
∆c
∆c
= ′
=
= x
x
∆p
p −p
p + td − p
td

(9)

Thus we have learning coefficient (say ∆c), as shown by Eq. (10).
∆c =

mx
td

(10)

The newly learned words x and total words d are constant. To maximize the
benefit, we need to maximize the increment of ∆c since it represents the improvement of a learner with the same amount of newly learned words. In short,
we need to maximize m
d , which we define in this study as learning coefficient.

5

Assessment and Discussion

Experiments are conducted by simulating the Scrabble matches between AI
with the various knowledge base, using various dictionary sizes.

5.1

Possible Enhancement with focus on Complexity

The experimental results are analyzed using game refinement measure, as shown
in Fig. 2. Game refinement measure of the original Scrabble is slightly higher
than appropriate zone. This reveals that the original setting of Scrabble yields
excess branching factors. One possible enhancement is to reduce the board size,
and it is found that 13x13 board size gives the best setting. The complete board
is shown in Fig. 3. This results in 24.89% smaller compared to the standard
Scrabble, thus can significantly reduce the branching factor. The results are
shown in Fig. 4, which is much closer to the appropriate zone.

5.2

Tendency of Game Refinement Measure Changes

The interesting part is that Scrabble with various size of a dictionary has
different game refinement tendency, as shown in Table 5. Note that ‘Dec’ and
‘Inc’ stand for decreasing and increasing, respectively.
From the earlier study [4], a measure of game refinement would reflect the
balance between chance and skill in playing the game under consideration.
Higher game refinement value means that chance is a stronger factor than skill.
Game refinement tendency would indicate the user experience of the game. If
it is increasing, we know that chance has more effect for a match-up between
6

Game refinement value

d = 0.1
d = 0.2
d = 0.3
d = 0.4
d = 0.5
d = 0.6
d = 0.7
d = 0.8
d = 0.9
d=1
Appropriate zone
8 · 10−2
7 · 10−2
0.1

0.2

0.3

0.4

0.5
0.6
0.7
AI knowledge base

0.8

0.9

1

Figure 2: Measures of game refinement and AI knowledge base: standard
Scrabble matches between AIs with different knowledge base on various dictionary size
2W

3L
2W

3L
2L

2W

2W

2L

2W

2W
3L

2W
3L

2L

3L
2L

2L

3L

2L

2L

2W

2L

2L

3L

2L
2L

3L

2L
3L

2W

3L
2W

2W

2L

2W
2W

2W

2L

2L

2W
2L

3L

2W
3L

2W

Figure 3: 13x13 variant of Scrabble board
expert players, and less for novice players respectively. Novice used to enjoy
this type of game as a skill-based game, but once they become an expert player,
only individual skill is not enough to beat the opponent since there are some
other unexpected factors towards a game, e.g., chance, teamwork and imperfect
information. This would offer the fun-game experience. On the contrary, tendency of decreasing would indicate the competitive-game experience. In case

7

Game refinement value

d = 0.1
d = 0.2
d = 0.3
d = 0.4
d = 0.5
d = 0.6
d = 0.7
d = 0.8
d = 0.9
d=1
Appropriate zone
8 · 10−2
7 · 10−2
0.1

0.2

0.3

0.4

0.5
0.6
0.7
AI knowledge base

0.8

0.9

Figure 4: Measures of game refinement and AI knowledge base: 13x13 variation
of Scrabble matches between AIs with different knowledge base on various
dictionary size
Table 5: Tendency of game refinement measure changes for different size of
dictionary
Dictionary size
GR tendency
0.01 - 0.1
Dec
0.2 - 0.6
Dec then Inc
0.7 - 0.9
Inc
1
Inc then Dec

where tendency is decreasing-then-increasing, it combines both experiences in
the different phase. At the beginning, players may feel the competitive-game
experience, which is followed by the fun-game experience. On the other hand,
the results using complexity measure are shown in Fig. 5.

5.3

Learning Coefficient and Summary

For every dictionary size, we have compared the learning coefficient as shown
in Fig. 6. The peak of the highest complexity slope per dictionary size is at 4%
of dictionary size. Despite that, the corresponding game refinement measure is
relatively uncomfortable. The good balance between entertainment and education would be around 10% to 20% of dictionary size. We show, in Table 6, the
summary of our results in this study.

8

1

250
d = 0.1
d = 0.2
d = 0.3
d = 0.4
d = 0.5
d = 0.6
d = 0.7
d = 0.8
d = 0.9
d=1

Complexity

200

150

100

50

0
0.1

0.2

0.3

0.4

0.5
0.6
0.7
AI knowledge base

0.8

0.9

1

Complexity slope per dictionary size

Figure 5: Complexity and AI knowledge base: standard Scrabble matches
between AIs with different knowledge base on various dictionary size

1,200
1,000
800
600
400
200
0

0.1

0.2

0.3

0.4
0.5
0.6
Dictionary size

0.7

0.8

0.9

1

Figure 6: Relation between dictionary size and corresponding learning coefficient

9

Table 6: Summary of analyzing Scrabble using game
learning coefficient
Variation
Board size
d
GR
Standard
15x15
1
0.0751 – 0.0926
Entertainment
13x13
1
0.0771 – 0.0808
Education
15x15
0.04 0.0951 – 0.3843
Balance
15x15
0.1 0.0731 – 0.2204

6

refinement measure and
GR tendency
Inc then Dec
Dec then inc
Dec
Dec

Learning coefficient
98.7323
140.9987
1333.335
1073.907

Concluding Remarks

According to the study, Scrabble players would feel fun-game experience (i.e.,
entertainment) more than the educational experience. We proposed three ways
of possible improvement: entertainment enhancement, education enhancement,
and the good balance between them. Scrabble yields excessive branching factors. This results in that game refinement measures are higher than the appropriate zone. Entertainment enhancement can be done by reducing the standard
board size (15x15) to 13x13. This can improve game refinement values significantly, specifically for native speakers to enjoy the competitive environment.
On the other hand, we can also enhance Scrabble with the educational dimension. For this purpose we proposed two new models with focus on complexity
and learning coefficient. Educational enhancement can be done by maximizing
the learning coefficient value, while the good balance between entertainment
and enhancement can be found by trading off.
Acknowledgments.
This research is funded by a grant from the Japan Society for the Promotion
of Science, in the framework of the Grant-in-Aid for Challenging Exploratory
Research.

References
[1] Victor, A. L.: Searching for Solutions in Games and Artificial Intelligence,
Ph.D.Thesis, Maastricht University. (1994)
[2] Kananat, Suwanviwatana, Jean-Christophe Terrillon, and Hiroyuki Iida.
”Gamification and Scrabble.” Games and Learning Alliance. Springer International Publishing, 2016. 405-414.
[3] Iida, H., Takahara, K., Nagashima, J., Kajihara, Y., Hashimoto, T.: An
application of game-refinement theory to Mah Jong. In: Rauterberg, M. (ed.)
ICEC 2004. LNCS, vol. 3166, pp. 333—338. Springer, Heidelberg (2004)

10

[4] Sutionon, A.P., Purwarianti, A., Iida, H.: A mathematical model of game
refinement. International Conference on Intelligent Technologies for Interactive Entertainment. pp. 148–151. Springer International Publishing (2014)
[5] Nathan, N., and Iida H.: Game refinement theory and its application to
score limit games. 2014 IEEE Games Media Entertainment (GEM). (2014)
[6] Ramadhan, A., Maulidevi N.U., Iida H.: Game refinement theory and multiplayer games: case study using UNO. The Seventh International Conference
on Information, Process, and Knowledge Management. pp.119–125. (2015)
[7] Xiong, S., Zuo, L., Chiewvanichakorn R., Iida H.: Quantifying engagement
of various games. The 19th Game Programming Workshop 2014. pp.101–106.
(2014)
[8] Campbell, Murray, A. Joseph Hoane, and Feng-hsiung Hsu. ”Deep blue.”
Artificial intelligence 134.1-2 (2002): 57-83.
[9] Silver, David, et al. ”Mastering the game of Go with deep neural networks
and tree search.” Nature 529.7587 (2016): 484-489.
[10] cross-tables.com - SCRABBLE R tournaments, statistics, and community,
http://www.cross-tables.com/

11

