abstract: In this paper, we propose a novel network design mechanism for efficient embedded computing. Inspired by the limited computing patterns, we propose to fix the number of channels in a group convolution, instead of the existing practice that fixing the total group numbers. Our solution based network, named Variable Group Convolutional Network (VarGNet), can be optimized easier on hardware side, due to the more unified computing schemes among the layers. Extensive experiments on various vision tasks, including classification, detection, pixel-wise parsing and face recognition, have demonstrated the practical value of our VarGNet.
author: Qian Zhang, Jianjun Li, Meng Yao, Liangchen Song, Helong Zhou,
**Zhichao Li, Wenming Meng, Xuezhi Zhang, Guoli Wang**
*Horizon Robotics*
bibliography: varg.bib
title: VarGNet: Variable Group Convolutional Neural Network for Efficient Embedded Computing

# Introduction

Empowering embedded systems to run the well-known deep learning architectures, such as convolutional neural networks (CNNs), has been a hot topic in recent years. For smart Internet of Things applications, the challenging part is that the whole system is required to be both energy-constrained and of small size. To meet the challenge, the work of improving the efficiency of the whole computing process can be roughly broken into two directions: The first is to design lightweight networks which has a small MAdds , thus friendly to low power consumption platforms; The second is to optimize hardware-side configurations, such as FPGA based accelerators , or to make the whole computing process more efficient by improving the compiler and generating more smart instructions .

All of the mentioned works above have demonstrated their great practical value in various applications. However, the real performance may not live up to the designer's expectations, due to the gap between the two different optimization directions. Specifically, for elaborately tuned networks with small MAdds, the overall latency may be high , while for carefully designed compilers or accelerators, the real networks may be hard to be processed.

In this work, we intend to close the exiting gap by systematically analyze the necessary properties of a lightweight network that is friendly to the embedded hardware and the corresponding compilers. More precisely, since the computation patterns of a chip in a embedded system is strictly limited, we propose that a embedded-system-friendly network should fit into the targeted computation patterns and also the ideal data layout. By fitting into the ideal data layout, we can reduce the communication cost between on-chip memory and off-chip memory, thus fully exploit the computation throughput.

Inspired by the observation that the computation graph of a network is easier to be optimized, if the computational intensity of the operations in a network is more balanced. We propose the variable group convolution, which is based on depthwise separable convolution . In variable group convolution, the number of input channels in each group is fixed and can be tuned as a hyperparameter, which is different from the group convolution where the number of groups are fixed. The benefits are two folds: Fixing the number of channels is more suitable for optimization from the perspective of compilers, due to the more coherent computation pattern and data layout; Compared with depthwise convolution in , which set the group number to be the channel number, variable group convolution has a larger network capacity , thus allowing the smaller channel numbers, which helps relief the time consuming off-chip communication.

Another key component in our network is to better exploit the on-chip memory based on the inverted residual block . However, in MobileNetV2 , the number of channels are adjusted by pointwise convolutions, which has a different computing pattern with the $3\times 3$ depthwise convolution in between and then is hard to be optimized due to limited computation patterns. Therefore, we propose that the input feature with $C$ channels is first expanded to $2C$ by variable group convolution and returned to $C$ by pointwise convolution. In this manner, the computational costs between the two types of layers are more balanced, thus being more hardware and compiler friendly. To sum up, our contributions can be summed as follows:

-   We systematically analyze how to optimize the computation of CNNs from the perspective of both network architectures and hardware/compilers on embedded systems. We found that there exists a gap between the two optimization directions that some elaborately designed architectures are hard to be optimized due to limited computation patterns in an embedded system.

-   Observing that more unified computation pattern and data layout are more friendly to an embedded system, we propose the variable group convolution and the corresponding improved whole network, named variable group network and VarGNet for short.

-   Experiments on prevalent vision tasks, such as classification, detection, segmentation, face recognition and etc., and corresponding large scale datasets verify the practical value of our proposed VarGNet.

## Related works

#### Lightweight CNNs.

Designing lightweight CNNs has been a hot topic in recent years. Representative manual designed networks include SqueezeNet , Xception , MobileNets , ShuffleNets and IGC . Besides, neural architecture search (NAS) is a promising direction for automatically designing lightweight CNNs. The above methods are capable to effectively speed up the recognition process. More recently, platforms aware NAS methods are proposed to search some specific networks that are efficient on certain hardware platforms. Our network, VarGNet, is complementary to the existing platforms aware NAS methods, since the proposed variable group convolution is helpful for setting the search space in NAS methods.

#### Optimizations on CNN accelerators.

To accelerate neural networks, FPGAs and ASIC designs have been widely studied. Generally speaking, Streaming Architectures (SAs) and Single Computation Engines (SCEs) are two kinds of FPGA based accelerators . The difference between the two directions is on customization and generality. SAs designs seek customization more than generality, while SCEs emphasize the tradeoff between flexibility and customization. In this work, we hope to propose a network that can be optimized by existing accelerators more easily, thus improve the overall performance.

# Designing Efficient Networks on Embedded Systems

For chips used on embedded systems, such as FPGA or ASIC, a low unit price as well as a fast time to market are critical factors in designing the whole system. Such crucial points result in a relative simple chip configuration. In other words, the computation schemes are strictly limited when compared with general-purpose processing units. However, operators in a SOTA network are so complex that some layers can be accelerated by hardware design while others not. Thus, for designing efficient networks on embedded systems, the first intuition here is that the layers in a network should be similar as each other in some sense.

Another important intuition is based on two properties of convolutions used in CNNs. The first property is the computation pattern. In convolution, several filters (kernels) slide over the whole feature map, indicating that the kernels are repeatedly used while values from the feature map are only used once. The second property is the data size of convolutional kernels and feature maps. Typically, the size of convolutional kernels is much lower than the size of feature maps, such as $k^2C$ for kernels and $2HWC$ for feature maps in 2D convolutions. In light of the above two properties, an ingenious solution is to load all the data of kernels first and then perform the convolution with popping and popping out feature data sequentially . Such practical solution is the second intuition for our following two guidelines for efficient network design on embedded systems:

-   It will be better if the size of intermediate feature maps between blocks is smaller.

-   The computational intensity of layers in a block should be balanced.

Next, we introduce the two guidelines in detail.

#### Small intermediate feature maps between blocks.

In SOTA networks, a common practice is to first design a normal block and a down sampling block first, and then stack several blocks together to get a deep network. Also, in these blocks, residual connections are widely adopted. So, in recent compiler-side optimizations , layers in a block are usually grouped and computed together. In such manner, off-chip memory and on-chip memory only communicates when starting or ending computing a block in the network. Therefore, a smaller intermediate feature map between blocks will certainly help reduce the data transfer time.

#### Balanced computational intensity inside a block.

As mentioned before, in practice, weights in several layers are loaded before performing convolution. If the loaded layers have a large divergence in terms of the computational intensity, extra on-chip memory is needed to store the intermediate slices of feature maps. In MobileNetV1 , a depthwise conv and a pointwise conv are used. Different from previous definitions, in our implementation, weights are already loaded. So, computational intensity is computed as MAdds divide the size of feature maps. Then, if the feature map is of size $28\times28\times256$, the computational intensity of depthwise convolution and pointwise convolution are 9 and 256, respectively. As a result, when running the two layers, we have to increase the on-chip buffer to satisfy the pointwise, or not grouping the computation of the two layers together.

# Variable Group Network

Based on the previous mentioned two guidelines, we propose a novel network in this section. To balance the computation intensity, we set the channel numbers in a group in a network to be constant, resulting in variable groups in each convolution layers. The motivation of fixing the channel numbers is not hard to understand if we look at the MAdds of a convolution, $$\frac{k^2\times \mathrm{Height} \times \mathrm{Width} \times \mathrm{Channels}^2}{\mathrm{Groups}}.$$ Thus, if the size of feature map is a constant, then by fixing $G=\frac{\mathrm{Channels}}{\mathrm{Groups}}$, the computational intensity inside a block is more balanced. Further, the number of channels in a group can be set to satisfy the configurations of the processing elements, in which channels of a certain number will be processed every time.

Compared with depthwise convolution, the variable group convolution increases the MAdds as well as the expressiveness . Thus, now we are able to reduce the channel number of intermediate feature maps, while keeping the same generalizing ability as previous networks. Specifically, we design novel network blocks as shown in Fig. . For the normal block used in the early stages in the whole network, since the size of weights are relatively small at this time, the weights of the four layers can be all cached into the on-chip memory. When entering the late stages, where channel numbers increase and the size of weights increase as well, the normal block is also able to be optimized by only loading a variable group conv and a pointwise conv. Similarly, the operations in down sampling block are also friendly to the compiler-side and hardware-side optimizations. The whole computing process for a normal block is demonstrated in Fig. . Then, based on the architecture of MobileNetV1 , we substitute their basic blocks to ours and the whole detailed network architecture is shown in Tab. . Also, another ShuffleNet v2 based architecture is shown in Tab. .

```latex
\resizebox{\textwidth}{!}{
  % \begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c|c}
  \begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c}
  \hline\hline
  \multirow{2}{*}{Layer} & \multirow{2}{*}{Output Size} & \multirow{2}{*}{KSize} & \multirow{2}{*}{Stride} & \multirow{2}{*}{Repeat} &
  \multicolumn{7}{c}{Output Channels}
   \\  \cline{6-12}
  % \multicolumn{8}{c}{Output Channels}
  %  \\  \cline{6-13}
  %  & & & & & 0.25x & 0.5x & 0.75x & 1x & 1.25x & 1.5x & 1.75x & 2x \\ \hline
  %  Image & 224 x 224 &  &  &  & 3 & 3 & 3 & 3 & 3 & 3 & 3 & 3 \\ \hline
  %  Conv 5 & 112 x 112 & 3 x 3 & 2 & 1 & 8 & 16 & 24 & 32 & 40 & 48 & 56 & 64 \\ \hline
  %  DownSample & 56 x 56 &  & 2 & 3 & 16 & 32 & 48 & 64 & 80 & 96 & 112 & 128 \\ \hline
  %  DownSample & 28 x28 &  & 2 & 1 & 32 & 64 & 96 & 128 & 160 & 192 & 224 & 256 \\ \hline
  %  DownSample & 14 x 14 &  & 2 & 1 & 64 & 128 & 192 & 256 & 320 & 384 & 448 & 512 \\ \hline
  %  Stage Block & 14 x 14 &  & 1 & 2 & 64 & 128 & 192 & 256 & 320 & 384 & 448 & 512 \\ \hline
  %  DownSample & 7 x 7 &  & 2 & 1 & 128 & 256 & 384 & 512 & 640 & 768 & 896 & 1024 \\ \hline
  %  Stage Block & 7 x 7 &  & 1 & 1 & 128 & 256 & 384 & 512 & 640 & 768 & 896 & 1024 \\ \hline
  %  Conv 5 & 7 x 7 & 1 x 1 & 1 & 1 & 1024 & 1024 & 1024 & 1024 & 1280 & 1536 & 1792 & 2048 \\ \hline
  %  Global Pool & 1 x 1 & 7 x 7 &  &  &  &  &  &  &  &  &  &  \\ \hline
  %  FC &  &  &  &  & 1000 & 1000 & 1000 & 1000 & 1000 & 1000 & 1000 & 1000 \\  \hline\hline
   & & & & & 0.25x & 0.5x & 0.75x & 1x & 1.25x & 1.5x & 1.75x\\ \hline
   Image & 224 x 224 &  &  &  & 3 & 3 & 3 & 3 & 3 & 3 & 3\\ \hline
   Conv 1 & 112 x 112 & 3 x 3 & 2 & 1 & 8 & 16 & 24 & 32 & 40 & 48 & 56\\ \hline
   DownSample & 56 x 56 &  & 2 & 3 & 16 & 32 & 48 & 64 & 80 & 96 & 112\\ \hline
   DownSample & 28 x28 &  & 2 & 1 & 32 & 64 & 96 & 128 & 160 & 192 & 224\\ \hline
   DownSample & 14 x 14 &  & 2 & 1 & 64 & 128 & 192 & 256 & 320 & 384 & 448\\ \hline
   Stage Block & 14 x 14 &  & 1 & 2 & 64 & 128 & 192 & 256 & 320 & 384 & 448\\ \hline
   DownSample & 7 x 7 &  & 2 & 1 & 128 & 256 & 384 & 512 & 640 & 768 & 896\\ \hline
   Stage Block & 7 x 7 &  & 1 & 1 & 128 & 256 & 384 & 512 & 640 & 768 & 896\\ \hline
   Conv 5 & 7 x 7 & 1 x 1 & 1 & 1 & 1024 & 1024 & 1024 & 1024 & 1280 & 1536 & 1792\\ \hline
   Global Pool & 1 x 1 & 7 x 7 &  &  &  &  &  &  &  &  &  \\ \hline
   FC &  &  &  &  & 1000 & 1000 & 1000 & 1000 & 1000 & 1000 & 1000\\  \hline\hline
\end{tabular}}
```
```latex
\resizebox{\textwidth}{!}{
  \begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c|c}
  \hline\hline
  \multirow{2}{*}{Layer} & \multirow{2}{*}{Output Size} & \multirow{2}{*}{KSize} & \multirow{2}{*}{Stride} & \multirow{2}{*}{Repeat} &
  \multicolumn{8}{c}{Output Channels}
   \\  \cline{6-13}
   & & & & & 0.25x & 0.5x & 0.75x & 1x & 1.25x & 1.5x & 1.75x & 2x \\ \hline
   Image& 224 x 224& & & & 3& 3& 3& 3& 3& 3& 3& 3 \\ \hline
   Conv 1& 112 x 112& 3 x 3& 2& 1& 8& 16& 24& 32& 40& 48& 56& 64 \\ \hline
   Head Block& 56 x 56& & 2& 1& 8& 16& 24& 32& 40& 48& 56& 64 \\ \hline
   \multirow{2}{*}{Stage 2}& 28 x28& & 2  & 1 &  \multirow{2}{*}{16}
   & \multirow{2}{*}{32} & \multirow{2}{*}{48} & \multirow{2}{*}{64} & \multirow{2}{*}{80}
   & \multirow{2}{*}{96} & \multirow{2}{*}{112} & \multirow{2}{*}{128}  \\ \cline{2-5}
   & 28 x 28& & 1& 2& & & & & & & &  \\ \hline
   \multirow{2}{*}{Stage 3}& 14 x 14& & 2 & 1 & \multirow{2}{*}{32}
   & \multirow{2}{*}{64}& \multirow{2}{*}{96}& \multirow{2}{*}{128}& \multirow{2}{*}{160}
   & \multirow{2}{*}{192}& \multirow{2}{*}{224}& \multirow{2}{*}{256} \\ \cline{2-5}
   & 14 x 14& & 1& 6& & & & & & & &  \\ \hline
   \multirow{2}{*}{Stage 4}& 7 x 7& & 2 & 1 & \multirow{2}{*}{64}
   & \multirow{2}{*}{128}& \multirow{2}{*}{192}& \multirow{2}{*}{256}& \multirow{2}{*}{320}
   & \multirow{2}{*}{384}& \multirow{2}{*}{448}& \multirow{2}{*}{512} \\ \cline{2-5}
   & 7 x 7& & 1& 3& & & & & & & &  \\ \hline
   Conv 5& 7 x 7& 1 x 1& 1& 1& 1024& 1024& 1024& 1024& 1280& 1536& 1792& 2048 \\ \hline
   Global Pool& 1 x 1& 7 x 7& & & & & & & & & &  \\ \hline
   FC& & & & & 1000& 1000& 1000& 1000& 1000& 1000& 1000& 1000 \\ \hline\hline
  \end{tabular}}
```

VarGNet v1 performance on ImageNet. (G is the number of channels in a group.)

(a) G = 4

VarGNet v1 performance on ImageNet. (G is the number of channels in a group.)

Model Scale
Acc-top1 (float32)
Acc-top1 (int8)
Model size
MAdds
Max Channels

0.25
64.57%
65.02%
1.44M
55M
128

0.5
69.67%
70.33%
2.23M
157M
256

0.75
72.36%
72.56%
3.43M
309M
384

1
74.04%
74.11%
5.02M
509M
512

(b) G = 8

VarGNet v1 performance on ImageNet. (G is the number of channels in a group.)

Model Scale
Acc-top1 (float32)
Acc-top1 (int8)
Model size
MAdds
Max Channels

0.25
65.44%
65.61%
1.5M
75M
128

0.5
70.67%
70.84%
2.37M
198M
256

0.75
73.28%
73.35%
3.66M
370M
384

1
74.87%
74.90%
5.33M
590M
512

(c) Comparison network: MobileNet v1

VarGNet v1 performance on ImageNet. (G is the number of channels in a group.)

Model Scale
Acc(top1)
Model size
MAdds
Max Channels

0.35
60.4%
0.7 M
72 M
358

0.6
68.6%
1.7 M
201 M
614

0.85
72.0%
3.1M
394 M
870

1.0
73.3%
4.1M
542 M
1024

1.05
73.5%
4.4 M
594 M
1075

1.3
74.7%
6.4 M
903 M
1331

1.5
75.1%
8.3 M
1.17 G
1536

VarGNet v2 performance on ImageNet. (G is the number of channels in a group.)

(a) G = 4

VarGNet v2 performance on ImageNet. (G is the number of channels in a group.)

Model Scale
Acc-top1 (float32)
Acc-top1 (int8)
Model size
MAdds
Max Channels

0.25
60.53%
61.33%
1.27M
35M
64

0.5
67.52%
68.60%
1.72M
92M
128

0.75
70.97%
71.84%
2.35M
173M
192

1
73.18%
73.55%
3.19M
278M
256

(b) G = 8

VarGNet v2 performance on ImageNet. (G is the number of channels in a group.)

Model Scale
Acc-top1 (float32)
Acc-top1 (int8)
Model size
MAdds
Max Channels

0.25
61.74%
62.46%
1.35M
51M
64

0.5
68.40%
68.83%
1.87M
124M
128

0.75
71.48%
72.07%
2.58M
222M
192

1
73.76%
73.76%
3.49M
343M
256

(c) Comparison network: ShuffleNet v2

VarGNet v2 performance on ImageNet. (G is the number of channels in a group.)

Model Scale
Acc(top1)
Model size
MAdds
Max Channels

0.25 (60)
63.85%
1.47M
51M
240

0.5 (108)
68.74%
2.1M
123M
432

0.75 (154)
71.65%
2.92M
223M
616

1 (196)
73.17%
3.87M
342M
784

1.25 (228)
74.15%
6.63M
494M
912

1.5 (270)
74.56%
8.06M
666M
1080

1.75 (312)
75.24%
9.68M
863M
1248

# Experiments

## ImageNet Classification

The results of our model on ImageNet are presented in Tab. and Tab. . Training hyperparameters are set as: batch size 1024, crop ratio 0.875, learning rate 0.4, cosine learning rate schedule, weight decay 4e-5 and training epochs 240. We can observe that VarGNet v1 performs better than MobileNet v1, as shown in Tab. . From (c) in Tab. , we can see that when the model scale is small, the performance of VarGNet v2 is worse than ShuffleNet v2, due to less channels used in our VarGNet v2. Then, when the model size is large, our network performs better.

## Object Detection

In Tab. , we present the performance of our proposed VarGNet as well as comparison methods. We evaluate the object detection performance of our proposed networks on COCO datasets and compare them with other state-of-the-art lightweight architectures. We choose FPN-based Faster R-CNN as the framework and all the experiments are implemented under the same settings with the input resolution being 800$\times$`<!-- -->`{=html}1333 and the number of epochs being 18. Specially, we find that ShuffleNet v2 achieves better accuracy if trained with more epochs so a model with 27 epochs is trained for ShuffleNet v2. 1000 proposals per image are evaluated in RPN stage at test time. We use train+val set for training except 8000 minimal images and finally test on minival set.

| Network                       | MAdds (G) | mAP  |
|:------------------------------|:---------:|:----:|
| MobileNet v1 1.0              |   24.15   | 31.1 |
| MobileNet v2 1.0              |   18.71   | 31.0 |
| ShuffleNet v1 1.0             |   15.31   | 27.9 |
| ShuffleNet v2 1.0             |   15.55   | 27.5 |
| ShuffleNet v2 1.0 (27 epochs) |   15.55   | 28.9 |
| VarGNet v1 1.0                |   24.91   | 33.7 |
| VarGNet v2 0.5                |   14.98   | 28.6 |
| VarGNet v2 1.0                |   19.61   | 33.3 |

Performance on COCO object detection with FPN based Faster R-CNN. The input image size is 800 $\times$ 1333.

## Pixel Level Parsing

### Cityscapes

On Cityscapes dataset , we designed a multi-task structure (Fig. ) to conduct two important pixel level parsing tasks: single image depth prediction and segmentation.

#### Traning setup.

We use the standard Adam Optimizer with weight decay set to 1e-5 and batch size set to 16. The learning rate is initialized as 1e-4 and follows a polynomial decay with power of 0.9. Total training epochs are set as 100. For data augmentation, random horizontal flip is used and images are resized with scale randomly chosen from 0.6-1.2. For multitask training, we have the loss function defined as $$L_{\mathrm{total}}=\lambda_{\mathrm{instance}} L_{\mathrm{instance}}+\lambda_{\mathrm{semantic}} L_{\mathrm{semantic}}+\lambda_{\mathrm{depth}} L_{\mathrm{depth}}.$$ When the task is panoptic segmentation, we set $\lambda_{\mathrm{instance}}=0.2, \lambda_{\mathrm{semantic}}=1.0$. After adding depth task, we set $\lambda_{\mathrm{depth}}=0.08$.

#### Results.

Parameters and MAdds of comparison methods are presented in Table. . Results and some visual examples on segmentation and depth prediction are shown in Table. and Fig. , respectively. The priority of the proposed VarGNet v1 and v2 is proved by the above tables. VarGNet v1 and v2 are efficient and can perform equally well when compared with large networks.

| Method       |   Backbone   | MAdds(G) | Params |
|:-------------|:------------:|:--------:|:------:|
| SegNet       |    VGG16     |  286.0   | 29.5M  |
| Enet         | From scratch |   3.8    |  0.4M  |
| BiSeNet      |  Xception39  |   2.9    |  5.8M  |
| BiSeNet      |    Res18     |   10.8   | 49.0M  |
| MobileNet v2 |      \-      |   6.82   | 7.64M  |
| VarGNet v1   |      \-      |   6.16   | 13.23M |
| VarGNet v2   |      \-      |   2.76   | 7.41M  |

Details of comparison methods and ours on pixel level parsing tasks with input size 640$\times$`<!-- -->`{=html}360.

Results on Cityscapes validation set.

(a) Semantic Segmentation (image size 2048×1024)

Results on Cityscapes validation set.

Method
Backbone
Mean IoU(%)

BiSeNet
Xception39
69.0

BiSeNet
Res18
74.8

MobileNet v2
-
64.8

VarGNet v1
-
76.6

VarGNet v2
-
74.2

Results on Cityscapes validation set.

(b) Depth

Results on Cityscapes validation set.

Method
AbsRel
SqlRel
RMSE
RMSE Log

MobileNet v2
0.167
3.22
15.46
0.553

VarGNet v1
0.092
1.327
8.864
0.163

VarGNet v2
0.096
1.404
8.85
0.168

Results on Cityscapes validation set.

(c) Panoptic Segmentation (MAdds calculated with 2048×1024 input size.)

Results on Cityscapes validation set.

Method
Backbone
MAdds
PQ
PQ(Things)
PQ(Stuff)
Mean IoU(%)

PFPnet
resnet101
533G
58.1
52
62.5
75.1

VarGNet v1
-
104G
57.1
50
62.3
73.4

VarGNet v2
-
68G
54.5
45.1
59.8
71.4

Results on Cityscapes validation set.

(d) Panoptic Segmentation + Depth (MAdds calculated with 2048×1024 input size.)

Results on Cityscapes validation set.

Method
MAdds
PQ
PQ(Things)
PQ(Stuff)
Mean IoU(%)
AbsRel
RMSE

VarGNet v1
109G
56
48.8
61.3
71
0.1
9.2

VarGNet v2
70G
53.9
46.2
59.5
70.5
0.116
10.06

## KITTI

#### Traning setup.

For single image depth prediction and stereo tasks on KITTI dataset , we present the performance of our VarGNet based models. A U-Net style architecture () is employed in the experiments. All the depth models are trained on KITTI RAW datasets, We test on 697 images from 29 scenes split by Eigen et al. , and train on about 23488 images from the remaining 32 scenes. All the experiment results are evaluated with the depth ranging from 0m to 80m and 0m to 50m. The evaluation metrics are the same as previous works. All the stereo models are trained on KITTI RAW datasets, We test on test set split by Eigen et al. , and train set of KITTI15. The evaluation metrics for stereo are EPE and D1. During training, standard SGD Optimizer is used, and the momentum set to 0.9. The standard weight decay is set to 0.0001 for resnet18 and resnet50, and 0.00004 for others. The iteration number is set to 300 epochs. The initial learning rate is 0.001, and learning rate decay 0.1 at \[120, 180, 240\] epoch. We use 4 GPU to train models, and the batch size is set to 24.

#### Results.

In Table. and Table. , we show our depth results and stereo results under various evaluation metrics. Also, we report our implemented MobileNet and ResNet as comparison. Further, visual effects are presented in Fig. and Fig. .

```latex
\resizebox{\textwidth}{!}{
  \begin{tabular}{c}
    (a) 0-80m \\
    \begin{tabular}{lccccccccc}
      \toprule
      Method & AbsRel & SqlRel & RMSE & RMSE Log & $\delta$<1.25 & $\delta$<1.25$^2$ & $\delta$<1.25$^3$ & MAdds(G) & Params  \\ \midrule
      MobileNet v2 1.0 &0.103 & 0.744 & 4.686 & 0.17 & 0.888 & 0.966 & 0.987 & 36.8 & 7.6 M \\
      MobileNet v2 0.5 &0.112 & 0.865 & 5.01 & 0.183 & 0.869 & 0.959 & 0.983 & 10.0 & 1.9 M \\
      MobileNet v2 0.25 &0.113 & 0.831 & 4.988 & 0.183 & 0.866 & 0.96 & 0.985 & 2.9 & 539.2 K \\
      % (*)MobileNet v2 1.0 & 0.112 & 0.836 & 4.969 & 0.181 & 0.869 & 0.961 & 0.985 & 9.7 & 3.4 M \\
      % (*)MobileNet v2 0.5 & 0.116 & 0.886 & 5.123 & 0.188 & 0.861 & 0.958 & 0.984 & 2.7 & 904.1 K \\
      % (*)MobileNet v2 0.25 & 0.127 & 0.976 & 5.348 & 0.199 & 0.838 & 0.952 & 0.982 & 0.8 & 237.5 K \\
      ResNet 18 & 0.109 & 0.767 & 4.76 & 0.178 & 0.869 & 0.961 & 0.986 & 203.4 & 30.6 M \\
      ResNet 50 & 0.109 & 0.788 & 4.796 & 0.18 & 0.868 & 0.959 & 0.984 & 247.5 & 46.7 M \\ \midrule
      VarGNet v1 1.0 &  0.105 & 0.798 & 4.92 & 0.175 & 0.883 & 0.965 & 0.986  & 36.0   &13.2 M\\
      VarGNet v1 0.5 &  0.107 & 0.803 & 4.86 & 0.175 & 0.881 & 0.964 & 0.986  & 12.8   &3.8 M\\
      VarGNet v1 0.25 & 0.113 & 0.845 & 5.003 & 0.18 & 0.87 & 0.962 & 0.986   &  5.1 &1.2 M\\ \midrule
      VarGNet v2 1.0 &  0.108 & 0.823 & 4.898 & 0.176 & 0.881 & 0.965 & 0.986 & 20.0& 7.4 M\\
      VarGNet v2 0.5 &  0.111 & 0.851 & 4.98 & 0.179 & 0.874 & 0.961 & 0.985  & 7.7& 2.2 M\\
      VarGNet v2 0.25 & 0.118 & 0.9 & 5.11 & 0.186 & 0.863 & 0.959 & 0.985    & 3.3& 788.1 K\\ \bottomrule
    \end{tabular}
     \\
    (b) 0-50m \\
    % \resizebox{\textwidth}{!}{
      \begin{tabular}{lccccccccc}
      \toprule
      Method & AbsRel & SqlRel & RMSE & RMSE Log & $\delta$<1.25 & $\delta$<1.25$^2$ & $\delta$<1.25$^3$ & MAdds(G) & Params  \\ \midrule
      MobileNet v2 1.0 & 0.097 & 0.557 & 3.424 & 0.155 & 0.903 & 0.972 & 0.989 & 36.8 & 7.6 M \\
      MobileNet v2 0.5 & 0.106 & 0.649 & 3.665 & 0.167 & 0.886 & 0.966 & 0.986 & 10.0 & 1.9 M \\
      MobileNet v2 0.25 & 0.106 & 0.63 & 3.693 & 0.168 & 0.883 & 0.966 & 0.988 & 2.9 & 539.2 K \\
      % (*)MobileNet v2 1.0 & 0.106 & 0.634 & 3.669 & 0.166 & 0.886 & 0.968 & 0.988 & 9.7    & 3.4 M \\
      % (*)MobileNet v2 0.5 & 0.11 & 0.675 & 3.775 & 0.172 & 0.878 & 0.965 & 0.987 & 2.7    & 904.1 K \\
      % (*)MobileNet v2 0.25 & 0.121 & 0.746 & 3.974 & 0.183 & 0.856 & 0.96 & 0.985 & 0.8    & 237.5 K \\
      ResNet 18 & 0.104 & 0.584 & 3.525 & 0.164 & 0.883 & 0.967 & 0.988 & 203.4    & 30.6 M \\
      ResNet 50 & 0.104 & 0.592 & 3.521 & 0.165 & 0.883 & 0.965 & 0.987 & 247.5    & 46.7 M \\ \midrule
      VarGNet v1 1.0 &  0.098 & 0.578 & 3.534 & 0.158 & 0.899 & 0.973 & 0.99 &  36.0   &13.2 M\\
      VarGNet v1 0.5 &  0.1 & 0.603 & 3.535 & 0.159 & 0.897 & 0.97 & 0.989 &  12.8   &3.8 M\\
      VarGNet v1 0.25 & 0.106 & 0.637 & 3.648 & 0.165 & 0.887 & 0.969 & 0.989 &  5.1 &1.2 M\\\midrule
      VarGNet v2 1.0 & 0.101 & 0.612 & 3.556 & 0.16 & 0.896 & 0.971 & 0.989 & 20.0& 7.4 M\\
      VarGNet v2 0.5 & 0.104 & 0.635 & 3.639 & 0.163 & 0.89 & 0.968 & 0.988 & 7.7& 2.2 M\\
      VarGNet v2 0.25 & 0.112 & 0.681 & 3.768 & 0.171 & 0.88 & 0.966 & 0.988 & 3.3& 788.1 K\\ \bottomrule
    \end{tabular}
  \end{tabular}
  }
```

Stereo results on KITTI.

(a) On KITTI RAW

Stereo results on KITTI.

Method
EPE
D1
MAdds(G)
Params

MobileNet v2 1.0
1.424
0.0777
37.0
7.6 M

MobileNet v2 0.5
1.4904
0.0832
10.1
1.9 M

MobileNet v2 0.25
1.5897
0.0902
2.9
539.5 K

ResNet 18
1.5269
0.0886
205.4
30.6 M

ResNet 50
1.531
0.0887
249.5
46.7 M

VarGNet v1 1.0
1.3296
0.0703
36.1
13.2 M

VarGNet v1 0.5
1.4045
0.0757
12.9
3.8 M

VarGNet v1 0.25
1.5111
0.0835
5.1
1.2 M

VarGNet v2 1.0
1.3582
0.0728
20.7
7.4 M

VarGNet v2 0.5
1.44
0.079
8.0
2.2 M

VarGNet v2 0.25
1.5346
0.0862
3.4
790.2 K

(b) On KITTI 15

Stereo results on KITTI.

Method
EPE
D1
MAdds
Params

MobileNet v2 1.0
1.7387
0.0753
37.0
7.6 M

MobileNet v2 0.5
1.6861
0.0772
10.1
1.9 M

MobileNet v2 0.25
1.6754
0.0819
2.9
539.5 K

ResNet 18
1.7318
0.0873
205.4
30.6 M

ResNet 50
1.7305
0.0868
249.5
46.7 M

VarGNet v1 1.0
1.5767
0.07
36.1
13.2 M

VarGNet v1 0.5
1.5868
0.0708
12.9
3.8 M

VarGNet v1 0.25
1.6685
0.0747
5.1
1.2 M

VarGNet v2 1.0
1.5856
0.0697
20.7
7.4 M

VarGNet v2 0.5
1.5994
0.0735
8.0
2.2 M

VarGNet v2 0.25
1.6302
0.0777
3.4
790.2 K

## Face Recognition

All the networks are trained on the DeepGlint MS-Celeb-1M-v1c dataset cleaned from MS-Celeb-1M . There are 3,923,399 aligned face images from 86,876 ids. The LFW , CFP-FP and AgeDB-30 are used as the validation datasets. Finally, all network models are evaluated on MegaFace Challenge 1 . Table. lists the best face recognition accuracies on validation datasets, as well as face verification true accepted rates under 1e-6 false accepted rate on the refined version of MegaFace dataset . We use MobileNet v1 and MobileNet v2 as baseline models. To adapt the input image size of 112x112, the stride of the first convolutional layer is set to 1 for each baseline and vagnet model. To achieve better performance, we further replace the pooling layer by a "BN-Dropout-FC-BN" structure as InsightFace , followed by the ArcFace loss . The standard SGD optimizer is used with momentum 0.9 and the batch-size is set to 512 with 8 GPUs. The learning rate begins with 0.1 and is divided by 10 at the 100K, 140K and 160K iterations. We set the weight decay to be 5e-4. The embedding feature dimension is 256 with 0.4 dropout rate. The normalization scale is 64 and the ArcFace margin is set to 0.5. All training are based on the InsightFace toolbox .

| Networks     | MAdds |   LFW   | CFP-FP  | AgeDB-30  | MegaFace  |
|:-------------|:-----:|:-------:|:-------:|:---------:|:---------:|
| MobileNet v1 | 554M  | 0.99617 | 0.89714 |  0.96600  | 0.935848  |
| MobileNet v2 | 313M  | 0.99500 | 0.86386 |  0.95583  | 0.898219  |
| VarGNet v1   | 603M  | 0.99733 | 0.88929 |  0.97583  | 0.961499  |
| VarGNet v2   | 355M  | 0.99733 | 0.89829 |  0.97333  | 0.954261  |

Face recognition results.


abstract: We review the Mpemba effect, where intially hot water freezes faster than initially cold water. While the effect appears impossible at first sight, it has been seen in numerous experiments, was reported on by Aristotle, Francis Bacon, and Descartes, and has been well-known as folklore around the world. It has a rich and fascinating history, which culminates in the dramatic story of the secondary school student, Erasto Mpemba, who reintroduced the effect to the twentieth century scientific community. The phenomenon, while simple to describe, is deceptively complex, and illustrates numerous important issues about the scientific method: the role of skepticism in scientific inquiry, the influence of theory on experiment and observation, the need for precision in the statement of a scientific hypothesis, and the nature of falsifiability. We survey proposed theoretical mechanisms for the Mpemba effect, and the results of modern experiments on the phenomenon. Studies of the observation that hot water pipes are more likely to burst than cold water pipes are also described.
author: Monwhea Jeng
title: Hot water can freeze faster than cold?!?

# Introduction

The Mpemba effect occurs when two bodies of water, identical in every way, except that one is at a higher temperature than the other, are exposed to identical subzero surroundings, and the initially hotter water freezes first. The effect appears theoretically impossible at first sight. Nevertheless, it has been observed in numerous experiments, and we will see that it is in fact possible.

Readers who are quite certain that the effect is forbidden by the laws of thermodynamics should pause for a moment to do two things. First, to try to explain, as precisely as possible, why it is impossible. And second, to decide how to respond to a non-scientist who insists that they have observed the Mpemba effect in a home experiment. Whether or not the effect is real, careful consideration of these tasks will bring up a wealth of important issues about the scientific method, that can be understood and discussed by students without any knowledge of advanced physics.

In section , we describe early observations and experiments on this phenomenon. The effect was discussed by, among others, Aristotle, Roger Bacon, Francis Bacon, and Descartes. The effect was repeatedly discussed in support of an *incorrect* theory of heat, and appears to have been forgotten once more modern conceptions of heat, which it *appears* to contradict, were developed. In fact, we will see that Kuhn incorrectly claimed that modern experiments cannot replicate these early observations. In section , we describe the reintroduction of this phenomenon to the modern scientific community by a secondary school student, Erasto Mpemba. Mpemba's dramatic story cautions against dismissing the observations of non-scientists, and raises questions about the degree to which our theoretical understanding can and should bias our acceptance and interpretation of experiments.

In section , we see that that the Mpemba effect also provides a good illustration of the need to formulate a scientific hypothesis carefully, and the need for theory in the design of an experiment. We see that the Mpemba effect is much harder to study experimentally than it might appear at first glance, and discuss some common problems with lay experiments on the Mpemba effect. Analysis of the experiments naturally brings up Popper's thesis that a scientific hypothesis must be falsifiable.

The discussions in sections - should be comprehensible regardless of why the Mpemba effect occurs, and indeed, regardless of whether it occurs. It is not until section  that we discuss possible theoretical mechanisms for the effect, and readers uninterested in the history and background can jump straight to this section. We explain why a common proof that the Mpemba effect is impossible, is in fact flawed. Multiple explanations have been proposed for the effect. Evaporative cooling is one of the strongest explanations, but the effects of convection, dissolved gasses, and the surrounding environment, may all also be important. We discuss the results of modern experiments on the effect, which are confusing, but for interesting reasons.

It has sometimes been reported that hot water pipes are more likely to burst from freezing than adjacent cold water pipes. Experiments on this phenomenon, which we discuss in section , have been more conclusive than those on the Mpemba effect, and target supercooling as the cause. The experiments on pipes are closely related to the Mpemba effect, because they provide a clear situation where the water can "remember" what has happened to it. In section , we look at the possible importance of supercooling in the Mpemba effect, ultimately concluding that its role is unclear. Finally, in section  we discuss possibilities for future experiments that could be done by students.

# Experiments before the Scientific Revolution

The Mpemba effect has long been known in the Western world (although not by this name until fairly recently). Around 350 B.C., Aristotle wrote

If water has been previously heated, this contributes to the rapidity with which it freezes: for it cools more quickly. (Thus so many people when they want to cool water quickly first stand it in the sun: and the inhabitants of Pontus when they encamp on the ice to fish. . . pour hot water on their rods because it freezes quicker, using the ice like solder to fix their rods.) And water that condenses in the air in warm districts and seasons gets hot quickly.

Aristotle used this observation in support of antiperistasis, which is the "sudden increase in the intensity of a quality as a result of being surrounded by its contrary quality, for instance, the sudden heating of a warm body when surrounded by a cold."

While the idea of antiperistasis today sounds ridiculous, with the hindsight of our modern understanding of heat, energy, and temperature, it should be remembered that Aristotle was working without these paradigms, and indeed, without a thermometer. The fact that ice requires cold, yet hail comes in the summer, rather than the winter, requires an explanation—Aristotle's explanation was antiperistasis. Later, a number of medieval scientists used antiperistasis to explain the (apparent) facts that bodies of water are colder in the summer, and that human bodies are hotter in the winter . While we can now explain these observations with our modern theory of heat, the explanations are not obvious. The concept of temperature, and the zeroth law of thermodynamics, are quite counterintuitive to anyone who has touched metal and wood, outside on a cold day.

In the 13th century, well before the Scientific Revolution, Roger Bacon wrote *Opus Majus*, in which he argued repeatedly for the importance of experiments in science. He wrote

Moreover, it is generally believed that hot water freezes more quickly than cold water in vessels, and the argument in support of this is advanced that contrary is excited by contrary, just like enemies meeting each other. But it is certain that cold water freezes more quickly for any one who makes the experiment. People attribute this to Aristotle in the second book of Meteorologics; but he certainly does not make this statement, but he does make one like it, by which they have been deceived, namely, that if cold water and hot water are poured on a cold place, as upon ice, the hot water freezes more quickly, and this is true. But if hot water and cold are placed in two vessels, the cold will freeze more quickly. Therefore all things must be verified by experience.

What is particularly interesting about this is that Roger Bacon agrees that hot water can, *under some circumstances*, freeze faster than cold water, but argues that specification of the precise experimental conditions is important. We will see that this is a crucial observation, equally important in discussions about modern experiments on the Mpemba effect.

In the Middle Ages, debates raged over whether objects could only be cooled by extrinsic sources, or whether some objects might be able to cool themselves. In the middle of this debate, around 1461, Giovanni Marliani reported on experiments, described here by Clagett :

. . . To support his contention that heated water freezes more rapidly \[than cold\], Marliani first points to a passage in Aristotle's *Meteorologica* affirming it. However, \[Marliani\] does not depend on Aristotle's statement alone. He claims that not only has he often tested its truth during a very cold winter night, but that anyone may do so. You take four ounces of boiling water and four ounces of non-heated water and place them in similar containers. Then the containers are exposed to the air on a cold winter's morning at the same time. The result is that the boiling water will freeze the faster.

The belief in the Mpemba effect continued strong into the 17th century. Francis Bacon and Descartes both wrote extensive works on the scientific method, and experiments, and both wrote about the Mpemba effect. In the *Novum Organum*, Francis Bacon wrote

. . . water a little warmed is more easily frozen than that which is quite cold. . .

And in 1637, Descartes wrote about this phenomenon in *Les Meteores*, a work which was published attached to his more famous *Discourse on Method* . He emphasized the importance of experiment, described how to analyze the density-dependence of water, and stated results about the freezing times:

We can see this by experiment, if we fill a beaker—or some other such container having a long, straight neck—with hot water, and expose it to freezing cold air; for the water level will go down visibly, little by little, until the water reaches a certain level of coldness, after which it will gradually swell and rise, until it is completely frozen. Thus the same cold which will have condensed or shrunk it in the beginning will rarefy it afterwards. *And we can also see by experiment that water which has been kept hot for a long time freezes faster than any other sort*, because those of its parts which can least cease to bend evaporate while it is being heated. (Emphasis added.)

A modern writer on Descartes has commented on the italicized statement: "This statement, which the simplest of experiments could have refuted, was repeated with elaborate details in a letter to Mersenne, and it emphasizes Descartes' readiness to rely on *$\grave{a}$ priori* conclusions" . But this modern writer's position is contradicted by the letter to Mersenne, in which Descartes makes clear that he has, in fact, done this experiment. In this 1638 letter, Descrates wrote :

As with Roger Bacon's earlier experiment, we again see that the details of the experiment are crucial. Descartes is *not* measuring the time for the hot water to freeze, but is saying that when water has been heated, it is somehow changed so that it cools more easily, even after being brought back to room temperature. While the observation described is different than our modern statement of the Mpemba effect, it is similar in that some sort of history-dependence (i.e. memory) of the water is implied by the results. Descartes's letter also indicates that both he, and others, have done this experiment, and that the results are contradictory, a problem that we will also see in more modern experiments.

With the advent of the modern theory of heat, these earlier observations were forgotten. Since these experiments *appear* to contradict what we know about heat, it is natural to dismiss them as mistakes.

Presentations in textbooks typically show the progress of science as a simple, straight-line progression, with experiments pointing in a straightforward and unambiguous manner to new scientific theories. But, as Kuhn has pointed out, the development of scientific theories is not so simple . Most of the time, scientists are engaged in what Kuhn calls "normal science," during which research ". . . is a strenuous and devoted attempt to force nature into the conceptual boxes supplied by professional education. . . ."  When scientists are working under a certain paradigm, results that cannot be forced into the existing paradigm may be ignored, as attention is focused on experiments that appear more promising for advancing knowledge. To give but one of Kuhn's examples :

In the eighteenth century, for example, little attention was paid to the experiments that measured electrical attention with devices like the pan balance. Because they yielded neither consistent nor simple results, they could not be used to articulate the paradigm from which they derived. Therefore, they remained mere facts, unrelated and unrelatable to the continuing progress of electrical research. Only in retrospect, posessed of a subsequent paradigm, can we see what characteristics of electrical phenomena they display.

Whatever one's opinion on Kuhn's more controversial theories, it is clear that scientists' theoretical views influence what experiments they choose to do, to trust and to think about.

The Mpemba effect illustrates the points raised by Kuhn. In modern times, because the Mpemba effect *appears* to contradict modern theories of heat, scientists are much more skeptical, or even dismissive, of experiments that observe the Mpemba effect. Furthermore, like the eighteenth century experiments with pan balances, experiments on the Mpemba effect, for reasons we will discuss, yield "neither consistent nor simple results." Thus, the Mpemba effect, while interesting, is a factual curiosity, and not fundamental to our modern understanding of heat.

In *The Structure of Scientific Revolutions*, Kuhn briefly mentions these experiments by Marliani and Bacon :

. . . the natural histories often juxtapose \[correct\] descriptions. . . with others, say, heating by antiperistasis (or by cooling), that we are now quite unable to confirm.

Kuhn does not cite any experimental evidence that we are unable to confirm these older results, and one suspects that he is simply assuming this point . In fact, as we will see, at the time that Kuhn wrote this, there were multiple papers confirming the existence of the Mpemba effect. Kuhn thus unintentionally, and ironically, demonstrates how our theoretical expectations can color our experimental beliefs.

# Erasto B. Mpemba and 20th century knowledge

This strange phenomenon was reintroduced to the modern scientific community by Erasto Mpemba, a secondary school student in Tanzania, in 1963. Mpemba told his story in *Physics Education* . Mpemba and his fellow students were making ice cream, which used a mixture that included boiled milk. Because excessively hot objects could damage the refrigerator, they were supposed to let their mixture cool before putting it in the refrigerator. However, space in the refrigerator was scarce, and when another student put his mixture in without boiling the milk, Mpemba decided to put his hot mixture in, without waiting for it to cool. Later, Mpemba found that his hot mixture froze first.

Mpemba asked his teacher for an explanation, but his teacher said Mpemba must have been confused. When the teacher later covered Newton's law of cooling, Mpemba persisted in his questioning. The teacher responded that "Well, all I can say is that is Mpemba physics and not the universal physics," and from then on the teacher and the class would mock his mistakes by saying "That is Mpemba's mathematics," or "That is Mpemba's physics."

Mpemba ran more systematic experiments, both with water and milk, and continued to get similar results. When Dr. Osborne, a professor at a nearby university, visited Mpemba's school, Mpemba asked him why water at $100^O C$ froze faster than water at $35^O C$. Upon returning to his university, Dr. Osborne asked a technician to test Mpemba's question. When Dr. Osborne later asked for the results, "The technician reported that the water that started hot did indeed freeze first and added in a moment of unscientific enthusiasm *'But we'll keep on repeating the experiment until we get the right result'* " (emphasis added) . But future experiments gave similar results, and Mpemba and Osborne later published their results . In the same year, Dr. Kell of Canada independently reported the phenomenon, along with a theoretical explanation that we will consider later .

Subsequent discussions in journals showed that the effect was already known and believed by non-scientists in diverse regions of the world. Kell stated that it was widely believed in Canada, and that

Some will say that a car should not be washed with hot water because the water will freeze on it more quickly than cold water will, or that a skating rink should be flooded with hot water because it will freeze more quickly.

Mpemba reported that after his initial experience, he found that ice cream makers in Tanga City, Tanzania, used hot liquids to make the ice cream faster. British letters to the *New Scientist* reveal a wealth of lay observations. One writer stated that it was well-known that in the winter, hot water pipes were more likely to freeze than cold water pipes  (an issue we consider in section ). Another stated that the phenomenon was well-known in the food-freezing industry . A number of letters reported that their (non-scientist) friends and family had known of this effect even in the 1920's . One writer was a science teacher, whose story closely parallels Mpemba's. The teacher describes his experiences with a student who asked him why hot water froze faster than cold

. . . I told him that it was most unlikely, but he replied that he had seen it happen when his mother threw out her washing water onto the path. I explained to him that the particles in the hot liquid would be escaping more readily to the atmosphere due to evaporation and that this would leave a thinner layer of liquid to freeze than in the colder one where the particles would be escaping more slowly. He was not, however, convinced, and a few days later he reported that he had placed two cans, one containing hot water, and the other cold water, outside and that the hot one was still the first to freeze. Finally, he challenged me to explain that one, if I could. Still in no doubt I criticized his experiment and suggested he attempt a more accurate one under laboratory conditions.

He obtained two identical specimen jars and placed hot water (approx $50^O$ C) in one and colder water (approx $20^O$ C) in the other; both tops were left off and they were placed in the freezing compartment of a refrigerator. To my disbelief, and his delight, the hotter one was indeed the first to form a layer of ice on the surface.

He was, to say the least, no longer quite so impressed by my capabilities as a Science Teacher.

The skepticism with which scientists react to the Mpemba effect is quite common. In this author's experience, scientists are much more likely to react with disbelief than laypersons; this is not surprising, since scientists know enough to "know that this is impossible."

Mpemba's story provides a dramatic parable against dismissing the observations of non-scientists. But his story is particularly interesting because it is more than just a story of bad, close-minded, scientists. There is, of course, no excuse for the Tanzanian teacher's mockery of Mpemba. But is it "unscientific" for scientists to, upon hearing of the Mpemba effect, immediately suspect errors in the experiment? Kuhn emphasizes that with the operation of "normal science," scientists interpret experiments in light of the reigning paradigm—i.e. with their preexisting theoretical understanding .

What is interesting about the Mpemba effect is that unlike the examples commonly given in science textbooks, where theory and experiment march hand-in-hand, always leading to further progress, here theory (rightly or wrongly) prevents acceptance of experiment. We are certainly not arguing that the reaction of the scientific community to surprising experimental results is arbitrary, or necessarily hostile. Our point is simply that the reaction to an experiment depends significantly on how well that experiment matches accepted theoretical preconceptions. Because experimental claims can be in error, scientists do not just accept all published claims. While few scientists would find this claim controversial, it is quite different than the impression one gets from science textbooks, and from what appears in certain positivistic views of science. The Mpemba effect provides a lovely case for considering these issues, because while it provokes skepticism, it has been observed in multiple experiments; yet, in support of the skeptical position, we will see that the experimental results are not entirely consistent, and that the theoretical situation is still unsettled.

# What is the question, and is it scientific?

To analyze the Mpemba effect, we first need to precisely state what we are trying to test. At first sight, the question is quite simple: "Does hot water freeze faster than cold?" However, a little thought shows that this formulation is not adequate. Clearly, a small drop of hot water can freeze faster than a cold ocean. Hot water in a freezer will freeze faster than cold water on a warm day (as the latter will not freeze at all). These examples are silly, but illustrate the need to state the question clearly.

A better second attempt at stating the problem might be "Given two bodies of water, which are identical in all parameters (mass, shape, surroundings, etc. . .), except that one is initially at a higher uniform temperature than the other, the hotter water will freeze first." But further thought shows that this cannot be correct. If the initially hotter water is at $99.9999^O C$, and the initially colder water is at $0.0001^OC$, then the initially cold water is just seconds away from freezing, and it is clear that the hot water cannot possibly overtake it. Furthermore, there is clearly no reason to expect the Mpemba effect to occur for *all* possible initial parameters.

So a better phrasing might be "There exists a set of initial parameters, and a pair of temperatures, such that given two bodies of water identical in these parameters, and differing only in their initial uniform temperatures, the hot one will freeze sooner." This is a much better statement of the question, although we will see later that deficiencies remain.

Once the Mpemba effect is properly stated, it is clear that we are only looking for *some* set of parameters, such that if we plot freezing time versus initial temperature, there is *some* range of the graph that is downwards sloping—not that there exist initial parameters such that the graph is downwards-sloping all the way from $0^O C$ to $100^O C$, or that the graph has downwards-sloping parts for any set of initial parameters. This is logically necessary for the problem to be at all reasonable, but this point is not always appreciated in popular discussions. Consider the discussion of the Mpemba effect in Ann Landers' column, as described in *Myth-Informed*

warm water \[freeze\] faster than cold water. . . The redoubtable Ann Landers got into an ongoing row with readers in 1983 when she addressed this question, as well as the related cosmic issue of whether cold water boils faster than hot water. She "went to the top" and consulted Dr. Jermore Weisner, chancellor of the Massacusetts Institute of Technology, who kicked the problem over to the MIT dean of science, Dr. John W. Deutch. Landers never recorded what Deutch thought of being given such a problem by an advice column, but the eminent scientist reported "Neither statement was true." Whereupon "Self-Reliant in Riverdale". . . upbraided her for using "argument by authority" rather than doing her own experiment. "Self-reliant" said she reached the same conclusion as Deutch by using a pan of hot water, a thermometer, a stove, a refrigerator, and a watch with a second hand.

In his popular column, *The Straight Dope*, Cecil Adams also discussed the Mpemba effect :

. . . I carefully measured a whole passel of water into the Straight Dope tea kettle and boiled it for about five minutes. This was so I could compare the freezing rate of boiled $H_2O$ with that of regular hot water from the tap. (Somehow I had the idea that water that had been boiled would freeze faster.)

Finally I put equal quantities of each type into trays in the freezer, checked the temp (125 degrees Fahrenheit all around), and sat back to wait, timing the process with my brand new Swatch watch, whose precision and smart styling have made it the number one choice of scientists the world over.

I subsequently did the same with two trays of cold water, which had been chilled down to a starting temperature of 38 degrees.

The results? The cold water froze about 10 or 15 minutes faster than the hot water, and there was no detectable difference between the boiled water and the other kind. Another old wives' tale thus emphatically bites the dust. Science marches on.

These discussions fail to appreciate that a single test cannot show that the effect never occurs for any parameters and initial temperatures.

Further consideration of this point brings up another issue. Logically, our statement about the Mpemba effect can never be proven false. Regardless of how many experiments Cecil Adams runs, a true believer in the Mpemba effect can always claim that the effect occurs for other sets of initial parameters, that differ slightly from the ones that Cecil Adams used. Popper has famously argued that the hallmark of a scientific hypothesis is that it be falsifiable . That is, not that it can be proven true, but that it can be proven false. Is our most recent statement of the Mpemba effect thus unscientific?

It is not unusual that a scientific phenomenon, strictly speaking, cannot be proven impossible, because the parameter space in which it might occur is, in principle, infinite. However, if we search a "representative sample" of the parameter space over which the phenomenon might be thought to occur, and the phenomenon is not seen, that would be fairly convincing evidence against it.

We thus need a list of experimental parameters that we might vary when studying the Mpemba effect. The list is rather long (infinite). On a first page, we might list the mass of the water, the shape of the container, the surrounding environment of the refrigerator, and the gas content of the water. Note that several items on this list are not single parameters—characterizing the shape of the container requires multiple parameters. We may also want to include boolean parameters, such as whether the container has a lid. On a second page, we might list the color of the container, the electrical conductivity of the walls of the refrigerator, the gender of the experimenter, etc. . . If we simply list all the parameters we might think of, the list is infinite, and we are at a complete loss as to how to proceed with an experiment. Without a theoretical framework in which to design and conduct the experiment, we are reduced to randomly collecting facts, such as the color of the container, and the situation is impossible. However, we have strong theoretical reasons for ignoring all the parameters on the second page, and the experimenter can safely ignore the color of the container, neither varying, nor recording, its color.

Unfortunately, as we will see in next section, all the items of the first page can plausibly be considered important for the Mpemba effect. Furthermore, their effects are not independent of one another. But an experimenter can hardly be expected to establish a vast multidimensional array of containers of different dimensions and shapes, independently varying masses, gas contents, and refrigeration methods.

We are, of course, not claiming that scientific investigation of the Mpemba effect is impossible. But a common response, upon first hearing of the Mpemba effect, is that it should be a straightforward matter to study experimentally. But for even this (deceptively) simple problem, productive experimental design requires at least some theoretical understanding of why the effect might occur—otherwise we will not know whether we should be paying attention to, for example, the gas content of the water. We will see in the next section that because the time to freezing is sensitive to all the "first page" parameters, the experimental results become very confusing.

Our statement of the Mpemba effect now reads "There exists a set of initial parameters (water mass, gas content of water, container shape and type, and refrigeration method), and a pair of temperatures, such that given two bodies of water identical in these parameters, and differing only in their temperatures, the hot one will freeze sooner." One final difficulty that must be considered is how to define the time of freezing—are we considering it frozen when ice crystals first appear, or only when the entire body of water is frozen? Or, to simplify the experiment, we might just measure the time until some specified part of the water reaches $0^O$ C. This might seem like a minor issue, but we will see in section , on supercooling, that it is potentially crucial.

# How could the Mpemba effect occur?

We have deliberately avoided discussing the theoretical explanations for the Mpemba effect until now. We have done this both because the historical reaction to the Mpemba effect is only comprehensible in light of the effect's *apparent* inconsistency with modern conceptions of heat, and to emphasize the need for an experimental framework when designing experiments on the effect. Here, we discuss some proposed mechanisms for the Mpemba effect, but will not attempt to analyze their relative plausibility in depth.

To see how the effect might occur, it is useful to carefully think about why the effect appears impossible. The careful reader may already have come up with a proof of impossibility that goes something like this:

Suppose that the initial temperatures for the hot and cold water are $70^O C$ and $30^O C$. Then the $70^O C$ must first cool to $30^O C$, after which it must do everything the $30^O C$ water must do. Since the $70^O C$ water has to do everything the $30^O C$ water must do, plus a little more, it must take longer to freeze.

A good way of systematically analyzing the Mpemba effect is to think about why this proof is wrong.

The problem with this proof is that it implicitly assumes that the water is completely characterized by a single parameter—the temperature. We need to think of a parameter that might change during the course of the experiment; then, the $70^O C$ water cooled to $30^O C$ will not be the same as the water initially at $30^O C$.

One possible parameter is the mass of the water. Both bodies of water initially have the same mass. But if the initially hotter water loses mass to evaporation, then the $70^O C$ water cooled to $30^O C$ will, having less mass, be easier to freeze (i.e. less energy will need to be removed to cool and freeze it). This is one of the strongest theoretical explanations for the Mpemba effect. Kell numerically integrated the heat loss equations, assuming that the cooling was by evaporation alone, and that the mass lost to evaporation never recondensed—-he found that with these assumptions, there were initial temperatures where the hot water would freeze faster than the cold water . But this does not prove that evaporative cooling is the *only* factor behind the Mpemba effect. A number of experimenters have claimed that the amount of mass they lost to evaporation was insufficient to explain their results . And Wojciechowski et. al. observed the Mpemba efect in a *closed* container, which further suggests that evaporative cooling is not the sole cause of the effect .

A more complex "parameter" is the temperature distribution of the water. As the water cools it will develop convection currents, and the temperature will become non-uniform. This also defeats the impossibility proof above, since the water is no longer characterized by a single number. Analysis of the situation is now quite complex, since we are no longer considering a single parameter, but a scalar function, and computational fluid dynamics is notoriously difficult. Nevertheless, at least one general point can be made. For temperatures above $4^O C$, hot water is less dense than cold water, and will thus rise to the top. So we can generally expect that when the $70^O C$ water has cooled to an average temperature of $30^O C$, the top of the water will be hotter than $30^O C$, and the bottom of the water will be below $30^O C$. If the water primarily cools at its surface, the non-uniform distribution with an average temperature of $30^O C$ will thus lose heat faster than uniformly $30^O C$ water. Consistent with this, Deeson found that gentle stirring substantially raised to time to freezing . Convection could work in concert with other factors, such as evaporative cooling. Convection currents are sensitive to the shape and dimensions of the container, so this explanation may play very different roles for different container shapes.

This brings up the question of what "parameters" should be associated with the surrounding air. Modeling the cooling process should take into account the convection currents of the air, which will depend on the shape of the refrigerator. Firth's studies of the Mpemba effect found that this was an important factor :

What these experiments have shown, however, is that it is not just the beaker or the initial temperature of its contents that are important, but that its environment probably influences the cooling rates to a greater extent than any aspect of the beaker itself.

On a related note, one letter writer to *The New Scientist* suggested that the Mpemba effect could be explained if the containers of water were sitting on layers of frost. He argued that the frost conducts heat poorly, and the hot water causes the layer of frost to melt, establishing better thermal conduct with the refrigerator floor . This may explain some everyday observations of the effect, but the experiments published in refereed journals generally used containers on thermal insulators.

Another possibility is that the hot and cold water, while they appear identical to the naked eye, differ in their gas content. Hot water can hold less dissolved gas than cold water, and the gas content affects the properties of the water. Mpemba and Osborne's original experiments were done with recently boiled water, to remove dissolved air , as were the experiments by Walker . These experiments suggest that dissolved gasses are not necessary to the Mpemba effect. (Although under typical conditions, the degassed water will slowly regain dissolved gasses from the atmosphere, confusing matters.) On the other hand, Freeman only observed the Mpemba effect when carbon dioxide was dissolved in the water . Similarly, Wojciechowski et. al. only saw the effect for non-degassed water . A number of explanations of have been proposed for how the amount of dissolved gas could affect the properties of the water, and thus cause the Mpemba effect, although they remain largely speculative. One of the few quantitative findings was by Wojciechowski et. al., who reported that for water saturated with carbon dioxide, the enthalpy of freezing was smaller for the initially warmer water (but that preheating was irrelevant to the enthalphy if dissolved gasses were absent) . We will return to the effects of dissolved gases, and other impurities, when we discuss supercooling.

All the factors discussed are at least potentially important in explaining the Mpemba effect. What makes the situation particularly difficult to analyze is that the factors are not independent of each other—for example, rates of evaporative cooling will depend on the shape of the container. The experimental results described here do not all point to a single, clear, conclusion, and the reader who further inspects the original papers will find more facts, but will not find the overall picture much clearer .

Because there are so many factors that can be varied, and the results of the experiments can depend sensitively on any of these factors, experimental results are varied and difficult to organize into a consistent picture. (Recall Kuhn's statement about 18th century pan balance experiments, quoted in section .) It is thus unclear which of these explanations is *the* explanation, or indeed, whether it is appropriate to look for a single explanatory factor, isolated from the other factors. As Firth wrote ,

There is a wealth of experimental variation in the problem so that any laboratory undertaking such investigations is guaranteed different results from all others.

In figure  we see experimental results by Walker, showing the dependence of the time of freezing on the initial temperature, under various initial conditions. We see that some graphs show a strong Mpemba effect, some only show a weak one, and some show no Mpemba effect at all. This indicates that the cooling is indeed sensitive to a number of parameters. Furthermore, Walker reported that while his results were mostly repeatable, he "still obtained strange large deviations on some of the results." .

# Supercooling and Bursting water pipes

It has often been said that hot water pipes burst from freezing more often than adjacent cold water pipes . This is different from the Mpemba effect, but it is similar to it, in requiring the water to have a "memory." The mechanism behind the bursting water pipes is better understood than that behind the Mpemba effect.

The water pipe claim was first investigated by F. C. Brown, in 1916 . He confirmed the claim by taking 100 glass test tubes, and filling 50 with tap water, and 50 with tap water that had been boiled. *After allowing all water to first reach room temperature*, he placed them outside in subzero temperatures. He found that 44 of the tubes with the boiled water burst, while only 4 of the tubes with non-boiled water first. Since all water was at the same temperature when placed outside (as with Descartes's experiment), and he was looking at the occurence of bursting, rather than the time until freezing, these are not tests of the Mpemba effect.

Freezing water will generally supercool. Supercooling to $-4^OC$ to $-6^OC$ is common, and much greater supercooling can occur for small samples . Once freezing starts, the ice-water mixture must go to $0^O C$. So when freezing begins, a finite fraction of the water must lose energy, and turn to ice, transferring heat to the remaining subzero water, whose temperature will rise to $0^O C$. Thus, the more supercooling occurs (i.e. the lower the temperature at which freezing begins), the larger the volume fraction of water that will freeze initially (i.e. the larger the fraction of $H_2 O$ molecules that will be in ice structures). Also, a certain volume fraction of ice will *not* always correspond to the same volume of region spanned by the ice. The ice will form a dendritic structure, interspersed with liquid. If more supercooling has occurred, more of the water will have reached subzero temperatures, and the dendritic structure will span a larger region, for a fixed volume fraction of ice.

Brown observed that the water that had been boiled first, later supercooled, while the nonboiled water did not, and argued that this difference was responsible for the difference in bursting behavior . He argued as follows: If water in a pipe freezes near $0^O C$, only a small amount of ice will be formed initially. This ice will be localized to the coldest regions, at the sides of the pipes, leaving a hole in the center. With further cooling, the hole will shrink, but until all the water has frozen, liquid water will still be able to flow through the hole. Furthermore, the water flow can break away the ice on the sides, relieving pressure. On the other hand, if the water supercools significantly before freezing, a larger structure of dendritic ice will form throughout the pipe—while only a finite fraction of the water will be turned to ice, this dendritic ice will span more of the pipe, possibly leaving no hole, thus blocking the flow of water, and resulting in a burst pipe. Consistent with this explanation, Brown found that the ice rose higher from expansion in the tubes with nonboiled water, indicating the existence of a central region in which the water was mobile, and the ice able to push up.

In 1977, Gilpin performed quantitative experiments that confirmed Brown's qualitative explanation . Gilpin exposed pipes to subzero temperatures, and measured the pressure gradient necessary to induce the flow of water, at various times after the ice had formed. He found that the more supercooling, the greater the pressure gradient needed, *for the same amount of total ice formed* (figure ). He concluded, as Brown had, that with more supercooling, the ice formed would be more likely to form a dendritic structure, spanning the pipe, and causing blockage. He also confirmed this picture with photographs of the cross-section of the pipe during the freezing process.

These experiments convincingly demonstrated that greater supercooling would lead to burst pipes, leading to the question of why initially hot water would supercool more than initially cold water. Brown argued it was because boiled water had less dissolved gas, and that the dissolved gas prevented supercooling . However, Dorsey, who carried out an extensive series of experiments on the factors affecting supercooling, over the course of ten years, disputed this . Dorsey found that dissolved gasses were not a significant barrier to supercooling; he also pointed out that, unlike boiled water, the water in hot water pipes does contain significant amounts of dissolved gas. Dorsey agreed that heated water supercooled more, and that this would result in burst pipes, but argued that the greater supercooling occurred because heating served to deactivate or expel "motes" (what we would today call nucleation sites) . Both explanations agree that some sorts of nucleation sites are deactivated by heating the water. Gilpin not only confirmed that hot tap water supercooled more than cold tap water, but that tap water left in an open container supercooled least of all—this can be explained by the fact that water in an open container will absorb impurities from the air, and these impurities can then act as nucleation sites .

Again, the phenomenon here is not the same as the Mpemba effect, but is related, in that it explains how the water can remember its history. However, note that the results here are in the wrong direction to explain the Mpemba effect. If initially hot water supercools more, then to freeze it has to reach an even lower temperature than initially cold water, which will lengthen the time that it takes to freeze.

# Supercooling and the Mpemba effect

Consideration of supercooling greatly complicates the Mpemba effect, and it is not clear how, or whether, it helps to explain it. We first need to decide precisely how we measure the "time to freezing." If we wait until the first appearance of ice, then the experimental situation is complicated by the randomness of supercooling, and multiple trials will be needed to get precise average times to freezing. For simplicity, many experiments have studied the time for some specified location in the water to reach $0^O$ C . Supercooling is irrelevant for these experiments, if the specified location is at or near the place that first reaches $0^O$ C.

Auerbach considered the relevance of supercooling to the effect . He found that initially hot water would supercool less (measuring the time to the first appearance of ice crystals) than initially cold water. Auerbach did not determine why this happened, but pointed out that the initially hotter water should have greater temperature shear, and that this shear is known to trigger crystallization . However, Auerbach's finding that heated water supercools less than nonheated water is directly opposite to the findings of Brown and Dorsey, described in the previous section. Furthermore, Auerbach had a relatively small number of trials, so the significance of his results is unclear.

While Auerbach's result is in the correct direction to explain the Mpemba effect (since if the hot water supercools less, this will help it freeze sooner), he actually found that the initially hot water actually took longer to freeze on average (due to the greater time the hot water took to reach $0^O C$) .

Given this, it is unclear whether Auerbach's experiments can be described as actually observing the Mpemba effect. Due to the random fluctuations in the times till freezing, Auerbach found that the hot water might freeze first by chance. He found that when the ambient temperature was $-5^O C >T_a>-8^O C$, the probability for a randomly chosen container of initially hotter water to freeze before a randomly chosen container of initially colder water was 53% . For ambient temperatures of $-8^O C>T_a>-11^O C$, the probability was 24%. If the hot water, on average, takes longer to freeze, but only happens to freezes first in some specific samples due to random fluctuations, it is not clear that this should be called a Mpemba effect. (Given Auerbach's small number of samples, 53% is not signficantly greater than 50%.) While the experiments on pipes show that supercooling can induce significant memory effects, the role of supercooling in the Mpemba effect remains uncertain.

# Future prospects

It is clear that evaporative cooling can play an important role in the Mpemba effect, and that the history of the water can affect the amount of supercooling. But beyond that, the experiments together paint a very muddled picture. More experiments are needed to solve this 2000+ year-old puzzle. Despite the theoretical complexity of the Mpemba effect, the experiments needed to probe it can be done at the K-12 and undergraduate level. Indeed, simple experiments on the Mpemba effect are a common science fair project.

A fair amount of thought needs to go into the experimental design before even the first data point is taken. Walker discusses the basic set-up, and while you can come up with your own design, reading Walker's article is a good way to appreciate some of the hidden complexities in the experiment . For example, Walker points out that the container should be heated along with the water, for if hot water is poured into a cold container, the sudden change in the water's temperature causes problems. To make sure that all samples of water have the same mass, masses need to be measured after heating, rather than before, as a fair amount of mass is lost during the heating process. The precise environment surrounding the container is important, and it can make a difference whether the water is in a middle of an empty freezer, or jammed between a frozen pizza and a frost-covered bucket of ice cream. The temperature can be read with a common mercury thermometer, but a device that can more quickly and accurately register changes in temperature is better, if available. The reader is directed to Walker's article for further discussion of potential problems and solutions .

A series of trials will produce a graph of freezing time vs. initial temperature, of the sort shown in figure . A single curve may or may not show a Mpemba effect, but is not, on its own, particularly useful for probing the cause(s) of this phenomenon. To see how the Mpemba effect depends on the various parameters of interest (initial mass, gas content, container shape and type, etc. . .) several curves need to be produced, under different parameter settings. If you had infinite resources and time, you could vary all the initial parameters, producing a giant multidimensional matrix of freezing times. In practice, you will have to decide which factors you are most interested in, and design your experiment accordingly.

You can decide on your own what to vary to produce your series of curves. I give some suggestions below, but the number of reasonable experimental designs is essentially unlimited.

Kell claimed that because the Mpemba effect relies on surface cooling, it is more likely to be observed in wooden pails than in metal ones, since in metal pails a fair amount of heat is lost through the sides. This claim can be tested by producing a series of curves of freezing time vs. initial temperature, for containers with differing degrees of thermal insulation on the sides. This would demonstrate how the Mpemba effect is affected as the relative importance of evaporation to the cooling process is varied. Alternatively, varying the height of the water, while keeping the base fixed, provides another way of varying the importance of evaporation.

The importance of evaporation can be largely eliminated by putting the water in a closed container, or by putting a layer of oil over the surface of the water. Such experiments would be extremely useful in assessing claims that evaporation is *the* cause of the Mpemba effect. As already discussed, several authors have claimed that evaporation is insufficient to explain their results. But I know of only one published paper studying the Mpemba effect in a closed container , and a single experiment can always be in error. If you run a series of experiments in closed containers, and regularly fail to find a Mpemba effect, that would provide good support for the claims that evaporation is the primary cause of the effect. On the other hand, observations of the Mpemba effect in closed containers would show that it can occur without evaporation.

Rather than looking at freezing time versus initial temperature, you could investigate supercooling. Simply reproducing earlier experiments would be valuable in resolving the discrepancy between the recent results of Auerbach, and the older results of Brown and Dorsey . The number of runs in Auerbach's experiment was too limited for firm conclusions to be drawn, so a repetition of his experiment with a greater number of runs would be useful. A modern repetition of Descartes' experiment would also be interesting.

More systematic studies of how the history of the water affects its properties would be helpful. For example, if you find that pre-boiled tap water has different properties than water straight out of the tap, you could investigate why it differs (dissolved oxygen? impurities?), by looking at how long it takes pre-boiled water's properties to return to those of tap water's, and under what conditions.

You should try rerunning the experiments several times with exactly the same parameters, to get an idea of how big the error bars are. The size of the error bars is crucial, since if a graph of freezing time vs. initial temperature shows only a weak local maximum, it will be unclear if this is a true Mpemba effect, or just the result of fluctuations. And, as with any experiment, you want to make sure that your results are repeatable. This greatly increases the time required for the experiment, and limits the amount you can vary the parameters, but it's better to have a small amount of reliable data than a large amount of unreliable data!

What will constitute experimental success? You do not need to observe the Mpemba effect for your experiment to be a success. Finding that the Mpemba effect does not occur under certain conditions is still a good experimental result. On the other hand, it is certainly more dramatic and psychologically satisfying if you can find conditions under which the effect occurs. And if you find conditions under which it occurs, you can then study what changes destroy the effect, which provides a potentially valuable probe of the phenomenon. You may want to do some preliminary testing to find parameters where the Mpemba effect occurs, and then decide how to proceed.

If you complete experiments on the Mpemba effect, I'd be interested in hearing the results—if you have a chance, send me an e-mail telling me what you found. If enough experiments are done, perhaps this 2000+ year-old problem can be solved!

Finally, those who like the counterintuitive nature of the Mpemba effect might be interested in a similar phenomenon: water drops will last longer on a skillet well above $100^O C$, than on one only a little above $100^O C$. This is easier to explain than the Mpemba effect .