|
--- |
|
license: apple-amlr |
|
--- |
|
<h1 align="center"> Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions </h1> |
|
|
|
<p align="center"> |
|
<a href="https://arxiv.org/abs/2502.18435">📃 Paper</a> |
|
• |
|
<a href="https://machinelearning.apple.com" >📝 Blog</a> |
|
</p> |
|
|
|
This model card accompanies the research paper, [Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions](https://arxiv.org/abs/2502.18435). |
|
|
|
Here we release 4 models' checkpoints trained with simulation data described in out paper Section 4. |
|
Please follow our github README to download and evaluate these models. |
|
|
|
|
|
<div align="center"> |
|
|
|
### Results of the Controlled Simulation Study of 4-Digits Multiplication |
|
|
|
| || **Forward X** | || **Reverse X** | | |
|
|:--|:--:|:--:|:--:|:--:|:--:|:--:| |
|
| | **L2R** | **R2L(m,n)** | **R2L(m)** | **R2L** | **L2R(m,n)** | **L2R(n)** | |
|
| **Test Accuracy (%)** | **99.81±0.15** | 59.71±1.99 | 60.93±0.88 | **100±0** | 97.82±0.35 | 99.85±0.10 | |
|
| **Train Accuracy (%)** | **99.76±0.15** | 59.03±1.66 | 61.22±1.12 | **100±0** | 97.90±0.42 | 99.98±0.04 | |
|
| **Test Cond. Ent. (nats)** | 0.06 | 1.18 | 0.08 | 0 | 0.84 | 0.01 | |
|
| **Train Cond. Ent. (nats)** | 0.06 | 1.17 | 0.08 | 0 | 0.83 | 0.01 | |
|
| **Theo. Cond. Ent. (nats)** | 0 | 1.49 | 0 | 0 | 1.49 | 0 | |
|
| **Training loss** | **0.86** | 0.94 | 0.94 | **0.86** | 0.94 | 0.94 | |
|
|
|
</div> |
|
|
