How to use a Hall effect sensor in a rotary encoder?
As a simple explanation, consider that the distance between the Hall sensor and the magnets changes as the gear rotates.
The sensor is flat, and wider than it is thick.  Because of this, the distance between the magnets and the sensor varies as the gear rotates.
The magnetic fields of the magnets are also concentrated at two points, so you get a maximum effect only when the sensor is properly aligned so that it is exposed to the maximum flux from the magnets.
As Marko Buršič noted in the comments, it would help to spread the magnetic flux around the whole circle.
A ring of iron inside the magnets and around the sensor would help.  The magnets should be &quot;stuck&quot; on the outside of the iron ring, with the sensor inside the iron ring.