How does sensor fusion work? I want to understand the math/physics/algorithms
Sensors provide analog or digital insights into a reality that is difficult to grasp. Just like your senses do. 

Sensors are designed to seize a special portion of a measurable phenomenon: electrical, chemical, physical...  Just like your senses do. The usual five ones are sight, hearing, touch, smell and taste, but there are more to human skills.

Understanding a physical phenomenon via signal processing is like trying to tell the main ingredients from the  recipe of the meal you ordered in a restaurant last evening. You do not know what happened in the kitchen, yet you have chosen what to order, and have seen, tasted, smelled, even touched the plate. 

Each of your senses and understanding provided you with hints or knowledge, but you can never be sure, as some cooks master sense deception. What you see as an artifial green marmelade made of strawberries can taste like apple jelly.

Based on you cooking expertise, the combination of all your senses can bring you closer to the the actual recipe, provided that:


you can use your senses often enough: if you take only one bite every 10 days, it is unlikely you get the menu. This is data sampling;
each sense is sensitive enough. This is sensor sensitivity;
your sense list is close to complete, and the coverage is sufficient for your purpose. If you cannot taste sugar anymore, you won't be able to cook some meals for people who still can taste it. This is measurement span;
you are able infer, or model some of the processes you are unaware of. This is modeling.


Sensor fusion is the art and science of combining sensory data, knowledge and models from disparate sources so that the resulting information has more validity or less uncertainty than individual  sources. 

Examples: The act of averaging $N$ sensor data for the same deterministic signal, with different stochastic realizations of a noise term, is the most basic sensor fusion operation. With the model of independent Gaussian noises of deviation $\sigma$, the average theoretically yields a $\frac{\sigma}{\sqrt{N}}$ deviation(less uncertain). The combination of three channels (Red, Blue, Green) offers a color image (more valid than a monochannel gray-scale). 



The first example combines information from a single sensor model. The second one from sensors operating in the same domain (electromagnetic waves), yet in different portions of the spectrum. In general, fusion operates on different sensors with different rates, ranges, domains and mostly units.

The problem is highly dependent on the phenomenon you are looking at, the available sensors, and the information you are looking for. 

The physics tell you the potential information you can get from your sensors. The mathematics can model how they are related or complementary, or what is irrelevant information (noise). The algorithms will combine the previous knowledge as optimally as possible, in terms of precision, accuracy or speed. 

The topic is related to the realms of Sensor fusion, Data fusion or Information integration, with a short overview in Principles and Techniques for Sensor Data Fusion. Many more books are available, as in Best book for learning sensor fusion, specifically regarding IMU and GPS integration.

Regarding your actual problem, a first step would consist in understanding What are the differences between a gyroscope, accelerometer and magnetometer? which could help you push techniques a little further. And achieve the goal of fusion: using sensor differences, sum their data, in the most clever way.