Why is the polar-nonpolar electronegativity difference defined to be 0.4?
Both the $0.4$ threshold for nonpolar/polar bonds and the $1.7$ threshold for polar/ionic bonds are arbitrary choices. When defining nonpolar, polar and ionic bonds, it is important to allow $\ce{C-H}$ bonds ($\Delta \chi = 0.4$) to be nonpolar — it is the best description of alkanes’ low polarities and it facilitates discussions so much. Likewise, a few compounds commonly known as salts should fall just within the ionic boundary for ease of discussion. Teaching introductory levels is just so much easier if you can draw things black and white.

At higher levels, it should always be remembered that not only the thresholds are arbitrary but even electronegativity as a concept follows quite arbitrary definitions.


Except for one electronegativity scale that links electronegativity to ionisation enthalpy and electron affinity, all definitions and all defined reference points (typically fluorine having $\chi = 4.0$) are arbitrary.
Choosing a definite value for a specific element is also arbitrary since the physical implication of electronegativity depends on:


bonding partners
structure (i.e. hybridisation)