Is this paragraph about the dihedral effect in the FAA's Glider Flying Handbook correct?
No, the explanation is not correct.

Tanner, you are right when you say that lift does not depend on bank angle. Lift is caused by a pressure difference, and pressure can only act perpendicularly to a surface. Therefore, the lift on each wing and its lever arm to the center of gravity won't change with bank and no "correcting" rolling moment is created.

Instead, what does happen is a side force from the wing's bank angle which will accelerate the airplane sideways. This in turn will result in a sideslip condition, and only now will a dihedral effect show up: By changing the angle of attack on each side differently, the sideslip in combination with dihedral will create a correcting rolling moment.

As you correctly observe, a stabilising effect can only come from a difference in angle of attack between both wings, and that only occurs in a sideslip.