Explained intuitively, why does aspect ratio affect induced drag?
Let's imagine 2 theoretical wings, both of which have the same area, but differ in aspect ratio.

Then the wing with the higher aspect ratio also has more span. This is what counts.

If induced drag depends on the downwash angle, why would a longer wingspan reduce the angle?

Because the wider wing will affect more air. Think of the air affected by the wing as what flows through a tube, the diameter of which equals wing span. Like in the picture below.

Of course, for clarity, the amount of bending the stream tube undergoes due to the wing is greatly exaggerated here. In reality it is a few degrees at most.

how can that happen when the wing area is equal?

Wing chord does not count here as long as the amount of stream tube bending can be done by the skinnier wing. More chord will not affect the result. Only when too much being is asked of a limited chord will the wing stall. At lower lift coefficients, more chord will only increase friction drag.
In reality the influence of the wing fades the more you move away from it. This picture of a stream tube is a simplification which you will also find in propeller theory, and it gives the correct result, because what you neglect in the far field you compensate by assuming a constant influence over the near field.

All of the sources I am able to find blame induced drag solely on wingtip vortexes

You do well to disregard those sources. They have the causality backwards.