Relationship between wing aspect ratio and speed
It's really the other way around, speed of the aircraft affects aspect ratio.

Hawks fold their wings to dive at high speeds, and have them straight out for lower speeds.
At high speeds the main concern is drag.  The less drag, the faster you can go.  At lower speeds, the main concern is lift.  Here a higher aspect wing is more efficient.


  Why not use a smaller high aspect wing at higher speeds, and just vary the sweep?


This is done, but along with higher speed comes greater aerodynamic forces.  This is one reason why modern fighters feature low aspect wings and deltas.  These are simply much more able to withstand higher G forces.

The variable sweep wing concept was developed in the 1950s and 1960s, but generally carries a weight penalty.  Variable camber is widely seen in airliners today.
Some military aircraft, such as the B-1 bomber, use variable sweep to expand their useful 
flight envelope, giving them the ability to "get low and fast".

Cruise missiles, optimized for one speed only with little abrupt maneuvering, can be found with very simple high aspect swept wings, which offer an improvement in drag over a lower aspect wing, thereby increasing fuel efficiency and extending range 

There is no direct relationship with speed and aspect ratio, indeed, high subsonic aircraft such as airliners, that strive for maximum efficiency, are seen with increasingly higher aspect ratios in recent years.

Structural strength is the main limitation which continues to challenge engineers today.