Difference between criterion validity and reliability equivalence
Criterion validity concerns with measuring the right thing. For instance, GPA is likely to have criterion validity to measure a student's academic understanding. While the change in weight in the last semester has much less criterion validity to measure the same trait. Basically, if the measurement you use and the trait you want to measure has a high correlation, then there is likely criterion validity.

Reliability concerns mostly with measuring the thing right. For instance, if GPA can measure a student's academic understanding, and percent attendance can also measure a student's academic understanding, then GPA and percent attendance should correlate, aka, they are reliable. Before subjected to reliability assessment, the tests are usually checked if they are criterion-valid. However, it's possible to have two tests that are highly correlated (reliable) but invalid. Such as using dietary fat intake and serum lipid to predict a college graduate's earning potential.

Notice that there a few different types of reliabilities, the one you cited is more about alternate forms reliability, there are also test-retest reliability and inter-rater reliability, etc. 

Practically, they are not interchangeable. Validity happens between the true trait (or behavior) and the measurements. Reliability happens between two measurements (or modes/instances of measurement.)