ɪn ɐ bɹˈiːf ɹᵻvjˈuː, ˈɔːl mjˈuːzɪkz wˈɪljəm ɹˈuːlmən kˈɔːld ɹˈɛɹəɾiz ɐ kəlˈɛkʃən ʌv "ɔːltˈɜːnət tˈeɪks, sɚpɹˈaɪzɪŋ kˈʌvɚ vˈɜːʒənz, ænd dˈɪfɹənt mˈɪksᵻz ʌv fəmˈɪlɪɹ sˈɔŋz mˌeɪk ðɪs ɐ dᵻlˈaɪt fɔːɹ kəlˈɛktɚz ænd hˈɑːɹdkˈoːɹ fˈænz.
In a brief review, AllMusic's William Ruhlmann called Rarities a collection of "alternate takes, surprising cover versions, and different mixes of familiar songs make this a delight for collectors and hard-core fans.