So, the goal of this project was to scrape the results of the top 100 list, query a database to see if those titles were within it, and return back information of all top 100 songs not contained within said database. The datasets are as follows:

```
top 100 = {{"1": {"artist": "Lil Nas X Featuring Billy Ray Cyrus", "rank": 1, "title": "Old Town Road"}, "2": {"artist": "Taylor Swift", "rank": 2, "title": "You Need To Calm Down"}, "3": {"artist": "Billie Eilish", "rank": 3, "title": "Bad Guy"}, "4": {"artist": "Khalid", "rank": 4, "title": "Talk"}, "5": {"artist": "Ed Sheeran & Justin Bieber", "rank": 5, "title": "I Don't Care"}, "6": {"artist": "Jonas Brothers", "rank": 6, "title": "Sucker"}, "7": {"artist": "Drake Featuring Rick Ross", "rank": 7, "title": "Money In The Grave"}, "8": {"artist": "Post Malone", "rank": 8, "title": "Wow."}, "9": {"artist": "DaBaby", "rank": 9, "title": "Suge"}, "10": {"artist": "Chris Brown Featuring Drake", "rank": 10, "title": "No Guidance"}, "11": {"artist": "Post Malone & Swae Lee", "rank": 11, "title": "Sunflower (Spider-Man: Into The Spider-Verse)"}, "12": {"artist": "Sam Smith & Normani", "rank": 12, "title": "Dancing With A Stranger"}, "13": {"artist": "Shawn Mendes", "rank": 13, "title": "If I Can't Have You"}, "14": {"artist": "Lizzo", "rank": 14, "title": "Truth Hurts"}, "15": {"artist": "Taylor Swift Featuring Brendon Urie", "rank": 15, "title": "ME!"}, "16": {"artist": "Halsey", "rank": 16, "title": "Without Me"}, "17": {"artist": "Ava Max", "rank": 17, "title": "Sweet But Psycho"}, "18": {"artist": "Polo G Featuring Lil Tjay", "rank": 18, "title": "Pop Out"}, "19": {"artist": "Ariana Grande", "rank": 19, "title": "7 Rings"}, "20": {"artist": "Marshmello & Bastille", "rank": 20, "title": "Happier"}, "21": {"artist": "Blake Shelton", "rank": 21, "title": "God's Country"}, "22": {"artist": "Morgan Wallen", "rank": 22, "title": "Whiskey Glasses"}, "23": {"artist": "Panic! At The Disco", "rank": 23, "title": "High Hopes"}, "24": {"artist": "Panic! At The Disco", "rank": 24, "title": "Hey Look Ma, I Made It"}, "25": {"artist": "Lee Brice", "rank": 25, "title": "Rumor"}, "26": {"artist": "Young Thug, J. Cole & Travis Scott", "rank": 26, "title": "The London"}, "27": {"artist": "Daddy Yankee & Katy Perry Featuring Snow", "rank": 27, "title": "Con Calma"}, "28": {"artist": "Luke Combs", "rank": 28, "title": "Beer Never Broke My Heart"}, "29": {"artist": "Katy Perry", "rank": 29, "title": "Never Really Over"}, "30": {"artist": "J. Cole", "rank": 30, "title": "Middle Child"}, "31": {"artist": "benny blanco, Halsey & Khalid", "rank": 31, "title": "Eastside"}, "32": {"artist": "City Girls", "rank": 32, "title": "Act Up"}, "33": {"artist": "Mustard & Migos", "rank": 33, "title": "Pure Water"}, "34": {"artist": "Meek Mill Featuring Drake", "rank": 34, "title": "Going Bad"}, "35": {"artist": "Drake", "rank": 35, "title": "Omerta"}, "36": {"artist": "Tyler, The Creator", "rank": 36, "title": "Earfquake"}, "37": {"artist": "Thomas Rhett", "rank": 37, "title": "Look What God Gave Her"}, "38": {"artist": "Khalid", "rank": 38, "title": "Better"}, "39": {"artist": "Lady Gaga & Bradley Cooper", "rank": 39, "title": "Shallow"}, "40": {"artist": "A Boogie Wit da Hoodie", "rank": 40, "title": "Look Back At It"}, "41": {"artist": "Ariana Grande", "rank": 41, "title": "Break Up With Your Girlfriend, I'm Bored"}, "42": {"artist": "Travis Scott", "rank": 42, "title": "Sicko Mode"}, "43": {"artist": "Dan + Shay", "rank": 43, "title": "Speechless"}, "44": {"artist": "Halsey", "rank": 44, "title": "Nightmare"}, "45": {"artist": "Billie Eilish", "rank": 45, "title": "When The Party's Over"}, "46": {"artist": "Ed Sheeran Featuring Chance The Rapper & PnB Rock", "rank": 46, "title": "Cross Me"}, "47": {"artist": "Calboy", "rank": 47, "title": "Envy Me"}, "48": {"artist": "Kane Brown", "rank": 48, "title": "Good As You"}, "49": {"artist": "YG, Tyga & Jon Z", "rank": 49, "title": "Go Loko"}, "50": {"artist": "Jonas Brothers", "rank": 50, "title": "Cool"}, "51": {"artist": "Blanco Brown", "rank": 51, "title": "The Git Up"}, "52": {"artist": "Lil Tecca", "rank": 52, "title": "Ran$om"}, "53": {"artist": "DJ Khaled Featuring SZA", "rank": 53, "title": "Just Us"}, "54": {"artist": "Lewis Capaldi", "rank": 54, "title": "Someone You Loved"}, "55": {"artist": "P!nk", "rank": 55, "title": "Walk Me Home"}, "56": {"artist": "YK Osiris", "rank": 56, "title": "Worth It"}, "57": {"artist": "Cardi B & Bruno Mars", "rank": 57, "title": "Please Me"}, "58": {"artist": "Offset Featuring Cardi B", "rank": 58, "title": "Clout"}, "59": {"artist": "Luke Bryan", "rank": 59, "title": "Knockin' Boots"}, "60": {"artist": "Cardi B", "rank": 60, "title": "Press"}, "61": {"artist": "Maren Morris", "rank": 61, "title": "GIRL"}, "62": {"artist": "5 Seconds Of Summer", "rank": 62, "title": "Easier"}, "63": {"artist": "Meek Mill Featuring Ella Mai", "rank": 63, "title": "24/7"}, "64": {"artist": "Summer Walker X Drake", "rank": 64, "title": "Girls Need Love"}, "65": {"artist": "Eric Church", "rank": 65, "title": "Some Of It"}, "66": {"artist": "Dan + Shay", "rank": 66, "title": "All To Myself"}, "67": {"artist": "NLE Choppa", "rank": 67, "title": "Shotta Flow"}, "68": {"artist": "Bad Bunny & Tainy", "rank": 68, "title": "Callaita"}, "69": {"artist": "Jason Aldean", "rank": 69, "title": "Rearview Town"}, "70": {"artist": "Kelsea Ballerini", "rank": 70, "title": "Miss Me More"}, "71": {"artist": "Brett Eldredge", "rank": 71, "title": "Love Someone"}, "72": {"artist": "Beyonce", "rank": 72, "title": "Before I Let Go"}, "73": {"artist": "Florida Georgia Line", "rank": 73, "title": "Talk You Out Of It"}, "74": {"artist": "DJ Khaled Featuring Cardi B & 21 Savage", "rank": 74, "title": "Wish Wish"}, "75": {"artist": "Dreamville Featuring JID, Bas, J. Cole, EARTHGANG & Young Nudy", "rank": 75, "title": "Down Bad"}, "76": {"artist": "Chase Rice", "rank": 76, "title": "Eyes On You"}, "77": {"artist": "Lunay, Daddy Yankee & Bad Bunny", "rank": 77, "title": "Soltera"}, "78": {"artist": "Lil Uzi Vert", "rank": 78, "title": "Sanguine Paradise"}, "79": {"artist": "Marshmello Featuring CHVRCHES", "rank": 79, "title": "Here With Me"}, "80": {"artist": "Joji", "rank": 80, "title": "Sanctuary"}, "81": {"artist": "Sech Featuring Darell", "rank": 81, "title": "Otro Trago"}, "82": {"artist": "The Chainsmokers & Bebe Rexha", "rank": 82, "title": "Call You Mine"}, "83": {"artist": "Chris Young", "rank": 83, "title": "Raised On Country"}, "84": {"artist": "SHAED", "rank": 84, "title": "Trampoline"}, "85": {"artist": "Eli Young Band", "rank": 85, "title": "Love Ain't"}, "86": {"artist": "Billie Eilish", "rank": 86, "title": "Ocean Eyes"}, "87": {"artist": "Yella Beezy, Gucci Mane & Quavo", "rank": 87, "title": "Bacc At It Again"}, "88": {"artist": "Pedro Capo X Farruko", "rank": 88, "title": "Calma"}, "89": {"artist": "Travis Scott", "rank": 89, "title": "Wake Up"}, "90": {"artist": "Bryce Vine Featuring YG", "rank": 90, "title": "La La Land"}, "91": {"artist": "Jonas Brothers", "rank": 91, "title": "Only Human"}, "92": {"artist": "Marshmello Featuring A Day To Remember", "rank": 92, "title": "Rescue Me"}, "93": {"artist": "Megan Thee Stallion", "rank": 93, "title": "Big Ole Freak"}, "94": {"artist": "Nicky Jam X Ozuna", "rank": 94, "title": "Te Robare"}, "95": {"artist": "NAV Featuring Meek Mill", "rank": 95, "title": "Tap"}, "96": {"artist": "Ozuna x Daddy Yankee x J Balvin x Farruko x Anuel AA", "rank": 96, "title": "Baila Baila Baila"}, "97": {"artist": "Ali Gatie", "rank": 97, "title": "It's You"}, "98": {"artist": "Juice WRLD", "rank": 98, "title": "Robbery"}, "99": {"artist": "Nipsey Hussle Featuring Roddy Ricch & Hit-Boy", "rank": 99, "title": "Racks In The Middle"}, "100": {"artist": "Justin Moore", "rank": 100, "title": "The Ones That Didn't Make It Back Home"}}
database_results = [(u'Old Town Road', u'Lil Nas X featuring Billy Ray Cyrus'), (u'Talk', u'Coldplay'), (u'Talk', u'Khalid'), (u'Sucker', u'Jonas Brothers'), (u"I Don't Care", u'Buck Owens'), (u"I Don't Care", u'Fallout Boy'), (u"I Don't Care", u'Justin Bieber'), (u'Sunflower (Spider-Man: Into the Spider-Verse)', u'Post Malone & Swae Lee'), (u'Dancing With A Stranger', u'Sam Smith'), (u"If I Can't Have You", u'Shawn Mendes'), (u'Sweet But Psycho', u'Ava Max'), (u'Without Me', u'Halsey'), (u'Happier', u'Ed Sheeran'), (u'Happier', u'Marshmello'), (u"God's Country", u'Blake Shelton'), (u'Whiskey Glasses', u'Morgan Wallen'), (u'High Hopes', u'Panic! At  the disco'), (u'Beer Never Broke My Heart', u'Luke Combs'), (u'Never Really Over', u'Katy Perry'), (u'Hey Look Ma, I Made It', u'Panic! At the Disco'), (u'Speechless', u'Dan + Shay'), (u'Speechless', u'Hanson'), (u'Shallow', u'Lady Gaga & Bradley Cooper'), (u'BETTER', u'GUNS N" ROSES'), (u'Better', u'Khalid'), (u'Rumor', u'Lee Brice'), (u'Look What God Gave Her', u'Thomas Rhett'), (u"when the party's over", u'Billie Eilish'), (u'Cool', u'Gwen Stefani'), (u'Cool', u'Jonas Brothers'), (u'Beautiful Crazy', u'Luke Combs'), (u'Good As You', u'Kane Brown'), (u'Love Someone', u'Brett Eldredge'), (u'Love Someone', u'Lucas Graham'), (u'Someone You Loved', u'Lewis Capaldi'), (u'Miss Me More', u'Kelsea Ballerini'), (u'Walk Me Home', u'Mandy Moore'), (u'Walk Me Home', u'P!nk'), (u'Girl', u'Destiny\xb4s Child'), (u'Girl', u'Maren Morris'), (u'Girl', u'The Beatles'), (u"Knockin' Boots", u'Luke Bryan'), (u'Rearview Town', u'jason Aldean'), (u'All To Myself', u'Dan + Shay'), (u'Eyes On You', u'Chase Rice'), (u'Some of It', u'Eric Church'), (u'Here With Me', u'Mercyme'), (u'Talk You Out of It', u'Florida Georgia Line'), (u"Love Ain't", u'Eli Young Band'), (u'Heaven', u'Bryan Adams'), (u'Heaven', u'Derek Miller'), (u'Heaven', u'Kane Brown'), (u'Heaven', u'Salvador'), (u'Heaven', u'State of Sound'), (u'Heaven', u'Three Doors Down'), (u'Heaven', u'Warrant'), (u'Call You Mine', u'Chainsmokers (Feat. Bebe Rexha)'), (u'Ocean Eyes', u'Billie Eilish'), (u'On My Way to You', u'Cody Johnson'), (u'On My Way To You', u'Mercy Me'), (u'Raised on Country', u'Chris Young')]

```

I have been able to format the results so that they are structured into separate data frames.

*Top 100 List*

[![Top 100 List](https://i.stack.imgur.com/9ChNL.png)](https://i.stack.imgur.com/9ChNL.png)

*Database Search for Title from Top 100 List*

[![Database Results](https://i.stack.imgur.com/g8OGW.png)](https://i.stack.imgur.com/g8OGW.png)

What I would like to do is check for which values, from the top 100 list are not contained within the Database search results. The idea being, that I'd be able to generate a list of songs that would need to be purchased in order to put together a top 100 playlist. 

So far, I've been able to return back a list of items that are in each of the two data frames, with the following example: 

```
set(df_t['title']).intersection(set(df2['title']))

```

Which yields: 

```
{'All To Myself',
 'Beer Never Broke My Heart',
 'Better',
 'Call You Mine',
 'Cool',
 'Dancing With A Stranger',
 'Eyes On You',
 "God's Country",
 'Good As You',
 'Happier',
 'Here With Me',
 'Hey Look Ma, I Made It',
 'High Hopes',
 "I Don't Care",
 "If I Can't Have You",
 "Knockin' Boots",
 'Look What God Gave Her',
 "Love Ain't",
 'Love Someone',
 'Miss Me More',
 'Never Really Over',
 'Ocean Eyes',
 'Old Town Road',
 'Rearview Town',
 'Rumor',
 'Shallow',
 'Someone You Loved',
 'Speechless',
 'Sucker',
 'Sweet But Psycho',
 'Talk',
 'Walk Me Home',
 'Whiskey Glasses',
 'Without Me'}

```

But, this is sort of the opposite of what I'd like - I want to know which values from the top 100 list are not returned by the query - and it has the added problem that it doesn't account for the fact that, if another artist has a song by the same title, you could end up with a false positive. So, I tried something like this: 

```
set(df_t['artist'] + ': ' + df_t['title']).intersection(set(df2['artist']+ ': ' + df2['title']))

```

Which yields:

```
{'Ava Max: Sweet But Psycho',
 'Billie Eilish: Ocean Eyes',
 "Blake Shelton: God's Country",
 'Brett Eldredge: Love Someone',
 'Chase Rice: Eyes On You',
 'Dan + Shay: All To Myself',
 'Dan + Shay: Speechless',
 "Eli Young Band: Love Ain't",
 'Halsey: Without Me',
 'Jonas Brothers: Cool',
 'Jonas Brothers: Sucker',
 'Kane Brown: Good As You',
 'Katy Perry: Never Really Over',
 'Kelsea Ballerini: Miss Me More',
 'Khalid: Better',
 'Khalid: Talk',
 'Lady Gaga & Bradley Cooper: Shallow',
 'Lee Brice: Rumor',
 'Lewis Capaldi: Someone You Loved',
 "Luke Bryan: Knockin' Boots",
 'Luke Combs: Beer Never Broke My Heart',
 'Morgan Wallen: Whiskey Glasses',
 'P!nk: Walk Me Home',
 "Shawn Mendes: If I Can't Have You",
 'Thomas Rhett: Look What God Gave Her'}

```

So, it's able to filter out some results by including the need for a matching artist, but is not able to account for slight differences like: 

Lil Nas X Featuring Billy Ray Cyrus

and

Lil Nas X featuring Billy Ray Cyrus

So, if there's a way that I could handle slight typos/case sensitivity while returning back the values that are not within the respective data frames, please let me know. 

--**Update**-- 

So, I've gotten a little further, by trying the following: 

```
set(df_t['artist'].str.lower() + ': ' + df_t['title'].str.lower()).symmetric_difference(set(df2['artist'].str.lower()+ ': ' + df2['title'].str.lower()))

```

This is able to give me the differences, but it returns back the differences between both data frames whereas I only want to see which of the top 100 results are missing.