Scraping Premier League results I have a code that scrapes list of URL and appends dataframe as below:
from selenium import webdriver
import pandas as pd
from tabulate import tabulate
import os
os.chdir(r&quot;C:\Users\harsh\Google Drive\sportsintel.shop\Files&quot;)
cwd = os.getcwd()
print(cwd)

browser = webdriver.Chrome()

browser.get(&quot;https://www.oddsportal.com/soccer/england/premier-league/results/&quot;)

df = pd.read_html(browser.page_source, header=0)[0]

dateList = []
gameList = []
scoreList = []
home_odds = []
draw_odds = []
away_odds = []

for row in df.itertuples():
    if not isinstance(row[1], str):
        continue
    elif ':' not in row[1]:
        date = row[1].split('-')[0]
        continue
    time = row[1]
    dateList.append(date)
    gameList.append(row[2])
    scoreList.append(row[3])
    home_odds.append(row[4])
    draw_odds.append(row[5])
    away_odds.append(row[6])

result_comp_1 = pd.DataFrame({'date': dateList,
                       'game': gameList,
                       'score': scoreList,
                       'Home': home_odds,
                       'Draw': draw_odds,
                       'Away': away_odds})

print(tabulate(result_comp_1))

browser.get(&quot;https://www.oddsportal.com/soccer/england/premier-league/results/#/page/2/&quot;)

df = pd.read_html(browser.page_source, header=0)[0]

dateList = []
gameList = []
scoreList = []
home_odds = []
draw_odds = []
away_odds = []

for row in df.itertuples():
    if not isinstance(row[1], str):
        continue
    elif ':' not in row[1]:
        date = row[1].split('-')[0]
        continue
    time = row[1]
    dateList.append(date)
    gameList.append(row[2])
    scoreList.append(row[3])
    home_odds.append(row[4])
    draw_odds.append(row[5])
    away_odds.append(row[6])

result_comp = pd.DataFrame({'date': dateList,
                       'game': gameList,
                       'score': scoreList,
                       'Home': home_odds,
                       'Draw': draw_odds,
                       'Away': away_odds})


new_df =result_comp_1.append(result_comp, ignore_index=True)

Can I make my code better to avoid redundancy?
Seems like there's one major point of redundancy, in that you're performing the same logic on the two different pages you're scraping (requesting site, parsing results, storing in dataframe).
You could eliminate the redundancy with a function that takes in the site URL and performs the logic... something like def get_matches_data(url)...