title: Replace column names in a pandas data frame that partially match a string
body: <h3>Background</h3>
  
  <p>I would like to identify column names in a dataframe that partially match a string, and replace them with the original names plus some new elements added to them. The new elements are integers defined by a list. Here is a <a href="https://stackoverflow.com/questions/40202268/partially-renaming-columns-in-pandas-dataframe">similar question</a>, but I'm afraid the suggested solution will not be flexible enough in my particular case. And <a href="https://stackoverflow.com/questions/11346283/renaming-columns-in-pandas">here</a> is another post with a few excellent answers that come close to the problem I'm facing.</p>
  
  <h3>Some research</h3>
  
  <p>I know I can combine two lists of strings, map them pairwise <a href="https://stackoverflow.com/questions/209840/map-two-lists-into-a-dictionary-in-python">into a dictionary</a> , and  <a href="https://stackoverflow.com/questions/11346283/renaming-columns-in-pandas/11354850#11354850">rename the columns</a> using the dictionary as input in the function <code>df.rename</code>. But this seems a bit too complicated, and not very flexible taking into consideration that the number of existing columns will vary. As will the numbers of columns to be renamed.</p>
  
  <p>The following snippet will produce an input example:</p>
  
  <pre><code># Libraries
  import numpy as np
  import pandas as pd
  import itertools
  
  # A dataframe
  Observations = 5
  Columns = 5
  np.random.seed(123)
  df = pd.DataFrame(np.random.randint(90,110,size=(Observations, Columns)),
                columns = ['Price','obs_1','obs_2','obs_3','obs_4'])
  
  datelist = pd.date_range(pd.datetime.today().strftime('%Y-%m-%d'),
                       periods=Observations).tolist()
  df['Dates'] = datelist
  df = df.set_index(['Dates'])
  print(df)
  </code></pre>
  
  <h3>Input</h3>
  
  <p><a href="https://i.stack.imgur.com/XfoeA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XfoeA.png" alt="enter image description here"></a></p>
  
  <p>I want to identify the column names that start with <code>obs_</code>, and add elements (integers) from the list <code>newElements = [5, 10, 15, 20]</code> following an <code>=</code> sign. The column named <code>Price</code> remains the same. Other columns appearing after the <code>obs_</code> columns should also stay the same.</p>
  
  <p>The following snippet will demonstrate the desired output:</p>
  
  <pre><code># Desired output
  Observations = 5
  Columns = 5
  np.random.seed(123)
  df2 = pd.DataFrame(np.random.randint(90,110,size=(Observations, Columns)),
                columns = ['Price','Obs_1 = 5','Obs_2 = 10','Obs_3 = 15','Obs_4 = 20'])
  
  df2['Dates'] = datelist
  df2 = df2.set_index(['Dates'])
  print(df2)
  </code></pre>
  
  <h3>Output</h3>
  
  <p><a href="https://i.stack.imgur.com/Vp8eY.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Vp8eY.png" alt="enter image description here"></a></p>
  
  <h3>My attempt</h3>
  
  <pre><code># Define the partial string I'm lookin for
  stringMatch = 'Obs_'
  
  # Put existing column names in a list
  oldnames = list(df)
  
  # Put elements that should be added to the column names
  # where the three first letters match 'obs_'
  newElements = [5, 10, 15, 20]
  oldElements = [1, 2, 3, 4]
  
  # Change types of the elements in the list
  str_newElements = [str(x) for x in newElements]
  str_oldElements = [str(y) for y in oldElements]
  str_newNames = str_newElements.copy()
  
  # Since I know the first column should not be renamed,
  # I start with 'Price' in a list
  newnames = ['Price']
  
  # Then I add the renamed parts to the same list
  i = 0
  for oldElement in str_oldElements:   
      #print(repr(oldElement) + repr(str_newElements[i]))
      newnames.append(stringMatch + oldElement + ' = ' + str_newElements[i])
      i = i + 1
  
  # Rename columns using the dict as input in df.rename
  df.rename(columns = dict(zip(oldnames, newnames)), inplace = True)
  
  print('My attempt: ', df)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/lTvbx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/lTvbx.png" alt="enter image description here"></a></p>
  
  <p>Having already made a complete list of the new column names
  I could just as well have used <code>df.columns = newnames</code>,
  but hopefully one of you have got a suggestion using
  <code>df.rename</code> in a more pythonic way.</p>
  
  <p>Thank you for any suggestions!</p>
  
  <p>Here's the whole code for an easy copy-paste:</p>
  
  <pre><code># Libraries
  import numpy as np
  import pandas as pd
  import itertools
  
  # A dataframe
  Observations = 5
  Columns = 5
  np.random.seed(123)
  df = pd.DataFrame(np.random.randint(90,110,size=(Observations, Columns)),
                    columns = ['Price','obs_1','obs_2','obs_3','obs_4'])
  
  datelist = pd.date_range(pd.datetime.today().strftime('%Y-%m-%d'),
                           periods=Observations).tolist()
  df['Dates'] = datelist
  df = df.set_index(['Dates'])
  print('Input: ', df)
  
  # Desired output
  Observations = 5
  Columns = 5
  np.random.seed(123)
  df2 = pd.DataFrame(np.random.randint(90,110,size=(Observations, Columns)),
                    columns = ['Price','Obs_1 = 5','Obs_2 = 10','Obs_3 = 15','Obs_4 = 20'])
  
  df2['Dates'] = datelist
  df2 = df2.set_index(['Dates'])
  print('Desired output: ', df2)
  
  # My attempts
  # Define the partial string I'm lookin for
  stringMatch = 'Obs_'
  
  # Put existing column names in a list
  oldnames = list(df)
  
  # Put elements that should be added to the column names
  # where the three first letters match 'obs_'
  newElements = [5, 10, 15, 20]
  oldElements = [1, 2, 3, 4]
  
  # Change types of the elements in the list
  str_newElements = [str(x) for x in newElements]
  str_oldElements = [str(y) for y in oldElements]
  str_newNames = str_newElements.copy()
  
  
  # Since I know the first column should not be renamed,
  # I start with 'Price' in a list
  newnames = ['Price']
  
  # Then I add the renamed parts to the same list
  i = 0
  for oldElement in str_oldElements:
  
      #print(repr(oldElement) + repr(str_newElements[i]))
      newnames.append(stringMatch + oldElement + ' = ' + str_newElements[i])
      i = i + 1
  
  # Rename columns using the dict as input in df.rename
  df.rename(columns = dict(zip(oldnames, newnames)), inplace = True)
  
  print('My attempt: ', df)
  </code></pre>
  
  <h3>EDIT: Aftermath</h3>
  
  <p>So many good answers after only one day is just amazing! This made it really hard to decide which answer to accept. I don't know if the following will add much value to the post as a whole, but I went ahead and wrapped all suggestions into functions and tested them with %timeit.</p>
  
  <p>Here are the results:
  <a href="https://i.stack.imgur.com/Vk1wG.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Vk1wG.png" alt="enter image description here"></a></p>
  
  <p>The suggestion fram HH1 was the first to be posted, and is also one of the fastest in terms of execution time.  I'll include the code later if anyone is interested.</p>
  
  <h3>EDIT 2</h3>
  
  <p>The suggestion from suvy rendered these results when I tried it:<a href="https://i.stack.imgur.com/sHpfS.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/sHpfS.png" alt="enter image description here"></a></p>
  
  <p>The snippet worked fine until the last line. After running the line <code>df = df.rename(columns=dict(zip(names,renames)))</code> the data frame looked like this:</p>
  
  <p><a href="https://i.stack.imgur.com/9U6GN.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/9U6GN.png" alt="enter image description here"></a></p>