title: Run Python script over multiple folders
body: <p>Today im working with a Python script, this script takes multiple CSV files that are inside a folder and generate a CSV output file with averages by colums, in addition i also have multiple similar folders.
  But what i do, is change manually the name of the folder and the name of the output file, as you can imagine, this process is very hard.
  What I am looking for, is to do it for each folder respecting the order and the name of each corresponding folder in name and content of each output file.</p>
  <p>I was trying using the next in my python script</p>
  <pre><code>scriptname,f1name,f2name = sys.argv
  </code></pre>
  <p>I also use a bash script to make it, but in this bash script, i can change the python script to run it over multiple files inside a folder, but not for multiple folders</p>
  <pre><code>#!/usr/bin/env bash
  
  find ./escen10 -type f -name '*.csv' -print0 |
      while IFS= read -r -d $'\0' line; do
          nname=$(echo &quot;$line&quot; | sed &quot;s/ecn/encn/&quot;) 
          python3 generator2.py $line $nname
      done
  </code></pre>
  <p>The python script is the next:</p>
  <pre><code>import os
  import sys
  from os import listdir
  from csv import DictReader
  from os import walk
  from os import scandir
  import numpy as np
  import pandas as pd
  scriptname,f1name,f2name = sys.argv
  
  #delimiter = ';'
  #dataInFolderPath = './nodes_escen1_1/'
  #outputFile = 'results_escen1_1.csv'
  
  delimiter = ';'
  dataInFolderPath = f1name
  outputFile = f2name
  
  
  dataInFiles = sorted(listdir(dataInFolderPath),key=lambda x: int(x.split('_')[1]))
  
  #sorted_files = sorted(dataInFolderPath)
  lines = []
  nodes = {}
  for fileName in dataInFiles:
  
      file = open(dataInFolderPath + fileName)
      data = DictReader(file, delimiter=delimiter)
      for line in data:
          if not line['node_code'] in nodes.keys():
              nodes[line['node_code']] = []
          nodes[line['node_code']].append([
              line['node_code'],
              line['throughput[Mbps]'],
              line['data_packets_sent'],
              line['data_packets_lost'],
              line['rts_cts_sent'],
              line['rts_cts_lost']
          ])
  
      file.close()
  
  output = ''
  maxData = max([len(nodes[n]) for n in nodes])
  keys = list(nodes.keys())
  keys.sort()
  
  for i in range(maxData):
      if i == 0:
          output += ';'.join(['node_code;throughput[Mbps];Data packets_sent;Data_packets lost;rts_cts_sent;rts_cts_lost'] * len(keys)) + '\n'
  
      for key in keys:
          if key != keys[0]:
              output += ';'
  
          if i &gt;= len(nodes[key]):
              output += ';;;;;'
              continue
  
          output += ';'.join(nodes[key][i])
      output += '\n'
  
  f = open(outputFile, 'w')
  f.write(output)
  f.close()
  
  data = pd.read_csv(outputFile,sep=';')
  
  data = data.append({'node_code':&quot;Promedio&quot;},ignore_index=True)
  
  lenght = len(data[&quot;node_code&quot;])
  
  for i in range(len(data.columns)//6):
      if i ==0:
          data['throughput[Mbps]'].iloc[lenght-1] = np.average(data['throughput[Mbps]'][:-1])
      else:
          data['throughput[Mbps].%i'%i].iloc[lenght-1] = np.average(data['throughput[Mbps].%i'%i][:-1])
  
  data.to_csv(outputFile,sep=';')
  
  print(outputFile)
  </code></pre>
  <p>For this reason, i hope you can help me to automate this process, using what I have available or with some other more efficient solution to realize this task, thank you in advance for any useful help, regards!</p>