title: How to extract data from large NetCDF files stored on Azure File shares and send to Azure Web Page
body: <p>I have downloaded some weather data (wind, wave, etc.) for 40 years duration and stored these data files which comes in a NetCDF format in Azure File shares. I have about 8 TB of total data stored. Each weather parameter, say wind speed for one year for the whole earth surface is saved in one single file which is about 35GB.</p>
  <p>Next, I have developed a simple Azure website using Python and Dash package, where the user can define a location (Latitude, Longitude), select a weather parameter, date range and submit the request. See website picture below:</p>
  <p><a href="https://i.stack.imgur.com/IxfEb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IxfEb.png" alt="enter image description here" /></a></p>
  <p>Now, I would like to be able to run a script once the user clicks the submit button to extract the specified data, save in a csv file, and give a download link to the file.</p>
  <p>The Azure Storage File Share client library for Python (azure-storage-file-share) allows connecting to the file and download the file. Since one year of data file is 35GB, downloading each year of data and extracting a single grid point is not an option.</p>
  <p>Is there anyway I can run scripts directly on Azure File shares to extract the required data, and then retrieve it from the webpage?</p>
  <p>I am trying to avoid the situation where I need to extract data from NetCDF files and push it into a SQL database, which a website can access easily.</p>