title: Issues with using AWS-Data-Wrangler in Glue to SQL Server
body: <p>I've run into an issue with AWS-Data-Wrangler in AWS Glue that I can't seem to find the resolution to. In my searches, I came across <a href="https://stackoverflow.com/questions/66216063/aws-glue-spark-job-error-modulenotfounderror-you-need-to-install-pyodbc-respe">this stackoverflow discussion</a> that accurately describes my issue, but the answer isn't helpful because I'm writing the script in GlueStudio. The answer repeats what the source documentation reads about installing unixODBC-devel to install pyodbc, but nothing seems to refer to using the Job parameters function in GlueStudio to install awswrangler and pyodbc together.</p>
  <p>To describe what I'm trying to accomplish: we have a bunch of data stored as parquet files that came from our on-premise SQL Server instance. I am trying to take this data and load it into an RDS SQL Server instance so we can run already-existing T-SQL SQL Server scripts against it so we don't have to re-write all of our SQL to match Aurora/Redshift/PostgreSQL/etc. The results would then be stored again in S3 or in another permanent SQL Server instance for use in an application and the RDS instance would be deleted as it is no longer in use. This is to be a first step in migrating our processes into cloud-native technologies and we want to minimize the amount of changes that we would have to make (this would be the ideal end-state of our processes) while also minimizing costs (as permanent SQL Server instances can become quite expensive).</p>
  <p>I've tried the following using --additional-python-modules:</p>
  <ul>
  <li><p>pyarrow==2,awswrangler</p>
  <ul>
  <li>This results in it missing pyodbc and reporting &quot;ModuleNotFoundError: You need to install pyodbc respectively the AWS Data Wrangler package with the sqlserver extra for using the sqlserver module&quot;</li>
  </ul>
  </li>
  <li><p>pyarrow==2,awswrangler,pyodbc</p>
  <ul>
  <li>This results in awswrangler not being found: &quot;ModuleNotFoundError: No module named 'awswrangler'&quot;</li>
  </ul>
  </li>
  <li><p>pyodbc,pyarrow==2,awswrangler</p>
  <ul>
  <li>This is the same as above.</li>
  </ul>
  </li>
  <li><p>pyarrow==2,awswrangler[sqlserver]</p>
  <ul>
  <li>This is the same as above.</li>
  </ul>
  </li>
  <li><p>pyarrow==2,awswrangler==2.14.0</p>
  <ul>
  <li>This is the same as above</li>
  </ul>
  </li>
  <li><p>s3://&lt;path&gt;/&lt;to&gt;/&lt;wheel&gt;.whl</p>
  <ul>
  <li>I manually built my own wheel file that contained pyarrow, awswrangler, and pyodbc (with all necessary dependencies), but it resulted in the same issue as above.</li>
  </ul>
  </li>
  </ul>
  <p>I've done the above configurations using both Glue 2.0 and 3.0, and one test using Glue 1.0 which took way too long to fail to be viable. At this point I'm stuck and can't figure out how to get this to work out. Without trying to make a connection to SQL Server, awswrangler works fine reading my S3 files and creating dataframes. I've seen mention of using awswranger[sqlserver] but I can't seem to find any actual usage of it in this kind of use-case. Does anyone have any ideas?</p>