Extensions
Additional Information
#### Logging {#docs:extensions:azure::logging}  
The Azure extension relies on the Azure SDK to connect to Azure Blob storage and supports printing the SDK logs to the console.
To control the log level, set the [`AZURE_LOG_LEVEL`](https://github.com/Azure/azure-sdk-for-cpp/blob/main/sdk/core/azure-core/README.md#sdk-log-messages) environment variable.  
For instance, verbose logs can be enabled as follows in Python:  
```python
import os
import duckdb

os.environ["AZURE_LOG_LEVEL"] = "verbose"

duckdb.sql("CREATE SECRET myaccount (TYPE AZURE, PROVIDER CREDENTIAL_CHAIN, SCOPE 'az://myaccount.blob.core.windows.net/')")
duckdb.sql("SELECT count(*) FROM 'az://myaccount.blob.core.windows.net/path/to/blob.parquet'")
```  
#### Difference between ADLS and Blob Storage {#docs:extensions:azure::difference-between-adls-and-blob-storage}  
Even though ADLS implements similar functionality as the Blob storage, there are some important performance benefits to using the ADLS endpoints for globbing, especially when using (complex) glob patterns.  
To demonstrate, lets look at an example of how the a glob is performed internally using respectively the Glob and ADLS endpoints.  
Using the following filesystem:  
```text
root
├── l_receipmonth=1997-10
│   ├── l_shipmode=AIR
│   │   └── data_0.csv
│   ├── l_shipmode=SHIP
│   │   └── data_0.csv
│   └── l_shipmode=TRUCK
│       └── data_0.csv
├── l_receipmonth=1997-11
│   ├── l_shipmode=AIR
│   │   └── data_0.csv
│   ├── l_shipmode=SHIP
│   │   └── data_0.csv
│   └── l_shipmode=TRUCK
│       └── data_0.csv
└── l_receipmonth=1997-12
├── l_shipmode=AIR
│   └── data_0.csv
├── l_shipmode=SHIP
│   └── data_0.csv
└── l_shipmode=TRUCK
└── data_0.csv
```  
The following query performed through the blob endpoint  
```sql
SELECT count(*)
FROM 'az://root/l_receipmonth=1997-*/l_shipmode=SHIP/*.csv';
```  
will perform the following steps:  
* List all the files with the prefix `root/l_receipmonth=1997-`
* `root/l_receipmonth=1997-10/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-10/l_shipmode=AIR/data_0.csv`
* `root/l_receipmonth=1997-10/l_shipmode=TRUCK/data_0.csv`
* `root/l_receipmonth=1997-11/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-11/l_shipmode=AIR/data_0.csv`
* `root/l_receipmonth=1997-11/l_shipmode=TRUCK/data_0.csv`
* `root/l_receipmonth=1997-12/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-12/l_shipmode=AIR/data_0.csv`
* `root/l_receipmonth=1997-12/l_shipmode=TRUCK/data_0.csv`
* Filter the result with the requested pattern `root/l_receipmonth=1997-*/l_shipmode=SHIP/*.csv`
* `root/l_receipmonth=1997-10/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-11/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-12/l_shipmode=SHIP/data_0.csv`  
Meanwhile, the same query performed through the datalake endpoint,  
```sql
SELECT count(*)
FROM 'abfss://root/l_receipmonth=1997-*/l_shipmode=SHIP/*.csv';
```  
will perform the following steps:  
* List all directories in `root/`
* `root/l_receipmonth=1997-10`
* `root/l_receipmonth=1997-11`
* `root/l_receipmonth=1997-12`
* Filter and list subdirectories: `root/l_receipmonth=1997-10`, `root/l_receipmonth=1997-11`, `root/l_receipmonth=1997-12`
* `root/l_receipmonth=1997-10/l_shipmode=SHIP`
* `root/l_receipmonth=1997-10/l_shipmode=AIR`
* `root/l_receipmonth=1997-10/l_shipmode=TRUCK`
* `root/l_receipmonth=1997-11/l_shipmode=SHIP`
* `root/l_receipmonth=1997-11/l_shipmode=AIR`
* `root/l_receipmonth=1997-11/l_shipmode=TRUCK`
* `root/l_receipmonth=1997-12/l_shipmode=SHIP`
* `root/l_receipmonth=1997-12/l_shipmode=AIR`
* `root/l_receipmonth=1997-12/l_shipmode=TRUCK`
* Filter and list subdirectories: `root/l_receipmonth=1997-10/l_shipmode=SHIP`, `root/l_receipmonth=1997-11/l_shipmode=SHIP`, `root/l_receipmonth=1997-12/l_shipmode=SHIP`
* `root/l_receipmonth=1997-10/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-11/l_shipmode=SHIP/data_0.csv`
* `root/l_receipmonth=1997-12/l_shipmode=SHIP/data_0.csv`  
As you can see because the Blob endpoint does not support the notion of directories, the filter can only be performed after the listing, whereas the ADLS endpoint will list files recursively. Especially with higher partition/directory counts, the performance difference can be very significant.