prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  [parquet] can't load parquet directory anymore: `IsADirectoryError`
  **Small description**
  
  Hi @saulpw @anjakefala @takacsd - it seems that forcing opening the path as file with `.open()` - introduced with #2133 - breaks the use case where the multiple parquet files are stored in a directory, and this directory is then read by visidata. This is common with Hive partitioning or when working with spark. A simple fix would be to check if the path is a directory with `os.path.is_dir()` and then retaining old behavior of passing it as a string to `read_table()`. If it is not an existing directory, we move to the new way of opening as a binary buffer.
  
  I have already added this workaround to my clone of visidata, and it fixes my issue, but maybe you have some better ideas how to handle it instead of `if-else` statement in the `ParquetSheet`.
  
  **Expected result**
  
  ```bash
  vd -f parquet parquet_dir
  ```
  should load a parquet into visidata
  
  **Actual result with screenshot**
  ![image](https://github.com/saulpw/visidata/assets/9008816/f262c403-704e-47f0-8994-68bf214b3db6)
  
  **Additional context**
  
  ```bash
  # freshest develop
  visidata@9fd728b72c115e50e99c24b455caaf020381b48e
  
  pyarrow==12.0.0
  python 3.10.2
  ```
  
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `visidata/loaders/parquet.py`
  Content:
  ```
  1 from visidata import Sheet, VisiData, TypedWrapper, anytype, date, vlen, Column, vd
  2 from collections import defaultdict
  3 
  4 
  5 @VisiData.api
  6 def open_parquet(vd, p):
  7     return ParquetSheet(p.name, source=p)
  8 
  9 
  10 class ParquetColumn(Column):
  11     def calcValue(self, row):
  12         val = self.source[row["__rownum__"]]
  13         if val.type == 'large_string':
  14             return memoryview(val.as_buffer())[:2**20].tobytes().decode('utf-8')
  15         else:
  16             return val.as_py()
  17 
  18 
  19 class ParquetSheet(Sheet):
  20     # rowdef: {'__rownum__':int, parquet_col:overridden_value, ...}
  21     def iterload(self):
  22         pa = vd.importExternal("pyarrow", "pyarrow")
  23         pq = vd.importExternal("pyarrow.parquet", "pyarrow")
  24         from visidata.loaders.arrow import arrow_to_vdtype
  25 
  26         with self.source.open('rb') as f:
  27             self.tbl = pq.read_table(f)
  28 
  29         self.columns = []
  30         for colname, col in zip(self.tbl.column_names, self.tbl.columns):
  31             c = ParquetColumn(colname,
  32                               type=arrow_to_vdtype(col.type),
  33                               source=col,
  34                               cache=(col.type.id == pa.lib.Type_LARGE_STRING))
  35             self.addColumn(c)
  36 
  37         for i in range(self.tbl.num_rows):
  38             yield dict(__rownum__=i)
  39 
  40 
  41 @VisiData.api
  42 def save_parquet(vd, p, sheet):
  43     pa = vd.importExternal("pyarrow")
  44     pq = vd.importExternal("pyarrow.parquet", "pyarrow")
  45 
  46     typemap = {
  47         anytype: pa.string(),
  48         int: pa.int64(),
  49         vlen: pa.int64(),
  50         float: pa.float64(),
  51         str: pa.string(),
  52         date: pa.date64(),
  53         # list: pa.array(),
  54     }
  55 
  56     for t in vd.numericTypes:
  57         if t not in typemap:
  58             typemap[t] = pa.float64()
  59 
  60     databycol = defaultdict(list)  # col -> [values]
  61 
  62     for typedvals in sheet.iterdispvals(format=False):
  63         for col, val in typedvals.items():
  64             if isinstance(val, TypedWrapper):
  65                 val = None
  66 
  67             databycol[col].append(val)
  68 
  69     data = [
  70         pa.array(vals, type=typemap.get(col.type, pa.string()))
  71         for col, vals in databycol.items()
  72     ]
  73 
  74     schema = pa.schema(
  75         [(c.name, typemap.get(c.type, pa.string())) for c in sheet.visibleCols]
  76     )
  77     with p.open_bytes(mode="w") as outf:
  78         with pq.ParquetWriter(outf, schema) as writer:
  79             writer.write_batch(
  80                 pa.record_batch(data, names=[c.name for c in sheet.visibleCols])
  81             )
  82
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/visidata/loaders/parquet.py b/visidata/loaders/parquet.py
  --- a/visidata/loaders/parquet.py
  +++ b/visidata/loaders/parquet.py
  @@ -23,8 +23,11 @@
           pq = vd.importExternal("pyarrow.parquet", "pyarrow")
           from visidata.loaders.arrow import arrow_to_vdtype
   
  -        with self.source.open('rb') as f:
  -            self.tbl = pq.read_table(f)
  +        if self.source.is_dir():
  +            self.tbl = pq.read_table(str(self.source))
  +        else: 
  +            with self.source.open('rb') as f:
  +                self.tbl = pq.read_table(f)
   
           self.columns = []
           for colname, col in zip(self.tbl.column_names, self.tbl.columns):
  
verification_info: {"golden_diff": "diff --git a/visidata/loaders/parquet.py b/visidata/loaders/parquet.py\n--- a/visidata/loaders/parquet.py\n+++ b/visidata/loaders/parquet.py\n@@ -23,8 +23,11 @@\n         pq = vd.importExternal(\"pyarrow.parquet\", \"pyarrow\")\n         from visidata.loaders.arrow import arrow_to_vdtype\n \n-        with self.source.open('rb') as f:\n-            self.tbl = pq.read_table(f)\n+        if self.source.is_dir():\n+            self.tbl = pq.read_table(str(self.source))\n+        else: \n+            with self.source.open('rb') as f:\n+                self.tbl = pq.read_table(f)\n \n         self.columns = []\n         for colname, col in zip(self.tbl.column_names, self.tbl.columns):\n", "issue": "[parquet] can't load parquet directory anymore: `IsADirectoryError`\n**Small description**\r\n\r\nHi @saulpw @anjakefala @takacsd - it seems that forcing opening the path as file with `.open()` - introduced with #2133 - breaks the use case where the multiple parquet files are stored in a directory, and this directory is then read by visidata. This is common with Hive partitioning or when working with spark. A simple fix would be to check if the path is a directory with `os.path.is_dir()` and then retaining old behavior of passing it as a string to `read_table()`. If it is not an existing directory, we move to the new way of opening as a binary buffer.\r\n\r\nI have already added this workaround to my clone of visidata, and it fixes my issue, but maybe you have some better ideas how to handle it instead of `if-else` statement in the `ParquetSheet`.\r\n\r\n**Expected result**\r\n\r\n```bash\r\nvd -f parquet parquet_dir\r\n```\r\nshould load a parquet into visidata\r\n\r\n**Actual result with screenshot**\r\n![image](https://github.com/saulpw/visidata/assets/9008816/f262c403-704e-47f0-8994-68bf214b3db6)\r\n\r\n**Additional context**\r\n\r\n```bash\r\n# freshest develop\r\nvisidata@9fd728b72c115e50e99c24b455caaf020381b48e\r\n\r\npyarrow==12.0.0\r\npython 3.10.2\r\n```\r\n\n", "before_files": [{"content": "from visidata import Sheet, VisiData, TypedWrapper, anytype, date, vlen, Column, vd\nfrom collections import defaultdict\n\n\n@VisiData.api\ndef open_parquet(vd, p):\n    return ParquetSheet(p.name, source=p)\n\n\nclass ParquetColumn(Column):\n    def calcValue(self, row):\n        val = self.source[row[\"__rownum__\"]]\n        if val.type == 'large_string':\n            return memoryview(val.as_buffer())[:2**20].tobytes().decode('utf-8')\n        else:\n            return val.as_py()\n\n\nclass ParquetSheet(Sheet):\n    # rowdef: {'__rownum__':int, parquet_col:overridden_value, ...}\n    def iterload(self):\n        pa = vd.importExternal(\"pyarrow\", \"pyarrow\")\n        pq = vd.importExternal(\"pyarrow.parquet\", \"pyarrow\")\n        from visidata.loaders.arrow import arrow_to_vdtype\n\n        with self.source.open('rb') as f:\n            self.tbl = pq.read_table(f)\n\n        self.columns = []\n        for colname, col in zip(self.tbl.column_names, self.tbl.columns):\n            c = ParquetColumn(colname,\n                              type=arrow_to_vdtype(col.type),\n                              source=col,\n                              cache=(col.type.id == pa.lib.Type_LARGE_STRING))\n            self.addColumn(c)\n\n        for i in range(self.tbl.num_rows):\n            yield dict(__rownum__=i)\n\n\n@VisiData.api\ndef save_parquet(vd, p, sheet):\n    pa = vd.importExternal(\"pyarrow\")\n    pq = vd.importExternal(\"pyarrow.parquet\", \"pyarrow\")\n\n    typemap = {\n        anytype: pa.string(),\n        int: pa.int64(),\n        vlen: pa.int64(),\n        float: pa.float64(),\n        str: pa.string(),\n        date: pa.date64(),\n        # list: pa.array(),\n    }\n\n    for t in vd.numericTypes:\n        if t not in typemap:\n            typemap[t] = pa.float64()\n\n    databycol = defaultdict(list)  # col -> [values]\n\n    for typedvals in sheet.iterdispvals(format=False):\n        for col, val in typedvals.items():\n            if isinstance(val, TypedWrapper):\n                val = None\n\n            databycol[col].append(val)\n\n    data = [\n        pa.array(vals, type=typemap.get(col.type, pa.string()))\n        for col, vals in databycol.items()\n    ]\n\n    schema = pa.schema(\n        [(c.name, typemap.get(c.type, pa.string())) for c in sheet.visibleCols]\n    )\n    with p.open_bytes(mode=\"w\") as outf:\n        with pq.ParquetWriter(outf, schema) as writer:\n            writer.write_batch(\n                pa.record_batch(data, names=[c.name for c in sheet.visibleCols])\n            )\n", "path": "visidata/loaders/parquet.py"}], "after_files": [{"content": "from visidata import Sheet, VisiData, TypedWrapper, anytype, date, vlen, Column, vd\nfrom collections import defaultdict\n\n\n@VisiData.api\ndef open_parquet(vd, p):\n    return ParquetSheet(p.name, source=p)\n\n\nclass ParquetColumn(Column):\n    def calcValue(self, row):\n        val = self.source[row[\"__rownum__\"]]\n        if val.type == 'large_string':\n            return memoryview(val.as_buffer())[:2**20].tobytes().decode('utf-8')\n        else:\n            return val.as_py()\n\n\nclass ParquetSheet(Sheet):\n    # rowdef: {'__rownum__':int, parquet_col:overridden_value, ...}\n    def iterload(self):\n        pa = vd.importExternal(\"pyarrow\", \"pyarrow\")\n        pq = vd.importExternal(\"pyarrow.parquet\", \"pyarrow\")\n        from visidata.loaders.arrow import arrow_to_vdtype\n\n        if self.source.is_dir():\n            self.tbl = pq.read_table(str(self.source))\n        else: \n            with self.source.open('rb') as f:\n                self.tbl = pq.read_table(f)\n\n        self.columns = []\n        for colname, col in zip(self.tbl.column_names, self.tbl.columns):\n            c = ParquetColumn(colname,\n                              type=arrow_to_vdtype(col.type),\n                              source=col,\n                              cache=(col.type.id == pa.lib.Type_LARGE_STRING))\n            self.addColumn(c)\n\n        for i in range(self.tbl.num_rows):\n            yield dict(__rownum__=i)\n\n\n@VisiData.api\ndef save_parquet(vd, p, sheet):\n    pa = vd.importExternal(\"pyarrow\")\n    pq = vd.importExternal(\"pyarrow.parquet\", \"pyarrow\")\n\n    typemap = {\n        anytype: pa.string(),\n        int: pa.int64(),\n        vlen: pa.int64(),\n        float: pa.float64(),\n        str: pa.string(),\n        date: pa.date64(),\n        # list: pa.array(),\n    }\n\n    for t in vd.numericTypes:\n        if t not in typemap:\n            typemap[t] = pa.float64()\n\n    databycol = defaultdict(list)  # col -> [values]\n\n    for typedvals in sheet.iterdispvals(format=False):\n        for col, val in typedvals.items():\n            if isinstance(val, TypedWrapper):\n                val = None\n\n            databycol[col].append(val)\n\n    data = [\n        pa.array(vals, type=typemap.get(col.type, pa.string()))\n        for col, vals in databycol.items()\n    ]\n\n    schema = pa.schema(\n        [(c.name, typemap.get(c.type, pa.string())) for c in sheet.visibleCols]\n    )\n    with p.open_bytes(mode=\"w\") as outf:\n        with pq.ParquetWriter(outf, schema) as writer:\n            writer.write_batch(\n                pa.record_batch(data, names=[c.name for c in sheet.visibleCols])\n            )\n", "path": "visidata/loaders/parquet.py"}]}