# PySpark dependencies (required)
py4j

# PySpark dependencies (optional)
numpy
pyarrow
pandas
scipy
plotly
mlflow>=1.0
sklearn
matplotlib<3.3.0

# PySpark test dependencies
unittest-xml-reporting

# PySpark test dependencies (optional)
coverage

# Linter
mypy==0.920
pytest-mypy-plugins==1.9.3
flake8==3.9.0
# See SPARK-38680.
pandas-stubs<1.2.0.54

# Documentation (SQL)
mkdocs

# Documentation (Python)
pydata_sphinx_theme
ipython
nbsphinx
numpydoc
jinja2<3.0.0
sphinx<3.1.0
sphinx-plotly-directive
docutils<0.18.0
# See SPARK-38279.
markupsafe==2.0.1

# Development scripts
jira
PyGithub

# pandas API on Spark Code formatter.
black==22.6.0

# Spark Connect
grpcio==1.48.1
protobuf==4.21.6
mypy-protobuf

# PySpark dependencies (required)
py4j

# PySpark dependencies (optional)
numpy
pyarrow
pandas
scipy
plotly
mlflow>=1.0
sklearn
matplotlib<3.3.0

# PySpark test dependencies
unittest-xml-reporting

# PySpark test dependencies (optional)
coverage

# Linter
mypy==0.920
pytest-mypy-plugins==1.9.3
flake8==3.9.0
# See SPARK-38680.
pandas-stubs<1.2.0.54

# Documentation (SQL)
mkdocs

# Documentation (Python)
pydata_sphinx_theme
ipython
nbsphinx
numpydoc
jinja2<3.0.0
sphinx<3.1.0
sphinx-plotly-directive
docutils<0.18.0
# See SPARK-38279.
markupsafe==2.0.1

# Development scripts
jira
PyGithub

# pandas API on Spark Code formatter.
black==22.6.0

# Spark Connect (required)
grpcio==1.48.1
protobuf==4.21.6

# Spark Connect python proto generation plugin (optional)
mypy-protobuf

Mark mypy-protobuf as an optional dependency for Spark Connect
[SPARK-40725][INFRA][FOLLOWUP] Mark mypy-protobuf as an optional dependency for Spark Connect

### What changes were proposed in this pull request?

`mypy-protobuf` is only needed when the connect proto is changed and then to use [generate_protos.sh](https://github.com/apache/spark/blob/master/connector/connect/dev/generate_protos.sh) to update python side generated proto files. We should mark this dependency as optional for people who do not care.

### Why are the changes needed?

`mypy-protobuf` can be optional dependency for people who do not touch connect proto files.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

N/A

Closes #38195 from amaliujia/dev_requirements.

Authored-by: Rui Wang <5520b64c3217afc10cb881636dce6d49816c0d83@databricks.com>
Signed-off-by: Ruifeng Zheng <21ef94aec54fae1a36abbd011347596ae66d9959@apache.org>
