# Spark Connect

**Spark Connect is a strictly experimental feature and under heavy development.
All APIs should be considered volatile and should not be used in production.**

This module contains the implementation of Spark Connect which is a logical plan
facade for the implementation in Spark. Spark Connect is directly integrated into the build
of Spark. To enable it, you only need to activate the driver plugin for Spark Connect.

## Build

1. Build Spark as usual per the documentation.

2. Build and package the Spark Connect package

   ```bash
   ./build/mvn -Phive package
   ```

   or

   ```bash
   ./build/sbt -Phive package
   ```
   
## Run Spark Shell

```bash
./bin/spark-shell \
  --packages org.apache.spark:spark-connect_2.12:3.4.0 \
  --conf spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin
```

## Run Tests

```bash
./run-tests --testnames 'pyspark.sql.tests.test_connect_basic'
```


# Spark Connect

**Spark Connect is a strictly experimental feature and under heavy development.
All APIs should be considered volatile and should not be used in production.**

This module contains the implementation of Spark Connect which is a logical plan
facade for the implementation in Spark. Spark Connect is directly integrated into the build
of Spark. To enable it, you only need to activate the driver plugin for Spark Connect.

## Build

```bash
./build/mvn -Phive clean package
```

or

```bash
./build/sbt -Phive clean package
```
   
## Run Spark Shell

To run Spark Connect you locally built:

```bash
# Scala shell
./bin/spark-shell \
  --jars `ls connector/connect/target/**/spark-connect*SNAPSHOT.jar | paste -sd ',' -` \
  --conf spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin

# PySpark shell
./bin/pyspark \
  --jars `ls connector/connect/target/**/spark-connect*SNAPSHOT.jar | paste -sd ',' -` \
  --conf spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin
```

To use the release version of Spark Connect:

```bash
./bin/spark-shell \
  --packages org.apache.spark:spark-connect_2.12:3.4.0 \
  --conf spark.plugins=org.apache.spark.sql.connect.SparkConnectPlugin
```

## Run Tests

```bash
./python/run-tests --testnames 'pyspark.sql.tests.test_connect_basic'
```


Fix docs to run Spark Connect locally built
[MINOR][DOCS][CONNECT] Fix docs to run Spark Connect locally built

### What changes were proposed in this pull request?

This PR adds some more command examples to run Spark Connect that you built locally.

### Why are the changes needed?

To guide developers to run Spark Connect they built locally.

### Does this PR introduce _any_ user-facing change?

No, dev-only and doc-only.

### How was this patch tested?

The commands were tested manually in my local.

Closes #38236 from HyukjinKwon/minor-docs-spark-cnnect.

Authored-by: Hyukjin Kwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>
Signed-off-by: Hyukjin Kwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>
